MAAS commission failed due to no IP address after shutdown
Dave.Chen at Dell.com
Dave.Chen at Dell.com
Fri Nov 16 09:16:31 UTC 2018
Hi Dmitrii,
Thanks for your explanation, really appreciate that! Pls allow me to continue the discussion here, I will move the discussion to https://discourse.maas.io/ later.
Based on your information you tell me, I guess I MAAS is good only with enlistment, it has not reached commission phase at all, is that right?
I bet the network in our lab is okay, if the NIC of the BMC (Dell IDRAC) is connected with our lab where another DHCP is effective, I can still ping the BMC IP successfully after the node is powered off. But if the node is connect with MAAS, after the enlistment, the node will power off and the BMC IP will miss, I cannot see the IP from GUI of MAAS either after the system power off. I can *only* see the IP released from MAAS’ DHCP service during the enlistment, I think this is the root cause of the following failure on the commission.
Any suggestion on how to debug this issue?
Best Regards,
Dave Chen
From: Dmitrii Shcherbakov <dmitrii.shcherbakov at canonical.com>
Sent: Friday, November 16, 2018 4:48 PM
To: Chen2, Dave
Cc: maas-devel at lists.ubuntu.com
Subject: Re: MAAS commission failed due to no IP address after shutdown
[EXTERNAL EMAIL]
Hi Dave,
The MAAS team has decided to move the community discussions to https://discourse.maas.io/ so it's best to create a topic there.
This doc was written some time ago but is (mostly) still relevant.
https://github.com/maas/maas/blob/master/docs/development/notes/anatomy-of-recommissioning-in-maas-2.0.rst
> MAAS release IP to nodes
This happens several times via DHCP during commissioning: (1) PXE boot (2) at the initrd stage. This is different from "auto assign" static configuration during the deployment or DHCP configuration. In other words, it is quite natural that MAAS temporary assigns IP addresses using DHCP from a pool for (1) and (2) and then during the deployment uses whatever was allocated via MAAS IPAM for "auto assign" or via DHCP at the deployment time (3).
> run some cloud init script and then shutdown
There is a metadata server component in MAAS which should be reachable using a routing table of a booted ephemeral image - this is what this step is about.
https://github.com/maas/maas/blob/master/docs/development/notes/anatomy-of-recommissioning-in-maas-2.0.rst#9-cloud-init-requests-its-metadata
> My question is why MAAS claim back the IP from the node? Is there any configuration needed to make it works?
The process to deploy a node from scratch is as follows:
a) enlistment (manually or via auto-enlistment if supported as described in this table https://docs.maas.io/2.5/en/nodes-power-types#bmc-driver-support). (1) and (2) stages are used here with temporary DHCP-provided addresses;
b) commissioning - (1) and (2) stages are with temporary DHCP-provided addresses;
c) deployment - (1), (2) and (3).
From the log that you pasted, I can see that this is the auto-enlistment log and that enlistment has completed successfully.
Nov 9 18:16:10 maas-enlisting-node cloud-init[2717]: === Fri, 09 Nov 2018 18:16:10 +0000: successfully enlisted to 'http://10.20.0.1:5240/MAAS/api/2.0/machines/'
My guess is that "Failed to query node's BMC" happens because there is a problem in IP connectivity between your MAAS node and the server BMC network interface - you need to check that network path and see if there is a problem because this is not the interface you PXE boot from (it is a dedicated BMC NIC).
Best Regards,
Dmitrii Shcherbakov
Field Software Engineer
IRC (freenode): Dmitrii-Sh
On Fri, Nov 16, 2018 at 6:49 AM <Dave.Chen at dell.com<mailto:Dave.Chen at dell.com>> wrote:
(Resend to avoid size limitation.)
Hi Developers,
I am not sure whether this email can reach you well, just cannot find a mailing list for user,
Here is my issue, I have MAAS installed on the controller, and want to commission some nodes (Dell 9630), looks like everything is okay, MAAS release IP to nodes, nodes run some cloud init script and then shutdown, but after for an while, MAAS claim back the IP (since I cannot find the IP anymore after clicking the DNS), and hence, MAAS cannot finish the commission, cannot get the information on Cores, RAM, disk etc.
"ping" the IP address shows me the connection cannot be established as well.
I can also see the below messages from the MAAS GUI,
“Failed to query node's BMC - Connection timed out while performing power action. Check BMC configuration and connectivity and try again.”
My question is why MAAS claim back the IP from the node? Is there any configuration needed to make it works?
Attach the logs from the commission, it’s
$ dpkg -l '*maas*'|cat
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-===============================-============================-============-=================================================
ii maas 2.3.5-6511-gf466fdb-0ubuntu1 all "Metal as a Service" is a physical cloud and IPAM
ii maas-cli 2.3.5-6511-gf466fdb-0ubuntu1 all MAAS client and command-line interface
un maas-cluster-controller <none> <none> (no description available)
ii maas-common 2.3.5-6511-gf466fdb-0ubuntu1 all MAAS server common files
ii maas-dhcp 2.3.5-6511-gf466fdb-0ubuntu1 all MAAS DHCP server
ii maas-dns 2.3.5-6511-gf466fdb-0ubuntu1 all MAAS DNS server
ii maas-proxy 2.3.5-6511-gf466fdb-0ubuntu1 all MAAS Caching Proxy
ii maas-rack-controller 2.3.5-6511-gf466fdb-0ubuntu1 all Rack Controller for MAAS
ii maas-region-api 2.3.5-6511-gf466fdb-0ubuntu1 all Region controller API service for MAAS
ii maas-region-controller 2.3.5-6511-gf466fdb-0ubuntu1 all Region Controller for MAAS
un maas-region-controller-min <none> <none> (no description available)
un python-django-maas <none> <none> (no description available)
un python-maas-client <none> <none> (no description available)
un python-maas-provisioningserver <none> <none> (no description available)
ii python3-django-maas 2.3.5-6511-gf466fdb-0ubuntu1 all MAAS server Django web framework (Python 3)
ii python3-maas-client 2.3.5-6511-gf466fdb-0ubuntu1 all MAAS python API client (Python 3)
ii python3-maas-provisioningserver 2.3.5-6511-gf466fdb-0ubuntu1 all MAAS server provisioning libraries (Python 3)
Logs from “maas-enlisting-node” directory is pasted here (http://paste.openstack.org/show/734898/), highly appreciated for your help!
Best Regards,
Dave Chen
--
Maas-devel mailing list
Maas-devel at lists.ubuntu.com<mailto:Maas-devel at lists.ubuntu.com>
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/maas-devel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/maas-devel/attachments/20181116/71a1f530/attachment-0001.html>
More information about the Maas-devel
mailing list