MAAS commission failed due to no IP address after shutdown
Dmitrii Shcherbakov
dmitrii.shcherbakov at canonical.com
Tue Nov 20 04:28:52 UTC 2018
Glad to hear it!
On Tue, Nov 20, 2018, 03:49 <Dave.Chen at dell.com wrote:
> Hi all,
>
>
>
> The issue has been solved, we tried a lot of ways and found it is properly
> caused by the incompatibility between the switch (cumulus OS) and LOM that
> was used, we changed to use another LOM and removed the negotiation related
> configuration in the cumulus switch, and then everything looks good.
>
>
>
> Thanks all for your help!
>
>
>
> Best Regards,
>
> Dave Chen
>
>
>
> *From:* Maas-devel <maas-devel-bounces at lists.ubuntu.com> *On Behalf Of *Chen2,
> Dave
> *Sent:* Friday, November 16, 2018 5:17 PM
> *To:* dmitrii.shcherbakov at canonical.com
> *Cc:* maas-devel at lists.ubuntu.com
> *Subject:* RE: MAAS commission failed due to no IP address after shutdown
>
>
>
> [EXTERNAL EMAIL]
>
> Hi Dmitrii,
>
>
>
> Thanks for your explanation, really appreciate that! Pls allow me to
> continue the discussion here, I will move the discussion to
> https://discourse.maas.io/ later.
>
>
>
> Based on your information you tell me, I guess I MAAS is good only with
> enlistment, it has not reached commission phase at all, is that right?
>
>
>
> I bet the network in our lab is okay, if the NIC of the BMC (Dell IDRAC)
> is connected with our lab where another DHCP is effective, I can still ping
> the BMC IP successfully after the node is powered off. But if the node is
> connect with MAAS, after the enlistment, the node will power off and the
> BMC IP will miss, I cannot see the IP from GUI of MAAS either after the
> system power off. I can **only** see the IP released from MAAS’ DHCP
> service during the enlistment, I think this is the root cause of the
> following failure on the commission.
>
>
>
> Any suggestion on how to debug this issue?
>
>
>
> Best Regards,
>
> Dave Chen
>
>
>
> *From:* Dmitrii Shcherbakov <dmitrii.shcherbakov at canonical.com>
> *Sent:* Friday, November 16, 2018 4:48 PM
> *To:* Chen2, Dave
> *Cc:* maas-devel at lists.ubuntu.com
> *Subject:* Re: MAAS commission failed due to no IP address after shutdown
>
>
>
> [EXTERNAL EMAIL]
>
> Hi Dave,
>
>
>
> The MAAS team has decided to move the community discussions to
> https://discourse.maas.io/ so it's best to create a topic there.
>
>
>
> This doc was written some time ago but is (mostly) still relevant.
>
>
> https://github.com/maas/maas/blob/master/docs/development/notes/anatomy-of-recommissioning-in-maas-2.0.rst
>
>
>
> > MAAS release IP to nodes
>
>
>
> This happens several times via DHCP during commissioning: (1) PXE boot (2)
> at the initrd stage. This is different from "auto assign" static
> configuration during the deployment or DHCP configuration. In other words,
> it is quite natural that MAAS temporary assigns IP addresses using DHCP
> from a pool for (1) and (2) and then during the deployment uses whatever
> was allocated via MAAS IPAM for "auto assign" or via DHCP at the deployment
> time (3).
>
>
>
> > run some cloud init script and then shutdown
>
>
>
> There is a metadata server component in MAAS which should be reachable
> using a routing table of a booted ephemeral image - this is what this step
> is about.
>
>
>
>
> https://github.com/maas/maas/blob/master/docs/development/notes/anatomy-of-recommissioning-in-maas-2.0.rst#9-cloud-init-requests-its-metadata
>
>
>
> > My question is why MAAS claim back the IP from the node? Is there any
> configuration needed to make it works?
>
>
>
> The process to deploy a node from scratch is as follows:
>
>
>
> a) enlistment (manually or via auto-enlistment if supported as described
> in this table
> https://docs.maas.io/2.5/en/nodes-power-types#bmc-driver-support). (1)
> and (2) stages are used here with temporary DHCP-provided addresses;
>
> b) commissioning - (1) and (2) stages are with temporary DHCP-provided
> addresses;
>
> c) deployment - (1), (2) and (3).
>
>
>
> From the log that you pasted, I can see that this is the auto-enlistment
> log and that enlistment has completed successfully.
>
>
>
> Nov 9 18:16:10 maas-enlisting-node cloud-init[2717]: === Fri, 09 Nov 2018 18:16:10 +0000: successfully enlisted to 'http://10.20.0.1:5240/MAAS/api/2.0/machines/'
>
>
>
> My guess is that "Failed to query node's BMC" happens because there is a
> problem in IP connectivity between your MAAS node and the server BMC
> network interface - you need to check that network path and see if there is
> a problem because this is not the interface you PXE boot from (it is a
> dedicated BMC NIC).
>
>
> Best Regards,
>
> Dmitrii Shcherbakov
>
>
>
> Field Software Engineer
> IRC (freenode): Dmitrii-Sh
>
>
>
>
>
> On Fri, Nov 16, 2018 at 6:49 AM <Dave.Chen at dell.com> wrote:
>
> (Resend to avoid size limitation.)
>
>
>
> Hi Developers,
>
>
>
> I am not sure whether this email can reach you well, just cannot find a
> mailing list for user,
>
>
>
> Here is my issue, I have MAAS installed on the controller, and want to
> commission some nodes (Dell 9630), looks like everything is okay, MAAS
> release IP to nodes, nodes run some cloud init script and then shutdown,
> but after for an while, MAAS claim back the IP (since I cannot find the IP
> anymore after clicking the DNS), and hence, MAAS cannot finish the
> commission, cannot get the information on Cores, RAM, disk etc.
>
>
>
> "ping" the IP address shows me the connection cannot be established as
> well.
>
>
>
> I can also see the below messages from the MAAS GUI,
>
> “Failed to query node's BMC - Connection timed out while performing power
> action. Check BMC configuration and connectivity and try again.”
>
>
>
> My question is why MAAS claim back the IP from the node? Is there any
> configuration needed to make it works?
>
>
>
> Attach the logs from the commission, it’s
>
>
>
> $ dpkg -l '*maas*'|cat
>
> Desired=Unknown/Install/Remove/Purge/Hold
>
> |
> Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
>
> |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
>
> ||/ Name Version
> Architecture Description
>
>
> +++-===============================-============================-============-=================================================
>
> ii maas 2.3.5-6511-gf466fdb-0ubuntu1
> all "Metal as a Service" is a physical cloud and IPAM
>
> ii maas-cli 2.3.5-6511-gf466fdb-0ubuntu1
> all MAAS client and command-line interface
>
> un maas-cluster-controller <none>
> <none> (no description available)
>
> ii maas-common 2.3.5-6511-gf466fdb-0ubuntu1
> all MAAS server common files
>
> ii maas-dhcp 2.3.5-6511-gf466fdb-0ubuntu1
> all MAAS DHCP server
>
> ii maas-dns 2.3.5-6511-gf466fdb-0ubuntu1
> all MAAS DNS server
>
> ii maas-proxy 2.3.5-6511-gf466fdb-0ubuntu1
> all MAAS Caching Proxy
>
> ii maas-rack-controller 2.3.5-6511-gf466fdb-0ubuntu1
> all Rack Controller for MAAS
>
> ii maas-region-api 2.3.5-6511-gf466fdb-0ubuntu1
> all Region controller API service for MAAS
>
> ii maas-region-controller 2.3.5-6511-gf466fdb-0ubuntu1
> all Region Controller for MAAS
>
> un maas-region-controller-min <none>
> <none> (no description available)
>
> un python-django-maas <none>
> <none> (no description available)
>
> un python-maas-client <none>
> <none> (no description available)
>
> un python-maas-provisioningserver <none>
> <none> (no description available)
>
> ii python3-django-maas 2.3.5-6511-gf466fdb-0ubuntu1
> all MAAS server Django web framework (Python 3)
>
> ii python3-maas-client 2.3.5-6511-gf466fdb-0ubuntu1
> all MAAS python API client (Python 3)
>
> ii python3-maas-provisioningserver 2.3.5-6511-gf466fdb-0ubuntu1
> all MAAS server provisioning libraries (Python 3)
>
>
>
>
>
> Logs from “maas-enlisting-node” directory is pasted here (
> http://paste.openstack.org/show/734898/), highly appreciated for your
> help!
>
>
>
> Best Regards,
>
> Dave Chen
>
>
>
> --
> Maas-devel mailing list
> Maas-devel at lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/maas-devel
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/maas-devel/attachments/20181120/aa1fa0c5/attachment-0001.html>
More information about the Maas-devel
mailing list