<div dir="auto"><div>Glad to hear it! <div class="gmail_quote"><div dir="ltr">On Tue, Nov 20, 2018, 03:49 <<a href="mailto:Dave.Chen@dell.com">Dave.Chen@dell.com</a> wrote: </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> <div lang="EN-US" link="blue" vlink="purple"> <div class="m_-8509039955086979712WordSection1"> Hi all, The issue has been solved, we tried a lot of ways and found it is properly caused by the incompatibility between the switch (cumulus OS) and LOM that was used, we changed to use another LOM and removed the negotiation related configuration in the cumulus switch, and then everything looks good. Thanks all for your help! <div> Best Regards, Dave Chen </div> <div style="border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt"> <div> <div style="border:none;border-top:solid #e1e1e1 1.0pt;padding:3.0pt 0in 0in 0in"> From: Maas-devel <<a href="mailto:maas-devel-bounces@lists.ubuntu.com" target="_blank" rel="noreferrer">maas-devel-bounces@lists.ubuntu.com</a>> On Behalf Of Chen2, Dave Sent: Friday, November 16, 2018 5:17 PM To: <a href="mailto:dmitrii.shcherbakov@canonical.com" target="_blank" rel="noreferrer">dmitrii.shcherbakov@canonical.com</a> Cc: <a href="mailto:maas-devel@lists.ubuntu.com" target="_blank" rel="noreferrer">maas-devel@lists.ubuntu.com</a> Subject: RE: MAAS commission failed due to no IP address after shutdown </div> </div> <div> [EXTERNAL EMAIL] </div> Hi Dmitrii, Thanks for your explanation, really appreciate that! Pls allow me to continue the discussion here, I will move the discussion to <a href="https://discourse.maas.io/" target="_blank" rel="noreferrer">https://discourse.maas.io/</a> later. Based on your information you tell me, I guess I MAAS is good only with enlistment, it has not reached commission phase at all, is that right? I bet the network in our lab is okay, if the NIC of the BMC (Dell IDRAC) is connected with our lab where another DHCP is effective, I can still ping the BMC IP successfully after the node is powered off. But if the node is connect with MAAS, after the enlistment, the node will power off and the BMC IP will miss, I cannot see the IP from GUI of MAAS either after the system power off. I can *only* see the IP released from MAAS’ DHCP service during the enlistment, I think this is the root cause of the following failure on the commission. Any suggestion on how to debug this issue? Best Regards, Dave Chen <div style="border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt"> <div> <div style="border:none;border-top:solid #e1e1e1 1.0pt;padding:3.0pt 0in 0in 0in"> From: Dmitrii Shcherbakov <<a href="mailto:dmitrii.shcherbakov@canonical.com" target="_blank" rel="noreferrer">dmitrii.shcherbakov@canonical.com</a>> Sent: Friday, November 16, 2018 4:48 PM To: Chen2, Dave Cc: <a href="mailto:maas-devel@lists.ubuntu.com" target="_blank" rel="noreferrer">maas-devel@lists.ubuntu.com</a> Subject: Re: MAAS commission failed due to no IP address after shutdown </div> </div> <div> [EXTERNAL EMAIL] </div> <div> <div> <div> <div> <div> <div> <div> <div> <div> <div> Hi Dave, <div> </div> <div> The MAAS team has decided to move the community discussions to <a href="https://discourse.maas.io/" target="_blank" rel="noreferrer">https://discourse.maas.io/</a> so it's best to create a topic there. </div> <div> </div> <div> This doc was written some time ago but is (mostly) still relevant. </div> <div> <a href="https://github.com/maas/maas/blob/master/docs/development/notes/anatomy-of-recommissioning-in-maas-2.0.rst" target="_blank" rel="noreferrer">https://github.com/maas/maas/blob/master/docs/development/notes/anatomy-of-recommissioning-in-maas-2.0.rst</a> </div> <div> </div> <div> > MAAS release IP to nodes </div> <div> </div> <div> This happens several times via DHCP during commissioning: (1) PXE boot (2) at the initrd stage. This is different from "auto assign" static configuration during the deployment or DHCP configuration. In other words, it is quite natural that MAAS temporary assigns IP addresses using DHCP from a pool for (1) and (2) and then during the deployment uses whatever was allocated via MAAS IPAM for "auto assign" or via DHCP at the deployment time (3). </div> <div> </div> <div> > run some cloud init script and then shutdown </div> <div> </div> <div> There is a metadata server component in MAAS which should be reachable using a routing table of a booted ephemeral image - this is what this step is about. </div> <div> </div> <div> <a href="https://github.com/maas/maas/blob/master/docs/development/notes/anatomy-of-recommissioning-in-maas-2.0.rst#9-cloud-init-requests-its-metadata" target="_blank" rel="noreferrer">https://github.com/maas/maas/blob/master/docs/development/notes/anatomy-of-recommissioning-in-maas-2.0.rst#9-cloud-init-requests-its-metadata</a> </div> <div> </div> <div> > My question is why MAAS claim back the IP from the node? Is there any configuration needed to make it works? </div> <div> </div> <div> The process to deploy a node from scratch is as follows: </div> <div> </div> <div> a) enlistment (manually or via auto-enlistment if supported as described in this table <a href="https://docs.maas.io/2.5/en/nodes-power-types#bmc-driver-support" target="_blank" rel="noreferrer"> https://docs.maas.io/2.5/en/nodes-power-types#bmc-driver-support</a>). (1) and (2) stages are used here with temporary DHCP-provided addresses; </div> <div> b) commissioning - (1) and (2) stages are with temporary DHCP-provided addresses; </div> <div> c) deployment - (1), (2) and (3). </div> <div> </div> <div> From the log that you pasted, I can see that this is the auto-enlistment log and that enlistment has completed successfully. </div> <div> </div> <div> <pre>Nov 9 18:16:10 maas-enlisting-node cloud-init[2717]: === Fri, 09 Nov 2018 18:16:10 +0000: successfully enlisted to '<a href="http://10.20.0.1:5240/MAAS/api/2.0/machines/" target="_blank" rel="noreferrer">http://10.20.0.1:5240/MAAS/api/2.0/machines/</a>'</pre> </div> <div> My guess is that "Failed to query node's BMC" happens because there is a problem in IP connectivity between your MAAS node and the server BMC network interface - you need to check that network path and see if there is a problem because this is not the interface you PXE boot from (it is a dedicated BMC NIC). </div> <div> <div> <div> <div> <div> <div> <div> <div> <div> Best Regards, <div> Dmitrii Shcherbakov </div> <div> </div> <div> <div> Field Software Engineer IRC (freenode): Dmitrii-Sh </div> </div> </div> </div> </div> </div> </div> </div> </div> </div> </div> </div> </div> </div> </div> </div> </div> </div> </div> </div> </div> <div> <div> On Fri, Nov 16, 2018 at 6:49 AM <<a href="mailto:Dave.Chen@dell.com" target="_blank" rel="noreferrer">Dave.Chen@dell.com</a>> wrote: </div> <blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt"> <div> <div> (Resend to avoid size limitation.) Hi Developers, I am not sure whether this email can reach you well, just cannot find a mailing list for user, Here is my issue, I have MAAS installed on the controller, and want to commission some nodes (Dell 9630), looks like everything is okay, MAAS release IP to nodes, nodes run some cloud init script and then shutdown, but after for an while, MAAS claim back the IP (since I cannot find the IP anymore after clicking the DNS), and hence, MAAS cannot finish the commission, cannot get the information on Cores, RAM, disk etc. "ping" the IP address shows me the connection cannot be established as well. I can also see the below messages from the MAAS GUI, “Failed to query node's BMC - Connection timed out while performing power action. Check BMC configuration and connectivity and try again.” My question is why MAAS claim back the IP from the node? Is there any configuration needed to make it works? Attach the logs from the commission, it’s $ dpkg -l '*maas*'|cat Desired=Unknown/Install/Remove/Purge/Hold | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad) ||/ Name Version Architecture Description +++-===============================-============================-============-================================================= ii maas 2.3.5-6511-gf466fdb-0ubuntu1 all "Metal as a Service" is a physical cloud and IPAM ii maas-cli 2.3.5-6511-gf466fdb-0ubuntu1 all MAAS client and command-line interface un maas-cluster-controller <none> <none> (no description available) ii maas-common 2.3.5-6511-gf466fdb-0ubuntu1 all MAAS server common files ii maas-dhcp 2.3.5-6511-gf466fdb-0ubuntu1 all MAAS DHCP server ii maas-dns 2.3.5-6511-gf466fdb-0ubuntu1 all MAAS DNS server ii maas-proxy 2.3.5-6511-gf466fdb-0ubuntu1 all MAAS Caching Proxy ii maas-rack-controller 2.3.5-6511-gf466fdb-0ubuntu1 all Rack Controller for MAAS ii maas-region-api 2.3.5-6511-gf466fdb-0ubuntu1 all Region controller API service for MAAS ii maas-region-controller 2.3.5-6511-gf466fdb-0ubuntu1 all Region Controller for MAAS un maas-region-controller-min <none> <none> (no description available) un python-django-maas <none> <none> (no description available) un python-maas-client <none> <none> (no description available) un python-maas-provisioningserver <none> <none> (no description available) ii python3-django-maas 2.3.5-6511-gf466fdb-0ubuntu1 all MAAS server Django web framework (Python 3) ii python3-maas-client 2.3.5-6511-gf466fdb-0ubuntu1 all MAAS python API client (Python 3) ii python3-maas-provisioningserver 2.3.5-6511-gf466fdb-0ubuntu1 all MAAS server provisioning libraries (Python 3) Logs from “maas-enlisting-node” directory is pasted here (<a href="http://paste.openstack.org/show/734898/" target="_blank" rel="noreferrer">http://paste.openstack.org/show/734898/</a>), highly appreciated for your help! Best Regards, Dave Chen </div> </div> -- Maas-devel mailing list <a href="mailto:Maas-devel@lists.ubuntu.com" target="_blank" rel="noreferrer">Maas-devel@lists.ubuntu.com</a> Modify settings or unsubscribe at: <a href="https://lists.ubuntu.com/mailman/listinfo/maas-devel" target="_blank" rel="noreferrer"> https://lists.ubuntu.com/mailman/listinfo/maas-devel</a> </blockquote> </div> </div> </div> </div> </div> </blockquote></div></div></div>