<div dir="ltr">Lee,<div><br></div><div>Removing the argument prevents it from booting.</div><div><br></div><div>I get a repeating error:</div><div>> ERR: When syslinux is not booted from physical disk (or its emulation),</div><div>> 'boot' and 'fs' are meaningless.</div><div>> boot:</div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Dec 6, 2017 at 3:28 AM, Lee Trager <span dir="ltr"><<a href="mailto:lee.trager@canonical.com" target="_blank">lee.trager@canonical.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>What happens if you remove the argument(the append line) completely? On a test host it automatically selects the right drive.<span class="HOEnZb"><font color="#888888"><br><br></font></span></div><span class="HOEnZb"><font color="#888888">Lee<br></font></span></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Dec 5, 2017 at 11:56 PM, Dmitrii Shcherbakov <span dir="ltr"><<a href="mailto:dmitrii.shcherbakov@canonical.com" target="_blank">dmitrii.shcherbakov@<wbr>canonical.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">For non-UEFI systems it appears to be that MAAS 2.3.0 only generates a curtin operation with "gpt" when storage devices are larger than 2 TiB:<div><br></div><div><a href="https://github.com/maas/maas/blob/2.3.0/src/maasserver/preseed_storage.py#L224-L237" target="_blank">https://github.com/maas/maas/b<wbr>lob/2.3.0/src/maasserver/prese<wbr>ed_storage.py#L224-L237</a><br></div><div><div> def _generate_disk_operation(self, block_device):</div></div><div>...</div><div><div> if bios_boot_method in [</div><div> "uefi", "powernv", "powerkvm"]:</div><div> disk_operation["ptable"] = "gpt"</div><div> if node_arch == "ppc64el":</div><div> add_prep_partition = True</div><div> elif (block_device.size >= GPT_REQUIRED_SIZE and # <---- this</div><div> node_arch == "amd64"):</div><div> disk_operation["ptable"] = "gpt"</div><div> add_bios_grub_partition = True</div><div> else: # <---- and this</div><div> disk_operation["ptable"] = "msdos"</div></div><div><br></div><div><div>class CurtinStorageGenerator:</div></div><div>... def generate(self):</div><div>...</div><div><div> # Generate each YAML operation in the storage_config.</div><div> self._generate_disk_operations<wbr>()</div></div><div>...</div><div><div> def _generate_disk_operations(self<wbr>):</div><div> """Generate all disk operations."""</div><div> for block_device in self.operations["disk"]:</div><div> self._generate_disk_operation(<wbr>block_device)</div></div><div><br></div><div><a href="https://github.com/maas/maas/blob/2.3.0/src/maasserver/models/partitiontable.py#L51-L53" target="_blank">https://github.com/maas/maas/b<wbr>lob/2.3.0/src/maasserver/model<wbr>s/partitiontable.py#L51-L53</a><br></div><div>GPT_REQUIRED_SIZE = 2 * 1024 * 1024 * 1024 * 1024<br></div><div><br></div><div><br></div></div><div class="gmail_extra"><br clear="all"><div><div class="m_2505316094371395718m_-939005646189548821gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div dir="ltr">Best Regards,<div>Dmitrii Shcherbakov</div><div><br></div><div><div style="color:rgb(136,136,136);font-size:12.8px"><span style="color:rgb(68,68,68);font-size:12.8px">Field Software Engineer</span><br style="color:rgb(68,68,68);font-size:12.8px"><span style="font-size:12.8px">IRC (freenode): Dmitrii-Sh</span><br></div></div></div></div></div></div></div></div></div></div><div><div class="m_2505316094371395718h5">
<br><div class="gmail_quote">On Wed, Dec 6, 2017 at 7:40 AM, Daniel K <span dir="ltr"><<a href="mailto:sathackr@gmail.com" target="_blank">sathackr@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Digging into chain.c32, it looks like several options could be passed for the boot drive.<div><div><br></div><div>> Usage:</div><div>> chain.c32 hd<disk#> [<partition>] [options]</div><div>> chain.c32 fd<disk#> [options]</div><div>> chain.c32 mbr:<id> [<partition>] [options]</div><div>> chain.c32 boot [<partition>] [options]</div><div>> chain.c32 fs [options]</div><div>> chain.c32 label=<label> [options]</div><div>> chain.c32 guid=<label> [options]</div></div><div><br></div><div>It would seem that label= or guid= would be the most sure-fire way to boot the drive you want to boot, but that requires a GPT partition instead of MBR. Fallback method for mbr could be use the mbr:<id> option:</div><div><div><br></div><div>> The mbr: syntax means search all the hard disks until one with a specific MBR serial number (bytes 440-443) is found.</div><div>> You can get the MBR serial number, by running the following command (change /dev/sda to the correct device):</div><div>> $ hexdump -s 440 -n 4 -e '"0x%08x\n"' /dev/sda</div><div>> 0x0ec8694c</div><div>> Or by running:</div><div>> $ fdisk -l /dev/sda</div><div>> ...</div><div>> Disk identifier: 0x0ec8694c</div><div>> Example:</div><div>> LABEL mbr_serial</div><div>> COM32 chain.c32</div><div>> APPEND mbr:0x0ec8694c</div></div><div><br></div><div>So for a MBR boot it would seem to make sense that if during commissioning, grub is installed on /dev/sdc then it should pass hd2 instead of hd0 to chain.c32, or pass mbr:<serial>. That way regardless of the bios configuration, the correct drive would always boot. Looks like there are some options to use variables in the pxe template files, but I doubt a guid or mbr serial number would be availble.</div><div><br></div><div>Looks like I may be able to sidestep this by hardcoding something like "label=boot" instead of hd0 in the template file, then forcing curtin to use a gpt table instead of mbr, and ensuring the disk/partition with grub is labeled "boot" and no others are labeled as such. Still not quite familiar enough with MAAS to know where to make that adjustment.<br></div><div><br></div><div>This of course would also not be a problem if HPE would put the drives in the right order. Or UEFI, which is not supported by these servers that I can tell.</div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div></div><div class="m_2505316094371395718m_-939005646189548821HOEnZb"><div class="m_2505316094371395718m_-939005646189548821h5"><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Dec 5, 2017 at 11:06 PM, Daniel K <span dir="ltr"><<a href="mailto:sathackr@gmail.com" target="_blank">sathackr@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Looks like the hd0 may be hardcoded :-/<div><br></div><div><div>> root@maas1:~# cat /usr/lib/python3/dist-packages<wbr>/provisioningserver/templates/<wbr>pxe/config.local.amd64.templat<wbr>e</div><span><div>> DEFAULT local</div><div>> <br></div><div>> LABEL local</div><div>> SAY Booting local disk ...</div><div>> KERNEL chain.c32</div><div>> APPEND hd0</div></span></div><div><br></div><div><br></div></div><div class="m_2505316094371395718m_-939005646189548821m_-6422418454607960183HOEnZb"><div class="m_2505316094371395718m_-939005646189548821m_-6422418454607960183h5"><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Dec 5, 2017 at 10:38 PM, Daniel K <span dir="ltr"><<a href="mailto:sathackr@gmail.com" target="_blank">sathackr@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">So attacking this from the angle I'm most familiar with I've captured the traffic between the maas server and a booting node to see if I can catch the data in flight.<div><br></div><div>PXE first downloads a file called pxelinux.0 - I see this file in /var/lib/maas/boot-resources.</div><div>Then requests(and receives) a file called ldlinux.c32.</div><div><br></div><div>Then requests a non existent file: pxelinux.cfg/<some sort of uuid/guid?></div><div><div>> # Packet 344 from C:\Users\user\asdf.pcap</div><div>> - 345</div><div>> - 166.825599</div><div>> - 10.20.128.111</div><div>> - 10.20.4.30</div><div>> - TFTP</div><div>> - 121</div><div>> - Read Request, File: pxelinux.cfg/36383031-3839-325<wbr>5-5831-353130303732, Transfer type: octet, tsize=0, blksize=1408</div></div><div>> - # Packet 345 from C:\Users\user\asdf.pcap</div><div>> - 346</div><div>> - 166.836864</div><div>> - 10.20.4.30</div><div>> - 10.20.128.111</div><div>> - TFTP</div><div>> - 61</div><div>> - Error Code, Code: File not found, Message: File not found</div><div><br></div><div>Then requests and receives a file called pxelinux.cfg/01-<mac address></div><div><div>> # Packet 346 from C:\Users\user\asdf.pcap</div><div>> - 347</div><div>> - 166.837008</div><div>> - 10.20.128.111</div><div>> - 10.20.4.30</div><div>> - TFTP</div><div>> - 105</div><div>> - Read Request, File: pxelinux.cfg/01-e8-39-35-2b-c9<wbr>-5c, Transfer type: octet, tsize=0, blksize=1408</div></div><div><br></div><div>which contains the following printable text:</div><div><div>> 95+\W1ExL@@T</div><div>> oad*DEFAULT local<br></div><div>> LABEL local<br></div><div>> SAY Booting local disk ...</div><div>> KERNEL chain.c32</div><div>> APPEND hd0</div></div><div><br></div><div>which I can correlate to log entries:</div><div><div>> 2017-12-05 22:17:49 provisioningserver.rackdservic<wbr>es.tftp: [info] ldlinux.c32 requested by e8:39:35:2b:c9:5c</div><div>> 2017-12-05 22:17:49 provisioningserver.rackdservic<wbr>es.tftp: [info] pxelinux.cfg/36383031-3839-325<wbr>5-5831-353130303732 requested by e8:39:35:2b:c9:5c</div><div>> 2017-12-05 22:17:49 provisioningserver.rackdservic<wbr>es.tftp: [info] pxelinux.cfg/01-e8-39-35-2b-c9<wbr>-5c requested by e8:39:35:2b:c9:5c</div></div><div><br></div><div>I assume the "APPEND hd0" is what is telling the pxelinux loader which disk to boot.</div><div>I searched but I cannot find a directory called pxelinux.cfg anywhere on the maas servers, nor a file with any part of the mac address in it's name. I'll assume then that some piece of maas is responding to that request after fetching the config from some sort of database for that MAC address/node. </div><div><br></div><div>So then there must be a knob somewhere in MAAS that I can tweak to cause a different disk to be sent in the APPEND hd0 command.<br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div></div><div class="m_2505316094371395718m_-939005646189548821m_-6422418454607960183m_7114649815275344096HOEnZb"><div class="m_2505316094371395718m_-939005646189548821m_-6422418454607960183m_7114649815275344096h5"><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Dec 5, 2017 at 5:24 PM, Lloyd Parkes <span dir="ltr"><<a href="mailto:lloyd+lp@must-have-coffee.gen.nz" target="_blank">lloyd+lp@must-have-coffee.gen<wbr>.nz</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">I originally sent this from the wrong email address and so it got hung<br>
up on list moderation.<br>
<div class="m_2505316094371395718m_-939005646189548821m_-6422418454607960183m_7114649815275344096m_8705230647351122613HOEnZb"><div class="m_2505316094371395718m_-939005646189548821m_-6422418454607960183m_7114649815275344096m_8705230647351122613h5"><br>
<br>
On 5 December 2017 at 10:45, Daniel K <<a href="mailto:sathackr@gmail.com" target="_blank">sathackr@gmail.com</a>> wrote:<br>
><br>
> There must be something that tells the PXE loader which physical disk to try<br>
> to boot<br>
<br>
This is almost certainly hard coded in the PXELinux boot script to<br>
default to BIOS disk 0x80. Have a look at<br>
<a href="http://www.syslinux.org/wiki/index.php?title=SYSLINUX#LOCALBOOT_type" rel="noreferrer" target="_blank">http://www.syslinux.org/wiki/i<wbr>ndex.php?title=SYSLINUX#LOCALB<wbr>OOT_type</a><br>
and see if it helps.<br>
<br>
I would dig into this myself because I want to make my HPE servers<br>
boot as well, but I'm 3265km and two months away from my MAAS servers.<br>
<br>
Cheers,<br>
Lloyd<br>
<br>
</div></div><div class="m_2505316094371395718m_-939005646189548821m_-6422418454607960183m_7114649815275344096m_8705230647351122613HOEnZb"><div class="m_2505316094371395718m_-939005646189548821m_-6422418454607960183m_7114649815275344096m_8705230647351122613h5">--<br>
Maas-devel mailing list<br>
<a href="mailto:Maas-devel@lists.ubuntu.com" target="_blank">Maas-devel@lists.ubuntu.com</a><br>
Modify settings or unsubscribe at: <a href="https://lists.ubuntu.com/mailman/listinfo/maas-devel" rel="noreferrer" target="_blank">https://lists.ubuntu.com/mailm<wbr>an/listinfo/maas-devel</a><br>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div><br></div>
</div></div><br>--<br>
Maas-devel mailing list<br>
<a href="mailto:Maas-devel@lists.ubuntu.com" target="_blank">Maas-devel@lists.ubuntu.com</a><br>
Modify settings or unsubscribe at: <a href="https://lists.ubuntu.com/mailman/listinfo/maas-devel" rel="noreferrer" target="_blank">https://lists.ubuntu.com/mailm<wbr>an/listinfo/maas-devel</a><br>
<br></blockquote></div><br></div></div></div>
<br>--<br>
Maas-devel mailing list<br>
<a href="mailto:Maas-devel@lists.ubuntu.com" target="_blank">Maas-devel@lists.ubuntu.com</a><br>
Modify settings or unsubscribe at: <a href="https://lists.ubuntu.com/mailman/listinfo/maas-devel" rel="noreferrer" target="_blank">https://lists.ubuntu.com/mailm<wbr>an/listinfo/maas-devel</a><br>
<br></blockquote></div><br></div>
</div></div></blockquote></div><br></div>