[ubuntu-cloud] Greater than 1TiB boot disks for the Cloud Image

[Patrick Devine]

Hi Scott,

Thanks for looking in to this, and thank you for subscribing me to the bug.

The simplest solution I can think of would be to create two partitions
on the image;  one for the boot and one for root.  If they're created
with partition labels (which is already done on the root disk), you
can easy find the partition to expand with the command (as root):

$ blkid -L cloudimg-rootfs
/dev/vda<id>

which can be fed into the various grow operations for ext, btrfs, xfs,
etc... and I think cloud-init already supports something close to
this, no?  As to the limitation with GRUB, I haven't seen it
documented anywhere official, only from finding people talking about
it while searching with Google.  The problem is most people use
multiple partitions, and EC2 doesn't have any > 1 TiB local boot
instances, so there's not much to go on.  I think it's mostly going to
be a problem with people using custom flavours on OpenStack, which
(hopefully) will probably become more prevalent soon.

Ideally though, yes we should all move to GPT and be done with the 2
TiB problem.  Is there a chance of getting Canonical to produce a BIOS
(ie. non-UEFI) image with GPT labeled partitions?  That would solve
the > 2 TiB limitation without forcing everyone to move to UEFI.

For the docs, the real problem is figuring out how to get the image
into glance and then figuring out the "minimum size" value.  What
would be great is a page linked off of the cloud image page which has
step-by-step instructions on the import process.  I realize this may
sound like overkill, but when you're dealing with OpenStack you're
already thinking about a million things, so one less thing the better.
It'd be much easier to just plug-and-chug.

Thanks again,
--Patrick.





On Mon, Jul 28, 2014 at 12:10 PM, Scott Moser <smoser at ubuntu.com> wrote:
> On Fri, 25 Jul 2014, Patrick Devine wrote:
>
>> Hey peeps,
>>
>> I ran into an interesting problem yesterday with regards to the cloud
>> image.  The image contains a single ext4 partition across an MSDOS
>> partition table, which I believe is done to make it expedient to
>> resize the partition; it's simple to add space to the image and extend
>> the the single partition to different instance sizes.
>>
>> There ends up being two problems, however.  Since it's partitioned
>> with MSDOS, the total size of the disk is limited to 2 TiB.  This is
>> annoying, but if you're OK with the limitation it will work out of the
>> box with OpenStack and KVM.  That is, it will work until a new grub
>
> Right, there is not much we can do at this point about that limitation.
> As you pointed out, we do make the uefi images available, and given
> appropriate kvm invocation, they should load properly.  growpart (from
> cloud-utils) should successfully be able to make them grow to > 2TB.
>
>> package is released.  Grub has a limitation that it can only read
>> partitions which are less than 1TiB in size, so if you've resized your
>> instance flavour to larger than 1 TiB, you're going to be in trouble.
>
> I talked with grub maintainer Colin Watson and he did not seem to know
> about such a limitation.  Do you have a link to such ?
>
>> Everything will work fine initially because Grub gets installed
>> initially as part of image creation when the partition size is small.
>> When it gets _re-installed_ however (as was the case yesterday when a
>> new grub-pc package was released), things will blow up leaving your
>> image in a broken state.
>
> I've filed this as bug 1349538 and taken the liberty of subscribing you.
>   https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/1349538
>
>> I haven't tried the uefi image as I'm not certain whether or not it
>> will have issues with KVM.  My guess is that it would at least have a
>> GPT partition table, but I don't know if it still suffers from the
>> single partition problem which chokes up Grub.
>>
>> Is there any documentation on the cloud images?  The Community Help
>> Page does not mention any limitations, and doesn't even explain how to
>> import the image into OpenStack.  It would be nice to know what the
>
> Manually, as you can find in the openstack doc, this is:
>   http://docs.openstack.org/user-guide/content/cli_manage_images.html
>   glance image-create --name centos63-image --disk-format=qcow2 \
>     --container-format=bare --is-public=True --file=your.img
>
> For automated use cases, we provide machine formated data at
>   http://cloud-images.ubuntu.com/releases/streams/v1/
> That is in 'simplestreams' format, which can be read and used by the
> simplestreams package/library.  There is a full example of how to "sync"
> images from cloud-images.ubuntu.com into your local glance at:
>   http://bazaar.launchpad.net/~smoser/simplestreams/example-sync/view/head:/README
>
> That also creates simplestreams data and uploads it to swift.  Then, if
> you know your swift url.  Then, a tool (sstream-query) can be used to
> query that data and get the image id of the latest ubuntu image.  Note,
> this is what juju reads, and is available for public clouds from that same
> streams/v1/ link above.
>
> There is some more information about simplestreams in a talk I did for
> UDS.  See https://github.com/smoser/talk-simplestreams .
>
>> limitations are up front so that when you are using them in a
>> production environment they don't end up biting you.
>
> Agreed, better documentation would be useful.