size of uec images

[Steve Langasek]

On Wed, Oct 07, 2009 at 11:35:58PM -0400, Scott Moser wrote:
> Hello all,
>    The folks working on UEC have found the 10G filesystem images that we
> create in our UEC builds [1] to be a bit unwieldy [2].  In addition to
> download times, the uncompress takes quite a while (in the order of 10
> minutes on "normal" developer hardware), and the 10G image doesn't run on
> the several of the default instance sizes in UEC.
>    The 10G size was chosen for good reason [3] to target ec2.
> (Essentially on ec2 you get a '/' partition that of whatever size you want
> up to 10G with no cost.) We had been interested in keeping the 10G image
> size to make UEC images identical to those of ec2.

>    I started off trying to address this issue by producing a 2G image
> along side the 10G image (named with a '-2G' suffix).  The result was a
> bit surprising to me:
>       |10G img  | 2G img
>       |------------------
> amd64 |  476M   | 222M
>  i386 |  470M   | 216M

>    The numbers in the table are the compressed image size for the 10G and
> 2G partitions. The filesystems are filled with somewhere around 635M of
> filesystem data (per 'df').

Well, and here's the result if we encapsulate the 10GB image in a tarball
with -S before compressing (i.e., take advantage of the fact that tar
understands sparse files, instead of asking gzip to compress a whole lot of
zeroes):

      |10G img
      |-------
amd64 | 175M
 i386 | 168M

This has the added benefit that it will *decompress* as a sparse file on the
far end, saving lots of disk space on the user's system and making the
decompress operation itself a lot quicker.

Is there any reason we can't ship these as img.tar.gz instead of img.gz?  My
understanding is that images have to be uncompressed for rebundling on the
end user's system before they can be used on UEC anyway, right?  In that
case, 'tar xf' should be no harder than 'gunzip', hopefully it's just a
small documentation change.

Note that these advantages apply whether you go with a 10GB or a 2GB
filesystem, so this change is worth doing anyway - but maybe if the 10GB fs
is a small enough download, there's no need to switch to 2GB?

-- 
Steve Langasek                   Give me a lever long enough and a Free OS
Debian Developer                   to set it on, and I can move the world.
Ubuntu Developer                                    http://www.debian.org/
slangasek at ubuntu.com                                     vorlon at debian.org
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 827 bytes
Desc: Digital signature
Url : https://lists.ubuntu.com/archives/ubuntu-devel/attachments/20091008/0186a9da/attachment.pgp