ISO size breakdown? ISO diff?

Wed Jan 20 13:57:17 UTC 2010

Because the question was multiposted, I have to multipost my answer. Sorry
if you read it before, I'll combine it on one email.

On Mon, January 18, 2010 09:53, giovanni_re wrote:
> Is there an breakdown analysis by size of the major categories of the
> files on the ISOs?
>
> Like, every component that is >1% of the size?  Like, Kernel, GUI,
> Applications, & breakdown for some of the major applications, like OO, FF,
> whatever?
>
> If it doesn't exist, could someone do that analysis quickly? Can it be
> done easily?  Could you do it? if so, TIA :)

This is how far I got:

----------CUT----------
amedee at intrepid:~/download$ mkdir -p /tmp/karmic
amedee at intrepid:~/download$ sudo mount -o loop
ubuntu-9.10-desktop-i386.iso /tmp/karmic
amedee at intrepid:~/download$ sudo aptitude install squashfs-tools
amedee at intrepid:~/download$ apt-cache search squashfs
squashfs-modules-2.6-amd64 - Compression filesystem for Linux 2.6 on AMD64
squashfs-modules-2.6-openvz-amd64 - Compression filesystem for Linux 2.6
on AMD64
squashfs-modules-2.6-vserver-amd64 - Compression filesystem for Linux 2.6
on AMD64
squashfs-modules-2.6-xen-amd64 - Compression filesystem for Linux 2.6 on
AMD64
squashfs-modules-2.6.26-2-amd64 - Compression filesystem for Linux 2.6.26
on AMD64
squashfs-modules-2.6.26-2-openvz-amd64 - Compression filesystem for Linux
2.6.26 on AMD64
squashfs-modules-2.6.26-2-vserver-amd64 - Compression filesystem for Linux
2.6.26 on AMD64
squashfs-modules-2.6.26-2-xen-amd64 - Compression filesystem for Linux
2.6.26 on AMD64
squashfs-source - Source for the squash filesystem
live-initramfs - Debian Live initramfs hook
squashfs-tools-dbg - Tool to create and append to squashfs filesystems
(debug)
squashfs-tools - Tool to create and append to squashfs filesystems
amedee at intrepid:~/download$ uname -r
2.6.26-2-xen-amd64
amedee at intrepid:~/download$ sudo aptitude install
squashfs-modules-2.6-xen-amd64
amedee at intrepid:~/download$ sudo modprobe squashfs
amedee at intrepid:~/download$ lsmod | grep squash
squashfs               40336  0
zlib_inflate           18944  2 squashfs,isofs
amedee at intrepid:~/download$ mkdir -p /tmp/karmic-squashfs
amedee at intrepid:~/download$ sudo mount
/tmp/karmic/casper/filesystem.squashfs /tmp/karmic-squashfs -t squashfs
mount: /tmp/karmic/casper/filesystem.squashfs is geen blok-apparaat 
(misschien '-o loop' proberen?)
amedee at intrepid:~/download$ sudo mount
/tmp/karmic/casper/filesystem.squashfs /tmp/karmic-squashfs -t squashfs -o
loop
mount: onjuiste bestandssysteemsoort, ongeldige optie, ontbrekende
codepagina,
       ontbrekend hulpprogramma, slecht superblok op /dev/loop1, of een
andere fout
       Soms staat er nuttige informatie in het systeemlog --
       probeer zoiets als:  dmesg | tail

amedee at intrepid:~/download$ dmesg | tail
[11236478.952075] Shorewall:net2fw:DROP:IN=eth0 OUT=
MAC=00:16:3e:df:76:94:fe:ff:ff:ff:ff:ff:08:00 SRC=85.17.73.36
DST=188.40.34.110 LEN=48 TOS=0x00 PREC=0x00 TTL=121 ID=27588 DF PROTO=TCP
SPT=4324 DPT=8443 WINDOW=65535 RES=0x00 SYN URGP=0
[11236481.966756] Shorewall:net2fw:DROP:IN=eth0 OUT=
MAC=00:16:3e:df:76:94:fe:ff:ff:ff:ff:ff:08:00 SRC=85.17.73.36
DST=188.40.34.110 LEN=48 TOS=0x00 PREC=0x00 TTL=121 ID=28944 DF PROTO=TCP
SPT=4324 DPT=8443 WINDOW=65535 RES=0x00 SYN URGP=0
[11236500.876868] loop: module loaded
[11236501.004036] ISO 9660 Extensions: Microsoft Joliet Level 3
[11236501.035969] ISO 9660 Extensions: RRIP_1991A
[11236923.474303] Shorewall:net2fw:DROP:IN=eth0 OUT=
MAC=00:16:3e:df:76:94:fe:ff:ff:ff:ff:ff:08:00 SRC=198.63.210.173
DST=188.40.34.110 LEN=48 TOS=0x00 PREC=0x00 TTL=116 ID=892 DF PROTO=TCP
SPT=3014 DPT=8443 WINDOW=65535 RES=0x00 SYN URGP=0
[11236926.471165] Shorewall:net2fw:DROP:IN=eth0 OUT=
MAC=00:16:3e:df:76:94:fe:ff:ff:ff:ff:ff:08:00 SRC=198.63.210.173
DST=188.40.34.110 LEN=48 TOS=0x00 PREC=0x00 TTL=116 ID=3853 DF PROTO=TCP
SPT=3014 DPT=8443 WINDOW=65535 RES=0x00 SYN URGP=0
[11237015.859457] squashfs: version 3.3-CVS (2008/04/04) Phillip Lougher
[11237260.576381] SQUASHFS error: Major/Minor mismatch, trying to mount
newer 4.0 filesystem
[11237260.576391] SQUASHFS error: Please update your kernel
----------CUT----------

Like, meh...
Could anyone please try to mount the squashfs filesystem on the iso with
the correct version of squashfs? I don't want to frak up my precious
Debian box.
Once it's mounted, it is just a simple matter of using du. Probably
something like
du /tmp/karmic-squashfs | sort -n | head -n 50

That would give you a list of the 50 biggest files inside the squashed
filesystem on the iso. It would probably need a bit of refinement
somewhere, but that's how I would approach this question.

> Also, is it possible to diff the different ISOs, so that if you have one
> already, & want one other, you don't have to dl all 700MB, just the diff?
> Like, between 9.10 Ub & KUb for x86?

>> No.
>
> I think you are wrong. There are programs that can diff binary files,
> for example xdelta. Is this not what he ask for? I haven't used xdelta
> and have no clue how it will work on ISO files.

It is an exercise in futility.

----------CUT----------
amedee at intrepid:~$ sudo aptitude install xdelta
amedee at intrepid:~$ cd download/
amedee at intrepid:~/download$ wget
http://ftp.heanet.ie/pub/ubuntu-releases/karmic/ubuntu-9.10-desktop-i386.iso
http://ftp.heanet.ie/pub/ubuntu-releases/jaunty/ubuntu-9.04-desktop-i386.iso
*snip*
723488768   9,84M/s   in 85s
*snip*
732909568   9,74M/s   in 1m 58s
amedee at intrepid:~/download$ xdelta delta ubuntu-9.04-desktop-i386.iso
ubuntu-9.10-desktop-i386.iso jaunty-karmic.delta
amedee at intrepid:~/download$ ls -hl *.iso *.delta
-rw-r--r-- 1 amedee amedee 576M jan 20 13:50 jaunty-karmic.delta
-rw-r--r-- 1 amedee amedee 699M apr 20  2009 ubuntu-9.04-desktop-i386.iso
-rw-r--r-- 1 amedee amedee 690M okt 28 22:14 ubuntu-9.10-desktop-i386.iso
----------CUT----------

Using a delta for the ISO files would mean a 17% decrease in network
traffic (from 690MB to 576MB) but a 185% increase in storage (from 690 MB
to 1965MB), because you would have to keep the old ISO before you can
apply the path. And Canonical would have to store them on their servers
too. And they would have to replicate all the diffs to all their mirrors,
so that would mean a HUGE increase in network traffic. Plus you would need
checksums for all the diffs. A lot can go wrong in transport.

In other words, the answer is no. Not because is technically impossible,
but because it isn't practical and hardly worth the effort. As we say in
Dutch, er zijn nog andere katten te geselen. (There are other cats to whip
= there are more important things to do)

-- 
Amedee Van Gasse