Delta debs / zsync (etc) discussions gone quiet?
paran at lysator.liu.se
Fri Aug 21 12:57:01 UTC 2009
I got interested in this while reading the previous threads. Recently I
helped a friend install jauntu and downloading updates right after the
installation took more than 12 hours which was not very nice.
Lars Wirzenius <lars at ubuntu.com> writes:
> The summary of my conclusions:
> * zsync works very well for ISO images (for testers)
> * zsync works very well for Packages files
Have you benchmarked zsync against the Packages.diff files used by
Debian? I don't doubt that zsync works well, but is it so much better
that it is worth all the implementation work when something very similar
is already supported by APT?
> * debdelta works well for .deb files
I have done some investigation about debdelta that I have been meaning
to send to this list, might as well dump everything into this mail.
Debdelta works well, but there is some problems that needs to be fixed.
The maximum memory parameter should be set quite high. I have used "-M
2048" for my tests, and this reduced the size of the deltas a few
I implemented support for LZMA compressed debs, and this is included
From debdelta 0.29. However these packages are very bad for debdelta.
LZMA compression comes at the cost of high CPU and memory usage during
compression. Usually this is OK, as compression is only performed once
on a build, clients only decompress which is not so slow.
Debdelta changes this as every single client that applies the delta have
to redo the compression to get the exact same data.tar.lzma file.
This highlights the problem that the checksums is of the compressed
data. The problem exists with gzipped debs as well, but is less
noticeable. It is really a big waste of CPU cycles in both cases, as the
next step will usually be to install the package, which decompresses the
Debdelta can't find deltas for new or renamed packages. The most
important example is the kernel image that change name when the kernel
Debdelta is not integrated with APT, see bug #498778,
There is no version information in the debdelta file format, so changing
it without breaking older debdelta clients is currently hard.
Delta files are currently not signed in any way.
This is IMHO the biggest problem with the design of debdelta. Each
debdelta file includes patch.sh, a shell script that does parts of the
work of applying the delta.
This script can contain arbitrary commands, which opens up a big
security problem as the deltas are not signed.
It also makes the deltas bigger than necessary, as a lot of the script
code is repeated in every delta. The compressed patch.sh file is usually
15-30% of a deltas size.
Deltas can't apply themselves anyway, so I think it would be better to
move as much of the logic as possible into the debdelta program itself
I have generated deltas for all packages in jaunty-updates main amd64.
Deltas had to be generated manually for the renamed kernel packages,
2.6.28-11.42 to 2.6.28-15.49. For some reason -updates also contain
2.6.28-13.45 packages, i ignored those.
bsdiff encounters the OOM-killer on my 4G laptop when generating the
delta for libqt4-webkit-dbg. "-M 2048" does not prevent this, I
generated this delta manually using "--delta-algo xdelta" instead.
$ du -ks jaunty-updates
$ du -ks jaunty-updates-deltas
The 2.6.28-13.45 packages adds up to 67484k, and does not have
deltas. This means that the deltas are 9.9% the size of the original
I did a default jaunty amd64 desktop install, and installed debdelta
Downloading updates using "aptitude full-upgrade" would have required a
debdelta-upgrade downloaded 4527k and created 111M of deb archives.
The three deltas for 2.6.28-15 kernel had to be downloaded manually,
total size 5204k.
After this "aptitude full-upgrade" downloads 1276kB. A gnutls update
released after I made my deltas and linux-restricted-modules as I only
made deltas for packages in main.
Total download size using debdelta: 4527+5204+1276 = 11007k = 10MB, this
is 7% of the 146MB normal download.
All my deltas are available here . To repeat the test you need
debdelta 0.30  and /etc/debdelta/sources.conf from here . Most of
the output from my test is also available .
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Size: 196 bytes
Desc: not available
More information about the Ubuntu-devel-discuss