Using zsync for .deb downloads: initial benchmark results

[John McCabe-Dansted]

On Wed, Jul 15, 2009 at 12:19 AM, Lars Wirzenius<lars at ubuntu.com> wrote:
> Before I continue working on this, I'd like to have some feedback on
> this: is a 25% reduction in download sizes worthwhile to pursue? It
> would seem to require changing dpkg to call the external gzip binary to
> use --rsyncable, rather than use the internal zlib library.

I think a better solution would be to fix zsync so it can look inside
ar files. In principle all this requires is adding an offset to every
http request so we can use the zsync look-inside, but the zsync source
code doesn't have this make a clear distinction between the http
offset and the local offset. We could try asking the author to add
this capability.

> What do other people think?

My opinion is that deb deltas are more useful. The average size was
only 11% of the original, according to
  http://tonelli.sns.it/pub/mennucc1/debdelta/run/tests/README
This is similar to my experiences

The disadvantage of debdeltas is that they take more space than .zsync files.

> PS. I've fully automated my benchmark, and am happy to share the
> scripts. If anyone wants to play with them, drop me a note, and I'll set
> up a public bzr branch. You'll need fast access to a mirror, since they
> download snapshots of hardy-security and the corresponding packages from
> hardy, for a total of about three gigabytes.

So if we were using n->n+1 debdeltas, we would need ~300MB of extra
packages for Hardy? This seems reasonable given that we can delete old
packages, omit unimportant packages etc.

FYI, the debdelta in Ubuntu cannot handle lzma packages, and gives a
cryptic 'debdelta failed  :   <type 'exceptions.AssertionError'>'
error if it comes across them. However it seems easy to add this
functionality to debdelta.