Zsync for Packages files

[Robert Collins]

On Tue, 2009-07-21 at 09:52 +0300, Lars Wirzenius wrote:
> ma, 2009-07-20 kello 22:46 +0200, Tollef Fog Heen kirjoitti:
> > ]] Max Bowsher 
> > 
> > | That's interesting - I would have thought that it would simply lead to
> > | many small pdiff files being created, but the system overall still
> > | working fine. How is the usability damaged?
> > 
> > You'll need to get quite a lot of small files, which means you get hit
> > by TCP's slow start algorithm again and again rather than just grabbing
> > one big file once.
> 
> Surely apt uses a single TCP connection to fetch all the pdiff files?
> That is, HTTP pipelining, or whatever it is called, to do several HTTP
> requests over a singel TCP connection.

Even if apt doesn't, if you have:
 - a local http proxy
 - an ISP using an intercepting proxy
 - a localish mirror

then you shouldn't be making that many high latency round trips. HTTP
pipelining is dead in the water though, don't depend on it. There are
other things that can be done to mitigate that though: if there is a
pdiff per day, you could (for instance) create a skip tree with one per
day that goes back a day. One every even day that goes back two days,
one every 4th day going back 8, etc.

> At this point we're in speculation territory. Does someone have the
> ability to do a quick benchmark to see how much data a week's worth of
> Packages file updates for karmic takes, using pdiffs? I'd be happy to
> provide the files I snapshotted for this, but I don't have the time to
> actually do the benchmark.

Data is good.

-Rob
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part
Url : https://lists.ubuntu.com/archives/ubuntu-devel/attachments/20090721/07876827/attachment.pgp