Changing dpkg-deb default compression from gzip to lzma for Hardy

[Scott Ritchie]

Krzysztof Lichota wrote:
> Scott Ritchie napisał(a):
>> I did some experimentation with my Wine package.  Here's the filesize of
>> the latest .deb passing different options to dpkg-deb:
>>
>> 11081456 default
>> 10090930 bzip2
>> 7682608 lzma
>>
>> That's over a 30% reduction in bandwidth for me and my humble third
>> party repository.
>>
>> I've heard that lzma will be included by default in main for Hardy.
>> This is a very good idea.  Changing package build scripts to manually
>> pass lzma compression using dh_builddeb -- -Z lzma would be very
>> tedious, however.  In IRC pitti proposed that we do this centrally -
>> changing the default of dpkg-deb (currently gzip) seems to be the best
>> place for this.
>>
>> Thoughts?
> 
> It is hard to judge best compression using only one package. It is
> possible that for other packages other compression schemes would be
> better. Have you run built other packages? ?The best would be to rebuild
> whole repo with new compression scheme and compare the results, so that
> it does not appear, for example, that packages stop fitting into one CD.
> 
It's been shown that lzma is, in general, much better.  If we happen to
find a specific case where it's not, then we can always set that package
to a non-default by tweaking the dh_builddeb line.

> Another thing is decompression time - on some machines the limiting
> resource is CPU, not bandwidth nor disk space and changing compression
> would mean significant burden as packages would be unpacked much longer
> and put more stress on system, making user experience unpleasant. This
> could be mitigated by running unpacking process with "nice", but AFAIK
> it is not the case now.
> 
> If these 2 issues are addressed, I think it in general a good idea :)
> 

I believe lzma has a fairly efficient decompression time.  We should
note, however, that package installation time is one of the least
important places to optimize CPU usage - it's not user-interactive, and
is very frequently done after the user has stopped doing other things.

I don't have any data, however from my own personal experience with
moderately fast broadband it seems like most of my package installation
time is during downloading rather than unpacking/configure by a very
wide margin.  A 30% reduction there would require a much larger amount
of time to unpack to make it not worthwhile.

More importantly, however, is that by using a better compression scheme
we greatly reduce the stress put on the mirrors, especially during
release time when package downloads get REALLY slow.

Thanks,
Scott Ritchie