to LZMA or not to LZMA
apachelogger at ubuntu.com
Tue May 26 22:12:18 BST 2009
Proposal for long-term LZMA usage:
Add it to our CDBS magic.
In Jaunty Jonathan Thomas introduced LZMA compression to some packages,
especially decreasing the size of dbg packages a lot. In Karmic those changes
were lost due to merging, which made me remember the original idea I had.
The compression method should be handled within our CDBS makefile and not
within the individual rules files.
There are two reasons for that:
1. It can be managed in a central place (which also helps keeping an overview
of what is LZMAified and what is not)
2. Disabling it on a global scale becomes a lot easier
So, my current concept is to have it enabled by default for every build using
debian-qt-kde.mk. LZMA in general allows for much better compression than gzip
or even bzip2. The downside however is that it requires a lot more resources
at compression time, but at decompression is somewhere in between bzip2.
Simply put: lzma is between gzip and bzip2 rating by decompression time but
mostly providing the best compression of all 3.
A blog post about LZMA:
"Lzma is definitely worth a look due to its performance - if you unpack much
more often than you pack (three times slower than already-not-so-quick
bzip2!). Also, lzma works better on large files. From a distribution
standpoint, if/when the library/dependency problems have been sorted out, lzma
is quite preferable, if you don't do much compression yourself, yet have to
watch your bandwidth usage."
Slax, for example, uses LZMA for every package AFAIK (Slax, being a Live
System, is also facing the "how much stuff can I squeeze on one CD" problem)
The increased requirements at compression make the build time sometimes a lot
longer, which is why you can deactivate LZMA compression (thus fall back to
gzip, which is default) by exporting DEB_NO_LZMA=1.
If someone finds out how to check if we are on a buildd we could also reverse
that and only use LZMA if the build is done on a buildd. That just made me
realise, maybe we should add another option: no compression at all ... that
might speed up testbuilds a bit ;-)
Anyway, please state any concerns regarding this proposal
More information about the kubuntu-devel