to LZMA or not to LZMA

Harald Sitter apachelogger at
Tue May 26 22:12:18 BST 2009

Proposal for long-term LZMA usage:

Add it to our CDBS magic.

In Jaunty Jonathan Thomas introduced LZMA compression to some packages, 
especially decreasing the size of dbg packages a lot. In Karmic those changes 
were lost due to merging, which made me remember the original idea I had.

The compression method should be handled within our CDBS makefile and not 
within the individual rules files.
There are two reasons for that:
1. It can be managed in a central place (which also helps keeping an overview 
of what is LZMAified and what is not)
2. Disabling it on a global scale becomes a lot easier

So, my current concept is to have it enabled by default for every build using LZMA in general allows for much better compression than gzip 
or even bzip2. The downside however is that it requires a lot more resources 
at compression time, but at decompression is somewhere in between bzip2.
Simply put: lzma is between gzip and bzip2 rating by decompression time but 
mostly providing the best compression of all 3.

A blog post about LZMA:
"Lzma is definitely worth a look due to its performance - if you unpack much 
more often than you pack (three times slower than already-not-so-quick 
bzip2!). Also, lzma works better on large files. From a distribution 
standpoint, if/when the library/dependency problems have been sorted out, lzma 
is quite preferable, if you don't do much compression yourself, yet have to 
watch your bandwidth usage."

Slax, for example, uses LZMA for every package AFAIK (Slax, being a Live 
System, is also facing the "how much stuff can I squeeze on one CD" problem)

The increased requirements at compression make the build time sometimes a lot 
longer, which is why you can deactivate LZMA compression (thus fall back to 
gzip, which is default) by exporting DEB_NO_LZMA=1.

If someone finds out how to check if we are on a buildd we could also reverse 
that and only use LZMA if the build is done on a buildd. That just made me 
realise, maybe we should add another option: no compression at all ... that 
might speed up testbuilds a bit ;-)

Anyway, please state any concerns regarding this proposal

More information about the kubuntu-devel mailing list