Speed up package builds

[Harald Sitter]

Following are essentials on how to speed up test building:

a) env varibles
All KDE binary packages get compressed using LZMA which is a jolly nice 
compression algorithm that decompresses almost as fast/light as GZIP while 
permitting much smaller binaries most of the time. This however comes at the 
cost of horrible CPU and RAM usage at compression time hence you defenitely 
want the following envioronment variable set in a build environment:
  export DEB_NO_LZMA=1
This turns of LZMA and use the default setting (GZIP, which is generally less 
aggressive at compression).
Additionally you might find the following very useful too:
  export DEB_NO_COMPRESSION=1
I only added support for this today (pkg-kde-tools 0.9.2ubuntu7) in addition 
to no LZMA compression it also enforces no compression at all, which is of 
course faster than GZIP too.
If you have disk space to spare I must recommend setting those two variables. 
Especially if you build a lot of stuff you will notice a lot less waiting.

You can add those simply to your pbuilderrc to get them applied inside the 
pbuilder environment.

2) use cowbuilder or similar madness
I would recommend cowbuilder as the waepon of choice becaues it is just a 
wrapper around pbuilder so essentially everything stays the same you just have 
cowbuilder lay ontop of pbuilder. Cowbuilder basically eliminates the need of 
having tgz chroots but instead maintains your copy on disk and then hardlinks 
to the build envrionments (there are obvious disadvantages to this, so for 
review builds of new software I would still go with pbuilder...).
This should speed up environment setups nice (according to Quintasan, I did 
not try it myself ;))

3) ninjas: know your stack!
If you are doing KDE release building you will probably have seen the nice dep 
graph [1] already. What is nice about this is, that you can work on things in 
layers ... someone nukes down the blue "parenting" packages and others work on 
the various layers. In any case you will notice that like 92.4% of the time 
packaging is done on packages depending on one or more other packages 
underneath. Now in the situation of release building where you are actually 
walking up the graph you can make very smart use of this fact. Instead of 
having to download and install kdelibs+deps just create a pbuilder chroot and 
login using
  sudo pbuilder login --save-after-login
then add the ninjas PPA (and apt-transport-https to get that working) and 
install kdelibs5-dev. That way you cut pretty much half the effort off the 
dependency installation. As you are moving up the graph you of course might 
add other parenting packages (such as kdepimlibs).
BUT!!! Do not do this for regular builds! Do not install kdepimlibs or any 
"higher" parent unless you are positive that you do not need to build 
something underneath it...
Simply, unless you are very attentive, DO NOT install anything other than 
kdelibs5-dev and DO NOT forget to switch to a 100% clean pbuilder once you are 
done.
Really, while this speeds up things A LOT it is also a potential spot where 
things can be screwed up A LOT, so be very careful.

[1] https://wiki.kubuntu.org/Kubuntu/Ninjas/DependencyGraph

-- 
Harald Sitter
Kubuntu Core Developer
http://www.kubuntu.org
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part.
Url : https://lists.ubuntu.com/archives/kubuntu-devel/attachments/20100730/17d33e0f/attachment.pgp