speeding up package builds

[Steve Langasek]

Hi Matthias,

On Thu, Aug 08, 2013 at 08:09:57PM +0200, Matthias Klose wrote:

> I had to look at many "core" packages, which are needed for a normal and a
> buildd chroot, and and which are needed to build these packages for the
> chroots.

Given that the core packages are almost all maintained in Debian with
minimal if any changes, perhaps it would be useful to discuss this on
debian-devel [as well|instead]?

> Parallel builds

> Unfortunately not enabled very often. The buildds set
> DEB_BUILD_OPTIONS="parallel=<N>", and if the build supports parallel builds, it
> can speed up the build substantially. Common mistakes:

>  - debhelper based packages don't call with --parallel

So for dh(1) rules, this just means calling 'dh $@ --parallel'.  Easy to
implement once you know it, but not very widely advertised.  Perhaps we
should get this into the default debhelper rules.tiny example, so that
maintainers are encouraged to do the "sensible" thing by default and only
disable parallel builds when there are problems?

>  - debhelper based packages overwrite dh_auto_build and don't call
>    --parallel

If you are overriding dh_auto_build to call dh_auto_build with additional
options, this isn't needed.  dh --parallel will set the following variable
in the environment:

  DH_INTERNAL_OPTIONS=-O--parallel

This will be picked up by dh_auto_build.  So there's only an issue if you
are skipping dh_auto_build in favor of your own invocation, in which case
--parallel probably doesn't do what you want anyway?

> Documentation builds

> Many packages always build the documentation, even when only building the
> architecture dependent packages (that is, on our slowest buildds).  The
> easy solution where the upstream docs are not built by default is to build
> these in an extra target.

> However I didn't find any good recipes how to write a modern debhelper or
> semi-modern cdbs rules file how to correctly configure a package to
> disable the docs build for an arch-only build, and enable the docs for an
> arch-indep build.  Pointers welcome.

In the past, the problem here was that the build-arch target was optional,
so dpkg-buildpackage always called the 'build' target instead which meant
documentation was included.

But although the build-arch target will now be used by dpkg-buildpackage,
this isn't a very good interface for controlling dh(1) behavior, since
overriding these targets would bypass more than just dh_auto_build.

I would suggest something like the following:

override_dh_auto_build:
ifneq (,$(findstring foo-doc, $(shell dh_listpackages)))
	dh_auto_build -- all docs
else
	dh_auto_build
endif

Tweaked of course for whatever the right targets are for building docs vs.
building programs in your case.

If you need to change options to both dh_auto_build and dh_auto_configure to
disable the docs build, it's probably easier to override the
build{,-indep,-arch} targets directly.

(Interestingly, it seems that dh assumes that when called for binary-indep,
it doesn't need to call dh_auto_configure... strange...)

Cheers,
-- 
Steve Langasek                   Give me a lever long enough and a Free OS
Debian Developer                   to set it on, and I can move the world.
Ubuntu Developer                                    http://www.debian.org/
slangasek at ubuntu.com                                     vorlon at debian.org
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: <https://lists.ubuntu.com/archives/ubuntu-devel/attachments/20130808/6b76e1bb/attachment.pgp>