Launchpad builders - increasing memory spec

Colin Watson cjwatson at
Thu Jul 6 13:36:01 UTC 2017

On Tue, Jul 04, 2017 at 01:22:05PM -0700, Steve Langasek wrote:
> On Tue, Jul 04, 2017 at 04:31:32PM +0000, James Page wrote:
> > Mathias and I where discussing some challenges around fitting the Ceph
> > build with any level of parallelism in to a virtualized launchpad builder
> > (as used for the majority of architectures today).
> > Basically the current memory spec only permits a max-parallel of 2 (on
> > Xenial amd64).  Ceph is a big C++ project so takes some time to build with
> > such a low level of parallelism.
> > Would it be possible to nudge up the default memory allocation on the
> > builders? I use a 16G machine for test builds of i386/amd64 and I've not
> > seen any memory exhaustion with parallel=8.
> > Or maybe have a mechanism to allow a source package to request a higher
> > spec of builder?
> You should probably consult the Launchpad team on this, since the Launchpad
> builder configuration is owned by them and not by the Ubuntu developers
> directly.
> As far as raising the default memory allocation, understand that there is a
> tradeoff between being able to parallelize /within/ a build and being able
> to parallelize across /all/ builds.  If the sole justification for this
> request is that it would speed up the build of one particular package, I
> don't think that's a sufficient reason to reduce the capacity of the
> launchpad build farm as a whole.

Indeed - increasing the size of a single builder would mean reducing the
number of builders we can fit on scalingstack, and I don't think that's
a good trade-off globally.  The current size seems to be a sweet spot,
even if it means that some individual packages are a bit slower to

The build farm spends a lot of time on lots of relatively small builds,
so parallelising across all builders tends to win as a strategy.

> Providing a mechanism to request a builder of a different size seems ok, but
> would need to be designed/implemented in Launchpad; and how do you prevent
> this from being abused?

Our experience is that the build farm delivers best overall performance
when it's divided into as small a number of pools as possible, and as
Dimitri says we pre-boot builder VMs before knowing what build is going
to be dispatched to them, so having some larger builders would require
dividing the build farm into more pools: we'd have to have amd64+i386
(small) and amd64+i386 (large) rather than just amd64+i386, for
instance.  We'd be very resistant to this.

Colin Watson                                       [cjwatson at]

More information about the ubuntu-devel mailing list