Deploying with Bazaar (or how a big repo can make you crazy)

Leonardo Santagada santagada at gmail.com
Thu Mar 8 16:41:50 UTC 2012


People on the IRC channel said I should share my use story with the
mailing list, and that is what I will try to do.

Yesterday I was having problems trying to deploy a bazaar repo
(lp:openobject-addons/6.1) to a server with  600mb ram and no swap
space (ubuntu 11.10 bzr 1.4.1). Doing a bzr branch using http was
getting the process killed by the OOM killer because it was using all
the avaliable ram. The repository is 493mb in size locally but I think
it tries to transfer around 600mb+ of data. So here were my steps to
try to fix the problem:

First people on IRC told me that using ssh is better because it would
have to transfer less data and that the protocol is smart. My first
problem is that launchpad doesn't support anonymous access to ssh
although doing so seems to be very simple (is it still made using the
twisted ssh implementation?)  here is the bug report
https://bugs.launchpad.net/launchpad/+bug/493389 marked as won't fix.
So I had to create a launchpad user for a headless machine that only
downloads code and create/upload a ssh key to launchpad (something
that isn't easily automated, but I will reuse the key to other
machines). Surprise, it did not work, actually it transfered like 480
mb before being killed and was 10x slower than http. Looking at
.bzr.log you can see the process was still only fecthing data. So
simple bzr branch was out of the question.

Then someone said to bzr branch -rN where N is a small number to see
if that would work. I tried to get the speed back so tried http again,
apparently it tried to download the whole repo again so I stoped it to
try ssh. SSH worked but then I would have to manually split the
download so bzr doesn't eat the whole memory. That is way too manual
to my taste so I gave up on that also.

I don't really need the whole history in the servers so what I tried
was using bzr branch --stacked and it worked fine over ssh (again over
http it tries to download everything just to throw it out after). Now
I don't know if bzr pull is going to be as fast using a stacked branch
as a complete repo and if the bugs people tell about --stacked are
going to show their ugly head on our servers, but at least it's
simpler than using bzr for development and then a complex mix of shell
and rsync for deployment.

Talking about a huge repo, this one has 493mb and around 40k rev. I
used fastexport to see how big it would be in git and another bad news
it gets a tad smaller there 401mb after a git repack -a -d -f -F
(don't ask about all those flags, git cli is crazy). This goes against
the benchmarks posted on bzr, should they be updated or something?

Why does bzr uses so much memory do to a simple branch, and is
--stacked the best way to do source deployment?


-- 

Leonardo Santagada



More information about the bazaar mailing list