Deploying with Bazaar (or how a big repo can make you crazy)

[INADA Naoki]

> space (ubuntu 11.10 bzr 1.4.1). Doing a bzr branch using http was

I think newer Bazaar may help you. bzr 1.4.1 is too old to work with Launchpad.
Could you try bzr 2.5 ?

On Fri, Mar 9, 2012 at 1:41 AM, Leonardo Santagada <santagada at gmail.com> wrote:
> People on the IRC channel said I should share my use story with the
> mailing list, and that is what I will try to do.
>
> Yesterday I was having problems trying to deploy a bazaar repo
> (lp:openobject-addons/6.1) to a server with  600mb ram and no swap
> space (ubuntu 11.10 bzr 1.4.1). Doing a bzr branch using http was
> getting the process killed by the OOM killer because it was using all
> the avaliable ram. The repository is 493mb in size locally but I think
> it tries to transfer around 600mb+ of data. So here were my steps to
> try to fix the problem:
>
> First people on IRC told me that using ssh is better because it would
> have to transfer less data and that the protocol is smart. My first
> problem is that launchpad doesn't support anonymous access to ssh
> although doing so seems to be very simple (is it still made using the
> twisted ssh implementation?)  here is the bug report
> https://bugs.launchpad.net/launchpad/+bug/493389 marked as won't fix.
> So I had to create a launchpad user for a headless machine that only
> downloads code and create/upload a ssh key to launchpad (something
> that isn't easily automated, but I will reuse the key to other
> machines). Surprise, it did not work, actually it transfered like 480
> mb before being killed and was 10x slower than http. Looking at
> .bzr.log you can see the process was still only fecthing data. So
> simple bzr branch was out of the question.
>
> Then someone said to bzr branch -rN where N is a small number to see
> if that would work. I tried to get the speed back so tried http again,
> apparently it tried to download the whole repo again so I stoped it to
> try ssh. SSH worked but then I would have to manually split the
> download so bzr doesn't eat the whole memory. That is way too manual
> to my taste so I gave up on that also.
>
> I don't really need the whole history in the servers so what I tried
> was using bzr branch --stacked and it worked fine over ssh (again over
> http it tries to download everything just to throw it out after). Now
> I don't know if bzr pull is going to be as fast using a stacked branch
> as a complete repo and if the bugs people tell about --stacked are
> going to show their ugly head on our servers, but at least it's
> simpler than using bzr for development and then a complex mix of shell
> and rsync for deployment.
>
> Talking about a huge repo, this one has 493mb and around 40k rev. I
> used fastexport to see how big it would be in git and another bad news
> it gets a tad smaller there 401mb after a git repack -a -d -f -F
> (don't ask about all those flags, git cli is crazy). This goes against
> the benchmarks posted on bzr, should they be updated or something?
>
> Why does bzr uses so much memory do to a simple branch, and is
> --stacked the best way to do source deployment?
>
>
> --
>
> Leonardo Santagada
>



-- 
INADA Naoki  <songofacandy at gmail.com>