Work flow on large repositories

[Chris Hecker]

Ah, sorry, I thought the whole .bzr directory was only 20mb and was 
confused!

Very interested to hear responses!

Chris


On 2010/07/27 21:01, Michael Hope wrote:
> 'bzr revno 4.4' shows 93541 revisions.  The mirror .bzr directory is
> 549 MB.  The 4.4 branch directory is 20 MB.  The exported size is 559
> MB over 63000 files.  It's reasonably big.
>
> -- Michael
>
> On Wed, Jul 28, 2010 at 3:57 PM, Chris Hecker<checker at d6.com>  wrote:
>>
>> It seems like "large repository" means amount of history?  20mb is not very
>> big at all for the .bzr, I think.  How many revisions are there?
>>
>> My repo is 1k revisions, but 55mb, and operations are reasonably fast
>> (creating a new working copy is a little pokey, 1300 files, but not too bad,
>> maybe 10 seconds).
>>
>> I'm very interested in this topic.  I have binary files in my repro (maya
>> models and animations, photoshop psds, and audio files), and I'm worried
>> about it blowing up in my face at some point, but so far it's been fine and
>> I'm loving bzr relative to svn.
>>
>> Chris
>>
>>
>> On 2010/07/27 19:40, Michael Hope wrote:
>>>
>>> Hi there.  I'm working on the gcc-linaro branch which is stored under
>>> bzr and hosted on Launchpad.  This is a fairly big branch as it was
>>> imported from upstream SVN and contains a large amount of history.
>>>
>>> Most of the work is day-to-day changes on topic branches.  I also want
>>> to run a buildbot style program that continually updates and builds
>>> the latest.
>>>
>>> My issue is that the various operations are taking too long.  Could
>>> anyone suggest tricks or a different work flow to speed things up?
>>>
>>> Some of the operations include:
>>>
>>> Creating a mirror branch by doing init-repo, branch lp:gcc-linaro/4.4.
>>>   The finding revisions stage takes about 10 minutes at 1kB/s.  The
>>> download stage is much faster.
>>>
>>> Day-to-day work is done on topic branches.  Creating the branch takes
>>> 46 s, 250 MB of RAM, and creates a 20 MB .bzr directory.  Pushing this
>>> branch to LP for merging involves pushing the full 20 MB, but this is
>>> acceptable.
>>>
>>> Doing a bzr pull on the 4.4 mirror directory may more than half an
>>> hour and more than 500 MB of memory.
>>>
>>> Doing a bzr checkout takes over 20 minutes and 800 MB of memory on my
>>> fastest machine.  On my netbook and ARM board this causes significant
>>> swapping.  I've yet to complete a checkout on either.
>>>
>>> I'd also like to share the mirror with other local machines to skip
>>> downloading the same 500 MB many times.  Running bzr serve and then
>>> checking out causes 100 % CPU usage for more than 10 minutes on the
>>> host.
>>>
>>> These numbers were with 2.2b4.  2.2 is significantly better than 2.1.
>>>
>>> -- Michael
>>>
>>>
>>
>