Work flow on large repositories

Wed Jul 28 07:54:55 BST 2010

I am currently happily using bzr with a 1.2Gb branch through judicious use
of shared repositories and switching.  Basically, I have one heavy-weight
checkout of the 1.2Gb branch in a shared repository, where branches are
created with no working trees by default.  Then I switch the heavy-weight
checkout between "features", before committing to the mainline.  The obvious
downside is that you can only work on X number of features at the same time
(where X is the number of heavy-weight checkouts you have)... though
switching this checkout between features is a very rapid operation (~10s on
mine roughly).

Basic set up is:
# Setup the shared bzr repo
cd /some/repo/
bzr init-repo --no-trees --default .

# Make a clean branch of mainline
bzr branch /mainline/bzr/branch mainline

# Make a feature to play with
bzr branch mainline some-new-feature

# Make a checkout to play with
bzr co some-new-feature working_1
*hack and play....*
*bzr commit etc.*

# Make a new feature to play with
bzr branch mainline some-other-feature

# Switch the heavy workout to the new feature DONT FORGET TO COMMIT FIRST
(though bzr won't lose your changes... it's polite that way)
cd working_1
bzr switch ..\some-other-feature
# Commits will now go to some-other-feature

To make this convenient on a day-to-day- basis, I've wrapped up common
operations (like creating the branch and then binding the heavy checkout to
the new feature branch) up in script files.  So to start a new feature, I do
"create-feature some-other-feature", and the scripts know my directory
layout to take care of the branching etc.

Anywho... not sure if that is at all relevant to your use case, but it is
working quite well here for ~8devs who have only just migrated from SVN...
no data has been accidently lost yet either which is a bonus :).  If it is
relevant for you, I also have more information about how we mirror our
branches onto a shared network repository and other such magics that go to
making the setup work well for us.

Philip

On Wed, Jul 28, 2010 at 3:29 PM, Chris Hecker <checker at d6.com> wrote:

>
> Ah, sorry, I thought the whole .bzr directory was only 20mb and was
> confused!
>
> Very interested to hear responses!
>
> Chris
>
>
>
> On 2010/07/27 21:01, Michael Hope wrote:
>
>> 'bzr revno 4.4' shows 93541 revisions.  The mirror .bzr directory is
>> 549 MB.  The 4.4 branch directory is 20 MB.  The exported size is 559
>> MB over 63000 files.  It's reasonably big.
>>
>> -- Michael
>>
>> On Wed, Jul 28, 2010 at 3:57 PM, Chris Hecker<checker at d6.com>  wrote:
>>
>>>
>>> It seems like "large repository" means amount of history?  20mb is not
>>> very
>>> big at all for the .bzr, I think.  How many revisions are there?
>>>
>>> My repo is 1k revisions, but 55mb, and operations are reasonably fast
>>> (creating a new working copy is a little pokey, 1300 files, but not too
>>> bad,
>>> maybe 10 seconds).
>>>
>>> I'm very interested in this topic.  I have binary files in my repro (maya
>>> models and animations, photoshop psds, and audio files), and I'm worried
>>> about it blowing up in my face at some point, but so far it's been fine
>>> and
>>> I'm loving bzr relative to svn.
>>>
>>> Chris
>>>
>>>
>>> On 2010/07/27 19:40, Michael Hope wrote:
>>>
>>>>
>>>> Hi there.  I'm working on the gcc-linaro branch which is stored under
>>>> bzr and hosted on Launchpad.  This is a fairly big branch as it was
>>>> imported from upstream SVN and contains a large amount of history.
>>>>
>>>> Most of the work is day-to-day changes on topic branches.  I also want
>>>> to run a buildbot style program that continually updates and builds
>>>> the latest.
>>>>
>>>> My issue is that the various operations are taking too long.  Could
>>>> anyone suggest tricks or a different work flow to speed things up?
>>>>
>>>> Some of the operations include:
>>>>
>>>> Creating a mirror branch by doing init-repo, branch lp:gcc-linaro/4.4.
>>>>  The finding revisions stage takes about 10 minutes at 1kB/s.  The
>>>> download stage is much faster.
>>>>
>>>> Day-to-day work is done on topic branches.  Creating the branch takes
>>>> 46 s, 250 MB of RAM, and creates a 20 MB .bzr directory.  Pushing this
>>>> branch to LP for merging involves pushing the full 20 MB, but this is
>>>> acceptable.
>>>>
>>>> Doing a bzr pull on the 4.4 mirror directory may more than half an
>>>> hour and more than 500 MB of memory.
>>>>
>>>> Doing a bzr checkout takes over 20 minutes and 800 MB of memory on my
>>>> fastest machine.  On my netbook and ARM board this causes significant
>>>> swapping.  I've yet to complete a checkout on either.
>>>>
>>>> I'd also like to share the mirror with other local machines to skip
>>>> downloading the same 500 MB many times.  Running bzr serve and then
>>>> checking out causes 100 % CPU usage for more than 10 minutes on the
>>>> host.
>>>>
>>>> These numbers were with 2.2b4.  2.2 is significantly better than 2.1.
>>>>
>>>> -- Michael
>>>>
>>>>
>>>>
>>>
>>
>

-- 
Philip Peitsch
Mob: 0439 810 260
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://lists.ubuntu.com/archives/bazaar/attachments/20100728/3df6835b/attachment.htm 