Work flow on large repositories
Chris Hecker
checker at d6.com
Wed Jul 28 08:06:52 BST 2010
> If it is relevant for you, I also have more information about how we
> mirror our branches onto a shared network repository and other such
> magics that go to making the setup work well for us.
I'd be very interested to hear more details.
Chris
On 2010/07/27 23:54, Philip Peitsch wrote:
> I am currently happily using bzr with a 1.2Gb branch through judicious
> use of shared repositories and switching. Basically, I have one
> heavy-weight checkout of the 1.2Gb branch in a shared repository, where
> branches are created with no working trees by default. Then I switch
> the heavy-weight checkout between "features", before committing to the
> mainline. The obvious downside is that you can only work on X number of
> features at the same time (where X is the number of heavy-weight
> checkouts you have)... though switching this checkout between features
> is a very rapid operation (~10s on mine roughly).
>
> Basic set up is:
> # Setup the shared bzr repo
> cd /some/repo/
> bzr init-repo --no-trees --default .
>
> # Make a clean branch of mainline
> bzr branch /mainline/bzr/branch mainline
>
> # Make a feature to play with
> bzr branch mainline some-new-feature
>
> # Make a checkout to play with
> bzr co some-new-feature working_1
> *hack and play....*
> *bzr commit etc.*
>
> # Make a new feature to play with
> bzr branch mainline some-other-feature
>
> # Switch the heavy workout to the new feature DONT FORGET TO COMMIT
> FIRST (though bzr won't lose your changes... it's polite that way)
> cd working_1
> bzr switch ..\some-other-feature
> # Commits will now go to some-other-feature
>
>
> To make this convenient on a day-to-day- basis, I've wrapped up common
> operations (like creating the branch and then binding the heavy checkout
> to the new feature branch) up in script files. So to start a new
> feature, I do "create-feature some-other-feature", and the scripts know
> my directory layout to take care of the branching etc.
>
> Anywho... not sure if that is at all relevant to your use case, but it
> is working quite well here for ~8devs who have only just migrated from
> SVN... no data has been accidently lost yet either which is a bonus :).
> If it is relevant for you, I also have more information about how we
> mirror our branches onto a shared network repository and other such
> magics that go to making the setup work well for us.
>
> Philip
>
> On Wed, Jul 28, 2010 at 3:29 PM, Chris Hecker <checker at d6.com
> <mailto:checker at d6.com>> wrote:
>
>
> Ah, sorry, I thought the whole .bzr directory was only 20mb and was
> confused!
>
> Very interested to hear responses!
>
> Chris
>
>
>
> On 2010/07/27 21:01, Michael Hope wrote:
>
> 'bzr revno 4.4' shows 93541 revisions. The mirror .bzr directory is
> 549 MB. The 4.4 branch directory is 20 MB. The exported size
> is 559
> MB over 63000 files. It's reasonably big.
>
> -- Michael
>
> On Wed, Jul 28, 2010 at 3:57 PM, Chris Hecker<checker at d6.com
> <mailto:checker at d6.com>> wrote:
>
>
> It seems like "large repository" means amount of history?
> 20mb is not very
> big at all for the .bzr, I think. How many revisions are there?
>
> My repo is 1k revisions, but 55mb, and operations are
> reasonably fast
> (creating a new working copy is a little pokey, 1300 files,
> but not too bad,
> maybe 10 seconds).
>
> I'm very interested in this topic. I have binary files in
> my repro (maya
> models and animations, photoshop psds, and audio files), and
> I'm worried
> about it blowing up in my face at some point, but so far
> it's been fine and
> I'm loving bzr relative to svn.
>
> Chris
>
>
> On 2010/07/27 19:40, Michael Hope wrote:
>
>
> Hi there. I'm working on the gcc-linaro branch which is
> stored under
> bzr and hosted on Launchpad. This is a fairly big
> branch as it was
> imported from upstream SVN and contains a large amount
> of history.
>
> Most of the work is day-to-day changes on topic
> branches. I also want
> to run a buildbot style program that continually updates
> and builds
> the latest.
>
> My issue is that the various operations are taking too
> long. Could
> anyone suggest tricks or a different work flow to speed
> things up?
>
> Some of the operations include:
>
> Creating a mirror branch by doing init-repo, branch
> lp:gcc-linaro/4.4.
> The finding revisions stage takes about 10 minutes at
> 1kB/s. The
> download stage is much faster.
>
> Day-to-day work is done on topic branches. Creating the
> branch takes
> 46 s, 250 MB of RAM, and creates a 20 MB .bzr directory.
> Pushing this
> branch to LP for merging involves pushing the full 20
> MB, but this is
> acceptable.
>
> Doing a bzr pull on the 4.4 mirror directory may more
> than half an
> hour and more than 500 MB of memory.
>
> Doing a bzr checkout takes over 20 minutes and 800 MB of
> memory on my
> fastest machine. On my netbook and ARM board this
> causes significant
> swapping. I've yet to complete a checkout on either.
>
> I'd also like to share the mirror with other local
> machines to skip
> downloading the same 500 MB many times. Running bzr
> serve and then
> checking out causes 100 % CPU usage for more than 10
> minutes on the
> host.
>
> These numbers were with 2.2b4. 2.2 is significantly
> better than 2.1.
>
> -- Michael
>
>
>
>
>
>
>
>
> --
> Philip Peitsch
> Mob: 0439 810 260
More information about the bazaar
mailing list