large files and storage formats

[Chad Dombrova]

On Jul 8, 2010, at 10:14 PM, Robert Collins wrote:

> Hi.
> 
> You certainly can implement a loose blob store for bzr. We had one in
> the past but they scale very poorly which is why we have the pack
> based formats today.

what aspects did you find did not scale well?  status?  update/checkout?  i know that git uses a combination of loose objects plus packs.  i might have to do something similar.

> Yes, you can add a new format - a git like one, or other weirder
> things - without altering the high level APIs. We currently have:
> - 2a
> - svn
> - git
> - hg
> - team foundation server
> 
> implementations of the "repository' and 'branch' interfaces which the
> high level code can use. So you can cd to a svn directory and run 'bzr
> commit' or 'bzr status' and it will do the right thing.
> 
> There are two challenges you'll have:
> - some of the higher level API's won't do what you want in terms of
> memory use; these will need adjusting (many should - we have been
> working on bigger and bigger file support incrementally for a while
> now).
> - the repository api (the one you need to implement) is fairly thick.
> 
> I wouldn't want to judge man hours for you. I know I can whip up a
> crude blob store in a few hours; a polished one in a few days. But - I
> know the architecture and code backwards.

sounds promising.

before i go down this path, i'd like to better understand the current 2a format.  is there a document somewhere that explains the inner workings of this?  i've been digging through the docs and have not found anything useful in that regard.  i made some commits and watched the .bzr directory and that gave me a very basic understanding:  i gleaned that there is a pack file and a set of 4 index files for each commit, but (among other things) i'm curious if the packs contain deltas or full files.

to learn how to implement a new format should i start by looking through bzrlib.repofmt?

-chad





-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://lists.ubuntu.com/archives/bazaar/attachments/20100708/c78a95d1/attachment-0001.htm