large files and storage formats

[Robert Collins]

Hi.

You certainly can implement a loose blob store for bzr. We had one in
the past but they scale very poorly which is why we have the pack
based formats today.

"
so, i'd like some honest opinions:
- is bazaar really so well layered that new storage formats can be
added without the need to rewrite higher level code?
- how difficult a task is this, approximately, in man hours?  (keep in
mind, git's object model has already been implemented in python
(http://samba.org/~jelmer/dulwich/), so i'm mostly concerned with the
time it would take to interface this with bazaar in all the right
places.)

i've looked through the docs and i can't find any information on how
to get started on writing a new storage format (which i take as a sign
that it is probably very difficult).  assuming that this goal is not
laughably lofty, and that there are not other better alternatives, i'd
love some guidance on how this might be pulled off.
"

Yes, you can add a new format - a git like one, or other weirder
things - without altering the high level APIs. We currently have:
 - 2a
 - svn
 - git
 - hg
 - team foundation server

implementations of the "repository' and 'branch' interfaces which the
high level code can use. So you can cd to a svn directory and run 'bzr
commit' or 'bzr status' and it will do the right thing.

There are two challenges you'll have:
 - some of the higher level API's won't do what you want in terms of
memory use; these will need adjusting (many should - we have been
working on bigger and bigger file support incrementally for a while
now).
 - the repository api (the one you need to implement) is fairly thick.

I wouldn't want to judge man hours for you. I know I can whip up a
crude blob store in a few hours; a polished one in a few days. But - I
know the architecture and code backwards.

Someone recently added the team foundation server support, and I
recall him only asking for assistance a few times on IRC; so I'd say
its reasonably straight forward, at least.

-Rob