Extending the storage format with a plugin

[Evan Pipho]

Hello everyone,

I am currently evaluating VCSs for a project and Bazaar is a strong
contender for a few reasons.

1) Multiple workflows, checkouts vs branches both supported against the
same central server
2) GUI Support, several of the contributors to this projects are non
technical (artists, etc) so having fewer steps and a GUI is important.
3) Ability to use lightweight checkouts and stacked branches.

However I have a specific need that seems to not be met well in any of the
current DVCSs, support for large binary files.  There is a large amount of
content that needs to be versioned alongside the code base, including
executables, art assets, and content patch data.

The requirements are fairly simple:
1) Content is not mergeable, can only accept yours or theirs.
2) All content history must be preserved and accessible
3) Should be transparent to users of the existing bazaar tools if possible.
4) Content should automatically use the new storage, possibly based on a
file size trigger (i.e. all files > 5MB) or if they contain a nul byte.
5) Files and content not using the new storage system would fall back to
the current system which works quite well for code and text.

My initial thought is to write a plugin that uses a rolling checksum
similar to rsync to break up the files and store the chunks along with
metadata on how to put them back together again similar to bup (
https://github.com/apenwarr/bup#readme).  When commiting the plugin would
break the new file up in the same way and add any new chunks and update the
metadata for the new revision.

Obviously this is still a half baked idea, but is it even possible?

If so, is there any documentation on this, other than
http://doc.bazaar.canonical.com/plugins/en/plugin-development.html mentioning
that plugins could add new storage formats, my searches for bazaar storage
format plugins came up empty.
Could this be developed as a fictional external source control plugin
similar to
http://doc.bazaar.canonical.com/migration/en/foreign/bzr-on-svn-projects.html ?
or is there a better way?
Are there any existing projects, not only to handle binary files better,
but anything that changes or extends bazaar storage?

Thanks for your time.
-Evan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/bazaar/attachments/20120606/d24cdda6/attachment-0001.html>