[RFC] Bundles as repositories

[Aaron Bentley]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Robert Collins wrote:
> I think this has been roughly discussed, but wanted to be sure it had..
> 
> I think we want a compact binary bundle format, with the human text
> overall delta just an ignored-prelude. We can check the prelude matches
> the binary data when processing, or encourage people to run the bundle
> through bzr

This is a good description of what I'm working on now, but I think that
checking preludes should always be done.

> ... Alternatively we could have no human readable section but
> a specific mime type so mutt etc can get bzr to format it for human
> reading.

I worry about the potential for mischief.  People could create a
binary-only bundle that looked like it had a prelude that would be
validated, when in fact, it wouldn't be validated.  And the user
wouldn't see that.

For binary-only bundles, is there a need for base64?

> AIUI we want bundles to have the following properties:
>  - compact representation
>  - able to be used without their contained data being added to
> repositories

^^^ This was not one of my goals.

>  - fast to create
>  - fast to extract data from

I'm trying to accomplish fast installation, (e.g. of knit records), not
fast extraction of fulltexts.  And I'm specifically choosing size over
speed, because of how bundles are usually used.


> To me it makes sense that bundles should then be a
> 'branch-and-repository-in-a-file':
>  - one with the derivable data discarded
>  - one where the data within it has been stored in the most compact
> representation
>  - one that only contains partial history data

Well, actually making them a branch is a new idea to me.  With the
Branch API as fat as it is, I don't know if that'll really be
productive.  It would work best if it was readonly, and if we only had
to simulate the control files, it might be manageable.

> The actual representation might look something like:
> the bundle is a pack file.
> The pack contains records named:
> branch-data
>   tip revision id

^^^ not strictly necessary-- the tip is already provided as "target
revision" with bundles.  But Branch6 stores tip revision_id *and revno*,
 so storing that would make sense.

>   branch nick

^^^ Wouldn't it be better to just put the whole config file in a record?
 With dirstate-tags, the config already controls 90% of the behavior.

> branch-tags
>   tag ...
> [repository data here, representation still unclear to me, though I'm
> sure Aaron has some solid thoughts on this as he has been hacking on
> something that is conceptually very similar]

Current representation is
- - multiparent diffs of file texts, named like so: file:revisionid/filid
- - multiparent diffs of inventories, named like so: inventory:revisionid
- - fulltexts of revisions, named list so: revision:revisionid
- - testament of the tip revision

Support for revision signatures should come in the next day or two.

Also, a table-of-contents, or at least a count of the number of records
is probably a good idea, so we can indicate progress more easily.

> If this is agreeable, I'll create a design document trying to ensure we
> have solid motivation and so on documented.

Yeah, generally speaking, it feels like a good move.

Aaron
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFGcj250F+nu1YWqI0RApjVAJ9HMF3jZ3twbNhDt0/43K3Cj13HwACeNtHu
Tv/f7VY6Pa/A27Gf/jfeCW8=
=W2oD
-----END PGP SIGNATURE-----