[MERGE] Developer doc: container format

[Aaron Bentley]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Andrew Bennetts wrote:
> +This document describes the proposed container format for streaming and
> +storing collections of data in Bazaar.  Initially this will be used for
> +streaming revision data for incremental push/pull in the smart server for
> +0.18, but the intentional is that this will be the basis for much more

^^^ intention, not intentional

> +It would be nice if we could combine multiple containers into a single
> +stream by something no more expensive than concatenation (e.g. by omitting
> +end/start marker pairs).  

^^^ Because names are not required to be universally unique but must be
locally unique, I don't believe this has been achieved.  Names must be
examined and potentially rewritten when concatenating two containers.

Concatenation could be safely performed without checking names if you
had some external guarantee that names between two containers were
unique, but that's a subset of cases, and I'm not sure how commonly we
will have that external guarantee.

> +No length-prefixing of entire container
> +---------------------------------------
> +
> +The overall container is not length prefixed.

For consistency, this should be "length-prefixed".

> +It is acceptable to have records with no explicit name, if the expected
> +use of them does not require them.  For example:
> +
> +  * a record's content could be self-describing in the context of a
> +    particular container, or
> +  * a record could be accessed via an index based on SHA-1, or 
> +  * when streaming, the first record could be treated specially.


Thanks for including that explanation.  It makes more sense now.

> +Specification
> +=============
> +
> +This describes just a basic layer for storing simple series of "records".

I think you mean "storing a simple series"

Aaron
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFGaEdh0F+nu1YWqI0RApU+AJ0W1s11lmrCHr3S2lyhJOFq3f8n7wCfU3qh
rUPISRrp4oRNoiqplWk8dEA=
=2duM
-----END PGP SIGNATURE-----