[RFC] Blob format

[Johan Rydberg]

Matthieu Moy <Matthieu.Moy at imag.fr> writes:

>> If it turns out to be something as fast as knits, then it could very
>> well replace them.  But it is more of a research format. 
>
> It can also be a complement to knits. I think git has something
> similar. For example, you could keep the knit archive, and
> periodically add a big "bundle" or "blob" in addition to this. This
> way, branch would do
>
> 1) get the big bundle (takes bandwidth, but amost no rtt)
> 2) See which revisions are still missing
> 3) fetch them in the usual way for knits.

I suppose we could couple the blob format tightly with the knit format,
and make a blob the concatenated knit-deltas of a (set of) revision(s).

An optimized fetcher could simply bring down the blob and insert the
deltas directly into the knits.  

Use cases:

 * To upload to a blob format repository (e.g., doing a "push"), the
   fetcher would have to identify all changes introduced by the missing
   revisions, iterate over all files and concatenate the deltas together
   into a blob.

   This includes comparing inventories to detect changes, or the
   fileids_altered* API could be used.

 * To install a remotely fetched blob into the local repository, the
   file deltas has to be inserted into the knits and revisions added
   to the revision store.  If done right, there should be no need to
   extract any fulltexts at all.

In other words; a blob could be seen as a bundle, but serialized in
another way to fit our current format. 

An open question is; should BlobRepository implement the whole
functionality of Repository, or just enough to make push/pull work?

~j