[RFC] VersionedFiles.get_data_stream/insert_data_stream and factory objects etc

[Andrew Bennetts]

Robert Collins wrote:
> So I'm thinking about the best way to keep generalising
> get_data_stream/insert_data_stream down at the VersionedFile(s) level.
[...]
> 
> So I'm thinking of something along the following lines:
> 
> Add a field type to represent a full text *uncompressed* record. These
> would be handled during insertion the same as insertion during commit
> (e.g. with possibly conversion to deltas etc).

Just to be clear, what do you mean by a "field type"?  Is that something to do
with the item_key in a repository data stream?

> get_data_stream on a weave returning a signature of 'weave'.

+1

> Having a base insert_data_stream which does:
>   if format != self.get_format_signature():
>     format, stream_data = adapt_stream(self, \
>         self.get_format_signature, format, stream_data)
>   return self._insert_data_stream(format, stream_data)

+1

> And writing the following adapters:
>  - weave to any knit by yielding everything as a full
>    text
>  - any knit to weave by using the weave as a basis file and using the
>    StreamIndex and StreamAccess facilities but generalised to layer on
>    VersionedFiles not on knits specifically, and yielding full texts
>  - unannotated knit to annotated knit by using the target as the basis
>    to reconstruct the unannotated full text and yielding that as a
>    full text.
>  - annotated knit to unannotated knit by stripping annotations from the
>    hunks and outputting compressed records.

+1

-Andrew.