RFC: streaming knit repository fetch interface

[Robert Collins]

On Sat, 2007-06-09 at 12:35 +1000, Martin Pool wrote:
> That sounds fine.
> 
> So this would be activated by an InterRepository that matches only
> when the two are using both knits and the same internal formats?

Kindof. In the smart server there will be a proxy object, like
RemoteRepository, that represents the clients repository. Conversely on
the client the RemoteRepository object is the one that will match when
the client is pushing.

This is why the data stream will include its own signature such that the
streams can be checked for sanity on each end.

> As it inserts the hunks it probably needs to check that they reproduce
> the correct sha.  There maybe should be a knit method to extract and
> insert these hunks, if there is not already one.

knit joining already exists, I don't plan to rewrite that at this point.
This is largely a stop gap measure. We do check we get the right sha on
data extraction: this method will be *no more* prone to bad data than
our current fetch logic.

> On 6/8/07, Robert Collins <robertc at robertcollins.net> wrote:
> > As part of Andrew's work on the smart server it would be great to
> beat
> > the latency demon in the short term. The following sketch is for an
> > interface to go on Repository to allow structured retrieval of all
> the
> > data needed to perform a fetch, given that you know the revisions to
> be
> > included.
> >
> > I thought the easiest way to show the interface would be prototypes:
> >
> > def _get_revision_knit_data_stream(revision_ids_to_include):
> >     """Obtain an iterator over revision data from this repository.
> >
> >     :param revision_ids_to_include: The revision ids to be gather
> up.
> >         For each revision id, the data stream will include:
> >          - File texts (as a blob of knit gzip hunks)
> >          - File graph data (as a blob of compressed text)
> >          - Inventory texts (as a blob of knit gzip hunks)
> >          - Inventory graph data (as a block of compressed text)
> >          - Signature texts (as a series of knit gzip hunks)
> >          - Revision texts (as a series of knit gzip hunks)
> >     :return: An iterator. Each item yielt by the iterator will be a
> 
> "yielded"  <http://en.wiktionary.org/wiki/yield>, cute though 'yielt'
> is :)
> 
> As I said in Andrew's review, I think it's a good idea if new rpcs
> explicitly declare their parameters in the Remote layer, plus the bzr
> version that added them.

This isn't an RPC. Its a local method that the rpc which is yet to be
finely designed or proposed will use. I think that recording the added
bzr version makes sense for all api's actually. However adding it before
its accepted to merge doesn't make that much sense :).

Rob

-- 
GPG key available at: <http://www.robertcollins.net/keys.txt>.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : https://lists.ubuntu.com/archives/bazaar/attachments/20070612/a1b1c5d9/attachment.pgp