plugin: sftp transport

[John A Meinel]

Martin Pool wrote:
> On 20/10/05, John A Meinel <john at arbash-meinel.com> wrote:
>
>>>>I know there has been some discussion (and I believe it was  decided)
>>>>about
>>>>using Twisted, which also provides sftp support. I'm guessing what
>>>>you've
>>>>done can probably be adapted for it. But I know one reason I wasn't
>>>>persuing it heavily, is that I was waiting for Twisted to be  integrated
>>>>first.
>>>
>>>A large part of it could probably be salvaged if we switch to  Twisted,
>>>but I won't hold my breath for that.  Twisted is an event- based
>>>framework, so it would mean changing a lot of the flow of bzr  to suit
>>>twisted.  (This is one reason I prefer synchronous code over
>>>event-based: Simpler is better.)
>>
>>Simpler, but not without loss of functionality. And it looks like we
>>need to go async for performance reasons when downloading lots of code,
>>and to enable nice responsive guis to be built. I'm only about 60%
>>convinced, but it seems to be a general consensus that it is the least
>>evil thing. :)
>
>
> I'm still only about 75% convinced myself.  If we can acheive our
> performance and function goals without using twisted that would make
> me more comfortable.  Being purely synchronous is probably not enough
> but perhaps it can be confined to one part of the code.
>

That was the original idea about Transport.*_multi (at least the
performance concerns. It doesn't really handle the event based GUI very
well)

But I did think of a use case which would be nicer with async. If you
are branching or pulling from a remote tree, you can create a queue of
files which need to be downloaded. You start with revision-history, once
that comes in, you grab the last revision, and request that, along with
that entries inventory.
When the inventory comes in, you can add the texts you don't have to the
queue.
When the revision comes in, you can request it's ancestors, along with
their inventories.
This at least seems like a fancy async system, where you keep making
requests for new stuff as the information comes in.

On the other hand....
I guess if you just requested the entire inventory.weave you can get
most of this from local operations. Since it has all inventories, and
the ancestry information, etc (you still need to make a request for all
the revisions, but that is easily a get_multi).

However, .bzr/inventory.weave is now 1.5MB, and can't be broken up
(yet). So if you are doing a bzr pull of 1 revision, it has to download
at least 1.5MB.
In the future, when we have an indexed append only weave format, in
theory you could only download the portions of the inventory.weave that
you don't already have. This should be contained in the index file, so
really you probably could make some pretty efficient requests just with
get('inventory.weave.idx'), and then get_partial_multi() all of the
chunks that you don't have in the local weave, and a get_multi() all of
the revision store entries you don't have.
...

>>>>We might also consider having a weakref dictionary of remote hosts
>>>>that we
>>>>are currently connected to, and their connection object. I know that I
>>>>believe everything will be called through "clone()" but I suppose  it
>>>>would
>>>>be possible for a second connection to be attempted. A weakref
>>>>dictionary
>>>>lets you use the connection if it exists, but it also doesn't hang  on to
>>>>it (letting it disconnect) when all real references go away.
>
>
> Or maybe just hang on to the indefinitely, or until the connection drops.

My concern here is front-ends that are using bzrlib. For instance,
Emacs, or some other gui. The common thing to do is to connect for a
single operation, and then disconnect. (status, diff, merge, etc). And I
believe your gui is going to be running for hours, versus bzr the cli
running for seconds (in a perfect world <1s).

I don't *think* we want to make gui programmers call a "disconnect()"
operation periodically, though that would also be a solution.

John
=:->

>
> --
> Martin
>

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 253 bytes
Desc: OpenPGP digital signature
Url : https://lists.ubuntu.com/archives/bazaar/attachments/20051019/80a0b72e/attachment.pgp