Suggested changed to the transport API.

[Robert Collins]

On Sat, 2005-12-03 at 12:01 -0600, John A Meinel wrote:

> HTTP objects are not seekable. If you want to only get a portion of the
> file, then you can use "get_partial".

That api was a kludge: 

  [ For readers following along at home, get_partial returned an file
like object that started at an offset and went forward from there. ]

It had essentially the same result for a user as 'open the file; seek to
the offset; start reading', but it made Transport responsible for
seeking, and you could only seek once: you had to open the file again
later to get more data, and there is no possibility of (for instance)
building up a memory map of the regions of the file that *have* been
read.

> By switching to get_partial, it means that not only can you get the
> added bandwidth improvements for sftp and local, but you also can get it
> for http (I believe it doesn't work for ftp).

FTP has 'REST' for most FTP servers these days, certainly an
implementation could try it. You dont need the kludgy get_partial api to
get the bandwidth improvements though - having files that can seek with
a dumb backend protocol is completely feasible.


> If you switch to expecting to have access to the entire file, and then
> seek around it, then for HTTP you have to download the entire thing into
> either a local temporary file, or a StringIO.

That would surely be only a first-slice implementation of seeking with
HTTP. Once you have one data block from HTTP you can request that the
response be for the same instance using the content md5, and that lets
you issue range requests as many times as needed.

> I can certainly see some benefits of switching from a get/put model to
> an open/seek/etc model. But it just isn't supported well under http.

???

Rob
-- 
GPG key available at: <http://www.robertcollins.net/keys.txt>.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : https://lists.ubuntu.com/archives/bazaar/attachments/20051204/e270019c/attachment.pgp