annotate & stacking

[John Arbash Meinel]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Robert Collins wrote:
| On Mon, 2008-06-23 at 14:09 -0500, John Arbash Meinel wrote:
|> -----BEGIN PGP SIGNED MESSAGE-----
|> Hash: SHA1
|>
|> Robert Collins wrote:
|> | So,
|> |     knit.py has an Annotator object that looks inside knits to perform
|> | annotations. This appears to duplicate a bunch of logic and also looks
|> | inside the knit at the _index object etc.
|> |
|> | To stack cleanly, I need something that can live within the knit, and
|> | then build on top of the public VersionedFiles interface (or indeed
|> | something that lives solely outside and using the public interface).
|> |
|> | John - I know you have been working on annotate() recently; are we
are a
|> | position to remove annotate() from VersionedFiles, or otherwise tweak
|> | things to meet the constraints above?
|> |
|> | -Rob
|>
|> There is quite a bit that is knit/pack specific about _KnitAnnotator.
|> There are parts that could be made more generic, but it certainly
|> expects knit-deltas (for example).
|>
|> It would certainly be possible to write an Annotator class that would
|> just process fulltexts and have it be fairly simple. However, I'm
|> guessing it would be quite a bit slower. Being able to re-use the
|> existing deltas for left-hand parents is a pretty big win for a lot of
|> the test files I was using.
|>
|> However, if we go with a full "policy" based system, then we may not
|> even be able to use the left-hand blocks, and if the underlying storage
|> uses xdelta, it won't have something we can use either. (xdelta is much
|> more of a gzip across 2 texts, and throw out the first set, so you get
|> stuff like RLE, etc.)
|>
|> So... I would probably go with a generic Annotator that just works on
|> full texts and ancestries, and implement a custom one that can take
|> advantage of more intimate knowledge of the layout of knits/packs.
|>
|> Also, some stuff is pulled into the annotator, so that it can be smart
|> about which pieces need to be cached, and what can be freed, etc.
|
| So, I think its fine for the hunks that are *in* the local knit to be
| directly examined (though get_record_stream allows them to be accessed
| without using private functions). But once the knit doesn't have the
| content locally is where the problem arose.
|
| As you're working on annotation already, is this something we can
| parallelise?
|
| -Rob

I'm working on a different layer of annotate, so you can change this at
your will.

John
=:->

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkhgDqQACgkQJdeBCYSNAANHQgCgzqflSujjW43cpkWUbdV1qNLD
9hoAoIPOz3R6cAMfyPispW3Oblrn8EA9
=yLvM
-----END PGP SIGNATURE-----