[RFI] Annotations cache & progressive display of annotations

[Martin Pool]

On 16 February 2010 01:37, John Arbash Meinel <john at arbash-meinel.com> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Ian Clatworthy wrote:
>> The MySQL guys recently raised the fact that annotate is slower is 2a
>> format than pack-0.92. Talking to poolie last Friday, this looks like
>> something I could help improve in coming weeks.
>>
>> Before I look into it though, I'd like to start from the hard-earned
>> knowledge and great ideas of others. I'm sure John, Robert, Vincent and
>> others know 10X (well 100X) more than I do in this area. Any deep
>> insights, tips or branches I should start from?
>>
>> I'd also like to hear from end users who use annotate frequently. How
>> often are you using the GUI vs command-line to view annotations? Any
>> thoughts on "lazy-loading" of the data into the GUI so we can begin
>> showing some annotations immediately?
>>
>> Ian C.
>
> "lazy-loading..."
>
> My primary feeling is that it isn't worthwhile for 2 reasons:
>
> 1) You have to work backwards, which isn't always obvious how to make it
> work. Forward annotation is more 'obvious'. Not to say you couldn't work
> out the logic, just that it will be more complex.
> 2) In my testing, almost every file has a Copyright header whose source
> goes all the way back to the first revision of the file. So you have to
> grab the whole ancestry anyway. (Otherwise you could stop as soon as all
> lines were annotated.)
>
> Mostly because of (2), I think a cache is really the only way to make it
> reasonable. *if* we supported just annotating a region of a file, then
> (1) could be a reasonable win. (People tend to care about recently
> introduced text.)

Obviously if you're going to do it incrementally you can't just start
at the first line and iterate through the file, you need something by
which lines can be colored in arbitrary order and perhaps repeatedly.
However it does seem like there's a couple of useful options:

1- first mark everything against the mainline revision in which it was
introduced, then go back and (also?) show the specific revision where
it happened; this might cut it by a factor of 2-3x for typical
branches

2- show lines just as "older" until you discover where they were
changed; so the copyright will light up last but on the other hand
it's probably moderately recent changes that are most interesting.

It's a bit interesting to consider how this would happen in emacs but
you could possibly emit a text stream of linenumber:origin_details
that emacs could reassemble.

These changes probably aren't trivial but they don't seem impossibly hard...
-- 
Martin <http://launchpad.net/~mbp/>