[MERGE] Prevent knit corruption by checking parents

[John Arbash Meinel]

Aaron Bentley wrote:
> John Arbash Meinel wrote:
>>> Aaron Bentley wrote:
>>>
>>> I'm a little concerned about the performance implications of this
>>> change, because we do have to extract the parent contents in order to
>>> find the sha1 sum.
> 
> I agree.  I would like the sha1s to go in the knit index, so that
> maintaining integrity isn't expensive.

Though then every read of the index is more expensive because you are
adding a lot of data. Most likely it is the best tradeoff, though.

...

>>> So, locally, it is costing us 20-40ms to do this. Which I am okay with.
>>> However, over the remote connection, it is costing 540ms. Which equates
>>> to something like 18 round trips (at the 30ms latency of SocketDelay). I
>>> assume that it has something to do with the number of files affected, etc.
> 
> Of course, the check is more necessary when dealing with remote
> repositories...
> 
>>> So I like the integrity checking, but I'm concerned about how much
>>> overhead it generates. Because of that, I'm not comfortable giving it
>>> the green light without having more people look over it.
> 
> I understand your caution.  It really disturbs me that using the
> standard API correctly can allow knits to become corrupt, though.
> 
> Aaron

I'm not vetoing it. In fact, I think we probably do need something like
it. I don't think that 'using the standard API correctly can allow knits
to become corrupt' is a correct statement, though. The only way it can
become corrupt is if parents are corrupt (because there is a
disagreement). Then that will cascade to all derived texts, up until
another full cacherev. And it will be detected the next time the texts
are read. So we do have some safety catches present.

I think what I would like to see the most, is for revisions to get the
testament sha1 as part of the revision text. Then you can check that all
file contents match, just by checking a single sha1. But that is for the
future.

For now, I just want Martin & Robert to decide if the performance impact
is worth, considering this is a fairly unlikely and rare case of
corruption. I'm probably about +0.6 myself. But I would rather the
system be slower and more robust. But I know one of our main complaints
right now is that we are slow. (Sort of a: users see performance, they
don't *see* robustness, until it bites them really hard).

John
=:->

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 254 bytes
Desc: OpenPGP digital signature
Url : https://lists.ubuntu.com/archives/bazaar/attachments/20060922/c25accdf/attachment.pgp