[MERGE] Prevent knit corruption by checking parents

[Aaron Bentley]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

John Arbash Meinel wrote:
> Aaron Bentley wrote:
> 
> I'm a little concerned about the performance implications of this
> change, because we do have to extract the parent contents in order to
> find the sha1 sum.

I agree.  I would like the sha1s to go in the knit index, so that
maintaining integrity isn't expensive.

> However, it does look like you are restricting the check to a small set,
> which seems like the best that we can do. 

Technically, we only need to check the leftmost parent, since the delta
is only against that.

> (We don't actually have to
> extract the entire gzip hunk, just the first line, but we probably have
> to read the whole chunk, which is the real performance penalty over a
> remote connection).

This will really suck if the parent happens to be a fulltext.  I don't
really know what to do here, because if we read less, we risk causing
even more readvs.

> 'bzr push' is already pretty slow, and now, on top of pushing data, we
> have to read back data, and verify the sha1 sums.

Yeah.  I don't see an alternative, though.

> So, locally, it is costing us 20-40ms to do this. Which I am okay with.
> However, over the remote connection, it is costing 540ms. Which equates
> to something like 18 round trips (at the 30ms latency of SocketDelay). I
> assume that it has something to do with the number of files affected, etc.

Of course, the check is more necessary when dealing with remote
repositories...

> So I like the integrity checking, but I'm concerned about how much
> overhead it generates. Because of that, I'm not comfortable giving it
> the green light without having more people look over it.

I understand your caution.  It really disturbs me that using the
standard API correctly can allow knits to become corrupt, though.

Aaron
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFFFAEE0F+nu1YWqI0RAjRdAJ940ogjhxF9Z66/jo2Vh+w+MmQdeQCfWOQ6
bbeixzozfr+NyEyL6tQEZDA=
=UWY+
-----END PGP SIGNATURE-----