[MERGE] be stricter in knit record serialisation, and bundle-generation-cornercase

[John Arbash Meinel]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Robert Collins wrote:
| I encountered a possible knit corruption issue with mpdiffs during the
| versionedfiles work. It occurred because I had not ported over an
| optimised code path, which caused an error, and pulling on this thread I
| uncovered the following, which is pithily reproducible via the test in
| the patch at the end of this mail (which fails on bzr.dev). It turns out
| that what I describe below was caused by bug 234748, which Martin was
| already working on - so I've just committed my tests and the correctness
| changes, to be integrated with his work.
|
| I chatted with Aaron to learn more about mpdiffs and he helped me get a
| handle on this, and sanity checked the early test cases - thanks!
|
| Firstly the corruption. It happens when generating the knit record for
| an annotated knit, for a fulltext, on top of a delta, with the last line
| unchanged, and the last line has no trailing eol. We get the wrong
| output from lower_fulltext() because the content's internal
| representation after merging annotations has no trailing newline (but it
| needs one for serialisation).
|
| Secondly, we have a glitch in make_mpdiffs (rooted in
| VF._extract_blocks) where noeol files do not reuse as much as possible -
| I haven't dug into this as it is quite orthogonal to what I'm working
| towards (by having better mpdiffs I discovered this though:)). To
| reproduce, just inspect the output in the make_mpdiffs test case of the
| 'noeoldup' version - we should reuse the parent:
| MultiParent([ParentText(0, 0, 1, 1)])
| but instead we create from scratch:
| MultiParent([NewText(['line\n'])])
|
| I was going to say that we can't fix the bundle generation because there
| are bzr's out there that will handle the tighter mpdiffs badly; but that
| was before it turned out to be a much more general and common issue - I
| now believe it is possible/likely to cause this just by editing and
| committing files as normal, and thus we won't significantly reduce the
| likelyhood of triggering it by not allowing bundles with the same
| changes.
|
| -Rob
|

BB:approve


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkg/c5gACgkQJdeBCYSNAAPnxgCgjNvdKlIbf8FbqWTED/1o4nFh
lT8AoMgVnu5wvuFd0jbKbQP8O5AW7fAw
=ZyY/
-----END PGP SIGNATURE-----