[RFC] binary files and merging

[John Arbash Meinel]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

We just had some discussion on IRC, because PDF files may have a header
longer than 1K before they get into their compressed content. So just
reading the first 1K bytes may yield a false negative as text data.

In digging into the code, all the merge algorithms call
'check_text_lines' at some point before they start processing.
Which will check the first 1K bytes for a '\0' character.

On IRC the question was brought up as to whether it would be worthwhile
to check all lines while you are merging. We already have to read the
data, and the code seems to be structured such that you could raise
BinaryFile at any time.

Just something to think about. To help prevent silently corrupting
binary files.

John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2.2 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFEvO9vJdeBCYSNAAMRAjlMAKDB8yPf/sAXk/M+zGDNq5wOPZ5dogCeOd0m
tgrMEqKZIiU5BT/f/otiLxk=
=PWJL
-----END PGP SIGNATURE-----