[RFC] TreeTransform and 'iter_files_bytes()'

[John Arbash Meinel]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I was looking over the time for "bzr co" with the new brisbane-core
formats. And I was surprised to see that the time for "writelines()" was
6.6s, when it was only ~1.0s if we had an 'accelerator_tree'.

It turns out that TreeTransform.create_files(content, ...) is treating
'content' as an iterator, and calling "out.writelines(content)".

However, when you start with Repository.iter_files_bytes() the content
object is a *string*.

And writelines() faithfully iterates the entire string, and writes the
content one byte at a time.... :(

With this simple fix:
=== modified file 'bzrlib/repository.py'
- --- bzrlib/repository.py        2009-03-24 19:19:26 +0000
+++ bzrlib/repository.py        2009-03-24 21:16:11 +0000
@@ -1892,7 +1892,7 @@
         for record in self.texts.get_record_stream(text_keys,
'unordered', True):
             if record.storage_kind == 'absent':
                 raise errors.RevisionNotPresent(record.key, self)
- -            yield text_keys[record.key], record.get_bytes_as('fulltext')
+            yield text_keys[record.key], record.get_bytes_as('chunked')

     def _generate_text_key_index(self, text_key_references=None,
         ancestors=None):


The time for 'bzr co' of a launchpad-gc format tree drops from 17s down
to 11s.

Now, I don't have a great knowledge for the expected user's of
'iter_files_bytes()', but I don't think we want to be passing strings to
TreeTransform.create_file().

Either that, or TT.create_file() should do:

if isinstance(content, str):
  out.write(content)
else:
  out.writelines(content)

This is a rather huge difference for 'bzr co' times....

At least, I think that is the easier fix, rather than changing the api
of "iter_files_bytes()" to return a 'chunked' format. Mostly because it
is a more significant API change.

Thoughts?

John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAknJURMACgkQJdeBCYSNAAOwxQCgzfgXJUwjxpJLwwCahHGZ8bJv
6fQAnRybSIsJ/sYR52Ne5/v5jPggon4B
=TnZy
-----END PGP SIGNATURE-----