[MERGE] fetch refactoring resend

[John Arbash Meinel]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Robert Collins wrote:
> Better submit location ftw.
> 
> 

So one thing that sticks out for me, is that you didn't change the logic
ordering in "get_stream()".

Specifically, you still do:

+        revs = search.get_keys()
+        graph = self.from_repository.get_graph()
+        revs = list(graph.iter_topo_order(revs))
+        data_to_fetch = self.from_repository.item_keys_introduced_by(
+            revs, self.pb)
+        text_keys = []

So you still sort the revs topologically, and call
"item_keys_introduced_by".

This is necessary for *knit* compatibility, but not for *pack*
compatibility. And has the major downside that it reads the inventory
objects 2x.

Instead, what we really want is to be able to stream across the
inventory records, and build up what we need to fetch next. (Perhaps you
have another 'get_stream()' implementation planned?

+                # Before we process the inventory we generate the root
+                # texts (if necessary) so that the inventories references
+                # will be valid.
+                for _ in self._generate_root_texts(revs):
+                    yield _

^- Using "_" as a real variable seems wrong. It certainly seems like
this *should* be:

for result in self._generate_root_texts(revs):
  yield result


John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkmex/QACgkQJdeBCYSNAANQCwCgw+4oWlWxI6HayPBzjHoC4HOd
ztcAnjuuvBTi65UI8eWXshVMZuFVHAj1
=PJaN
-----END PGP SIGNATURE-----