[MERGE] Allow stacking for --dev6

[John Arbash Meinel]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Attached is a patch which has a few improvements. I'm leaving the
original text around so people don't have to read both.

John Arbash Meinel wrote:
> The attached patch implements stacking support for
> --development6-rich-root formats.
> 
> 1) I chose to just allow stacking, rather than require a format bump. No
> on-disk structures really changed, and it is a dev format. Older clients
> will abort complaining that it isn't stackable, but that's ok. (They
> won't violate any stacking invariants.)
> 
> 2) GroupcompressVersionedFiles already supported stacking, as part of
> the versionedfiles interface tests I had to implement it a while ago.
> 
> 3) The only real changes where to update
>   a) GroupCHKStreamSource.get_stream_for_missing_keys() to return a
>      stream that includes all referenced CHK pages for a given
>      inventory.
>   b) Update ResumedPack to support CHK indexes.
>   c) Add ResumedGCPack just because I needed a way to force
>      _get_external_refs() to return the empty set, rather than thinking
>      it new how to understand the 'value' field of indexes.
> 
> 4) There were a few tests updated, for example:
> 
>    TestPackRepository.test_suspend_write_group (et al), weren't passing
>    make_repository(..., format=self.get_format()), which meant they were
>    all only testing the default format.
> 
>    The only complaint here, is that I only caught this by accident (the
>    test wasn't failing when I new it should.)
> 
>    I think the problem is that our other per-X tests override something
>    so that 'self.make_branch()' creates the format-under-test, while
>    TestPackRepository requires each test do it manually.
> 
> 
> I'm pretty confident with this patch. I'm doing manual testing now. So
> far, it is looking good.
> 
> I'd like to get this into 1.15 if possible. (Since rc1 is this Friday,
> that doesn't seem onerous.)
> 
> John
> =:->

This adds a few code cleanups (like removing 'bork'). It also fixes
'get_search_for_missing_keys()' when the format *doesn't* support
chk_bytes, and a small improvement that we don't try to get an empty
stream anymore.

Now with the manual testing, I've run into a bug when accessing via
bzr+ssh (it works fine with local access).

I tracked it down to code trying to get the 'ghost' revisions from the
stacked-on location. The specific issue is with:

  revisions = search.get_keys()

This does:

  keys = [key for key, parents in graph.iter_ancestry(heads)
          if key != NULL_REVISION]

However, that ignores the fact that 'iter_ancestry()' returns ghosts.
You can find them by checking "parents is None".

The reason it works in XML formats, is because there we actually do:

        revs = search.get_keys()
        graph = self.from_repository.get_graph()
        revs = list(graph.iter_topo_order(revs))

^- And "graph.iter_topo_order()" silently strips ghosts. (I assume
because it just calls get_parent_map() and iterates over the results,
which happens to not include any ghosts.)

I'm trying to understand what code is broken. My gut feeling is that
"search.get_keys()" should not be returning the ghosts...

Anyway, this patch shouldn't be merged as is, until this gets resolved,
but I think the bulk of it can be reviewed now.

Thoughts?
John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkoIo5gACgkQJdeBCYSNAAPVNQCglxbEMyVcSruAoEbrWcUVQdwW
iTkAn28DiXZbj8yC3+qpPkd/k74a+4vh
=JxSc
-----END PGP SIGNATURE-----
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: 1.15-gc-stacking.patch
Url: https://lists.ubuntu.com/archives/bazaar/attachments/20090511/a800eb98/attachment-0001.diff