New Emacs Bazaar Repository II

[Jason Earl]

I wasn't entirely sure where to send this email, but since most of the
Emacs folk that are involved in the switch to Bazaar seem to be
subscribed to this list I chose the bazaar mailing list instead of
emacs-devel.[1]

For one thing, posting here allows me to thank the bzr developers for
their fine work and specifically thank Ian for his work on the
fast-import plugin.  I really appreciate it.  For another, the Emacs
development crowd tends to be a bit judgmental about bzr and there are
some issues with the newest conversion.  I'm hoping that I can get some
help sorting things out before involving emacs-devel.

Anyway, I've made some progress with the Emacs bzr repository.
Actually, it can more honestly be said that Ian's changes to fast-import
now do a better job with the Emacs repository.  Specifically the missing
tags have appeared.  Unfortunately, however, these tags have appeared as
branches.  The overall size of the repository has also ballooned
significantly, probably due to the increase in the number of branches.
In fact, it has almost doubled in size.  Even the smaller 1.9 formatted
repository still weighs in at over 700M with no trees.  As a comparison
the old repositories that were missing tags (but apparently not
revisions) weighed in around 400M (without trees) and the git repository
that the bzr repository is based on weighs in at a svelte 306M with a
tree.

You can get the new repositories at:

http://bzr.notengoamigos.org/emacs-merges.tar.gz (or lzma)
http://bzr.notengoamigos.org/emacs-merges-ce.tar.gz (or lzma)

The emacs-merges-ce repository is uses the 1.9 format, and the
emacs-merges repository uses the 0.92 pack format.  These repositories
are somewhat improved over their predecessors.  For example, I upgraded
the branch format in the the 1.9 format repository to be 1.9 as well.
In the old ce repositories I used a 1.9 format for the repository but
the branches were actually still in 0.92 format as that is the default.
The parent of each branch in these new repositories is also set
correctly, in the old branch it was set to one of my test servers.

Now, here are my observations (and questions) for the bzr community.

First of all, how would I go about turning some of these branches back
into tags?

Secondly, I am somewhat confused about why 0.92 pack branches are the
default branch type in a 1.9 format repository.  Everything worked the
same, but you would get warnings when stacking on the branches.  To my
limited experience that seems like a very odd default.  When pulling
branches into a 1.9 formatted repository it seems like you should get
1.9 format branches.

Thirdly, the fast-import-info bit of fast-import is broken with regards
to the Emacs repository.  Emacs has so many "stickies" that it creates a
ConfigObj that ConfigObj can't read in a reasonable amount of time (I
let it run for 24 hours).  I fixed this by using a special cache_manager
that put stickies in an anydbm hash (it creates a 27G file).  I am
somewhat embarrassed by the sheer hackiness of this, but it actually
works surprisingly well, and runs faster than scanning the huge
fast-import file twice.  Perhaps if I had access to a machine with a
ridiculous amount of memory this wouldn't be a problem, but with 4G of
memory and twice that in swap I had problems.

Finally, should I consider creating a 1.9-rich-root format repository?
The fast-import plugin will now create rich-root repositories, but the
serializer will not successfully convert between a rich-root repository
and a non-rich-root repository.  It tries, but it fails with an XML
error (which, I understand, is a known problem).  Is rich root likely to
become a default anytime soon?  If it is going to become the default
does it make sense to bite the bullet and save a re-serialization later?

Thanks for your attention.

Jason

Footnotes: 
[1]  Plus Karl suggested I post here.  Blame him. :)