2.0 upgrade experiences

Tue Aug 11 22:20:26 BST 2009

I'm looking to get my branches and repositories upgraded to the 2a 
format before the school year starts, so I'm trying to follow 
doc/en/upgrade-guide.txt for my setup.  I just wanted to share a couple 
of experiences so that this process can continue to get more 
streamlined.  I will just post my comments on what I run into along with 
numerous questions.

All of the following is using bzr-1.18rc1 installed from the tarball 
with Python 2.5.2 on Ubuntu Hardy.

1.  Running ``bzr check`` in a standalone branch that was originally in 
format 1.9 gives me a load of output like:

[...]
Missing inventory {('TREE_ROOT', 
'svn-v2:1179 at 8f9f66e6-09d4-0310-8052-96bf1e615463-')}
Missing inventory {('TREE_ROOT', 
'nmb at wartburg.edu-20090417201757-chtr18w6ddnpcal3')}
Missing inventory {('TREE_ROOT', 
'nmb at quaggy.local-20071106145233-vr33wd75mx91n97l')}
Missing inventory {('TREE_ROOT', 
'svn-v2:724 at 8f9f66e6-09d4-0310-8052-96bf1e615463-')}
Missing inventory {('TREE_ROOT', 
'nmb at wartburg.edu-20071109170840-o838ndsrs1prwl1m')}
Missing inventory {('TREE_ROOT', 
'nmb at wartburg.edu-20071115142153-pkeip1evaq353ubc')}
checked branch file:///labhome/nmb/repos/career/ format Branch format 7

It is not clear from ``bzr help check`` whether or not a missing 
inventory for the tree root "indicates a problem".  Going ahead with the 
upgrade (since I have a full backup, I feel free to try it) gives me a 
branch in which ``bzr check`` runs cleanly.  Are these missing 
inventories an error?  When are they an error?  The upgrade 
documentation or the check help should indicate this more clearly.

2. The space savings is not as big as my wildest dreams (This is a 
standalone branch, even though the parent directory is called "repos".)

nmb at rameses[~/repos/career]$ du -sk .
6848	.
nmb at rameses[~/repos/career]$ du -sk ../../repos.bak/career
7028	../../repos.bak/career

When I thought that a pack operation might clear up additional space, I 
got the following error:

nmb at rameses[~/repos/career]$ bzr pack
bzr: ERROR: Pack '6ffbb186d5539f27e54609f64e5fb92e' already exists in 
<bzrlib.repofmt.groupcompress_repo.GCRepositoryPackCollection object at 
0x126c3d0>

Is that error serious?  Does it mean that I cannot pack this branch 
further?  Are there obsolete packs still sitting around somewhere?  Can 
these be gotten rid of somehow?  This suggests that the space savings 
could be quite a bit larger:

nmb at rameses[~/repos/career]$ du -sk .bzr/repository/obsolete_packs/
2260	.bzr/repository/obsolete_packs/

What processes can be used to empty out these obsolete packs?  When is 
that process safe?

3.  In a different branch, ``bzr check`` tells me that I have "sha1 
mismatch"

nmb at rameses[~/repos/software]$ bzr check
Checking working tree at '/labhome/nmb/repos/software'.
Checking branch at 'file:///labhome/nmb/repos/software/'.
Checking repository at 'file:///labhome/nmb/repos/software/'.
checked repository <bzrlib.transport.local.LocalTransport 
url=file:///labhome/nmb/repos/software/> format <RepositoryFormatKnitPack6>
    189 revisions
    677 file-ids
Missing inventory {('TREE_ROOT', 
'svn-v2:245 at 8f9f66e6-09d4-0310-8052-96bf1e615463-')}
[...many, many more of the same...]
Missing inventory {('TREE_ROOT', 
'svn-v2:243 at 8f9f66e6-09d4-0310-8052-96bf1e615463-')}
sha1 mismatch: 
('svn-v2:1099 at 8f9f66e6-09d4-0310-8052-96bf1e615463--software%2fIPChange%2fbuild%2fIPChange.app%2fContents%2fResources%2flib%2fpython2.5%2fsite.py', 
'svn-v2:1099 at 8f9f66e6-09d4-0310-8052-96bf1e615463-') has sha1 
793c6808e6057fe40c2d99e35aec4e3d347b98a5 expected 
da39a3ee5e6b4b0d3255bfef95601890afd80709 referenced by 
svn-v2:1099 at 8f9f66e6-09d4-0310-8052-96bf1e615463-
[...more of the same...]
sha1 mismatch: 
('svn-v2:1099 at 8f9f66e6-09d4-0310-8052-96bf1e615463--software%2fIPChange%2fbuild%2fIPChange.app%2fContents%2fResources%2flib%2fpython2.5%2fconfig', 
'svn-v2:1099 at 8f9f66e6-09d4-0310-8052-96bf1e615463-') has sha1 
013734b9cc010c4a77db9af6348141908d0dfc7b expected 
da39a3ee5e6b4b0d3255bfef95601890afd80709 referenced by 
svn-v2:1099 at 8f9f66e6-09d4-0310-8052-96bf1e615463-
checked branch file:///labhome/nmb/repos/software/ format Branch format 7

As suggested in the upgrade docs, I ran ``bzr reconcile`` which 
completed successfully, but the output of ``bzr check`` was unchanged. 
Again, proceeding will-he-nill-he with a conversion (``bzr upgrade 
--2a``) provides a branch that stills has the same "sha1 mismatch" 
errors.  How does one go about fixing these SHA1 errors?  What will be 
the future impact of them?  How did they get in there?  (This branch is 
a bzr-svn conversion of part of an old SVN repository.)

4.  The progress bar for upgrade goes 70% of the way to finished 
immediately and then takes all of its time working on "Copying content 
into repository.:Transferring revisions".  Is there any way to make the 
progress on the progress bar corrspond more nearly to the time that 
various parts of the process actually take?

5. Standalone branches are much easier to upgrade than those living in 
shared repositories.  Shared repositories for me can rapidly get quite 
large.  My worst case is the ~/src/bzr directory on my main development 
machine where I have a shared repository that I intended for my various 
bzr.dev working branches that live in that directory.  As things have 
gone on, I've branched the trunks for all of my plugins into that shared 
repository (using symlinks in ~/.bazaar/plugins), as well as started a 
couple of Bazaar-related projects of my own in that directory (thus 
using the same shared repository).  It's over 30k revisions, while no 
one branch that lives there has over 5000 revisions.  As we move forward 
with the 2.0 series, I hope that shared repository usage and policy will 
be back on the table, because a single repository that large is taking a 
*long* time to check before even starting an upgrade.

Also around shared repositories, ``bzr check`` checks the branches 
within that subdirectory when run at the repository level (prior to 
upgrading the repository).  Does this mean that I don't need to run 
check again when I am converting each branch in the following step?

Also for shared repositories, I find myself doing an unreasonable amount 
of "for branch in `ls`; do bzr <something> $branch; done".  Can we 
create a sensible recursive policy for this upgrade process?  Check is 
already recursive.  Should upgrade be?

Overall, I'm pleased with the upgrade process and the smooth rhythm of

$ bzr check # WT... oh well
$ bzr upgrade --2a

but I think that more effort needs to go into documentation and error 
reporting so that the process is self-explanatory for the large part. 
Just another data point...

-Neil