Feedback on merging via bzr

Mon Jan 18 03:45:53 GMT 2010

I spent some time over the holidays giving merging via bzr and the UDD tools.  
I understand that development of the tools to support this is still a work in 
progress and the much of this feedback probably represents work that you 
already know needs doing.  This mail is based on notes I took recently while 
doing a merge of regina-normal.  As merges go, it was not a complex one.

I'm writing this from the perspective of someone who is reasonably familiar 
with merging using merge-o-matic (~3 years of experience), but relatively knew 
to bzr.  I would have been dead almost immediately if not for the w.u.c 
documentation [1].

Step 1: Getting the source:

Using MoM, I would have grab-merge.sh regina-normal and then had local copies 
of the common ancestor package, the current Debian package, the current Ubuntu 
package, and a proposed merge with any conflicting files listed.

The UDD equivalent seems to be:

$bzr branch lp:ubuntu/regina-normal regina-normal
$password for ssh key
$cd regina-normal
$bzr merge-package lp:debian/squeeze/regina-normal

Text conflict in debian/changelog
1 conflicts encountered.
The merge resulted in 1 conflicts. Please resolve these and commit the changes 
with "bzr commit".

This gives me the proposed merge.  The conflict in this case seems to occur 
with every merge I try.  The most recent Debian and Ubuntu debian/changelog 
entries conflict with each other and the file has to be edited to move the most 
recent Debian entry above the most recent Ubuntu entry.  Additionally, MoM 
would create a stub changelog entry for the new merged package which was quite 
convenient.  None of this additional work is difficult, but it is tedious and 
seems ripe for automation.

Additionally, I still don't have the previous packages available (Debian, 
Ubuntu, and common ancestor).

To get ready to start to work on the actual merge, I need to also do:

$ vim debian/changelog (re-arrange as described)
$ bzr resolve
All conflicts resolved.
$ bzr ci -m "Fix debian/changelog conflict."
$ dch -i (manually add the new changelog entry)

Step 2 working on the package:

I needed to update the build-depends for this package (why I was doing the 
merge right now) and so that was pretty easy:

$ vim debian/control
$ bzr ci -m "* Add new debian/changelog entry
> * Build-depend on libboost-python1.40-dev instead of libboost-python1.38-
dev"
Committing to: ~/devel/boost/merge/regina-normal/
modified debian/changelog
modified debian/control
Committed revision 19.

Step 3 checking the package:

At this point I want to check the package against the previous Debian and 
Ubuntu packages to make sure I have it correct.  Traditionally, I would 
locally debdiff the proposed merge with both the previous Debian and Ubuntu 
packages to make sure I had documented all of the Ubuntu diff and not lost any 
needed changes in the merge.  To do it the new way, I did:

$bzr diff --old lp:debian/squeeze/regina-normal | less
ssh key
(repeat, including redownloading each time the diff is done)

I assume "bzr diff --old lp:ubuntu/regina-normal" would have worked for the diff 
from the previous Ubuntu package, but it isn't documented and I didn't think 
to try it at the time.

Then commit the result:

$ bzr ci -m "* Update debian/changelog to include undocumented differences from 
Debian
> * Remove extraneous whitespace changes"
Committing to: ~/devel/boost/merge/regina-normal/
modified debian/changelog
modified python/testsuite/trigeneral.test
Committed revision 20.

This method of diffing works fine, except that the previous branch has to be re-
downloaded each time the diff is done.  In this case I was trying to remove 
extraneous white space changes that had crept into the packages so it took 
several tries to get them all.  For larger packages or people with a slow 
internet connection, the need to re-download the diff will have a substantial 
negative impact.  Additionally, I miss a way to diff just the debian 
directories.  For new upstream releases (which this wasn't, so I didn't hit 
it) I find this a critical way to review the packaging differences between the 
old and new packages.

Step 4: Building the package:

In the old method, I would debuild -S (-sa if needed) -v and have a package 
ready for uploading. 

bzr builddeb -S -- -v4.6-1ubuntu1

Now this one looks easy, but has the hidden trap of bzr builddeb providing 
only -S from dpkg-buildpackage and requiring an extra flag to pass other 
options to dpkg-buildpackage ("--").  I think this needs a rethink [2].  

Step 5: Testing the package:

This is orthogonal to UDD, so I won't cover it.

Step 6: Uploading the package:

$ dput ubuntu regina-normal_4.6-1.1ubuntu1_source.changes

At this point, I'd be done under the old method (and I could have stopped here 
now and let the branch be created from the uploaded package, but I decided to 
persevere).

$ bzr mark-uploaded

$ bzr push lp:ubuntu/regina-normal
ssh key ...

Now we are done and have both an updated package in the archive and an updated 
branch on Launchpad [3].

Currently, merging this way has a huge advantage over the traditional MoM 
method in that the data for it is being updated and MoM is not.  I suppose 
that more granularity in the branch is an advantage, but I don't think it has 
any real value for cases like there where only one person has worked on the 
package (this will virtually always be true for merges).  

I feel like I've gone through a process that is far more complex than the old 
one for no real benefit.  I have some recommendations for improving and 
simplifying this process.  I think simplification is an essential element 
because the learning curve for new contributors is very steep already and 
raising the barrier to entry is not something that will benefit Ubuntu.

1.  The most important change would be to have some kind of a wrapper for 
getting the source.  It would be nice to have a script that would download 
branches for the common ancestor, current Ubuntu, and current Debian branches, 
and do the proposed merge.  Without local access to the different versions of 
the package, it is very difficult to know if you've got a correct merge.

Additionally, if there were checkouts for the previous Debian and Ubuntu 
packages locally, then it should be easy enough to diff the debian directories 
to check for packaging changes when a new upstream version is involved.

2.  As you no doubt know, the changelog merging could be better and would 
reduce repetitive, boring work potential contributors have to do.

3.  As mentioned, the bzr builddeb interface and documentation needs work.  
Ideally it would act similarly to debuild and pass all the dpkg-buildpackage 
options to it without special flags.  Yet another interface that is almost the 
same is problematic.

I know a huge amount of work has gotten to getting things as far as they are.  
This feedback is not meant to miminize this accomplishment.  I do not, 
however, feel like this facility is really ready for general use yet.

Scott K

[1] https://wiki.ubuntu.com/DistributedDevelopment/Documentation
[2] https://bugs.launchpad.net/ubuntu/+source/bzr-builddeb/+bug/499684
[3] https://code.launchpad.net/~ubuntu-branches/ubuntu/lucid/regina-
normal/lucid