Feedback on merging via bzr

[Dmitrijs Ledkovs]

2010/1/18 Scott Kitterman <ubuntu at kitterman.com>:
> I spent some time over the holidays giving merging via bzr and the UDD tools.
> I understand that development of the tools to support this is still a work in
> progress and the much of this feedback probably represents work that you
> already know needs doing.  This mail is based on notes I took recently while
> doing a merge of regina-normal.  As merges go, it was not a complex one.
>
> I'm writing this from the perspective of someone who is reasonably familiar
> with merging using merge-o-matic (~3 years of experience), but relatively knew
> to bzr.  I would have been dead almost immediately if not for the w.u.c
> documentation [1].
>
> Step 1: Getting the source:
>
> Using MoM, I would have grab-merge.sh regina-normal and then had local copies
> of the common ancestor package, the current Debian package, the current Ubuntu
> package, and a proposed merge with any conflicting files listed.
>
> The UDD equivalent seems to be:
>
> $bzr branch lp:ubuntu/regina-normal regina-normal
> $password for ssh key
> $cd regina-normal
> $bzr merge-package lp:debian/squeeze/regina-normal
>

I do

$bzr init-repo package
$cd package
$bzr branch lp:ubuntu/package lucid
$bzr branch lp:debian/package squeeze

> Text conflict in debian/changelog
> 1 conflicts encountered.
> The merge resulted in 1 conflicts. Please resolve these and commit the changes
> with "bzr commit".
>

Hopefully per-file hook will solve this


> Step 2 working on the package:
>
>
> Step 3 checking the package:
>
> At this point I want to check the package against the previous Debian and
> Ubuntu packages to make sure I have it correct.  Traditionally, I would
> locally debdiff the proposed merge with both the previous Debian and Ubuntu
> packages to make sure I had documented all of the Ubuntu diff and not lost any
> needed changes in the merge.  To do it the new way, I did:
>
> $bzr diff --old lp:debian/squeeze/regina-normal | less
> ssh key
> (repeat, including redownloading each time the diff is done)
>

At this point in the package/lucid branch you have all package
revisions and tags so you can do this :

$bzr diff -r tag:0.1-3ubuntu5      #debdiff against old ubuntu
$bzr diff -r tag:0.1-6                 #debdiff against debian

Or any package release for that matter..... see $bzr tags

Also you might want to add --prefix=old-version/:new-version/ such
that bzr diff looks like debdiff.

> I assume "bzr diff --old lp:ubuntu/regina-normal" would have worked for the diff
> from the previous Ubuntu package, but it isn't documented and I didn't think
> to try it at the time.
>

Again use your local revisions. After you grab debian & ubuntu branch
disconnect your network =) It will make you think UDD way.

> Then commit the result:
>
> $ bzr ci -m "* Update debian/changelog to include undocumented differences from
> Debian
>> * Remove extraneous whitespace changes"
> Committing to: ~/devel/boost/merge/regina-normal/
> modified debian/changelog
> modified python/testsuite/trigeneral.test
> Committed revision 20.
>

Better use debcommit it takes new entries from debian/changlog and
appends --fixes lp:XX such that when you push this branch it will get
linked to bugs in lp.


> This method of diffing works fine, except that the previous branch has to be re-
> downloaded each time the diff is done.  In this case I was trying to remove
> extraneous white space changes that had crept into the packages so it took
> several tries to get them all.  For larger packages or people with a slow
> internet connection, the need to re-download the diff will have a substantial
> negative impact.  Additionally, I miss a way to diff just the debian
> directories.  For new upstream releases (which this wasn't, so I didn't hit
> it) I find this a critical way to review the packaging differences between the
> old and new packages.
>

No see above

> Step 4: Building the package:
>
> In the old method, I would debuild -S (-sa if needed) -v and have a package
> ready for uploading.
>
> bzr builddeb -S -- -v4.6-1ubuntu1
>

? huh I just use
$bzr bd                #regular build
$bzr bd --quick     #customised to build with pbuilder (should be named "slow")
$bzr bd --source   #create source package

> Now this one looks easy, but has the hidden trap of bzr builddeb providing
> only -S from dpkg-buildpackage and requiring an extra flag to pass other
> options to dpkg-buildpackage ("--").  I think this needs a rethink [2].
>

See builddeb manual for builddeb.conf

> Step 5: Testing the package:
>
> This is orthogonal to UDD, so I won't cover it.
>
> Step 6: Uploading the package:
>

I don't have upload rights so I can't comment.

> $ dput ubuntu regina-normal_4.6-1.1ubuntu1_source.changes
>
> At this point, I'd be done under the old method (and I could have stopped here
> now and let the branch be created from the uploaded package, but I decided to
> persevere).
>
> $ bzr mark-uploaded
>
> $ bzr push lp:ubuntu/regina-normal
> ssh key ...
>
> Now we are done and have both an updated package in the archive and an updated
> branch on Launchpad [3].
>
> Currently, merging this way has a huge advantage over the traditional MoM
> method in that the data for it is being updated and MoM is not.  I suppose
> that more granularity in the branch is an advantage, but I don't think it has
> any real value for cases like there where only one person has worked on the
> package (this will virtually always be true for merges).
>
> I feel like I've gone through a process that is far more complex than the old
> one for no real benefit.  I have some recommendations for improving and
> simplifying this process.  I think simplification is an essential element
> because the learning curve for new contributors is very steep already and
> raising the barrier to entry is not something that will benefit Ubuntu.
>
> 1.  The most important change would be to have some kind of a wrapper for
> getting the source.  It would be nice to have a script that would download
> branches for the common ancestor, current Ubuntu, and current Debian branches,
> and do the proposed merge.  Without local access to the different versions of
> the package, it is very difficult to know if you've got a correct merge.
>

All of them have common ancestor and this way you choose what you want
to merge: experimental, sid, squeeze, lenny..... And also you can
choose to merge ubuntu proposed, security etc. So I believe this
method is superior cause now you can use the same workflow for any
package upload to any ubuntu pocket.

Note in addition you get the "hidden" upstream branch.

$bzr branch lucid -r tag:upstream-0.1

And you get pure upstream branch with release per commit. Quite cool
if there were accidental direct modifications in tarball while patch
system present and you want to diff them back.


> Additionally, if there were checkouts for the previous Debian and Ubuntu
> packages locally, then it should be easy enough to diff the debian directories
> to check for packaging changes when a new upstream version is involved.
>

$bzr diff debian/ -r tag:debian-release

Or again any release you want.

> 2.  As you no doubt know, the changelog merging could be better and would
> reduce repetitive, boring work potential contributors have to do.
>
> 3.  As mentioned, the bzr builddeb interface and documentation needs work.
> Ideally it would act similarly to debuild and pass all the dpkg-buildpackage
> options to it without special flags.  Yet another interface that is almost the
> same is problematic.
>

It does more =) I like it as it is. But that's matter of opinion.


> I know a huge amount of work has gotten to getting things as far as they are.
> This feedback is not meant to miminize this accomplishment.  I do not,
> however, feel like this facility is really ready for general use yet.
>

Maybe I should write a guide.....

> Scott K
>
> [1] https://wiki.ubuntu.com/DistributedDevelopment/Documentation
> [2] https://bugs.launchpad.net/ubuntu/+source/bzr-builddeb/+bug/499684
> [3] https://code.launchpad.net/~ubuntu-branches/ubuntu/lucid/regina-
> normal/lucid
>



-- 
With best regards


Dmitrijs Ledkovs (for short Dima),
Ледков Дмитрий Юрьевич

()  ascii ribbon campaign - against html e-mail
/\  www.asciiribbon.org   - against proprietary attachments