First impressions (long)

[Barry Warsaw]

Hi all,

I think I've actually patched my first Ubuntu package, with much thanks to
Colin and James W for their help.  Having done this from essentially no
knowledge of how to do it, and no experience with the tools, James suggested I
email this list with some feedback and suggestions.  Remember, I'm a n00b so
be gentle because I have No Idea What I'm Talking About. :)

As I understand it, there are essentially two approaches to submitting bug
fixes for Ubuntu packages, the "traditional way" (of which there are some
variants), and the "Ubuntu distributed development (udd)" way, which this list
is interested in.  I was keen to try both approaches, and kept notes of my
steps, both what succeeded and what failed.  I then wanted to be able to
compare how easy I thought it was to accomplish the task using each approach.
There are of course steps in common, like setting up ~/.devscripts and such.

Colin found a good first bug for me:

https://bugs.edge.launchpad.net/ubuntu/+source/system-config-lvm/+bug/390423

This was great because it's pure Python (something I'm familiar with, even if
I didn't know much about the tool itself), there was already a patch attached
to the bug, and there was good information and links back to the bug report in
the Fedora tracker.  So mostly I had to learn a bit of process and experiment
with the tools.

I won't talk too much about my experiences doing things the traditional way,
except to say that I used the pbuilder approach on a kvm VM (host: karmic,
guest: lucid).  I went thorough a number of fits and starts, but eventually
ended up with a debdiff that I could have submitted for sponsorship.  I
ultimately didn't submit this though, because I then wanted to see what it was
like to develop the patch using Bazaar and bzr-builddeb.

I'll go in to some detail in moment, but I have to say that for someone who is
very comfortable with Bazaar for general software development, using
bzr-builddeb was fantastic.  Of course I have some suggestions for
improvements. :)  I /really/ like being able to do almost everything from
within bzr because so much of the (seemingly) grunt work of the traditional
way was just hidden from me.  All I really had to worry about was making sure
the patch was sane, testing, an submitting a merge proposal -- all things I
already knew how to do.  I had to learn a little bit of extra process to build
the package the right way, but those were not overwhelming at all (perhaps in
part because I had just gone through the exercise using the traditional way,
where I did learn a lot).  For packages that fit into a similar class as the
one I worked on, bzr-builddeb was an easy win.

I did go down one false path, and Colin had to teach me about patch systems
and dpatch in particular.  I think bzr-builddeb could actually help me not go
down that path, or point me to the right path, so I'll make some concrete
suggestions in a moment.

So what did I do?  Here are the steps I took (minus some machinations to get
debsign to DTRT):

* bzr init-repo system-config-lvm
* cd system-config-lvm
* bzr branch lp:ubuntu/system-config-lvm
* bzr branch system-config-lvm 390423-ext4
* cd 390423-ext4
* (download patch from upstream tracker)
* patch -p1 < patch.txt
* dch -i
* (add changelog entry)
* debcommit
* bzr commit --fixes lp:390423 --unchanged
* bzr builddeb
* (win!  or was it?)
* (dpkg -i blah; test; confirm fix)
* bzr push lp:~barry/ubuntu/lucid/390423-ext4
* bzr lp-open
* (submit merge proposal)

You probably saw immediately where I went wrong!  I didn't know anything about
patch systems at this point, so I just committed my changes to the source tree
in place.  Colin (politely and helpfully!) schooled me that I should have
created a patch file instead, left the original source unchanged, and added my
patch file to debian/patches/00list.  My diff should have included just the
new .dpatch file, along with the changes to the changelog and 00list.

Instead of backing out my changes, I extracted them from the relevant
revisions, doing something like this (at the time):

* bzr diff -r6.. | dpatch patch-template -p "ext4xfssupport" "blah" > debian/patches/ext4xfssupport.dpatch"

I had to hand edit edit ext4xfssuport.dpatch because it contained a diff to
debian/changelog, and then I reverse-merged the relevant revisions to back out
the changes to the package's source code.  Then I ran 'bzr builddeb' again,
but it failed!

The problem I think was that dpatch expects to run 'patch -p1' or its moral
equivalent on the files in debian/patches, but 'bzr diff' does not include the
parent directory by default.  Now I understand why 'bzr diff' has a --prefix
option. :)  So I ran this:

* bzr diff -r6.. --prefix system-config-lvm.old/:system-config-lvm/ | dpatch patch-template -p "ext4xfssupport" "blah" > debian/patches/ext4xfssupport.dpatch"

This time 'bzr builddeb' ran successfully.  I installed the resulting package,
tested again, committed my last change and pushed.  I now await Colin's
re-review.

I might almost say the experience was a delight! :)

Now, I might not be using the tools at all correctly, and may have just
stumbled an a right-ish answer, but here are a few things I think could be
added to bzr-builddeb to make fixing bugs in Ubuntu almost crazy easy.

* Use looms (or pipes I suppose, but I prefer looms over pipes).

Here's the way (I think) I'd organize my branch threads:

- review-comments
- hack-the-source-code
- dpatches
- changelog
- system-config-lvm

Thus the source package trunk is at the bottom thread.  At the top are the
threads that actually change the source code in place.  This can sometimes be
handy if you can/want to run the code right from there, as I did a few times
since it's just Python and easily run in-place.  It's certainly way easier
manage code this way for someone who's used to normal bzr development.

In the middle are the threads that pertain to package building.  The reason
I'm putting them in the middle is because eventually I'm going to slice off
the top two threads since they are not appropriate for sponsorship.  Actually,
what I'm going to do is export the loom and push only the dpatches thread, so
having the irrelevant local stuff at the top makes that easy.

I'm separating the changelog and dpatches threads though perhaps in practice
they can be combined.  The important thing, and where I think bzr-builddeb
could really shine, would be to generate the dpatches, and maybe even the
changelog threads for me.

Imagine that I'm sitting in the hack-the-source-code thread and I'm now ready
to build my package for local testing.  I've already committed several
revisions to the source code in-place, so why couldn't bzr automatically run
'dch -i' and generate the changelog thread?  It could take its input out of
the commit messages (with command line override).

Why too couldn't bzr generate the dpatches for me?  It /knows/ that it needs
to diff hack-the-source-code against system-config-lvm so it could
automatically remove any changes in the changelog or dpatches threads to
generate the .dpatch file.

I'm sure there are all kinds of gotchas, tricky bits, and corner cases to
consider, and I don't know how representative system-config-lvm is as a
package.  I've also not actually /tried/ the looms approach, although I think
I will for the next bug I tackle.  But anyway, it feels like bzr-builddeb is
very close, and could really help an Ubuntu developer with that extra mile.

What do you think?

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
Url : https://lists.ubuntu.com/archives/ubuntu-distributed-devel/attachments/20091211/352abc6d/attachment.pgp