Distributed Development toolset [was: Re: ArchiveReorganisation and sponsoring]

[Matt Zimmerman]

On Mon, Sep 01, 2008 at 09:36:51AM -0400, Scott Kitterman wrote:
> On Mon, 1 Sep 2008 10:34:45 +0100 Matt Zimmerman <mdz at ubuntu.com> wrote:
> >Some cases where it could be significantly easier include:
> >
> > * The package has been updated since the patch was created, and it needs a
> >   three-way merge
> >
> > * There is more than one patch outstanding to include
> >
> > * The patch requires changes before it can be included
> 
> In Universe, except for the last one these are all atypical.

In Hardy, there were 6533 uploads to universe and 4976 to main.  Even if it
were only useful in main (which I don't think is the case at all), it would
still be worth pursuing.

An additional goal of this project is to streamline the process of
contributing to Ubuntu.  If we're successful, there will be more patches
coming in, and we'll be grateful for good tools to manage a higher volume of
them.

> >This is not even a minority, much less a small one.  Most packages are
> >touched by at least the Debian maintainer, upstream, and one or more Ubuntu
> >developers during a typical development cycle.  Many also receive
> >third-party contributions, and I expect that more would if the process were
> >more lightweight.
> 
> I don't the syncs from Debian/upstream updates are particularly relevant 
> (except for the forensics use case that Colin mentioned in another message 
> in this thread).

Merge-o-matic is very nice, but managing multiple branches of development is
a problem that modern version control systems were designed to address.  If
you've had to do merges by hand and resolve conflicts, you know how much
more convenient this is with a good VCS.

Have a look at https://wiki.ubuntu.com/UbuntuDevelopment/Merging, how many
steps there are just to figure out what to do and whether it's worked.  It's
a ~2500-word document.  I doubt any other project needs this much
documentation for merging a branch.

Several sections of that document could be replaced with a 'bzr merge'
command, and the result would be less error-prone, take less time, and be
easier to audit.

> >Fortunately for us, Bazaar is very easy to learn, particularly if you've
> >used other version control systems.
> >
> >I can't help but get the impression that you're biased against Bazaar,
> >though.
> 
> I do have a bias, but it isn't anything particular to Bazaar.  My bias is 
> against having to learn new tools.  I don't qualify that as fun.  I have 
> not used Bazaar enough to have formed any specifc opinions about it beyond 
> too slow (there has been a lot of testimoney that its faster now, but most 
> of the benifit seems to come from knowing what options to use).

Our goal is to allow a smooth transition which doesn't require that anyone's
existing tools stop working.  I think Colin and James have explained that in
some detail now.

None of us were born with software development tools, and few of us are
using the same tools we were 10 years ago.  Change is a necessary part of
improving our work.  So while it's reasonable to expect justification for
change, opposing it outright is counter-productive.

> >There are at least two options: 1. everything works exactly the same as it
> >does today, or 2. the git repository is imported.
> 
> Since we're planning on maintaining the Ubuntu branches on alioth, there 
> would have to be some way to indicate that changes should not be added to 
> the Bazaar repository, but to the Git repo on alioth.  Otherwise the repos 
> would get out of sync.

They wouldn't get any more out of sync than they do today when someone
uploads a package to Ubuntu.

> In this paradigm, I don't think continues as it does today is an option.  
> We have to have a way to mark packages as maintained elsewhere (and then 
> import from there so there is a correct history available).

We will support the use of different tools where there is a good reason to
do so (the kernel is a good example of this).

> >> I say at best because many disadvantages are quite clear to me.
> >
> >The only one you've pointed out so far is that it involves Bazaar, which
> >you see as a disadvantage.  What are the others?
> 
> 1.  New toolset to learn which takes time, causes a productivity hit during 
> the learning process, and may increase error rates.

This is a one-time cost of the transition, not a disadvantage of the
proposed system.  Change may be inconvenient, but we need to make decisions
based on the longer term cost/benefit balance.

> 2.  Slower.  This is apparently getting better, but I think is still a 
> problem.

Working in revision control is necessarily slower than without it, but only
testing will show whether the difference is "small enough".  Let's give it a
chance before deciding that it is too slow.

> 3.  Potentially complicates inter-distro collaboration (I think this is 
> solvable - see the pkg-clamav discussion above)

I agree that your concerns here can be addressed.

Also, one of the main problems in inter-distro (and upstream) collaboration
at present are that it's often hard to see what changes have been made and
where they came from, and standardized revision control will be a huge step
forward in this regard.

> As Colin points out, use of this new facility is entirely optional, so I 
> don't need to get too stressed out about it in the short term.  As I said 
> in another message, I don't expect to block this change.  I just want to 
> make sure it's thought through.

We're all invested in making it the best it can be, but you're coming off as
being opposed to doing it at all, rather than making sure that it's done
right and concerns are addressed.

-- 
 - mdz