Distributed development toolset (Re: ArchiveReorganisation and sponsoring)

[Colin Watson]

On Mon, Sep 01, 2008 at 01:49:04AM -0400, Michael Casadevall wrote:
> Having used VCS for packages, I find its generally much more difficult
> to get a patch in. Launchpad's branch merger makes it easier, but in
> general the process, for someone who just wants to generate a simple
> branch, and posting a debdiff is impossible (i.e., what we are
> proposing), the workflow becomes MUCH more complex.

Contrary to what you say, we are *not* proposing that posting a debdiff
will be impossible - I think that's been stated several times now in
this thread! People who are unfamiliar with version control could
continue to do exactly what they're doing now and send a patch. People
who are familiar with version control will be able to use it, and their
work will become easier.

Remember also that you're looking at the workflow involved when you
cannot rely on Bazaar branches existing for most source packages, and
when tool development to support this model is not complete. That
inevitably introduces complications into the process, and there are
definitely complications right now that need to be ironed out. Don't
throw the baby of revision control out with the bathwater of current
tool problems, though.

> 2. Branch the path, which requires finding out where in the bazaar
> repo the version you wish to change is.

The intent is to provide every package with a standard repository URL
built on a common template, so you would be able to do something like
'bzr get lp:ubuntu/packagename' (I don't think the syntax has quite been
nailed down, so there may be minor variations after ubuntu/ there). Once
we have an import of every package in Ubuntu (well in progress) and have
a few Launchpad changes made to give us this branch namespace (also in
progress), this will work for every package in Ubuntu.

> 3. Make modifications and make sure it works in bzr-buildpackage which
> also requires you to manually download the tarball last time I tried
> it.

None of this is necessary if you want to carry on using 'apt-get
source'. Furthermore, we'll likely provide tools to do things like
downloading the tarball for you. For instance, it probably wouldn't be
very hard to plumb .orig.tar.gz fetching into debcheckout or a similar
tool.

> 5. Pray that it gets merged before someone commits a change that
> conflicts (in my general experience, Launchpad merger and bazaar's
> merge algorithm leaves something to be desired)

It's considerably better than having two uploads collide and having to
resolve that (been there, cursed a lot; note that if that happens you
find out several minutes later when you've already moved on to something
else, rather than immediately).

I use this system a lot, and conflicts of the type you describe due to
somebody else committing a patch at the same time typically take me
seconds to resolve. I have also used many other revision control
systems, and have seen none with significantly better facilities for
conflict resolution. I have had very few practical problems with
Bazaar's merge algorithms that I considered to be fixable by them with
remotely reasonable design effort.

Can you provide a specific example of a scenario in Ubuntu package
development that you thought Bazaar handled unreasonably? I'm genuinely
interested. Certainly Bazaar is software and therefore contains bugs,
but I have trouble reconciling your statement with the fact that I've
found it to be very much better than the alternatives for Ubuntu package
development (namely, throwing diffs around by hand and resolving the
same conflicts without the assistance of any revision control system).

> Now I realize that normal uploads will still be allowed,

I think this needs to be stressed again, since in this mail you're
focusing on a pattern that you would not have to use if you didn't want
to. Remember that some of your fellow developers *have* successfully
built good workflows around this type of system, and you seem to be
saying that they're all onto a loser. I find that puzzling; it's more
usual to try to learn from your fellow developers.

> but I feel that getting rid of source packages is a problem looking
> for a solution.

(Did you mean "a solution looking for a problem"?)

Note that getting rid of source packages is the very final stage, and is
as yet a tenuous, sketchy plan at best (not to mention IMO a bad idea; I
want to make it optionally possible to create uploads by committing and
tagging, but I think we should continue publishing source packages for
the foreseeable future). The much more important early stages simply
involve getting everything imported into Bazaar.

> Since there are no maintainers in Ubuntu, it doesn't make sense that
> we have to maintain VCS branches for what is likely a one off upload.

It's precisely when you don't have maintainers that VCS branches are
most useful, in my opinion and experience. If you want to make a tidy-up
kind of change, an upload is often gross overkill in terms of effort;
you just want to get the change off your disk and out of your way, and
whoever has good cause to upload the package at some point in the future
can do it.

Furthermore, if you don't want to maintain VCS branches, our software
will do it for you. You can just ignore them if you like; you won't
"have to maintain" them.

> Teams that want or need it have already changed, the rest of us seem
> to be doing fine without, and anyone could create a bzr branch to
> manage their own pages.

The current effort is simply creating a "default" bzr branch for
everyone, with a sensible and useful seeding of history from previous
package uploads. Over a mere release cycle or two, this default seeding
of history will become quite substantial for many packages. Going to the
effort of creating a Bazaar branch with even that much detail is enough
work to put many people off (I've done it myself for a large number of
packages; that doesn't mean I relish the thought of doing it for many
more). If I contribute to a package by means of commits to a Bazaar
branch that had helpfully been created for me (and an upload at the
end), and somebody else contributes to it by means of traditional
uploads, that's still more fine-grained history than we would have had
otherwise, and I see it as a win. The person contributing by means of
traditional uploads doesn't even have to know Bazaar exists if it scares
them too much.


I am amazed at the number of people who seem to be intent on shooting
down an optional facility they won't have to use, that other developers
have explicitly said would make their lives easier, and that is already
well in progress after quite a bit of discussion! Why not let other
developers have what they consider to be an easier life, even if you
would prefer not to use the same tools and don't see the advantages?

I think we should definitely examine the weaknesses of our current tools
for a broad spectrum of developers. However, it really doesn't help to
describe this effort as "a solution looking for a problem". That shows
little respect for the many developers who have already contributed to
this effort in one way or another, most of whom have long experience of
Ubuntu development and feel that it would be an asset. If we can keep
the debate toned down, we may be able to get somewhere more useful.

Cheers,

-- 
Colin Watson                                       [cjwatson at ubuntu.com]