Orchestra integration

[William Reade]

Hey all

As you will have noticed, Orchestra integration is proceeding apace,
thanks to Andres' excellent work over the last couple of weeks. I've
been doing my best to get his changes moved over to trunk in a series of
small branches, and this has yielded some useful stuff, but we're
starting to find that the development model we've been using has a
couple of drawbacks. The forces in play are:

1) Each change to trunk has to be reasonably small, lest it drive the
reviewers insane.

2) Andres has done a lot of work on a branch which does some great
things, but which diverged from trunk about 2 weeks ago.

3) My changes to trunk have been drawn from his code, but are notably
different in a number of ways (mainly just to ensure test coverage, and
consistency with the rest of trunk)... and haven't been applied back to
his branch.

This puts us in a tricky situation right now: as he fixes bugs in his
own branch, he inevitably touches functionality that I've already moved
to trunk, but there's no way for us to track the changes and ensure they
end up where they should.

In future, we hope to avoid this situation by following this protocol:

1) Andres wants to add a feature, or fix a bug, so he branches from
trunk; when it's complete, he merges the latest changes from trunk,
pushes the branch, and informs me.

2) As soon as I get a completed branch from Andres, I make whatever
modifications I need to ensure it's fully tested, using twisted, and
properly integrated into the project (avoiding duplication, following
conventions, etc).

3) As soon as that's ready, I propose a merge and hassle everyone
mercilessly until it gets into trunk.

4) For the next feature, Andres creates a new branch (probably from his
previous branch, but (where possible) from my own version of it or
(better still) from trunk; even if my work lags behind his a certain
amount, he should still be able to merge my own changes and those from
trunk, as they land, with a minimum of conflict and confusion.

To illustrate: 

trunk   ____________\3_____________________________________\12____
           \           \    /            /8      /10  \         \
william     \        ___\4_/      ___.../   ____/      \         \
             \      /      \5    /6  \7    /9           \         \
andres        \1___/2_______\___/     \___/              \11_______\13__


1) Andres branches from trunk for feature A.

2) Andres completes A, we each branch from A: I start A', he starts B.

3) Someone changes trunk.

4) I merge from trunk into A'.

5) I finish A' and merge it into trunk; feature A is complete, and
Andres merges from A' into B.

6) Andres completes B, I branch B'.

7) I finish B' but it gets stuck in review; Andres branches from B' to
start work on C.

8) B' gets merged into trunk; feature B is complete.

9) C is completed; I branch C'.

10) C' is finished, and gets merged into trunk.

11) Andres branches from trunk for feature D.

12) Someone changes trunk.

13) Andres merges directly from trunk into D.

...&c.

Now, I think this model is fine, even if I haven't explained it as well
as it could be (if you don't think it's fine, please shout now, before
we start doing it ;)).

However, we're not currently in a situation in which we can start doing
this, because what we currently have is one branch (Andres') with
features A-Q, and several of my branches with features A', C', P' and
Q', which are (I think) ready to merge into trunk.

To reconcile Andres' current branch with trunk, and get us into a state
where we can work as described above, all I can think of is to:

1) Do a hideous monster-merge from trunk, A', C', P', Q' into Andres'
current branch.

2) Get Andres to branch from the result to start work on R, and keep
branching from there, or from his own new branches where necessary, for
S, T, &c.

3) Do B' and D'-O' as I have been, but make sure I merge them into
Andres' branch as well as into trunk.

4) While this is going on, Andres will have to keep merging from his
original branch into R, S, T, &c.

5) Finally, I catch up to R and can implement R', S', T', &c, as
described previously.

...but, well, I think that's going to be painful. If anyone can think of
a better way, or can improve the suggested process even slightly, please
let us know :-).

Cheers
William