[storm] Moving Storm Forward

[James Henstridge]

On Tue, Jun 9, 2009 at 9:50 AM, Gustavo Niemeyer <gustavo at niemeyer.net> wrote:
> Thanks for putting these ideas together James!
>
>> = Get More People Working on Trunk =
>>
>> If someone were to look at the revisions in Storm's trunk, they'd see
>> that pretty much all the recent commits come from Canonical employees.
>>  However, if they looked at the other branches registered on
>> Launchpad, it is clear that there are a number of long lived feature
>> branches containing significant contribution from the community.  The
>> prominent ones are:
>
> A few notes on these specific branches:
>
>>  * twisted-integration
>
> Even though that's not entirely connected to what is being done by
> Canonical today, this branch is actually from Thomas Hervé, who works
> for Canonical too and is an active Storm developer.  We've been
> discussing details about how to integrate it and there are a few edges
> we agree should be polished before integration.

Okay.  It'd be good to clear the air w.r.t. this branch.  This branch
is already on the review queue, so perhaps  we can try and nail down
exactly what edges need polishing.


>>  * Oracle support
>
> I've exchanged some ideas about this with Gustavo (kov), but we both
> are a bit slow on answering each other's comments, so it's being a bit
> slower than it should.
>
>>  * Microsoft SQL Server support
>
> This came from the community, but I think one of the branches is
> actually from Sidnei da Silva, who was then hired by Canonical as
> coincidence (this Storm work wasn't involved in the hire,
> specifically).   Sidnei, are you listening? :-)  Can you comment on
> the status of this branch?

There has also been development on branches owned by the ~zeomega team
for these two, so it'd be worth considering those too.


>> While the current state of affairs might be workable for their current
>> users, I think it would be beneficial to try and get them merged to
>> trunk ASAP.  Some benefits of merging the branches include:
> (...)
>
> These look like good points.
>
>> Of course, there are some problems to overcome in merging Oracle and
>> MS-SQL support.  I am currently able to run tests for all supported DB
>> backends on my machine when committing to trunk, but that will be
>> difficult with these two due to licensing and ties to Windows for
>> MS-SQL.  Perhaps a buildbot instance might be the answer, if we can
>> get appropriate slaves set up.
>
> Agreed.  I think buildbots are the way to go.  It would be good if at
> least some of the main developers had all the backends working locally
> too.

Yep.  That probably means access to multiple (virtual?) machines.


>> Another issue is test coverage for the new backends.  If they don't
>> currently pass all the tests, might it be appropriate to merge them
>> anyway and work out a process where we can ratchet down the set of
>> failures to zero post merge?
>
> I'm a bit more concerned about this one aspect.  We're currently quite
> strict about the test coverage in Storm, and there's a great benefit
> to people which use it in production systems.  If some of these
> branches have incomplete test suites (either broken or missing some
> tests entirely), once they are integrated, there's less motivation to
> get good testing in place.

So there are two aspects to this:
 1. the new backends may not fully pass their tests now.
 2. a developer without access to the relevant databases may break a
backend unintentionally.

This problem can only get worse as more backends.

For (1), I'd consider merging the backends but setting up the test
suite to expect certain tests to fail.  That way the test suite can be
used to ensure that things haven't regressed, and it provides a goal
for fixing bugs.

For (2), I think we'd need to recognise certain databases as being
second class citizens in the sense that their tests may not pass for
every mainline revision of Storm trunk.  To alleviate this, we might
introduce reduced review requirements for simple patches that fix
regressions in those backends.

While there might be a drop in motivation to fix bugs post merge
(which is questionable: there doesn't seem to be much motivation to
get these branches merged at the moment), I think the benefits of
having them in trunk make it worthwhile.  In particular:
 1. People using these backends can help fix DB independent bugs on trunk.
 2. With more backend code in view, it should become clearer when it
makes sense to refactor some DB dependent code into the DB independent
portion of Storm.



> That said, I guess it's correct to say that being strict about tests
> do reduce the community motivation for contributions, since it's a
> steeper learning curve to do things *and* to get them properly tested.
>
> So, how do we balance, good testing coverage with good community practices?

I would like to see us keep our current level of test coverage going
forward.  I see it as one of the strengths of Storm: it makes it less
of a risk to move forward to new versions of Storm since there is a
base level of behaviour that we know hasn't changed.  Where we have
changed behaviour, it is generally deliberate and documented in the
NEWS file.

If we want to improve the contribution process, probably the best
thing we could do would be to revise
https://storm.canonical.com/DevelopmentProcedure to match current
practice.  That document hasn't been updated since we switched over to
using Launchpad's code review system, and isn't completely clear whose
input is needed to move forward at each point.

I suspect that with an up to date version of that document it would be
easier to pin point where potential contributors might get
discouraged.


>> = Reviews =
>>
>> Changes make their way to trunk through Launchpad's code review
>> system.  A list of branches waiting for review is available here:
>>
>>    https://code.launchpad.net/storm/+activereviews
>>
>> While most review work is being done by the core Storm team at the
>> moment, I'd welcome others to help out.  Even if you don't have a deep
>> knowledge of Storm, a general knowledge of Python programming can
>> often be enough to see things the original developer missed.
>
> I think we can try to schedule "Storm review days" as well, where all
> of the main developers would get together over IRC and clean up the
> queues entirely.  This would probably be effective, as long as we
> manage to do these often enough.

Sounds good.  Providing timely feedback is a must if we want to
encourage contribution.


>> = IRC Meetings =
>>
>> Gustavo mentioned this on the mailing list a few months back, but we
>> haven't actually organised it.  It'd be helpful to have a regular IRC
>> meeting to discuss the development of Storm.  I think it'd be good to
>> aim for a monthly meeting at least initially, but we could do it more
>> frequently if we feel it is necessary.
>>
>> The meeting need not be too long: if there isn't much to discuss, then
>> the meeting can end quickly.  This would be open
>>
>> Picking a time might be a bit tricky, given that we're a
>> geographically dispersed team with people in America (North and
>> South), Europe and Australia, but hopefully we can find a time that
>> works.
>
> Agreed.  I suggest we do the first one next week, on Friday.   What
> time would work for you, James?  I guess you're used to that kind of
> conflict due to previous Launchpad work, so will probably have some
> well-known times where it's best for most people.

Probably some time late night for me would work best.  That would be
afternoon in Europe and morning in America.  It looks like 11pm for me
is 8am for Jamu, so perhaps some time around then would work.


>> Comments are welcome, as are further suggestions to improve our
>> development process.
>
> Indeed!  Again, thank you very much for pushing these ideas forward!


James.