Did we really release 8.04?
ubuntu at kitterman.com
Mon Jul 7 13:12:13 UTC 2008
On Monday 07 July 2008 08:50, Cory K. wrote:
> Scott Kitterman wrote:
> > I think the only path to better tested releases is higher quality test
> > releases.
> Sad fact is people don't install the alpha/betas at the same level as
> final. So more actual testing/feedback was done after release. I can
> tell you for sure this was the case for Ubuntu Studio.
> For me, it's all about how do we get people to use the later alphas or
> betas so final is better? Age old issue it seems.
Agreed. The problem is that the easy solution of moving the problem down a
step (fix post-release and .1 is good) is a never ending slippery slope.
Personally, I think it means we need to be doing a much better job of testing
and bug fixing as developers.
There is one small upstream project that I'm the sole developer for. The
first three releases I did were well received and I didn't have any trouble
with regressions/failures. Then with the fourth release I started having
some significant bugs. Five and six were problematic too. Finally, for the
seventh (and current) release, I took the time to write a comprehensive test
suite that allowed me to automatically exercise virtually all the code paths.
This one I've had no reports of regressions or problems (my test suite did
find a major regression I'd missed in my other testing, but it did it before
The moral of the story is that my project had increased in complexity beyond
what my QA approach could sustain and I had to re-engineer QA, even at the
cost of some features. Once I had a QA approach that matched the complexity
of the project, then release quality went up again.
I suspect that we may be in a similar position with Ubuntu. We need to
radically rethink testing and how test results get back into fixes. I
believe that Ubuntu has gotten more complex and we need to match our test/fix
methodology to match. I would like to hear ideas on the subject.
There was some discussion at UDS about developing the ability to trivially
clone a host machine into a VM so that users could easily test their setups.
I can see this being very useful. I am not currently willing to run Intrepid
on any of my servers, but if we had a capability like this, I would be
willing to take my test server, clone a VM, upgrade to Intrepid, and run with
it. That would potentially expose more problems earlier.
I'm not certain, however, that we need more bugs. We need better bugs and we
need more fixes. Suggestions?
More information about the Ubuntu-devel-discuss