LTS and release methodology

[Matt Zimmerman]

On Mon, Jul 07, 2008 at 07:48:03PM +0200, Vincenzo Ciancia wrote:
> Il giorno lun, 07/07/2008 alle 18.04 +0100, Matt Zimmerman ha scritto:
> > 
> > Instead, we focus on defining a subset of functionality which can be
> > tested in practice.  You can find the corresponding test plans here:
> > https://wiki.ubuntu.com/Testing along with instructions for how you can
> > participate in the testing effort and find the problems which matter to
> > you.
> 
> I think I can add two notices:
> 
> 1) what about giving really high priority to _regressions_ of these test
> cases? For example pdf printing has been and is broken in ubuntu since I
> think one year or so, due to evince not being capable of printing
> correctly many (but not all) of these files. This means the LTS does *not*
> pass the test cases.

I print a PDF from evince at least once per week and am happy with the
output, so there seems to be more subtlety to the problem than "PDF printing
is broken in Ubuntu".  In fact, it seems clear from the upstream bug
(correctly forwarded by an Ubuntu developer) that the problem is not
specific to Ubuntu either.

> The bug 
> 
> https://bugs.edge.launchpad.net/poppler/+bug/150187
> 
> is open but stuck in a dead end. This bug must be paid a much greater
> attention than it is now: I have to teach people to install acrobat
> reader, that sucks on linux in any point except for its very good printing
> abilities. People will _not_ use ubuntu if they can't print pdf files.
> Indeed, any regression regarding such basic functionality as the thest
> cases you kindly provided should be given a very high importance for the
> quality of the distribution.

Your bug seems to have received an appropriate response both from Ubuntu and
from the upstream project.  The process, in this case, seems to be mostly
working, but the upstream bug simply isn't fixed yet.  The next step would
be to continue to work with the upstream developers via
https://bugs.freedesktop.org/show_bug.cgi?id=12769

We can try, in Ubuntu, to minimize regressions and address as many as we
can, but we cannot insulate Ubuntu from upstream regressions entirely.  to
attempt to do so would severely hinder the overall progress of the project.

Naturally, though, we should be open to practical suggestions for how we
could do better.

> 2) What about adding some basic hardware testing to these test cases?  For
> example, vga out support never survives a release or two before being
> killed by X progressing, in my experience, but it is very important for
> the whole academic community which is one of the primary targets of
> linux-based environments at the moment. As of now, I own three different
> laptops, of different ages. For different bugs none of them can project on
> a VGA projector in hardy, and ALL of them have been able in past releases.
> This gives a very bad impression of ubuntu to newcomers.

This is a very tricky problem, as such bugs are notoriously
hardware-specific and can affect only certain combinations of graphics
adapter and projector.  Some projectors have bugs which make it unlikely
that they can work at all without manual configuration.

We (Canonical) test a certain amount of hardware on pre-release versions,
but can't possibly cover this exhaustively.  A community approach is the
only practical way to do this, and the testing needs to happen early in the
release cycle, when there is still time to analyze and fix the failures.

Have you tested your hardware with Intrepid?

-- 
 - mdz