Do you really want developers to be on this list was (Re: Very bad status of hardware (especially wifi) support in ubuntu, due to the too many accumulated regressions)

[Stephan Hermann]

Hi Markus,

On Thu, 2008-11-13 at 11:56 +0100, Markus Hitter wrote:
> Am 13.11.2008 um 10:32 schrieb Stephan Hermann:
> 
> > But reality told me different.
> 
> Stephan, your points about the unfortunate truth are valid.  

Sad, but true.

> Nevertheless, software quality is one of the keys to success.

> I've just filed the second bug where one of the Gnome applets  
> segfaults in a standard situation. Many developers obviously code  
> really sloppy, a la "it worked once in my situation, so it works  
> always in all situations". Some developers even consider a segfault  
> as a normal way to end the execution of an application. This is a  
> more general observation of mine, this is ridiculous.
> 
> While we can't "fix" developers, we can put more automatic helpers  
> into place:
> 
>   - Keep Apport enabled even on stable releases. Hiding bugs doesn't  
> help.
> 
> While this doesn't fix bugs by it's self, it greatly helps to fix  
> them after the fact (and timely educate developers about their  
> practices).

Yes...this can help us, to shape applications which are running actually
on the user's desktops, but doesn't prevent it from happening.
If the bug is found after a release, it's already too late. Well, not
too late to fix it in an upcoming release, but too late today.

But here is a point: Why did the bug occur after the release first, or
when it occurred during development, why nobody took care to fix it?

And here are some answers (hopefully not all, but some, and mostly not
correct):

1. The bug occurred after the release:

a) The application in question is not used by a wide range of users. If
it would have been used by a broader community, the bug would have
occurred during development
b) Nobody, using this software before release, was actually able to file
a bug report to the distro bug tracker. That's not good. And this starts
another flow of questions, but those I won't raise here.

2. The bug occurred during development, why wasn't it fixed by someone?

a) There was no bug report, look at 1.b)
b) Most likely the application package waits in the Universe/Multiverse
pocket, and no non-paid/paid dev took care, because it's not important
for the release goal and nobody was interested, because it's
unsupported.
c) The application is in one of the supported pockets (main/restricted),
the core devs had it on the radar, but decided to take it as a
regression which could be fixed later, and is not so important for the
release in general.
d) the bug is so difficult and non-trivial to reproduce, or to fix, and
the bug was pushed upstream, and the distro team just have to wait for a
fix or an answer.

This is belongs to the application level so far.

Coming to the more delicate kernel level:

> 
> Additionally, this opens the door to get some automatic measure about  
> the quality of drivers or other software. Count open bugs and you  
> know what you roughly can expect. If you count too many of them, drop  
> the hardware in the compatibility list.

As said in one of my mails:

The problem here is, that some users with the hardware on a list don't
have problems, but others have.
Now, how can we determine what the difference between the hardware is,
between those with and those without problems?

This task is not easy. There needs to be input from the users with the
non-working hardware. Most likely, that this information can be gathered
with some magic commands on CLI, which is also provided by a nice
developer. But user thinks: "Damn, this takes more time, more that I
want to invest in this...this OS is crap...the devs are lazy bastards,
because the hardware is on the list...but as I can see, it doesn't work,
wait I'll tell them that on the ML or whereever".

So, for the kernel devs or other devs in other parts of the distro, it's
quite difficult sometimes to get the necessary infos, when people are
not coming back and providing the infos about the hardware, or if they
did, then they won't come back to test the fix, because they already
installed another OS or switched back to something else.

There are so many variables, which are playing a part, starting from
non-working hardware revision to the decision: "Ok, this card is only 10
days old, most likely that there are not many people who are using it,
we need to forget about this, during this release cycle, and yes, we
screw the people who have this card, but the majority is not affected at
all." to "Shit, we didn't even know that this wasn't working, yeah there
was a report, but we didn't get the infos back we needed to
investigate..shit happens, but shit happens all the time, let's document
it".

And in reality, only one or two newer revisions of chipset are not
working anymore...but to get this revision it takes time to get the
right info from the users.

> 
> To keep more users happy:
> 
>   - Allow downgrades. This should help narrowing potential causes of  
> the trouble.

This is something I don't understand.
When I upgrade to a new release, I always think (or is it knowing): "Ok,
for the next 4 hours I'll sit in front of this computer, and I expect
something to break...because it's software made by people". If nothing
breaks, then I'm really surprised and happy. But when something breaks,
I already expected that. And when I find the cause for the breakage,
I'll try to fix it, AND/OR file a bug report about that issue. 

Therefore, I don't upgrade my production machine without any real
testing. But this won't help for everybody, I know.




> Ideally, there would be a big regression testing facility, like Wine  
> has one. Each time a Wine developer fixes a bug, he's pushed to  
> create a test for his case. These test cases are run automatically  
> for each commited patch and pretty well avoid introducing a bug a  
> second time.

Test Driven Development is a good thing...Much better is, when you write
a test before you implement...but we all know how it works ;)

Wine is not safe, too...there were many releases where CS worked, and
the next release wasn't working..and TBH, this is real...

To come to an end...

It's hard to please everybody...shit happens and will happen in the
future. Hardware and Hardware Drivers are not simple applications from
someone. They need more love then anything else...that's why kernel devs
are in need of hardware samples and tests from users.
Applications can break, too..whysoever..development for applications in
our area is done by contributors who do it most of the time in their
spare time, without being paid, and only short ammount of time. We can
help those people when we file bugs, giving them examples how to
reproduce those bugs. etc. but it won't help to demand something which
is (quite) impossible to accomplish.

Support / Unsupported Hardware Lists are ok, up to a certain level of
information. 

Regards,

\sh