brainstorming for UDS-N - Application Developers

[Matthew Paul Thomas]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Krzysztof Klimonda wrote on 01/10/10 20:40:
>
> On Fri, 2010-10-01 at 17:56 +0100, Evan Dandrea wrote:
>...
>> Buggy software is going to get into Ubuntu no matter what we do, /but
>> that's okay/.  Equally, if we patch these applications ourselves, what
>> kind of message is that sending to developers about our trust of them,
>> and whose responsibility broken software is?
> 
> We patch software ourselves not because we believe we can fix bugs on
> our own - we do it so our users are not exposed to potential new bugs
> in the newer version of the software.

Yes, that was Evan's point. The only way we would have time to do that
well for every Ubuntu application, would be if Ubuntu's application
selection was even tinier than it is.

The usual consequence of the current approach is that people have to
suffer a bug for up to six months after the application developer has
released a fixed version.

>                                       The message we send to upstream
> developers is that we care about their software (after all we have to
> care to prepare and test patches ourselves), but our users come first.
> I don't think there are that many upstream developers who don't like it
> - that's how most big distribution works. Some of them are even helping
> us locate bugs worth fixing in our releases.

There has never been a "big distribution". No distribution, Ubuntu
included, has ever had more than 2 percent market share.

Every general-purpose operating system has gone through a stage where
the primary publisher of software for it was the OS vendor themselves.
Every popular general-purpose operating system quickly outgrew that stage.

>...
>> If we have to package the applications ourselves because it requires
>> years of experience and understanding of an inordinate amount of
>> complexity, what does that say about our package management system?
> 
> Packaging requires experience because of the way software is installed,
> updated and interacts with other software. It requires understanding of
> the system.

Yes, that's part of the problem. Packaging should not require that
experience and understanding of the system. Thirteen-year-olds write and
sell iPad applications.

>             Adding a detailed changelog makes it easier for our users
> to make sure what can they do with their software. detailed
> dependencies make it easy to make transitions etc.

Do you think that more than one in a thousand users of the average
Ubuntu application have ever looked at its changelog?

> Also, packaging a simple desktop application is really easy this days.

<https://wiki.ubuntu.com/PackagingGuide/Complete> is 16518 words long.
Even the "Basic Packaging" section alone is longer than the US
Declaration of Independence, and the original US Constitution, combined.

> Making it even easier wouldn't really help us to provide a better
> software to our users.

It would help application developers provide better software to their users.

>> Instead of handicapping developers based on these misconceptions, we
>> need to free them to deploy applications and make choices on their
>> own, 
> 
> Okay, but what about our users? Are they supposed to deal with errors
> made by developers?

That's between the developers and their users. Our job is to make that
interaction easier.

>> then deal with the repercussions themselves.  We need to empower our
>> users to make the best decisions on what applications they should use
>> without having to think hard about their options.  So lets introduce
>> some new metrics in Ubuntu Software Center, and lets make it so that
>> they are applied to the sorting automatically. We can start by
>> lowering the page rank of applications that not only have low ratings
>> or low download counts, but also have high bug counts, that have high
>> crash reports, and that have a large time delta between bugs and
>> their fixes.
> 
> Isn't it what would reviews and voting for application achieve?
> I don't think that the application that crashes often enough for it to
> be important or have other severe bugs is going to get a lot of
> positive reviews and a good score.

That's true. But submitting a crash report, to a real crash database
(instead of abusing a login-required bug tracker for it, as we do now),
would take a tiny fraction of the effort required to submit a rating and
review. Therefore there would be many more crash reports than ratings
and reviews. So we could show better rankings by using both.

>> Lets further build on this to create a tight feedback loop between
>> developers and users.
> 
> What does it really mean? Our users already have contact informations
> for the software they use, upstream developers have an unlimited access
> to Launchpad (they can easily get accepted to the ~ubuntu-bugcontrol
> and triage bugs on LP). We are already providing an infrastructure for
> nightly builds. What else should be done? Probably worth discussing on
> the UDS.

Hence the subject of this thread.

>> We need to make it so that developers can quickly deploy an
>> application that then appears in Ubuntu Software Center for anyone on
>> that release of Ubuntu, regardless of where we our in our own cycle.
> 
> The can add software to the current development release and request a
> backport to previous releases.

Which, as Jorge said, they don't.

> (...)
>> This application deployment system I describe should at no point block
>> on a human being from the Ubuntu project like our existing systems,
>> both the regular archive and -extras, do.  These quite simply cannot
>> scale.  Instead, lets create an automatic system that takes an
>> application submission, runs its test suite, does some static
>> analysis, and conducts any other sanity checks we can think of before
>> letting it through to the archive.  If any of these tests fail, it
>> simply rejects the application back to the developer.  Let them worry
>> about fixing it.  Let them take complete ownership of their work.
> 
> No, we create a distribution for our users not as a place for
> developers to dump their software to.

In the past four years, not one but two operating systems have each,
starting from zero, accumulated more than 20 *times* as many
applications as Ubuntu has. What we are doing is not working. It's time
to try something else.

>                            Unit tests, static analysis, any sanity
> checks we can think of can be either bypassed or plain wrong.

Why would a developer want to bypass checks that would help save them
from disappointing their users?

Do you have any examples of sanity checks that produce unfixably wrong
results?

>                                                               Your idea
> could work if all applications were sandboxed (i.e. Android model) but
> not in Ubuntu.

Ubuntu Software Center, for example, has unit tests without being sandboxed.

> There is a scaling issue and the burden on both maintainers and QA team
> is big but removing this burden by removing them from the process is
> not going to make Ubuntu a better distribution (for our users [1]).

Why not?

> [1] I use "for our users" quite a lot - that's because what is good for
> developers (or even for us) is not always good for the users of our
> distribution - for example developers want to push their new software
> to as many people as possible in as little time as possible but, on the
> other hand, not all users are interested in the newest and greatest
> stuff.
>...

That decision should be up to individual users and organizations.

>> This system wont catch everything and we shouldn't ever operate on the
>> false hope that it will.  All it's meant to do is filter out the very
>> worst that we can catch, and leave the rest up to the users to try and
>> report back on.
> 
> When broken software reaches users it's already too late. By publishing
> something in *.ubuntu.com "namespace" we take responsibility for it and
> users are going to complain to us.

That's not credible. Neither Ubuntu Software Center nor Update Manager
reveal the domain name of the software they install, even for people who
know what a "namespace" is.

>> We should also give the developers flexibility in the way in which
>> they stage out these deployments.  We should allow them to offer an
>> application to say, 5% of the Ubuntu audience, wait for feedback, and
>> if it doesn't have any grave bug reports coming in, push it to the
>> remaining 95%.
> 
> Now, the idea sounds pretty nice but I'm not sure it would work well.
> Leaving aside the question how to do that I'm concerned that it would
> make community support harder and making people feel bad that they
> "don't have an access" to the newer software while their friends do.
> Things like this are done - beta tests etc. but not at this level. It
> would feel like we, as Ubuntu, are limiting our users.

Things like this are often done at this level, and at much larger
levels. Twitter is doing it right now with 105 million registered users.
Microsoft does it for Office's half a billion users. And Facebook also
often does it with half a billion active users. They would be
incompetent if they didn't.

>...
>> We need to make it our goal that any software developer should be able
>> to package their application without difficulty and without causing
>> harm to the system.
> 
> Isn't it a goal for Quickly?

Only for the type of applications created with Quickly.

>                              We can obviously work on making it easier
> for developers to prepare a preliminary packaging but it still has to
> be checked by someone familiar with Ubuntu and Debian.

Yes, that is part of the problem.

> Upstream developers shouldn't have to understand various distribution,
> their packaging methods, policies they have in place.

That is true only for unpopular operating systems.

- -- 
Matthew Paul Thomas
http://mpt.net.nz/
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkyp5UcACgkQ6PUxNfU6ecqwrACdEkIxZWUc6qCLG6vy1Io/emxK
jDAAoIHh8n+DbMhQ9y5Q3g793P+d7Wer
=JTvM
-----END PGP SIGNATURE-----