brainstorming for UDS-N - Application Developers

[Scott Kitterman]

On Monday, October 04, 2010 10:31:36 am Matthew Paul Thomas wrote:
> Krzysztof Klimonda wrote on 01/10/10 20:40:
> > On Fri, 2010-10-01 at 17:56 +0100, Evan Dandrea wrote:
> >...
> >
> >> Buggy software is going to get into Ubuntu no matter what we do, /but
> >> that's okay/.  Equally, if we patch these applications ourselves, what
> >> kind of message is that sending to developers about our trust of them,
> >> and whose responsibility broken software is?
> > 
> > We patch software ourselves not because we believe we can fix bugs on
> > our own - we do it so our users are not exposed to potential new bugs
> > in the newer version of the software.
> 
> Yes, that was Evan's point. The only way we would have time to do that
> well for every Ubuntu application, would be if Ubuntu's application
> selection was even tinier than it is.
> 
> The usual consequence of the current approach is that people have to
> suffer a bug for up to six months after the application developer has
> released a fixed version.

They equally often manage to avoid dealing with regressions associated with 
new versions of the software.  It annoys me to no end to get an update on my 
Android that then breaks something and I get three updates in four days 
getting stuff that was working before back to working.  Currently it's pretty 
safe to apply updates in Ubuntu and I don't (on desktops) spend a lot of time 
considering the situation.  Stability of a release is a feature not a bug.

> >                                       The message we send to upstream
> > 
> > developers is that we care about their software (after all we have to
> > care to prepare and test patches ourselves), but our users come first.
> > I don't think there are that many upstream developers who don't like it
> > - that's how most big distribution works. Some of them are even helping
> > us locate bugs worth fixing in our releases.
> 
> There has never been a "big distribution". No distribution, Ubuntu
> included, has ever had more than 2 percent market share.
> 
> Every general-purpose operating system has gone through a stage where
> the primary publisher of software for it was the OS vendor themselves.
> Every popular general-purpose operating system quickly outgrew that stage.

The Linux distribution paradigm is different than other operating systems.  I 
don't think the analogy is relevant.

> >...
> >
> >> If we have to package the applications ourselves because it requires
> >> years of experience and understanding of an inordinate amount of
> >> complexity, what does that say about our package management system?
> > 
> > Packaging requires experience because of the way software is installed,
> > updated and interacts with other software. It requires understanding of
> > the system.
> 
> Yes, that's part of the problem. Packaging should not require that
> experience and understanding of the system. Thirteen-year-olds write and
> sell iPad applications.

We have fourteen year old MOTU also.  Packaging should not require 
understanding of the system and software should write itself.  Both are 
interesting theories, but will take some work to accomplish.  We can't just 
banish packaging by fiat.  It will take some work to really develop a system 
that, given certain standardized choices and constraints, makes packaging 
trivial.

> >             Adding a detailed changelog makes it easier for our users
> > 
> > to make sure what can they do with their software. detailed
> > dependencies make it easy to make transitions etc.
> 
> Do you think that more than one in a thousand users of the average
> Ubuntu application have ever looked at its changelog?

On Android Marketplace, the information about updates is displayed prominently 
and is (I think) very important.  What we currently ship is not the most user 
friendly information, but in a world where application updates are not 
mediated by the distribution development process, users will need to be more 
aware of what's changing in their software, not less.

> > Also, packaging a simple desktop application is really easy this days.
> 
> <https://wiki.ubuntu.com/PackagingGuide/Complete> is 16518 words long.
> Even the "Basic Packaging" section alone is longer than the US
> Declaration of Independence, and the original US Constitution, combined.
> 
> > Making it even easier wouldn't really help us to provide a better
> > software to our users.
> 
> It would help application developers provide better software to their
> users.
> 
> >> Instead of handicapping developers based on these misconceptions, we
> >> need to free them to deploy applications and make choices on their
> >> own,
> > 
> > Okay, but what about our users? Are they supposed to deal with errors
> > made by developers?
> 
> That's between the developers and their users. Our job is to make that
> interaction easier.

Yes, but it needs to be done in a reasoned, technically well founded way.  
Appointing a new board and declaring a new world order isn't enough.

> >> then deal with the repercussions themselves.  We need to empower our
> >> users to make the best decisions on what applications they should use
> >> without having to think hard about their options.  So lets introduce
> >> some new metrics in Ubuntu Software Center, and lets make it so that
> >> they are applied to the sorting automatically. We can start by
> >> lowering the page rank of applications that not only have low ratings
> >> or low download counts, but also have high bug counts, that have high
> >> crash reports, and that have a large time delta between bugs and
> >> their fixes.
> > 
> > Isn't it what would reviews and voting for application achieve?
> > I don't think that the application that crashes often enough for it to
> > be important or have other severe bugs is going to get a lot of
> > positive reviews and a good score.
> 
> That's true. But submitting a crash report, to a real crash database
> (instead of abusing a login-required bug tracker for it, as we do now),
> would take a tiny fraction of the effort required to submit a rating and
> review. Therefore there would be many more crash reports than ratings
> and reviews. So we could show better rankings by using both.

Except we turn crash reporting off post-release so we don't get inundated with 
reports.

> >> Lets further build on this to create a tight feedback loop between
> >> developers and users.
> > 
> > What does it really mean? Our users already have contact informations
> > for the software they use, upstream developers have an unlimited access
> > to Launchpad (they can easily get accepted to the ~ubuntu-bugcontrol
> > and triage bugs on LP). We are already providing an infrastructure for
> > nightly builds. What else should be done? Probably worth discussing on
> > the UDS.
> 
> Hence the subject of this thread.
> 
> >> We need to make it so that developers can quickly deploy an
> >> application that then appears in Ubuntu Software Center for anyone on
> >> that release of Ubuntu, regardless of where we our in our own cycle.
> > 
> > The can add software to the current development release and request a
> > backport to previous releases.
> 
> Which, as Jorge said, they don't.

The current design doesn't suite this terribly well.  The amount of work to 
get to something that does is small, but has sat on the TODO low priority pile 
for at least a couple of years.

> > (...)
> > 
> >> This application deployment system I describe should at no point block
> >> on a human being from the Ubuntu project like our existing systems,
> >> both the regular archive and -extras, do.  These quite simply cannot
> >> scale.  Instead, lets create an automatic system that takes an
> >> application submission, runs its test suite, does some static
> >> analysis, and conducts any other sanity checks we can think of before
> >> letting it through to the archive.  If any of these tests fail, it
> >> simply rejects the application back to the developer.  Let them worry
> >> about fixing it.  Let them take complete ownership of their work.
> > 
> > No, we create a distribution for our users not as a place for
> > developers to dump their software to.
> 
> In the past four years, not one but two operating systems have each,
> starting from zero, accumulated more than 20 *times* as many
> applications as Ubuntu has. What we are doing is not working. It's time
> to try something else.

I fundamentally think this is wrong.  As a user I really don't need MORE 
applications.  I need applications that are better, more stable, faster.  The 
new use case is rare.  Just going for volume isn't the right answer and the 
lack of a certain volume isn't indicative of a problem.

> >                            Unit tests, static analysis, any sanity
> > 
> > checks we can think of can be either bypassed or plain wrong.
> 
> Why would a developer want to bypass checks that would help save them
> from disappointing their users?
> 
> Do you have any examples of sanity checks that produce unfixably wrong
> results?
> 
> >                                                               Your idea
> > 
> > could work if all applications were sandboxed (i.e. Android model) but
> > not in Ubuntu.
> 
> Ubuntu Software Center, for example, has unit tests without being
> sandboxed.

So?  Software Center is maintained and developed by trusted developers and so 
doesn't require sandboxing.  If you want to get beyond the standard 
distribution model of software deployment, then you are going to need some 
technical measures to isolate untrusted applications from other applications, 
user data, and system elements they are not authorized to use.

> > There is a scaling issue and the burden on both maintainers and QA team
> > is big but removing this burden by removing them from the process is
> > not going to make Ubuntu a better distribution (for our users [1]).
> 
> Why not?
> 
> > [1] I use "for our users" quite a lot - that's because what is good for
> > developers (or even for us) is not always good for the users of our
> > distribution - for example developers want to push their new software
> > to as many people as possible in as little time as possible but, on the
> > other hand, not all users are interested in the newest and greatest
> > stuff.
> >
> >...
> 
> That decision should be up to individual users and organizations.

We can't make an infinite variety of an infinite number of versions of 
applications to satisfy everyone.  We need to make some opinionated choices 
about where to balance between the latest and greatest and stability and 
predictability (we, reasonably, choose differently in different release cycles - 
e.g. generally favoring stability for LTS releases).  There are also often 
third party resources (e.g. PPAs) for users that want a different experience.  
That's OK and is a data point that says the distribution model is wrong.

> >> This system wont catch everything and we shouldn't ever operate on the
> >> false hope that it will.  All it's meant to do is filter out the very
> >> worst that we can catch, and leave the rest up to the users to try and
> >> report back on.
> > 
> > When broken software reaches users it's already too late. By publishing
> > something in *.ubuntu.com "namespace" we take responsibility for it and
> > users are going to complain to us.
> 
> That's not credible. Neither Ubuntu Software Center nor Update Manager
> reveal the domain name of the software they install, even for people who
> know what a "namespace" is.

If it's installed through the "Ubuntu Software Center" of course the attach 
the Ubuntu brand to the quality of the experience.  I'm not sure how you could 
imagine it otherwise.

> >> We should also give the developers flexibility in the way in which
> >> they stage out these deployments.  We should allow them to offer an
> >> application to say, 5% of the Ubuntu audience, wait for feedback, and
> >> if it doesn't have any grave bug reports coming in, push it to the
> >> remaining 95%.
> > 
> > Now, the idea sounds pretty nice but I'm not sure it would work well.
> > Leaving aside the question how to do that I'm concerned that it would
> > make community support harder and making people feel bad that they
> > "don't have an access" to the newer software while their friends do.
> > Things like this are done - beta tests etc. but not at this level. It
> > would feel like we, as Ubuntu, are limiting our users.
> 
> Things like this are often done at this level, and at much larger
> levels. Twitter is doing it right now with 105 million registered users.
> Microsoft does it for Office's half a billion users. And Facebook also
> often does it with half a billion active users. They would be
> incompetent if they didn't.

This sounds like a reasonable capability to want if one is going to bypass the 
distro development process.

> >...
> >
> >> We need to make it our goal that any software developer should be able
> >> to package their application without difficulty and without causing
> >> harm to the system.
> > 
> > Isn't it a goal for Quickly?
> 
> Only for the type of applications created with Quickly.
> 
> >                              We can obviously work on making it easier
> > 
> > for developers to prepare a preliminary packaging but it still has to
> > be checked by someone familiar with Ubuntu and Debian.
> 
> Yes, that is part of the problem.
> 
> > Upstream developers shouldn't have to understand various distribution,
> > their packaging methods, policies they have in place.
> 
> That is true only for unpopular operating systems.

For other than a narrowly defined set of special cases, packaging concerns are 
not going to go away.  For a development target like Android, it is only the 
special case that is accessible to third party developers.  In the Ubuntu 
project we have the potential to add such a thing but also keep our more 
powerful/general packaging solutions available for applications that need it.  
We can match what these other systems do and still do more.

Scott K