brainstorming for UDS-N - Application Developers

[Scott Kitterman]

On Wednesday, October 06, 2010 07:23:00 am Matthew Paul Thomas wrote:
> Scott Kitterman wrote on 06/10/10 06:20:
> > On Tuesday, October 05, 2010 09:19:41 am Matthew Paul Thomas wrote:
> >...
> >
> >> For example, in Natty we'll be doing work that makes it easier to
> >> visually distinguish core updates from application updates.
> >> <https://wiki.ubuntu.com/SoftwareUpdates?action=AttachFile&do=get&target
> >> =up dates-list.jpg> That alone would make it much easier for someone to
> >> (for example) postpone an application update to a weekend when they
> >> have time to troubleshoot any problems. In future, it might even be
> >> possible for them to roll back to the previous version.
> > 
> > So it's better to accept a library update when you don't have time to
> > troubleshoot?  I don't think this distinction is terribly helpful.
> 
> By spending less time on other people's applications, we could spend
> more time testing fixes to Ubuntu core.

What's Ubuntu's core?  Most of it is other people's applications.  
Distribution developers are primarily integrators.  It's a roll that doesn't, 
IMO, have an equivalent on proprietary platforms because none of them are 
trying to provide what we are trying to provide (a complete, well integrated 
base operating systems, standard applications, etc.).

> > Open Office may be a leaf application, but if I've got a presentation
> > I have to get finished to give to a customer tomorrow, it's a lot more
> > mission critical to me at the moment than any number of core libraries.
> 
> Exactly.

Note that this is a core Ubuntu application and also somebody else's.  The 
things we spend most of our effort on are core no matter who the upstream 
developer is.

> >...
> >
> >>>> Every general-purpose operating system has gone through a stage
> >>>> where the primary publisher of software for it was the OS vendor
> >>>> themselves. Every popular general-purpose operating system quickly
> >>>> outgrew that stage.
> >>> 
> >>> The Linux distribution paradigm is different than other operating
> >>> systems.  I don't think the analogy is relevant.
> >> 
> >> I'm interested to know why.
> > 
> > Why it's different, why it's a good thing it's different, or why
> > drawing analogies from things that are substantially different has
> > little relevance?
> 
> Why the difference is relevant. (I don't think it's even an analogy, but
> rather a general pattern.)

I don't recall having used a general purpose operating system where I viewed 
the O/S developer as the primary publisher of software for it (and I've been 
at this for a long time).  As distribution developers we aim to provide an 
integrated experience for our users.  Most of the work we do is integration.

Some of this is necessity (packaging systems are painful and distro unique, so 
distro developers are the ones that deal with them) and some of this is a true 
feature.  If you install one of the *buntu flavors you get a complete system 
that's ready to support it's primary use case.  You can install additional 
applications to extend that, but the system is complete and useful for many 
things as installed.  This is a good thing.  

I don't see this as something to outgrow. I see it as one of the (many) 
significant features that makes the platform attractive.

> >...
> >
> > But it can't be done by fiat.  We didn't achieve the packaging system
> > we have now accident and it will take good engineering work to get to
> > something that is both simpler and functional.  I don't know of anyone
> > that defends the current complexity of packaging as a good thing, what
> > we lack is tools to to get there.  Additional theorizing that it should
> > be easier, doesn't really help.
> 
> Several concrete ideas have already emerged from this discussion.
> Calling them "theorizing" doesn't really help.

I agree that we are starting to get some good ideas.  I think the current ARB 
process is predicated on the theory that the current review process is 
randomly too complicated and by writing some process documents we can make 
things easier.

> >...
> >
> > Debian/changelog isn't meant for non-technical end users.  If you want
> > something that is, we need something different.  The packaging
> > changelog serves it's own useful purpose.  The fact that it doesn't
> > also serve this purpose is not a deficiency in debian/changelog.
> > That's out of scope for it.
> > 
> > OTOH, it's difficult enough for technical people to write text that is
> > sensible to non-technical users.  Please don't insist I learn some new
> > markup language too.
> 
> Fair enough. We don't want to be in Microsoft's situation of waiting to
> release a security update until a corresponding Knowledge Base article
> has been written and translated into 23 languages.
> 
> So, one thing that might help is a log of "What this update does"
> separate from debian/changelog (or maybe this can be demarcated inside
> debian/changelog somehow).
> 
> Second, providing template plain-English text for maintainers to use for
> different kinds of update -- even as simple as "Fixes a problem where
> ______ would ______" or "Fixes a security problem where an infected
> document could ______".
> 
> Third, setting up a team of recognized tech writers for small tasks like
> this, so someone preparing an SRU can hop into an IRC channel and get
> advice on text for their update.

I agree this would be useful.  Someone (not me, I don't have time to deal with 
it) should capture this and write up a spec for a UDS session on it.

> >...
> >
> >> That's true. The App Review Process is a very small step.
> > 
> > It's not a technically well founded step.  We should be doing the
> > engineering of something that is suitable for a lightweight review and
> > approval process before we deploy one.  The current App Review Process
> > if more like shuffling the deck chairs on the Titanic.
> 
> So, given that the ARP already exists, what would a lightweight process
> to replace it look like?

I think that's putting the cart before the horse.  I think we need to design a 
system that makes it safe to allow applications to be deployed with minimal, 
rapid review before we design the process to manage it.  All the process in 
the world doesn't change the fact that maintainer scripts run as root and can 
do ~ anything to a system, that applications can read user data, have network 
access, etc.  Let's design the technical measures to reduce risk both to 
systems and user data and then once we have successfully mitigated the risks 
associated with providing lightly reviewed software to non-technical end 
users, then we can determine how much and what kind of process is needed.

Fundamentally, things like the ARB don't scale to the level that Android 
covers.  If this became wildly successful, any manual review process would 
become a roadblock.  It should be a design goal for it to be "safe" for new 
applications to be added to software center with no manual review at all.  If 
we don't get that, as soon as we move from developing an "unpopular" OS to 
developing a "popular" OS, we'll be stuck.

> >...
> >
> >> <http://launchpad.net/ubuntu/+spec/foundations-n-backports-notautomatic>
> > 
> > That's the one.  It covers changes to the apt resolver to deal with
> > dependencies in a non-automatic repository, changes to make -backports
> > not automatic, and U/I to present multiple versions of packages to
> > users sensibly (once backports is not automatic, they'd need to be able
> > to pick if they want the stable release one or the new one from
> > backports).
> 
> Ok, that seems like it would involve tweaks to the design of Update
> Manager (and maybe Ubuntu Software Center too). Subscribed.

Yes.  I believe that changes to both would be beneficial to present this well 
to users (kpackagekit too).

> >...
> >
> > Right, but Android Marketplace is primarily a source for external
> > applications.  In a Linux distribution that isn't the case.  I
> > relatively routinely run into bugs or mailing list messages  where
> > people are complaining about PPA packages, so I already see this as a
> > problem.  Third party applications (or changes to existing packages)
> > are already too closely identified with Ubuntu.
> >
> >...
> 
> We already distinguish "Provided by Ubuntu" vs. "Independent" in Ubuntu
> Software Center. But as we accumulate independent applications, it will
> become more important for USC to say *who* develops a particular
> application.

We don't, from my experience, yet distinguish adequately.  Perhaps Maverick 
will be better.

Scott K