Proposing a New App Developer Upload Process

[David Planella]

Al 05/09/12 05:18, En/na Emmet Hikory ha escrit:
> Steve Langasek wrote:
>> On Tue, Sep 04, 2012 at 05:20:47PM -0400, Michael Hall wrote:
>>>> It's possible that namespacing within /usr - for instance,
>>>> requiring each subdirectory and binary name to be prefixed with
>>>> 'extras.' - would give better results with regards to the upstream
>>>> build systems, I'm not sure. But if we are going to try to do
>>>> namespacing of extras packages to avoid the need for coordination,
>>>> we definitely would need improved package helpers that support
>>>> namespacing of packages (i.e., debhelper support).
>>
>>> Would it be reasonable for MyApps to add a package name prefix to the
>>> submitted package, rather than requiring the developer to do so?
>>> MyApps will already be modifying the submitted package to include the
>>> generator AppArmor profile.
>>
>>> So, for example, if I submit quickly-gtk as a source package to
>>> MyApps, it would convert it to extras-quickly-gtk, add the AppArmor
>>> profile, and re-build the source package before submitting it to the
>>> PPA/buildds.
>>
>> For package renaming that seems easy enough to do, but if we're worried
>> about file conflicts, dynamically prefixing filenames when building the
>> package is going to have an even worse impact on the upstream code than
>> changing directories does.
> 
>     Changing the base filename indeed generates more complicated issues,
> not only in terms of collisions, but also in terms of coordination between
> code and data (e.g. code expects to find ${CONFIG_DIR}/package and fails
> to initialise properly with only ${CONFIG_DIR}/extras-package.
> 
>     However, if we consider "renaming" to apply to the entire path, rather
> than the bare filename, this becomes considerably less of an issue.  If
> MyApps were to call tooling that changed the default installation location
> for files while preserving bare filenames, then there is less potential
> for conflict.  The standard mechanism for this is to place the non-system
> files in per-package namespaced directories in /opt.
> 

I fully sympathize and understand the advocacy for the use of /opt at
the packaging level and I would in principle support it: it is the
standard FHS location for add-on software packages and it creates a
separate installation namespace that prevents file collisions. In
theory, it seems the best approach.

However, reality has shown that (a) it is a big hurdle for app
developers (who are actually the people we're trying to make the process
easier for) to follow this policy and (b) we're enforcing a policy not
even our tools support.

For (b), what I believe is that it is also clear that no one is going to
work to improve /opt support in tooling or in developer toolkits in the
near or distant future. So for this alone I consider it to be a dead end.

We are assuming that build systems and libraries are flexible enough to
cater for an alternative installation prefix, and that it will all just
work at runtime. Unfortunately, this has proven not to be the case. And
I think the amount of coordination and work that'd be required to
provide solid /opt support in Ubuntu would be best put in other parts of
the spec, such as its central one: sandboxing.

Just to illustrate the kind of issues we've bumped into in MyApps
submissions, here's just an example [1]: GtkBuilder does not work with
the gettext Python library if you specify an alternative location to
load translations from (e.g.
'/opt/extras.ubuntu.com/$APP/share/locale/'). So we had to either wait
or contribute to fix the upstream bug (unlikely as per the complexity
involved) or implement a workaround. The workaround was to get Quickly
to modify the source code to use 'locale' instead of 'gettext': an
effective but nasty solution.

And that's been the journey with /opt so far: continually playing catch
with the next exception we have to fix or work around. This does not
sound like a very solid approach or a good experience to provide to app
developers. And again, I think we should rather direct resources where
higher priorities lie.

While I like Emmet's idea of delegating the changes required to work
with /opt to MyApps, this would also mean that the complexity in the
logic behind the server would greatly increase. And in some cases (e.g.
hardcoded paths) we would also need to actually modify the sources to
ensure an app runs.

Summarizing, I think these are the 3 main points of discussion right now:

* Package name conflicts: it seems that we've agreed that a solution for
package name conflicts is to rely on 'extras-' prefixing on the server
side (i.e. MyApps). This seems to be easy enough to do, fit well with
other parts of the spec and would be transparent to app developers.

* File name conflicts: there I would suggest exploring Daniel's proposal
of relying on a conflict checker that works across all archives, so that
before an upload is accepted this service checks for any potential
clashes and informs the uploader to fix the package before they can do
the next submission. The uploader would either be an Ubuntu developer
(through the main archive) or an app developer (through extras, via
MyApps). This would not only benefit the app developer process, but also
fix the existing issue in the regular Ubuntu upload process.

* Namespace ownership: even with conflict checking there is the issue of
who gets to own a particular file name or namespace. E.g. would "Mad
Feathered Creatures" (/usr/bin/birds, from Extras) have priority in
owning the binary's name if it had been submitted before "Jolly Flying
Animals" (also /usr/bin/birds, from Universe)? I think if we want to
make apps first-class citizens, even if not part of the distro, a simple
suggestion would just be to do it on a first-come-first-serve basis.

What are your thoughts on these?

Finally, I believe we do need to provision for those cases, but my
understanding is that the potential amount of occurrences would be
rather low. So do you think they justify additional Engineering work, or
rather they could be dealt manually on a case-by-case basis?

Cheers,
David.

[1]
http://askubuntu.com/questions/140552/how-to-make-glade-load-translations-from-opt

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 551 bytes
Desc: OpenPGP digital signature
URL: <https://lists.ubuntu.com/archives/ubuntu-devel/attachments/20120905/5649ccd5/attachment.pgp>