Packaging large Java software stacks ?

[Thierry Carrez]

Daniel Holbach wrote:
> Thierry Carrez schrieb:
>> Creating a separate package for each version of each JAR is indeed the
>> most debianic approach, however I'm not sure it's realistic. Glassfish
>> alone needs ~230 separate versions of dependencies to build (of which we
>> have ~30 in the tree currently). Add the dependencies to those
>> dependencies to the mix, and the 1000 mark is probably not very far. And
>> Glassfish has a rather clean dependency approach compared to Geronimo...
> 
> I have a set of questions:
>  - How many of these do we have already?

I would say around 500. 99% of them are providing a single version
(usually a recent one).

>  - How many of these ever change?

Most of them still regularly change.

>  - How many of these break compatibility?

That's a difficult one to answer, since there is no API versioning in
JARs, but apparently a lot of new releases do. The Geronimo developers
told me the slightest version change frequently breaks the TCK JavaEE
compliance tests, and using the latest is usually not the best. To get
an idea of the (runtime) components versioning complexity, see [1]. The
-Gnnnnnn versions means it's a Geronimo-patched version of the JAR that
is used.

[1] http://cwiki.apache.org/GMOxDOC22/component-versions.html

>  - How hard is the dependency? Is this always "== JAR_VERSION" or often
> a ">= JAR_VERSION" too?

Dependencies are "== JAR_VERSION" for Maven-based projects. Ant-based
things are generally friendlier.

Note that the JARs needed at runtime are usually shipped together with
the software in the upstream binary release. They really consider JARs
as pre-written blocks of their own code rather than as external deps...
The Geronimo guys monitor security issues in all components and release
a new version of the stack every time there is a security issue with
some external thing.

>> Yes, we still have to advocate the right way. Note that from Java
>> developers point of view, their way of doing things (including precise
>> external libraries as part of the package) is the right way, and our way
>> (trying to run several things on top of the same pile of libraries) is
>> wrong.
> 
> Maybe there needs to be a "Java Open Source Distribution Interest Group"
> or something. The other distributions will have the same problem like we do.
> 
> Open Source Distributions essentially value and care about:
>  - building from open source
>  - shipping built binaries and being authoritative about what is shipped
> and installed on the users machines
>  - stable interfaces and a clear indication (versioning, tools, etc) of
> changes
>  - <add more items here>

The JPackage project has been trying to do that, provide a sufficiently
common platform to try to influence smaller upstream projects. The
bigger projects (like Geronimo) have much more brittle stacks: they
don't see the point of going through a very complex path while they
already have a hard job assembling their stack without caring about
anything else.

-- 
Thierry Carrez
Ubuntu server team