Juju's dependencies, juju/utils and a road forward

[roger peppe]

I agree wholeheartedly about splitting up the utils package.

We talked about this potential issue when it was initially created
(and named "trivial", for trivial functionality with no better place
and no dependencies).

The dependency graph is crucial - one of the main reasons that utils
is problematic is because it has so many external dependencies.

> I'd like to move juju to only use "stable" dependency branches. No more
> '-unstable'.

It's perhaps worth observing that "unstable" is the default for any
package that's not under gopkg.in. At current count there are 99 such
packages in juju's dependencies, all potentially unstable, so I'm not
sure it's worth picking out the
explicit gopkg.in unstable paths for blame here.

Of course, this depends on how careful package authors are to maintain
backward compatibility. Some are pretty good, some not so much.

That said, I agree that using explicitly stable dependencies is a
worthwhile goal. In particular, the ongoing move towards using
bakery.v2-unstable is probably not a great direction, because that
uses a particular commit that just happens to be the one that the
charm store currently uses. Given that the charmstore dependency is
only there for some tests, I'd suggest that the effort might be better
expended in changing those tests so that they don't need to import the
charmstore code directly (for example by writing a fake charmstore
server, or making the tests talk to an external charmstore server -
or, ideally, both). Then it becomes more feasible to make the Juju
code use bakery.v2.

What I'd really like is for us to fast forward a year so that we can
use the the upcoming vgo dependency stuff, but time travel isn't
currently an option, sadly.

One important thing is to try to commit to API stability as much as
possible. A large number (maybe most) of the issues come from places
where APIs have been changed without regard for backward
compatibility. Using semantic versions to tag releases is also a good
idea in general.



On 4 April 2018 at 01:52, Tim Penhey <tim.penhey at canonical.com> wrote:
> Hi folks,
>
> Juju has a problem with its dependencies. We have been trying to upgrade
> a number of our dependencies recently have have been hitting road
> blocks. Mostly because of changes to common libraries that are
> incompatible with other libraries that we use.
>
> I'm pretty sure that the root of all these problems is the juju/utils
> package. It has become a dumping ground for any common functions that
> packages may want to share. However these common functions are changed
> arbitrarily with only thoughts around how they impact a subset of the
> downstreams of juju/utils.
>
> I think that the only sane way forward is to move all the functionality
> we care about out of juju/utils into more cohesive, smaller, packages
> that have meaningful names and you should be able to determine what they do.
>
> Where the current clashes came from were a few places.
>  1) we wanted to change our mgo dependency
>  2) a new file path utility function was wanted
>  3) the debugstatus package inside utils was updated in a backward
> incompatible way
>
> The root problem here is that the utils package tries to do too many
> things. Any package called "utils" should be a warning, and we should
> have listened to Dave Cheney when he brought this up originally.
>
> I think the way forward is to break up the utils package. I'd like to
> propose that we don't add anything new to the utils package, and only
> deletions are allowed. We create new packages that contain the
> functionality we require, and have them be much more controlled on their
> own dependencies.
>
> Thoughts?
>
> Tim
>
> --
> Juju-dev mailing list
> Juju-dev at lists.ubuntu.com
> Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev