Handling active dependencies in Go

Tue Dec 18 12:45:05 UTC 2012

Hi all,

I'm personally -1 on A. "go get" is working pretty well for our needs so
far, and the issue being debated here is well known in the wild too. We're
not a special case there, and there are conventions that handle it fine, as
we debated live in the meeting last week.

John's suggestion in the original mail sound fine to solve the reported
issue.

With that said, we may certainly have tools that facilitate our workflow.
We already have lbox, for example. It addresses specific issues uncovered
by the existing tooling we have, though, and don't invalidate it.
 On Dec 18, 2012 4:42 AM, "David Cheney" <david.cheney at canonical.com> wrote:

> I vote for A. While I don't say this publicly, the go get tool is a toy.
> It's a good start, but we've clearly exceeded its capacity.
>
>
> On Monday, 17 December 2012 at 9:03 PM, John Arbash Meinel wrote:
>
> > -----BEGIN PGP SIGNED MESSAGE-----
> > Hash: SHA1
> >
> > The primary underlying issue we are trying to solve is that we want to
> > make sure that juju-core's trunk branch always builds and passes its
> > test suite. So that people have a known-working reference when
> > developing their own code.
> >
> > The secondary issue is that the main tool for downloading building and
> > installing go code is the "go get" tool. Which generally translates
> > import paths like: import "launchpad.net/goose (
> http://launchpad.net/goose)" into "bzr branch
> > lp:goose $GOPATH/src/launchpad.net/goose (http://launchpad.net/goose)"
> and installs the result.
> >
> >
> > This is a thread on golang-nuts about this:
> > https://groups.google.com/d/topic/golang-nuts/0avuiWURSQk/discussion
> >
> >
> > It seems the current best practice is to make sure your 'trunk/master'
> > branch is always backwards compatible. And if you want to introduce
> > API breakage, then you should switch to a versioned import url. (For
> > example, you: import "labix.org/v2/mgo (http://labix.org/v2/mgo)")
> >
> >
> > I'd like to bring up some limitations and possible solutions, and at
> > present what we've come up with as a medium term solution.
> >
> > 1. Using 'go get' means that any dependency can trivially accidentally
> > break 'go get launchpad.net/juju-core' (http://launchpad.net/juju-core').
> If goyaml/gocheck/goose/mgo
> > commit a new revision on trunk they risk breaking juju-core's test
> > suite.
> >
> > 2. It would be nice if we break current trunk, that we can roll back to
> > a previous version, and expect tests to pass. At present, there is
> > no record of what version of dependencies was used when a given
> > revision was committed on trunk (assuming we always have a green
> > test suite before a new revision is committed).
> >
> > However, rolling back trunk implies rolling back dependencies if
> > there was any API changes made.
> >
> > 3. At present it is desirable that "lp:goose" have a fair amount of
> > flexbility wrt its API stability.
> > Eg, we recently had a discussion about whether functions should
> > return structs or pointers to structs. Once that is settled, it
> > would be good to update most APIs to be consistent with that.
> >
> >
> > Some possible solutions:
> >
> > A. Don't use 'go get'. Have a 3rd-party tool that works more like
> > "buildout". Where you specify both what the dependency is, and what
> > exact version of it to use (versions.cfg).
> > This can be as trivial as a meta-branch that just has a "Makefile"
> > with a list commands like:
> > bzr branch -rXXX lp:FOO $GOPATH/src/launchpad.net/FOO (
> http://launchpad.net/FOO)
> >
> > In this case, dependencies are pinned, and you have to land a
> > specific commit to update a dependency to a new version.
> >
> > B. Put version numbers in your import statements, and publicly version
> > your projects APIs.
> >
> > Eg, instead of doing:
> > import "launchpad.net/goose/identity (
> http://launchpad.net/goose/identity)"
> >
> > You do:
> > import "launchpad.net/~gophers/goose/v1/identity (
> http://launchpad.net/~gophers/goose/v1/identity)"
> >
> > The one caveat here, is that the code inside "lp:~gophers/goose/v1"
> > *also* has to reference other bits of the project using the same
> > identifier.
> >
> > So if you wanted to have a fluid 'trunk' branch, it will involve a
> > bit of rewriting whenever you want to roll out your trunk branch to
> > whatever is the current stable branch.
> >
> > For reference, this is what that looks like in lp:goose's case:
> > https://code.launchpad.net/~gophers/goose/unstable-001/+merge/140140
> >
> > This doesn't insulate juju-core from dependencies breaking the
> > build, but it does allow juju-core to say "bad dependency, if you
> > want to break the API, you need to roll out a new URL."
> >
> > C. Lock-step deployment. We wait on landing a patch to a dependency,
> > until we have an approved patch on juju-core that uses the new API.
> >
> > So we don't guarantee that any revision of trunk will build an work,
> > but we do guarantee that "go get launchpad.net/juju-core/ (
> http://launchpad.net/juju-core/)..." at the
> > current tip will work. (Note that this means that libraries should
> > run the test suite of their dependencies before landing a trunk
> > commit, so as to not accidentally break them.)
> >
> > D. Slightly removed from (C) is to use a separate integration branch
> > (which is effectively trunk, but not actually associated with
> > 'lp:FOO'). And then wait on updating trunk until we know the test
> > suite passes, or we have a patch landing that makes it patch with
> > the updated code.
> >
> >
> >
> > My personal opinions:
> >
> > (A) is tempting, because then things work more like they have worked
> > elsewhere. The main caveat is having yet-another tool to maintain.
> > (Though possibly this is something like poking at config-manager,
> > rather than writing something from scratch.) The primary downside is
> > you don't guarantee 'go get' just works, and it requires changes to
> > how the 'juju' team does their work.
> >
> > (B) Has the very nice property of atomic updates to juju-core. (which
> > A also has). It lets you rollback trunk, and still get things working.
> > However, it adds a *lot* of process overhead to libraries.
> >
> > The actual diff to juju-core isn't terrible:
> >
> >
> https://code.launchpad.net/~jameinel/juju-core/goose-unstable-001/+merge/140141
> > But the diff inside lp:goose every time we make an unstable bump is at
> > the least ugly:
> > https://code.launchpad.net/~gophers/goose/unstable-001/+merge/140140
> >
> > I haven't quite sorted out how much of that could be scripted. Could
> > we still just develop on trunk, and script pulling trunk changes into
> > a 'current stable' API branch. Do the rewrite, run juju-core test
> > suite, if it passes, update the branch. Else set up a new stable
> > branch, put the code there.
> >
> > It also potentially leaves behind a lot of 'unstable-XXX' branches. It
> > also means there is a bit of a disconnect of what branch you develop
> > on, vs what is being used.
> >
> > It seems like a fair amount of overhead to have every dependency that
> > is undergoing active development follow this layout, though.
> >
> > I suppose if you went into a bi-weekly minor release cycle (0.0.1,
> > 0.0.2, etc) that might work out as a reasonable tradeoff of effort vs
> > benefit. You still are very tempted to re-use a branch if the API
> > hasn't actually changed, in which case you still lose the ability to
> > roll back to an exact state.
> >
> > To get going, I'm looking to do (D: lock-step staged deployment) with
> > an end goal of getting to (B: versioned stable APIs)
> >
> > John
> > =:->
> > -----BEGIN PGP SIGNATURE-----
> > Version: GnuPG v1.4.12 (Cygwin)
> > Comment: Using GnuPG with undefined - http://www.enigmail.net/
> >
> > iEYEARECAAYFAlDO7goACgkQJdeBCYSNAAN14wCfe15DJEXgIwnJyXHX4zNgKdTH
> > V94An09EeRpLogCw4Kg0Qmb82mLx8Np3
> > =pkbY
> > -----END PGP SIGNATURE-----
> >
> > --
> > Juju-dev mailing list
> > Juju-dev at lists.ubuntu.com (mailto:Juju-dev at lists.ubuntu.com)
> > Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju-dev
>
>
>
>
> --
> Juju-dev mailing list
> Juju-dev at lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/juju-dev/attachments/20121218/d1bf64be/attachment.html>