Juju Stable Update Policy
curtis at canonical.com
Mon Jul 28 16:47:11 UTC 2014
>> From: Martin Pitt <martin.pitt at ubuntu.com>
>> Date: Tue, Jul 22, 2014 at 9:05 AM
>> Subject: Re: Juju Stable Update Policy
>> Alexis Bruemmer [2014-07-08 14:05 -0700]:
>>> > > Juju developers and have a specific QA review, and a variety of fresh
>>> > > install and upgrade combinations must be explicitly tested on every
>>> > > cloud
>>> > > that Juju supports (for details see the “Testing a revision” section
>>> > > of
>>> > the
>>> > > Juju CI Design and Operation document linked below)
>>> > This really seems to be the crux of regression testing as far as SRUs
>>> > and existing cloud deployments are concerned. Whenever I try to open
>>> > the google docs I just get a "File unavailable. Sorry, there's a
>>> > problem with this file. Please reload.", so I can't look at the
>>> > details. Does that mean that newer clients are tested on existing
>>> > deployments with older servers and agents? Which combinations does
>>> > that cover?
>>> I have updated the sharing policies for the Juju Ci doc, let me know if
>>> still have issues accessing the file.
>> I can see it now, but it still doesn't explain how you test backwards
>> compatibility to existing deployments? Or I can't find it.
>> I can see teh google doc now, but it still crashes on the first mouse
>> click. I think for long-term documentation we don't want to bury that
>> in google docs, but perhaps underneath
>> https://wiki.ubuntu.com/StableReleaseUpdates ?
>> So in summary, I have reasonably good confidence in the currently
>> documented CI, but I need some more convincing about backwards
>> compatibility testing.
We don't have automated tests of minor-to-minor-client-to-server yet.
We are not scheduled to start work on this feature for a a few weeks.
We are added infrastructure to Juju CI to support this and we have
done some manual testing to prove how we will automate this.
1. CI will have a store of stable minor jujus: 1.18.4, 1.20.1, 1.21.0...
2. For each new revision of juju,
* For each stable minor juju an env will be bootstrapped.
* The juju client under test will be run through a series of
commands that demonstrate the env can be maintained.
* The test will culminate in a destroy environment
to show the env can reach its EOL.
* An env will be bootstrapped with the new juju
* For each stable minor juju
a series of commands will test that old client can
maintain the new env.
* reports.vapour.ws will have a report showing the all combinations
pass test suite.
3. Only revisions that pass all tests are considered for release,
CI will provide the actual release tarball.
4. CI will gain a mechanism to test a packages built for Ubuntu
(ubuntu's packaging rules) that will test the packages pass all the
tests the release tarball passed. Demonstrating that juju and the ubuntu
packaging are compatible with the versions published in Ubuntu.
Canonical Cloud Development and Operations
More information about the technical-board