Policy proposal: All new charms must include tests.

[Jorge O. Castro]

On Fri, Dec 13, 2013 at 9:20 AM, Vincent JOBARD <vinzjobard at gmail.com> wrote:
>> This way, we let people get charms in that work-for-them-and-others, but
>> the limitations are reflected in our rating of the charm.
>
>
> Well with only the 2 stars rating, users will not know that the charm is
> correctly tested. I mean an other charms that doesn't include test, but
> which be loved by people can have a two stars. Maybe an "approuve" mark or a
> trusted label can be used ?

We have the equivalent of the "approve" mark, it's the charms that say
"Recommended by the Juju team" and have full icons and can be deployed
from the store as the default `juju deploy mysql`. This policy would
apply to those charms only. I think that if we're going to recommend
things that we should ensure that what we give people is well tested.

An errant commit or simple MP can break a charm subtlety (and this has
happened) that a reviewer might have missed. People need to know that
what they deploy out of the store has been tested; a guy deploying an
entire bundle should be spending his time on his application, not
worrying about whether haproxy is going to work or not. Also, personal
namespaces are always available. In the case of the popular charms
that work for us and others, that would be reflected in the stats, and
we'd notice popular personal charms and that would lead us to ask the
maintainers about what they need to get tests in, is there anyway we
can help them, and so on. The onus is on us to make adding tests easy
for charm authors.

I also think that it's much easier to start strict now, and possibly
loosen up over time depending on what charm authors and contributors
tell us, than it is to start loose and try to tighten up after the
fact.

-- 
Jorge Castro
Canonical Ltd.
http://juju.ubuntu.com/ - Automate your Cloud Infrastructure