What's the future of Juju?

[Mark Shuttleworth]

On 25/03/15 20:01, Merlijn Sebrechts wrote:
> I'm interested in what the future of Juju is. From the small experience
> I've had with it, it seems like a product with a lot of potential. It fills
> a gap in our project that no other technology can fill. Its biggest
> strength is how relations between services are managed. This is something
> that, to my knowledge, does not exist in any tool today. It enables a very
> modular approach and solves a lot of problems we would have with other
> tools.

Yes, agreed!

> However, I've also seen some things that worry me. Even after three years,
> there are still a lot of bugs in the project. The documentation is lacking,
> especially in the parts of Juju that are the most competitive. The
> community is also very small. The fact that it can still only manage Ubuntu
> servers worries me too. I could go more into detail here, but I don't think
> it is relevant to this question.

I understand what you mean. In part I think this comes from the team
being focused on getting to our key Juju 2.0 milestone, and perhaps not
taking as much time to celebrate the smaller milestones on the way,
especially when they are things that we know need a little more work.
For example, the Windows and CentOS support that has landed is a huge
step, but we know it is still a bit rough and needs more polish. Perhaps
we should be louder about some of those steps to draw in people who
could help provide feedback and patches!


> I'm considering starting a big long-term project on top of Juju. The
> project would be very dependent on Juju, so I don't want to do this if
> there is a chance that Juju will be abandoned in 5 years...

No chance of that.

In my travels now I am glad to say I see a shift in the way people
engage with us about Juju. It used to be "why would you want to compete
with Puppet?" but now folks understand that we do not want to compete
with Puppet, we want to come up a level and enable people to collaborate
and reuse their puppet / chef / bash / python across different clouds
and environments.

That focus on collaboration and reuse is what makes Juju special. In the
long run we will integrate Juju into places like HEAT so you get the
benefit of charms, together with the underlying cloud information about
performance, so that things like autoscaling are magical, but in the
short term it's easy to think the projects compete. But folks are
starting to understand that now, so I don't get asked "why do you not
just do HEAT?".

I also think that it will emerge that there are many levels of
orchestration; Juju will be the BEST for the lowest level of
infrastructure orchestration, but people will use Juju to deploy things
like PAAS which themselves offer a kind of orchestration. So end-users
might use a PAAS, which is like a model or API or orchestration system,
and underneath that, administrators will use Juju for the base level. We
won't try to push Juju into every layer; just like we have kept MAAS and
Juju separate with different interests and different responsibilities,
so now the Chef guys are happy to use MAAS, which is good for us both.


> What can you tell me about the future of Juju? Things I'm interested in:
>
> - Big companies building services on top of Juju

Pretty much the biggest telco's in the world, and many of the companies
that supply them, are writing charms.

Pretty much the biggest investment banks in the world, are writing charms.

Pretty much the biggest media companies in the world, are writing charms.

Pretty much the biggest big data and machine learning companies, are
writing charms.

Now, of course, those companies also do a lot of OTHER things :) so I
can't say they will all move solely to Juju because they naturally will
not. But smart folks are seeing what you see - the model is amazing. And
if we are able to get the next round of model pieces in place - status,
network, disk - then we'll be able to do some really incredible rapid
deployment and service evolution things.

My biggest concern at the moment is that I think it's too hard to add
interfaces to existing charms. I think about adding say a Nagios
interface to an existing charm - that should be really easy, but I think
we haven't optimised the whole charm development process very well, so I
will host the lead charmers at my house next week for a mini sprint so
they can show me what I'm missing and we can discuss how to make it much
better.

Also, I think we need to really socialise the status work, because it's
too easy for charms to fail mysteriously, and that makes Juju look bad.
So automated testing of charms is going to get even more important.

> - Statements of long-term commitment from Canonical

You heard it from me. I'm personally interested in Juju and it has a
future, for sure, under all the scenarios I can influence. I think the
cloud world needs something like Juju that is cross-platform and
cross-cloud, and I find the project personally challenging and
interesting. Looking at the thread, you also heard from folks who are
enjoying working on it :)
 
> - Usage statistics

Best indicator is probably to look at the usage stats on individual
charms you care about.

> - Statements of commitment to support other distro's

Windows and CentOS is a pretty diverse list. RHEL and SUSE will be easy
once CentOS is A-grade, and I expect we'll get contributions of that or
perhaps a Canonical customer will ask us to do that work. We will
certainly accept patches for those or other platforms, being a
cross-platform service oriented architecture modelling system is the
point of the project!

> - .. or else, signs that Juju doesn't have a bright future.

It's been a long road so far, as you noted. My take is that:

 * modelling *real* systems is very hard compared to, say, docker
containers, but much more valuable at the infrastructure level
 * doing it across AWS, OpenStack, Azure, GCE and bare metal means it
takes time for things to get complete
 * writing charms has been too hard - poorly documented, poorly considered

That said, I don't see any other project stepping up at the baseline
infrastructure level (the Docker command systems are interesting for
internal business apps).

For example, the HP Triple-O project for OpenStack deployment just
imploded, because it's hard to get this stuff right. By contrast,
installing OpenStack with Juju and charms got even better this last
three months, and when we have status in the model, it will be pretty
much incredible. Any time you set out to model real systems, it's hard!
And in that world, I think Juju has the best chance of success. That's
why the telco's seem to be growing comfortable with it for their
OpenStack and other network function management.

Mark