Docker compose in Juju

Tue Jun 7 09:17:38 UTC 2016

Sort of yeah.
You'll have to manage collocation of several charms on the units to run
several containers on the same instance. The Docker layer will take care of
avoiding to replicate install several times though, so the second charm you
deploy will essentially be docker pull / docker run. You can actually force
this behavior in the layer options (do not install docker)
I haven't played that much with the latest versions of the Docker layer, so
I'll let Chuck pick up this one, but some stuff I think you'll need to take
into account:

* Ports / networking: containers usually expose a network port. If you run
several colocated Docker charms, you will have to manage yourself the fact
the ports don't overlap. That means your install section will also have to
be clever about the ports it opens in the cloud fw.
* Relations: app containers are supposed to be orchestrated. By themselves,
they know nothing about the state of the rest of the world. Also, in a Juju
bundle, you don't have a warranty on the order in which they will spin. So
you will have to create a relation logic that reacts to the availability of
other components to, for example, reload a docker image with the new set of
env variables, or just make sure such container waits for others before it
runs. Essentially, you'll have to recreate the notion of dependency you may
have experienced with Fleet for example.
* Scale out:
  * Juju so far doesn't have a notion of ELB or other similar cloud level
load balancer. So if you run your containers in a cluster, and spawn
several replicas, that won't be managed by Juju either, you'll have to do
the load balancing yourself
  * The way Juju scales out today is by adding units/instances. In the case
of Docker containers, this can be tricky, as you are actually spinning more
containers, but in a limited cluster. You don't necessarily want to scale
that cluster. So my recommendation is not to use the default juju add-unit,
but rather add a config option in the charm to scale out the container,
which you'll trigger via "juju set config-option"
* Storage / resiliency: Not sure if we have a story to integrate storage
resiliency yet. If you need to use stateful apps, I'd recommend using a
classic charm and use Juju's abstractions for storage. Again, I haven't
used the latest Docker stuff in Juju, maybe Chuck has better insight.

I'd personally be interested in your scale out usage, and if you usually
consume helpers like Flocker, or Weave for the storage and network layers.
Any info would be appreciated.

++
Sam

--
Samuel Cozannet
Cloud, Big Data and IoT Strategy Team
Business Development - Cloud and ISV Ecosystem
Changing the Future of Cloud
Ubuntu <http://ubuntu.com>  / Canonical UK LTD <http://canonical.com> / Juju
<https://jujucharms.com>
samuel.cozannet at canonical.com
mob: +33 616 702 389
skype: samnco
Twitter: @SaMnCo_23
[image: View Samuel Cozannet's profile on LinkedIn]
<https://es.linkedin.com/in/scozannet>

On Tue, Jun 7, 2016 at 7:30 AM, Gayan Gunarathne <csegayan at gmail.com> wrote:

> Hi Samuel,
>
> Thank you very much for the details.
>
> As you mention if we wrap the Docker image in the Charm, it is just like a
> single instance.Isn't? It just like that I run this docker containers in a
> single instance in IaaS like EC2, GCE etc. Within this instance it will
> spawn the containers. So for my deployment I need to manager it within the
> docker image in the Charm.Isn't?
>
> Thanks,
> Gayan
>
>
>
>
>
> On Mon, Jun 6, 2016 at 7:39 PM, Samuel Cozannet <
> samuel.cozannet at canonical.com> wrote:
>
>> Hi Gayan,
>>
>> Juju cannot operate with app containers as it operates with "os-based"
>> systems.
>> The difference is that Juju expects the node it spins to run a Juju
>> agent. In the case of a Docker container, which runs a single process, that
>> would mean the sole purpose of the container is running... Juju itself. So
>> long for your app.
>> So that means Juju cannot provision Docker containers "directly" (which
>> is what you seem to expect with the provider idea).
>>
>> If you want to run Docker containers via Juju, you have the following
>> methods:
>> * Wrap the Docker image in the charm. Essentially, the charm would then
>> be a set of commands to run, scale, integrate... the container with the
>> rest of the world (for example, managing the fact that the host port is not
>> necessarily the same as the container port, or managing storage for the
>> image and so on and so on)
>> In this context, you would run docker images, under the form of a docker
>> compose, on a single host.
>> * Wrap the docker-compose in the Swarm layer: here you would do the same
>> as above, but Juju would run at the host level of your Swarm cluster.
>> * Create a specific charm to interact with the cluster API (k8s, Mesos,
>> Swarm): this charm would be your deployer. It could receive hooks from the
>> outside world, triggering deployment or other stuff. In this case, the
>> charm can be a subordinate (running on all cluster instances), which you
>> could potentially load balance, or a standalone service running on the
>> side.
>>
>> Now, if you are looking into building a container based IaaS equivalent,
>> I suggest you look into LXD
>> <https://linuxcontainers.org/lxd/getting-started-cli/>, which would spin
>> containers that look and behave like VMs.
>> From a Juju perspective, there is a LXD provider.
>> The diff with Docker is that LXD doesn't natively provide image based
>> deployment, but follows the Juju philosophy to always start with a blank
>> image.
>> So your charms will have to deploy the code in LXD from scratch.
>>
>> Additional nodes: in the docs, pick the "/devel" branch, it's better if
>> you use Juju 2.x
>>
>> I hope this helps.
>> ++
>> Sam
>>
>>
>>
>>
>> --
>> Samuel Cozannet
>> Cloud, Big Data and IoT Strategy Team
>> Business Development - Cloud and ISV Ecosystem
>> Changing the Future of Cloud
>> Ubuntu <http://ubuntu.com>  / Canonical UK LTD <http://canonical.com> /
>> Juju <https://jujucharms.com>
>> samuel.cozannet at canonical.com
>> mob: +33 616 702 389
>> skype: samnco
>> Twitter: @SaMnCo_23
>> [image: View Samuel Cozannet's profile on LinkedIn]
>> <https://es.linkedin.com/in/scozannet>
>>
>> On Mon, Jun 6, 2016 at 2:36 PM, Gayan Gunarathne <csegayan at gmail.com>
>> wrote:
>>
>>>
>>> Hi Charles/Samuel,
>>>
>>> Thanks for the information.
>>>
>>> What I am try to do is to run a docker compose to start docker
>>> containers in a single machine. Still I am not clear how we can use the
>>> docker with Juju.I am following this[1]
>>>
>>> Simple I need to use the Juju UI[2] and create my deployment there.Then
>>> can I do the deployment with the docker.How can I select  the docker as a
>>> IaaS. I couldn't find the docker configuration section in
>>> environment.yaml.I am not clear how can I provide the docker as a cloud
>>> provider.
>>>
>>> [1]https://jujucharms.com/docs/stable/getting-started
>>> [2]https://demo.jujucharms.com/
>>>
>>> Thanks,
>>> Gayan
>>>
>>> On Wed, Jun 1, 2016 at 3:08 AM, Samuel Cozannet <
>>> samuel.cozannet at canonical.com> wrote:
>>>
>>>> And finally to add to this, there is a community effort around Mesos
>>>> and DCOS to provide the same idea as the k8s stuff.
>>>>
>>>> I am also interested in your use case and feedback :)
>>>>
>>>> Best,
>>>> Sam
>>>> On May 31, 2016 7:17 PM, "Charles Butler" <charles.butler at canonical.com>
>>>> wrote:
>>>>
>>>>> Greetings Gayan!
>>>>>
>>>>> You most certainly can compose and use a full docker-provided stack,
>>>>> mix and match with charms - thats the power of wrapping your compose-based
>>>>> service with layer-docker.
>>>>>
>>>>> https://github.com/juju-solutions/layer-docker
>>>>>
>>>>> Which is a great starting place for charming up your dockerized app
>>>>> using charms.docker to lend a hand with the ops knowledge to get it running
>>>>> :)
>>>>>
>>>>> https://github.com/juju-solutions/charms.docker
>>>>> http://pythonhosted.org/charms.docker/modules.html
>>>>>
>>>>> There are several examples in the charm store from myself (lazypower)
>>>>> and Matt Bruzek (mbruzek)  that are docker based. I believe the most
>>>>> straight forward example that I can illustrate today is the swarm layer,
>>>>> which builds the swarm charm.
>>>>>
>>>>> https://github.com/juju-solutions/layer-swarm
>>>>> https://jujucharms.com/u/containers/swarm-core
>>>>>
>>>>> This is a multi-series charm that delivers swarm via containers,
>>>>> backended by consul or etcd as the discovery mechanism. This has a good mix
>>>>> of relationships, base layers, top-layers (what you would be writing), and
>>>>> uses the docker-native tooling to bring everything up with some help from
>>>>> juju.
>>>>>
>>>>> There are even more complex examples, such as Kubernetes - all being
>>>>> brought up and controlled in a similar manner
>>>>>
>>>>> https://github.com/kubernetes/kubernetes/tree/master/cluster/juju
>>>>> https://jujucharms.com/u/containers/kubernetes-core
>>>>>
>>>>> If you need any help charming with Docker you can get in touch with
>>>>> myself on the mailing list here, or join us on irc in #juju on
>>>>> irc.freenode.net
>>>>>
>>>>> I'd love to hear any feedback/questions/comments about the developer
>>>>> tooling we have here, as its really ramped up our capacity to churn out
>>>>> high quality charms quickly that are docker based, and if there's any rough
>>>>> edges we can sand out for other developers would be great starting points.
>>>>>
>>>>> All the best,
>>>>>
>>>>> Charles
>>>>>
>>>>> On Tue, May 31, 2016 at 9:41 AM Gayan Gunarathne <csegayan at gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi Marco,
>>>>>>
>>>>>> Thanks for the details.
>>>>>>
>>>>>> Actually I just want to do something like this. Lets say I have
>>>>>> tomcat and mysql composite application. So I need to deploy these two
>>>>>> application in docker with depends on(I am really glad if I can use demo
>>>>>> UI[1]). How can I do that?
>>>>>>
>>>>>> Do you already have charms that run in docker with the same? Can you
>>>>>> point me to some sample?
>>>>>>
>>>>>> [1]https://demo.jujucharms.com/
>>>>>>
>>>>>> Thanks,
>>>>>> Gayan
>>>>>>
>>>>>> On Tue, May 31, 2016 at 6:52 PM, Marco Ceppi <
>>>>>> marco.ceppi at canonical.com> wrote:
>>>>>>
>>>>>>> Hi Gayan,
>>>>>>>
>>>>>>> I've added the general Juju list which covers more of these general
>>>>>>> topics.
>>>>>>>
>>>>>>> So, because of the nature of LXC machines and Docker style
>>>>>>> application containers it's hard to model that style application container
>>>>>>> in Juju in the same way LXC machines work. However, it's quite easy to wrap
>>>>>>> something like a Docker container, which works really well as a
>>>>>>> payload/software delivery tool, but then you can use Juju to wrap that
>>>>>>> immutable object and make it mutable inside of a Juju deployment.
>>>>>>>
>>>>>>> I know there are quite a few people on the juju mailing list doing
>>>>>>> this today, so I'll let them weigh in. In short, yes you can use Docker and
>>>>>>> docker style application containers with Juju, but not in the same direct
>>>>>>> way you would a LXC machine just because of the differences in function and
>>>>>>> form.
>>>>>>>
>>>>>>> Marco
>>>>>>>
>>>>>>> On Tue, May 31, 2016 at 7:09 AM Gayan Gunarathne <csegayan at gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hello,
>>>>>>>>
>>>>>>>> Can we run docker directly with Juju? I saw Juju is supporting the
>>>>>>>> LXC containers. I need to know whether we can spawn docker containers as
>>>>>>>> the same.
>>>>>>>>
>>>>>>>> If we support this can you point me to any document?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Gayan
>>>>>>>> --
>>>>>>>> Juju-dev mailing list
>>>>>>>> Juju-dev at lists.ubuntu.com
>>>>>>>> Modify settings or unsubscribe at:
>>>>>>>> https://lists.ubuntu.com/mailman/listinfo/juju-dev
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Best Regards,
>>>>>> Gayan
>>>>>> --
>>>>>> Juju-dev mailing list
>>>>>> Juju-dev at lists.ubuntu.com
>>>>>> Modify settings or unsubscribe at:
>>>>>> https://lists.ubuntu.com/mailman/listinfo/juju-dev
>>>>>>
>>>>> --
>>>>> Juju Charmer
>>>>> Canonical Group Ltd.
>>>>> Ubuntu - Linux for human beings | www.ubuntu.com
>>>>> Juju - The fastest way to model your service | www.jujucharms.com
>>>>>
>>>>> --
>>>>> Juju mailing list
>>>>> Juju at lists.ubuntu.com
>>>>> Modify settings or unsubscribe at:
>>>>> https://lists.ubuntu.com/mailman/listinfo/juju
>>>>>
>>>>>
>>>
>>>
>>> --
>>> Best Regards,
>>> Gayan
>>>
>>
>>
>
>
> --
> Best Regards,
> Gayan
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/juju/attachments/20160607/43cc0fbb/attachment-0001.html>