DCOS on Juju

Tue Apr 26 13:20:09 UTC 2016

Cool, I'll have a look later today. I have been thinking about the
networking for this as DCOS is pretty picky apparently.
I believe we could use Ubuntu Fan to create a more controllable overlay
network, from which we could predict available IPs. Then we could use vrrp
on that, and avoid sharing a fixed list of IPs.
So we could have a layer for the fan, and build on top.
Still early thinking, will get sharper...

Also I want to understand how we can have GPUs in there but that's a second
step.
On Apr 25, 2016 5:54 PM, "Tom Barber" <tom at analytical-labs.com> wrote:

> Okay this isn't my finest work, barely tested and in need of a lot of work
> to get it production ready, but it works, mostly, so its a start.
>
> https://github.com/buggtb/dcos-interface
> https://github.com/buggtb/dcos-master-charm
> https://github.com/buggtb/dcos-agent-charm
>
> Currently only tested on Wily, should work on Xenial but there are no EC2
> images available. Doesn't work on Trusty or before due to Upstartness.
>
> Only supports 1 master currently, but thats just a config thing. Multiple
> agents is fine.
>
> Also NGINX only listens on the internal IP for a reason I've not yet dug
> out, clearly the main reason being I pass it the internal IP at config
> time, but that is deliberate, but I must be able to tweak it somewhere so
> NGINX listens to the world. Because of that to get the dashboard I've just
> been using SSH  port forwarding to test it eg:  sudo ssh bugg at 52.30.59.98
> -L 80:localhost:80 works just fine.
>
> Smoke tested it 10 minute ago with 1 master, 1 agent, then added a 2nd
> agent and installed Jenkins from the repo, all came up fine, thats as far
> as it goes currently.
>
> --------------
>
> Director Meteorite.bi - Saiku Analytics Founder
> Tel: +44(0)5603641316
>
> (Thanks to the Saiku community we reached our Kickstart
> <http://kickstarter.com/projects/2117053714/saiku-reporting-interactive-report-designer/>
> goal, but you can always help by sponsoring the project
> <http://www.meteorite.bi/products/saiku/sponsorship>)
>
> On 25 April 2016 at 21:07, Andy Petrella <noootsab at data-fellas.guru>
> wrote:
>
>> yeah this is what rad did
>>
>> On Mon, 25 Apr 2016 10:05 pm Tom Barber, <tom at analytical-labs.com> wrote:
>>
>>> I installed it using the custom installer (
>>> https://dcos.io/docs/1.7/administration/installing/custom/advanced/)
>>> and then reverse engineered the shell script it creates to run the
>>> installation routines into something Juju could cope with.
>>>
>>> That installer downloads some stuff from the web and then mounts a
>>> docker container to distribute the config across the network, so I ditched
>>> that extracted the content and incoporated it into the charm deployment
>>> instead.
>>>
>>> --------------
>>>
>>> Director Meteorite.bi - Saiku Analytics Founder
>>> Tel: +44(0)5603641316
>>>
>>> (Thanks to the Saiku community we reached our Kickstart
>>> <http://kickstarter.com/projects/2117053714/saiku-reporting-interactive-report-designer/>
>>> goal, but you can always help by sponsoring the project
>>> <http://www.meteorite.bi/products/saiku/sponsorship>)
>>>
>>> On 25 April 2016 at 21:01, Samuel Cozannet <
>>> samuel.cozannet at canonical.com> wrote:
>>>
>>>> Sure! More people more fun. it seems they are currently extremely busy
>>>> focusing on Velocity & Infinity, but having guidance from them will help.
>>>>
>>>> Currently the installer they provide does really help if you want to
>>>> deploy from scratch without any tooling, so that would help.
>>>>
>>>> BTW Tom, been looking into the DC/OS docs lately, what was your
>>>> starting point for this? the Vagrant stuff? So far I find it pretty
>>>> obscure, I am probably not looking in the right direction.
>>>>
>>>> ++
>>>> Sam
>>>>
>>>>
>>>> --
>>>> Samuel Cozannet
>>>> Cloud, Big Data and IoT Strategy Team
>>>> Business Development - Cloud and ISV Ecosystem
>>>> Changing the Future of Cloud
>>>> Ubuntu <http://ubuntu.com>  / Canonical UK LTD <http://canonical.com> /
>>>> Juju <https://jujucharms.com>
>>>> samuel.cozannet at canonical.com
>>>> mob: +33 616 702 389
>>>> skype: samnco
>>>> Twitter: @SaMnCo_23
>>>> [image: View Samuel Cozannet's profile on LinkedIn]
>>>> <https://es.linkedin.com/in/scozannet>
>>>>
>>>> On Mon, Apr 25, 2016 at 8:11 PM, Andy Petrella <
>>>> noootsab at data-fellas.guru> wrote:
>>>>
>>>>> well, I can ppl from mesosphere of necessary... I dunno if you are
>>>>> okay with this but that would make a bigger announcement
>>>>>
>>>>> On Mon, 25 Apr 2016 8:09 pm Samuel Cozannet, <
>>>>> samuel.cozannet at canonical.com> wrote:
>>>>>
>>>>>> Hi Tom,
>>>>>>
>>>>>> Adding also Artyom from DataArt as initial writers of the Mesos
>>>>>> charms. Artyom, it looks like we have a PoC for DC/OS charms. Let's
>>>>>> coordinate also with your team to see how we can get that further.
>>>>>>
>>>>>> @Tom: No worries. Whenever you're ready send us links to the layers,
>>>>>> and we'll have a look.
>>>>>>
>>>>>> So I have created this repo :
>>>>>> https://github.com/SaMnCo/layers-dcos.git
>>>>>> so we can have all the code for all layers involved in one place.
>>>>>> Empty for now.
>>>>>>
>>>>>> I've added a gitter here : https://gitter.im/SaMnCo/layers-dcos
>>>>>>
>>>>>> Everyone interested please join this channel and let's move the
>>>>>> conversation there.
>>>>>>
>>>>>> ++
>>>>>> Sam
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Samuel Cozannet
>>>>>> Cloud, Big Data and IoT Strategy Team
>>>>>> Business Development - Cloud and ISV Ecosystem
>>>>>> Changing the Future of Cloud
>>>>>> Ubuntu <http://ubuntu.com>  / Canonical UK LTD <http://canonical.com> /
>>>>>> Juju <https://jujucharms.com>
>>>>>> samuel.cozannet at canonical.com
>>>>>> mob: +33 616 702 389
>>>>>> skype: samnco
>>>>>> Twitter: @SaMnCo_23
>>>>>> [image: View Samuel Cozannet's profile on LinkedIn]
>>>>>> <https://es.linkedin.com/in/scozannet>
>>>>>>
>>>>>> On Mon, Apr 25, 2016 at 2:31 PM, Tom Barber <tom at analytical-labs.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi smart people!
>>>>>>>
>>>>>>> I'll tidy up my initial DCOS dump and publish is over the next day
>>>>>>> or so. As a disclaimer, I'm no MESOS/DCOS expert so I'll defer to you guys
>>>>>>> on pretty much everything! ;)
>>>>>>>
>>>>>>> Tom
>>>>>>>
>>>>>>> --------------
>>>>>>>
>>>>>>> Director Meteorite.bi - Saiku Analytics Founder
>>>>>>> Tel: +44(0)5603641316
>>>>>>>
>>>>>>> (Thanks to the Saiku community we reached our Kickstart
>>>>>>> <http://kickstarter.com/projects/2117053714/saiku-reporting-interactive-report-designer/>
>>>>>>> goal, but you can always help by sponsoring the project
>>>>>>> <http://www.meteorite.bi/products/saiku/sponsorship>)
>>>>>>>
>>>>>>> On 25 April 2016 at 12:31, Andy Petrella <noootsab at data-fellas.guru>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> awesome, my turn to add Rad, who had created a terraform dcos
>>>>>>>> integration to be open sourced soon.
>>>>>>>> in combination with François you'll have all the information
>>>>>>>> necessary
>>>>>>>>
>>>>>>>> On Mon, 25 Apr 2016 13:20 Samuel Cozannet, <
>>>>>>>> samuel.cozannet at canonical.com> wrote:
>>>>>>>>
>>>>>>>>> Adding some smart people working with us on the Mesos stack.
>>>>>>>>>
>>>>>>>>> Andy, Adam, François, have a look at this DCOS charms.
>>>>>>>>> Tom, look at the project in my github (samnco/ods-dl)
>>>>>>>>>
>>>>>>>>> Let's open a gutter room later this week and see how we can
>>>>>>>>> collaborate on this
>>>>>>>>> On Apr 25, 2016 03:03, "Tom Barber" <tom at analytical-labs.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> GPU Mesos on Power, that sounds cool!
>>>>>>>>>>
>>>>>>>>>> I did originally just stand up Mesos again, but DCOS is in the
>>>>>>>>>> news so I figured, why not.
>>>>>>>>>>
>>>>>>>>>> Okay sounds like a plan. Tomorrow is a bit rubbish, I'm around
>>>>>>>>>> today and Wednesday though, just ping me off list with a time.
>>>>>>>>>>
>>>>>>>>>> Tom
>>>>>>>>>>
>>>>>>>>>> --------------
>>>>>>>>>>
>>>>>>>>>> Director Meteorite.bi - Saiku Analytics Founder
>>>>>>>>>> Tel: +44(0)5603641316
>>>>>>>>>>
>>>>>>>>>> (Thanks to the Saiku community we reached our Kickstart
>>>>>>>>>> <http://kickstarter.com/projects/2117053714/saiku-reporting-interactive-report-designer/>
>>>>>>>>>> goal, but you can always help by sponsoring the project
>>>>>>>>>> <http://www.meteorite.bi/products/saiku/sponsorship>)
>>>>>>>>>>
>>>>>>>>>> On 25 April 2016 at 01:06, Samuel Cozannet <
>>>>>>>>>> samuel.cozannet at canonical.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Tom,
>>>>>>>>>>>
>>>>>>>>>>> I've been working with a bunch of people on this as well for GPU
>>>>>>>>>>> enabled Mesos on Power. We're also doing more regarding DCOS Open Source.
>>>>>>>>>>>
>>>>>>>>>>> Let's connect and optimize who works on what. I am at OpenStack
>>>>>>>>>>> summit in Austin this week, but can do a call early in my mornings. Pick
>>>>>>>>>>> your time and day.
>>>>>>>>>>>
>>>>>>>>>>> ++
>>>>>>>>>>> Sam
>>>>>>>>>>> On Apr 24, 2016 5:00 PM, "Tom Barber" <tom at analytical-labs.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Figured this was worth distributing to try and prevent
>>>>>>>>>>>> duplicated effort etc:
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> 
>>>>>>>>>>>> Not ready for distribution yet, but I've stood up DCOS
>>>>>>>>>>>> orchestration as part of my ApacheCon demo (as its based on MESOS). Deploys
>>>>>>>>>>>> master and slave nodes to EC2 and works a treat(from a novice perspective).
>>>>>>>>>>>>
>>>>>>>>>>>> If anyone is interested in collaboration as my bandwidth is
>>>>>>>>>>>> pretty much maxed at the mo, let me know and we'll sort this stuff out.
>>>>>>>>>>>>
>>>>>>>>>>>> Tom
>>>>>>>>>>>> --------------
>>>>>>>>>>>>
>>>>>>>>>>>> Director Meteorite.bi - Saiku Analytics Founder
>>>>>>>>>>>> Tel: +44(0)5603641316
>>>>>>>>>>>>
>>>>>>>>>>>> (Thanks to the Saiku community we reached our Kickstart
>>>>>>>>>>>> <http://kickstarter.com/projects/2117053714/saiku-reporting-interactive-report-designer/>
>>>>>>>>>>>> goal, but you can always help by sponsoring the project
>>>>>>>>>>>> <http://www.meteorite.bi/products/saiku/sponsorship>)
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> Juju mailing list
>>>>>>>>>>>> Juju at lists.ubuntu.com
>>>>>>>>>>>> Modify settings or unsubscribe at:
>>>>>>>>>>>> https://lists.ubuntu.com/mailman/listinfo/juju
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>
>>>>>>
>>>>
>>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/juju/attachments/20160426/f3bf0c3c/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Untitled.png
Type: image/png
Size: 97190 bytes
Desc: not available
URL: <https://lists.ubuntu.com/archives/juju/attachments/20160426/f3bf0c3c/attachment.png>