DCOS on Juju

Tue Apr 26 14:07:00 UTC 2016

Yeah, networking, and scalability stuff are top of the list, also hooking
it up to a Juju provided monitoring service will be required early on, I
would have thought.

A few simple configs I'll add shortly, opt out of oauth security and
disabling telemetry, I'll leave them in their enabled state by default
though so Mesosphere continue to get their stats.

There will also be some work around the loadbalancing and MESOS DNS to make
them configurable/replaceable/extendible.

Up until ApacheCon I'm pretty much out of spare room to look at it, but
I'll certainly be picking it up post ApacheCon.

Tom

--------------

Director Meteorite.bi - Saiku Analytics Founder
Tel: +44(0)5603641316

(Thanks to the Saiku community we reached our Kickstart
<http://kickstarter.com/projects/2117053714/saiku-reporting-interactive-report-designer/>
goal, but you can always help by sponsoring the project
<http://www.meteorite.bi/products/saiku/sponsorship>)

On 26 April 2016 at 14:20, Samuel Cozannet <samuel.cozannet at canonical.com>
wrote:

> Cool, I'll have a look later today. I have been thinking about the
> networking for this as DCOS is pretty picky apparently.
> I believe we could use Ubuntu Fan to create a more controllable overlay
> network, from which we could predict available IPs. Then we could use vrrp
> on that, and avoid sharing a fixed list of IPs.
> So we could have a layer for the fan, and build on top.
> Still early thinking, will get sharper...
>
> Also I want to understand how we can have GPUs in there but that's a
> second step.
> On Apr 25, 2016 5:54 PM, "Tom Barber" <tom at analytical-labs.com> wrote:
>
>> Okay this isn't my finest work, barely tested and in need of a lot of
>> work to get it production ready, but it works, mostly, so its a start.
>>
>> https://github.com/buggtb/dcos-interface
>> https://github.com/buggtb/dcos-master-charm
>> https://github.com/buggtb/dcos-agent-charm
>>
>> Currently only tested on Wily, should work on Xenial but there are no EC2
>> images available. Doesn't work on Trusty or before due to Upstartness.
>>
>> Only supports 1 master currently, but thats just a config thing. Multiple
>> agents is fine.
>>
>> Also NGINX only listens on the internal IP for a reason I've not yet dug
>> out, clearly the main reason being I pass it the internal IP at config
>> time, but that is deliberate, but I must be able to tweak it somewhere so
>> NGINX listens to the world. Because of that to get the dashboard I've just
>> been using SSH  port forwarding to test it eg:  sudo ssh bugg at 52.30.59.98
>> -L 80:localhost:80 works just fine.
>>
>> Smoke tested it 10 minute ago with 1 master, 1 agent, then added a 2nd
>> agent and installed Jenkins from the repo, all came up fine, thats as far
>> as it goes currently.
>>
>> --------------
>>
>> Director Meteorite.bi - Saiku Analytics Founder
>> Tel: +44(0)5603641316
>>
>> (Thanks to the Saiku community we reached our Kickstart
>> <http://kickstarter.com/projects/2117053714/saiku-reporting-interactive-report-designer/>
>> goal, but you can always help by sponsoring the project
>> <http://www.meteorite.bi/products/saiku/sponsorship>)
>>
>> On 25 April 2016 at 21:07, Andy Petrella <noootsab at data-fellas.guru>
>> wrote:
>>
>>> yeah this is what rad did
>>>
>>> On Mon, 25 Apr 2016 10:05 pm Tom Barber, <tom at analytical-labs.com>
>>> wrote:
>>>
>>>> I installed it using the custom installer (
>>>> https://dcos.io/docs/1.7/administration/installing/custom/advanced/)
>>>> and then reverse engineered the shell script it creates to run the
>>>> installation routines into something Juju could cope with.
>>>>
>>>> That installer downloads some stuff from the web and then mounts a
>>>> docker container to distribute the config across the network, so I ditched
>>>> that extracted the content and incoporated it into the charm deployment
>>>> instead.
>>>>
>>>> --------------
>>>>
>>>> Director Meteorite.bi - Saiku Analytics Founder
>>>> Tel: +44(0)5603641316
>>>>
>>>> (Thanks to the Saiku community we reached our Kickstart
>>>> <http://kickstarter.com/projects/2117053714/saiku-reporting-interactive-report-designer/>
>>>> goal, but you can always help by sponsoring the project
>>>> <http://www.meteorite.bi/products/saiku/sponsorship>)
>>>>
>>>> On 25 April 2016 at 21:01, Samuel Cozannet <
>>>> samuel.cozannet at canonical.com> wrote:
>>>>
>>>>> Sure! More people more fun. it seems they are currently extremely busy
>>>>> focusing on Velocity & Infinity, but having guidance from them will help.
>>>>>
>>>>> Currently the installer they provide does really help if you want to
>>>>> deploy from scratch without any tooling, so that would help.
>>>>>
>>>>> BTW Tom, been looking into the DC/OS docs lately, what was your
>>>>> starting point for this? the Vagrant stuff? So far I find it pretty
>>>>> obscure, I am probably not looking in the right direction.
>>>>>
>>>>> ++
>>>>> Sam
>>>>>
>>>>>
>>>>> --
>>>>> Samuel Cozannet
>>>>> Cloud, Big Data and IoT Strategy Team
>>>>> Business Development - Cloud and ISV Ecosystem
>>>>> Changing the Future of Cloud
>>>>> Ubuntu <http://ubuntu.com>  / Canonical UK LTD <http://canonical.com> /
>>>>> Juju <https://jujucharms.com>
>>>>> samuel.cozannet at canonical.com
>>>>> mob: +33 616 702 389
>>>>> skype: samnco
>>>>> Twitter: @SaMnCo_23
>>>>> [image: View Samuel Cozannet's profile on LinkedIn]
>>>>> <https://es.linkedin.com/in/scozannet>
>>>>>
>>>>> On Mon, Apr 25, 2016 at 8:11 PM, Andy Petrella <
>>>>> noootsab at data-fellas.guru> wrote:
>>>>>
>>>>>> well, I can ppl from mesosphere of necessary... I dunno if you are
>>>>>> okay with this but that would make a bigger announcement
>>>>>>
>>>>>> On Mon, 25 Apr 2016 8:09 pm Samuel Cozannet, <
>>>>>> samuel.cozannet at canonical.com> wrote:
>>>>>>
>>>>>>> Hi Tom,
>>>>>>>
>>>>>>> Adding also Artyom from DataArt as initial writers of the Mesos
>>>>>>> charms. Artyom, it looks like we have a PoC for DC/OS charms. Let's
>>>>>>> coordinate also with your team to see how we can get that further.
>>>>>>>
>>>>>>> @Tom: No worries. Whenever you're ready send us links to the layers,
>>>>>>> and we'll have a look.
>>>>>>>
>>>>>>> So I have created this repo :
>>>>>>> https://github.com/SaMnCo/layers-dcos.git
>>>>>>> so we can have all the code for all layers involved in one place.
>>>>>>> Empty for now.
>>>>>>>
>>>>>>> I've added a gitter here : https://gitter.im/SaMnCo/layers-dcos
>>>>>>>
>>>>>>> Everyone interested please join this channel and let's move the
>>>>>>> conversation there.
>>>>>>>
>>>>>>> ++
>>>>>>> Sam
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Samuel Cozannet
>>>>>>> Cloud, Big Data and IoT Strategy Team
>>>>>>> Business Development - Cloud and ISV Ecosystem
>>>>>>> Changing the Future of Cloud
>>>>>>> Ubuntu <http://ubuntu.com>  / Canonical UK LTD
>>>>>>> <http://canonical.com> / Juju <https://jujucharms.com>
>>>>>>> samuel.cozannet at canonical.com
>>>>>>> mob: +33 616 702 389
>>>>>>> skype: samnco
>>>>>>> Twitter: @SaMnCo_23
>>>>>>> [image: View Samuel Cozannet's profile on LinkedIn]
>>>>>>> <https://es.linkedin.com/in/scozannet>
>>>>>>>
>>>>>>> On Mon, Apr 25, 2016 at 2:31 PM, Tom Barber <tom at analytical-labs.com
>>>>>>> > wrote:
>>>>>>>
>>>>>>>> Hi smart people!
>>>>>>>>
>>>>>>>> I'll tidy up my initial DCOS dump and publish is over the next day
>>>>>>>> or so. As a disclaimer, I'm no MESOS/DCOS expert so I'll defer to you guys
>>>>>>>> on pretty much everything! ;)
>>>>>>>>
>>>>>>>> Tom
>>>>>>>>
>>>>>>>> --------------
>>>>>>>>
>>>>>>>> Director Meteorite.bi - Saiku Analytics Founder
>>>>>>>> Tel: +44(0)5603641316
>>>>>>>>
>>>>>>>> (Thanks to the Saiku community we reached our Kickstart
>>>>>>>> <http://kickstarter.com/projects/2117053714/saiku-reporting-interactive-report-designer/>
>>>>>>>> goal, but you can always help by sponsoring the project
>>>>>>>> <http://www.meteorite.bi/products/saiku/sponsorship>)
>>>>>>>>
>>>>>>>> On 25 April 2016 at 12:31, Andy Petrella <noootsab at data-fellas.guru
>>>>>>>> > wrote:
>>>>>>>>
>>>>>>>>> awesome, my turn to add Rad, who had created a terraform dcos
>>>>>>>>> integration to be open sourced soon.
>>>>>>>>> in combination with François you'll have all the information
>>>>>>>>> necessary
>>>>>>>>>
>>>>>>>>> On Mon, 25 Apr 2016 13:20 Samuel Cozannet, <
>>>>>>>>> samuel.cozannet at canonical.com> wrote:
>>>>>>>>>
>>>>>>>>>> Adding some smart people working with us on the Mesos stack.
>>>>>>>>>>
>>>>>>>>>> Andy, Adam, François, have a look at this DCOS charms.
>>>>>>>>>> Tom, look at the project in my github (samnco/ods-dl)
>>>>>>>>>>
>>>>>>>>>> Let's open a gutter room later this week and see how we can
>>>>>>>>>> collaborate on this
>>>>>>>>>> On Apr 25, 2016 03:03, "Tom Barber" <tom at analytical-labs.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> GPU Mesos on Power, that sounds cool!
>>>>>>>>>>>
>>>>>>>>>>> I did originally just stand up Mesos again, but DCOS is in the
>>>>>>>>>>> news so I figured, why not.
>>>>>>>>>>>
>>>>>>>>>>> Okay sounds like a plan. Tomorrow is a bit rubbish, I'm around
>>>>>>>>>>> today and Wednesday though, just ping me off list with a time.
>>>>>>>>>>>
>>>>>>>>>>> Tom
>>>>>>>>>>>
>>>>>>>>>>> --------------
>>>>>>>>>>>
>>>>>>>>>>> Director Meteorite.bi - Saiku Analytics Founder
>>>>>>>>>>> Tel: +44(0)5603641316
>>>>>>>>>>>
>>>>>>>>>>> (Thanks to the Saiku community we reached our Kickstart
>>>>>>>>>>> <http://kickstarter.com/projects/2117053714/saiku-reporting-interactive-report-designer/>
>>>>>>>>>>> goal, but you can always help by sponsoring the project
>>>>>>>>>>> <http://www.meteorite.bi/products/saiku/sponsorship>)
>>>>>>>>>>>
>>>>>>>>>>> On 25 April 2016 at 01:06, Samuel Cozannet <
>>>>>>>>>>> samuel.cozannet at canonical.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Tom,
>>>>>>>>>>>>
>>>>>>>>>>>> I've been working with a bunch of people on this as well for
>>>>>>>>>>>> GPU enabled Mesos on Power. We're also doing more regarding DCOS Open
>>>>>>>>>>>> Source.
>>>>>>>>>>>>
>>>>>>>>>>>> Let's connect and optimize who works on what. I am at OpenStack
>>>>>>>>>>>> summit in Austin this week, but can do a call early in my mornings. Pick
>>>>>>>>>>>> your time and day.
>>>>>>>>>>>>
>>>>>>>>>>>> ++
>>>>>>>>>>>> Sam
>>>>>>>>>>>> On Apr 24, 2016 5:00 PM, "Tom Barber" <tom at analytical-labs.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Figured this was worth distributing to try and prevent
>>>>>>>>>>>>> duplicated effort etc:
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Not ready for distribution yet, but I've stood up DCOS
>>>>>>>>>>>>> orchestration as part of my ApacheCon demo (as its based on MESOS). Deploys
>>>>>>>>>>>>> master and slave nodes to EC2 and works a treat(from a novice perspective).
>>>>>>>>>>>>>
>>>>>>>>>>>>> If anyone is interested in collaboration as my bandwidth is
>>>>>>>>>>>>> pretty much maxed at the mo, let me know and we'll sort this stuff out.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Tom
>>>>>>>>>>>>> --------------
>>>>>>>>>>>>>
>>>>>>>>>>>>> Director Meteorite.bi - Saiku Analytics Founder
>>>>>>>>>>>>> Tel: +44(0)5603641316
>>>>>>>>>>>>>
>>>>>>>>>>>>> (Thanks to the Saiku community we reached our Kickstart
>>>>>>>>>>>>> <http://kickstarter.com/projects/2117053714/saiku-reporting-interactive-report-designer/>
>>>>>>>>>>>>> goal, but you can always help by sponsoring the project
>>>>>>>>>>>>> <http://www.meteorite.bi/products/saiku/sponsorship>)
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> Juju mailing list
>>>>>>>>>>>>> Juju at lists.ubuntu.com
>>>>>>>>>>>>> Modify settings or unsubscribe at:
>>>>>>>>>>>>> https://lists.ubuntu.com/mailman/listinfo/juju
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>
>>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/juju/attachments/20160426/10189cd7/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Untitled.png
Type: image/png
Size: 97190 bytes
Desc: not available
URL: <https://lists.ubuntu.com/archives/juju/attachments/20160426/10189cd7/attachment.png>