Sprint Feedback

Tom Haddon tom.haddon at canonical.com
Tue Jun 7 08:57:02 UTC 2011

On Thu, 2011-06-02 at 11:45 -0300, Gustavo Niemeyer wrote:
> > I've just come back from a Sprint for a subset of the sysadmins at
> > Canonical who are responsible for deploying and managing some of the
> > services that Canonical runs (Landscape, Launchpad, Ubuntu One, etc.)
> Thanks a lot for this feedback Tom.
> > == Things we like about Puppet ==
> >
> > - Declarative state. This makes it easier to manage services over the
> > longer term, because you can be assured that systems are configured the
> > way you've told them to be configured.
> Ensemble is really an orchestration framework, and it doesn't really
> enforce on you a particular way to configure the formula environment.
> If you like Puppet for that, you can even use it in standalone mode to
> enforce the configuration Ensemble provides.
> > == Things we don't like about Ensemble ==
> >
> > - Ensemble seems to currently require a cloud infrastructure (EC2/S3
> > specifically) to run. Are there plans in the future to allow Ensemble to
> > run on bare metal? Our usage of EC2 has been limited for a number of
> Yes, there are plans for that, and isn't too hard even.  We just need
> to take some on our roadmap to do it.


> > - Doesn't seem to be a way to maintain state across the servers that
> > Ensemble is managing.
> Ensemble manages communication across the services using relations, in
> a pretty comfortable way, so I'm not sure I get what you mean by that.

I guess I'm meaning in a more comprehensive way in terms of managing the
state of the servers as a whole rather than the specific service we're
talking about. Since Ensemble seems to work in terms of executing
scripts, this means you know that certain scripts have been run, but you
don't know the end state that that's achieved in terms of configuration
files on disk, for example.

> > - Can't preview changes before they happen to determine if they will do
> > what you want them to do. Can't test out new versions of different
> > formulas with different "environments".
> We have to think a bit more around this.  Part of that is actually on
> purpose, since we want formulas to have freedom to implement things in
> different ways, so you're effectively a user of an encapsulated
> service.  We have to think a bit more about how to provide the
> visibility you want without compromising on that aspect of the system.

Ok, this would be kind of crucial to us (and I'm sure others too). Being
able to preview changes and confirming they'll do what you expect is key
to being happy with making changes on production systems.

> > == Some other comments based on the example formulas ==
> >
> > - The "utility instance" seems to be a single point of failure. If this
> > goes down do we lose access to everything?
> Only in the current incarnation.  ZooKeeper can very easily have
> multiple units in a resilient cluster.  We just need to take the time
> to put it in place.


> > - Once you've hooked items together, it's confusing to me that the
> > "mysql" service is saying it's relation is "db: wordpress" - wordpress
> > isn't a DB, so shouldn't this be saying "app: wordpress" or "db for:
> > wordpress"?
> This is wrong indeed.  The relation on the mysql side should not be called "db".
> > - When you add-unit to the wordpress instance, I don't see how this
> > actually provides any scalability. Presumably you'd need to be using
> These are very trivial examples formulas, demonstrating what you can
> do.  We hope someone that knows better than we do about how to put a
> wordpress topology in place (hint!) takes them over.
> > - Can you use your own AMI? Different instance sizes?
> You can use different AMIs, but you don't _have_ to.  THe AMIs are
> basic Ubuntu AMIs.. you can tune them for the workload at hand.
> Yes, different instance sizes are supported by configuring your
> environment file.
> > - How do you apply security updates to running instances, etc.?
> We'll start by having an upgrade hook, which defaults to sudo apt-get
> upgrade, but which the formula can customize.
> More details in the documentation:
>     https://ensemble.ubuntu.com/docs/upgrades.html
> > - Shouldn't the formulas include author info in the yaml? I'd be loathe
> Indeed.  We'll add support for that.
> > to create my own formulas based on those someone else has provided
> > unless I know who I can go to if I have problems with the formula. Also,
> > is there any promise of version compatibility, or is it possible that if
> > you create formulas that import other formulas that your own formula
> > will no longer work?
> Formulas do not really "import" other formulas, they relate to them,
> so if the protocol used in the relation is the same, it should
> continue to work across formula versions.  If it changes in an
> incompatible way, the relation name should change to reflect that.
> > - Can it use elastic IPs (DNS and for interacting with "static"
> > services)? Can it interact with services that are not part of Ensemble
> Not today.  It's a feature high in our roadmap, though.
> > - What security is there in terms of if one server in an ensemble
> > cluster is compromised? How much information is shared between the
> > instances with zookeeper and what's to prevent one server from querying
> > all information on other servers?
> Today, nothing.  We have somewhat of a clear path to follow on the
> security front, though.
> > - What is the Ensemble approach to firewalls? Is it expected that this
> > is a formula issue?
> It is, and that's already in progress.  Formulas will be able to
> declare which ports they want to open, and the admin can select which
> services to expose.  The firewall is only open when both the service
> is exposed by the admin and the formula declared it wants the port
> open.
> > - It's not entirely clear to me if you could use Ensemble to replace our
> > current deployment scripts - they are used to push out incremental code
> > updates to specific services, and work by copying code into a directory
> From your description, I don't see any blockers.  Formulas can be
> upgraded, and they have a proper hook which is called when that
> happens, allowing you to add custom logic at that stage.
> > == What's next? ==
> >
> > Our plans from here are to continue testing Ensemble so that we can try
> > to realistically get an idea of what works for us and what doesn't over
> > the long term. Initially this involves testing how it deals with a bunch
> > of error states, but then we'd also like to begin writing some formulas
> > (I guess participating in https://launchpad.net/principia would be the
> > best thing here).
> Yes, that would be fantastic!  Please feel free to hang around here in
> the mailing list and/or on the freenode #ubuntu-ensemble channel for
> us to learn what you need and also to help you out.
> > I think the overall takeaway as far as we can see is that Ensemble seems
> > suited for deploying services, but not necessarily managing services. Is
> Given the points above, I don't see any constraints in terms of it
> managing services.
> > the idea that you would need to deploy your own management layer through
> > Ensemble, or outside of Ensemble, or is the idea that in the future
> > Ensemble will be able to manage services for you?
> It manages services for you _today_.    It certainly has well-known
> shortcomings which are a side effect of us having it working at all
> for three months now, but it feels like we have your needs covered on
> the medium term roadmap.
> Again, thanks a lot for the feedback Tom.  That's really appreciated
> as it gives us good ground to follow on.

More information about the Ensemble mailing list