Django charms [Was: Re: [ubuntu-cloud] Ubuntu cloud use case]

Tue Sep 25 02:51:57 UTC 2012

On 09/24/2012 10:42 PM, Jim Baker wrote:
> Bruno,
>
> Thanks for raising these questions. Having great Django support is very
> important for Juju. I have outlined some of my thoughts below.
>
> On 09/24/2012 10:30 AM, Bruno Girin wrote:
>> On 21/09/12 20:45, Bruno Girin wrote:
>>> On 20/09/12 19:32, Mark Mims wrote:
>>>> <snip>
>>>> Please by all means help with Django!!  I'm not sure what extra features
>>>> / connections we should add to it at this stage and need input from
>>>> peeps who're running it in production.
>>> I will do. I'll probably start working on that in the next few weeks and
>>> will tell you how I progress.
>> Right, I've had a look and I need advice on how to proceed. If I look at
>> what exists today, I find a django-rainbow charm [1], an initial
>> Launchpad bug for a Django framework charm [2] initially worked on by
>> Mike Nelson, an additional bug for a python-django with WSGI container
>> charm [3] created by Patrick Hetu as a result of his idea on using a
>> WSGI subordinate and a blog post from Michael Hall [4] who is looking at
>> charming the app rather than the framework.
>>
>> Looking through this, django-rainbow [1] looks simple but doesn't suit
>> my needs at it assumes Apache, MySQL and applications stored in
>> Mercurial (I use nginx or gunicorn, PostgreSQL and have my application
>> code hosted in git).
> django-rainbow suits a very specific use case, as you mentioned.
>
>> Looking at [2], [3] and [4], there are a lot of good ideas and reading
>> on subordinate services [5], one way to get this to work would be:
>>
>> +- WSGI container as a principal service
>>      +- django framework subordinate to wsgi container
>>          +- django apps subordinate to django (could be any number of apps)
> Right, a specific WSGI container should be the primary service for the
> Django app, with a number of choices, including gunicorn, mod_wsgi, and
> uWSGI. However, we don't want to model the Django framework as a
> subordinate - we always want those bits to be installed: they're not
> optional. Subordinates, as initially conceived, are about aspects where
> there is a choice (so think of examples like logging or monitoring).
>
> So this implies something like django-gunicorn, django-modwsgi, etc.,
> similar to the tomcat6 and tomcat7 charms. uWSGI might be more
> complicated, given that it can work with Cherokee and Nginx. However, I
> don't think we have to support every possible combination; it's OK to be
> opinionated.
Well, my choice here is to work on django-gunicorn option. Like you say, 
uWSGI is more
complicated, but we can work later on it.
>
> The idea of using a subordinate to manage a specific Django app is an
> intriguing option. However, stepping back, the question I would ask is,
> would a Django stack benefit from having the lifecycle support of
> subordinates? Also, is this really a good mapping between Django and
> Juju? The Django-Mingus blog stack is a popular way to demonstrate
> reuse, and despite being pretty simple, it uses 28 apps. It's not clear
> to me that 28 subordinates is really going to work that well - although
> it's certainly worth trying. And of course, non-demo Django stacks can
> readily use many more apps.
>
> But I do see a role for a set of Django apps being bundled as a
> subordinate, especially when it may relate to other services. Consider
> Haystack. You might want to relate this Solr, Xapian, and other search
> engines. Deploying this subordinate also needs to rewrite settings.py.
Well, there are basic Django apps for every Django site, and Haystack is 
one of them.
For me, it should include it in the charm
>
>> This would mean having a generic django charm that could be deployed
>> into any WSGI container (Apache, nginx, gunicorn) plus simple charms for
>> the apps. It's then a case of identifying what should be the
>> responsibility of Django and what should be the responsibility of the
>> apps. Django allows you to make apps agnostic from the way they are
>> deployed so it should be feasible.
> So if we don't have apps correspond to subordinates, then the other key
> piece will be to ensure that all the bits for the apps are deployed to
> the Django service units, and that this can be updated over time. PyPI
> is an important source of truth, as well as launchpad, github,
> bitbucket, and the vcs of the user's choice.
>
>> When it comes to databases, Django uses a concept similar to the JavaEE
>> data source, where you define the different data sources in the
>> container and the apps just have to reference them by name. If you give
>> no reference, it uses the database called "default". So what this means
>> is the django charm should specify that it can have any number of "db"
>> relationships, the default one being called "default". It should then be
>> able to add new named db relationships to support applications that
>> require more than one data source.
> This is a key question, how to map database definitions in Django to Juju.
>
> At a very simple level, Django stacks that use one database ("default")
> can simply add a relation to the database service in question. The
> config-changed hook can then ensure that settings.py has this
> definition, including such info as host.
>
> For multiple databases, it becomes tricky. First, there's a quite
> relevant bug in Juju, see
> https://bugs.launchpad.net/juju-core/+bug/1026422 re relation edge settings.
>
> With that in mind, there are essentially two cases:
>
> The Django stack is relating to multiple database services. This seems
> the most plausible case, otherwise why not simply combine the relevant
> schema under the same database and keep the management easy? Juju can
> readily handle this case, so it becomes just a question, which Juju
> database service corresponds to what database in Django's settings.py
> (maybe DRY and use the same name, otherwise it requires config settings).
>
> The Django stack wants to use the same given database service for
> multiple databases. There's some degree of support for this in say the
> mysql-admin interface, but from my perspective, this seems clunky, so I
> don't see a reason to support this specific use case. Django makes it
> easy to have apps with their schema in the same database. So let's just
> use that functionality.
1+
>
>> The next question is how do you model the fact that a Django db
>> connection could accept different interfaces? Ideally, if you do
>> something like "juju add-relation django mysql", it should work out on
>> its own that it needs to ensure the correct client libraries are
>> installed for mysql on the django node and setup the correct
>> configuration parameters.
> Mark Mims recently wrote an email on this list re a charm's role
> (https://lists.ubuntu.com/archives/juju/2012-August/001785.html). In a
> nutshell, a Django charm should be able to relate via mysql or pgsql (or
> other databases). If using pgsql, then it could relate to something like
> pgpool, which if there was a charm for it, could act as middleware
> between Django and an actual set of postgres servers.
I would put here PgBouncer. It´s more efficient in terms of connection 
reuse than
PgPool.
>
> The act of adding the relation could install the appropriate client bits
> in the relation-joined hook.
>
>
>> Last question (but not least!), how do you add nodes to a service
>> defined like this with subordinate services? Are all the subordinate
>> services extended to the new node as well?
> Yes. That keeps things simple. And conversely, you cannot add units to a
> subordinate.
>
> Incidentally, there are other key pieces to deploying a scalable Django
> stack, eg memcached. But in the case of caching infrastructure, this is
> such a standard part of Django that it really should be part of the
> Django charm.
>
> Here's another good source for thinking about building scalable Django
> stacks, https://github.com/jacobian/django-deployment-workshop - it has
> the tutorial slides plus code using Fabric and Chef Solo to manage the
> deployment of a Mingus blog.
Yes, this is a obligated reading if we want to work on this.
Where is the main code for the django charm?

Best wishes

>
> - Jim
>
>
>

-- 

Marcos Luis Ortíz Valmaseda
*Data Engineer && Sr. System Administrator at UCI*
about.me/marcosortiz <http://about.me/marcosortiz>
My Blog <http://marcosluis2186.posterous.com>
Tumblr's blog <http://marcosortiz.tumblr.com/>
@marcosluis2186 <http://twitter.com/marcosluis2186>

10mo. ANIVERSARIO DE LA CREACION DE LA UNIVERSIDAD DE LAS CIENCIAS INFORMATICAS...
CONECTADOS AL FUTURO, CONECTADOS A LA REVOLUCION

http://www.uci.cu
http://www.facebook.com/universidad.uci
http://www.flickr.com/photos/universidad_uci
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/juju/attachments/20120924/d7bc4583/attachment.html>