Announcing Ubuntu Ensemble, mediawiki community feedback requested

[Ahmed Kamal]

On 06/09/2011 11:51 PM, Platonides wrote:
> Ahmed Kamal wrote:
>> Hi everyone,
>
> Thanks for contacting with us, Ahmed.
>

Hey, I really appreciate your feedback

>> This is Ahmed, I work for Ubuntu helping the cloud community. Ubuntu has
>> been working on some hot technology, that aims to be "apt-get for the
>> cloud"! Basically Install and manage large scale cloud deployments of
>> web applications (and others) with the equivalent ease of "apt-get
>> install". The reason I'm posting this here, is because the very first
>> use-case for this technology (called Ensemble) has been MediaWiki 
>> (yaay)!
>>
>> To get a feel for Ensemble, here is how you'd deploy a 4 machine
>> Mediawiki installation
>> DB ->  MediaWiki ->   memcached ->  HAproxy Loadbalancer
>> ----->8---------
>> # Deploy machines
>> ensemble deploy mysql wiki-db
>> ensemble deploy mediawiki demo-wiki
>> ensemble deploy memcached wiki-cache
>> ensemble deploy haproxy wiki-balancer
>> # Connect them together!
>> ensemble add-relation wiki-db demo-wiki:db
>> ensemble add-relation wiki-cache demo-wiki
>> ensemble add-relation wiki-balancer:reverseproxy demo-wiki:website
>> ----->8---------
>> Slashdotted? Want to scale out? You'd need something like
>> ensemble add-unit wiki-cache
>> ensemble add-unit demo-wiki
>>
>> Yes, it's that easy!
>
> Well, you still need to know that you need 4 machines and what is each 
> of them.

Yes, you'd need to know about that. However, Ensemble is working towards 
the concept of a mega formula. That is one big formula that will deploy, 
configure and link together HAproxy/MW/Memcached/DB+whatever-else (think 
automatic backup, monitoring ..etc) without you even knowing about the 
details of machines. So you'd tell Ensemble "Give me a scalable MW 
deployment, that is monitored, auto-scales, and is auto-backed-up!". 
Granted we don't have that yet, Ensemble is indeed a young project, this 
MW formula is about the first production ready formula (well almost :), 
but it's improving quite fast. And we feel at this stage it's beneficial 
to start getting more exposure

> I first got confused thinking that they were packages on a single 
> machine.
Currently every service is deployed on a separate cloud machine. Lots of 
changes are planned though (like supporting non-cloud deployments, and 
possibly more than one service per machine)
> So, when you add a second demo-wiki, how is the second mediawiki 
> configured? A simple copy ? How are further changes synced?

The same hooks that configure the first instance, are triggered to 
configure the second instance. Since hooks should be written in an 
idempotent manner, all instances reach the same configuration state 
driven by the provided Ensemble variables. I'm not sure what you mean by 
sync'ed. All MW instances use the same DB, thus get the same data. If 
you're talking about a shared file-system, I don't believe the formulas 
have that yet, however, there is work to do a glusterfs formula (and 
other shared FSs) to enable that. If that is something that interests 
anyone here, you can write that formula, it's really quite simple

> What about optional packages? If you are installing for a large scale 
> cloud deployment of mediawiki you probably also want php-apc, or 
> wikidiff2 extension.
Well if you want php-apc, just add it to the apt-get line of the install 
hook. Very soon, formulas will be configurable as well, so having 
php-apc as an "optional" install will be possible. I have no idea what 
wikidiff2 is, but can't see a reason why it cannot be installed as well. 
It really depends on what you (the expert community) define as best practice

> I see you are creating your own LocalSettings.php, but the way you are 
> installing it is really ugly. I advance you that it will break in the 
> new installer of MediaWiki 1.17 You will have an install.php cli 
> script there, though.
Well, that is exactly why I'm reaching out for your help. You see 
Ensemble plans to cover many pieces of software (wordpress, drupal, 
joomla...etc) basically anything that is cloud deployable! Maintaining 
the MW formulas such that it's in tip-top shape, would really benefit 
from being done by MW experts, and so on for other packages. What would 
be absolutely awesome, is if you could patch the formula and show me how 
it should be done. I am willing to help with absolutely anything to have 
that happen

> (i have to admit, it is really the fault of ubuntu package that it 
> doesn't create the database or thatit leaves the tables broken on  
> update)
>
> I recommend you to plan and test with 1.17 and tell us if you need 
> some additional feature. It still could enter in the release, if it's 
> simple enough.
> http://svn.wikimedia.org/svnroot/mediawiki/branches/REL1_17/phase3/
>

Thanks! I don't think we should need any modifications to MW (or any 
other package for that matter). At least I hope we don't! Regarding 
testing and tweaking for 1.17, I'd love to do that, however since 
Ensemble is on its way to supporting all cloud deployable software, the 
only scalable model I can see is for the expert community of each 
respective package to step-up and start tweaking the formulas.

As I mentioned earlier, I wish to see the experience of deploying and 
scaling MW on the cloud, be very smooth. I am very willing to help in 
any way I possibly can. If someone is interested in stepping up and 
playing with Ensemble, improving the MW formula, could you please reply 
and let me know about it. If ec2 time is an issue, I can probably help 
with that as well. Just let me know you'd like to give it a shot, who's 
up for the challenge :)

> Cheers

Cheers