[ec2-beta] Newbie Questions

[Eric Hammond]

Darren:

There are a number of different approaches that can be taken to launch EC2 instances with your desired system and application software.  I'll describe some best practices here without a lot of background as to the pros and cons, but feel free to ask further questions and to ask for help when you run into roadblocks.

1) You can write simple "user-data scripts" in bash/Perl/Python/... to install and configure your preferred software on new instances when they boot.  This highly recommended approach lets you upgrade to a newly released public AMI without a lot of hassle.  It's also easier to test adjustments to the startup process than with other approaches.

The official Ubuntu beta AMIs have been built with the same user-data startup hook approach as the (unofficial) Ubuntu AMIs I have been publishing on http://alestic.com

If you run an instance passing in user-data starting with #! then the instance will run this script on boot.

There are a couple cases where the user-data script approach fails and you need to go down a different path:

a) The software installation cannot be automated (more rare than you might think).

b) The software installation process takes far too long for new instances to be usable in a reasonable time frame.

In these situations you have the option of bundling your own AMIs. This can be done in a couple ways...

2) Run a standard public AMI, install the software on the running instance (preferably using an automated script for reproducibility), bundle the running instance, upload the bundle to S3, and register it as a new AMI.  Amazon's Getting Started Guide walks through this process:

  http://ec2gsg.notlong.com

I don't think the official Ubuntu beta AMIs come with the AMI command line tools installed; ask if you need instructions.

3) Use AMI building software to create your own AMI from scratch with custom software installed through hooks.  I currently use the script I publish on http://alestic.com but I support the development version of vmbuilder which was used to build the official Ubuntu beta AMIs.  The version of vmbuilder which works for building EC2 instances is currently only available through bazaar revision control in launchpad.net and the instructions for using it have been a bit fluid.  I'd recommend trying one of the above approaches in the short term.

Less popular approaches to the software installation challenge include:

4) It should be possible in some cases to install software on an EBS volume and mount this at boot (or rather a volume created from a snapshot of the original volume).

5) There has been some progress published on an approach which enables running the entire instance root partition off of an EBS volume using pivotroot at startup, but this is pretty advanced, has a number of potential drawbacks, and does not seem to be widely adopted.

..

The above describes options for running instances with your preferred software.  You also asked about running with your own database.  Databases and other data should generally not be included in AMIs but should be stored externally and attached to or retrieved by the instance.

The standard recommendation for databases and many other data classes at this point is to store them on EBS volumes for persistence.  I wrote a tutorial about this for Amazon:

 http://ec2ebs-mysql.notlong.com

The tutorial needs to be updated so that it works for the official Ubuntu beta AMIs which have AppArmor enabled, but you'll get the general idea.

Amazon S3 and SimpleDB are also great options for data in many cases, depending on your application requirements.

--
Eric Hammond
http://www.anvilon.com


-----Original Message-----
From: Darren Govoni <darren at ontrenet.com>
Date: Thursday, Jan 1, 2009 10:23 am
Subject: [ec2-beta] Newbie Questions
To: ec2-beta at lists.ubuntu.com

Hi,
>  I just signed up and have a couple quick questions I didn't see in the FAQ.
>
>1. How do I configure my Ubuntu image with my own apps/databases, etc? For easy replication and deployment?
>2. I didn't see a link to install the python-vmbuilder-ec2 package and couldn't find it with apt-get on ubuntu 8.04 (my desktop).
>
>Any tips appreciated!
>Happy new year
>
>Darren