[ec2-beta] EBS questions

[Eric Hammond]

Neal:

 > What is the best way to use EBS to simplify security updates that
 > require reboots?

I'm not sure I understand the question.

EBS volume attachments survive a simple reboot.  If your file system is 
listed in /etc/fstab it will be mounted as normal.

 > How early can they be mounted

After the instance is running, you can attach the EBS volume.  After the 
network is up, you can mount the volume.  There has been some discussion 
of auto attaching and mounting here:

http://developer.amazonwebservices.com/connect/thread.jspa?messageID=99100
http://developer.amazonwebservices.com/connect/thread.jspa?messageID=98885
http://developer.amazonwebservices.com/connect/thread.jspa?messageID=101017

 > and how much of the root filesystem can be EBS?

You can get as complicated as you like, though moving things like 
/var/run can be difficult after system processes are running.

Here is an interesting discussion about booting off of EBS:

http://developer.amazonwebservices.com/connect/thread.jspa?threadID=24091

Amazon has stated that they are considering supporting a native way to 
launch from an EBS volume, but their list of priorities being considered 
is long and their resources finite.  They definitely listen to what 
users ask for if you want to add your voice.

 > If you put /usr on EBS, how much would the I/O request charges cost
 > for typical usage patterns?

I don't know "typical" and doubt anybody does.  IO requests are heavily 
dependent on what's going on with the system and could vary by several 
orders of magnitude.

On a half dozen reasonably active servers with EBS volumes mounted we're 
running around 100 million IO requests ($10) per month each.  Your 
mileage will vary.

You can use IO transactions per second (tps) on your existing servers to 
get an estimate (though it's not entirely a 1-1 mapping).

The cool thing about EC2 is that you can just try it out for a short 
time and see how much it costs.  If it's too much to pay for over a long 
period of time you can optimize your application, use another approach, 
or user another service.

Note, however, that EBS transactions are not updated on your account 
activity page quite as often as, say, the hours your instances have been 
running.

FYI, questions that aren't directly related to the beta Ubuntu AMIs and 
the beta program can be asked in the larger EC2 Ubuntu community where 
more people can answer them and more people can benefit from the answers:

   http://ec2ubuntu-group.notlong.com

EC2 questions that aren't directly related to Ubuntu can be asked on 
Amazon's EC2 forum where there are even more experts:

   http://ec2forum.notlong.com

BTW, thanks for the tip on /srv.  I've been using Linux and Unix for a 
couple decades and never heard of /srv though it seems to be a standard 
now.  I've been using /var/local for this purpose on most of my systems.

http://www.pathname.com/fhs/pub/fhs-2.3.html#SRVDATAFORSERVICESPROVIDEDBYSYSTEM

--
Eric Hammond
ehammond at thinksome.com



Neal McBurnett wrote:
> Good info on EBS - thanks.  That was rolled out long after I did most
> of my EC2 investigation last year so I haven't tried it.
> 
> It sounds like Ubuntu images and packages should be configured to
> store all persistent data in a place that is mounted from EBS.
> Mounting /srv as EBS seems like the obvious choice, along with
> symlinking appropriate paths to point there, like /var/www
> and /usr/lib/cgi-bin/
> 
> What is the best way to use EBS to simplify security updates that
> require reboots?  How early can they be mounted and how much of the
> root filesystem can be EBS?  If you put /usr on EBS, how much would
> the I/O request charges cost for typical usage patterns?
> 
> Cheers,
> 
> Neal McBurnett                 http://neal.mcburnett.org/
> 
> On Wed, Dec 17, 2008 at 12:05:16PM -0800, Eric Hammond wrote:
>> EBS is a fantastic feature and in my opinion is by itself a reason to 
>> move services to Amazon EC2.
>>
>> I wrote a tutorial for Amazon describing one way to use EBS for hosting 
>> a MySQL database:
>>
>>   http://ec2ebs-mysql.notlong.com
>>
>> I need to update the tutorial so that it works with the official Ubuntu 
>> beta AMIs since the approach I originally described does not work with 
>> the default AppArmor configuration.
>>
>>
>> Minor clarification on performance:
>>
>> The current general consensus based on lots of different performance 
>> tests and statements from Amazon seems to be that EBS generally performs 
>> better than ephemeral (local) storage for random IO especially when you 
>> RAID multiple EBS volumes together.  At this level you would be using a 
>> large or xlarge instance for more network capacity.
>>
>> Ephemeral storage generally performs better than EBS for large 
>> sequential IO, especially after the initial write penalty has been paid 
>> (first write to a block on the instance takes longer than subsequent 
>> writes).
>>
>> It is also possible to RAID ephemeral partitions for better local 
>> performance if you don't need the persistence of EBS or want to pay the 
>> additional cost.
>>
>> --
>> Eric Hammond
>> ehammond at thinksome.com
>>
>>
>>
>> Michael Jensen wrote:
>>> Neal McBurnett wrote:
>>>>> PSS: Can you have persistant instances? it seams my instances
>>>>> dissapear after i shut them down, so if i were to build a server would
>>>>> i just loose everything each time? (wouldnt think thats the case).
>>>> This is a big surprise for most users.  No - you don't get any
>>>> persistence of the disk (or ram of course).  The way to get data
>>>> persistence is to use external storage like Amazon's S3 service,
>>>> or your own networked database.
>>> You actually can use Amazon's Elastic Block Store for persistent storage
>>> http://aws.amazon.com/ebs/
>>> Aside from the awesome snapshot feature, the disk performance is better
>>> than the "local" disks in EC2 instances.
>>> http://altj.org/content/performance-increase-amazons-ebs-persistent-storage
>>>
>>>
>>> Michael
>>>
>