[ec2-beta] Automated bundling?

[Eric Hammond]

Chris:

1. In order to bundle the instance and upload the image, the private key, certicifate, and secret key al need to be on the instance (unless you feel like downloading gigabytes to your local system)

You can delete or wipe them afterwards, but the private key and certificate need to be files on a file system (which I suppose could be in memory).

2.  Yes, you can --exclude directories that are mount points for EBS, s3fs, and the like.

No need to stop services, but keep in mind that if files are being modified by a service (think database) they might not be copied to the new AMI consistently, and if processes are creating and deleting files then the rsync phase might fail.  And, when the rsync fails, the bundling process does not give intuitive error messages.

--
Eric Hammond
http://www.anvilon.com

-----Original Message-----
From: Christopher Smith <chris.smith at nexops.com>
Date: Friday, Jan 30, 2009 3:07 pm
Subject: Re: [ec2-beta] Automated bundling?
To: ec2-beta at lists.ubuntu.com

Having this available in the Ubuntu image would be nice, but nicer I  
>think would be to have the command available on my local computer.   
>Maybe I'm overly worried, but I dislike having to place my secret key  
>on the instance itself(*).  And why have that extra step to have to  
>login to the instance first?
>
>How about a command that can be run locally which would activate a  
>bundling script on the instance and make use of my secret key stored  
>locally and securely.  Things like the directories to exclude from the  
>bundle and possibly other parameters for bundling could be set on the  
>instance itself, say /etc/ami/bundle.conf and/or /etc/ami/ 
>exclude.list.  Also there would be a /etc/ami/bundle.d where scripts  
>could be placed if anything needs to be run to clean up the instance  
>before bundling.
>
>ec2-bundle-upload-register  <user at public-dns> <s3-store> <bundle-prefix>
>
>and given I tend to keep my AMIs in one particular S3 store, that  
>parameter can probably be taken from a preset/default value somewhere.
>
>I'll have something that does the above soon, but my bash scripting is  
>rough and ready at best.  So sensible stuff, like checking that the  
>ami-tools are installed, you're not overwriting a pre-existing bundle  
>will probably be missing.
>
>As an aside, In my AMIs I mount S3 and EBS volumes using mount points  
>in the root partition to allow them to be remounted easily from fstab  
>on startup.  Does anyone know if I can keep the mount point but not  
>any of the volumes using the "exclude" option?  Currently, I stop all  
>services that use the storage, unmount the volumes, bundle the image,  
>remount and restart.  However, it would be nice to be able to bundle  
>while the instance keeps working.
>
>- Chris
>
>(*) Off-topic/Pet bugbear for AWS.  In order to access S3 from an  
>instance that instance needs to know my secret key which controls my  
>whole AWS account.  Why not a subsidiary id/key for single S3 bucket  
>access only, rather than requiring the account holder to open separate  
>AWS accounts.
>
>On 31 Jan 2009, at 06:15, Eric Hammond wrote:
>
>>
> ...not that I would object to an Ubuntu package which included a  
> command
> to do the bundle and upload of a running instance :)   Some other  
> public
> AMIs provide this and I think it's a fine idea.
>
>> The main considerations that pop into my head include:
>
>> 1. Figure out how the user provides all of the parameters and key  
> files
> by