ehammond at thinksome.com
Sun Jul 26 23:18:00 BST 2009
Did you mention which AMI (id) you are running? This is important when reporting problems on EC2.
Amazon has the ability to perform a "hard reboot" of an instance, if a normal reboot does not work. You can post the request with instance id to to the EC2 forum, though it might take a while for them to get around to it.
Not suitable for emergency response, but might be useful to know if this helps the situation for debugging purposes.
From: Darren Govoni <darren at ontrenet.com>
Date: Sunday, Jul 26, 2009 2:07 pm
Subject: Re: [ec2] swap?
To: Robert Coup <robert.coup at koordinates.com>
CC: ec2 at lists.ubuntu.com
> According to the launchpad bug, it seems related to a bug in the
>kernel at the moment, but as yet I am only assuming my issue is related to the documented bug because of the description.
>My new data shows this.
>My system runs fine for about 15 minutes and I see the memory dwindle down (I'm running 32 supervisord python app threads that make system
>calls). For the most part, memory is fairly static. But inches down.
>At one point it jumps from about 250MB free to 34MB free and that is the last update I see after it freezes. My 'top' indicates that kswapd with a nice of -5 has jumped to the top of the process list. Not sure if
>thats because of the memory situation or the cause of it.
>After that, the instance cannot be used, rebooted or contacted in
>I will post this on the launchpad thread too, which seems to indicate a bug in the process scheduler.
>But if kswapd is kicking in and there is no swap, I wonder what that would mean.
>On Mon, 2009-07-27 at 08:23 +1200, Robert Coup wrote:
> Hi Darren,
> On Mon, Jul 27, 2009 at 5:44 AM, Darren Govoni <darren at ontrenet.com>
> The problem I am running into may be related to the other
> posted issue
> here when instances freeze.
> Basically, all of my instances will freeze up after some time
> where they are reading/writing AWS services heavily as well as
> disk. I was concerned if its related to no virtual memory
> If you have no swap things will get killed, be it apache, a database,
> or something else. The system won't go down if it runs out of memory.
> If the system hangs it implies (to me) that it's a filesystem or
> kernel problem. What filesystem are you running? Any RAID/LVM? Are you
> using EBS or is this the local instance storage?
> once it freezes I cannot even reboot it. Its gone forever.
> So rebooting via the AWS console/tools doesn't help? Note that it can
> easily take 5 minutes or longer to reboot an instance via the AWS
> restart command. After your instance has rebooted, there may be
> something informative (about the problem) in the EC2 console output.
> Rob :)
More information about the Ec2