[ec2-beta] EC2 I/O Failure

John Hampton john at cleanoffer.com
Thu Mar 26 22:39:56 GMT 2009


Thanks Chuck,
I just got an email from amazon, apparently our server suffered hardware
failure, so I don't think filing a bug is necessary.

John

On Thu, Mar 26, 2009 at 12:58 PM, Chuck Short <chuck.short at canonical.com>wrote:

> Hi John,
>
>
> I was wondering how I can reproduce this also could you open up a bug in
> launchpad (http://bugs.launchpad.net/ubuntu-on-ec2) so we can track this?
>
> Regards
> chuck
>
> John Hampton wrote:
> > Just loaded one of our first ubuntu beta instances (ami-18d73071). Is
> > this due to a driver problem or a hardware failure at amazon? The
> > instance is dying due to I/O failures found this in the logs:
> >
> > Mar 26 12:25:31 ubuntu kernel: [183609.913991] lost page write due to
> > I/O error on dm-0
> > Mar 26 12:25:31 ubuntu kernel: [183609.947241] lost page write due to
> > I/O error on dm-0
> > Mar 26 12:25:31 ubuntu kernel: [183609.997125] lost page write due to
> > I/O error on dm-0
> > Mar 26 12:25:31 ubuntu kernel: [183610.047034] lost page write due to
> > I/O error on dm-0
> > Mar 26 12:25:31 ubuntu kernel: [183610.096987] lost page write due to
> > I/O error on sda1
> > Mar 26 12:25:31 ubuntu kernel: [183610.096997] lost page write due to
> > I/O error on sda1
> > Mar 26 12:25:31 ubuntu kernel: [183610.097005] lost page write due to
> > I/O error on sda1
> > Mar 26 12:25:31 ubuntu kernel: [183610.097013] lost page write due to
> > I/O error on sda1
> > Mar 26 12:25:31 ubuntu kernel: [183610.097039] JBD: Detected IO errors
> > while flushing file data on sda1
> > Mar 26 12:25:31 ubuntu kernel: [183610.146889] lost page write due to
> > I/O error on dm-0
> > Mar 26 12:25:31 ubuntu kernel: [183610.180132] lost page write due to
> > I/O error on dm-0
> > Mar 26 12:25:31 ubuntu kernel: [183610.280729] JBD: Detected IO errors
> > while flushing file data on dm-0
> > Mar 26 12:25:31 ubuntu kernel: [183610.327166] ------------[ cut here
> > ]------------
> > Mar 26 12:25:31 ubuntu kernel: [183610.327169] WARNING: at
> > /build/buildd/linux-xen-2.6.27/fs/buffer.c:1186
> > mark_buffer_dirty+0x84/0xb0()
> > Mar 26 12:25:31 ubuntu kernel: [183610.327171] Modules linked in:
> > iptable_filter ip_tables x_tables fbcon tileblit font bitblit
> > softcursor uvesafb xfs binfmt_misc dm_multipath dm_mod scsi_dh
> > scsi_mod ipv6 af_packet loop ext3 jbd fuse
> > Mar 26 12:25:31 ubuntu kernel: [183610.327188] Pid: 32062, comm: vi
> > Not tainted 2.6.27-22-xen #1
> > Mar 26 12:25:31 ubuntu kernel: [183610.327190]
> > Mar 26 12:25:31 ubuntu kernel: [183610.327191] Call Trace:
> > Mar 26 12:25:31 ubuntu kernel: [183610.327197]  [<ffffffff8023679f>]
> > warn_on_slowpath+0x5f/0x90
> > Mar 26 12:25:31 ubuntu kernel: [183610.327201]  [<ffffffff80236b1f>] ?
> > __call_console_drivers+0x6f/0x90
> > Mar 26 12:25:31 ubuntu kernel: [183610.327207]  [<ffffffff804e26e3>] ?
> > printk+0x67/0x6c
> > Mar 26 12:25:31 ubuntu kernel: [183610.327210]  [<ffffffff802e26e4>]
> > mark_buffer_dirty+0x84/0xb0
> > Mar 26 12:25:31 ubuntu kernel: [183610.327224]  [<ffffffffa002fae0>]
> > ext3_commit_super+0x60/0x90 [ext3]
> > Mar 26 12:25:31 ubuntu kernel: [183610.327233]  [<ffffffffa0032127>]
> > ext3_handle_error+0x57/0xd0 [ext3]
> > Mar 26 12:25:31 ubuntu kernel: [183610.327241]  [<ffffffffa00322c2>]
> > ext3_error+0xa2/0xb0 [ext3]
> > Mar 26 12:25:31 ubuntu kernel: [183610.327245]  [<ffffffff804e395b>] ?
> > __wait_on_bit+0x7b/0x90
> > Mar 26 12:25:31 ubuntu kernel: [183610.327248]  [<ffffffff802e1b00>] ?
> > sync_buffer+0x0/0x50
> > Mar 26 12:25:31 ubuntu kernel: [183610.327251]  [<ffffffff804e39e4>] ?
> > out_of_line_wait_on_bit+0x74/0x90
> > Mar 26 12:25:31 ubuntu kernel: [183610.327256]  [<ffffffff8024fbd0>] ?
> > wake_bit_function+0x0/0x40
> > Mar 26 12:25:31 ubuntu kernel: [183610.327259]  [<ffffffff802e1ea6>] ?
> > __wait_on_buffer+0x26/0x30
> > Mar 26 12:25:31 ubuntu kernel: [183610.327266]  [<ffffffffa00226a2>]
> > read_block_bitmap+0x162/0x170 [ext3]
> > Mar 26 12:25:31 ubuntu kernel: [183610.327274]  [<ffffffffa00228e7>]
> > ext3_free_blocks_sb+0xd7/0x4c0 [ext3]
> > Mar 26 12:25:31 ubuntu kernel: [183610.327277]  [<ffffffff802e3703>] ?
> > __find_get_block_slow+0x43/0x150
> > Mar 26 12:25:31 ubuntu kernel: [183610.327284]  [<ffffffffa0023911>]
> > ext3_free_blocks+0x21/0x90 [ext3]
> > Mar 26 12:25:31 ubuntu kernel: [183610.327292]  [<ffffffffa0029785>]
> > ext3_clear_blocks+0x195/0x1d0 [ext3]
> > Mar 26 12:25:31 ubuntu kernel: [183610.327300]  [<ffffffffa00298d7>]
> > ext3_free_data+0x117/0x160 [ext3]
> > Mar 26 12:25:31 ubuntu kernel: [183610.327308]  [<ffffffffa002a4a3>]
> > ext3_truncate+0x563/0x650 [ext3]
> > Mar 26 12:25:31 ubuntu kernel: [183610.327317]  [<ffffffffa0011c76>] ?
> > journal_start+0xa6/0xf0 [jbd]
> > Mar 26 12:25:31 ubuntu kernel: [183610.327325]  [<ffffffffa002a690>]
> > ext3_delete_inode+0x100/0x110 [ext3]
> > Mar 26 12:25:31 ubuntu kernel: [183610.327333]  [<ffffffffa002a590>] ?
> > ext3_delete_inode+0x0/0x110 [ext3]
> > Mar 26 12:25:31 ubuntu kernel: [183610.327338]  [<ffffffff802cf9af>]
> > generic_delete_inode+0xbf/0x1a0
> > Mar 26 12:25:31 ubuntu kernel: [183610.327341]  [<ffffffff802cfab5>]
> > generic_drop_inode+0x25/0x30
> > Mar 26 12:25:31 ubuntu kernel: [183610.327345]  [<ffffffff802ce48d>]
> > iput+0x5d/0x70
> > Mar 26 12:25:31 ubuntu kernel: [183610.327347]  [<ffffffff802c304c>]
> > do_unlinkat+0x10c/0x1d0
> > Mar 26 12:25:31 ubuntu kernel: [183610.327351]  [<ffffffff802b78fd>] ?
> > fput+0x1d/0x30
> > Mar 26 12:25:31 ubuntu kernel: [183610.327354]  [<ffffffff802b3d6b>] ?
> > filp_close+0x5b/0x90
> > Mar 26 12:25:31 ubuntu kernel: [183610.327356]  [<ffffffff802c3121>]
> > sys_unlink+0x11/0x20
> > Mar 26 12:25:31 ubuntu kernel: [183610.327360]  [<ffffffff8020c088>]
> > system_call_fastpath+0x16/0x1b
> > Mar 26 12:25:31 ubuntu kernel: [183610.327363]  [<ffffffff8020c020>] ?
> > system_call+0x0/0x52
> > Mar 26 12:25:31 ubuntu kernel: [183610.327364]
> > Mar 26 12:25:31 ubuntu kernel: [183610.327366] ---[ end trace
> > 27412ec0c2aa53a8 ]---
> > Mar 26 12:25:36 ubuntu kernel: [183613.270600] JBD: Detected IO errors
> > while flushing file data on sda1
> >
> >
> >
>
>
> --
> Ec2-beta mailing list
> Ec2-beta at lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/ec2-beta
>



-- 
John O. Hampton, Jr., CTO
CleanOffer, Inc.
101 California Street
Suite 2450 #612
San Francisco, CA  94111

Homepage: http://www.cleanoffer.com
Blog: http://blog.cleanoffer.com

(415) 240-4532 (office)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://lists.ubuntu.com/mailman/private/ec2/attachments/20090326/ee68ed02/attachment-0002.htm 


More information about the Ec2-beta mailing list