[ec2-beta] data corruption

John Hampton john at cleanoffer.com
Tue Apr 14 21:24:21 BST 2009


Hi Mark,
Interesting information about file system preferences. I was wondering what
is the status of XFS? We use it because of the quicker time to format.

Thanks
John

On Tue, Apr 14, 2009 at 12:54 PM, Mark Shuttleworth <mark at ubuntu.com> wrote:

>  Ben Hendrickson wrote:
>
> I use around 16 large instances for data processing tasks.  When using
> the beta AMIs, I had around a dozen instances of data corruption.
> Previously when using the alestic.com images, and now that I've
> switched back to them, I haven't seen any corruption.  Is there any
> known issue this could be related to?
>
> I use machines by raiding together the disks via RAID 0, and then
> installing ReiserFS on top of that.  The list of commands I use to do
> this is at the bottom of this email.  The workload of the machines
> changes somewhat, but generally it maxes out both of the cores, use
> around 15MB/s of disk throughput (split even read/writing), and has
> the disks around 60% full.  Our data is always compressed on disk
> (LZO), and we have checksums every 64KB of uncompressed data.  What I
> would see is that at a seemingly random point in a file the checksum
> wouldn't match, although the checksums for the rest of the file both
> before and after this point would be fine.  I didn't notice anything
> unusual in the system logs.
>
>
>  Thanks for the detailed information, that may help to narrow the search
> for the problem substantially.
>
> Is ReiserFS integral to the solution, or a personal preference? It jumped
> out at me as an area of risk, as it's not a filesystem we're particularly
> focused on. Ext3, and the newer ext4 and ultimately btrfs would be the
> "stable, next, future" default filesystems we'd recommend unless there was a
> specific technical reason to do otherwise. If Reiser isn't integral I'd be
> interested in your results with ext3, both performance and stability wise.
>
> There are different kernels, as I understand it, between the Alestic
> images. Chuck and Eric will be able to say in detail but AIUI the beta AMI's
> use newer kernels, which bring some benefits but are also quite possibly the
> source of new gotchas. You may have triggered one of those.
>
> Mark
>
> --
> Ec2-beta mailing list
> Ec2-beta at lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/ec2-beta
>
>


-- 
John O. Hampton, Jr., CTO
CleanOffer, Inc.
101 California Street
Suite 2450 #612
San Francisco, CA  94111

Homepage: http://www.cleanoffer.com
Blog: http://blog.cleanoffer.com

(415) 240-4532 (office)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://lists.ubuntu.com/mailman/private/ec2/attachments/20090414/518f4511/attachment-0002.htm 


More information about the Ec2-beta mailing list