current access to file systems

Peng Yu pengyu.ut at
Sat Jun 19 15:59:27 UTC 2010


I'm seeking for a solution in ubuntu to address the concurrent disk
access problem that are described below. I did a google search but
there are just too many information out there and I don't which is the
best for my situation.

I have one machine (8 CPUs and 40GB RAM) with 4 slots that can be
plugged in 4 local disks. Right now, there is one 2T disk (xfs
filesystem) and a 140GB disk plugged in. I run a speed test using the
gnu coreutils split command to split 8 500MB files (each into 10
smaller files), either in serial or in parallel (8 process running
simultaneously). Both run finish in about 15 sec, which indicates a
speed of about 270MB/s (read and write the same disk). The test is
performed on both on 2T and on 140GB disks.

However, I run 6 other 3rd party program simultaneously (I don't know
how the program handle disk buffering), iotop tells me that each
consume a bandwidth of only a few handred KB/s (both read and write on
2T disk). The 6 processes takes 100% CPU time each, which means that
they are CPU bound rather than disk bound. While they are running, I
start the split test program on the 2T disk. The speed now drops to
only a MB/sec. But if I run the split test program on 140GB disk. the
speed is about 80-90MB/sec.

This clearly indicated that the main bottleneck is on the 2T disk. So
far, I think of 2 solutions.

1. increase the buffers, which involves changing the kernel.
2. using a cluster filesystem, which has several disks to balance the
load on each disk.

But I'm not sure how much performance gain I can get from either
solutions? And how much time it need for a novice administrator? If
one or both solutions are viable, what detailed steps I need to do?

NOTE, in the future I may need to expand the system from just one
machine to multiples machines, which all access the same storage
space. When all the machines with many processes on each machine
access the same storage simultaneously, I'd think that cluster file
system probably is the solution that need to be adopt. But please let
me know if you have any better solutions.


More information about the ubuntu-users mailing list