Random access read speed from encrypted filesystem is very slow compared to other linux distributions

[Yakov Hrebtov]

Hello!

I've reported this issue on 2009-04-19 at the bug tracking system 
(https://bugs.launchpad.net/bugs/363763), but have not received any feedback.
So I decided to repost it here. Sorry, if it's a wrong place.

I have noticed that random-access reads from file located on encrypted device 
performs very slow -- 2 times slower compared with CentOS 5.3 distribution. 
There are very high IO wait values during the test.
I have reproduced this issue on different hardware with several ubuntu server 
kernels:
  - several 8.04 kernels (last one tested is  2.6.24-23-server)
  - 9.04 RC kernel

I wrote a simple script that creates a 100M file, performs "random-access" 
reads from it, and measures the time spent for reading all the file.
The script is attached (http://launchpadlibrarian.net/25755426/rand_access.sh).

How to reproduce the issue:
1. Create a mapping for encrypted device
    cryptsetup --cipher=aes-cbc-essiv:sha256 --key-size=256 --hash=ripemd160 
create cfs1 /dev/sdb
2. Create filesystem
    mkfs.ext3 /dev/mapper/cfs1
3. mkdir /mnt/test; mount /dev/mapper/cfs1 /mnt/test
4. copy attached script to /mnt/test;
5. Run the script:
    cd /mnt/test; ./random_access.sh
6. compare results with other distributions.

My results are:
1. ubuntu 2.6.24-23-server:
     not encrypted fs: 20 seconds
     encrypted fs: 80 seconds
2. centos 2.6.18-128.el5PAE (similar results on several fedora kernels)
     not encrypted fs: 20 seconds
     encrypted fs: 40 seconds
As you see, ubuntu is two times slower in case of encrypted device :-(

I began to discover this issue because we expecting problems working with 
database located on encrypted device.

The problem seems to appear only if the file is not in a system cache.

I'm ready to provide any additional info that can help to solve an
issue.

Thank you.