ureadahead improvements ( was: Readahead Maverick )

[Scott James Remnant]

Thanks Phillip,

I'll certainly take a look at this!

Scott

On 24/12/10 03:46, Phillip Susi wrote:
> This conversation died earlier this year.  I decided to rebase my 
> changes on natty and test again and I thought I would post the results 
> on the mailing list and encourage testing this time.  If Scott can 
> find time to review the changes, they are in 
> lp:~psusi/ubuntu/natty/ureadahead/mine.  Anyone interested in testing 
> can find the package in my PPA.
>
> I ran a few boot time trials with a clean install of today's natty 
> build on an lvm partition on my 1.5tb wd green drive.  The results were:
>
> Initial baseline: 17.01 seconds
> With my ureadahead: 15.33 seconds
> After defrag: 15.82 seconds
> Back to stock ureadahead: 16.25 seconds
>
> I defragmented ( with e2defrag ) the drive giving packing priority to 
> the files in the ureadahead list.  I am attaching before and after 
> graphs of the blocks made with fragraph.py ( in the upstream 
> ureadahead contrib directory ).  You can see that it looks better, and 
> it did result in ureadahead finishing in 3 seconds instead of 5, but 
> for some reason the overall boot took slightly longer.
>
> On 07/30/2010 04:44 PM, Phillip Susi wrote:
>> On 7/28/2010 10:15 AM, Scott James Remnant wrote:
>>> Right, this is the kind of thing that only works when you have
>>> explicitly laid the disk out in the right way, and is completely ruined
>>> as soon as any part of the disk is changed (e.g. packages installed).
>>>
>>> I've avoided relying on directories always being in depth order on the
>>> disk for the time being, because without a reprofile and defrag after
>>> every boot, it drifts over time and the performance penalty can be
>>> terrible.
>>
>> I had made similar modifications to do a two pass read getting the
>> directories first and found that it caused a very slight slowdown
>> without a defrag to place the directories before the normal files.  Once
>> that was done though it made a good improvement.  The unmodified
>> ureadahead spends about 1 full second in the open() loop with almost
>> zero disk throughput as it fetches a single 4kb block from the disk at a
>> time walking the directory tree.  Doing two passes over the disk is
>> obviously less desirable than one, but the directory pass would have to
>> take>  1 second to result in a net loss, which seems unlikely.
>>
>>> On an extents-based filesystem, reading the inodes will be entirely
>>> separate to reading the contents.  So you'll incur twice as many passes
>>> as on a non-extents-based filesystem.
>>
>> The currently live ureadahead already does this though; it preloads the
>> inode table with calls to e2fslibs.  My version slightly modifies this
>> to use readahead() to load the table without copying it to user space,
>> rather than calling e2fslibs which ends up using read() into a user
>> space buffer, and reads the next inode table beyond the one that
>> actually has relevant inodes before stopping.
>>
>>> (ext3 converted to ext4 is *not* extents based for files that 
>>> previously
>>> existed - they must be rewritten)
>>
>> Or you can chattr +e.
>>
>