Boot charting

[Phillip Susi]

David Nielsen wrote:
> 
> As a sidenote to this the upcoming Reiser4 filesystem is able to use a
> repacker (same thing as defragmentation tool really), so I wouldn't be
> surprised if we would have this functionality in the near future.
> Historically our filesystems have laid out data in a near optimal way so
> defraging hasn't been a problem untill now, and it always posed a
> problem being a slow cumbersome process - I'm my impression that the
> Reiser4 repacker will run little by little in the background using only
> idle time to repack the disk so this wouldn't even be felt by the user
> as compared to the full defrag tools someone like Norton would provide
> for Windows.

I actually wrote the maintainer of libreiser to ask if he thought it 
would be possible to write a utility using it to do that kind of 
repacking.  His reply was no, libreiser does not provide the required 
functionality, but libreiser4 does.  Unfortunately, reiser4 is caught up 
in politics at the moment and it doesn't look like it will make it into 
dapper.

Tristan Wibberley wrote:
> Or it could be single-threaded, so a read is issued, then completes,
> such that the head is in the wrong place for the next read, which is
> subsequently issued, leaving the head in the wrong place for the next,
> etc. If so, then using asynchronous mechanisms could provide a
> performance boost (the speed I'm getting for readahead-list seems far to
> small for the normal level of fragmentation I expect to have). If it
> already *is* asynchronous then playing with the kernel tunables before
> running readahead and resetting them afterwards could help by extending
> the I/O queue length and reducing the fairness to allow the reads to be
> ordered better.

I have been a big proponent of async IO since 1998, when I wrote an ftp 
server on NT using zero copy overlapped IO and IO completion ports and 
was blown away by network throughput I never dreamed was possible using 
virtually zero cpu time.  I actually modified dd the other day to use 
linux native async IO in conjection with O_DIRECT and the results were 
very positive.  In this case though, it is not really appropriate.

Async IO's main advantage is that you can carry out a very large amount 
of IO with many different targets with very little cpu overhead. 
Because of readahead mechanisms, conventional blocking IO is capable of 
basically just as much throughput for sequantial IO, even if it costs a 
good deal more cpu cycles.  In the case of readahead-list, it does not 
directly read but instead asks the kernel to precache the file with the 
readahead() system call, which it does quite efficiently.

This is why I am reasonably sure the problem is simply disk seeking 
between each file as they are scattered randomly around the disk, and a 
number of them are actually rather small.

> 
> I don't have more experience (none whatsoever), but that looks really
> interesting. Could you send me details of what you did, and any diffs
> against stock scripts? I'd like to see this and have a play.
> 

I actually didn't even modify any scripts.  I just asked grub to pass 
the break=init parameter to the kernel to have the initramfs drop me to 
a shell just before passing to the real root filesystem.  I then mounted 
the root filesystem in /root, loaded the loop, cramfs, and unionfs 
modules, configured loop0 to connect to /root/readahead.cramfs and 
mounted loop0 in /readahead.  Then I did a "mount -t unionfs -o 
dirs=/readahead=ro:/root=rw none /real-root" to mount the root 
filesystem with the readahead cramfs overlaid on top in /real-root. 
Then I executed run-init with /real-root in place of /root, and then 
there were a bunch of decompression errors I think from cramfs, then the 
kernel paniced because init terminated.  I think init was loading some 
libraries in the readahead cramfs.

> 
> The biggest problems I see during boot (before gdmgreeter goes quiet)
> are modprobe in the initramfs and elsewhere, hwclock, hald-probe-volu
> (whatever that is short for), Xorg and gdmgreeter startup, or the fact
> that things wait for others where they shouldn't need to. In most cases
> they *do* need to but ought not - that requires some big surgery to
> packages to fix it properly which I guess is certainly *not* going to
> happen for dapper.
> 

Yea, I also noticed the modprobe and things like X waiting for all the 
hardware detection to finish before starting.  I guess it needs the 
mouse and whatnot to be detected before it can start, but it would be 
nice if it could get started sooner, then connect to the mouse once it 
is detected.  I don't even see gdmgreeter listed in my bootchart at all 
though.