[v2] [PATCH 0/2] Populate rootfs asynchronously and early

[Andy Whitcroft]

On Tue, Dec 08, 2009 at 11:44:33AM +0200, Surbhi Palande wrote:
> These patches populate the rootfs asynchronously and early to improve the 
> boot performance. They are tested against a 2.6.32 kernel on KVM and Dell
> Mini 10v. On a  Dell Mini 10v, it was observed that the boot performance 
> improved by 0.35 s with the 1st patch and by 0.37 s with the 1st and
> the 2nd patch together. 
> 
> The 2nd patch starts the population of the rootfs as soon as the cores are 
> up and the asynchronous calls are available for use. This patch might reduce 
> the boot time, but this needs verification. On a Dell Mini 10v, the 2nd patch
> gave an additional boot performance improvement of only 0.2s.

I put these patches to some tests in combination with an additional patch
of my own to make isapnp_init asyncronous.  All test kernels are from
the same base lucid kernel version for comparability.  Three patches
are tested:

  async_populate_rootfs: an early version of 1/2 here which pushed
    populate_rootfs out to an sync function but leaves it in the original
    start location,
  async_isapnp: which pushed the isapnp_init call out of line to async,
    again in the original location, and
  earlyrootfs: which represents 2/2 here which pushes the start of the
    initramfs decode as early as possible.

The combinations tested are as below:

  test1: virgin lucid kernel,
  test2: virgin + async_populate_rootfs,
  test3: virgin + async_populate_rootfs + async_isapnp,
  test4: virgin + async_populate_rootfs + async_isapnp + earlyrootfs.

These were then tested on three machines, the reference Dell Mini 10v
(ssd), a Dell 1537, and an amd64 uni-processor system.  The link below
points to some comparitive graphs for these tests:

    http://people.canonical.com/~apw/boot-speed/index.html

The overall kernel initialisation time can be seen in the table below.
It should be noted that the variance in results is reasaonably high so
these figures should only be considers in broad terms:

          |  Dell 10v  | Dell 1537  |  amd64-up  |
  | test1 |    1.48    |    1.34    |    1.95    |
  | test2 |    1.11    |    1.15    |    1.75    |
  | test3 |    0.87    |    1.16    |    1.75    |
  | test4 |    0.87    |    1.17    |    1.95    |

What is clear is that making populate_initramfs async is worth a good
deal of time, as is making isapnp_init async.  Moving the init of the
initramfs earlier is a basically worth nothing on SMP boxes and a severe
negative on UP systems.

>From these numbers I would recommend that we add the async_populate_rootfs
and async_isapnp patches for Lucid, and keeping the earlyrootfs in our
back pocket; likely we would need to make it conditional on the machine
being SMP for it to be of use and only then should the rest of kernel
init mean we are waiting on this.

Caveat: it should also be noted that the isapnp_init being async has not
been tested with any actual ISA devices present, and would require testing.

Finally anyone wishing to perform these tests on a machine of interest,
perhaps one with ISA, can get the kernels used for this testing at the
URL below.  Please boot each of the four kernels with these additional
kernel command line options 'initcall_debug printk.time=1' and collect
a dmesg output immediatly after boot, send all four to me and I will add
them to the results.  Test kernels are here:

    http://people.canonical.com/~apw/async-init-lucid/

-apw