event based initramfs

Wed May 16 13:01:18 UTC 2012

On 11/05/12 04:54, Phillip Susi wrote:
> I was a bit confused when I saw this uds event.  It is unclear to me why we would want to add upstart to the initramfs.
The most obvious reason is that the initramfs has it's own init system
(a big shell script). So, an Ubuntu system with an initramfs has two
different init systems. By reducing this down to just Upstart, we get a
simpler and hence more readily maintainable system that:

- has the advantage of being event-based (but which of course can also
run jobs effectively "serially" too if desired).
- mirrors the main system thus making the initramfs less inscrutable (a
good thing IMHO :-)

 I think it is important to remember that the the whole purpose of the
initramfs is to help the kernel find the root fs, so things that aren't
needed to access the root fs really shouldn't be there,
That's not entirely true - we also need additional functionality to
provide graphical boot splash which is now an expectation and in fact
users are asking for more (see [1]).

lest it add bloat and slow down booting.
The majority of "bloat" in the initramfs (~60%) is in fact the kernel
modules. Adding upstart (upstart, mountall, libnih) consumes an
additional ~1.6MB (approximately a 3% increase).

After listening to the audio of the session, it seems like there are a
few issues people are trying to address with upstart in the initramfs:
> 
> 1)  Want to fix bugs with mdadm/luks
> 
> I don't see what this has to do with upstart at all.  If your root is encrypted, then you must prompt for the password before you can continue, so there is no reason to do this event driven rather than procedural.
LUKS support in the initramfs is currently handled using calls to "sleep
0.1". Who chose this value? How did they decide upon this value? Why 0.1
seconds? Why not just wait until the device appears? By having an
event-based system, we can adopt a saner approach where the system
simply reacts to the udev event when the kernel generates it - no
polling required.

> 
> 2)  Want to get boot splash going asap
> 
> Wouldn't it be better to just get through the initrd faster, or skip it entirely ( as discussed at uds-p )?
Dropping the initramfs is not an option for all cases (See [2]).

> If you're spending less than 2-3 seconds in the initrd anyhow, it seems counterproductive to make it larger/slower to display the splash screen now instead of after switching to the real root.
> 
> If there is still an appreciable time between the point where plymouth loads and where you can actually login where it might make sense to continue showing the splash screen instead of having X control the display, maybe we could let X initialize in the background and switch from the plymouth tty to the X tty only once it is ready for you to login.
> 
> 3)  Want to do friendly recovery when the root fs can't be mounted
> 
> I'm not sure putting recovery in the initrd is worth the bother given the rather small number and rarity of failures that can prevent finding the root fs, but if you do want to do this, it certainly would need to be in an alternate initrd rather than bloating the primary initrd with significant weight that is not needed the vast majority of the time.  Rather than putting it in an initrd, it may make more sense just to have a separate recovery partition to boot instead of the usual root fs.
I'm confused by this comment - we want to handle failure gracefully and
-- crucially -- "fail fast". The whole point is that *all* failure
scenarios should be handled as best we can. Just because a failure might
be rare does not make it any less important to handle IMHO.

> 4)  Want to move mountall to initramfs to support separate /usr
> 
> This requires a copy of /etc/fstab in initramfs, and a flag to mountall to only mount / and /usr read-only, then exit.  I don't get why anyone would want /usr on a different filesystem in the first place and therefore, why we should spend effort on supporting this.
I've worked on many servers that have different partitions for most
top-level directories. Ubuntu is a general-purpose O/S, so should
support such use cases where possible.

What does having upstart and mountall in the initramfs add to this, as
opposed to just adding a second call to mount to the existing init scripts?
My non-LVM, non-RAID systems initramfs contains 16 calls to mount. By
using mountall you end up with a single mountall.conf job that handles
all that complexity.

In the old days, the differentiation was that you could at least boot to
single user mode and try to do some recovery without /usr.  What is it
that we want to be able to do in rescue mode that requires things from
/usr?  If there is a good case for having them in a rescue mode, then
maybe they shouldn't be in /usr in the first place?
A number of upstream projects are starting to move functionality from /
to /usr (see [3]), hence we intend to support this model too.

Additional reasons for supporting an event-based initramfs:

- By utilizing Upstart, we get a clean, fast framework for mounting the
root FS. We should also be able to simplify the initramfs by removing
duplicate code.
- the initramfs spawns daemons (such as udevd) which can be better
managed via upstart.
-  the current initramfs kills the daemons it starts requiring them to
be restarted in the main system. This has caused a few race conditions
and clearly disallows state passing between the two systems. By
enhancing Upstart to support stateful re-exec [4] we get huge benefits
as we can retain state on processes started in the initramfs and hence
don't necessarily need to restart them. I don't know of any other system
that is capable of this.
- there is potential for extra "smarts" that cannot be achieved
easily/cleanly with the current shell approach.

"Phase 1" is to add the ability to generate an event-based initramfs.
The plan is to make this "opt in" so we can encourage users to try it
(they can always revert) and gather user feedback since clearly there
are a number of different boot scenarios to consider. Additionally, we
will also need to consider carefully the initramfs size and performance
impacts of any potential changes.

Kind regards,

James.
--
James Hunt

[1] - http://brainstorm.ubuntu.com/?keywords=boot
[2] - http://summit.ubuntu.com/uds-q/meeting/20296/foundations-q-usr-merge/
[3] - https://blueprints.launchpad.net/ubuntu/+spec/foundations-q-usr-merge
[4] -
https://blueprints.launchpad.net/ubuntu/+spec/foundations-q-upstart-stateful-re-exec

____________________________________
http://upstart.ubuntu.com/cookbook
http://upstart.ubuntu.com/cookbook/upstart_cookbook.pdf

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 900 bytes
Desc: OpenPGP digital signature
URL: <https://lists.ubuntu.com/archives/ubuntu-devel/attachments/20120516/b22c3586/attachment.pgp>