[Bug 1636912] Re: systemd-networkd runs too late for cloud-init.service (net)

[Martin Pitt]

> I really want to avoid having different cloud-init in any of
xenial/yakkety/zesty.

Me too, and I don't see why this would be conceptually required? 'After
=systemd-networkd.service" is appropriate in all releases (as that's
what you intend to do), after (!) we drop the After=dbus.service from
networkd. Simplifying the "Before=basic.target dbus.socket" to
"Before=sysinit.target" is something that should be done in y/z and then
cleanly backports to x as well.

> I've used After and Requires; but these are focused on when the units starts
rather than when we can expect networking to be up.

Correct. You can't use network-online.target in early boot, that would
be a too strong requirement (as e. g. NetworkManager implements this as
well). In early boot there can't ever be a *guarantee* that networking
works, so the best that you can do is to wait a little bit if you get a
default route, e. g. with /lib/systemd/systemd-networkd-wait-online. Of
course in desktop or snappy systems you might never get a connection
even in late boot, so there need to be some sensible timeouts.

> oddly though when using the ifupdown 'networking.service'; we don't
need to use that target.

Yes, that's a Type=oneshot, as it just calls "ifup -a". So that's more
or less equivalent to s-n-wait-online --timeout=30 or After=s-n-wait-
online.service. But the latter would block the entire boot process for
that long if there is no network (and this *did* hit us in snappy
already, like bug 1431836) -- my gut feeling is that this can be handled
more gracefully/asynchronously in code.

> networkd bringing up eth0 (virtio) on qemu user-net is taking like 40
seconds... why?

That's certainly unusual, it should only take ~ 5s or so. I suggest
filing a separate bug for that with your precise config (my suspicion is
that you enabled DHCPv6 or similar and it's waiting/timing out for that,
or something similar).

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to systemd in Ubuntu.
https://bugs.launchpad.net/bugs/1636912

Title:
  systemd-networkd runs too late for cloud-init.service (net)

Status in systemd:
  Unknown
Status in cloud-init package in Ubuntu:
  Triaged
Status in systemd package in Ubuntu:
  Triaged
Status in cloud-init source package in Xenial:
  New
Status in systemd source package in Xenial:
  Triaged

Bug description:
  Ubuntu Core 16 images using cloud-init fail to function when the
  DataSource is over the network (Like OpenStack) as networking is not
  yet available when cloud-init.service runs.

  cloud-init service unit deps look like this:

  [Unit]
  Description=Initial cloud-init job (metadata service crawler)
  DefaultDependencies=no
  Wants=cloud-init-local.service
  Wants=local-fs.target
  Wants=sshd-keygen.service
  Wants=sshd.service
  After=cloud-init-local.service
  After=networking.service
  Requires=networking.service
  Before=basic.target
  Before=dbus.socket
  Before=network-online.target
  Before=sshd-keygen.service
  Before=sshd.service
  Before=systemd-user-sessions.service
  Conflicts=shutdown.target

  Here's networkd unit deps:

  [Unit]
  Description=Network Service
  Documentation=man:systemd-networkd.service(8)
  ConditionCapability=CAP_NET_ADMIN
  DefaultDependencies=no
  # dbus.service can be dropped once on kdbus, and systemd-udevd.service can be
  # dropped once tuntap is moved to netlink
  After=systemd-udevd.service dbus.service network-pre.target systemd-sysusers.service systemd-sysctl.service
  Before=network.target multi-user.target shutdown.target
  Conflicts=shutdown.target
  Wants=network.target

  # On kdbus systems we pull in the busname explicitly, because it
  # carries policy that allows the daemon to acquire its name.
  Wants=org.freedesktop.network1.busname
  After=org.freedesktop.network1.busname

  
  And a critical-chain output:

  root at snap-test7:~# systemd-analyze critical-chain systemd-networkd
  Failed to get ID: Unit name systemd-networkd is not valid.
  The time after the unit is active or started is printed after the "@" character.
  The time the unit takes to start is printed after the "+" character.

  root at snap-test7:~# systemd-analyze critical-chain systemd-networkd.service
  The time after the unit is active or started is printed after the "@" character.
  The time the unit takes to start is printed after the "+" character.

  systemd-networkd.service +440ms
  └─dbus.service @11.461s
    └─basic.target @11.403s
      └─sockets.target @11.401s
        └─dbus.socket @11.398s
          └─cloud-init.service @10.127s +1.266s
            └─networking.service @9.305s +799ms
              └─network-pre.target @9.295s
                └─cloud-init-local.service @3.822s +5.469s
                  └─local-fs.target @3.813s
                    └─run-cgmanager-fs.mount @12.687s
                      └─local-fs-pre.target @1.393s
                        └─systemd-tmpfiles-setup-dev.service @1.116s +195ms
                          └─kmod-static-nodes.service @887ms +193ms
                            └─system.slice @783ms
                              └─-.slice @721ms

  
  cloud-init would need networkd to run at or before 'networking.service' so it can raise networking to then find and use network-based datasources.

  # grep systemd /usr/share/snappy/dpkg.list 
  ii  libnss-resolve:amd64          229-4ubuntu11                                            amd64        nss module to resolve names via systemd-resolved
  ii  libpam-systemd:amd64          229-4ubuntu11                                            amd64        system and service manager - PAM module
  ii  libsystemd0:amd64             229-4ubuntu11                                            amd64        systemd utility library
  ii  systemd                       229-4ubuntu11                                            amd64        system and service manager
  ii  systemd-sysv                  229-4ubuntu11                                            amd64        system and service manager - SysV links

  # grep cloud-init /usr/share/snappy/dpkg.list 
  ii  cloud-init                    0.7.8-201610260005-gf7a5756-0ubuntu1~trunk~ubuntu16.04.1 all          Init scripts for cloud instances

To manage notifications about this bug go to:
https://bugs.launchpad.net/systemd/+bug/1636912/+subscriptions