[Bug 1629972] Re: networking stop incorrectly disconnects from (network) root filesystem

[Scott Moser]

It seems that the issue here is networking.service does:
  /sbin/ifdown -a --read-environment
but it should do:
  ifdown -a --exclude=lo

As is seen in the equivalent upstart job (/etc/init/networking.conf).

$ m="ifdown -a --read-environment";
$ sed -i "s,$m, --exclude=lo," /lib/systemd/system/networking.service
$ systemctl daemon-reload
$ sudo poweroff

Demonstration below.

## http://paste.ubuntu.com/23319833/
## ipv6 networking is broken by
##    ip link set down dev lo
##
## This seems to be the root cause of bug 1629972 (http://pad.lv/1629972)
## where system networking (networking.service) is brought down on
## shutdown and causes hang.
##
## Note, that it is the link that is the issue, there is actually
## no problem seen in 'ping6' by either:
##    ip addr del 127.0.0.1/8 dev lo
##    ip addr del ::1/128 dev lo

$ for n in y1 y2; do lxc delete --force $n; lxc launch ubuntu-daily:yakkety $n; done
error: not found
Creating y1
Starting y1
error: not found
Creating y2
Starting y2

$ for n in y1 y2; do lxc exec $n -- dhclient -6 -1 -v eth0; done

$ lxc list y[12]
+------+---------+----------------------+------------------------------------------------+------------+-----------+
| NAME |  STATE  |         IPV4         |                      IPV6                      |    TYPE    | SNAPSHOTS |
+------+---------+----------------------+------------------------------------------------+------------+-----------+
| y1   | RUNNING | 10.75.205.102 (eth0) | fd42:eee5:7c43:3d62:c7fd:6ccc:6181:8d48 (eth0) | PERSISTENT | 0         |
+------+---------+----------------------+------------------------------------------------+------------+-----------+
| y2   | RUNNING | 10.75.205.5 (eth0)   | fd42:eee5:7c43:3d62:f285:69dd:7210:c0a5 (eth0) | PERSISTENT | 0         |
+------+---------+----------------------+------------------------------------------------+------------+-----------+

$ for n in y1 y2; do
   lxc exec $n -- ip route add fd42:eee5:7c43:3d62::0/64 dev eth0

$ lxc exec y1 --  ping6 fd42:eee5:7c43:3d62:f285:69dd:7210:c0a5
PING fd42:eee5:7c43:3d62:f285:69dd:7210:c0a5(fd42:eee5:7c43:3d62:f285:69dd:7210:c0a5) 56 data bytes
64 bytes from fd42:eee5:7c43:3d62:f285:69dd:7210:c0a5: icmp_seq=1 ttl=64 time=0.133 ms
64 bytes from fd42:eee5:7c43:3d62:f285:69dd:7210:c0a5: icmp_seq=2 ttl=64 time=0.061 ms
64 bytes from fd42:eee5:7c43:3d62:f285:69dd:7210:c0a5: icmp_seq=3 ttl=64 time=0.069 ms
...


<window 2>
$ lxc exec y1 -- ip link set down dev lo


immediately ping6 starts dropping packets in window1.


** Changed in: ifupdown (Ubuntu)
     Assignee: (unassigned) => Scott Moser (smoser)

** Changed in: ifupdown (Ubuntu Xenial)
     Assignee: (unassigned) => Scott Moser (smoser)

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to ifupdown in Ubuntu.
https://bugs.launchpad.net/bugs/1629972

Title:
  networking stop incorrectly disconnects from (network) root filesystem

Status in MAAS:
  Triaged
Status in ifupdown package in Ubuntu:
  Confirmed
Status in ifupdown source package in Xenial:
  Confirmed

Bug description:
  With the switch to systemd, all support for iscsi root (and other)
  filesystems disappeared, since shutdown yanks the rug out from under
  us.

  Rather than just relying on /etc/iscsi/iscsi.initramfs (which d-i
  creates..), the DEV check should be expanded to include iscsi devices,
  and networking.service ExecStop should honor those checks.

  Related bugs:
    * bug 1229458: grub2 needed changes
    * bug 1621615: network not configured when ipv6 netbooted into cloud-init
    * bug 1621507: ipv6 network boot does not work

  [Impact]

  With the changes from the above, the iscsi root (at least in the ipv6
  case) gets disconneceted prior to clean shutdown (ifdown downs the
  interface), resulting in a failure to enlist, commission, or deploy
  cleanly under MAAS. (and a failure to cleanly unmount the root
  filesystem when it is over iscsi.)

  [Test Case]

  Given a MAAS 2.0 installation, and the packages in the other bugs,
  attempt to enlist, commission, or deploy a host with xenial.

  [Regression potential]

  This restores the pre-xenial behavior of not shutting down the
  interface if there are network drives at the time that neworking is
  stopped (making it a no-op.)  The additional change is to detect
  "/dev/disk/by-path/*-iscsi-*" as a network disk, replacing the check
  for the existence of /etc/iscsi/iscsi.initramfs, which was only
  created by debian-installer (and maas until recently).

To manage notifications about this bug go to:
https://bugs.launchpad.net/maas/+bug/1629972/+subscriptions