[Bug 1160490] Re: race condition updating statefile

[Ray Link]

It can happen much more frequently than that.

I have three machines in particular that do weekly reboots.  They all
have two network interfaces, but only one of them is used.  Every week,
one or more of them will lose on an interface that matters (loopback or
the used network interface), and sometimes it will continue to fail
through several subsequent manual reboots before it comes back up again.

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to ifupdown in Ubuntu.
https://bugs.launchpad.net/bugs/1160490

Title:
  race condition updating statefile

Status in “ifupdown” package in Ubuntu:
  Confirmed

Bug description:
  Ubuntu 12.04.2
  ifupdown 0.7~beta2ubuntu8

  Symptom: Every so often, /etc/init/network-interface.conf fails to
  bring up the loopback interface.

  > Mar 25 16:39:37 XXXXXXXX kernel: [   28.793922] init: network-
  interface (lo) pre-start process (1079) terminated with status 1

  /var/log/upstart/network-interface-lo shows:

  > ifup: failed to overwrite statefile /run/network/ifstate: No such
  file or directory

  Relevant section of the ifup sources, in update_state():

          if (rename (tmpstatefile, statefile)) {
                  fprintf(stderr,
                          "%s: failed to overwrite statefile %s: %s\n",
                          argv0, statefile, strerror(errno));
                  exit (1);
          }

  update_state() opens the statefile, gets a F_SETLKW lock on it, opens
  a tmpfile, filters the contents of the statefile into the tmpfile,
  closes the tmpfile, then renames the tempfile over the statefile.

  Once the rename() happens in one instance of ifup, any other blocked
  instances are waiting around to lock a file that no longer exists in
  the filesystem.  Overlap enough instances of ifup just right and you
  have them all locking different copies of statefile, which then
  doesn't prevent any of them from rename()ing tmpstatefile out from
  underneath the others, thus causing their own rename()s to fail with
  ENOENT.

  Example:

  Process A starts, opens statefile.
  Process A locks statefile. 
  Process B starts, opens statefile.
  Process B waits for lock on statefile.
  Process A renames tmpstatefile to statefile and exits.
  Process B acquires lock on *outdated* statefile FILE pointer.
  Process C starts, opens current statefile (written by Process A).
  Process C locks current statefile.
  ** Two ifups now have locks **
  Process B renames tmpstatefile to statefile and exits.
  Process C tries to rename tmpstatefile, fails because tmpstatefile has already been renamed out from under it by Process B.

  NOTE:  Since Process B was operating on an outdated statefile, it has
  also stomped over any changes made by Process A, so simply making the
  tmpstatefile process-specific to avoid rename()ing out from under each
  other won't help.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/ifupdown/+bug/1160490/+subscriptions