Our Networking Story

[Robie Basak]

On Thu, Mar 06, 2014 at 02:48:52PM -0500, Bryan Quigley wrote:
> We've had a lot of internal Canonical discussions about our networking
> story and before going to a UDS session [1] it was suggested to post to
> ubuntu-devel.
> 
> *Network Restart*
> I'd like to start by asking each of you what you think is the correct way
> to restart networking on Ubuntu server?  Feel free to write it down and
> include it in any replies :).

I don't think it's possible to define "restart networking" in a way that
everybody understands, and in a way that will always work on complex
configurations.

Instead, I think that we need to focus on *why* you need to "restart
networking".

Did you change an interface entry in /etc/network/interfaces{,.d}? Then
ifdown before, and ifup after. Or do others disagree?

Something more complicated? Then I guess we need to consider your
specific use cases. What are they, please?

This reminds me of this bug:
https://bugs.launchpad.net/juju-core/+bug/1248283

Daemons can't be expected to continue working if they are (for example)
listening on an interface that disappears. This is why "restart
networking" is difficult. We could say that they should do so and patch
them all to add handling of a "network is down" state, or we could
adjust all upstart jobs to automatically stop and start daemons as
required as networking is "restarted". But I'm not sure this makes
sense.

> It turns out our documentation has been wrong and the following are not
> correct and more importantly don't work consistently over 10.04/12.04/14.04
> [2]:
> sudo /etc/init.d/networking restart
> sudo restart networking
> 
> The correct way I've been told is to use the ifupdown scripts. It's
> important to note that this is different on the desktop due to
> network-manager.
> 
> I feel we need to publicly discuss if we really want the ifupdown scripts
> to be the only supported way to manage/restart networking.  We've been
> communicating the opposite for quite some time now..

Thank you for bringing this up. It's certainly important if our
documentation is wrong, or we're short of clarity on the process. I
think we should certainly have a blueprint to assign work items to
ensure that all our understanding and documentation is consistent.
Exactly what would we discuss in a session, though, that we can't do on
a mailing list? Is there any controversy here that we need to resolve?

> Related question:
> Do we not support giving users the ability to restart networking equivalent
> to rebooting the system?  (Upstart is used when booting, not when manually
> doing the ifupdown scripts).

I'd say no, because "restarting networking" as a general concept isn't
well-defined and unsupported by various components that depend on it
being there. However, I would certainly like to see particular
operations work smoothly without requiring a reboot. Are there specific
cases that do not work well here?

> *More complicated network setups*
> There are many bugs in regards to bonds/vlans/bridging and other more
> complex networking setups.  It appears like it might be a limitation to how
> ifupdown is designed.
> 
> We have had cases where the MTU needs to be set using a pre-up or post-up
> option in the interfaces file instead of a plain MTU line.
> Bond interfaces can cause significant pausing in boot/network restart
> The ifupdown script doesn't actually work on bonded interfaces [3]
> 
> race condition updating statefile "sometimes networking interfaces won't
> come up" - was fixed [4]-
> 
> We are seeing many more of cases involving complicated networking setups
> and with more OpenStack deployments this is going to become more of the
> norm.

It would be great to see some effort on prioritising these bugs, and
driving them (presumably in Debian also?) as part of a blueprint.

> My understanding is that ifupdown was not designed to handle a parallel
> boot process like Upstart or systemd.  I'm guessing there are a lot more
> bugs lurking due to that, aside from some other issues with the codebase
> [5].

What specific solutions are being proposed here?

Robie
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: <https://lists.ubuntu.com/archives/ubuntu-devel-discuss/attachments/20140307/5525ffa9/attachment.sig>