Our Networking Story

[Scott Kitterman]

On Thursday, March 06, 2014 14:48:52 Bryan Quigley wrote:
> We've had a lot of internal Canonical discussions about our networking
> story and before going to a UDS session [1] it was suggested to post to
> ubuntu-devel.
> 
> *Network Restart*
> I'd like to start by asking each of you what you think is the correct way
> to restart networking on Ubuntu server?  Feel free to write it down and
> include it in any replies :).
> 
> It turns out our documentation has been wrong and the following are not
> correct and more importantly don't work consistently over 10.04/12.04/14.04
> [2]:
> sudo /etc/init.d/networking restart
> sudo restart networking
> 
> The correct way I've been told is to use the ifupdown scripts. It's
> important to note that this is different on the desktop due to
> network-manager.
> 
> I feel we need to publicly discuss if we really want the ifupdown scripts
> to be the only supported way to manage/restart networking.  We've been
> communicating the opposite for quite some time now..
> 
> Related question:
> Do we not support giving users the ability to restart networking equivalent
> to rebooting the system?  (Upstart is used when booting, not when manually
> doing the ifupdown scripts).
> 
> 
> *More complicated network setups*
> There are many bugs in regards to bonds/vlans/bridging and other more
> complex networking setups.  It appears like it might be a limitation to how
> ifupdown is designed.
> 
> We have had cases where the MTU needs to be set using a pre-up or post-up
> option in the interfaces file instead of a plain MTU line.
> Bond interfaces can cause significant pausing in boot/network restart
> The ifupdown script doesn't actually work on bonded interfaces [3]
> 
> race condition updating statefile "sometimes networking interfaces won't
> come up" - was fixed [4]-
> 
> We are seeing many more of cases involving complicated networking setups
> and with more OpenStack deployments this is going to become more of the
> norm.
> 
> My understanding is that ifupdown was not designed to handle a parallel
> boot process like Upstart or systemd.  I'm guessing there are a lot more
> bugs lurking due to that, aside from some other issues with the codebase
> [5].
> 
> *Future Releases*
> NetworkManager everywhere?  systemd-networkd?
> 
> Thanks for discussing,
> Bryan
> 
> 
> [1]
> https://blueprints.launchpad.net/ubuntu/+spec/servercloud-1403-networking
> [2] If you want to see where those actually work see my document here:
> https://docs.google.com/document/d/1OBN3efJ1LmA0-0DzD3K0eUkIuQdscxLQ-QO1yi3b
> HeM/edit [3]
> https://bugs.launchpad.net/ubuntu/+source/ifenslave-2.6/+bug/1254120 [4]
> https://bugs.launchpad.net/ubuntu/+source/ifupdown/+bug/1160490 [5]
> http://pureperl.blogspot.com/2013/01/the-debian-ifupdown-package-and.html

Whatever the solution is for this, it's a problem that Ubuntu and Debian 
share.  Now that (eventually) we will single up on a common, modern init 
system, I think it's critical to work with the relevant Debian stakeholders on 
a common approach.

Debian also has a stack of bugs around cases that ifupdown doesn't support 
well and there have been discussions about the best way to move forward, but 
they have (so far) foundered of a lack of knowledgeable manpower to do the 
work.  Perhaps working together, Debian and Ubuntu can get it done.

I don't know what the right answer is myself, but based on some of these prior 
discussions and my own experience with NetworkManager on the desktop, I'm 
pretty sure that's not it.

Scott K