Ubuntu Fan Updates for Trusty and Vivid

[Andy Whitcroft]

On Fri, Jul 10, 2015 at 04:15:47PM +0200, Martin Pitt wrote:
> Hey Andy,
> 
> Andy Whitcroft [2015-07-07 10:27 +0100]:
> > The Ubuntu Fan driver is an IPv4 address multiplication mechanism.  It
> > allows a group of host machines to participate in an expanded address
> > space with hundreds (potentially thousands) of directly addressable
> > addresses per host.  These are intended to allow direct communication
> > between individual host containers and virtual machines.
> 
> This doesn't sound much different than normal local DHCP serving from
> a private IP range, and using NAT, tunnelling, and similar; normally
> IP distribution and routing setup is pretty much exclusively an
> userspace matter. Out of interest, how is the kernel involved into
> this?

Correct, the key differentiator is that there is no NAT involved between
members of these Fan ranges.  This allows those members to be directly
addressed simplifying modelling of the resultant service interactions.
The kernel is involved in as much as it supplies the computed mapping for
the tunnel side of things (this is the totality of the kernel changes),
allowing per-host setup without regard for the quantiry, addressing of,
or indeed existance of other hosts on the Fan.

> What's the security surface of this? I. e. is this concerned with any
> actual data manipulation and routing, or limited to setting up virtual
> interfaces, but does not get in touch with remotely specified data? As
> this seems to be an Ubuntu specific feature, was that ever reviewed by
> the kernel network maintainers for some potential
> problems/bugs/security issues? I don't find anything on kernel.org or
> LKML about it.

The kernel module only interacts with source and destination IP
addresses, for the purposes of mapping those to the appropriate
destination on the "local" network.  All mapping information is loaded
from userspace locally.

> > As the 14.04 LTS point release will contain the 15.04 LTS backport
> > kernel we propose to apply the Ubuntu Fan changes currently in Wily to
> > the 15.04 kernel, and thereby the 15.04 LTS backport kernel in 14.04
> > LTS.
> 
> That's a normal process through which certainly a lot of other new
> features will land in the 14.04 kernel. That's fine.
> 
> > We also propose to apply the corresponding changes for the
> > iproute2 package in lock step in both 15.04 and 14.04 LTS.
> 
> This is the bit that causes me most worries.
> 
>  - What is the precise nature of these changes? Does it change the
>    default behaviour in any way or just add new options?
> 
>   - Does this touch any code paths for other functionality? How intrusive is this, is there a patch to look at?
> 
>   - Why does this need changes to such a central utility if we are
>     going to need a separate ubuntu-fan package anyway?

The mappings between Fan (overlay) addresses and local (underlay) addresses
needs to be supplied to the kernel, normally this kind of link level
information is loaded into the kernel via netlink and commonly by the ip
application; in particular the other configuration we need to load for
the tunnel is supplied via this interface.  The modifications there are
adding decoding of userspace specifications for this mapping and loading
it into the kernel via netlink, and allowing those to be visualised.
Those not using these new interfaces should not be affected and we do not
modify any semantics for any of the existing interfaces (modulo any bugs
in the implementation thereof).

The delta for the trusty iproute2 package should be here now:

  http://people.canonical.com/~apw/misc/fan-sru/trusty-iproute2-3.12.0-2ubuntu1.debdiff

> If we screw this up in any way and break networking for existing
> users, then -- not good things will happen. So I would like to
> understand this particular part very well.

Understood.

> > Finally we propose to update the ubuntu-fan package in 15.04 and
> > introduce it in 14.04 LTS.
> 
> That seems fine to me.
> 
> Do you plan to do any other changes besides that, i. e. will this be
> suddenly used by existing installations through some other packages
> (LXC and the like)? Or will this remain an "opt-in" feature for 14.04?

The intent is that this is available for LXC et al to opt-in to, they
will not be forced to use it.  It is intended to be easy to enable and
use however.

> > We also insulate the consumers of this functionality from later
> > changes as this feature further develops moving forward towards
> > 16.04 LTS.
> 
> What does that mean? It sounds like this somehow has a "versioned"
> interface so that configs from 14.04 won't break even if the
> kernel/userspace bits change in an incompatible way?

The primary thrust of the Fan functionality is to provide an expansion
of local address space.  Currently that is implemented on the wire using
IPIP encapsulation.  Some hardware platforms (and in this I include
cloud substrates) are able to accellerate some protocols and not others.
This means that is is desirable to be able to vary the underlying on
wire form without impact to the consumers configuration interfaces.
It this separation from the specific implementation at the kernel/wire
level that we are trying to solidify in this update.

> > These changes should present a reasonably low risk overall.  The feature
> > we are modifying was a never announced technology preview.  (The
> > announced test images for this utilised the proposed updated versions
> > and interfaces.)  The kernel updates apply to an isolated driver
> > component only activated when Fan address ranges are configured.  The
> > iproute2 changes introduce new interfaces specifiers which would not
> > normally exist.  The ubuntu-fan package is benign when installed until
> > configured.
> 
> OK, that touches some of the questions above, but I'd like to hear
> some more details particularly about the iproute2 changes and why they
> are necessary.

Thank you for your detailed review, hopefully the above answers them,
but please let me know if not or you want more detail.

-apw