[Ubuntu-US-CA] qemu-kvm/libvirt/virsh & MTU ...

[Michael Paoli]

qemu-kvm/libvirt/virsh & MTU ...
Question came up at Ubuntu Hour San Francisco (not sure if the OS in
question was Ubuntu, or Debian (or ???), but in any case ...)

Roughly paraphrasing scenario from memory - a guest Virtual Machine (VM)
under qemu-kvm/libvirt/virsh, and I'm presuming (and from what I seem to
recall) a relatively default network configuration (VM NATed on default
bridge behind the physical hosting host's interface(s) to upstream
network(s) (e.g. Internet or other larger network
environment/infrastructure).  The issue was described, at least very
roughly, as highly poor throughput from(/to?) Internet - very slow
performance and/or drops/stalls, etc.  It was also noted the MTU on
interface of VM, and MTU of the physical host's upstream network
connectivity were different - if I recall correctly, lower for the
upstream on physical host (perhaps through some VPN or tunnel or PPPoE
or whatever).  Anyway, there was slight bit of discussion regarding the
MTUs, and I was guestimating (it turns out incorrectly), that one could
probably tweak that for the default network/bridge device under
qemu-kvm/libvirt/virsh, or if not there, perhaps on the, again default,
built-in DHCP server on that subnet that qemu-kvm/libvirt/virsh sets up
there by default.  Seems I was incorrect on my guesses.  Anyway,
picking up from that point ...

Looked into it a wee bit.
I thought there was capability to configure MTU on the the
qemu-kvm/libvirt/virsh default network bridge, which it creates by
default.  At least for the versions I'm running (on Debian stable),
there appears to be no such configuration option there.  It also uses
its own built-in default DHCP server.  Such can be specified via DHCP,
but alas, that built-in DHCP server has no configuration option for MTU.
What about on the bridge device itself?  I didn't get so far as testing
it out, but that looks rather interesting - and a bit counter intuitive
compared to comparable physical switches and the like - have a peek
here:
https://joshua.hoblitt.com/rtfm/2014/05/dynamically_changing_the_mtu_of_a_linux_bridge_interface/
... I haven't verified that information, and that may or may not be
somewhat older versions, but, uhm, sounds at least "interesting", if
it's in fact so (and still so).

The other thing I was thinking about - PMTU - that generally *should*
work, and the host should be able to figure out appropriate MTUs to use,
depending upon IPs/routes it's communicating with.  Unfortunately on The
Internet, and across various networks, things can be broken with respect
to PMTU - e.g. this at least used to be pretty common with older
firewalls that were stupid and that rather than doing the proper RFC
thing, were instead going, "Gee, I don't understand that flag, it might
be an attack, I think I'll drop it" - and in the process they'd break
PMTU (dumb, particularly older, firewalls, continue to break stuff
across various networks - e.g. I've seen firewalls totally destroy TCP
Selective ACKnowledgement (SACK), because they were randomizing
(mangling) initial sequence numbers - using those generated by firewall
rather than passing along those generated and used by client - and
firewall altering them in the connection traffic thereafter, but the
firewall didn't understand SACK, and hence weren't likewise making
corresponding sequence number changes to SACK data, so the instant SACK
came into play (got drop of a packet in your >=2 GiB TCP stream?  Yes,
probably), they'd totally hose the connection (yes, thinking one has a
good secure firewall, when its OS or firmware is many years or more out
of date with regard to patches and/or updates, is a dumb fallacy - got
self-DoS?  8-O).

Also, off-the-top-of-my-head, I'm not sure how PMTU does/doesn't (or
fails to) work across NAT/SNAT and the like (I'm thinking it mostly
doesn't? - but perhaps it also depends at least on part on what's doing
the NAT/SNAT ... and how).  Host also keeps tables of its PMTU data, so
that can be inspected, and there are also tools to test and examine
PMTU behavior - so one may be able to isolate issues in that regard,
and make some relatively minimal changes where things may not be
working properly (particularly if they're quite adversely impacting
performance/throughput).  As far as MTU being different on two closely
associated networks/subnets, I'd think in theory that really shouldn't
be an issue, as host should figure that out properly by PMTU, so that
should effectively be a non-issue ... at least that's the theory.  :-)
Among other utilities, tracepath(8) and/or tracepath6(8) may be quite
useful.

Oh, also, for apt-get, if client and server are dual stack (IPv4 &
IPv6), but IPv6 is broken or semi-broken or randomly broken between
client and server, one can force apt-get to use IPv4 only
"
Add -o Acquire::ForceIPv4=true when running apt-get.

If you want to make the setting persistent just create  
/etc/apt/apt.conf.d/99force-ipv4 and put Acquire::ForceIPv4 "true"; in  
it.

Config options Acquire::ForceIPv4 and Acquire::ForceIPv6 were added to  
version 0.9.7.9~exp1 (see bug 611891) which is available since Ubuntu  
Saucy (released in October 2013) and Debian Jessie (released in April  
2015).
"
http://unix.stackexchange.com/questions/9940/convince-apt-get-not-to-use-ipv6-method
And for debian, if you want to avoid having such local customization
squashed or conflict with some name that comes up with debian in
future, best to include "local" in the name of the configuration file,
e.g.:
/etc/apt/apt.conf.d/99_local_force-ipv4
I've quite recently been using -o Acquire::ForceIPv4=true in certain
non-home environment(s), 'cause, ... well, very recently finally got
local IPv6 there (yeah!) ... uhm, but unfortunately it's relatively
semi-broken, and most all bets are off when trying to go more than
about 3 network hops away (mostly pretty darn fine on 'da Internet ...
but getting from certain non-home environments, out to 'da Internet via
IPv6 ... uhm, ... yeah, ... egad, like web proxy that doesn't even know
how to do IPv6 DNS lookups yet - egad!).

> From: "Elizabeth K. Joseph" <lyz at ubuntu.com>
> Subject: [Ubuntu-US-CA] Wednesday, December 9th: San Francisco  
> Ubuntu Hour and Bay Area Debian Dinner
> Date: Sat, 5 Dec 2015 09:40:56 -0800

> http://loco.ubuntu.com/events/ubuntu-california/3261-ubuntu-hour-san-francisco/
> Also on meetup: http://www.meetup.com/Ubuntu-California/events/227105090/
> Details: http://bad.debian.net/pipermail/bad/2015-December/003704.html