[Bug 997978] Re: KVM images lose connectivity with bridged network

Stéphane Graber stgraber at stgraber.org
Mon Sep 24 19:57:05 UTC 2012

** No longer affects: ifenslave (Ubuntu)

** No longer affects: libvirt (Ubuntu)

** No longer affects: bridge-utils (Ubuntu)

** No longer affects: linux (Ubuntu)

You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to bridge-utils in Ubuntu.

  KVM images lose connectivity with bridged network

Status in OpenStack Compute (Nova):
Status in “qemu-kvm” package in Ubuntu:
  Fix Released
Status in “qemu-kvm” source package in Precise:
  In Progress

Bug description:
  SRU Justification:
  1. Impact: networking breaks after awhile in kvm guests using virtio networking
  2. Development fix: The bug was fixed upstream and the fix picked up in a new
  3. Stable fix: 3 virtio patches are cherrypicked from upstream:
     a821ce5 virtio: order index/descriptor reads
     92045d8 virtio: add missing mb() on enable notification
     a281ebc virtio: add missing mb() on notification
  4. Test case: Create a bridge enslaving the real NIC, and use that as the bridge
     for a kvm instance with virtio networking.  See comment #44 for specific test
  5. Regression potential: Should be low as several people have tested the fixed
     package under heavy load.

  Dell R410 Dual processor 2.4Ghz w/16G RAM
  Distributor ID: Ubuntu
  Description:    Ubuntu 12.04 LTS
  Release:        12.04
  Codename:       precise

  We're running 3 KVM guests, all Ubuntu 12.04 LTS using bridged networking.

  From the host:
  # cat /etc/network/interfaces
  auto br0
  iface br0 inet static
          address 212.XX.239.98
          gateway 212.XX.239.97
          bridge_ports eth0
          bridge_fd 9
          bridge_hello 2
          bridge_maxage 12
          bridge_stp off

  # ifconfig eth0
  eth0      Link encap:Ethernet  HWaddr d4:ae:52:84:2d:5a
            UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
            RX packets:11278363 errors:0 dropped:3128 overruns:0 frame:0
            TX packets:14437384 errors:0 dropped:0 overruns:0 carrier:0
            collisions:0 txqueuelen:1000
            RX bytes:4115980743 (4.1 GB)  TX bytes:5451961979 (5.4 GB)
            Interrupt:36 Memory:da000000-da012800

  # ifconfig br0
  br0       Link encap:Ethernet  HWaddr d4:ae:52:84:2d:5a
            inet addr:212.XX.239.98  Bcast:212.XX.239.111  Mask:
            inet6 addr: fe80::d6ae:52ff:fe84:2d5a/64 Scope:Link
            UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
            RX packets:1720861 errors:0 dropped:0 overruns:0 frame:0
            TX packets:1708622 errors:0 dropped:0 overruns:0 carrier:0
            collisions:0 txqueuelen:0
            RX bytes:210152198 (210.1 MB)  TX bytes:300858508 (300.8 MB)

  # brctl show
  bridge name     bridge id               STP enabled     interfaces
  br0             8000.d4ae52842d5a       no              eth0

  I have no default network configured to autostart in libvirt as we're using bridged networking:
  # virsh net-list --all
  Name                 State      Autostart
  default              inactive   no

  # arp
  Address                  HWtype  HWaddress           Flags Mask            Iface
  mailer03.xxxx.com       ether   52:54:00:82:5f:0f   C                     br0
  mailer01.xxxx.com       ether   52:54:00:d2:f7:31   C                     br0
  mailer02.xxxx.com       ether   52:54:00:d3:8f:91   C                     br0
  dxi-gw2.xxxx.com        ether   00:1a:30:2a:b1:c0   C                     br0

  From one of the guests:
  <domain type='kvm' id='4'>
      <type arch='x86_64' machine='pc-1.0'>hvm</type>
      <boot dev='hd'/>
    <clock offset='utc'/>
      <disk type='file' device='disk'>
        <driver name='qemu' type='raw'/>
        <source file='/dev/mapper/vg_main-mailer01--root'/>
        <target dev='hda' bus='ide'/>
        <alias name='ide0-0-0'/>
        <address type='drive' controller='0' bus='0' unit='0'/>
      <disk type='file' device='disk'>
        <driver name='qemu' type='raw'/>
        <source file='/dev/mapper/vg_main-mailer01--swap'/>
        <target dev='hdb' bus='ide'/>
        <alias name='ide0-0-1'/>
        <address type='drive' controller='0' bus='0' unit='1'/>
      <controller type='ide' index='0'>
        <alias name='ide0'/>
        <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
      <interface type='bridge'>
        <mac address='52:54:00:d2:f7:31'/>
        <source bridge='br0'/>
        <target dev='vnet0'/>
        <model type='virtio'/>
        <alias name='net0'/>
        <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
      <serial type='pty'>
        <source path='/dev/pts/0'/>
        <target port='0'/>
        <alias name='serial0'/>
      <console type='pty' tty='/dev/pts/0'>
        <source path='/dev/pts/0'/>
        <target type='serial' port='0'/>
        <alias name='serial0'/>
      <input type='mouse' bus='ps2'/>
      <graphics type='vnc' port='5900' autoport='yes' listen=''>
        <listen type='address' address=''/>
        <model type='cirrus' vram='9216' heads='1'/>
        <alias name='video0'/>
        <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
      <memballoon model='virtio'>
        <alias name='balloon0'/>
        <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    <seclabel type='dynamic' model='apparmor' relabel='yes'>

  From within the guest:
  # cat /etc/network/interfaces
  # The primary network interface
  auto eth0
  iface eth0 inet static
          address 212.XX.239.100
          network 212.XX.239.96
          broadcast 212.XX.239.111
          gateway 212.XX.239.97

  # ifconfig
  eth0      Link encap:Ethernet  HWaddr 52:54:00:d2:f7:31
            inet addr:212.XX.239.100  Bcast:212.XX.239.111  Mask:
            inet6 addr: fe80::5054:ff:fed2:f731/64 Scope:Link
            UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
            RX packets:5631830 errors:0 dropped:0 overruns:0 frame:0
            TX packets:6683416 errors:0 dropped:0 overruns:0 carrier:0
            collisions:0 txqueuelen:1000
            RX bytes:2027322829 (2.0 GB)  TX bytes:2076698690 (2.0 GB)

  A commandline which starts the KVM guest:
  /usr/bin/kvm -S -M pc-1.0 -enable-kvm -m 2048 -smp 1,sockets=1,cores=1,threads=1 -name mailer01 -uuid d41d1355-84e8-ae23-e84e-227bc0231b97 -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/mailer01.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -drive file=/dev/mapper/vg_main-mailer01--root,if=none,id=drive-ide0-0-0,format=raw -device ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 -drive file=/dev/mapper/vg_main-mailer01--swap,if=none,id=drive-ide0-0-1,format=raw -device ide-drive,bus=ide.0,unit=1,drive=drive-ide0-0-1,id=ide0-0-1 -netdev tap,fd=18,id=hostnet0 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:d2:f7:31,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -usb -vnc -vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x4

  Periodically (at least once a day), one or more of the guests lose network connectivity.  Ping responds with 'host unreachable', even from the dom host. Logging in via the serial console shows no problems: eth0 is up, can ping the local host, but no outside connectivity.  Restart the network (/etc/init.d/networking restart) does nothing. Reboot the machine and it comes alive again.

  I've verified there's no arp games going on on the primary host (the
  arp tables remain the same before - when it had connectivity - and
  after - when it doesn't.

  This is a critical issue affecting production services on the latest
  LTS release of Ubuntu. It's similar to an issue which was 'resolved'
  in 10.04 but appears to have risen its ugly head again.

To manage notifications about this bug go to:

More information about the foundations-bugs mailing list