[Bug 1759056] Re: [Ubuntu18.04][Witherspoon DD2.2] netplan fails to configure network bridge interface (tg3/BCM5719)

Frank Heimes 1759056 at bugs.launchpad.net
Tue Mar 27 05:47:50 UTC 2018 

** Tags added: triage-g

** Also affects: ubuntu-power-systems
   Importance: Undecided
       Status: New

** Changed in: ubuntu-power-systems
   Importance: Undecided => High

** Changed in: ubuntu-power-systems
     Assignee: (unassigned) => Canonical Foundations Team (canonical-foundations)

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to netplan.io in Ubuntu.
Matching subscriptions: foundations-bugs
https://bugs.launchpad.net/bugs/1759056

Title:
  [Ubuntu18.04][Witherspoon DD2.2] netplan fails to configure network
  bridge interface (tg3/BCM5719)

Status in The Ubuntu-power-systems project:
  New
Status in netplan.io package in Ubuntu:
  New

Bug description:
  Problem:
  -----------
  Network bridge interface setup fails with netplan configuration

  br0: port 1(enP52p1s0f0) entered blocking state 
  br0: port 1(enP52p1s0f0) entered disabled state 
  device enP52p1s0f0 entered promiscuous mode
  br0: port 1(enP52p1s0f0) entered blocking state 
  br0: port 1(enP52p1s0f0) entered forwarding state
  IPv6: ADDRCONF(NETDEV_UP): br0: link is not ready
  br0: port 1(enP52p1s0f0) entered disabled state 
  device enP52p1s0f0 left promiscuous mode
  br0: port 1(enP52p1s0f0) entered disabled state 

  
  Machine Type: Witherspoon BMC baremetal DD2.2
  uname -a
  Linux ltciofvtr-spoon1 4.15.0-12-generic #13-Ubuntu SMP Wed Mar 7 21:37:03 UTC 2018 ppc64le ppc64le ppc64le GNU/Linux

  package:
  netplan.io                                 0.34

  lspci -nn | grep 0034:01:00.0
  0034:00:00.0 PCI bridge [0604]: IBM Device [1014:04c1]
  0034:01:00.0 Ethernet controller [0200]: Broadcom Limited NetXtreme BCM5719 Gigabit Ethernet PCIe [14e4:1657] (rev 01)

  # ethtool -i  enP52p1s0f0
  driver: tg3
  version: 3.137
  firmware-version: 5719-v1.38i
  expansion-rom-version: 
  bus-info: 0034:01:00.0
  supports-statistics: yes
  supports-test: yes
  supports-eeprom-access: yes
  supports-register-dump: yes
  supports-priv-flags: no

  t# ifconfig  enP52p1s0f0
  enP52p1s0f0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
          inet 65.65.65.32  netmask 255.255.255.0  broadcast 65.65.65.255
          ether 98:be:94:01:02:80  txqueuelen 1000  (Ethernet)
          RX packets 81026  bytes 6025544 (6.0 MB)
          RX errors 0  dropped 0  overruns 0  frame 0
          TX packets 4946  bytes 431946 (431.9 KB)
          TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
          device interrupt 76  

  - - -Steps to recreate - - -
  1. Edit /etc/netplan/01-netcfg.yaml and add bridge interface entry br0

  # cat /etc/netplan/01-netcfg.yaml
  # This file describes the network interfaces available on your system
  # For more information, see netplan(5).
  network:
    version: 2
    renderer: networkd
    ethernets:
      enP5p1s0f0:
        addresses: [ 9.xx.xx.xx/20 ]
        gateway4: 9.xx.xx.xx
        nameservers:
            search: [ xx.xx.ibm.com ]
            addresses:
                - 9.xx.xx.x
      enP52p1s0f0:
        dhcp4: no
        dhcp6: no
    bridges:
      br0:
        interfaces: [ enP52p1s0f0 ]
        addresses: [ 65.65.65.32/24 ]
        dhcp4: no

  2.  Apply the changes by running below command
  # netplan --debug apply
  ** (generate:26009): DEBUG: 01:41:49.588: Processing input file //etc/netplan/01-netcfg.yaml..
  ** (generate:26009): DEBUG: 01:41:49.588: starting new processing pass
  ** (generate:26009): DEBUG: 01:41:49.588: enP52p1s0f0: setting default backend to 1
  ** (generate:26009): DEBUG: 01:41:49.588: br0: setting default backend to 1
  ** (generate:26009): DEBUG: 01:41:49.588: enP5p1s0f0: setting default backend to 1
  ** (generate:26009): DEBUG: 01:41:49.588: Generating output files..
  ** (generate:26009): DEBUG: 01:41:49.588: NetworkManager: definition enP52p1s0f0 is not for us (backend 1)
  ** (generate:26009): DEBUG: 01:41:49.588: NetworkManager: definition br0 is not for us (backend 1)
  ** (generate:26009): DEBUG: 01:41:49.588: NetworkManager: definition enP5p1s0f0 is not for us (backend 1)
  DEBUG:netplan generated networkd configuration exists, restarting networkd
  DEBUG:no netplan generated NM configuration exists
  DEBUG:device enP52p1s0f1 operstate is up, not replugging
  DEBUG:netplan triggering .link rules for enP52p1s0f1
  DEBUG:device enP5p1s0f0 operstate is up, not replugging
  DEBUG:netplan triggering .link rules for enP5p1s0f0
  DEBUG:device lo operstate is unknown, not replugging
  DEBUG:netplan triggering .link rules for lo
  DEBUG:replug enP52p1s0f2: unbinding 0034:01:00.2 from /sys/bus/pci/drivers/tg3
  DEBUG:replug enP52p1s0f2: rebinding 0034:01:00.2 to /sys/bus/pci/drivers/tg3
  DEBUG:replug enP5p1s0f1: unbinding 0005:01:00.1 from /sys/bus/pci/drivers/tg3
  DEBUG:replug enP5p1s0f1: rebinding 0005:01:00.1 to /sys/bus/pci/drivers/tg3
  DEBUG:device sit0 operstate is unknown, not replugging
  DEBUG:netplan triggering .link rules for sit0
  DEBUG:device enP52p1s0f0 operstate is up, not replugging
  DEBUG:netplan triggering .link rules for enP52p1s0f0
  DEBUG:replug enP52p1s0f3: unbinding 0034:01:00.3 from /sys/bus/pci/drivers/tg3
  DEBUG:replug enP52p1s0f3: rebinding 0034:01:00.3 to /sys/bus/pci/drivers/tg3

  
  $ ifconfig -a | grep br0

  It has failed to configured bridge interface.

  dmesg:
  tg3 0034:01:00.2: Using 64-bit DMA iommu bypass
  tg3 0034:01:00.2 eth0: Tigon3 [partno(00E2872) rev 5719001] (PCI Express) MAC address 98:be:94:01:02:82
  tg3 0034:01:00.2 eth0: attached PHY is 5719C (10/100/1000Base-T Ethernet) (WireSpeed[1], EEE[1])
  tg3 0034:01:00.2 eth0: RXcsums[1] LinkChgREG[0] MIirq[0] ASF[0] TSOcap[1]
  tg3 0034:01:00.2 eth0: dma_rwctrl[00000000] dma_mask[64-bit]
  tg3 0034:01:00.2 enP52p1s0f2: renamed from eth0
  tg3 0005:01:00.1: Using 64-bit DMA iommu bypass
  tg3 0005:01:00.1 eth0: Tigon3 [partno(BCM95719) rev 5719001] (PCI Express) MAC address 70:e2:84:14:29:fb
  tg3 0005:01:00.1 eth0: attached PHY is 5719C (10/100/1000Base-T Ethernet) (WireSpeed[1], EEE[1])
  tg3 0005:01:00.1 eth0: RXcsums[1] LinkChgREG[0] MIirq[0] ASF[1] TSOcap[1]
  tg3 0005:01:00.1 eth0: dma_rwctrl[00000000] dma_mask[64-bit]
  tg3 0005:01:00.1 enP5p1s0f1: renamed from eth0
  IPv6: ADDRCONF(NETDEV_UP): enP52p1s0f2: link is not ready
  tg3 0005:01:00.1 enP5p1s0f1: Link is up at 1000 Mbps, full duplex
  tg3 0005:01:00.1 enP5p1s0f1: Flow control is on for TX and on for RX
  tg3 0005:01:00.1 enP5p1s0f1: EEE is disabled
  net_ratelimit: 5 callbacks suppressed
  IPv4: martian source 255.255.255.255 from 9.47.72.146, on dev enP5p1s0f1
  ll header: 00000000: ff ff ff ff ff ff ea e3 8e 00 2d 02 08 00        ..........-...
  IPv6: ADDRCONF(NETDEV_UP): enP5p1s0f1: link is not ready 
  IPv6: ADDRCONF(NETDEV_CHANGE): enP5p1s0f1: link becomes ready
  tg3 0034:01:00.3: Using 64-bit DMA iommu bypass 
  tg3 0034:01:00.3 eth0: Tigon3 [partno(00E2872) rev 5719001] (PCI Express) MAC address 98:be:94:01:02:83
  tg3 0034:01:00.3 eth0: attached PHY is 5719C (10/100/1000Base-T Ethernet) (WireSpeed[1], EEE[1])
  tg3 0034:01:00.3 eth0: RXcsums[1] LinkChgREG[0] MIirq[0] ASF[0] TSOcap[1]
  tg3 0034:01:00.3 eth0: dma_rwctrl[00000000] dma_mask[64-bit]
  tg3 0034:01:00.3 enP52p1s0f3: renamed from eth0
  br0: port 1(enP52p1s0f0) entered blocking state
  br0: port 1(enP52p1s0f0) entered disabled state
  device enP52p1s0f0 entered promiscuous mode
  device enP52p1s0f0 left promiscuous mode
  br0: port 1(enP52p1s0f0) entered disabled state
  IPv6: ADDRCONF(NETDEV_UP): enP52p1s0f3: link is not ready

  --- expected ---
  bridge interface br0 should come online

  Guys I tried and failed using netplan to configure a bridge. 
  We better let canonical answer us how to proceed here.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu-power-systems/+bug/1759056/+subscriptions

More information about the foundations-bugs mailing list