[Bug 1915829] Re: FQDN / hostname recorded in OVSDB is unreliable

Tue May 11 13:49:27 UTC 2021

$ juju run --unit ovn-chassis/0 'ovs-vsctl remove open-vswitch . external-ids hostname'
$ juju run --unit ovn-chassis/0 'sudo shutdown -r now'
$ sudo ovs-vsctl list open-vswitch
...
external_ids        : {hostname=juju-d4e71d-0-lxd-0, rundir="/var/run/openvswitch", system-id="424e6f88-2e79-436f-99a8-d9e4dccf3c55"}
$ hostname -f
juju-d4e71d-0-lxd-0.maas
# sudo apt install openvswitch-common openvswitch-switch python3-openvswitch
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Suggested packages:
  openvswitch-doc
The following packages will be upgraded:
  openvswitch-common openvswitch-switch python3-openvswitch
3 upgraded, 0 newly installed, 0 to remove and 21 not upgraded.
Need to get 2783 kB of archives.
After this operation, 28.7 kB of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu focal-proposed/main amd64 python3-openvswitch all 2.13.3-0ubuntu0.20.04.1 [94.8 kB]
Get:2 http://archive.ubuntu.com/ubuntu focal-proposed/main amd64 openvswitch-common amd64 2.13.3-0ubuntu0.20.04.1 [1152 kB]
Get:3 http://archive.ubuntu.com/ubuntu focal-proposed/main amd64 openvswitch-switch amd64 2.13.3-0ubuntu0.20.04.1 [1536 kB]
Fetched 2783 kB in 1s (1986 kB/s)          
(Reading database ... 53438 files and directories currently installed.)
Preparing to unpack .../python3-openvswitch_2.13.3-0ubuntu0.20.04.1_all.deb ...
Unpacking python3-openvswitch (2.13.3-0ubuntu0.20.04.1) over (2.13.1-0ubuntu0.20
.04.4) ...
Preparing to unpack .../openvswitch-common_2.13.3-0ubuntu0.20.04.1_amd64.deb ...
Unpacking openvswitch-common (2.13.3-0ubuntu0.20.04.1) over (2.13.1-0ubuntu0.20.
04.4) ...
Preparing to unpack .../openvswitch-switch_2.13.3-0ubuntu0.20.04.1_amd64.deb ...
Unpacking openvswitch-switch (2.13.3-0ubuntu0.20.04.1) over (2.13.1-0ubuntu0.20.
04.4) ...
Setting up python3-openvswitch (2.13.3-0ubuntu0.20.04.1) ...
Setting up openvswitch-common (2.13.3-0ubuntu0.20.04.1) ...
Setting up openvswitch-switch (2.13.3-0ubuntu0.20.04.1) ...
Processing triggers for man-db (2.9.1-1) ...
Processing triggers for systemd (245.4-4ubuntu3.6) ...
$ juju run --unit ovn-chassis/0 'ovs-vsctl remove open-vswitch . external-ids hostname'
$ juju run --unit ovn-chassis/0 'sudo shutdown -r now'
$ sudo ovs-vsctl list open-vswitch
...
external_ids        : {hostname=juju-d4e71d-0-lxd-0.maas, rundir="/var/run/openvswitch", system-id="424e6f88-2e79-436f-99a8-d9e4dccf3c55"}

** Description changed:

  [ Impact ]

  The Open vSwitch init script `ovs-ctl` determines the system hostname
  and records it in the `external-ids:hostname` field of the `Open-
  vSwitch` table in the local OVSDB.

  This value may be consumed by downstream software and having it unset or
  set to a incorrect value could lead to erratic behavior of a system.

  In recent Ubuntu development we have also made it possible to configure
  Open vSwitch with netplan.io, which allows for a Open vSwitch controlled
  data path to be the only external network connection on a system.

  As such the current startup sequence of Open vSwitch is insufficient to
  allow for reliable recording of FQDN / hostname at system boot-up time.

  Examples of downstream issues can be viewed in bug 1896630 and bug
  1912844 comment #26.

  [ Test Case ]

- 1) Use MAAS 2.9, configure a machine with Open vSwitch bridge for networking and deploy.
- 2) Confirm that without the change the hostname recorded in Open vSwitch is `ubuntu`.
- 3) Add a PPA with the updated Open vSwitch package to MAAS.
- 4) Re-deploy and confirm that the hostname recorded in Open vSwitch matches the FQDN as provided by MAAS.
+ 1) Deploy bundle on metal:
+ 
+ series: focal
+ machines:
+   '0':
+     constraints: cores=48
+ relations:
+  - - octavia:ovsdb-subordinate
+    - ovn-chassis:ovsdb-subordinate
+ applications:
+   ovn-chassis:
+     charm: cs:ovn-chassis
+     bindings:
+       "": public-space
+   octavia:
+     charm: cs:octavia
+     options:
+       openstack-origin: distro
+     num_units: 1
+     to:
+     - 'lxd:0'
+     bindings:
+       "": public-space
+ 
+ 2) juju run --unit ovn-chassis/0 'ovs-vsctl remove open-vswitch . external-ids hostname'
+ 3) juju run --unit ovn-chassis/0 'sudo shutdown -r now'
+ 4) Observe external-ids:hostname being recorded as a shortname
+ 5) Add proposed and install openvswitch packages, repeat step 2-3
+ 6) Observe external-ids:hostname  being recorded as a FQDN

  [ Regression potential ]

  Minimal, the update uses the exact same methods to accomplish the end
  result of recording the hostname as before, we have only re-ordered when
  the events occur to ensure successful operation.

  [racb] A new service is added in packaging. Users who have overridden
  existing services to change behaviour (eg. "disable all OVS-related
  services") may have that configuration not have the same effect after
  this update (eg. "I thought I disabled all OVS-related services but
  what's this ovs-record-hostname thing now?")

  [ Other Info ]

  A possible solution to this would be to extend the upstream `ovs-ctl`
  script to allow controlling when the hostname is recorded and adding a
  new Systemd service file to the package that calls the `record-hostname`
  function at a more appropriate time, i.e. after or as part of the
  `network-online.target`.

  An example of downstream issues can be viewed in bug 1896630.

** Tags removed: verification-needed-focal
** Tags added: verification-done-focal

-- 
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to openvswitch in Ubuntu.
https://bugs.launchpad.net/bugs/1915829

Title:
  FQDN / hostname recorded in OVSDB is unreliable

Status in Ubuntu Cloud Archive:
  Fix Released
Status in Ubuntu Cloud Archive ussuri series:
  Fix Committed
Status in Ubuntu Cloud Archive victoria series:
  Triaged
Status in Ubuntu Cloud Archive wallaby series:
  Fix Released
Status in openvswitch package in Ubuntu:
  Fix Released
Status in openvswitch source package in Focal:
  Fix Committed
Status in openvswitch source package in Groovy:
  Fix Committed
Status in openvswitch source package in Hirsute:
  Fix Released

Bug description:
  [ Impact ]

  The Open vSwitch init script `ovs-ctl` determines the system hostname
  and records it in the `external-ids:hostname` field of the `Open-
  vSwitch` table in the local OVSDB.

  This value may be consumed by downstream software and having it unset
  or set to a incorrect value could lead to erratic behavior of a
  system.

  In recent Ubuntu development we have also made it possible to
  configure Open vSwitch with netplan.io, which allows for a Open
  vSwitch controlled data path to be the only external network
  connection on a system.

  As such the current startup sequence of Open vSwitch is insufficient
  to allow for reliable recording of FQDN / hostname at system boot-up
  time.

  Examples of downstream issues can be viewed in bug 1896630 and bug
  1912844 comment #26.

  [ Test Case ]

  1) Deploy bundle on metal:

  series: focal
  machines:
    '0':
      constraints: cores=48
  relations:
   - - octavia:ovsdb-subordinate
     - ovn-chassis:ovsdb-subordinate
  applications:
    ovn-chassis:
      charm: cs:ovn-chassis
      bindings:
        "": public-space
    octavia:
      charm: cs:octavia
      options:
        openstack-origin: distro
      num_units: 1
      to:
      - 'lxd:0'
      bindings:
        "": public-space

  2) juju run --unit ovn-chassis/0 'ovs-vsctl remove open-vswitch . external-ids hostname'
  3) juju run --unit ovn-chassis/0 'sudo shutdown -r now'
  4) Observe external-ids:hostname being recorded as a shortname
  5) Add proposed and install openvswitch packages, repeat step 2-3
  6) Observe external-ids:hostname  being recorded as a FQDN

  [ Regression potential ]

  Minimal, the update uses the exact same methods to accomplish the end
  result of recording the hostname as before, we have only re-ordered
  when the events occur to ensure successful operation.

  [racb] A new service is added in packaging. Users who have overridden
  existing services to change behaviour (eg. "disable all OVS-related
  services") may have that configuration not have the same effect after
  this update (eg. "I thought I disabled all OVS-related services but
  what's this ovs-record-hostname thing now?")

  [ Other Info ]

  A possible solution to this would be to extend the upstream `ovs-ctl`
  script to allow controlling when the hostname is recorded and adding a
  new Systemd service file to the package that calls the `record-
  hostname` function at a more appropriate time, i.e. after or as part
  of the `network-online.target`.

  An example of downstream issues can be viewed in bug 1896630.

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-archive/+bug/1915829/+subscriptions