[Bug 2059721] Re: ovn db upgrade

[Vadim Kuznetsov]

I reinstalled host OS and rebuild it from scratch including all containers.
I reinstalled compute nodes one by one, not to overload OpenStack Infrastructure.
I'm using venv_wheel_build_enable=False to get around.
Alternative can be to reinstall and rebuild only infra (repo, galera, rabbitmq) one of the controller, then  rebuild compute nodes one by one with limit, then come back and finish first controller.
btw glusterfs on repo needs to be upgraded to 10 before rebuilding first repo server.
Also imo more services (not just keystone) do not support limit with recent releases.

Question: You said "you need to restart ovn-northd service and ovn-central right after it"
Why "right after it" is significant?

It seems to me I did restarted services "right after it".

I aslo restarted it after I completed upgrade of all controllers and the
day after :)

After restarting ovn-northd I only see ovn-northd. 2 ovsdb-server are not started. 
According ovn docs that that the only thing is needed to finish ovn upgrade.
But you also restarted ovn-central. 
Me too. otherwise ovsdb-servers will stay down.

-- 
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to ovn in Ubuntu.
https://bugs.launchpad.net/bugs/2059721

Title:
  ovn db upgrade

Status in OpenStack-Ansible:
  Triaged
Status in ovn package in Ubuntu:
  New

Bug description:
  After distro upgrade of the openstack-ansible 2023.1 site from Ubuntu
  20.04 to Ubuntu 22.04 OVN cluster DB was not upgraded.

  root at b-mgmt-neutron-ovn-northd-container-cb715707:/var/log/ovn# ovn-nbctl --version
  ovn-nbctl 23.03.1
  Open vSwitch Library 3.1.2
  DB Schema 7.0.0

  root at b-mgmt-neutron-ovn-northd-container-cb715707:/var/log/ovn# ovn-sbctl --version
  ovn-sbctl 23.03.1
  Open vSwitch Library 3.1.2
  DB Schema 20.27.0

  However

  root at b-mgmt-neutron-ovn-northd-container-cb715707:/var/log/ovn# ovsdb-client get-schema-version unix:/var/run/ovn/ovnnb_db.sock OVN_Northbound
  6.1.0
  root at b-mgmt-neutron-ovn-northd-container-cb715707:/var/log/ovn# ovsdb-client get-schema-version unix:/var/run/ovn/ovnsb_db.sock OVN_Southbound
  20.21.0

  Restrarting ovn-central did not help.

  ovn-northd.log:
  2024-03-28T17:28:52.521Z|00053|ovsdb_idl|WARN|OVN_Southbound database lacks Chassis_Template_Var table (database needs upgrade?)
  2024-03-28T17:28:52.521Z|00054|ovsdb_idl|WARN|Load_Balancer table in OVN_Southbound database lacks datapath_group column (database needs upgrade?)
  2024-03-28T17:28:52.521Z|00055|ovsdb_idl|WARN|MAC_Binding table in OVN_Southbound database lacks timestamp column (database needs upgrade?)
  2024-03-28T17:28:52.521Z|00056|ovsdb_idl|WARN|OVN_Southbound database lacks Mirror table (database needs upgrade?)
  2024-03-28T17:28:52.521Z|00057|ovsdb_idl|WARN|Port_Binding table in OVN_Southbound database lacks additional_chassis column (database needs upgrade?)
  2024-03-28T17:28:52.521Z|00058|ovsdb_idl|WARN|Port_Binding table in OVN_Southbound database lacks additional_encap column (database needs upgrade?)
  2024-03-28T17:28:52.521Z|00059|ovsdb_idl|WARN|Port_Binding table in OVN_Southbound database lacks mirror_rules column (database needs upgrade?)
  2024-03-28T17:28:52.521Z|00060|ovsdb_idl|WARN|Port_Binding table in OVN_Southbound database lacks port_security column (database needs upgrade?)
  2024-03-28T17:28:52.521Z|00061|ovsdb_idl|WARN|Port_Binding table in OVN_Southbound database lacks requested_additional_chassis column (database needs upgrade?)
  2024-03-28T17:28:52.521Z|00062|ovsdb_idl|WARN|OVN_Southbound database lacks Static_MAC_Binding table (database needs upgrade?)

To manage notifications about this bug go to:
https://bugs.launchpad.net/openstack-ansible/+bug/2059721/+subscriptions