[Bug 1863257] Re: Neutron db sync fails

James Page james.page at ubuntu.com
Tue Mar 24 12:51:51 UTC 2020 

That primary key was added to deal with bug 1831986 and also appears to
work fine with MySQL8 (in addition to PXC 5.7.x)

I suspect that this is something todo with the mode that your have
MariaDB operating in but I'm not 100% sure.

** Changed in: neutron-fwaas (Ubuntu)
       Status: New => Incomplete

** Changed in: neutron-fwaas (Ubuntu)
   Importance: Undecided => Medium

-- 
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to neutron-fwaas in Ubuntu.
https://bugs.launchpad.net/bugs/1863257

Title:
  Neutron db sync fails

Status in neutron-fwaas package in Ubuntu:
  Incomplete

Bug description:
  When trying to sync db with python3-neutron-fwaas, db sync fails due
  to this code:

      op.create_primary_key(
          'pk_firewall_group_port_associations_v2',
          'firewall_group_port_associations_v2',
          ['firewall_group_id', 'port_id'])

  when commenting the code the db sync the db sync works, the change is described in: 
  https://launchpadlibrarian.net/453954118/neutron-fwaas_1%3A14.0.0-0ubuntu1.1_1%3A14.0.0-0ubuntu1.2.diff.gz

  DB sync stack:

  neutron-db-manage --config-file /etc/neutron/neutron.conf     --config-file /etc/neutron/plugins/ml2/ml2_conf.ini  upgrade heads
  INFO  [alembic.runtime.migration] Context impl MySQLImpl.
  INFO  [alembic.runtime.migration] Will assume non-transactional DDL.
    Running upgrade for neutron ...
  INFO  [alembic.runtime.migration] Context impl MySQLImpl.
  INFO  [alembic.runtime.migration] Will assume non-transactional DDL.
  INFO  [alembic.runtime.migration] Running upgrade  -> kilo
  INFO  [alembic.runtime.migration] Running upgrade kilo -> 354db87e3225
  INFO  [alembic.runtime.migration] Running upgrade 354db87e3225 -> 599c6a226151
  INFO  [alembic.runtime.migration] Running upgrade 599c6a226151 -> 52c5312f6baf
  INFO  [alembic.runtime.migration] Running upgrade 52c5312f6baf -> 313373c0ffee
  INFO  [alembic.runtime.migration] Running upgrade 313373c0ffee -> 8675309a5c4f
  INFO  [alembic.runtime.migration] Running upgrade 8675309a5c4f -> 45f955889773
  INFO  [alembic.runtime.migration] Running upgrade 45f955889773 -> 26c371498592
  INFO  [alembic.runtime.migration] Running upgrade 26c371498592 -> 1c844d1677f7
  INFO  [alembic.runtime.migration] Running upgrade 1c844d1677f7 -> 1b4c6e320f79
  INFO  [alembic.runtime.migration] Running upgrade 1b4c6e320f79 -> 48153cb5f051
  INFO  [alembic.runtime.migration] Running upgrade 48153cb5f051 -> 9859ac9c136
  INFO  [alembic.runtime.migration] Running upgrade 9859ac9c136 -> 34af2b5c5a59
  INFO  [alembic.runtime.migration] Running upgrade 34af2b5c5a59 -> 59cb5b6cf4d
  INFO  [alembic.runtime.migration] Running upgrade 59cb5b6cf4d -> 13cfb89f881a
  INFO  [alembic.runtime.migration] Running upgrade 13cfb89f881a -> 32e5974ada25
  INFO  [alembic.runtime.migration] Running upgrade 32e5974ada25 -> ec7fcfbf72ee
  INFO  [alembic.runtime.migration] Running upgrade ec7fcfbf72ee -> dce3ec7a25c9
  INFO  [alembic.runtime.migration] Running upgrade dce3ec7a25c9 -> c3a73f615e4
  INFO  [alembic.runtime.migration] Running upgrade c3a73f615e4 -> 659bf3d90664
  INFO  [alembic.runtime.migration] Running upgrade 659bf3d90664 -> 1df244e556f5
  INFO  [alembic.runtime.migration] Running upgrade 1df244e556f5 -> 19f26505c74f
  INFO  [alembic.runtime.migration] Running upgrade 19f26505c74f -> 15be73214821
  INFO  [alembic.runtime.migration] Running upgrade 15be73214821 -> b4caf27aae4
  INFO  [alembic.runtime.migration] Running upgrade b4caf27aae4 -> 15e43b934f81
  INFO  [alembic.runtime.migration] Running upgrade 15e43b934f81 -> 31ed664953e6
  INFO  [alembic.runtime.migration] Running upgrade 31ed664953e6 -> 2f9e956e7532
  INFO  [alembic.runtime.migration] Running upgrade 2f9e956e7532 -> 3894bccad37f
  INFO  [alembic.runtime.migration] Running upgrade 3894bccad37f -> 0e66c5227a8a
  INFO  [alembic.runtime.migration] Running upgrade 0e66c5227a8a -> 45f8dd33480b
  INFO  [alembic.runtime.migration] Running upgrade 45f8dd33480b -> 5abc0278ca73
  INFO  [alembic.runtime.migration] Running upgrade 5abc0278ca73 -> d3435b514502
  INFO  [alembic.runtime.migration] Running upgrade d3435b514502 -> 30107ab6a3ee
  INFO  [alembic.runtime.migration] Running upgrade 30107ab6a3ee -> c415aab1c048
  INFO  [alembic.runtime.migration] Running upgrade c415aab1c048 -> a963b38d82f4
  INFO  [alembic.runtime.migration] Running upgrade kilo -> 30018084ec99
  INFO  [alembic.runtime.migration] Running upgrade 30018084ec99 -> 4ffceebfada
  INFO  [alembic.runtime.migration] Running upgrade 4ffceebfada -> 5498d17be016
  INFO  [alembic.runtime.migration] Running upgrade 5498d17be016 -> 2a16083502f3
  INFO  [alembic.runtime.migration] Running upgrade 2a16083502f3 -> 2e5352a0ad4d
  INFO  [alembic.runtime.migration] Running upgrade 2e5352a0ad4d -> 11926bcfe72d
  INFO  [alembic.runtime.migration] Running upgrade 11926bcfe72d -> 4af11ca47297
  INFO  [alembic.runtime.migration] Running upgrade 4af11ca47297 -> 1b294093239c
  INFO  [alembic.runtime.migration] Running upgrade 1b294093239c -> 8a6d8bdae39
  INFO  [alembic.runtime.migration] Running upgrade 8a6d8bdae39 -> 2b4c2465d44b
  INFO  [alembic.runtime.migration] Running upgrade 2b4c2465d44b -> e3278ee65050
  INFO  [alembic.runtime.migration] Running upgrade e3278ee65050 -> c6c112992c9
  INFO  [alembic.runtime.migration] Running upgrade c6c112992c9 -> 5ffceebfada
  INFO  [alembic.runtime.migration] Running upgrade 5ffceebfada -> 4ffceebfcdc
  INFO  [alembic.runtime.migration] Running upgrade 4ffceebfcdc -> 7bbb25278f53
  INFO  [alembic.runtime.migration] Running upgrade 7bbb25278f53 -> 89ab9a816d70
  INFO  [alembic.runtime.migration] Running upgrade 89ab9a816d70 -> c879c5e1ee90
  INFO  [alembic.runtime.migration] Running upgrade c879c5e1ee90 -> 8fd3918ef6f4
  INFO  [alembic.runtime.migration] Running upgrade 8fd3918ef6f4 -> 4bcd4df1f426
  INFO  [alembic.runtime.migration] Running upgrade 4bcd4df1f426 -> b67e765a3524
  INFO  [alembic.runtime.migration] Running upgrade a963b38d82f4 -> 3d0e74aa7d37
  INFO  [alembic.runtime.migration] Running upgrade 3d0e74aa7d37 -> 030a959ceafa
  INFO  [alembic.runtime.migration] Running upgrade 030a959ceafa -> a5648cfeeadf
  INFO  [alembic.runtime.migration] Running upgrade a5648cfeeadf -> 0f5bef0f87d4
  INFO  [alembic.runtime.migration] Running upgrade 0f5bef0f87d4 -> 67daae611b6e
  INFO  [alembic.runtime.migration] Running upgrade b67e765a3524 -> a84ccf28f06a
  INFO  [alembic.runtime.migration] Running upgrade a84ccf28f06a -> 7d9d8eeec6ad
  INFO  [alembic.runtime.migration] Running upgrade 7d9d8eeec6ad -> a8b517cff8ab
  INFO  [alembic.runtime.migration] Running upgrade a8b517cff8ab -> 3b935b28e7a0
  INFO  [alembic.runtime.migration] Running upgrade 3b935b28e7a0 -> b12a3ef66e62
  INFO  [alembic.runtime.migration] Running upgrade b12a3ef66e62 -> 97c25b0d2353
  INFO  [alembic.runtime.migration] Running upgrade 97c25b0d2353 -> 2e0d7a8a1586
  INFO  [alembic.runtime.migration] Running upgrade 2e0d7a8a1586 -> 5c85685d616d
  INFO  [alembic.runtime.migration] Running upgrade 5c85685d616d -> 33e74c25a5b9
  INFO  [alembic.runtime.migration] Running upgrade 67daae611b6e -> 6b461a21bcfc
  INFO  [alembic.runtime.migration] Running upgrade 6b461a21bcfc -> 5cd92597d11d
  INFO  [alembic.runtime.migration] Running upgrade 5cd92597d11d -> 929c968efe70
  INFO  [alembic.runtime.migration] Running upgrade 929c968efe70 -> a9c43481023c
  INFO  [alembic.runtime.migration] Running upgrade a9c43481023c -> 804a3c76314c
  INFO  [alembic.runtime.migration] Running upgrade 804a3c76314c -> 2b42d90729da
  INFO  [alembic.runtime.migration] Running upgrade 2b42d90729da -> 62c781cb6192
  INFO  [alembic.runtime.migration] Running upgrade 62c781cb6192 -> c8c222d42aa9
  INFO  [alembic.runtime.migration] Running upgrade c8c222d42aa9 -> 349b6fd605a6
  INFO  [alembic.runtime.migration] Running upgrade 349b6fd605a6 -> 7d32f979895f
  INFO  [alembic.runtime.migration] Running upgrade 7d32f979895f -> 594422d373ee
  INFO  [alembic.runtime.migration] Running upgrade 594422d373ee -> 61663558142c
  INFO  [alembic.runtime.migration] Running upgrade 61663558142c -> 867d39095bf4, port forwarding
  INFO  [alembic.runtime.migration] Running upgrade 867d39095bf4 -> d72db3e25539, modify uniq port forwarding
  INFO  [alembic.runtime.migration] Running upgrade d72db3e25539 -> cada2437bf41
  INFO  [alembic.runtime.migration] Running upgrade cada2437bf41 -> 195176fb410d, router gateway IP QoS
  INFO  [alembic.runtime.migration] Running upgrade 195176fb410d -> fb0167bd9639
  INFO  [alembic.runtime.migration] Running upgrade fb0167bd9639 -> 0ff9e3881597
  INFO  [alembic.runtime.migration] Running upgrade 0ff9e3881597 -> 9bfad3f1e780
  INFO  [alembic.runtime.migration] Running upgrade 9bfad3f1e780 -> 68f83c3e007f
    OK
  INFO  [alembic.runtime.migration] Context impl MySQLImpl.
  INFO  [alembic.runtime.migration] Will assume non-transactional DDL.
    Running upgrade for neutron-fwaas ...
  INFO  [alembic.runtime.migration] Context impl MySQLImpl.
  INFO  [alembic.runtime.migration] Will assume non-transactional DDL.
  INFO  [alembic.runtime.migration] Running upgrade  -> start_neutron_fwaas, start neutron-fwaas chain
  INFO  [alembic.runtime.migration] Running upgrade start_neutron_fwaas -> 4202e3047e47, add_index_tenant_id
  INFO  [alembic.runtime.migration] Running upgrade 4202e3047e47 -> 540142f314f4, FWaaS router insertion
  INFO  [alembic.runtime.migration] Running upgrade 540142f314f4 -> 796c68dffbb, cisco_csr_fwaas
  INFO  [alembic.runtime.migration] Running upgrade 796c68dffbb -> kilo, kilo
  INFO  [alembic.runtime.migration] Running upgrade kilo -> c40fbb377ad, Initial Liberty no-op script.
  INFO  [alembic.runtime.migration] Running upgrade c40fbb377ad -> 4b47ea298795, add reject rule
  INFO  [alembic.runtime.migration] Running upgrade 4b47ea298795 -> d6a12e637e28, neutron-fwaas v2.0
  Traceback (most recent call last):
    File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/base.py", line 1246, in _execute_context
      cursor, statement, parameters, context
    File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/default.py", line 588, in do_execute
      cursor.execute(statement, parameters)
    File "/usr/lib/python3/dist-packages/pymysql/cursors.py", line 165, in execute
      result = self._query(query)
    File "/usr/lib/python3/dist-packages/pymysql/cursors.py", line 321, in _query
      conn.query(q)
    File "/usr/lib/python3/dist-packages/pymysql/connections.py", line 860, in query
      self._affected_rows = self._read_query_result(unbuffered=unbuffered)
    File "/usr/lib/python3/dist-packages/pymysql/connections.py", line 1061, in _read_query_result
      result.read()
    File "/usr/lib/python3/dist-packages/pymysql/connections.py", line 1349, in read
      first_packet = self.connection._read_packet()
    File "/usr/lib/python3/dist-packages/pymysql/connections.py", line 1018, in _read_packet
      packet.check_error()
    File "/usr/lib/python3/dist-packages/pymysql/connections.py", line 384, in check_error
      err.raise_mysql_exception(self._data)
    File "/usr/lib/python3/dist-packages/pymysql/err.py", line 107, in raise_mysql_exception
      raise errorclass(errno, errval)
  pymysql.err.InternalError: (1832, "Cannot change column 'firewall_group_id': used in a foreign key constraint 'firewall_group_port_associations_v2_ibfk_1'")

  The above exception was the direct cause of the following exception:

  Traceback (most recent call last):
    File "/usr/bin/neutron-db-manage", line 10, in <module>
      sys.exit(main())
    File "/usr/lib/python3/dist-packages/neutron/db/migration/cli.py", line 656, in main
      return_val |= bool(CONF.command.func(config, CONF.command.name))
    File "/usr/lib/python3/dist-packages/neutron/db/migration/cli.py", line 180, in do_upgrade
      desc=branch, sql=CONF.command.sql)
    File "/usr/lib/python3/dist-packages/neutron/db/migration/cli.py", line 81, in do_alembic_command
      getattr(alembic_command, cmd)(config, *args, **kwargs)
    File "/usr/local/lib/python3.6/dist-packages/alembic/command.py", line 298, in upgrade
      script.run_env()
    File "/usr/local/lib/python3.6/dist-packages/alembic/script/base.py", line 489, in run_env
      util.load_python_file(self.dir, "env.py")
    File "/usr/local/lib/python3.6/dist-packages/alembic/util/pyfiles.py", line 98, in load_python_file
      module = load_module_py(module_id, path)
    File "/usr/local/lib/python3.6/dist-packages/alembic/util/compat.py", line 173, in load_module_py
      spec.loader.exec_module(module)
    File "<frozen importlib._bootstrap_external>", line 678, in exec_module
    File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
    File "/usr/lib/python3/dist-packages/neutron_fwaas/db/migration/alembic_migrations/env.py", line 86, in <module>
      run_migrations_online()
    File "/usr/lib/python3/dist-packages/neutron_fwaas/db/migration/alembic_migrations/env.py", line 77, in run_migrations_online
      context.run_migrations()
    File "<string>", line 8, in run_migrations
    File "/usr/local/lib/python3.6/dist-packages/alembic/runtime/environment.py", line 846, in run_migrations
      self.get_context().run_migrations(**kw)
    File "/usr/local/lib/python3.6/dist-packages/alembic/runtime/migration.py", line 518, in run_migrations
      step.migration_fn(**kw)
    File "/usr/lib/python3/dist-packages/neutron_fwaas/db/migration/alembic_migrations/versions/newton/expand/d6a12e637e28_neutron_fwaas_v2_0.py", line 108, in upgrade
      ['firewall_group_id', 'port_id'])
    File "<string>", line 8, in create_primary_key
    File "<string>", line 3, in create_primary_key
    File "/usr/local/lib/python3.6/dist-packages/alembic/operations/ops.py", line 290, in create_primary_key
      return operations.invoke(op)
    File "/usr/local/lib/python3.6/dist-packages/alembic/operations/base.py", line 374, in invoke
      return fn(self, operation)
    File "/usr/local/lib/python3.6/dist-packages/alembic/operations/toimpl.py", line 151, in create_constraint
      operation.to_constraint(operations.migration_context)
    File "/usr/local/lib/python3.6/dist-packages/alembic/ddl/impl.py", line 244, in add_constraint
      self._exec(schema.AddConstraint(const))
    File "/usr/local/lib/python3.6/dist-packages/alembic/ddl/impl.py", line 140, in _exec
      return conn.execute(construct, *multiparams, **params)
    File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/base.py", line 982, in execute
      return meth(self, multiparams, params)
    File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/sql/ddl.py", line 72, in _execute_on_connection
      return connection._execute_ddl(self, multiparams, params)
    File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/base.py", line 1044, in _execute_ddl
      compiled,
    File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/base.py", line 1250, in _execute_context
      e, statement, parameters, cursor, context
    File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/base.py", line 1474, in _handle_dbapi_exception
      util.raise_from_cause(newraise, exc_info)
    File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/util/compat.py", line 398, in raise_from_cause
      reraise(type(exception), exception, tb=exc_tb, cause=cause)
    File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/util/compat.py", line 152, in reraise
      raise value.with_traceback(tb)
    File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/base.py", line 1246, in _execute_context
      cursor, statement, parameters, context
    File "/usr/local/lib/python3.6/dist-packages/sqlalchemy/engine/default.py", line 588, in do_execute
      cursor.execute(statement, parameters)
    File "/usr/lib/python3/dist-packages/pymysql/cursors.py", line 165, in execute
      result = self._query(query)
    File "/usr/lib/python3/dist-packages/pymysql/cursors.py", line 321, in _query
      conn.query(q)
    File "/usr/lib/python3/dist-packages/pymysql/connections.py", line 860, in query
      self._affected_rows = self._read_query_result(unbuffered=unbuffered)
    File "/usr/lib/python3/dist-packages/pymysql/connections.py", line 1061, in _read_query_result
      result.read()
    File "/usr/lib/python3/dist-packages/pymysql/connections.py", line 1349, in read
      first_packet = self.connection._read_packet()
    File "/usr/lib/python3/dist-packages/pymysql/connections.py", line 1018, in _read_packet
      packet.check_error()
    File "/usr/lib/python3/dist-packages/pymysql/connections.py", line 384, in check_error
      err.raise_mysql_exception(self._data)
    File "/usr/lib/python3/dist-packages/pymysql/err.py", line 107, in raise_mysql_exception
      raise errorclass(errno, errval)
  oslo_db.exception.DBError: (pymysql.err.InternalError) (1832, "Cannot change column 'firewall_group_id': used in a foreign key constraint 'firewall_group_port_associations_v2_ibfk_1'")
  [SQL: ALTER TABLE firewall_group_port_associations_v2 ADD CONSTRAINT pk_firewall_group_port_associations_v2 PRIMARY KEY (firewall_group_id, port_id)]
  (Background on this error at: http://sqlalche.me/e/2j85)

  Package Info:

  Package: python3-neutron-fwaas
  Version: 1:14.0.1-0ubuntu1~cloud0
  Priority: optional
  Section: python
  Source: neutron-fwaas
  Maintainer: Ubuntu Developers <ubuntu-devel-discuss at lists.ubuntu.com>
  Installed-Size: 1195 kB
  Depends: neutron-fwaas-common (= 1:14.0.1-0ubuntu1~cloud0), python3-alembic (>= 0.8.10), python3-eventlet (>= 0.18.2), python3-netaddr (>= 0.7.18), python3-neutron (>= 2:14.0.0~), python3-neutron-lib (>= 1.25.0), python3-os-ken (>= 0.3.0), python3-oslo.config (>= 1:5.2.0), python3-oslo.db (>= 4.37.0), python3-oslo.log (>= 3.36.0), python3-oslo.messaging (>= 5.29.0), python3-oslo.privsep (>= 1.32.0), python3-oslo.serialization (>= 2.18.0), python3-oslo.service (>= 1.24.0), python3-oslo.utils (>= 3.33.0), python3-pbr (>= 2.0.0), python3-pyroute2 (>= 0.4.21), python3-six (>= 1.10.0), python3-sqlalchemy (>= 1.2.0), python3:any
  Suggests: python3-zmq
  Supported: 48m
  Download-Size: 138 kB
  APT-Manual-Installed: no
  APT-Sources: http://ubuntu-cloud.archive.canonical.com/ubuntu bionic-updates/stein/main amd64 Packages
  Description: Firewall-as-a-Service driver for OpenStack Neutron

  Db info:

  Package: mariadb-server
  Version: 1:10.1.44+maria-1~bionic
  Priority: optional
  Section: database
  Source: mariadb-10.1
  Maintainer: MariaDB Developers <maria-developers at lists.launchpad.net>
  Installed-Size: 10.2 kB
  Depends: mariadb-server-10.1 (= 1:10.1.44+maria-1~bionic)
  Homepage: http://mariadb.org/
  Download-Size: 3012 B
  APT-Manual-Installed: yes
  APT-Sources: http://lon1.mirrors.digitalocean.com/mariadb/repo/10.1/ubuntu bionic/main amd64 Packages

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/neutron-fwaas/+bug/1863257/+subscriptions

More information about the Ubuntu-openstack-bugs mailing list