[Bug 1713059] Re: nova db sync is executed when the 'connection' is not defined

Felipe Reyes 1713059 at bugs.launchpad.net
Mon Aug 28 18:58:41 UTC 2017 

On Mon, Aug 28, 2017 at 03:44:42PM -0300, Felipe Reyes wrote:
> On Mon, Aug 28, 2017 at 06:25:17PM -0000, Corey Bryant wrote:
> > Hi Felipe
> > 
> > Historically we've configured OpenStack services to use sqlite at
> > install time, and it makes sense to run the db sync commands in that
> > case. However, I'm not sure why we also run the db sync commands if
> > connection isn't defined (! grep -qE "^(sql_)?connection( )?=.*"
> > /etc/nova/nova.conf).
> > 
> > We do this in a number of packages so I want to figure out why we do
> > that. We do the same in cinder, panko, aodh, nova, manila, to name a
> > few.
> > 
> > What we don't want to do is to run the db sync commands for non-sqlite
> > db's. If not using sqlite we leave it up to the user or config mgmt
> > tooling.
> 
> gotcha, then that's what that negation is meant for, the regexp needs to be tweaked a little bit to match only when "connection is config is set, but it's value contains sqlite"

Dropping the first grep:

--- /var/lib/dpkg/info/nova-common.postinst     2017-06-19 08:12:49.000000000 +0000
+++ ./nova-common.postinst      2017-08-28 18:51:01.367912483 +0000
@@ -47,8 +47,7 @@
     chown root:root /etc/nova/rootwrap.d
     chmod 0755 /etc/nova/rootwrap.d
 
-    if ! grep -qE "^(sql_)?connection( )?=.*" /etc/nova/nova.conf || \
-         grep -qE "^(sql_)?connection.*sqlite.*" /etc/nova/nova.conf
+    if grep -qE "^(sql_)?connection.*sqlite.*" /etc/nova/nova.conf
     then
       su -s /bin/sh -c 'nova-manage db sync' nova
     fi


Connection NOT defined:

root at juju-675da2-mno-17:~# bash -ex ./nova-common.postinst 
+ '[' '' = configure ']'
root at juju-675da2-mno-17:~# bash -ex ./nova-common.postinst configure
+ '[' configure = configure ']'
+ getent group nova
+ getent passwd nova
+ '[' -z '' ']'
+ chown -R nova:nova /var/lib/nova/
+ chown -R nova:nova /etc/nova
+ chown -R nova:adm /var/log/nova
+ '[' -z '' ']'
+ chown -R nova:nova /var/lib/nova/
+ chmod 0640 /etc/nova/nova.conf
+ chmod 0640 /etc/nova/api-paste.ini
+ chmod 0750 /etc/nova
+ chmod 0750 /var/log/nova
+ chown root:root /etc/nova/rootwrap.conf
+ chown root:root /etc/nova/rootwrap.d
+ chmod 0755 /etc/nova/rootwrap.d
+ grep -qE '^(sql_)?connection.*sqlite.*' /etc/nova/nova.conf
+ '[' -e /var/lib/nova/nova.sqlite ']'
+ chown nova:nova /var/lib/nova/nova.sqlite
+ chmod 0640 /var/lib/nova/nova.sqlite
root at juju-675da2-mno-17:~# grep connection /etc/nova/nova.conf

Connection defined:

root at juju-675da2-mno-17:~# bash -ex ./nova-common.postinst configure
+ '[' configure = configure ']'
+ getent group nova
+ getent passwd nova
+ '[' -z '' ']'
+ chown -R nova:nova /var/lib/nova/
+ chown -R nova:nova /etc/nova
+ chown -R nova:adm /var/log/nova
+ '[' -z '' ']'
+ chown -R nova:nova /var/lib/nova/
+ chmod 0640 /etc/nova/nova.conf
+ chmod 0640 /etc/nova/api-paste.ini
+ chmod 0750 /etc/nova
+ chmod 0750 /var/log/nova
+ chown root:root /etc/nova/rootwrap.conf
+ chown root:root /etc/nova/rootwrap.d
+ chmod 0755 /etc/nova/rootwrap.d
+ grep -qE '^(sql_)?connection.*sqlite.*' /etc/nova/nova.conf
+ '[' -e /var/lib/nova/nova.sqlite ']'
+ chown nova:nova /var/lib/nova/nova.sqlite
+ chmod 0640 /var/lib/nova/nova.sqlite
root at juju-675da2-mno-17:~# grep connection /etc/nova/nova.conf
connection = mysql://nova:XXXXXXXX@10.5.1.33/nova

-- 
You received this bug notification because you are a member of Ubuntu
Sponsors Team, which is subscribed to the bug report.
https://bugs.launchpad.net/bugs/1713059

Title:
  nova db sync  is executed when the 'connection' is not defined

Status in nova package in Ubuntu:
  New

Bug description:
  When the 'connection' configuration key is NOT defined in
  /etc/nova/nova.conf the nova-common.postinst script still tries to run
  'nova-manage db sync'

  root at juju-b24bc9-mno-18:~# grep connection /etc/nova/nova.conf
  root at juju-b24bc9-mno-18:~# dpkg-reconfigure nova-common
  Option "logdir" from group "DEFAULT" is deprecated. Use option "log-dir" from group "DEFAULT".
  Option "verbose" from group "DEFAULT" is deprecated for removal.  Its value may be silently ignored in the future.
  ERROR: could not access cell mapping database - has api db been created?
  An error has occurred:
  Traceback (most recent call last):
    File "/usr/lib/python2.7/dist-packages/nova/cmd/manage.py", line 1602, in main
      ret = fn(*fn_args, **fn_kwargs)
    File "/usr/lib/python2.7/dist-packages/nova/cmd/manage.py", line 645, in sync
      return migration.db_sync(version)
    File "/usr/lib/python2.7/dist-packages/nova/db/migration.py", line 26, in db_sync
      return IMPL.db_sync(version=version, database=database, context=context)
    File "/usr/lib/python2.7/dist-packages/nova/db/sqlalchemy/migration.py", line 57, in db_sync
      repository, version)
    File "/usr/lib/python2.7/dist-packages/migrate/versioning/api.py", line 186, in upgrade
      return _migrate(url, repository, version, upgrade=True, err=err, **opts)
    File "<decorator-gen-15>", line 2, in _migrate
    File "/usr/lib/python2.7/dist-packages/migrate/versioning/util/__init__.py", line 160, in with_engine
      return f(*a, **kw)
    File "/usr/lib/python2.7/dist-packages/migrate/versioning/api.py", line 366, in _migrate
      schema.runchange(ver, change, changeset.step)
    File "/usr/lib/python2.7/dist-packages/migrate/versioning/schema.py", line 93, in runchange
      change.run(self.engine, step)
    File "/usr/lib/python2.7/dist-packages/migrate/versioning/script/py.py", line 148, in run
      script_func(engine)
    File "/usr/lib/python2.7/dist-packages/nova/db/sqlalchemy/migrate_repo/versions/345_require_online_migration_completion.py", line 44, in upgrade
      raise exception.ValidationError(detail=msg)
  ValidationError: Migration cannot continue until all these have been migrated to the api database. Please run `nova-manage db online_migrations' on Newton code before continuing.There are still 5 unmigrated flavors. 

  Running the postinst script manually with -x

  root at juju-b24bc9-mno-18:~# bash -ex /var/lib/dpkg/info/nova-common.postinst configure
  + '[' configure = configure ']'
  + getent group nova
  + getent passwd nova
  + '[' -z '' ']'
  + chown -R nova:nova /var/lib/nova/
  + chown -R nova:nova /etc/nova
  + chown -R nova:adm /var/log/nova
  + '[' -z '' ']'
  + chown -R nova:nova /var/lib/nova/
  + chmod 0640 /etc/nova/nova.conf
  + chmod 0640 /etc/nova/api-paste.ini
  + chmod 0750 /etc/nova
  + chmod 0750 /var/log/nova
  + chown root:root /etc/nova/rootwrap.conf
  + chown root:root /etc/nova/rootwrap.d
  + chmod 0755 /etc/nova/rootwrap.d
  + grep -qE '^(sql_)?connection( )?=.*' /etc/nova/nova.conf
  + su -s /bin/sh -c 'nova-manage db sync' nova
  Option "logdir" from group "DEFAULT" is deprecated. Use option "log-dir" from group "DEFAULT".
  Option "verbose" from group "DEFAULT" is deprecated for removal.  Its value may be silently ignored in the future.
  ERROR: could not access cell mapping database - has api db been created?
  An error has occurred:
  Traceback (most recent call last):
  ...
  ValidationError: Migration cannot continue until all these have been migrated to the api database. Please run `nova-manage db online_migrations' on Newton code before continuing.There are still 5 unmigrated flavors. 

  
  The condition to run the sync operation should be that the connection config key is defined in nova.conf

  # diff -u /var/lib/dpkg/info/nova-common.postinst ./nova-common.postinst 
  --- /var/lib/dpkg/info/nova-common.postinst     2017-06-19 08:12:49.000000000 +0000
  +++ ./nova-common.postinst      2017-08-25 13:08:48.852744427 +0000
  @@ -47,8 +47,8 @@
       chown root:root /etc/nova/rootwrap.d
       chmod 0755 /etc/nova/rootwrap.d
   
  -    if ! grep -qE "^(sql_)?connection( )?=.*" /etc/nova/nova.conf || \
  -         grep -qE "^(sql_)?connection.*sqlite.*" /etc/nova/nova.conf
  +    if grep -qE "^(sql_)?connection( )?=.*" /etc/nova/nova.conf || \
  +       grep -qE "^(sql_)?connection.*sqlite.*" /etc/nova/nova.conf
       then
         su -s /bin/sh -c 'nova-manage db sync' nova
       fi

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/nova/+bug/1713059/+subscriptions

More information about the Ubuntu-sponsors mailing list