[Bug 1832075] Re: [19.04][Queens -> Rocky] python3-pymysql is not installed before use

[Corey Bryant]

I also was unable to recreate this with an upgrade from queens to rocky
via the charms.

I think we're going to need to have the charm install python3-mysqldb
(for python3-django suggested dep) for eoan/train and above though, but
that is a different issue. I opened a bug for it at
https://bugs.launchpad.net/charm-openstack-dashboard/+bug/1835301.

-- 
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to horizon in Ubuntu.
https://bugs.launchpad.net/bugs/1832075

Title:
  [19.04][Queens -> Rocky] python3-pymysql is not installed before use

Status in OpenStack openstack-dashboard charm:
  Incomplete
Status in horizon package in Ubuntu:
  Invalid

Bug description:
  Ran an upgrade from bionic-queens to bionic-rocky and encountered the
  following:

  /var/log/apt/term.log

  https://paste.ubuntu.com/p/n6WZJbDy6p/

    File "/usr/lib/python3/dist-packages/django/db/backends/mysql/base.py", line 25, in <module>
      import pymysql
  ModuleNotFoundError: No module named 'pymysql'
  dpkg: error processing package openstack-dashboard

  The problem is that python3-django only "suggests" that
  python3-pymysql is installed and does not depend on it while

  root at juju-f2e601-4-lxd-2:~# dpkg -l | grep pymysql
  ii  python-pymysql                   0.8.0-1                           all          Pure-Python MySQL driver - Python 2.x

  root at juju-f2e601-4-lxd-2:~# dpkg -l | grep -P 'python[3]?-django\s'
  ii  python-django                    1:1.11.11-1ubuntu1.3              all          High-level Python web development framework (Python 2 version)
  ii  python3-django                   1:1.11.11-1ubuntu1.3              all          High-level Python web development framework (Python 3 version)

  # python3-django does not depend on pymysql - only suggests it

  ```
  apt-rdepends -f Depends,PreDepends,Suggests python3-django | grep pymysql
  Reading package lists... Done
  Building dependency tree
  Reading state information... Done

  python3-pymysql
  python-pymysql-doc
  ```

  While during openstack-dashboard package setup there is a code path
  leading to an import of pymysql. See below the following line in the
  term.log:

  "Setting up openstack-dashboard (3:15.0.0-0ubuntu1~cloud0) ..."

  The dashboard charm contains python3-pymysql, however, and apt looks
  to be processing packages in the order they are specified (which leads
  to openstack-dashboard being set up before python3-pymysql as there is
  no direct dependency).

  BASE_PACKAGES = [
      'haproxy',
      'memcached',
      'openstack-dashboard',
      'openstack-dashboard-ubuntu-theme',
      'python-keystoneclient',
      'python-memcache',
      'python-novaclient',
  ]

  PY3_PACKAGES = [
      'python3-django-horizon',
      'python3-designate-dashboard',
      'python3-heat-dashboard',
      'python3-neutron-lbaas-dashboard',
      'python3-neutron-fwaas-dashboard',
      'python3-keystoneclient',
      'python3-novaclient',
      'python3-memcache',
      'python3-pymysql',
      'libapache2-mod-wsgi-py3',
  ]

  
  BASE_PACKAGES go first, then PY3_PACKAGES:
  # ...
  def determine_packages():
      """Determine packages to install"""
      packages = deepcopy(BASE_PACKAGES)
  # ...
      if release >= 'rocky':
          packages = [p for p in packages if not p.startswith('python-')]
          packages.extend(PY3_PACKAGES)

To manage notifications about this bug go to:
https://bugs.launchpad.net/charm-openstack-dashboard/+bug/1832075/+subscriptions