[Bug 1929156] [NEW] can't use dogpile.cache.dbm cache in keystone

Shengjing Zhu 1929156 at bugs.launchpad.net
Fri May 21 04:00:56 UTC 2021 

Public bug reported:

There's regression in dogpile.cache 1.1, that dogpile.cache.dbm can't be
used with ProxyBackend, which is used in keystone.

Please see https://github.com/sqlalchemy/dogpile.cache/issues/202

```
2021-05-21 03:59:58.538 348 ERROR keystone Traceback (most recent call last):
2021-05-21 03:59:58.538 348 ERROR keystone   File "/usr/bin/keystone-manage", line 10, in <module>
2021-05-21 03:59:58.538 348 ERROR keystone     sys.exit(main())
2021-05-21 03:59:58.538 348 ERROR keystone   File "/usr/lib/python3/dist-packages/keystone/cmd/manage.py", line 41, in main
2021-05-21 03:59:58.538 348 ERROR keystone     cli.main(argv=sys.argv, developer_config_file=developer_config)
2021-05-21 03:59:58.538 348 ERROR keystone   File "/usr/lib/python3/dist-packages/keystone/cmd/cli.py", line 1376, in main
2021-05-21 03:59:58.538 348 ERROR keystone     CONF.command.cmd_class.main()
2021-05-21 03:59:58.538 348 ERROR keystone   File "/usr/lib/python3/dist-packages/keystone/cmd/cli.py", line 195, in main
2021-05-21 03:59:58.538 348 ERROR keystone     klass.do_bootstrap()
2021-05-21 03:59:58.538 348 ERROR keystone   File "/usr/lib/python3/dist-packages/keystone/cmd/cli.py", line 186, in do_bootstrap
2021-05-21 03:59:58.538 348 ERROR keystone     self.bootstrapper.bootstrap()
2021-05-21 03:59:58.538 348 ERROR keystone   File "/usr/lib/python3/dist-packages/keystone/cmd/bootstrap.py", line 64, in bootstrap
2021-05-21 03:59:58.538 348 ERROR keystone     self._bootstrap_project()
2021-05-21 03:59:58.538 348 ERROR keystone   File "/usr/lib/python3/dist-packages/keystone/cmd/bootstrap.py", line 103, in _bootstrap_project
2021-05-21 03:59:58.538 348 ERROR keystone     PROVIDERS.resource_api.create_project(project_id, project)
2021-05-21 03:59:58.538 348 ERROR keystone   File "/usr/lib/python3/dist-packages/keystone/common/manager.py", line 115, in wrapped
2021-05-21 03:59:58.538 348 ERROR keystone     __ret_val = __f(*args, **kwargs)
2021-05-21 03:59:58.538 348 ERROR keystone   File "/usr/lib/python3/dist-packages/keystone/resource/core.py", line 209, in create_project
2021-05-21 03:59:58.538 348 ERROR keystone     self._enforce_project_constraints(project)
2021-05-21 03:59:58.538 348 ERROR keystone   File "/usr/lib/python3/dist-packages/keystone/resource/core.py", line 150, in _enforce_project_constraints
2021-05-21 03:59:58.538 348 ERROR keystone     self._assert_regular_project_constraints(project_ref)
2021-05-21 03:59:58.538 348 ERROR keystone   File "/usr/lib/python3/dist-packages/keystone/resource/core.py", line 123, in _assert_regular_project_constraints
2021-05-21 03:59:58.538 348 ERROR keystone     domain = self.get_domain(project_ref['domain_id'])
2021-05-21 03:59:58.538 348 ERROR keystone   File "/usr/lib/python3/dist-packages/keystone/common/manager.py", line 115, in wrapped
2021-05-21 03:59:58.538 348 ERROR keystone     __ret_val = __f(*args, **kwargs)
2021-05-21 03:59:58.538 348 ERROR keystone   File "<decorator-gen-42>", line 2, in get_domain
2021-05-21 03:59:58.538 348 ERROR keystone   File "/usr/lib/python3/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func
2021-05-21 03:59:58.538 348 ERROR keystone     return self.get_or_create(
2021-05-21 03:59:58.538 348 ERROR keystone   File "/usr/lib/python3/dist-packages/dogpile/cache/region.py", line 974, in get_or_create
2021-05-21 03:59:58.538 348 ERROR keystone     key = self.key_mangler(key)
2021-05-21 03:59:58.538 348 ERROR keystone   File "/usr/lib/python3/dist-packages/keystone/common/cache/core.py", line 86, in key_mangler
2021-05-21 03:59:58.538 348 ERROR keystone     key = '%s:%s' % (key, invalidation_manager.region_id)
2021-05-21 03:59:58.538 348 ERROR keystone   File "/usr/lib/python3/dist-packages/keystone/common/cache/core.py", line 43, in region_id
2021-05-21 03:59:58.538 348 ERROR keystone     return self._invalidation_region.get_or_create(
2021-05-21 03:59:58.538 348 ERROR keystone   File "/usr/lib/python3/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create
2021-05-21 03:59:58.538 348 ERROR keystone     with Lock(
2021-05-21 03:59:58.538 348 ERROR keystone   File "/usr/lib/python3/dist-packages/dogpile/lock.py", line 185, in __enter__
2021-05-21 03:59:58.538 348 ERROR keystone     return self._enter()
2021-05-21 03:59:58.538 348 ERROR keystone   File "/usr/lib/python3/dist-packages/dogpile/lock.py", line 87, in _enter
2021-05-21 03:59:58.538 348 ERROR keystone     value = value_fn()
2021-05-21 03:59:58.538 348 ERROR keystone   File "/usr/lib/python3/dist-packages/dogpile/cache/region.py", line 977, in get_value
2021-05-21 03:59:58.538 348 ERROR keystone     value = self._get_from_backend(key)
2021-05-21 03:59:58.538 348 ERROR keystone   File "/usr/lib/python3/dist-packages/dogpile/cache/region.py", line 1265, in _get_from_backend
2021-05-21 03:59:58.538 348 ERROR keystone     self.backend.get_serialized(key)
2021-05-21 03:59:58.538 348 ERROR keystone   File "/usr/lib/python3/dist-packages/dogpile/cache/api.py", line 288, in get_serialized
2021-05-21 03:59:58.538 348 ERROR keystone     return cast(SerializedReturnType, self.get(key))
2021-05-21 03:59:58.538 348 ERROR keystone   File "/usr/lib/python3/dist-packages/keystone/common/cache/_context_cache.py", line 74, in get
2021-05-21 03:59:58.538 348 ERROR keystone     value = self.proxied.get(key)
2021-05-21 03:59:58.538 348 ERROR keystone   File "/usr/lib/python3/dist-packages/dogpile/cache/api.py", line 245, in get
2021-05-21 03:59:58.538 348 ERROR keystone     raise NotImplementedError()
2021-05-21 03:59:58.538 348 ERROR keystone NotImplementedError
2021-05-21 03:59:58.538 348 ERROR keystone 
```

My keystone.conf file

```
[auth]
methods = password,token,openid
[cache]
enabled = true
backend = dogpile.cache.dbm
backend_argument = filename:/var/lib/keystone/dogpile.cache
[database]
connection = mysql+pymysql://keystone:keystone@mysql/keystone
[oslo_policy]
enforce_scope = true
[openid]
remote_id_attribute = HTTP_OIDC_ISS
[security_compliance]
password_regex = ""
password_regex_description = ""
[token]
provider = fernet
expiration = "604800"
```

ProblemType: Bug
DistroRelease: Ubuntu 20.04
Package: python3-dogpile.cache 1.1.2-1~cloud0 [origin: Canonical]
Uname: Linux 5.10.0-6-amd64 x86_64
ApportVersion: 2.20.11-0ubuntu27.17
Architecture: amd64
CasperMD5CheckResult: skip
CrashDB:
 {
                "impl": "launchpad",
                "project": "cloud-archive",
                "bug_pattern_url": "http://people.canonical.com/~ubuntu-archive/bugpatterns/bugpatterns.xml",
             }
Date: Fri May 21 03:38:51 2021
PackageArchitecture: all
ProcEnviron:
 TERM=xterm
 PATH=(custom, no user)
SourcePackage: python-dogpile.cache
UpgradeStatus: No upgrade log present (probably fresh install)

** Affects: cloud-archive
     Importance: Undecided
         Status: New

** Affects: python-dogpile.cache (Ubuntu)
     Importance: Undecided
         Status: New


** Tags: amd64 apport-bug focal third-party-packages

** Also affects: python-dogpile.cache (Ubuntu)
   Importance: Undecided
       Status: New

-- 
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to Ubuntu Cloud Archive.
https://bugs.launchpad.net/bugs/1929156

Title:
  can't use dogpile.cache.dbm cache in keystone

Status in Ubuntu Cloud Archive:
  New
Status in python-dogpile.cache package in Ubuntu:
  New

Bug description:
  There's regression in dogpile.cache 1.1, that dogpile.cache.dbm can't
  be used with ProxyBackend, which is used in keystone.

  Please see https://github.com/sqlalchemy/dogpile.cache/issues/202

  ```
  2021-05-21 03:59:58.538 348 ERROR keystone Traceback (most recent call last):
  2021-05-21 03:59:58.538 348 ERROR keystone   File "/usr/bin/keystone-manage", line 10, in <module>
  2021-05-21 03:59:58.538 348 ERROR keystone     sys.exit(main())
  2021-05-21 03:59:58.538 348 ERROR keystone   File "/usr/lib/python3/dist-packages/keystone/cmd/manage.py", line 41, in main
  2021-05-21 03:59:58.538 348 ERROR keystone     cli.main(argv=sys.argv, developer_config_file=developer_config)
  2021-05-21 03:59:58.538 348 ERROR keystone   File "/usr/lib/python3/dist-packages/keystone/cmd/cli.py", line 1376, in main
  2021-05-21 03:59:58.538 348 ERROR keystone     CONF.command.cmd_class.main()
  2021-05-21 03:59:58.538 348 ERROR keystone   File "/usr/lib/python3/dist-packages/keystone/cmd/cli.py", line 195, in main
  2021-05-21 03:59:58.538 348 ERROR keystone     klass.do_bootstrap()
  2021-05-21 03:59:58.538 348 ERROR keystone   File "/usr/lib/python3/dist-packages/keystone/cmd/cli.py", line 186, in do_bootstrap
  2021-05-21 03:59:58.538 348 ERROR keystone     self.bootstrapper.bootstrap()
  2021-05-21 03:59:58.538 348 ERROR keystone   File "/usr/lib/python3/dist-packages/keystone/cmd/bootstrap.py", line 64, in bootstrap
  2021-05-21 03:59:58.538 348 ERROR keystone     self._bootstrap_project()
  2021-05-21 03:59:58.538 348 ERROR keystone   File "/usr/lib/python3/dist-packages/keystone/cmd/bootstrap.py", line 103, in _bootstrap_project
  2021-05-21 03:59:58.538 348 ERROR keystone     PROVIDERS.resource_api.create_project(project_id, project)
  2021-05-21 03:59:58.538 348 ERROR keystone   File "/usr/lib/python3/dist-packages/keystone/common/manager.py", line 115, in wrapped
  2021-05-21 03:59:58.538 348 ERROR keystone     __ret_val = __f(*args, **kwargs)
  2021-05-21 03:59:58.538 348 ERROR keystone   File "/usr/lib/python3/dist-packages/keystone/resource/core.py", line 209, in create_project
  2021-05-21 03:59:58.538 348 ERROR keystone     self._enforce_project_constraints(project)
  2021-05-21 03:59:58.538 348 ERROR keystone   File "/usr/lib/python3/dist-packages/keystone/resource/core.py", line 150, in _enforce_project_constraints
  2021-05-21 03:59:58.538 348 ERROR keystone     self._assert_regular_project_constraints(project_ref)
  2021-05-21 03:59:58.538 348 ERROR keystone   File "/usr/lib/python3/dist-packages/keystone/resource/core.py", line 123, in _assert_regular_project_constraints
  2021-05-21 03:59:58.538 348 ERROR keystone     domain = self.get_domain(project_ref['domain_id'])
  2021-05-21 03:59:58.538 348 ERROR keystone   File "/usr/lib/python3/dist-packages/keystone/common/manager.py", line 115, in wrapped
  2021-05-21 03:59:58.538 348 ERROR keystone     __ret_val = __f(*args, **kwargs)
  2021-05-21 03:59:58.538 348 ERROR keystone   File "<decorator-gen-42>", line 2, in get_domain
  2021-05-21 03:59:58.538 348 ERROR keystone   File "/usr/lib/python3/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func
  2021-05-21 03:59:58.538 348 ERROR keystone     return self.get_or_create(
  2021-05-21 03:59:58.538 348 ERROR keystone   File "/usr/lib/python3/dist-packages/dogpile/cache/region.py", line 974, in get_or_create
  2021-05-21 03:59:58.538 348 ERROR keystone     key = self.key_mangler(key)
  2021-05-21 03:59:58.538 348 ERROR keystone   File "/usr/lib/python3/dist-packages/keystone/common/cache/core.py", line 86, in key_mangler
  2021-05-21 03:59:58.538 348 ERROR keystone     key = '%s:%s' % (key, invalidation_manager.region_id)
  2021-05-21 03:59:58.538 348 ERROR keystone   File "/usr/lib/python3/dist-packages/keystone/common/cache/core.py", line 43, in region_id
  2021-05-21 03:59:58.538 348 ERROR keystone     return self._invalidation_region.get_or_create(
  2021-05-21 03:59:58.538 348 ERROR keystone   File "/usr/lib/python3/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create
  2021-05-21 03:59:58.538 348 ERROR keystone     with Lock(
  2021-05-21 03:59:58.538 348 ERROR keystone   File "/usr/lib/python3/dist-packages/dogpile/lock.py", line 185, in __enter__
  2021-05-21 03:59:58.538 348 ERROR keystone     return self._enter()
  2021-05-21 03:59:58.538 348 ERROR keystone   File "/usr/lib/python3/dist-packages/dogpile/lock.py", line 87, in _enter
  2021-05-21 03:59:58.538 348 ERROR keystone     value = value_fn()
  2021-05-21 03:59:58.538 348 ERROR keystone   File "/usr/lib/python3/dist-packages/dogpile/cache/region.py", line 977, in get_value
  2021-05-21 03:59:58.538 348 ERROR keystone     value = self._get_from_backend(key)
  2021-05-21 03:59:58.538 348 ERROR keystone   File "/usr/lib/python3/dist-packages/dogpile/cache/region.py", line 1265, in _get_from_backend
  2021-05-21 03:59:58.538 348 ERROR keystone     self.backend.get_serialized(key)
  2021-05-21 03:59:58.538 348 ERROR keystone   File "/usr/lib/python3/dist-packages/dogpile/cache/api.py", line 288, in get_serialized
  2021-05-21 03:59:58.538 348 ERROR keystone     return cast(SerializedReturnType, self.get(key))
  2021-05-21 03:59:58.538 348 ERROR keystone   File "/usr/lib/python3/dist-packages/keystone/common/cache/_context_cache.py", line 74, in get
  2021-05-21 03:59:58.538 348 ERROR keystone     value = self.proxied.get(key)
  2021-05-21 03:59:58.538 348 ERROR keystone   File "/usr/lib/python3/dist-packages/dogpile/cache/api.py", line 245, in get
  2021-05-21 03:59:58.538 348 ERROR keystone     raise NotImplementedError()
  2021-05-21 03:59:58.538 348 ERROR keystone NotImplementedError
  2021-05-21 03:59:58.538 348 ERROR keystone 
  ```

  My keystone.conf file

  ```
  [auth]
  methods = password,token,openid
  [cache]
  enabled = true
  backend = dogpile.cache.dbm
  backend_argument = filename:/var/lib/keystone/dogpile.cache
  [database]
  connection = mysql+pymysql://keystone:keystone@mysql/keystone
  [oslo_policy]
  enforce_scope = true
  [openid]
  remote_id_attribute = HTTP_OIDC_ISS
  [security_compliance]
  password_regex = ""
  password_regex_description = ""
  [token]
  provider = fernet
  expiration = "604800"
  ```

  ProblemType: Bug
  DistroRelease: Ubuntu 20.04
  Package: python3-dogpile.cache 1.1.2-1~cloud0 [origin: Canonical]
  Uname: Linux 5.10.0-6-amd64 x86_64
  ApportVersion: 2.20.11-0ubuntu27.17
  Architecture: amd64
  CasperMD5CheckResult: skip
  CrashDB:
   {
                  "impl": "launchpad",
                  "project": "cloud-archive",
                  "bug_pattern_url": "http://people.canonical.com/~ubuntu-archive/bugpatterns/bugpatterns.xml",
               }
  Date: Fri May 21 03:38:51 2021
  PackageArchitecture: all
  ProcEnviron:
   TERM=xterm
   PATH=(custom, no user)
  SourcePackage: python-dogpile.cache
  UpgradeStatus: No upgrade log present (probably fresh install)

To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-archive/+bug/1929156/+subscriptions

More information about the Ubuntu-openstack-bugs mailing list