[Bug 1782922] Re: LDAP: changing user_id_attribute bricks group mapping
Felipe Reyes
1782922 at bugs.launchpad.net
Wed Nov 27 02:54:19 UTC 2019
I tested the fix for this code following the instructions at
https://launchpadlibrarian.net/449185359/bug-1782922-testing.txt and
everything works ok, and no regressions were detected.
testing bed log:
$ tox -e func-smoke
func-smoke installed: DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support,amulet==1.21.0,aodhclient==1.3.0,appdirs==1.4.3,Babel==2.7.0,backports.os==0.1.1,blessings==1.6,bundletester==0.12.2,certifi==2019.9.11,cffi==1.13.1,chardet==3.0.4,charm-tools==2.7.2,charmhelpers==0.20.4,Cheetah3==3.2.4,cliff==2.16.0,cmd2==0.8.9,colander==1.7.0,configparser==4.0.2,contextlib2==0.6.0.post1,coverage==4.5.4,cryptography==2.8,debtcollector==1.22.0,decorator==4.4.0,dict2colander==0.2,distro==1.4.0,distro-info==0.0.0,dogpile.cache==0.8.0,entrypoints==0.3,enum34==1.1.6,extras==1.0.0,fasteners==0.15,fixtures==3.0.0,flake8==2.4.1,funcsigs==1.0.2,functools32==3.2.3.post2,future==0.18.1,futures==3.3.0,futurist==1.9.0,gnocchiclient==3.1.1,httplib2==0.14.0,idna==2.8,importlib-metadata==0.23,ipaddress==1.0.23,iso8601==0.1.12,Jinja2==2.10.3,jmespath==0.9.4,jsonpatch==1.24,jsonpointer==2.0,jsonschema==2.5.1,juju-deployer==0.11.0,juju-wait==2.5.0,jujubundlelib==0.5.6,jujuclient==0.54.0,keyring==18.0.1,keystoneauth1==3.18.0,launchpadlib==1.10.7,lazr.authentication==0.1.3,lazr.restfulclient==0.14.2,lazr.uri==1.0.3,libcharmstore==0.0.9,linecache2==1.0.0,macaroonbakery==1.2.3,MarkupSafe==1.1.1,mccabe==0.3.1,mock==3.0.5,monotonic==1.5,more-itertools==5.0.0,msgpack==0.6.2,munch==2.3.2,netaddr==0.7.19,netifaces==0.10.9,nose==1.3.7,oauth==1.0.1,oauthlib==3.1.0,openstacksdk==0.36.0,os-client-config==1.33.0,os-service-types==1.7.0,osc-lib==1.14.1,oslo.concurrency==3.30.0,oslo.config==6.11.1,oslo.context==2.23.0,oslo.i18n==3.24.0,oslo.log==3.44.1,oslo.serialization==2.29.2,oslo.utils==3.41.2,osprofiler==2.8.2,otherstuf==1.1.0,parse==1.12.1,path.py==11.5.2,pathlib2==2.3.5,pathspec==0.3.4,pbr==5.4.3,pep8==1.7.1,pika==0.13.1,pkg-resources==0.0.0,prettytable==0.7.2,protobuf==3.10.0,pycparser==2.19,pyflakes==0.8.1,pyinotify==0.9.6,pymacaroons==0.13.0,PyNaCl==1.3.0,pyOpenSSL==19.0.0,pyparsing==2.4.2,pyperclip==1.7.0,pyRFC3339==1.1,python-barbicanclient==4.9.0,python-ceilometerclient==2.9.0,python-cinderclient==4.3.0,python-dateutil==2.8.0,python-designateclient==3.0.0,python-glanceclient==2.17.0,python-heatclient==1.18.0,python-keystoneclient==3.22.0,python-manilaclient==1.29.0,python-mimeparse==1.6.0,python-neutronclient==6.14.0,python-novaclient==16.0.0,python-openstackclient==4.0.0,python-subunit==1.3.0,python-swiftclient==3.8.1,pytz==2019.3,pyudev==0.21.0,PyYAML==3.13,requests==2.22.0,requestsexceptions==1.4.0,rfc3986==1.3.2,ruamel.ordereddict==0.4.14,ruamel.yaml==0.15.100,scandir==1.10.0,SecretStorage==2.3.1,simplejson==3.16.0,six==1.12.0,stestr==2.5.1,stevedore==1.31.0,stuf==0.9.16,subprocess32==3.5.4,Tempita==0.5.2,testresources==2.0.1,testtools==2.3.0,theblues==0.5.2,traceback2==1.4.0,translationstring==1.3,unicodecsv==0.14.1,unittest2==1.1.0,urllib3==1.25.6,vergit==1.0.2,virtualenv==16.7.7,voluptuous==0.11.7,wadllib==1.3.3,warlock==1.3.3,wcwidth==0.1.7,WebOb==1.8.5,websocket-client==0.40.0,wrapt==1.11.2,wsgi-intercept==1.9.0,zipp==0.6.0,zope.interface==4.6.0
func-smoke run-test-pre: PYTHONHASHSEED='0'
func-smoke runtests: commands[0] | bundletester -vl DEBUG -r json -o func-results.json gate-basic-bionic-queens --no-destroy
DEBUG:bundletester.utils:Updating JUJU_MODEL: "" -> "laptop:admin/lp1782922-bionic"
DEBUG:root:Bootstrap environment: laptop:admin/lp1782922-bionic
DEBUG:deployer.env:Connecting to laptop:admin/lp1782922-bionic...
DEBUG:jujuclient.connector:Connecting to wss://10.5.0.7:17070/model/9869a39e-c6c2-4ecd-8e7d-e5736d15ca51/api
DEBUG:deployer.env:Connected.
DEBUG:deployer.env: Terminating machines forcefully
INFO:deployer.env: Waiting for machine termination
DEBUG:jujuclient.connector:Connecting to wss://10.5.0.7:17070/model/9869a39e-c6c2-4ecd-8e7d-e5736d15ca51/api
DEBUG:root:Waiting for applications to be removed...
DEBUG:runner:call ['/home/freyes/Projects/charms/openstack/builds/keystone-ldap/.tox/func-smoke/bin/charm-proof'] (cwd: /tmp/bundletester-AmwJen/keystone-ldap)
DEBUG:runner:I: `display-name` not provided, add for custom naming in the UI
DEBUG:runner:I: config.yaml: option ssl_key has no default value
DEBUG:runner:I: config.yaml: option ssl_cert has no default value
DEBUG:runner:I: config.yaml: option ldap-user has no default value
DEBUG:runner:I: config.yaml: option ldap-server has no default value
DEBUG:runner:I: config.yaml: option ssl_ca has no default value
DEBUG:runner:I: config.yaml: option ldap-password has no default value
DEBUG:runner:I: config.yaml: option domain-name has no default value
DEBUG:runner:I: config.yaml: option ldap-suffix has no default value
DEBUG:runner:I: config.yaml: option ldap-config-flags has no default value
DEBUG:runner:I: config.yaml: option tls-ca-ldap has no default value
DEBUG:runner:Exit Code: 0
DEBUG:deployer.env: Terminating machines forcefully
INFO:deployer.env: Waiting for machine termination
DEBUG:jujuclient.connector:Connecting to wss://10.5.0.7:17070/model/9869a39e-c6c2-4ecd-8e7d-e5736d15ca51/api
DEBUG:root:Waiting for applications to be removed...
DEBUG:runner:call ['/tmp/bundletester-AmwJen/keystone-ldap/tests/gate-basic-bionic-queens'] (cwd: /tmp/bundletester-AmwJen/keystone-ldap)
DEBUG:runner:2019-11-26 22:17:07,892 __init__ INFO: OpenStackAmuletDeployment: init
DEBUG:runner:2019-11-26 22:17:07,892 _add_services INFO: OpenStackAmuletDeployment: adding services
DEBUG:runner:2019-11-26 22:17:07,892 _determine_branch_locations INFO: OpenStackAmuletDeployment: determine branch locations
DEBUG:runner:2019-11-26 22:17:11 Starting deployment of laptop:admin/lp1782922-bionic
DEBUG:runner:2019-11-26 22:17:14 Deploying applications...
DEBUG:runner:2019-11-26 22:17:14 Deploying application keystone using cs:~openstack-charmers-next/keystone-467
DEBUG:runner:2019-11-26 22:17:30 Deploying application keystone-ldap using /tmp/charmZXGKer/bionic/keystone-ldap
DEBUG:runner:2019-11-26 22:20:30 Deploying application ldap-server using /tmp/charm7X3z5h/bionic/charm-ldap-test-fixture
DEBUG:runner:2019-11-26 22:20:37 Deploying application percona-cluster using cs:~openstack-charmers-next/percona-cluster-356
DEBUG:runner:2019-11-26 22:20:52 Config specifies num units for subordinate: keystone-ldap
DEBUG:runner:2019-11-26 22:29:46 Adding relations...
DEBUG:runner:2019-11-26 22:29:47 Adding relation keystone:shared-db <-> percona-cluster:shared-db
DEBUG:runner:2019-11-26 22:29:47 Adding relation keystone:domain-backend <-> keystone-ldap:domain-backend
DEBUG:runner:2019-11-26 22:32:51 Deployment complete in 940.16 seconds
DEBUG:runner:2019-11-26 22:34:07,698 _configure_services INFO: OpenStackAmuletDeployment: configure services
DEBUG:runner:2019-11-26 22:34:16,233 __init__ INFO: Waiting on extended status checks...
DEBUG:runner:2019-11-26 22:34:16,234 _auto_wait_for_status INFO: Waiting for extended status on units for 5400s...
DEBUG:runner:2019-11-26 22:34:16,234 _auto_wait_for_status DEBUG: Default extended status wait match: contains READY (case-insensitive)
DEBUG:runner:2019-11-26 22:34:16,235 _auto_wait_for_status DEBUG: Excluding services from extended status match: ['mysql', 'mongodb']
DEBUG:runner:2019-11-26 22:34:16,236 _auto_wait_for_status DEBUG: Waiting up to 5400s for extended status on services: ['keystone-ldap', 'keystone', 'ldap-server', 'percona-cluster']
DEBUG:runner:2019-11-26 22:36:04,410 _auto_wait_for_status INFO: OK
DEBUG:runner:2019-11-26 22:36:19,010 get_default_keystone_session DEBUG: Authenticating keystone admin...
DEBUG:runner:Exit Code: 0
DEBUG:bundletester.utils:Updating JUJU_MODEL: "laptop:admin/lp1782922-bionic" -> ""
_____________________________________________________________________________________________________ summary ______________________________________________________________________________________________________
func-smoke: commands succeeded
congratulations :)
$ juju status
Model Controller Cloud/Region Version SLA Timestamp
lp1782922-bionic laptop stsstack/stsstack 2.7-rc6 unsupported 23:50:14-03:00
App Version Status Scale Charm Store Rev OS Notes
keystone 13.0.2 active 1 keystone jujucharms 467 ubuntu
keystone-ldap 13.0.2 active 1 keystone-ldap local 0 ubuntu
ldap-server active 1 ldap-test-fixture local 0 ubuntu
percona-cluster 5.7.20 active 1 percona-cluster jujucharms 356 ubuntu
Unit Workload Agent Machine Public address Ports Message
keystone/0* active idle 0 10.5.0.9 5000/tcp Unit is ready
keystone-ldap/0* active idle 10.5.0.9 Unit is ready
ldap-server/0* active idle 1 10.5.0.23 Unit is ready
percona-cluster/0* active idle 2 10.5.0.6 3306/tcp Unit is ready
Machine State DNS Inst id Series AZ Message
0 started 10.5.0.9 183a9407-2a57-4695-a7fb-8381a572a9da bionic nova ACTIVE
1 started 10.5.0.23 7124ee40-07ad-490f-9284-0d55a2b09e26 bionic nova ACTIVE
2 started 10.5.0.6 4b225e27-f482-4e2f-a389-656201b2b532 bionic nova ACTIVE
##### scenario 1 #######
$ juju ssh keystone/0 sudo -i
root at juju-15ca51-lp1782922-bionic-0:~# vim /etc/keystone/domains/keystone.userdomain.conf
root at juju-15ca51-lp1782922-bionic-0:~# logout
Connection to 10.5.0.9 closed.
~/Projects/charms/openstack/builds/keystone-ldap $ juju config keystone debug=true'
> ^C
~/Projects/charms/openstack/builds/keystone-ldap $ juju config keystone debug=true
~/Projects/charms/openstack/builds/keystone-ldap $ source ~/Projects/charms/openstack/openstack-charm-testing/openrcv3_project
bash: /home/freyes/Projects/charms/openstack/openstack-charm-testing/openrcv3_project: No such file or directory
~/Projects/charms/openstack/builds/keystone-ldap $ source ~/Projects/charms/openstack/openstack-charm-testing/novarcv3_project
~/Projects/charms/openstack/builds/keystone-ldap $ openstack user list --domain userdomain
+------------------------------------------------------------------+----------+
| ID | Name |
+------------------------------------------------------------------+----------+
| ddd06b170639818c3e3823d61ad4ca4de62ba80389eab07af9c20853dc970eb5 | Jane Doe |
| f9f63cb865b8e4fde89962a861bc72515fbe8d0eac437f9dd03231fb0ddc1778 | John Doe |
+------------------------------------------------------------------+----------+
~/Projects/charms/openstack/builds/keystone-ldap $ openstack group list --domain userdomain
+------------------------------------------------------------------+-------+
| ID | Name |
+------------------------------------------------------------------+-------+
| d8cdd7b09be841e4fea7535cc408cb8858a67be0d91b933d078c33355fae43e2 | cloud |
+------------------------------------------------------------------+-------+
~/Projects/charms/openstack/builds/keystone-ldap $ openstack user list --group cloud --domain userdomain
$ juju ssh keystone/0
Welcome to Ubuntu 18.04.3 LTS (GNU/Linux 4.15.0-65-generic x86_64)
* Documentation: https://help.ubuntu.com
* Management: https://landscape.canonical.com
* Support: https://ubuntu.com/advantage
System information as of Wed Nov 27 01:45:18 UTC 2019
System load: 0.13 Processes: 99
Usage of /: 11.3% of 19.21GB Users logged in: 0
Memory usage: 26% IP address for ens3: 10.5.0.9
Swap usage: 0% IP address for fan-252: 252.0.9.1
0 packages can be updated.
0 updates are security updates.
*** System restart required ***
Last login: Wed Nov 27 01:37:31 2019 from 10.5.0.4
ubuntu at juju-15ca51-lp1782922-bionic-0:~$ sudo apt-cache policy keystone-common
N: Unable to locate package keystone-common
ubuntu at juju-15ca51-lp1782922-bionic-0:~$ sudo apt-cache policy keystone
keystone:
Installed: 2:13.0.2-0ubuntu1
Candidate: 2:13.0.2-0ubuntu1
Version table:
*** 2:13.0.2-0ubuntu1 500
500 http://nova.clouds.archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages
100 /var/lib/dpkg/status
2:13.0.0-0ubuntu1 500
500 http://nova.clouds.archive.ubuntu.com/ubuntu bionic/main amd64 Packages
ubuntu at juju-15ca51-lp1782922-bionic-0:~$ sudo vim /etc/apt/sources.list
[1]+ Stopped sudo vim /etc/apt/sources.list
ubuntu at juju-15ca51-lp1782922-bionic-0:~$ fg
sudo vim /etc/apt/sources.list
ubuntu at juju-15ca51-lp1782922-bionic-0:~$ sudo apt-get update -qq
ubuntu at juju-15ca51-lp1782922-bionic-0:~$ sudo apt-cache policy keystone
keystone:
Installed: 2:13.0.2-0ubuntu1
Candidate: 2:13.0.2-0ubuntu3
Version table:
2:13.0.2-0ubuntu3 500
500 http://nova.clouds.archive.ubuntu.com/ubuntu bionic-proposed/main amd64 Packages
*** 2:13.0.2-0ubuntu1 500
500 http://nova.clouds.archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages
100 /var/lib/dpkg/status
2:13.0.0-0ubuntu1 500
500 http://nova.clouds.archive.ubuntu.com/ubuntu bionic/main amd64 Packages
ubuntu at juju-15ca51-lp1782922-bionic-0:~$ sudo apt-get upgrade -qq -y
Preconfiguring packages ...
...
ubuntu at juju-15ca51-lp1782922-bionic-0:~$ sudo apt-cache policy keystone
keystone:
Installed: 2:13.0.2-0ubuntu3
Candidate: 2:13.0.2-0ubuntu3
Version table:
*** 2:13.0.2-0ubuntu3 500
500 http://nova.clouds.archive.ubuntu.com/ubuntu bionic-proposed/main amd64 Packages
100 /var/lib/dpkg/status
2:13.0.2-0ubuntu1 500
500 http://nova.clouds.archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages
2:13.0.0-0ubuntu1 500
500 http://nova.clouds.archive.ubuntu.com/ubuntu bionic/main amd64 Packages
ubuntu at juju-15ca51-lp1782922-bionic-0:~$ logout
Connection to 10.5.0.9 closed.
$ juju config keystone debug=false
$ openstack user list --domain userdomain
+------------------------------------------------------------------+----------+
| ID | Name |
+------------------------------------------------------------------+----------+
| ddd06b170639818c3e3823d61ad4ca4de62ba80389eab07af9c20853dc970eb5 | Jane Doe |
| f9f63cb865b8e4fde89962a861bc72515fbe8d0eac437f9dd03231fb0ddc1778 | John Doe |
+------------------------------------------------------------------+----------+
$ openstack group list --domain userdomain
+------------------------------------------------------------------+-------+
| ID | Name |
+------------------------------------------------------------------+-------+
| d8cdd7b09be841e4fea7535cc408cb8858a67be0d91b933d078c33355fae43e2 | cloud |
+------------------------------------------------------------------+-------+
$ openstack user list --group cloud --domain userdomain
+------------------------------------------------------------------+----------+
| ID | Name |
+------------------------------------------------------------------+----------+
| f9f63cb865b8e4fde89962a861bc72515fbe8d0eac437f9dd03231fb0ddc1778 | John Doe |
| ddd06b170639818c3e3823d61ad4ca4de62ba80389eab07af9c20853dc970eb5 | Jane Doe |
+------------------------------------------------------------------+----------+
##### scenario 2 #######
$ juju ssh keystone/0 sudo -i
root at juju-15ca51-lp1782922-bionic-0:~# vim /etc/keystone/domains/keystone.userdomain.conf
root at juju-15ca51-lp1782922-bionic-0:~# logout
Connection to 10.5.0.9 closed.
$ juju config keystone debug=true
$ openstack user list --domain userdomain
+------------------------------------------------------------------+----------+
| ID | Name |
+------------------------------------------------------------------+----------+
| 4904d77e920a12b81b44ea6f789d805c45540e21466df4e9bae4f1dda4f684e7 | Jane Doe |
| 0ca611873bdc0978e9ac55d1591019e0aa5d1b7f018232d81318e7eaa4336b75 | John Doe |
+------------------------------------------------------------------+----------+
~/Projects/charms/openstack/builds/keystone-ldap $ openstack group list --domain userdomain
+------------------------------------------------------------------+-------+
| ID | Name |
+------------------------------------------------------------------+-------+
| d8cdd7b09be841e4fea7535cc408cb8858a67be0d91b933d078c33355fae43e2 | cloud |
+------------------------------------------------------------------+-------+
~/Projects/charms/openstack/builds/keystone-ldap $ openstack user list --group cloud --domain userdomain
+------------------------------------------------------------------+----------+
| ID | Name |
+------------------------------------------------------------------+----------+
| 0ca611873bdc0978e9ac55d1591019e0aa5d1b7f018232d81318e7eaa4336b75 | John Doe |
| 4904d77e920a12b81b44ea6f789d805c45540e21466df4e9bae4f1dda4f684e7 | Jane Doe |
+------------------------------------------------------------------+----------+
** Tags removed: verification-failed-bionic
** Tags added: verification-done-bionic
--
You received this bug notification because you are a member of Ubuntu
OpenStack, which is subscribed to keystone in Ubuntu.
https://bugs.launchpad.net/bugs/1782922
Title:
LDAP: changing user_id_attribute bricks group mapping
Status in Ubuntu Cloud Archive:
Triaged
Status in Ubuntu Cloud Archive queens series:
Fix Committed
Status in Ubuntu Cloud Archive rocky series:
Fix Released
Status in Ubuntu Cloud Archive stein series:
Fix Released
Status in Ubuntu Cloud Archive train series:
Fix Released
Status in OpenStack Identity (keystone):
Fix Released
Status in keystone package in Ubuntu:
Fix Released
Status in keystone source package in Bionic:
Fix Committed
Status in keystone source package in Cosmic:
Won't Fix
Status in keystone source package in Disco:
Fix Released
Status in keystone source package in Eoan:
Fix Released
Bug description:
[Impact]
When using the keystone LDAP backend, changing user_id_attribute breaks group mapping. This is because the _dn_to_id() method only calculated the uid to be the first RDN of the DN. _dn_to_id() is updated in the fix to also deal with the case where the uid is set to a different attribute.
[Test Case]
See details in comment #25: https://bugs.launchpad.net/keystone/+bug/1782922/comments/25
[Regression Potential]
The patch takes a minimal approach to the fix and includes unit tests to help ensure the patched code doesn't regress. The patches have landed in all upstream releases back to stable/queens which helps get even more exposure with upstream reviews, gate testing and real deployments.
[Original Description]
Env Details:
Openstack version: Queens (17.0.5)
OS: CentOS 7.5
LDAP: Active Directory, Windows Server 2012R2
We changed the user_id_attribute to sAMAccountName when configuring
keystone. [ user_id_attribute = "sAMAccountName" ;
group_members_are_ids = False ]. Unfortunately this bricks the group
mapping logic in keystone.
The relevant code in keystone:
`list_users_in_group` [1] -> gets all groups from the LDAP server, and then calls `_transform_group_member_ids`. `_transform_group_member_ids` tries to match the user ids (for posixGroups e.g.) or the DN. However DN matching does not match the full DN. It rather takes the first RDN of the DN and computes the keystone user id [2]. The first RDN in Active Directory is the "CN". While the user-create part honors the user_id_attribute and takes "sAMAccountName" in our configuration. The generated user-ids in keystone now do not match anymore and hence group mapping is broken.
A fix could be looking up the user by the DN received from the
'member' attribute of a given group and compare the configured
'user_id_attribute' of the received ldap user id and the in keystone
stored user id. A quick fix could also be to mention that behavior in
the documentation.
/e: related
https://bugs.launchpad.net/keystone/+bug/1231488/comments/19
[1]
https://github.com/openstack/keystone/blob/master/keystone/identity/backends/ldap/common.py#L1285
[2]
https://github.com/openstack/keystone/blob/master/keystone/identity/backends/ldap/core.py#L126
[3]
https://github.com/openstack/keystone/blob/master/keystone/identity/backends/ldap/common.py#L1296
To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-archive/+bug/1782922/+subscriptions
More information about the Ubuntu-openstack-bugs
mailing list