[Bug 1820083] Re: TLS params not set for session
Heather Lemon
1820083 at bugs.launchpad.net
Thu Mar 4 20:34:36 UTC 2021
** Description changed:
[Impact]
A connection session is opened, but the TLS parameters (timeout, ca,
cert and key) are not actually set for the session. This prevents use
of TLS for the etcd3gw package.
[Test Plan]
# Create self signed certs
openssl req -x509 -out localhost.crt -keyout localhost.key -newkey rsa:4096 -nodes -sha256 -out localhost.csr
*make sure the key has an empty password
#download binaries & launch etcd locally with TLS enabled
wget https://github.com/etcd-
io/etcd/releases/download/v3.3.13/etcd-v3.3.13-linux-amd64.tar.gz
tar -zxvf etcd-v3.3.14-linux-amd64.tar.gz
cd etcd-v3.3.14-linux-amd64/
sudo cp etcd etcdctl /usr/bin/
# spin up ectd server
etcd --name infra0 --data-dir infra0 --cert-file=localhost.crt --key-file=localhost.key --advertise-client-urls=https://127.0.0.1:2379 --listen-client-urls=https://127.0.0.1:2379
*note I named my directory infra0
#test connection with health endpoint:
curl --cacert localhost.crt --key localhost.key --cert localhost.crt
https://127.0.0.1:2379/health
#if successful, the etcd server is configured with https
{"health": "true"}
Modify ~/python-etcd3gw-0.2.1/etcd3gw/tests/test_client.py
to add this unit test.
def test_client_tls(self):
client = Etcd3Client(host="127.0.0.1", protocol="https", ca_cert="/root/etcdserver.crt",
cert_key="/root/etcdserver.key",
cert_cert="/root/etcdserver.crt",
timeout=10)
client.create("foo", value="bar")
- client.put("foo")
+ client.put("foo", "bar")
resp = client.get("foo")
print(resp)
# Run the newly added unit test
python3 -m unittest test_client.TestEtcd3Gateway.test_client_tls
We get an error in both the unit test and an error from the etcd server unit test error we are looking for:
# error in etcd
OpenSSL.SSL.Error: [('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')] related etcd error: I | embed: rejected connection from "127.0.0.1:44244" (error "remote error: tls: bad certificate", ServerName "")
error in unit test
python3 -m unittest test_client.TestEtcd3Gateway.test_client_tls
E
======================================================================
ERROR: test_client_tls (test_client.TestEtcd3Gateway)
test_client.TestEtcd3Gateway.test_client_tls
----------------------------------------------------------------------
testtools.testresult.real._StringException: Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 601, in urlopen
chunked=chunked)
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 346, in _make_request
self._validate_conn(conn)
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 852, in _validate_conn
conn.connect()
File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 340, in connect
ssl_context=context)
File "/usr/lib/python3/dist-packages/urllib3/util/ssl_.py", line 332, in ssl_wrap_socket
return context.wrap_socket(sock, server_hostname=server_hostname)
File "/usr/lib/python3.6/ssl.py", line 407, in wrap_socket
_context=self, _session=session)
File "/usr/lib/python3.6/ssl.py", line 817, in __init__
self.do_handshake()
File "/usr/lib/python3.6/ssl.py", line 1077, in do_handshake
self._sslobj.do_handshake()
File "/usr/lib/python3.6/ssl.py", line 689, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/requests/adapters.py", line 440, in send
timeout=timeout
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 639, in urlopen
_stacktrace=sys.exc_info()[2])
File "/usr/lib/python3/dist-packages/urllib3/util/retry.py", line 398, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='127.0.0.1', port=2379): Max retries exceeded with url: /v3alpha/kv/txn (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)'),))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/etcd3gw/client.py", line 89, in post
resp = self.session.post(*args, **kwargs)
File "/usr/lib/python3/dist-packages/requests/sessions.py", line 567, in post
return self.request('POST', url, data=data, json=json, **kwargs)
File "/usr/lib/python3/dist-packages/requests/sessions.py", line 520, in request
resp = self.send(prep, **send_kwargs)
File "/usr/lib/python3/dist-packages/requests/sessions.py", line 630, in send
r = adapter.send(request, **kwargs)
File "/usr/lib/python3/dist-packages/requests/adapters.py", line 506, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='127.0.0.1', port=2379): Max retries exceeded with url: /v3alpha/kv/txn (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)'),))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/root/githubsource-pythonetcd3gw/etcd3-gateway/etcd3gw/tests/test_client.py", line 186, in test_client_tls
client.create("foo2", value="bar2")
File "/usr/local/lib/python3.6/dist-packages/etcd3gw/client.py", line 175, in create
result = self.transaction(txn)
File "/usr/local/lib/python3.6/dist-packages/etcd3gw/client.py", line 350, in transaction
data=json.dumps(txn))
File "/usr/local/lib/python3.6/dist-packages/etcd3gw/client.py", line 100, in post
raise exceptions.ConnectionFailedError(six.text_type(ex))
etcd3gw.exceptions.ConnectionFailedError
----------------------------------------------------------------------
Ran 1 test in 0.023s
FAILED (errors=1)
# ran unit test again with patch fix
+ # adds new server to trusted ca list
+ sudo cp server.crt /usr/share/ca-certificates/
+ sudo dpkg-reconfigure ca-certificates
+ press spcaebar to select the cert and tab, enter to save
+ or run `sudo update-ca-certificates` does the same thing.
+
Unit test console output:
root at ubuntu-bionic:~/githubsource-pythonetcd3gw/etcd3-gateway/etcd3gw/tests# python3 -m unittest test_client.TestEtcd3Gateway.test_client_tls
/usr/lib/python3/dist-packages/urllib3/connection.py:358: SubjectAltNameWarning: Certificate for 127.0.0.1 has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/shazow/urllib3/issues/497 for details.)
SubjectAltNameWarning
[b'bar']
/usr/lib/python3/dist-packages/testtools/testcase.py:719: ResourceWarning: unclosed <ssl.SSLSocket fd=3, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('127.0.0.1', 38392), raddr=('127.0.0.1', 2379)>
return self._get_test_method()()
.
----------------------------------------------------------------------
Ran 1 test in 0.048s
OK
The unit test I've made is an echo of the test_client.py code we've just
updated.
#
Testing out the new unit test from the source code changes.
I have added a new unit test that tests the setting of TLS params.
You can run the unit test with:
python3 -m unittest
again make sure there's no etcd server already running.
[Where Problems Could Occur]
This adds TLS parameters (if provided) to the session, so regressions
would involve failed connections, possibly those without TLS that had
TLS params incorrectly provided before.
[Other]
the upstream bug is https://github.com/dims/etcd3-gateway/issues/20
fixed upstream with pull request https://github.com/dims/etcd3-gateway/pull/21
via commit 90b7a19cdc4daa1230d7f15c10b113abdefdc8c0
that commit is contained in version 0.2.2, which is not yet pulled into
Debian, so this patch is needed in Debian, as well as Bionic and Focal.
This package was not included in Xenial.
--
You received this bug notification because you are a member of Ubuntu
Sponsors Team, which is subscribed to the bug report.
https://bugs.launchpad.net/bugs/1820083
Title:
TLS params not set for session
Status in python-etcd3gw package in Ubuntu:
Fix Released
Status in python-etcd3gw source package in Bionic:
In Progress
Status in python-etcd3gw source package in Cosmic:
Won't Fix
Status in python-etcd3gw source package in Disco:
Won't Fix
Status in python-etcd3gw source package in Eoan:
Won't Fix
Status in python-etcd3gw source package in Focal:
In Progress
Status in python-etcd3gw source package in Groovy:
In Progress
Status in python-etcd3gw source package in Hirsute:
Fix Released
Bug description:
[Impact]
A connection session is opened, but the TLS parameters (timeout, ca,
cert and key) are not actually set for the session. This prevents use
of TLS for the etcd3gw package.
[Test Plan]
# Create self signed certs
openssl req -x509 -out localhost.crt -keyout localhost.key -newkey rsa:4096 -nodes -sha256 -out localhost.csr
*make sure the key has an empty password
#download binaries & launch etcd locally with TLS enabled
wget https://github.com/etcd-
io/etcd/releases/download/v3.3.13/etcd-v3.3.13-linux-amd64.tar.gz
tar -zxvf etcd-v3.3.14-linux-amd64.tar.gz
cd etcd-v3.3.14-linux-amd64/
sudo cp etcd etcdctl /usr/bin/
# spin up ectd server
etcd --name infra0 --data-dir infra0 --cert-file=localhost.crt --key-file=localhost.key --advertise-client-urls=https://127.0.0.1:2379 --listen-client-urls=https://127.0.0.1:2379
*note I named my directory infra0
#test connection with health endpoint:
curl --cacert localhost.crt --key localhost.key --cert localhost.crt
https://127.0.0.1:2379/health
#if successful, the etcd server is configured with https
{"health": "true"}
Modify ~/python-etcd3gw-0.2.1/etcd3gw/tests/test_client.py
to add this unit test.
def test_client_tls(self):
client = Etcd3Client(host="127.0.0.1", protocol="https", ca_cert="/root/etcdserver.crt",
cert_key="/root/etcdserver.key",
cert_cert="/root/etcdserver.crt",
timeout=10)
client.create("foo", value="bar")
client.put("foo", "bar")
resp = client.get("foo")
print(resp)
# Run the newly added unit test
python3 -m unittest test_client.TestEtcd3Gateway.test_client_tls
We get an error in both the unit test and an error from the etcd server unit test error we are looking for:
# error in etcd
OpenSSL.SSL.Error: [('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')] related etcd error: I | embed: rejected connection from "127.0.0.1:44244" (error "remote error: tls: bad certificate", ServerName "")
error in unit test
python3 -m unittest test_client.TestEtcd3Gateway.test_client_tls
E
======================================================================
ERROR: test_client_tls (test_client.TestEtcd3Gateway)
test_client.TestEtcd3Gateway.test_client_tls
----------------------------------------------------------------------
testtools.testresult.real._StringException: Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 601, in urlopen
chunked=chunked)
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 346, in _make_request
self._validate_conn(conn)
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 852, in _validate_conn
conn.connect()
File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 340, in connect
ssl_context=context)
File "/usr/lib/python3/dist-packages/urllib3/util/ssl_.py", line 332, in ssl_wrap_socket
return context.wrap_socket(sock, server_hostname=server_hostname)
File "/usr/lib/python3.6/ssl.py", line 407, in wrap_socket
_context=self, _session=session)
File "/usr/lib/python3.6/ssl.py", line 817, in __init__
self.do_handshake()
File "/usr/lib/python3.6/ssl.py", line 1077, in do_handshake
self._sslobj.do_handshake()
File "/usr/lib/python3.6/ssl.py", line 689, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/requests/adapters.py", line 440, in send
timeout=timeout
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 639, in urlopen
_stacktrace=sys.exc_info()[2])
File "/usr/lib/python3/dist-packages/urllib3/util/retry.py", line 398, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='127.0.0.1', port=2379): Max retries exceeded with url: /v3alpha/kv/txn (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)'),))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/etcd3gw/client.py", line 89, in post
resp = self.session.post(*args, **kwargs)
File "/usr/lib/python3/dist-packages/requests/sessions.py", line 567, in post
return self.request('POST', url, data=data, json=json, **kwargs)
File "/usr/lib/python3/dist-packages/requests/sessions.py", line 520, in request
resp = self.send(prep, **send_kwargs)
File "/usr/lib/python3/dist-packages/requests/sessions.py", line 630, in send
r = adapter.send(request, **kwargs)
File "/usr/lib/python3/dist-packages/requests/adapters.py", line 506, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='127.0.0.1', port=2379): Max retries exceeded with url: /v3alpha/kv/txn (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)'),))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/root/githubsource-pythonetcd3gw/etcd3-gateway/etcd3gw/tests/test_client.py", line 186, in test_client_tls
client.create("foo2", value="bar2")
File "/usr/local/lib/python3.6/dist-packages/etcd3gw/client.py", line 175, in create
result = self.transaction(txn)
File "/usr/local/lib/python3.6/dist-packages/etcd3gw/client.py", line 350, in transaction
data=json.dumps(txn))
File "/usr/local/lib/python3.6/dist-packages/etcd3gw/client.py", line 100, in post
raise exceptions.ConnectionFailedError(six.text_type(ex))
etcd3gw.exceptions.ConnectionFailedError
----------------------------------------------------------------------
Ran 1 test in 0.023s
FAILED (errors=1)
# ran unit test again with patch fix
# adds new server to trusted ca list
sudo cp server.crt /usr/share/ca-certificates/
sudo dpkg-reconfigure ca-certificates
press spcaebar to select the cert and tab, enter to save
or run `sudo update-ca-certificates` does the same thing.
Unit test console output:
root at ubuntu-bionic:~/githubsource-pythonetcd3gw/etcd3-gateway/etcd3gw/tests# python3 -m unittest test_client.TestEtcd3Gateway.test_client_tls
/usr/lib/python3/dist-packages/urllib3/connection.py:358: SubjectAltNameWarning: Certificate for 127.0.0.1 has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/shazow/urllib3/issues/497 for details.)
SubjectAltNameWarning
[b'bar']
/usr/lib/python3/dist-packages/testtools/testcase.py:719: ResourceWarning: unclosed <ssl.SSLSocket fd=3, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('127.0.0.1', 38392), raddr=('127.0.0.1', 2379)>
return self._get_test_method()()
.
----------------------------------------------------------------------
Ran 1 test in 0.048s
OK
The unit test I've made is an echo of the test_client.py code we've
just updated.
#
Testing out the new unit test from the source code changes.
I have added a new unit test that tests the setting of TLS params.
You can run the unit test with:
python3 -m unittest
again make sure there's no etcd server already running.
[Where Problems Could Occur]
This adds TLS parameters (if provided) to the session, so regressions
would involve failed connections, possibly those without TLS that had
TLS params incorrectly provided before.
[Other]
the upstream bug is https://github.com/dims/etcd3-gateway/issues/20
fixed upstream with pull request https://github.com/dims/etcd3-gateway/pull/21
via commit 90b7a19cdc4daa1230d7f15c10b113abdefdc8c0
that commit is contained in version 0.2.2, which is not yet pulled
into Debian, so this patch is needed in Debian, as well as Bionic and
Focal. This package was not included in Xenial.
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/python-etcd3gw/+bug/1820083/+subscriptions
More information about the Ubuntu-sponsors
mailing list