[Bug 1946520] [NEW] magic-proxy does not gracefully handle error conditions

[Launchpad Bug Tracker]

You have been subscribed to a public bug by Thomas Bechtold (toabctl):

During an impish build of ubuntu-cpc images with magic-proxy enabled,
the installation of certain packages tried to reach an internet endpoint
(specifically http://169.254.169.254/latest/meta-data/instance-id -
which appears to be used for EC2 metadata)

The magic-proxy log indicated :

10.10.10.2 - - [01/Oct/2021 13:21:26] urlopen() failed for http://169.254.169.254/2009-04-04/meta-data/instance-id with [Errno 110] Connection timed out
10.10.10.2 - - [01/Oct/2021 13:21:26] code 501, message [Errno 110] Connection timed out
10.10.10.2 - - [01/Oct/2021 13:21:26] "GET /2009-04-04/meta-data/instance-id HTTP/1.1" 501 -
----------------------------------------
Exception occurred during processing of request from ('10.10.10.2', 49672)
Traceback (most recent call last):
  File "/usr/lib/python3.9/urllib/request.py", line 1346, in do_open
    h.request(req.get_method(), req.selector, req.data, headers,
  File "/usr/lib/python3.9/http/client.py", line 1279, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib/python3.9/http/client.py", line 1325, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.9/http/client.py", line 1274, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.9/http/client.py", line 1034, in _send_output
    self.send(msg)
  File "/usr/lib/python3.9/http/client.py", line 974, in send
    self.connect()
  File "/usr/lib/python3.9/http/client.py", line 945, in connect
    self.sock = self._create_connection(
  File "/usr/lib/python3.9/socket.py", line 844, in create_connection
    raise err
  File "/usr/lib/python3.9/socket.py", line 832, in create_connection
    sock.connect(sa)
TimeoutError: [Errno 110] Connection timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/share/livecd-rootfs/magic-proxy", line 873, in __get_request
    with urllib.request.urlopen(
  File "/usr/lib/python3.9/urllib/request.py", line 214, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.9/urllib/request.py", line 517, in open
    response = self._open(req, data)
  File "/usr/lib/python3.9/urllib/request.py", line 534, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
  File "/usr/lib/python3.9/urllib/request.py", line 494, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.9/urllib/request.py", line 1375, in http_open
    return self.do_open(http.client.HTTPConnection, req)
  File "/usr/lib/python3.9/urllib/request.py", line 1349, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [Errno 110] Connection timed out>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.9/socketserver.py", line 683, in process_request_thread
    self.finish_request(request, client_address)
  File "/usr/lib/python3.9/socketserver.py", line 360, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "/usr/lib/python3.9/socketserver.py", line 747, in __init__
    self.handle()
  File "/usr/lib/python3.9/http/server.py", line 427, in handle
    self.handle_one_request()
  File "/usr/lib/python3.9/http/server.py", line 415, in handle_one_request
    method()
  File "/usr/share/livecd-rootfs/magic-proxy", line 787, in do_GET
    self.__get_request()
  File "/usr/share/livecd-rootfs/magic-proxy", line 887, in __get_request
    self.send_error(501, e.reason)
  File "/usr/lib/python3.9/http/server.py", line 473, in send_error
    'message': html.escape(message, quote=False),
  File "/usr/lib/python3.9/html/__init__.py", line 19, in escape
    s = s.replace("&", "&") # Must be done first!
AttributeError: 'TimeoutError' object has no attribute 'replace'

Because magic-proxy is proxying all traffic to port 80, it will
intercept these sort of messages. The TimeoutError ends up bubbling up
to the serversocket handling code, which will then cause a broken pipe
for all future communications, effectively breaking future apt calls
from within the build environment.

The concrete problem here is that inside magic-proxy, self.send_error is
called with the argument of e.reason, which can be a string (no error)
or an exception (in case of a nested exception). If the nested exception
is passed into self.send_error, html.replace will try to be called on
the exception, hence the above exception.

In addition to fixing this specific problem, magic-proxy's request
handling should catch all exceptions to not interfere with higher up
socketserver handling.

** Affects: livecd-rootfs (Ubuntu)
     Importance: Undecided
     Assignee: Thomas Bechtold (toabctl)
         Status: In Progress

-- 
magic-proxy does not gracefully handle error conditions
https://bugs.launchpad.net/bugs/1946520
You received this bug notification because you are a member of Ubuntu Sponsors Team, which is subscribed to the bug report.