[Bug 1915819] [NEW] 'NoneType' object has no attribute 'encode' in requestReceived() when multipart body doesn't include content-disposition

Launchpad Bug Tracker 1915819 at bugs.launchpad.net
Thu Feb 18 16:29:59 UTC 2021


You have been subscribed to a public bug by Ubuntu Foundations Team Bug Bot (crichton):

[impact]

python-twisted errors out with "'NoneType' object has no attribute
'encode' in requestReceived()" when it tries to parse a multipart mime
message and python3.7+ is used. This happens because before commit
cc3fa20 in cpython, cgi.parse_multipart ignored parts without a name
defined in "content-disposition" (or parts without headers for that
matter) but after 3.7+ the return of the function can now contain
NoneType keys, which fail to encode.

[scope]

Even though this bug affects all python3-twisted releases, I'll backport
the fix just to Focal, Groovy and Hirsute. Bionic and Xenial do not have
Python 3.7 as the default interpreter (required to trigger the issue),
and the delta in python3-twisted might be to big to backport as the
current packages do not contemplate _PY37PLUS at all.

Fixed upstream with commit 310496249, available since 21.2.0rc1

[test case]

1. Save the following code as webserver.py

from twisted.application.internet import TCPServer
from twisted.application.service import Application
from twisted.web.resource import Resource
from twisted.web.server import Site

class Foo(Resource):
    def render_POST(self, request):
        newdata = request.content.getvalue()
        print(newdata)
        return ''

root = Resource()
root.putChild("foo", Foo())
application = Application("cgi.parse_multipart test")
TCPServer(8080, Site(root)).setServiceParent(application)

2. Save the following code as client.py (python3-httplib2 is required)

#!/usr/bin/env python
import httplib2

def http_request(url, method, body=None, headers=None, insecure=False):
    """Issue an http request."""
    http = httplib2.Http(disable_ssl_certificate_validation=insecure)
    if isinstance(url, bytes):
        url = url.decode("ascii")
    return http.request(url, method, body=body, headers=headers)

url = "http://localhost:8080"
method = "POST"
headers = {'Content-Type': 'multipart/form-data; boundary="8825899812428059282"'}
emptyh = '--8825899812428059282\n\n--8825899812428059282--'

print("== BODY: " + emptyh + "\n")
response, content = http_request(url, method, emptyh, headers)

3. Run the server with "twistd3 -y webserver.py"
4. Run the client
5. twistd will fail to encode the key and will drop this traceback in the log file (twistd.log)

2021-02-16T13:41:39+0100 [_GenericHTTPChannelProtocol,7,127.0.0.1] Unhandled Error
        Traceback (most recent call last):
          File "/usr/lib/python3/dist-packages/twisted/python/log.py", line 103, in callWithLogger
            return callWithContext({"system": lp}, func, *args, **kw)
          File "/usr/lib/python3/dist-packages/twisted/python/log.py", line 86, in callWithContext
            return context.call({ILogContext: newCtx}, func, *args, **kw)
          File "/usr/lib/python3/dist-packages/twisted/python/context.py", line 122, in callWithContext
            return self.currentContext().callWithContext(ctx, func, *args, **kw)
          File "/usr/lib/python3/dist-packages/twisted/python/context.py", line 85, in callWithContext
            return func(*args,**kw)
        --- <exception caught here> ---
          File "/usr/lib/python3/dist-packages/twisted/internet/posixbase.py", line 614, in _doReadOrWrite
            why = selectable.doRead()
          File "/usr/lib/python3/dist-packages/twisted/internet/tcp.py", line 243, in doRead
            return self._dataReceived(data)
          File "/usr/lib/python3/dist-packages/twisted/internet/tcp.py", line 249, in _dataReceived
            rval = self.protocol.dataReceived(data)
          File "/usr/lib/python3/dist-packages/twisted/web/http.py", line 2952, in dataReceived
            return self._channel.dataReceived(data)
          File "/usr/lib/python3/dist-packages/twisted/web/http.py", line 2245, in dataReceived
            return basic.LineReceiver.dataReceived(self, data)
          File "/usr/lib/python3/dist-packages/twisted/protocols/basic.py", line 579, in dataReceived
            why = self.rawDataReceived(data)
          File "/usr/lib/python3/dist-packages/twisted/web/http.py", line 2252, in rawDataReceived
            self._transferDecoder.dataReceived(data)
          File "/usr/lib/python3/dist-packages/twisted/web/http.py", line 1699, in dataReceived
            finishCallback(data[contentLength:])
          File "/usr/lib/python3/dist-packages/twisted/web/http.py", line 2115, in _finishRequestBody
            self.allContentReceived()
          File "/usr/lib/python3/dist-packages/twisted/web/http.py", line 2224, in allContentReceived
            req.requestReceived(command, path, version)
          File "/usr/lib/python3/dist-packages/twisted/web/http.py", line 898, in requestReceived
            self.args.update({
          File "/usr/lib/python3/dist-packages/twisted/web/http.py", line 899, in <dictcomp>
            x.encode('iso-8859-1'): \
        builtins.AttributeError: 'NoneType' object has no attribute 'encode'

[regression potential]

This affects the returned dictionaries with non-str keys, which were
discarded in python3.6 or earlier before they reached twisted, so
patching this will make its behavior consistent.

** Affects: twisted (Ubuntu)
     Importance: Undecided
         Status: New

** Affects: twisted (Ubuntu Focal)
     Importance: Undecided
         Status: New

** Affects: twisted (Ubuntu Groovy)
     Importance: Undecided
         Status: New

** Affects: twisted (Ubuntu Hirsute)
     Importance: Undecided
         Status: New


** Tags: patch sts
-- 
'NoneType' object has no attribute 'encode' in requestReceived() when multipart body doesn't include content-disposition
https://bugs.launchpad.net/bugs/1915819
You received this bug notification because you are a member of Ubuntu Sponsors Team, which is subscribed to the bug report.



More information about the Ubuntu-sponsors mailing list