[Bug 1655153] [NEW] long-running stunnel leaks memory

[Launchpad Bug Tracker]

You have been subscribed to a public bug by Scott Emmons (lscotte):

We are running a long-running stunnel4 daemon to proxy TLS connections
to another set of servers. After leaving it running for a few weeks, its
memory usage had grown to 1.5GB. Restarting it reduced its memory usage
to expected levels (VSZ and RSS) but while I've been watching it today
it has grown by more than 10MB.

The stunnel website indicates that there have been fixes relating to
memory leaks in versions 5.32 and 5.33, but Ubuntu LTS is still running
5.30.

ProblemType: Bug
DistroRelease: Ubuntu 16.04
Package: stunnel4 3:5.30-1
ProcVersionSignature: Ubuntu 4.4.0-45.66-generic 4.4.21
Uname: Linux 4.4.0-45-generic i686
ApportVersion: 2.20.1-0ubuntu2.4
Architecture: i386
Date: Mon Jan  9 16:03:37 2017
InstallationDate: Installed on 2015-10-31 (435 days ago)
InstallationMedia: Ubuntu-Server 15.10 "Wily Werewolf" - Release i386 (20151021)
ProcEnviron:
 TERM=xterm
 SHELL=/bin/bash
 PATH=(custom, no user)
 LANG=en_US.UTF-8
 XDG_RUNTIME_DIR=<set>
SourcePackage: stunnel4
UpgradeStatus: Upgraded to xenial on 2016-05-18 (236 days ago)
mtime.conffile..etc.default.stunnel4: 2016-10-26T22:22:28.166247

===

[Impact]

 * This bug results in a leak of TLS session objects in the stunnel4
server whenever a connection is closed. For a long running stunnel4
server, it can eventually consume all available memory.

 * This bug was introduced in stunnel 5.27, and subsequently fixed in
5.33. Ubuntu Xenial uses 5.30.

 * For Ubuntu, only Xenial is currently impacted by this bug, as
previous versions of Ubuntu use an older version of stunnel4 (prior to
5.27), and later versions of Ubuntu use a newer version of stunnel4 (at
least 5.33).

 * This patch backports a single specific fix to free TLS session
objects when a connection is closed, but contains no other changes from
newer stunnel4 versions.

[Test Case]

 * The bug and fix can be reproduced fairly easily by setting up an
stunnel4 server, then using openssl s_client to hammer against the
stunnel4 server. For example, with the server running on localhost port
443, proxying to a local Apache instance, and using a client
certificate:

=
#!/bin/bash
while true; do
  echo "" | openssl s_client -connect localhost:443 \
    -cert /etc/stunnel/client.pem
done
=

In another window, monitor RSS of the stunnel4 server process with
something like:

=
watch 'ps -p $(</var/run/stunnel4/pid) -o rss,comm'
=

 * The RSS of the stunnel4 process will continue to grow over time.

 * After installing the patched version via my PPA[1] and re-running the
test, the RSS of the stunnel4 process will grow for a few minutes and
then reach a steady state where it no longer continues to grow.

[1] https://launchpad.net/~lscotte/+archive/ubuntu/stunnel4

[Regression Potential]

 * None expected. This backports a fix in newer versions of upstream
stunnel4.

 * In my own environment, I've been running a production stunnel4 server
with my patch for over 85 days (zero restarts of the stunnel4 process).
With the current Xenial version I was unable to run for more than 1 day
without restarting stunnel4.

** Affects: stunnel4 (Ubuntu)
     Importance: Medium
         Status: Fix Released

** Affects: stunnel4 (Ubuntu Xenial)
     Importance: Medium
         Status: Confirmed

** Affects: stunnel4 (Debian)
     Importance: Unknown
         Status: Fix Released


** Tags: apport-bug i386 patch xenial
-- 
long-running stunnel leaks memory
https://bugs.launchpad.net/bugs/1655153
You received this bug notification because you are a member of Ubuntu Sponsors Team, which is subscribed to the bug report.