[Bug 1198180] Re: possible leak in upstart 1.5
coding
1198180 at bugs.launchpad.net
Mon Aug 5 08:09:08 UTC 2013
I confirm this behaviour.
I use lxc for testing purposes. For each set of tests a few containers are created. Lifetime span is about a few seconds, and the tests are beeing run one after another continuously. After a few days init takes more than 2GB of memory. What's interesting - after stopping the tests two days ago, upstart is still creating/destroying non-existing virtual interfaces for the containers:
$ sudo lxc-list
RUNNING
FROZEN
STOPPED
$
$ sudo strace -ff -p 1 2>&1 | grep execve | grep if
[pid 7511] execve("/bin/sh", ["/bin/sh", "-e", "-c", "exec ifdown --allow auto $INTERF"..., "/bin/sh"], [/* 16 vars */] <unfinished ...>
[pid 7511] execve("/sbin/ifdown", ["ifdown", "--allow", "auto", "vetha9Ks2v"], [/* 18 vars */]) = 0
[pid 7513] execve("/bin/sh", ["/bin/sh", "-e", "-c", "exec ifdown --allow auto $INTERF"..., "/bin/sh"], [/* 16 vars */] <unfinished ...>
[pid 7513] execve("/sbin/ifdown", ["ifdown", "--allow", "auto", "vethDDYJtI"], [/* 18 vars */]) = 0
[pid 7514] execve("/bin/sh", ["/bin/sh", "-e", "-c", "exec ifdown --allow auto $INTERF"..., "/bin/sh"], [/* 16 vars */] <unfinished ...>
[pid 7514] execve("/sbin/ifdown", ["ifdown", "--allow", "auto", "vetheaOt3H"], [/* 18 vars */]) = 0
[pid 7523] execve("/sbin/ifup", ["ifup", "--allow", "auto", "vethcqVeRj"], [/* 17 vars */] <unfinished ...>
[pid 7522] execve("/sbin/ifup", ["ifup", "--allow", "auto", "veth0m5onm"], [/* 17 vars */]) = 0
[pid 7530] execve("/sbin/ifup", ["ifup", "--allow", "auto", "vethh6VSTB"], [/* 17 vars */] <unfinished ...>
[pid 7529] execve("/sbin/ifup", ["ifup", "--allow", "auto", "vethfgF5ty"], [/* 17 vars */] <unfinished ...>
[pid 7538] execve("/bin/sh", ["/bin/sh", "-e", "-c", "exec ifdown --allow auto $INTERF"..., "/bin/sh"], [/* 16 vars */] <unfinished ...>
[pid 7538] execve("/sbin/ifdown", ["ifdown", "--allow", "auto", "vethcqVeRj"], [/* 18 vars */]) = 0
[pid 7539] execve("/bin/sh", ["/bin/sh", "-e", "-c", "exec ifdown --allow auto $INTERF"..., "/bin/sh"], [/* 16 vars */] <unfinished ...>
[pid 7539] execve("/sbin/ifdown", ["ifdown", "--allow", "auto", "vethfgF5ty"], [/* 18 vars */] <unfinished ...>
[pid 7537] execve("/sbin/ifup", ["ifup", "--allow", "auto", "vethkx0wFy"], [/* 17 vars */] <unfinished ...>
[pid 7536] execve("/sbin/ifup", ["ifup", "--allow", "auto", "veth9z1M5B"], [/* 17 vars */]) = 0
[pid 7542] execve("/bin/sh", ["/bin/sh", "-e", "-c", "exec ifdown --allow auto $INTERF"..., "/bin/sh"], [/* 16 vars */] <unfinished ...>
[pid 7542] execve("/sbin/ifdown", ["ifdown", "--allow", "auto", "vethkx0wFy"], [/* 18 vars */]) = 0
[pid 7548] execve("/sbin/ifup", ["ifup", "--allow", "auto", "vethgClCAf"], [/* 17 vars */] <unfinished ...>
[pid 7547] execve("/sbin/ifup", ["ifup", "--allow", "auto", "veth8baiZ7"], [/* 17 vars */]) = 0
[pid 7560] execve("/bin/sh", ["/bin/sh", "-e", "-c", "exec ifdown --allow auto $INTERF"..., "/bin/sh"], [/* 16 vars */] <unfinished ...>
[pid 7560] execve("/sbin/ifdown", ["ifdown", "--allow", "auto", "veth8baiZ7"], [/* 18 vars */]) = 0
[pid 7558] execve("/sbin/ifup", ["ifup", "--allow", "auto", "vethTcUttE"], [/* 17 vars */] <unfinished ...>
$ sudo pmap -x 1
1: /sbin/init
Address Kbytes RSS Dirty Mode Mapping
00007fde5cf0c000 0 2508 2508 rw--- [ anon ]
00007fde5d30d000 0 1256 1256 rw--- [ anon ]
00007fde5d50e000 0 2508 2508 rw--- [ anon ]
00007fde5da93000 0 1256 1256 rw--- [ anon ]
00007fde5dd96000 0 0 0 r-x-- libnss_files-2.15.so
00007fde5dda2000 0 0 0 ----- libnss_files-2.15.so
00007fde5dfa1000 0 0 0 r---- libnss_files-2.15.so
00007fde5dfa2000 0 0 0 rw--- libnss_files-2.15.so
00007fde5dfa3000 0 0 0 r-x-- libnss_nis-2.15.so
00007fde5dfad000 0 0 0 ----- libnss_nis-2.15.so
00007fde5e1ad000 0 0 0 r---- libnss_nis-2.15.so
00007fde5e1ae000 0 0 0 rw--- libnss_nis-2.15.so
00007fde5e1af000 0 0 0 r-x-- libnsl-2.15.so
00007fde5e1c6000 0 0 0 ----- libnsl-2.15.so
00007fde5e3c5000 0 0 0 r---- libnsl-2.15.so
00007fde5e3c6000 0 0 0 rw--- libnsl-2.15.so
00007fde5e3c7000 0 0 0 rw--- [ anon ]
00007fde5e3c9000 0 0 0 r-x-- libnss_compat-2.15.so
00007fde5e3d1000 0 0 0 ----- libnss_compat-2.15.so
00007fde5e5d0000 0 0 0 r---- libnss_compat-2.15.so
00007fde5e5d1000 0 0 0 rw--- libnss_compat-2.15.so
00007fde5e5d2000 0 404 0 r-x-- libc-2.15.so
00007fde5e787000 0 0 0 ----- libc-2.15.so
00007fde5e986000 0 16 16 r---- libc-2.15.so
00007fde5e98a000 0 8 8 rw--- libc-2.15.so
00007fde5e98c000 0 16 16 rw--- [ anon ]
00007fde5e991000 0 4 0 r-x-- librt-2.15.so
00007fde5e998000 0 0 0 ----- librt-2.15.so
00007fde5eb97000 0 4 4 r---- librt-2.15.so
00007fde5eb98000 0 0 0 rw--- librt-2.15.so
00007fde5eb99000 0 24 0 r-x-- libpthread-2.15.so
00007fde5ebb1000 0 0 0 ----- libpthread-2.15.so
00007fde5edb0000 0 4 4 r---- libpthread-2.15.so
00007fde5edb1000 0 4 4 rw--- libpthread-2.15.so
00007fde5edb2000 0 4 4 rw--- [ anon ]
00007fde5edb6000 0 148 0 r-x-- libdbus-1.so.3.5.8
00007fde5edf8000 0 0 0 ----- libdbus-1.so.3.5.8
00007fde5eff8000 0 4 4 r---- libdbus-1.so.3.5.8
00007fde5eff9000 0 4 4 rw--- libdbus-1.so.3.5.8
00007fde5effa000 0 24 0 r-x-- libnih-dbus.so.1.0.0
00007fde5f002000 0 0 0 ----- libnih-dbus.so.1.0.0
00007fde5f202000 0 4 4 r---- libnih-dbus.so.1.0.0
00007fde5f203000 0 0 0 rw--- libnih-dbus.so.1.0.0
00007fde5f204000 0 44 0 r-x-- libnih.so.1.0.0
00007fde5f21a000 0 0 0 ----- libnih.so.1.0.0
00007fde5f41a000 0 4 4 r---- libnih.so.1.0.0
00007fde5f41b000 0 4 4 rw--- libnih.so.1.0.0
00007fde5f41c000 0 4 0 r-x-- ld-2.15.so
00007fde5f61a000 0 4 4 rw--- [ anon ]
00007fde5f63c000 0 0 0 rw--- [ anon ]
00007fde5f63e000 0 4 4 r---- ld-2.15.so
00007fde5f63f000 0 0 0 rw--- ld-2.15.so
00007fde5f641000 0 92 0 r-x-- init
00007fde5f867000 0 8 8 r---- init
00007fde5f869000 0 4 4 rw--- init
00007fde60da8000 0 2027292 2027292 rw--- [ anon ]
00007fff0162a000 0 24 24 rw--- [ stack ]
00007fff01664000 0 4 0 r-x-- [ anon ]
ffffffffff600000 0 0 0 r-x-- [ anon ]
System: Ubuntu 12.04.2 LTS
Kernel: Linux ABC 3.2.0-49-generic #75-Ubuntu SMP Tue Jun 18 17:39:32 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
Upstart version: 1.5-0ubuntu7.2
--
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to upstart in Ubuntu.
https://bugs.launchpad.net/bugs/1198180
Title:
possible leak in upstart 1.5
Status in “upstart” package in Ubuntu:
Confirmed
Bug description:
Description: Ubuntu 12.04.2 LTS
Release: 12.04
Package: 1.5-0ubuntu7.2
Kernel: Linux XXX 3.8.0-23-generic #34~precise1-Ubuntu SMP Wed May 29 21:12:31 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
We're starting lxc containers on demand, so there's lots and lots of
network interfaces going up and down. Udev emits signal, that goes to
upstart-udev-bridge, and next to the upstart itself. After about 40K
containers created/destroyed, upstart takes about 250MB or RAM, and it
grows with every event received/send. Not only RAM is beeing used, but
also init spends more time for every new event on the socket.
What happens:
init uses more RAM and more CPU during lifetime, when lots of network interfaces are beeing created/destroyed
What is expected:
init work should not change depending on number of events served
technical detail:
# pmap -x 1
00007f446f865000 0 237036 237036 rw--- [ anon ]
00007fffc694d000 0 24 24 rw--- [ stack ]
00007fffc69fe000 0 4 0 r-x-- [ anon ]
ffffffffff600000 0 0 0 r-x-- [ anon ]
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/upstart/+bug/1198180/+subscriptions
More information about the foundations-bugs
mailing list