Unable to start new processes

[James Hogarth]

On 24 August 2010 07:31, Chris MacDonald <chris at fourthandvine.com> wrote:
> On Mon, Aug 23, 2010 at 10:13 PM, Cameron Hutchison <lists at xdna.net> wrote:
>> If some process does have a memory leak, you may be able to track it by
>> regularly getting a full process listing (say twice a day) and watching
>> the VSS and RSS of your processes. Also run free(1) periodically and see
>> if available memory is falling, and if the problem starts occur shortly
>> after low memory.
>
> I've added a peek at 'ps auxc' for apps with non-zero memory usage and
> 'free -m' to my watchdog script that loops though each remote host at
> an interval of every two minutes or so, we'll see if anything
> materializes. Thanks for the suggestion, I'll post back with results
> shortly.
>
> Chris
>
> --
> ubuntu-users mailing list
> ubuntu-users at lists.ubuntu.com
> Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users
>

Another interesting point you mention is the directed nature of the
connections having problems (remote -> you okay but not the other way
around). Although ICMP echo requests are returning okay I'm wondering
if there is something messing around with TCP activity.

As an example I semi-recently had a box dying on a process with two
many files open. The network weenies had broken the ACLs in such a way
that any TCP packets with the FIN flag set were not coming back
properly and all network processes to the remote machine got stuck in
an FIN_WAIT state and never closed the connection until the denial of
service eventually occurred.

It would be worth saving an exhaustive tcpdump on both ends of the
connection (just dump to file with no filtering) and examining it in
wireshark to check if the packets are being generated and reaching the
two sides etc.

James