Negative caching

[Rashkae]

Hal Burgiss wrote:
> I continue to see the PAG line showing a lot of 'stalls'. This is on a 2 day
> or so uptime. The man page is not real clear on exactly what this should be
> telling me, ie is this "normal"? The system is OK right now, with occasional
> small jerks and lags and zags, but certainly usable.
> 
> PRC | sys  62m17s | user 133m42s | #proc    183 | #zombie    0 | #exit      ? |
> CPU | sys      3% | user      7% | irq       0% | idle     89% | wait      2% |
> CPL | avg1   0.94 | avg5    0.92 | avg15   0.88 | csw 81656138 | intr 19335e3 |
> MEM | tot  992.4M | free   18.0M | cache 334.4M | buff    1.3M | slab   43.9M |
> SWP | tot    2.9G | free    1.9G |              | vmcom   3.1G | vmlim   3.3G |
> PAG | scan 5279e3 | stall      0 |              | swin  169500 | swout 326425 | <<
> DSK |         sda | busy      2% | read  730720 | write 617333 | avio    2 ms |
> NET | transport   | tcpi  149337 | tcpo  144953 | udpi   22085 | udpo   29058 |
> NET | network     | ipi   180372 | ipo   181330 | ipfrw      0 | deliv 177403 |
> NET | eth0   ---- | pcki  148334 | pcko  138115 | si    4 Kbps | so    1 Kbps |
> NET | lo     ---- | pcki   44914 | pcko   44914 | si    1 Kbps | so    1 Kbps |
> 
> 

Hal, from what I've seen so far in your posts, I think you're
overthinking this.  Quite simply, the amount of apps you run that
require lots and lots of memory is simply *way* exceeding 1GB.. Your
earlier versions of Ubuntu worked, but that just tells me as you update
to newer software, including chrome, you just upped your memory requirement.

Take a look at 'cat /proc/meminfo' Look at the line Commited_AS:

That's how much memory your applications have requested be allocated.
It's not as useful a metric as you might want, because several
applications request allocation of more memory than they actually use,
but the more this number exceeds your physical memory, the greater
chance you'll start running into massive swap storms (swap storms are
what I call periods where you have to swap memory out to swap in order
to read swap file contents back into memory.  As you have observed, this
kills performance.  Given the description of the apps you like to have
to running simultaneously, I wager your allocated memory is over 2GB,
whereas your ram is only 1GB.

As for why you notice the problem getting your applications working
again in the morning, that's probably simply a matter of cron jobs that
themselves consume large memory, render your file cache entirely
useless, and why it takes so long for your applications to start working
again.  Try moving all the files from your /etc/cron.daily directory
somewhere else and see if your 'morning after' hangover is alleviated.

But the core issue, assuming I'm right about the Commited_as: value of
meminfo, is simply that you are requesting way more memory than your
system possess, and when background / cron daily processes put memory
pressure on your system, what little memory you have left gets used by
those instead of your apps, forcing everything to swap like crazy.  No
matter how much tuning you do, allocating more than twice your physical
ram is guaranteed to lead to grief.