Negative caching

Hal Burgiss hal at burgiss.net
Fri Jun 19 17:57:58 UTC 2009


On Fri, Jun 19, 2009 at 01:12:57PM -0400, Rashkae wrote:
> Hal, from what I've seen so far in your posts, I think you're
> overthinking this.  Quite simply, the amount of apps you run that
> require lots and lots of memory is simply *way* exceeding 1GB.. Your
> earlier versions of Ubuntu worked, but that just tells me as you update
> to newer software, including chrome, you just upped your memory requirement.

Chrome is a very recent addition due to performance related issues with
Firefox. I have to say I like it. If it had the addons FF did, I'd switch
all the way.
 
> Take a look at 'cat /proc/meminfo' Look at the line Commited_AS:

CommitLimit:     3512236 kB
Committed_AS:    3340940 kB
VmallocTotal:     122880 kB
VmallocUsed:        6392 kB
VmallocChunk:      91200 kB


> That's how much memory your applications have requested be allocated.
> It's not as useful a metric as you might want, because several
> applications request allocation of more memory than they actually use,
> but the more this number exceeds your physical memory, the greater
> chance you'll start running into massive swap storms (swap storms are
> what I call periods where you have to swap memory out to swap in order
> to read swap file contents back into memory.  As you have observed, this
> kills performance.  Given the description of the apps you like to have
> to running simultaneously, I wager your allocated memory is over 2GB,
> whereas your ram is only 1GB.

You won the wager!
 
> As for why you notice the problem getting your applications working
> again in the morning, that's probably simply a matter of cron jobs that
> themselves consume large memory, render your file cache entirely
> useless, and why it takes so long for your applications to start working
> again.  Try moving all the files from your /etc/cron.daily directory
> somewhere else and see if your 'morning after' hangover is alleviated.

Makes sense.

> But the core issue, assuming I'm right about the Commited_as: value of
> meminfo, is simply that you are requesting way more memory than your
> system possess, and when background / cron daily processes put memory
> pressure on your system, what little memory you have left gets used by
> those instead of your apps, forcing everything to swap like crazy.  No
> matter how much tuning you do, allocating more than twice your physical
> ram is guaranteed to lead to grief.

But here is what I don't understand, and why I think there is a fundamental
problem with the way swap is managed with this particular kernel: this issue
gets progressively worse over time. It will probably take at least a week,
maybe two, before I starting thinking evil thoughts of restarting X or the
system. Its like the physical memory at some point is just not utilized well.
The stuff I use frequently and interact with, winds up on disk, and "something
else" has physical memory bottlenecked. 

-- 
Hal




More information about the ubuntu-users mailing list