Is this possible?

[Bob]

** Reply to message from Peter Silva <peter at bsqt.homeip.net> on Wed, 5 Oct 2016
07:46:09 -0400

> "swap is maxed out"
> 
> uh... if that's true, it doesn't matter how much or what kind of cpu
> you have, it will crawl and die from time to time.   your machine is
> sitting in wait i/o.

true


>   When "swap is maxed out" the kernel will kill
> processes randomly, (OOM Killer) you cannot expect a PC with it's
> memory (including swap space) entirely full to run correctly.

If this is what Linux does that is a very bad design.  I would never have
thought the system would do that.


>   One
> would routinely expect crashes as the processes that get killed might
> be important.

No, one, would not expect crashes.  There is no reason for a crash even if the
system is thrashing because of excessive paging.  Slowing down to a crawl is
what I expect the system to do.


> Here is a healthy amount of swap use:
> 
> KiB Mem : 11733384 total,  2327220 free,  3294000 used,  6112164 buff/cache
> KiB Swap: 24761340 total, 24761340 free,        0 used.  7137152 avail Mem
> 
> You should be aiming for all your work to fit in memory. Swap is only
> efficient when it is used to swap out bits of memory that are not
> being used by running processes (say initialization code, or code that
> isn't used often.)

This is why paging works and what it was designed to do.  Of course over
committing memory heavily is not a good idea.


> If your machine is going to swap often, well the
> swapping in from disk is 100x to 1000x slower than memory, so it's
> going to be slow.
> 
> You either need more memory, or you need to run less stuff at once.
> There are ways of doing that in an organized way (batch queueing
> systems), but it might be more straight forward just to put some
> sequencing in your work.  That is you don't just fire up many
> background tasks at once, but rather a few at a time, planning them
> out so that all running tasks always fit in memory.

Good advice.


> use top to look at your memory usage and keep swap down...  It doesn't
> have to be zero, but it will never work if it is 'maxed out'.

Very true.


> On Wed, Oct 5, 2016 at 4:52 AM, Liam Proven <lproven at gmail.com> wrote:
> > On 3 October 2016 at 16:35, rikona <rikona at sonic.net> wrote:
> >>
> >> My hope is to do things in parallel. I work with a fair amount of
> >> data, and a large data run may take 12 hours. Sometimes I can split
> >> that and run it as multiple processes. While that is running, I may
> >> have multiple browsers, each with perhaps 50 or more tabs open. This
> >> load makes my current box unusably slow, swap is maxed out, and
> >> something very often crashes - I may lose several days of work. And
> >> there's email, editing of docs, making diagrams, etc, etc. Perhaps
> >> Intel can do one 12 hour job in 4 hours, but I still have lots going
> >> on during that 4 hours.
> >
> >
> > I'm ignoring all the pointless advocacy here.
> >
> > If you have stability issues, you need to troubleshoot them properly.
> >
> > You need to profile your workloads and find the bottlenecks.
> >
> > And if it's background stuff and concerns with OSes struggling to
> > balance conflicting workloads then you should probably be looking at
> > VM solutions, and partitioning off the background  number-crunching
> > tasks.
> >
> > Throwing CPU cores at the problem is inane and a pointless waste of
> > cash. Throwing slower cores is burning banknotes. And throwing slower
> > cores *when you're not even sure it's CPU-bound* is just stupid.
> >
> > Sorry, but it is.
> > --
> > Liam Proven • Profile: http://lproven.livejournal.com/profile
> > Email: lproven at cix.co.uk • GMail/Twitter/Facebook/Flickr: lproven
> > Skype/MSN: lproven at hotmail.com • LinkedIn/AIM/Yahoo: liamproven
> > Cell/Mobiles: +44 7939-087884 (UK) • +420 702 829 053 (ČR)

-- 
Robert Blair


A government big enough to give you everything you want, is strong enough to take everything you have.  -- Thomas Jefferson