[9fans] Swap considered harmful (Sorry)

[Mike Schwartz]

On 8/11/06, Roderick <r.ford at cox.net> wrote:
>
> Wow! did you guys know that you could switch off the overcommitment of
> memory?  This posting was recently from the Plan9 o.s. mailing list.
> Thought you all might want this juicy tidbit of Linux hacking info.
> Rod
> --
> Roderick Ford, Software Engineer
> Open Design Alliance
> http://www.opendesign.com/
>
> ------- Forwarded message -------
> From: "Lluís Batlle i Rossell" <viriketo at gmail.com>
> To: "Fans of the OS Plan 9 from Bell Labs" <9fans at cse.psu.edu>
> Cc:
> Subject: Re: [9fans] Swap considered harmful (Sorry)
> Date: Mon, 07 Aug 2006 10:49:05 -0700
>
> Ronald G Minnich wrote:
> > Charles Forsyth wrote:
> >
> >>
> >> Linux apparently takes the Atlas approach and thrashes on demand.
> >>
> >
> > until it starts killing random processes. Usually the wrong one. But,
> > hey, heuristics, right?
>
> Maybe you already know, but by chance I got into the linux malloc(3)
> manpage, and I found its BUGS section:
>
>
> BUGS
> By default, Linux follows an  optimistic  memory  allocation strategy.
> This  means  that  when malloc() returns non-NULL there is no guarantee
> that the memory really is available. This is a really bad bug. In case it
> turns  out  that the system is out of memory, one or more processes will
> be killed by the infamous OOM killer.  In case Linux  is employed under
> circumstances  where it would be less desirable to suddenly lose some
> randomly picked processes, and moreover the kernel version is sufficiently
> recent, one can switch off this overcommitting behavior using a command
> like
>                # echo 2 > /proc/sys/vm/overcommit_memory
>
>
>
>
>
>
> ---------------------------------------------------
> PLUG-discuss mailing list - PLUG-discuss at lists.plug.phoenix.az.us
> To subscribe, unsubscribe, or to change  you mail settings:
> http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss
>

I was wondering, why [gnu/] Linux doesn't just use a
"safer" implementation by default.
I started doing some searches, using Google and yahoo,
and it turns out that there is way more to it, than what I
would have guessed.
Apparently there are some good reasons why, one
does not always necessarily prefer the last option
discussed here, in this excerpt from a discussion list
(( This quote is from the initial post of a big thread about
   "...strict VM opvercommit accounting...", see
http://marc2.theaimsgroup.com/?l=linux-kernel&m=113592878502058&w=2  )) :
<begin quote>
diff -ruN linux-2.4.33-pre1/Documentation/vm/overcommit-accounting \

linux-2.4.33-pre1-memA/Documentation/vm/overcommit-accounting
--- linux-2.4.33-pre1/Documentation/vm/overcommit-accounting	Wed Dec
31 16:00:00 1969
+++ linux-2.4.33-pre1-memA/Documentation/vm/overcommit-accounting	Thu
Dec 29 20:39:30 \
2005 @@ -0,0 +1,70 @@
+* This describes the overcommit management facility in the latest kernel
+  tree (FIXME: actually it also describes the stuff that isnt yet done)
+
+The Linux kernel supports four overcommit handling modes
+
+0	-	Heuristic overcommit handling. Obvious overcommits of
+		address space are refused. Used for a typical system. It
+		ensures a seriously wild allocation fails while allowing
+		overcommit to reduce swap usage
+
+1	-	No overcommit handling. Appropriate for some scientific
+		applications
+
+2	-	(NEW) strict overcommit. The total address space commit
+		for the system is not permitted to exceed swap + half ram.
+		In almost all situations this means a process will not be
+		killed while accessing pages but only by malloc failures
+		that are reported back by the kernel mmap/brk code.
+
+3	-	(NEW) paranoid overcommit The total address space commit
+		for the system is not permitted to exceed swap. The machine
+		will never kill a process accessing pages it has mapped
+		except due to a bug (ie report it!)
+
+Gotchas
+-------   [...snip...]

<end quote>

I would think that, as RAM gets cheaper, folks would want
to consider using something that is "safer", even though
it requires more RAM to get a certain amount (X) of work done.

However, during the searching on Google (and yahoo) that
I did while trying to barely scratch the surface, it appears that
there is a lot more to it, than I would ever have guessed...

I don't have time right now to pursue this any further just
out of curiosity.  If anyone has any comments, they would
be welcome...
(but I do not promise to read them all, 100%, if they are
 anywhere near the seeming avalanche of stuff that I
 started finding, when I started googling...)
-- 
Mike Schwartz
Glendale  AZ
schwartz at acm.org
Mike.L.Schwartz at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.plug.phoenix.az.us/pipermail/plug-discuss/attachments/20060811/de1beff0/attachment.htm