|Jeroen Demeyer on Mon, 03 Oct 2016 17:05:00 +0200|
[Date Prev] [Date Next] [Thread Prev] [Thread Next] [Date Index] [Thread Index]
|Re: Use PROT_NONE for unused virtual stack memory|
On 2016-09-29 17:46, Bill Allombert wrote:
Do you have some framework to test this ?
Not really because it depends a lot on the OS. In order to properly test this, you need a Linux system with vm.overcommit = 2. So you need
# echo 2 >/proc/sys/vm/overcommit_memory
The idea is to use PROT_NONE for the unused part of the PARI stack (between vbot and bot). Memory mmap()ed with PROT_NONE is not committed: it does not count against the available physical/swap memory available for the system.Do you have some relevant documentation for this behaviour ?
No. As far as I know, this is undocumented. But the trick is known and used in the wild, see for example the discussion
https://github.com/jemalloc/jemalloc/issues/255 and code https://github.com/jemalloc/jemalloc/blob/3c8c3e9e9b59b6e34a222816a05f0a01a68919b3/src/pages.c#L134
One consequence of this patch is that increasing the actual size (during some calculation which needs more stack space) can fail. This is not a problem since we can handle this failure the same way that we handle a usual PARI stack overflow.It is unclear from the documentation of mprotect that it can actually fail for that reason. Can you make it fail and what is the error code ?
Yes, it can fail with error code ENOMEM.
It remains the last point which is: is it the behaviour users of vm.overcommit = 2 and parisizemax want, or would they prefer the stack to be fully available up to parisizemax.
Well, the stack is always fully available up to "parisize". If you really want the stack to be available (regardless of overcommit options), you should use "parisize" and not "parisizemax".
The whole point of vm.overcommit = 2 is that the OOM killer should never kick in. Instead, memory allocations should fail. The increasing of the PARI stack above "parisize" is exactly such an allocation which might fail. I think it fits well within this philosophy.