C stack size

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

C stack size

Roberto Ierusalimschy
As already discussed, Lua 5.4 is using a "non stackless" implementation.
That means that it uses the C stack for recursion in Lua functions.
Because of that, the maximum depth of recursion is much lower in 5.4
than it was in 5.3. In Linux, that limit could be around 20000 calls,
which seems more than enough. On other systems, however (e.g., Windows),
that limit must be much lower.

I am attaching here a test file that should check whether the limit
is good enough for your machine. The output should be something like
this:

    $ lua cstack.lua
    testing C-stack overflow detection
    testing simple recursion:
    2164     final count: 2165
    testing stack overflow in message handling
    2413     final count: 2414
    testing recursion inside pattern matching
    testing stack-overflow in recursive 'gsub'
    181     final count: 181
    testing stack-overflow in recursive 'gsub' with metatables
    272     final count: 272
    OK

Or this if the limit is too high for the machine's stack:

    $ ulimit -S -s 400; lua cstack.lua
    testing C-stack overflow detection
    testing simple recursion:
    1758    Segmentation fault (core dumped)

(The '1758' is how many recursions it did before crashing.)

In our tests, the current limit of 2200 was about right for
Windows 7/VS 2010. For Linux/gcc, it works with a limit of 30000.

If you could run this test in your machine, that would be helpful.
It would be even more helpful if you could play with the
constant LUAI_MAXCSTACK and find its maximum value that the
test does not crash. (You can change LUAI_MAXCSTACK in luaconf.h
or define it with a compiler option.)

Many thanks,

-- Roberto

cstack.lua (1K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: C stack size

Andrew Gierth
>>>>> "Roberto" == Roberto Ierusalimschy <[hidden email]> writes:

 Roberto> As already discussed, Lua 5.4 is using a "non stackless"
 Roberto> implementation. [...]

 Roberto> If you could run this test in your machine, that would be
 Roberto> helpful. It would be even more helpful if you could play with
 Roberto> the constant LUAI_MAXCSTACK and find its maximum value that
 Roberto> the test does not crash. (You can change LUAI_MAXCSTACK in
 Roberto> luaconf.h or define it with a compiler option.)

I think the question "does it crash" is not the important one. The
question that someone needs to know when embedding the interpreter is
"what is the maximum stack space it can use without letting me check the
stack usage". Also, as an embeddable language, if there's going to be a
stack limit, it should be one which is _much smaller_ than the
platform's default or commonly-used limit.

Furthermore, "just compile it with a smaller limit" is NOT a good
response, since it blocks any attempt to have a system-wide shared
library used by many programs.

(My opinion, biased as it is, is that the default limit should be such
that the stack usage does not exceed 500kbytes. The current limit is
very close to that, except on clang on i386 where it's just over 2x too
large.)

What I've been testing with is actually instrumenting the stack depth
error to report the approximate bytes of stack used.

My major results:

1. Without compiler optimization the stack usage grows by a large
multiple (more than 10 times in some cases) presumably because of the
use of un-eliminated and un-reused stack-based temporaries in place of
register allocation.

2. Clang (8.0) uses somewhat more stack space than gcc does, on amd64.

3. Clang (8.0) uses significantly more stack space than gcc does on
i386.

4. i386 uses more stack space than amd64 on either compiler.

5. Disabling jumptables reduces stack usage a bit.

The following values are approximate bytes of stack usage at the point
of error, with the default limit of 2200, from executing the string
'local function f() f() end f()'.

with optimization:
#compiler,jumptables,  i386-O2   amd64-O2
gcc8    jumptables      523913    488985
gcc8    nojumptables    349593    349529
clang80 jumptables     1290705    523913
clang80 nojumptables   1238409    454185

without optimization:
#compiler,jumptables,  i386-O0   amd64-O0
gcc8    jumptables     4254497   6033001
gcc8    nojumptables   4149905   5893545
clang80 jumptables     5684277   6869813
clang80 nojumptables   5684277   6869813

--
Andrew.

Reply | Threaded
Open this post in threaded view
|

Re: C stack size

pocomane
In reply to this post by Roberto Ierusalimschy
On windows 10 and without any optimization I got a limit of 208 both with mingw and with visual studio 2012. Instead, enabling -02 and /02 I got:

msvs2012 limit: 2224 [1]
msvs2012 out:

testing C-stack overflow detection
testing simple recursion:
2188            final count: 2189
testing stack overflow in message handling
2440            final count: 2441
testing recursion inside pattern matching
testing stack-overflow in recursive 'gsub'
183             final count: 183
testing stack-overflow in recursive 'gsub' with metatables
275             final count: 275
OK

mingw limit: 7757
mingw output:

testing C-stack overflow detection
testing simple recursion:
7680            final count: 7722
testing stack overflow in message handling
5655    0
8408            final count: 8665
testing recursion inside pattern matching
testing stack-overflow in recursive 'gsub'
512             final count: 644
testing stack-overflow in recursive 'gsub' with metatables
954             final count: 966
OK
966

Both are 32 bit executables. The machine is an Intel i5 6200u, 8 GB ram.

pocomane

[1] With visual studio I got stocastic results with 2227 and 2228. From 2229 on, lua always crashes. I can not explain the random behaviour.
Reply | Threaded
Open this post in threaded view
|

Re: C stack size

pocomane
In reply to this post by Roberto Ierusalimschy


Mon 3 Jun 2019, 16:06 Roberto Ierusalimschy <[hidden email]>:
If you could run this test in your machine, that would be helpful.
It would be even more helpful if you could play with the
constant LUAI_MAXCSTACK and find its maximum value that the
test does not crash. (You can change LUAI_MAXCSTACK in luaconf.h
or define it with a compiler option.)

I tested on emscripten + webassembly + Firefox 64 bit windows, and I got 
a LUA_MAXCSTACK limit of 4375.

Reply | Threaded
Open this post in threaded view
|

Re: C stack size

Roberto Ierusalimschy
> > If you could run this test in your machine, that would be helpful.
> > It would be even more helpful if you could play with the
> > constant LUAI_MAXCSTACK and find its maximum value that the
> > test does not crash. (You can change LUAI_MAXCSTACK in luaconf.h
> > or define it with a compiler option.)
> >
>
> I tested on emscripten + webassembly + Firefox 64 bit windows, and I got
> a LUA_MAXCSTACK limit of 4375.

Many thanks for the feedback.

-- Roberto