[ 5.4 work2 ] Stack overflow causes crash

classic Classic list List threaded Threaded
25 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Re: 答复: [ 5.4 work2 ] Stack overflow causes crash

Viacheslav Usov
On Sat, Dec 22, 2018 at 11:58 AM Egor Skriptunoff <[hidden email]> wrote:

> Does a question about VM crash make sense?

I cannot say without it being asked first.

> The stack size must be known at link time on Windows.

No. It is a default that the application can override at run time. https://docs.microsoft.com/en-us/windows/desktop/ProcThread/thread-stack-size

> Yes, of course, a host application developer should take into consideration both "how much stack Lua VM needs" and "how much stack the application itself needs"

Even that is impossible if the host provides functions using varying amounts of stack size callable from Lua, and the user is free to supply Lua code, even assuming that Lua can safely predict and control its own stack usage.

> I believed that Lua VM does guarantee to never crash the host application whatever Lua script is given.

Unfortunately that is not the case (as stated).

Cheers,
V.
Reply | Threaded
Open this post in threaded view
|

Re: 答复: [ 5.4 work2 ] Stack overflow causes crash

Egor Skriptunoff-2
On Sat, Dec 22, 2018 at 2:15 PM Viacheslav Usov wrote:
> The stack size must be known at link time on Windows.
No. It is a default that the application can override at run time.

It could be overridden only for new threads when creating them.
You can't increase stack size of already running Lua VM thread.

 
> Yes, of course, a host application developer should take into consideration both "how much stack Lua VM needs" and "how much stack the application itself needs"
Even that is impossible if the host provides functions using varying amounts of stack size callable from Lua, and the user is free to supply Lua code, even assuming that Lua can safely predict and control its own stack usage.

Nothing prevents you from checking stack size in your own application (and raise an error if the next operation might use more stack space than you have available).

 
> I believed that Lua VM does guarantee to never crash the host application whatever Lua script is given.
Unfortunately that is not the case (as stated).

I'd like to hear Lua authors' opinion on should we consider Lua safe or not.
Reply | Threaded
Open this post in threaded view
|

Re: 答复: [ 5.4 work2 ] Stack overflow causes crash

Viacheslav Usov
On Sat, Dec 22, 2018 at 1:41 PM Egor Skriptunoff <[hidden email]> wrote:

> It could be overridden only for new threads when creating them.

I'd rather say you can _only_ control the size of the main thread's stack at compile/link time. Any other thread can have a non-default size.

> You can't increase stack size of already running Lua VM thread.

There is no such thing as a "Lua VM thread". Any native thread (or fiber)  can be running in a given Lua VM at any given time, what thread it is, whether it is the same thread all the time, or even whether there are concurrent threads in the VM, is beyond the control of the Lua VM.

> Nothing prevents you from checking stack size in your own application (and raise an error if the next operation might use more stack space than you have available).

I am sure you can, in principle, come up with a set of constraints and measures aimed at ensuring safety for any given application.

The problem is the qualification "for any given application".

Another problem is "in principle", because it can be had or impossible in practice.

I am all for Lua's having a way to ensure true safety of at least its own stack use, but compile/link time constants aren't that way (in general).

Cheers,
V.
Reply | Threaded
Open this post in threaded view
|

Re: 答复: [ 5.4 work2 ] Stack overflow causes crash

Andrew Gierth
In reply to this post by Egor Skriptunoff-2
>>>>> "Egor" == Egor Skriptunoff <[hidden email]> writes:

 >> Even that is impossible if the host provides functions using varying
 >> amounts of stack size callable from Lua, and the user is free to
 >> supply Lua code, even assuming that Lua can safely predict and
 >> control its own stack usage.

 Egor> Nothing prevents you from checking stack size in your own
 Egor> application (and raise an error if the next operation might use
 Egor> more stack space than you have available).

Well, the problem (at least for me) is that I _can't_ check stack usage
within the 5.4-work VM, since it can go deeper than my available stack
space while never calling any non-Lua function. I would have to assume
that any call into Lua might take 7MB of stack (or 2MB if I can
guarantee that Lua was compiled with optimization, which I can't) -
which is a problem because I can't guarantee to have more than 2MB of
stack at all, and even by having the admin change configuration settings
I may not be able to get more than 3.5MB available on Windows without
forcing the user to compile their own binaries. (And that's WITHOUT
counting the stack usage of gsub/concat/format which adds additional
megabytes of usage; I could if necessary at least intercept all of those
functions and check stack depth on each call.)

Testing on a stack-instrumented 5.3.5 suggests that I can't get more
than ~520kB deep on the stack without using gsub/concat/format, and even
that much requires both pcall and -O0; without pcall I can't get more
than about 200kB deep. 200kB is well within my allowed 512kB safety
margin, so if I were to intercept string.gsub, string.format and
table.concat (in addition to pcall which I already do) and add stack
depth checks to those, I wouldn't need any other checks.

--
Andrew.

Reply | Threaded
Open this post in threaded view
|

Re: 答复: [ 5.4 work2 ] Stack overflow causes crash

Roberto Ierusalimschy
In reply to this post by Egor Skriptunoff-2
> Roberto, what stack size does Lua 5.3+ VM expect?
> This information is essential for correct building Lua on Windows.

This is highly experimental, but we usually think the other way around:
what is the maximum value for LUAI_MAXCCALLS in a particular environment.
The only way to really know is by testing. Any porting of Lua to a new
environment can define this constant to an appropriate value that avoids
stack overflows.

I may be wrong, but I think none of those experiments reported here
with an official release of Lua created a C stack overflow. Anyway,
we probably could write a small test script that explores different
kinds of C stack overflows (coroutine recursion, gsub recursion, etc.)
as a way to validate LUAI_MAXCCALLS for a particular environment.

Of course, we can crash almost any C program given a sufficiently small
stack...

-- Roberto

12