fine-grained lua_pcall() error processing

classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|

fine-grained lua_pcall() error processing

petah
I need to add some "intelligence" to a Lua error handler (written in C++), f.ex. taking specific action on a missing require() (which throws LUA_ERRRUN, not LUA_ERRFILE) when remote-debugging from a client with a case-insensitive filesystem to a daemon that's not. Or suggesting fixes on a malformed regex, etc. There are only a handful of LUA_ERR consts but dozens of different (yet finite) luaL_error() messages.

Is it possible to get more structured information from a Lua error callback than a "mangled" string generated by luaL_error()/lua_error()?

If not, is there a less dicey way to reduce an error string to a discrete enum than cascading if/else regexes?

If not, is there code out there (whatever the language) to extract a robust regex mapping table from the Lua VM source code?

My debugger is C-only (no extraneous Lua) and I'd really like to avoid patching the Lua VM.

thx,

-- p

Reply | Threaded
Open this post in threaded view
|

Re: fine-grained lua_pcall() error processing

Sven Olsen

Is it possible to get more structured information from a Lua error callback than a "mangled" string generated by luaL_error()/lua_error()?

As far as I know, it's not.
 
If not, is there a less dicey way to reduce an error string to a discrete enum than cascading if/else regexes?

Without patching ldo.c, again, not that I know of.
 
If not, is there code out there (whatever the language) to extract a robust regex mapping table from the Lua VM source code?

That's a fascinating theoretical question.  I suspect the answer is "yes, but it's not worth the trouble."
 
My debugger is C-only (no extraneous Lua) and I'd really like to avoid patching the Lua VM.

The Lua source is designed to be easy to patch -- particularly where error handling is concerned.  (Grep for the macros LUAI_TRY and LUAI_THROW.)  In my opinion, when it's being embedded in a C++ engine, one of the most useful source changes one can make is to expose Lua's exception types to C++, and vice versa.  Good error handling is a wonderful thing; but of course, it's hard to do without deviating from "clean C".  But if you're willing to compile the sources as C++, the hooks are easy to write.

-Sven
Reply | Threaded
Open this post in threaded view
|

Re: fine-grained lua_pcall() error processing

petah
>> Is it possible to get more structured information from a Lua error
>> callback than a "mangled" string generated by luaL_error()/lua_error()?
>>
>
>As far as I know, it's not.

Would there be some lua_State trickery to uniquely identify where luaL_error() was called from? Maybe some twisted native (machine language) stack walk? The daemon currently uses wxWidgets (base) so can use x-platform functions.

>> If not, is there a less dicey way to reduce an error string to a discrete
>> enum than cascading if/else regexes?
>
>Without patching ldo.c, again, not that I know of.

Ok. Regexing error strings seems pretty dangerous anyway; no unit test could ensure its safety and it's clearly an "anti-pattern".

>> If not, is there code out there (whatever the language) to extract a
>> robust regex mapping table from the Lua VM source code?
>
>That's a fascinating theoretical question.  I suspect the answer is "yes,
>but it's not worth the trouble."

I guess "worth" depends on how costly bugs can be :)

Maybe in a new Lua version the different lua_error() invocations could be centralized in a separate #include with one #define per error type, which wouldn't affect functionality.

Too bad Lua doesn't use its own error() model to throw an error object/table before flattening it to a string. Error handling isn't performance-critical so making some lower-level functions client-overridable shouldn't have downsides.

>> My debugger is C-only (no extraneous Lua) and I'd really like to avoid
>> patching the Lua VM.
>
>The Lua source is designed to be easy to patch -- particularly where error
>handling is concerned.  (Grep for the macros LUAI_TRY and LUAI_THROW.)

It seems luaD_throw() is called after the error string is constructed by a bunch of upstream sprintf() macros, so C++ exceptions wouldn't provide more information than longjmp()s. The Lua VM doesn't have code to build an error struct/object; it generates an error string directly from the error location. Adding an enum to the string would make a huge difference.

>Good error handling is a wonderful thing; but of
>course, it's hard to do without deviating from "clean C".  But if you're
>willing to compile the sources as C++, the hooks are easy to write.

I understand but my employer agreed to open-source the debugger with the promise it'd give them visibility and help recruiting Lua scripters. Patching the VM would make the debugger useless to anyone using pre-built Lua libs.

I may have to patch it in the end anyway but it seems a shame given how much built-in debugging functionality Lua already has.

thx,

-- p



Reply | Threaded
Open this post in threaded view
|

Re: fine-grained lua_pcall() error processing

Luiz Henrique de Figueiredo
> Too bad Lua doesn't use its own error() model to throw an error object/table before flattening it to a string.

You can throw any Lua value with lua_error() or error(), but I guess
you mean that internally the Lua core could thrown something else.

Reply | Threaded
Open this post in threaded view
|

Re: fine-grained lua_pcall() error processing

petah
>> Too bad Lua doesn't use its own error() model to throw an error object/table before flattening it to a string.
>
>You can throw any Lua value with lua_error() or error(), but I guess
>you mean that internally the Lua core could thrown something else.

Exactly.

Lua core errors are potentially most lethal and though recovering from them is much more difficult, deterministic error handling would be a huge benefit.

-- p

Reply | Threaded
Open this post in threaded view
|

Re: fine-grained lua_pcall() error processing

Sven Olsen
In reply to this post by petah

It seems luaD_throw() is called after the error string is constructed by a bunch of upstream sprintf() macros, so C++ exceptions wouldn't provide more information than longjmp()s.  The Lua VM doesn't have code to build an error struct/object; it generates an error string directly from the error location. Adding an enum to the string would make a huge difference.

Yes, if you want a richer enum, you'd need to cut into the error generation pipeline earlier.  I think the two functions to consider patching are luaG_runerror and luaL_error -- making versions of both that accept an optional enum argument would seem a fairly straightforward way of getting the behavior you're asking for.
 
I understand but my employer agreed to open-source the debugger with the promise it'd give them visibility and help recruiting Lua scripters. Patching the VM would make the debugger useless to anyone using pre-built Lua libs.

Reading between the lines, it sounds like this is a custom debugger commissioned to work with a particular C++ project, but, it might also be distributed as a standalone open source app?

In that case, you could consider modding the VM, and posting the diff on the power patches page.  That way other projects that embed different versions of Lua could potentially patch in the hooks needed by your debugger.  And if the debugger is awesome enough, there's an outside chance that the Lua author's might take notice, and roll the diffs into 5.3 :)

But if you're serious about supporting a wide range of Lua VM's, pattern matching is probably you're only option.  If you go this route, I'd also grab a copy of the LuaJIT sources, as that's the other version of the VM you're likely to encounter in the wild.

-Sven
Reply | Threaded
Open this post in threaded view
|

Re: fine-grained lua_pcall() error processing

Sven Olsen
In reply to this post by petah

Would there be some lua_State trickery to uniquely identify where luaL_error() was called from? Maybe some twisted native (machine language) stack walk? The daemon currently uses wxWidgets (base) so can use x-platform functions.

Well, again, if the goal is to support a wide range of different Lua VM's, I think you'd need several different versions of this black magic; at least one for each major release of Lua and LuaJIT.  Pattern matching is probably both easier and more robust that trying something like this.

If you don't care so much about broad VM support, then I think you're back to having an extended error info patch be your best option.

-Sven
Reply | Threaded
Open this post in threaded view
|

Re: fine-grained lua_pcall() error processing

petah
In reply to this post by Sven Olsen
>Reading between the lines, it sounds like this is a custom debugger
>commissioned to work with a particular C++ project, but, it might also be
>distributed as a standalone open source app?

Actually the Lua debugger is a part of a large, internal, realtime 3D engine that's been powering "high-profile" corporate projects for years (www.inhance.com). Since my company's competitors are miles away technologically and we have trouble hiring qualified Lua coders I convinced them to release the debugger itself, under the same MIT license as Lua, for ethical reasons.

I'm one of the PITAs advocating splitting our platform into separate components; a reusable debugger is just one example.

>In that case, you could consider modding the VM, and posting the diff on
>the power patches page.  That way other projects that embed different
>versions of Lua could potentially patch in the hooks needed by your
>debugger.  And if the debugger is awesome enough, there's an outside chance
>that the Lua author's might take notice, and roll the diffs into 5.3 :)

Nah. The #1 thing that sets Lua apart is Teflon resistance to feature creep; I like it that way.

>But if you're serious about supporting a wide range of Lua VM's, pattern
>matching is probably you're only option.

I think it's too dangerous since the error string is unknown. Last thing you want is your debugger daemon crashing on an iOS app.

>If you go this route, I'd also
>grab a copy of the LuaJIT sources, as that's the other version of the VM
>you're likely to encounter in the wild.

LuaJIT isn't on our radar; over ~5 years we've had only one interpreted Lua bottleneck (gigantic memory requirements) which was solved in a few minutes by a custom type implemented in C++.

I don't know what LuaJIT-compatibility would imply, whoever wants it will be free to do so :)

-- p



Reply | Threaded
Open this post in threaded view
|

Re: fine-grained lua_pcall() error processing

petah
In reply to this post by Luiz Henrique de Figueiredo
>> Too bad Lua doesn't use its own error() model to throw an error object/table before flattening it to a string.
>
>You can throw any Lua value with lua_error() or error(), but I guess
>you mean that internally the Lua core could thrown something else.

Btw I'm not advocating an overhaul of Lua's error handling -- I cherish Lua's consistency/compatibility like any long-time user -- and understand a declarative struct/table covering all possible lua core errors may be "uma bagunça".

The current challenge, as I see it, is that LUA_ERRSYNTAX/LUA_ERRRUN cover too many different error types. If Lua could set a unique error enum from the error-triggering function (retrievable from the registry, a new lua_substatus() function or whatever), extracting info from a flattened error string would be reasonably safe.

thx,

-- p

Reply | Threaded
Open this post in threaded view
|

Re: fine-grained lua_pcall() error processing

petah
In reply to this post by Sven Olsen
>Well, again, if the goal is to support a wide range of different Lua VM's,

My company doesn't care about "ruling the world" (they repeatedly denied my funding requests for a Stromberg-style underwater lair); I'm just trying to align their interests with the Lua community.

Convincing management they have a debt to open source projects is a bitch, so it's a careful dance... which may just end with me falling on my face :)

-- p

Reply | Threaded
Open this post in threaded view
|

Re: fine-grained lua_pcall() error processing

Miles Bader-2
In reply to this post by petah
petah <[hidden email]> writes:
> LuaJIT isn't on our radar; over ~5 years we've had only one
> interpreted Lua bottleneck (gigantic memory requirements) which was
> solved in a few minutes by a custom type implemented in C++.

... and note that LuaJIT actually reduces the amount of memory you can
use (compared to the standard interpreter)!

-miles

--
永日の 澄んだ紺から 永遠へ

Reply | Threaded
Open this post in threaded view
|

Re: fine-grained lua_pcall() error processing

Joshua Jensen-2
In reply to this post by petah
----- Original Message -----
From: petah
Date: 3/12/2013 8:43 PM
>>> Is it possible to get more structured information from a Lua error
>>> callback than a "mangled" string generated by luaL_error()/lua_error()?
>>>
>> As far as I know, it's not.
> Would there be some lua_State trickery to uniquely identify where luaL_error() was called from? Maybe some twisted native (machine language) stack walk? The daemon currently uses wxWidgets (base) so can use x-platform functions.
>
The Tilde debugger made by Tantalus adds an error hook to catch the
location where error() was called from.  It works pretty well:

https://github.com/jjensen/lua-tilde/blob/master/lib/lua/lua_5.1.3.patch

-Josh

Reply | Threaded
Open this post in threaded view
|

Re: fine-grained lua_pcall() error processing

petah
>----- Original Message -----
>From: petah
>Date: 3/12/2013 8:43 PM
>>>> Is it possible to get more structured information from a Lua error
>>>> callback than a "mangled" string generated by luaL_error()/lua_error()?
>>>>
>>> As far as I know, it's not.
>> Would there be some lua_State trickery to uniquely identify where luaL_error() was called from? Maybe some twisted native (machine language) stack walk? The daemon currently uses wxWidgets (base) so can use x-platform functions.
>>
>The Tilde debugger made by Tantalus adds an error hook to catch the
>location where error() was called from.  It works pretty well:
>
>https://github.com/jjensen/lua-tilde/blob/master/lib/lua/lua_5.1.3.patch

Thx.

I develop mainly on Linux/CodeLite and only fire up Windows to check VC++ compatibility before commit (kicking and screaming:) so I just browsed the code / didn't run it / may be totally off:

I want to uniquely identify the different luaG_runerror() and luaL_error() invocations (~22 and 54, respectively) before they're flattened in a generic error string. From what I can tell Tilde doesn't accomplish that - am I correct?

I think the approx 76 fine-grained errors are differentiated by their 'fmt' strings where those two functions call va_start(argp, fmt).

cheers,

-- p

Reply | Threaded
Open this post in threaded view
|

Re: fine-grained lua_pcall() error processing

Francisco
In reply to this post by petah
every time a see a Lua big and apparently good Windows only project i say "whatthahell", and then a tear drops :(

--
xico
web developer at Jurema and Simbiose
55 11 97531.7438
about.leite.us