Lua error handling and C++

classic Classic list List threaded Threaded
23 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Lua error handling and C++

Glenn Maynard
How are people dealing with error handling in C++?  setjmp doesn't
get along with C++ destructors.  luaconf.h has an alternate to use
exceptions, but enabling exceptions bloats binaries massively (in
VC, and even more in g++).  Anyone found a good approach?

In principle, I could move my Lua code into separate source files,
enable exceptions for just those files, and make sure to catch all
exceptions there.  That would be a major hassle--dozens of bound
classes, with Lua binding code at the end of each source file being
bound, several build systems, and also a fair bit of inline Lua
interfacing.  (I also know no way to convince automake to send
different compiler flags to different source files, short of building
a library.)  I'd sooner live with a memory leak on error than deal
with that.

-- 
Glenn Maynard

Reply | Threaded
Open this post in threaded view
|

Re: Lua error handling and C++

Javier Guerra Giraldez
On Wednesday 11 October 2006 6:57 pm, Glenn Maynard wrote:
> How are people dealing with error handling in C++?  setjmp doesn't
> get along with C++ destructors.  luaconf.h has an alternate to use

any C++ -aware longjmp should call destructors; or am i too optimistic?  at 
the very least, i know the old 'almost C++' Metrowerks C compiler did that 
many many years ago


-- 
Javier

Attachment: pgpb0W79GYGjX.pgp
Description: PGP signature

Reply | Threaded
Open this post in threaded view
|

Re: Lua error handling and C++

Glenn Maynard
On Wed, Oct 11, 2006 at 09:24:53PM -0500, Javier Guerra wrote:
> On Wednesday 11 October 2006 6:57 pm, Glenn Maynard wrote:
> > How are people dealing with error handling in C++?  setjmp doesn't
> > get along with C++ destructors.  luaconf.h has an alternate to use
> 
> any C++ -aware longjmp should call destructors; or am i too optimistic?  at 
> the very least, i know the old 'almost C++' Metrowerks C compiler did that 
> many many years ago

I'm fairly sure that neither VC nor g++ (the two compilers I deal with
regularly) do not, and VC has a warning for longjmp nonportability in
C++.

-- 
Glenn Maynard

Reply | Threaded
Open this post in threaded view
|

Re: Lua error handling and C++

Nick Gammon
In reply to this post by Glenn Maynard

On 12/10/2006, at 9:57 AM, Glenn Maynard wrote:

How are people dealing with error handling in C++?  setjmp doesn't
get along with C++ destructors.

Just to help me understand the question, do you mean:

a. A Lua script calls some C++ code which might throw an exception?

or

b. Some C++ code calls a Lua script that might raise an error?

- Nick

Reply | Threaded
Open this post in threaded view
|

Re: Lua error handling and C++

Glenn Maynard
On Thu, Oct 12, 2006 at 02:59:34PM +1000, Nick Gammon wrote:
> >How are people dealing with error handling in C++?  setjmp doesn't
> >get along with C++ destructors.
> 
> Just to help me understand the question, do you mean:
> 
> a. A Lua script calls some C++ code which might throw an exception?
> 
> or
> 
> b. Some C++ code calls a Lua script that might raise an error?

A, and A': C++ calls the Lua C api, which might also throw an error.

-- 
Glenn Maynard

Reply | Threaded
Open this post in threaded view
|

Re: Lua error handling and C++

David Manura-2
In reply to this post by Glenn Maynard
Glenn,

Glenn Maynard <glenn <at> zewt.org> writes:
> How are people dealing with error handling in C++? 

There is some discussion on this topic here:

  http://lua-users.org/wiki/ErrorHandlingBetweenLuaAndCplusplus

Improvements are welcome.

--davidm


Reply | Threaded
Open this post in threaded view
|

Re: Lua error handling and C++

Glenn Maynard
On Sat, Oct 14, 2006 at 02:00:16AM +0000, David Manura wrote:
> Glenn Maynard <glenn <at> zewt.org> writes:
> > How are people dealing with error handling in C++? 
> 
> There is some discussion on this topic here:
> 
>   http://lua-users.org/wiki/ErrorHandlingBetweenLuaAndCplusplus
> 
> Improvements are welcome.

I don't have any ideas.  Unfortunately, the whole thing assumes that
exceptions are always OK in C++.  In my experience, they usually aren't.
I've seen binaries shrink by over a meg by disabling exceptions, which
I think is just too much waste, even when targetting modern systems.

-- 
Glenn Maynard

Reply | Threaded
Open this post in threaded view
|

Re: Lua error handling and C++

Joe Smith
In reply to this post by Glenn Maynard

"Glenn Maynard" <[hidden email]> wrote in message [hidden email]">news:20061011235743.GW18958@......
How are people dealing with error handling in C++?  setjmp doesn't
get along with C++ destructors.  luaconf.h has an alternate to use
exceptions, but enabling exceptions bloats binaries massively (in
VC, and even more in g++).  Anyone found a good approach?

In principle, I could move my Lua code into separate source files,
enable exceptions for just those files, and make sure to catch all
exceptions there.  That would be a major hassle--dozens of bound
classes, with Lua binding code at the end of each source file being
bound, several build systems, and also a fair bit of inline Lua
interfacing.  (I also know no way to convince automake to send
different compiler flags to different source files, short of building
a library.)  I'd sooner live with a memory leak on error than deal
with that.

I'm not really aware of any good solution.
A C++ project I worked on that used C++ exceptions
found some very nasty errors (segfaults deep inside
the bowels of g++). The bugs were hard to debug,
seemingly non-deterministic, which was a mess.

Apparently longjmp does not mess well with C++,
and especially does not mesh well with c++ exceptions.

So we ended up compiling lua as c++ using c++ exceptions in lua.

The C++ standard says this:
18.7.4 The function signature longjmp(jmp_buf jbuf, int val) has more restricted behavior in this International Standard. If any automatic objects would be destroyed by a thrown exception transferring control to another (destination) point in the program, then a call to longjmp(jbuf, val) at the throw point that transfers control to the same (destination) point has undefined behavior.

So use of longjmp in most C++ programs has undefined results.




Reply | Threaded
Open this post in threaded view
|

Re: Lua error handling and C++

Sam Roberts-2
In reply to this post by Glenn Maynard
On Thu, Oct 12, 2006 at 01:38:02AM -0400, Glenn Maynard wrote:
> On Thu, Oct 12, 2006 at 02:59:34PM +1000, Nick Gammon wrote:
> > >How are people dealing with error handling in C++?  setjmp doesn't
> > >get along with C++ destructors.
> > 
> > Just to help me understand the question, do you mean:
> > 
> > a. A Lua script calls some C++ code which might throw an exception?
> > 
> > or
> > 
> > b. Some C++ code calls a Lua script that might raise an error?
> 
> A,

Can't the C++ code catch all exceptions, and re-raise with lua_error()?

> and A': C++ calls the Lua C api, which might also throw an error.

Can't it use pcall(), and re-raise with throw?

Sam


Reply | Threaded
Open this post in threaded view
|

Re: Lua error handling and C++

Glenn Maynard
On Mon, Oct 16, 2006 at 11:31:28AM -0700, Sam Roberts wrote:
> Can't the C++ code catch all exceptions, and re-raise with lua_error()?

The problem is that C++ exceptions bloat executables too much to be
practical to leave enabled in many cases.  The use of a 150k library
requiring enabling an option that would bloat my binary by over 1MB
means it's effectively a 1.15MB library.

-- 
Glenn Maynard

Reply | Threaded
Open this post in threaded view
|

Re: Lua error handling and C++

Rici Lake-2

On 16-Oct-06, at 1:49 PM, Glenn Maynard wrote:

On Mon, Oct 16, 2006 at 11:31:28AM -0700, Sam Roberts wrote:
Can't the C++ code catch all exceptions, and re-raise with lua_error()?

The problem is that C++ exceptions bloat executables too much to be
practical to leave enabled in many cases.  The use of a 150k library
requiring enabling an option that would bloat my binary by over 1MB
means it's effectively a 1.15MB library.

You only have to catch exceptions if they might be thrown. If you're not using exceptions, and you're using no c++ library which uses exceptions, then there's no problem, surely?


Reply | Threaded
Open this post in threaded view
|

Re: Lua error handling and C++

Glenn Maynard
On Mon, Oct 16, 2006 at 02:00:11PM -0500, Rici Lake wrote:
> You only have to catch exceptions if they might be thrown. If you're 
> not using exceptions, and you're using no c++ library which uses 
> exceptions, then there's no problem, surely?

The suggestion (and expected use, in the headers) is to make Lua a C++
library that throws exceptions, instead of a C library that uses longjmp.

-- 
Glenn Maynard

Reply | Threaded
Open this post in threaded view
|

Re: Lua error handling and C++

Sam Roberts-2
In reply to this post by Glenn Maynard
On Mon, Oct 16, 2006 at 02:49:25PM -0400, Glenn Maynard wrote:
> On Mon, Oct 16, 2006 at 11:31:28AM -0700, Sam Roberts wrote:
> > Can't the C++ code catch all exceptions, and re-raise with lua_error()?
> 
> The problem is that C++ exceptions bloat executables too much to be
> practical to leave enabled in many cases.  The use of a 150k library
> requiring enabling an option that would bloat my binary by over 1MB
> means it's effectively a 1.15MB library.

Then I lost the thread. If you aren't using exceptions, what is the
C++-specific problem you have encountered?

Sam


Reply | Threaded
Open this post in threaded view
|

Re: Lua error handling and C++

Rici Lake-2
In reply to this post by Glenn Maynard

On 16-Oct-06, at 2:13 PM, Glenn Maynard wrote:

On Mon, Oct 16, 2006 at 02:00:11PM -0500, Rici Lake wrote:
You only have to catch exceptions if they might be thrown. If you're
not using exceptions, and you're using no c++ library which uses
exceptions, then there's no problem, surely?

The suggestion (and expected use, in the headers) is to make Lua a C++
library that throws exceptions, instead of a C library that uses longjmp.

That would be the suggestion if you were using a C++ program with exceptions. If you're not, then Lua error handling with C++ is no different than Lua error handling with C: to wit, if you acquire resources and then use a Lua API which might throw a Lua error, you must use lua_pcall or lua_cpcall in order to trap the error so that you can release the resources. (Alternatively, you can make sure that the resources are part of a Lua userdata with an appropriate __gc metamethod.)


Reply | Threaded
Open this post in threaded view
|

Re: Lua error handling and C++

Glenn Maynard
In reply to this post by Sam Roberts-2
On Mon, Oct 16, 2006 at 12:14:53PM -0700, Sam Roberts wrote:
> On Mon, Oct 16, 2006 at 02:49:25PM -0400, Glenn Maynard wrote:
> > On Mon, Oct 16, 2006 at 11:31:28AM -0700, Sam Roberts wrote:
> > > Can't the C++ code catch all exceptions, and re-raise with lua_error()?
> > 
> > The problem is that C++ exceptions bloat executables too much to be
> > practical to leave enabled in many cases.  The use of a 150k library
> > requiring enabling an option that would bloat my binary by over 1MB
> > means it's effectively a 1.15MB library.
> 
> Then I lost the thread. If you aren't using exceptions, what is the
> C++-specific problem you have encountered?

Lua error handling, currently, depends on either using longjmp or C++
exceptions.

longjmp is incompatible with C++ destructors, and will result in leaks.
That's why Lua allows C++ exceptions to be used instead.

But I find C++ exceptions are rarely an option: on large code, they
bloat the code massively, and if you're writing small code for a small
arch, then you can't afford the smaller bloat, either.  (I ported to
a 32MB arch, and disabling exceptions won me that 1MB, which was
enormous.)

int mybinding(lua_State *L)
{
    std::string s1 = luaL_checkstring(L, 1);
    std::string s2 = luaL_checkstring(L, 2);
    ...
}

works fine, unless the second arg throws an error, in which case you'll
probably leak s1.  (This one's easy to sidestep; as binding become
less trivial, it becomes harder.)

So, I'm asking for other options.  (Not with a great deal of hope, but
maybe someone's thought of something I havn't ...)

-- 
Glenn Maynard

Reply | Threaded
Open this post in threaded view
|

Re: Lua error handling and C++

Glenn Maynard
In reply to this post by Rici Lake-2
On Mon, Oct 16, 2006 at 02:21:19PM -0500, Rici Lake wrote:
> That would be the suggestion if you were using a C++ program with 
> exceptions. If you're not, then Lua error handling with C++ is no 
> different than Lua error handling with C: to wit, if you acquire 
> resources and then use a Lua API which might throw a Lua error, you 
> must use lua_pcall or lua_cpcall in order to trap the error so that you 
> can release the resources. (Alternatively, you can make sure that the 
> resources are part of a Lua userdata with an appropriate __gc 
> metamethod.)

An actual binding:

        static int SetSongOptions( T* p, lua_State *L )
        {
                ModsLevel m = Enum::Check<ModsLevel>( L, 1 );

                SongOptions so;

                so.FromString( SArg(2) );
                p->m_SongOptions.Assign( m, so );
                return 0;
        }

SArg(2) is shorthand for "lua_checkstring(L, 2), and convert to a std::string".
It might error; if that happens, SongOptions will leak.

I could write it like this (which may also not be portable, but is
definitely better):

        static int SetSongOptions( T* p, lua_State *L )
        {
                ModsLevel m = Enum::Check<ModsLevel>( L, 1 );
                size_t iOptionsSize;
                const char *pOptions = luaL_checklstring( L, 2, &iOptionsSize );

                // everything that might trip Lua errors above; all C++
                // ctors below
                std::string sOptions( pOptions, iOptionsSize );
                SongOptions so;

                so.FromString( sOptions );
                p->m_SongOptions.Assign( m, so );
                return 0;
        }

but this is cumbersome here and worse in the general case (we have helpers
like SArg for a reason: there are lots of them).  It's also hard to know
if it's correct; there's no way to get a compiler warning if it's wrong.

I suppose I could try something like this:

        static int SetSongOptions( T* p, lua_State *L )
        {
                ModsLevel m = Enum::Check<ModsLevel>( L, 1 );
                luaL_checkstring( L, 2 );

                // ...

                so.FromString( SArg(2) );
         }

where SArg is unlikely to error, since it checked the type and did any
string conversion already.  But this is just working around one source
of errors; errors detected during less trivial traversals are harder to
check in advance.  (Like the above, it's also hard to check for correctness.
Since they would be problems that would compile silently, manifest as
small memory leaks, and only happen as a result of uncommon Lua errors,
this would be a persistant source of small, hard to fix code errors.)

(As an aside, I'm not worried about OOM errors; popping up a dialog and
dying is acceptable here.  It's the other more common, less fatal errors
that would be nice to handle properly.)

-- 
Glenn Maynard

Reply | Threaded
Open this post in threaded view
|

Re: Lua error handling and C++

Sam Roberts-2
In reply to this post by Glenn Maynard
On Mon, Oct 16, 2006 at 03:27:30PM -0400, Glenn Maynard wrote:
> longjmp is incompatible with C++ destructors, and will result in leaks.

It is also "incompatible" with malloc/free, this:

> int mybinding(lua_State *L)
> {
>     std::string s1 = luaL_checkstring(L, 1);
>     std::string s2 = luaL_checkstring(L, 2);
>     ...
> }

is same problem as:

int mybinding(lua_State *L)
{
    char* s1 = strdup(luaL_checkstring(L, 1));
    char* s2 = strdup(luaL_checkstring(L, 2)); // -- leaks s1 on error
    ...
}

So the techniques used for C code apply equally here, simple is:

int mybinding(lua_State *L)
{
	// make checks that may error() so nothing will error() later
	luaL_checkstring(L, 1);
	luaL_checkstring(L, 2);
	...

	// allocate resources
    std::string s1 = lua_tostring(L, 1);
    std::string s2 = lua_tostring(L, 2);
    ...

	// do things that will not error()

....

	// free resources
....
}

Other idiom, as Rici (?) suggested and used in the PIL, is to first
create something that WILL be garbage-collected, like a user-data, then
put your two std::string inside the user-data. If lua longjmps out of
your code, lua will still have the UD on the stack, and lua will know it
needs to be garbage collected, and in the __gc metamethod you can
destroy the s1 and s2.

I guess if code is large enough, then refactor into parts of code that
allocates resources, and into a function that uses those resources and
may error(). pcall the function that may error.

Its hard to have the benefits of C++ automatic resource freeing with
exceptions, without paying the costs for exceptions!

Cheers,
Sam


Reply | Threaded
Open this post in threaded view
|

Re: Lua error handling and C++

Glenn Maynard
On Mon, Oct 16, 2006 at 12:51:05PM -0700, Sam Roberts wrote:
> ....
> 
> 	// free resources
> ....
> }

But this is stripping C++ of one of its biggest advantages: automatic
destruction.  If this has to free manually, then it reopens the old
human error path to memory leaks.  (That is to say, memory leaks in
the normal path, not just leaks in the error path, which is what I
have now.)  It's also very tedious.

> Other idiom, as Rici (?) suggested and used in the PIL, is to first
> create something that WILL be garbage-collected, like a user-data, then
> put your two std::string inside the user-data. If lua longjmps out of
> your code, lua will still have the UD on the stack, and lua will know it
> needs to be garbage collected, and in the __gc metamethod you can
> destroy the s1 and s2.

That means every object that I might want to create inside a binding
needs to be handled in this way.  Unfortunately, "delete p" where p is
a void* will not call destructors--that would make it easy.  I'm not
a big fan of the "give everything in the world a base class" paradigm ...

It might be worth it for strings, though.  SArg() could construct the
std::string, convert the stack element to the userdata containing it,
and return a reference to it (the C++ kind; string &), so it'd be mostly
transparent.  That way, at least the "Lua::Push(L, p->Func(SArg(1), IArg(2))"
shorthand bindings would remain concise (grab arguments, pass to
the bound function, push the result according to its type).

> I guess if code is large enough, then refactor into parts of code that
> allocates resources, and into a function that uses those resources and
> may error(). pcall the function that may error.
> 
> Its hard to have the benefits of C++ automatic resource freeing with
> exceptions, without paying the costs for exceptions!

Not at all.  If I have error cases, I return them in the traditional way:
through error returns.  I avoid doing things that can cause fatal errors
inside ctors.  This is a very practical scheme; this codebase is something
like 300k LOC (and seen four commercial releases), and if crypto++ is
disabled, can be built entirely without exceptions without hurting error
handling.

(You really do need exceptions if you want to handle OOM gracefully--there's
no other way to handle OOM from "new"--but, as a deliberate design tradeoff,
we don't do that.)

-- 
Glenn Maynard

Reply | Threaded
Open this post in threaded view
|

Re: Lua error handling and C++

Sam Roberts-2
On Mon, Oct 16, 2006 at 05:05:45PM -0400, Glenn Maynard wrote:
> That means every object that I might want to create inside a binding

Not every object.

> needs to be handled in this way.  Unfortunately, "delete p" where p is
> a void* will not call destructors--that would make it easy.  I'm not
> a big fan of the "give everything in the world a base class" paradigm ...

Probably no way to avoid the tedium, but I'm not sure if you understood.

Create a single user-data that contains a pointer to a struct MyArgs {
std::string localA; std::string localB; }, and with a __gc pointing to a
function that calls delete on (struct MyArgs*) when the UD is
collected, the struct members will be destroyed.

Basically, this is a process of taking your stack variables, and putting
them on the heap, wrapped in a UD, so lua can tell you when to destroy
them, longjmp or no longjmp. Example is here:

  http://www.lua.org/pil/29.1.html

but instead of having a DIR object, have a struct MyArgs, and instead of
calling opendir() call new MyArgs, and instead of calling closedir(),
call delete.

This may be tedious, but it is easy to be confident its "correct". C++
takes care of calling the dtors for all struct members, you make sure
that you call delete once in the __gc metamethods correctly, and lua
makes sure that the __gc metamethod is called. Since there is nothing on
the stack that is a leakable resource, there is nothing on the stack
that can be leaked. Also, "local" variables get freed the same way
error() or not, when lua cleans up the the lua->C stack.

You asked for suggestions on "other ways", this is one other way.

Cheers,
Sam

> > Its hard to have the benefits of C++ automatic resource freeing with
> > exceptions, without paying the costs for exceptions!
> 
> Not at all.

What does that mean?

If you have way of getting automatic resource freeing across longjmp
without paying the cost for exceptions, why does this thread exist?


Reply | Threaded
Open this post in threaded view
|

RE: Lua error handling and C++

Jerome Vuarand-2
In reply to this post by Glenn Maynard
luaL_check* functions are designed to throw an error if the type is not the correct one, and you don't want errors to be thrown, be it with C++ exceptions or C long jumps. There is no solution to that. If you don't want exceptions and long jumps, just don't use them. You don't have to use luaL_check* functions to convert your data. Here is a non-throwing version of your SArg (I made it a macro since in your code snippet you don't pass L explictly):

#define SArg(index) (lua_isstring(L, index)?lua_tostring(L, index):"")

I have troubles understanding why so much has been written around a problem that is so easy to solve.

-----Message d'origine-----
De : [hidden email] [[hidden email]] De la part de Glenn Maynard
Envoyé : 16 octobre 2006 15:48
À : [hidden email]
Objet : Re: Lua error handling and C++

On Mon, Oct 16, 2006 at 02:21:19PM -0500, Rici Lake wrote:
> That would be the suggestion if you were using a C++ program with 
> exceptions. If you're not, then Lua error handling with C++ is no 
> different than Lua error handling with C: to wit, if you acquire 
> resources and then use a Lua API which might throw a Lua error, you 
> must use lua_pcall or lua_cpcall in order to trap the error so that 
> you can release the resources. (Alternatively, you can make sure that 
> the resources are part of a Lua userdata with an appropriate __gc
> metamethod.)

An actual binding:

        static int SetSongOptions( T* p, lua_State *L )
        {
                ModsLevel m = Enum::Check<ModsLevel>( L, 1 );

                SongOptions so;

                so.FromString( SArg(2) );
                p->m_SongOptions.Assign( m, so );
                return 0;
        }

SArg(2) is shorthand for "lua_checkstring(L, 2), and convert to a std::string".
It might error; if that happens, SongOptions will leak.

I could write it like this (which may also not be portable, but is definitely better):

        static int SetSongOptions( T* p, lua_State *L )
        {
                ModsLevel m = Enum::Check<ModsLevel>( L, 1 );
                size_t iOptionsSize;
                const char *pOptions = luaL_checklstring( L, 2, &iOptionsSize );

                // everything that might trip Lua errors above; all C++
                // ctors below
                std::string sOptions( pOptions, iOptionsSize );
                SongOptions so;

                so.FromString( sOptions );
                p->m_SongOptions.Assign( m, so );
                return 0;
        }

but this is cumbersome here and worse in the general case (we have helpers like SArg for a reason: there are lots of them).  It's also hard to know if it's correct; there's no way to get a compiler warning if it's wrong.

I suppose I could try something like this:

        static int SetSongOptions( T* p, lua_State *L )
        {
                ModsLevel m = Enum::Check<ModsLevel>( L, 1 );
                luaL_checkstring( L, 2 );

                // ...

                so.FromString( SArg(2) );
         }

where SArg is unlikely to error, since it checked the type and did any string conversion already.  But this is just working around one source of errors; errors detected during less trivial traversals are harder to check in advance.  (Like the above, it's also hard to check for correctness.
Since they would be problems that would compile silently, manifest as small memory leaks, and only happen as a result of uncommon Lua errors, this would be a persistant source of small, hard to fix code errors.)

(As an aside, I'm not worried about OOM errors; popping up a dialog and dying is acceptable here.  It's the other more common, less fatal errors that would be nice to handle properly.)

--
Glenn Maynard


12