guarantees of exception safety in Lua

classic Classic list List threaded Threaded
18 messages Options
Reply | Threaded
Open this post in threaded view
|

guarantees of exception safety in Lua

David Manura
I was wondering what guarantees of exception safety[1] are provided in Lua.

To help in testing, the below patch to Lua prints "[r]" upon each
allocation attempt and implements a function failalloc(n) that
schedules a memory allocation failure in the next n-th allocation.
This is used in the following example.  Here, g() has a memory
allocation failure when trying to grow that stack in the OP_CALL.
This causes the program to prints out that x is 2 despite the intended
invariant that x == 1.

  local function g()
    local x=1,2,3,4,5,6,7,8,9,10, 11,12,13,14,15,16,17,18,19,20,
            21,22,23,24,25,26,27,28
  end
  local x = 1
  local function test()
    x = x + 1
    g()
    x = x - 1
  end

  failalloc(1)
  local ok,msg = pcall(test)
  failalloc(0)

  print(x, ok, msg)

Although the Lua Reference Manual[2] defines the behavior of stack
growing for C functions, I don't think it does so for Lua functions.

[1] http://en.wikipedia.org/wiki/Exception_handling
[2] http://www.lua.org/manual/5.1/manual.html#3.2

diff -ur lua-5.1.4/src/lbaselib.c lua-mem/src/lbaselib.c
--- lua-5.1.4/src/lbaselib.c    2008-02-14 11:46:22.000000000 -0500
+++ lua-mem/src/lbaselib.c      2008-10-25 16:36:44.437500000 -0400
@@ -19,7 +19,12 @@
 #include "lauxlib.h"
 #include "lualib.h"

+extern int g_failalloc;

+static int luaB_failalloc(lua_State *L) {
+    g_failalloc = lua_tonumber(L, 1);
+    return 0;
+}


 /*
@@ -449,6 +454,7 @@
   {"collectgarbage", luaB_collectgarbage},
   {"dofile", luaB_dofile},
   {"error", luaB_error},
+  {"failalloc", luaB_failalloc},
   {"gcinfo", luaB_gcinfo},
   {"getfenv", luaB_getfenv},
   {"getmetatable", luaB_getmetatable},
diff -ur lua-5.1.4/src/lmem.c lua-mem/src/lmem.c
--- lua-5.1.4/src/lmem.c        2007-12-27 08:02:25.000000000 -0500
+++ lua-mem/src/lmem.c  2008-10-25 17:04:41.062500000 -0400
@@ -6,6 +6,7 @@


 #include <stddef.h>
+#include <stdio.h>

 #define lmem_c
 #define LUA_CORE
@@ -42,6 +43,7 @@

 #define MINSIZEARRAY   4

+int g_failalloc = 0;

 void *luaM_growaux_ (lua_State *L, void *block, int *size, size_t size_elems,
                      int limit, const char *errormsg) {
@@ -69,13 +71,14 @@
 }


-
 /*
 ** generic allocation routine.
 */
 void *luaM_realloc_ (lua_State *L, void *block, size_t osize, size_t nsize) {
   global_State *g = G(L);
   lua_assert((osize == 0) == (block == NULL));
+  printf("[r]");
+  if (g_failalloc && --g_failalloc == 0) { luaD_throw(L, LUA_ERRMEM); }
   block = (*g->frealloc)(g->ud, block, osize, nsize);
   if (block == NULL && nsize > 0)
     luaD_throw(L, LUA_ERRMEM);

Reply | Threaded
Open this post in threaded view
|

Re: guarantees of exception safety in Lua

Sam Roberts
On Sat, Oct 25, 2008 at 3:30 PM, David Manura <[hidden email]> wrote:
> I was wondering what guarantees of exception safety[1] are provided in Lua.
>
>  local function g()
>    local x=1,2,3,4,5,6,7,8,9,10, 11,12,13,14,15,16,17,18,19,20,
>            21,22,23,24,25,26,27,28
>  end
>  local x = 1
>  local function test()
>    x = x + 1  -- INC
>    g()
>    x = x - 1   -- DEC
>  end

If you want to deal with memory allocation errors, you have to write
all your code as if an error can be raised at any time, don't you?
What other choices are there?

Sam

Reply | Threaded
Open this post in threaded view
|

Re: guarantees of exception safety in Lua

Roberto Ierusalimschy
In reply to this post by David Manura
> I was wondering what guarantees of exception safety[1] are provided in Lua.

We do extensive testing about the safety of Lua. When we compile Lua
in test mode, the allocation function (not luaM_realloc_) is redefined
so that it throws an error when the total memory reaches a limit. Then
we perform several tasks like this: we set this limit to memory_in_use
(so that any attempt to allocate more memory fails) and then go on
increasing it by 3 bytes each time, until we can complete the task. This
ensures that we have memory fails in practically every allocation point
in the code.


> To help in testing, the below patch to Lua prints "[r]" upon each
> allocation attempt and implements a function failalloc(n) that
> schedules a memory allocation failure in the next n-th allocation.
> This is used in the following example.  Here, g() has a memory
> allocation failure when trying to grow that stack in the OP_CALL.
> This causes the program to prints out that x is 2 despite the intended
> invariant that x == 1.
> 
>   local function g()
>     local x=1,2,3,4,5,6,7,8,9,10, 11,12,13,14,15,16,17,18,19,20,
>             21,22,23,24,25,26,27,28
>   end
>   local x = 1
>   local function test()
>     x = x + 1
>     g()
>     x = x - 1
>   end
> 
>   failalloc(1)
>   local ok,msg = pcall(test)
>   failalloc(0)
> 
>   print(x, ok, msg)

This result is expected, is it not? "test" was interrupted by an
exception (memory allocation error, caught by pcall) after
incrementing x but before decrementing it. What value did you expect?


> Although the Lua Reference Manual[2] defines the behavior of stack
> growing for C functions, I don't think it does so for Lua functions.

The Lua stack is not visible for the users. Memory allocation errors
may occur almost anywhere inside a Lua script.


>  void *luaM_realloc_ (lua_State *L, void *block, size_t osize, size_t nsize) {
>    global_State *g = G(L);
>    lua_assert((osize == 0) == (block == NULL));
> +  printf("[r]");
> +  if (g_failalloc && --g_failalloc == 0) { luaD_throw(L, LUA_ERRMEM); }
>    block = (*g->frealloc)(g->ud, block, osize, nsize);
>    if (block == NULL && nsize > 0)
>      luaD_throw(L, LUA_ERRMEM);

This patch is not correct: "Lua assumes that the allocator never fails
when osize >= nsize."  (http://www.lua.org/manual/5.1/manual.html#lua_Alloc)

-- Roberto

Reply | Threaded
Open this post in threaded view
|

Re: guarantees of exception safety in Lua

Peter Cawley
While Lua as a language provides the guarantee that no memory will be
leaked, it is impossible to provide anything further at the language
level. As is the case with C++, exception guarantees can only be given
at the per-function or per-library level. If the author of the
function wants to provide invariants in the case of exceptions, then
they need to assume that every line may fail and restore invariants if
they have been modified.

2008/10/27 Roberto Ierusalimschy <[hidden email]>:
>> I was wondering what guarantees of exception safety[1] are provided in Lua.
>
> We do extensive testing about the safety of Lua. When we compile Lua
> in test mode, the allocation function (not luaM_realloc_) is redefined
> so that it throws an error when the total memory reaches a limit. Then
> we perform several tasks like this: we set this limit to memory_in_use
> (so that any attempt to allocate more memory fails) and then go on
> increasing it by 3 bytes each time, until we can complete the task. This
> ensures that we have memory fails in practically every allocation point
> in the code.
>
>
>> To help in testing, the below patch to Lua prints "[r]" upon each
>> allocation attempt and implements a function failalloc(n) that
>> schedules a memory allocation failure in the next n-th allocation.
>> This is used in the following example.  Here, g() has a memory
>> allocation failure when trying to grow that stack in the OP_CALL.
>> This causes the program to prints out that x is 2 despite the intended
>> invariant that x == 1.
>>
>>   local function g()
>>     local x=1,2,3,4,5,6,7,8,9,10, 11,12,13,14,15,16,17,18,19,20,
>>             21,22,23,24,25,26,27,28
>>   end
>>   local x = 1
>>   local function test()
>>     x = x + 1
>>     g()
>>     x = x - 1
>>   end
>>
>>   failalloc(1)
>>   local ok,msg = pcall(test)
>>   failalloc(0)
>>
>>   print(x, ok, msg)
>
> This result is expected, is it not? "test" was interrupted by an
> exception (memory allocation error, caught by pcall) after
> incrementing x but before decrementing it. What value did you expect?
>
>
>> Although the Lua Reference Manual[2] defines the behavior of stack
>> growing for C functions, I don't think it does so for Lua functions.
>
> The Lua stack is not visible for the users. Memory allocation errors
> may occur almost anywhere inside a Lua script.
>
>
>>  void *luaM_realloc_ (lua_State *L, void *block, size_t osize, size_t nsize) {
>>    global_State *g = G(L);
>>    lua_assert((osize == 0) == (block == NULL));
>> +  printf("[r]");
>> +  if (g_failalloc && --g_failalloc == 0) { luaD_throw(L, LUA_ERRMEM); }
>>    block = (*g->frealloc)(g->ud, block, osize, nsize);
>>    if (block == NULL && nsize > 0)
>>      luaD_throw(L, LUA_ERRMEM);
>
> This patch is not correct: "Lua assumes that the allocator never fails
> when osize >= nsize."  (http://www.lua.org/manual/5.1/manual.html#lua_Alloc)
>
> -- Roberto
>

Reply | Threaded
Open this post in threaded view
|

Re: guarantees of exception safety in Lua

David Manura
On Mon, Oct 27, 2008 at 8:18 AM, Roberto Ierusalimschy wrote:
> Memory allocation errors may occur almost anywhere inside a Lua script.

On Mon, Oct 27, 2008 at 1:12 PM, Peter Cawley wrote:
> ...then they need to assume that every line may fail

In Lua must we then assume that every line may fail?  Here is a
function that transfers an amount between bank accounts:

  local total1 = 10
  local total2 = 10

  local function transfer_amount(total1, total2, amount)
    total1 = total1 - amount
    total2 = total2 + amount
  end

Could the line "total1 = total1 - amount" execute but not the line
"total2 = total2 + amount"?  If so, we might pcall each statement and
add code to recover (total1 = total1 + amount), but what if the
recovery code fails?  In C/C++, those lines always succeed, at least
assuming no extraordinary condition (power failure, hardware failure,
OS crash, etc.).  The reason it succeeds is that all variables used
here, including temporaries, are allocated on the stack, and the stack
is preallocated.  Given restrictions on things like recursion, stack
requirements can be determined by static analysis[1].  In Lua,
however, the stack memory is allocated incrementally from the heap
(lua_Alloc) at unspecified times.

[1] http://www.embedded.com/columns/technicalinsights/47101892?_requestid=149323

Reply | Threaded
Open this post in threaded view
|

Re: guarantees of exception safety in Lua

Mark Hamburg
The fundamental problem is that metamethods in Lua and operator overloading in C++ can change the meaning of + and - and hence potentially those operations can throw in either language.

If you know more about the types of the values involved, you can avoid that in particular cases. As written, the behavior of transfer_amount depends on the types of total1, total2, and amount. Given the full code, passing total1 and total2 as parameters was probably an error, but that still leaves the type of amount as a potential source of trouble.

Mark

On Oct 27, 2008, at 5:56 PM, David Manura wrote:

On Mon, Oct 27, 2008 at 8:18 AM, Roberto Ierusalimschy wrote:
Memory allocation errors may occur almost anywhere inside a Lua script.

On Mon, Oct 27, 2008 at 1:12 PM, Peter Cawley wrote:
...then they need to assume that every line may fail

In Lua must we then assume that every line may fail?  Here is a
function that transfers an amount between bank accounts:

 local total1 = 10
 local total2 = 10

 local function transfer_amount(total1, total2, amount)
   total1 = total1 - amount
   total2 = total2 + amount
 end

Could the line "total1 = total1 - amount" execute but not the line
"total2 = total2 + amount"?  If so, we might pcall each statement and
add code to recover (total1 = total1 + amount), but what if the
recovery code fails?  In C/C++, those lines always succeed, at least
assuming no extraordinary condition (power failure, hardware failure,
OS crash, etc.).  The reason it succeeds is that all variables used
here, including temporaries, are allocated on the stack, and the stack
is preallocated.  Given restrictions on things like recursion, stack
requirements can be determined by static analysis[1].  In Lua,
however, the stack memory is allocated incrementally from the heap
(lua_Alloc) at unspecified times.

[1] http://www.embedded.com/columns/technicalinsights/47101892?_requestid=149323



Reply | Threaded
Open this post in threaded view
|

Re: guarantees of exception safety in Lua

Roberto Ierusalimschy
In reply to this post by David Manura
> [...]  In C/C++, those lines always succeed, at least
> assuming no extraordinary condition (power failure, hardware failure,
> OS crash, etc.).  The reason it succeeds is that all variables used
> here, including temporaries, are allocated on the stack, and the stack
> is preallocated.  Given restrictions on things like recursion, stack
> requirements can be determined by static analysis[1].  In Lua,
> however, the stack memory is allocated incrementally from the heap
> (lua_Alloc) at unspecified times.

I guess this kind of failure is as "unspecified" in Lua as it is in
C/C++ (albeit much more frequent). As far as I know the C standard
does not specify stack requirements for functions and local variables.
So, any static analysis is implementation dependent. In Lua it is also
possible ("given restrictions on things like ...") to write programs
that use an implementation-dependent maximum amount of memory, so that
memory errors never arise.

That said, I would add that the correct answer is "Lua is not C". I must
confess that I would be very reluctant to board a plane with flight
control implemented in Lua or any other dynamic language. (Even C++
makes me nervous if I think about that :)

-- Roberto

Reply | Threaded
Open this post in threaded view
|

RE: guarantees of exception safety in Lua

Mark Hamburg
In reply to this post by David Manura
> -------- Original Message --------
> Subject: Re: guarantees of exception safety in Lua
> From: Roberto Ierusalimschy <[hidden email]>
> Date: Tue, October 28, 2008 4:54 am
> To: Lua list <[hidden email]>
>
>
> That said, I would add that the correct answer is "Lua is not C". I must
> confess that I would be very reluctant to board a plane with flight
> control implemented in Lua or any other dynamic language. (Even C++
> makes me nervous if I think about that :)
>
> -- Roberto

Interesting. I think Guido von Rossum once said he would be happier with
an airplane where the control logic was written in Python, but I may be
misremembering this.

Mark




Reply | Threaded
Open this post in threaded view
|

Re: guarantees of exception safety in Lua

Enrico Colombini
[hidden email] wrote:
Interesting. I think Guido von Rossum once said he would be happier with
an airplane where the control logic was written in Python, but I may be
misremembering this.

In that case, I'd rather fly in a plane piloted by Roberto ;-)

  .Erix.

Reply | Threaded
Open this post in threaded view
|

Re: guarantees of exception safety in Lua

Javier Guerra Giraldez
In reply to this post by Mark Hamburg
On Tue, Oct 28, 2008 at 11:51 AM,  <[hidden email]> wrote:
> Interesting. I think Guido von Rossum once said he would be happier with
> an airplane where the control logic was written in Python, but I may be
> misremembering this.

that says a lot more about the men, than about the language... or
maybe about the audience.  imagine if Guido had say something like
Roberto to a wide public!  that would mean a surge in Ruby popularity
;-)

-- 
Javier

Reply | Threaded
Open this post in threaded view
|

RE: guarantees of exception safety in Lua

Diego Nehab-3
In reply to this post by Mark Hamburg
Hi,

Interesting. I think Guido von Rossum once said he would be happier with
an airplane where the control logic was written in Python, but I may be
misremembering this.

I remember somebody from NASA at a Lua conference some time back saying
lots of people that write software there don't believe in dynamic memory
allocation. They don't even believe in functions from the C runtime. They
write their own stuff from scratch, just to make sure. Everything static, no recursion. Just for loops. That gives me a warm feeling of
safety.

Regards,
Diego

Reply | Threaded
Open this post in threaded view
|

Re: guarantees of exception safety in Lua

David Given
Diego Nehab wrote:
[...]
> I remember somebody from NASA at a Lua conference some time back saying
> lots of people that write software there don't believe in dynamic memory
> allocation. They don't even believe in functions from the C runtime. They
> write their own stuff from scratch, just to make sure. Everything
> static, no recursion. Just for loops. That gives me a warm feeling of
> safety.

That's actually very traditional in the hard embedded world --- if you
look at platforms like VxWorks or Nucleus you're frequently see that
dynamic memory allocation --- malloc --- is an optional feature.
Instead, all data is stored in globals, which get allocated by the
linker at compile time.

Going even further, if you don't need reentrant functions, you can do
away with the stack --- all function parameters become global, and each
function gets exactly one return address. This can be useful on really
resource-limited machines, because you can make much more efficient use
of memory; stacks inherently waste memory because you have to allow
space for them to expand into.

But it's also useful because it makes the application much more
predictable, and therefore easier to verify. It becomes much easier to
statically find programming errors and to verify that functions are
guaranteed to exit on time, etc.

As late as 1996, spacecraft like the Sojourner Mars rover were running
off 8085 and their ilk, and that early spacecraft like like the Viking
orbiters only had 4k of 18-bit words of memory --- 2k for code, 2k for
data. The Pioneers and the Mariner Mars probes had CPUs constructed out
of discrete TTL logic components! It's only relatively recently that
space vehicles ran anything like what we'd consider to be 'normal' code.
>From the computing perspective, it's an utterly different world in space...

In a desperate attempt to bring things back on topic, I'll point out
that it would be possible to make a no-dynamic-allocation Lua subset,
but it wouldn't be very like what we'd consider to be Lua: no table
creation, no assignment to arbitrary table keys, and a very limited
amount of function nesting. It'd probably be relatively straightforward
to compile it to machine code, TBH, because it wouldn't support any of
the dynamic features that make Lua Lua... I wonder if such a thing would
be useful?

-- 
David Given
[hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: guarantees of exception safety in Lua

Sam Roberts
On Tue, Oct 28, 2008 at 12:19 PM, David Given <[hidden email]> wrote:
> In a desperate attempt to bring things back on topic, I'll point out

To help you out in this attempt... :-)

I've spent a bunch of time testing libraries written for embedded
systems that were expected to deal with malloc failure (by returning
an error and releasing all memory,
as opposed to segfaulting and mem leaking).

What we did was wrap our unit tests in a loop that basically said:

e = ENOMEM;
for(failon=1;  e == ENOMEN; failon++)
{
     e = run_unit_test()
     assert(allocated_memory == 0);
     assert(e == ENOMEM || e == 0);
}

failon would force the ith call of malloc to fail. Interestingly,
Symbian is the only dev environment I've seen that seems to recognize
the utitility of this, and it has a mode in which you can  seed its
allocation library, and the seed will be used to "randomly" fail
allocations. An excellent idea.

Anyhow, all the code we tested segfaulted or leaked memory, and it had
been written knowing that it would be tested like this. All the code
"looked" like it did the right things - checked malloc returns,
cleaned up prior allocation when previous ones had failed, etc., but
always something would have been forgotten.

I've since come to the conclusion that malloc() NEVER fails on modern
operating systems, so the code theoretically written to recover from
failure never runs, and code that has never been run is not going to
work when it is run.

I used to think programming with an xmalloc() that just exited on
allocation failure was lazy. No longer. I know think that every
programmer who thinks their code survives malloc failure (but haven't
acutally tested it) is wrong. And I've proved it repeatedly on code
who's authors thought they were the exception.

Anyhow, what exactly would a lua program that caught an out of memory
exception do? There isn't enough memory to run lua, so how is any lua
code supposed to recover? Cleanup and get out are pretty much your
only options.

It would be trivial to do this kind of testing with lua, because you
can pass in your own malloc implementation, and a destroyed lua_State
should clean up all memory allocated during its lifetime. I might hack
up a lua exe some time to do this, it'd be interesting!

Cheers,
Sam

Reply | Threaded
Open this post in threaded view
|

Re: guarantees of exception safety in Lua

David Given
Sam Roberts wrote:
[...]
> I've since come to the conclusion that malloc() NEVER fails on modern
> operating systems, so the code theoretically written to recover from
> failure never runs, and code that has never been run is not going to
> work when it is run.

On most common configurations of Linux (and probably other Unices as
well), this is true --- malloc() will always return you a valid pointer,
unless you run out of address space. The physical memory is only
allocated when the page is accessed for the first time. If the
allocation fails, you'll get an address exception.

This apparently vastly improves performance and memory usage, because
you only every need to allocate as much memory as a program actually
uses, rather than how much it says it uses. (As a side effect, it means
that your stack autoextends as it grows, which is very useful.) It does
make me rather nervous, though. There is an option to turn this off.

[...]
> Anyhow, what exactly would a lua program that caught an out of memory
> exception do? There isn't enough memory to run lua, so how is any lua
> code supposed to recover? Cleanup and get out are pretty much your
> only options.

Well, a system I worked with in the past dealt with this by keeping a
preconstructed 'out of memory' exception in the thread structure. This
meant that you could always throw one without needing to do any memory
allocation. When it got thrown, destructors would run, which would free
memory, which would allow the system to recover, etc. I don't believe it
ever actually happened in real life.

-- 
ââââ ïïïïïïïïïïïïïï âââââ http://www.cowlark.com âââââ
â
â â'â',âNââSâ'âââ(3=T)âMâ2=Tââ+/(Vâ"âM),(Vâ"M),(V,âV)â"(V,Vâ1â1)â"âM)'
â --- Conway's Game Of Life, in one line of APL

Attachment: signature.asc
Description: OpenPGP digital signature

Reply | Threaded
Open this post in threaded view
|

Re: guarantees of exception safety in Lua

David Kastrup
In reply to this post by David Given
David Given <[hidden email]> writes:

> Diego Nehab wrote:
> [...]
>> I remember somebody from NASA at a Lua conference some time back saying
>> lots of people that write software there don't believe in dynamic memory
>> allocation. They don't even believe in functions from the C runtime. They
>> write their own stuff from scratch, just to make sure. Everything
>> static, no recursion. Just for loops. That gives me a warm feeling of
>> safety.
>
> That's actually very traditional in the hard embedded world --- if you
> look at platforms like VxWorks or Nucleus you're frequently see that
> dynamic memory allocation --- malloc --- is an optional feature.
> Instead, all data is stored in globals, which get allocated by the
> linker at compile time.
>
> Going even further, if you don't need reentrant functions, you can do
> away with the stack --- all function parameters become global, and each
> function gets exactly one return address. This can be useful on really
> resource-limited machines, because you can make much more efficient use
> of memory; stacks inherently waste memory because you have to allow
> space for them to expand into.

Fortran machine architectures tended to be stackless.  I remember the
CDC machines (Cyber 175 was what I used) having a "return jump
instruction".  Before jumping to the given destination, it wrote a jump
to _after_ the RJI instructions into the memory word preceding the
destination.  When the called function finished, it jumped to this
instruction preceding its body.

Of course, recursive function calls were not possible.

-- 
David Kastrup


Reply | Threaded
Open this post in threaded view
|

Re: guarantees of exception safety in Lua

David Kastrup
David Kastrup <[hidden email]> writes:

> David Given <[hidden email]> writes:
>
>> Diego Nehab wrote:
>> [...]
>>> I remember somebody from NASA at a Lua conference some time back saying
>>> lots of people that write software there don't believe in dynamic memory
>>> allocation. They don't even believe in functions from the C runtime. They
>>> write their own stuff from scratch, just to make sure. Everything
>>> static, no recursion. Just for loops. That gives me a warm feeling of
>>> safety.
>>
>> That's actually very traditional in the hard embedded world --- if you
>> look at platforms like VxWorks or Nucleus you're frequently see that
>> dynamic memory allocation --- malloc --- is an optional feature.
>> Instead, all data is stored in globals, which get allocated by the
>> linker at compile time.
>>
>> Going even further, if you don't need reentrant functions, you can do
>> away with the stack --- all function parameters become global, and each
>> function gets exactly one return address. This can be useful on really
>> resource-limited machines, because you can make much more efficient use
>> of memory; stacks inherently waste memory because you have to allow
>> space for them to expand into.
>
> Fortran machine architectures tended to be stackless.  I remember the
> CDC machines (Cyber 175 was what I used) having a "return jump
> instruction".  Before jumping to the given destination, it wrote a jump
> to _after_ the RJI instructions into the memory word preceding the
> destination.  When the called function finished, it jumped to this
> instruction preceding its body.
>
> Of course, recursive function calls were not possible.

I'm getting old.  The instruction was just named "RJ", and it wrote to
the memory word _at_ the destination and then continued.  Just took a
refresher from <URL:http://klausler.com/cdc6600.txt>

-- 
David Kastrup


Reply | Threaded
Open this post in threaded view
|

Re: guarantees of exception safety in Lua

Mark Hamburg
In reply to this post by David Given
Yes, common Unix behavior makes malloc failure safety basically non- sensical.

Failing only for lack of address space and then later failing for lack of physical resources when the memory is accessed means that many allocation failures simply can't be detected at the point of allocation which means that essentially any allocation is potentially a ticking time bomb. My work around in the past for the cases where I worried about this was to allocate the backing store on disk -- though one also has to avoid over commit there as well -- and then memory map it. But really one needs a "I'm going to allocate this big buffer and I'm fully prepared for the allocation to fail so either really give it to me or fail" call.

Then there's the matter of small allocations. Coming from machines with constrained memory, I was in the habit of being very careful to check all allocations and have a strategy for the cases where they failed. I felt dirty writing code that essentially assumed that small allocations on MacOS X would succeed. But the nature of the beast is that if you can't allocate a small amount of memory, there are enough pieces of the runtime that will be unhappy that you are toast anyway and one should spend effort dealing with error cases that can actually be addressed.

As for airplanes, I guess the answer is: Don't run too close to the allocation limits.

Mark


Reply | Threaded
Open this post in threaded view
|

Re: guarantees of exception safety in Lua

Florian Weimer
In reply to this post by David Given
* David Given:

> In a desperate attempt to bring things back on topic, I'll point out
> that it would be possible to make a no-dynamic-allocation Lua subset,
> but it wouldn't be very like what we'd consider to be Lua: no table
> creation, no assignment to arbitrary table keys, and a very limited
> amount of function nesting. It'd probably be relatively straightforward
> to compile it to machine code, TBH, because it wouldn't support any of
> the dynamic features that make Lua Lua... I wonder if such a thing would
> be useful?

It could be used to implement parts of Lua.  Scheme48 does something
similar.  Pypy has got some non-dynamic Python subset as well, IIRC.