RFE: refman - adding mention of os.exit not closing variables with default arguments.

classic Classic list List threaded Threaded
23 messages Options
12
Reply | Threaded
Open this post in threaded view
|

RFE: refman - adding mention of os.exit not closing variables with default arguments.

Lorenzo Donati-3
Hi Lua Team!

Sorry for bringing this up again, but I think it is important, and maybe
it was swamped by other threads.

In another thread it was discussed that calling os.exit without a "true"
second argument won't close pending "to close" variables because the Lua
state will not be closed itself.

This fact is not mentioned explicitly in the manual, i.e. the inference
that non closing the main Lua state won't close those vars.

This is not an easy inference to do, also because section 3.3.8 goes a
long way to explain other ways those variables will or won't be closed.

So a reader, especially a pure Lua programmer, would be highly
surprised, IMO, if calling os.exit() /won't/ close any pending vars.

I think that an explicit mention of this behavior should be made both in
3.3.8 AND in os.exit docs.

Something on these lines:

3.3.8
...
Note that calling os.exit without closing the Lua state won't close any
to-be-closed variables.
------

os.exit
...
If the optional second argument close is true, closes the Lua state
before exiting and also closes any pending to-be-closed variables.



To-be-closed variables are a new feature and a safety- and data
integrity-related one so, IMO, users should be given extremely clear
information on what are the guarantees of the new mechanisms, otherwise
they could get a false sense of security and put nasty bugs in their
program.

BTW, are there any undocumented cases in which to-be-closed variables
could not be closed?

Thanks!

-- Lorenzo



Reply | Threaded
Open this post in threaded view
|

Re: RFE: refman - adding mention of os.exit not closing variables with default arguments.

Gé Weijers
> To-be-closed variables are a new feature and a safety- and data
> integrity-related one so, IMO, users should be given extremely clear
> information on what are the guarantees of the new mechanisms, otherwise
> they could get a false sense of security and put nasty bugs in their
> program.

Another trap you can fall into is to put a local ... <close> variable
inside a coroutine. There are a number of problems with that:

- you have to explicitly call coroutine.close(thread) to do the
variable cleanup if the coroutine is suspended (has yielded).
- coroutine.close does not work on threads that are not suspended or
dead,  i.e. threads that are waiting for coroutine.resume to return.
- if you call os.exit(<code>, true) from a coroutine no __close
metamethods will be called in any thread, not even the main one.

The behavior of os.exit is in my opinion a bit inconsistent, it
unwinds the stack and closes to-be-closed variables in the main
thread, but only if you call it from that main thread, and it does not
do anything to other coroutines.



--

Reply | Threaded
Open this post in threaded view
|

Re: RFE: refman - adding mention of os.exit not closing variables with default arguments.

pocomane
On Wed, Jun 24, 2020 at 3:45 AM Gé Weijers wrote:
> - you have to explicitly call coroutine.close(thread) to do the
> variable cleanup if the coroutine is suspended (has yielded).

If the variable is in a sub-scope of the coroutine that does not
contain any yield, it will be closed at the end of scope. If there is
a yield, the close will be postponed until the next resume or
coroutine.close.  (example in [1]). It seems "Right" to me, in the
sense of the "Last surprise principle".

On Wed, Jun 24, 2020 at 3:45 AM Gé Weijers wrote:
> - coroutine.close does not work on threads that are not suspended or
> dead,  i.e. threads that are waiting for coroutine.resume to return.

I am not sure how to test this in pure lua. Can you make a short example?

On Wed, Jun 24, 2020 at 3:45 AM Gé Weijers wrote:
> - if you call os.exit(<code>, true) from a coroutine no __close
> metamethods will be called in any thread, not even the main one.

Actually it seems from a quick test [1] that the ones of the main
thread are closed. This is consistent with the manual, tought I agree
that a way to exit closing ALL the variables from ALL the threads
could be useful.

Well... at least it is consistent assuming that "Closing the Lua
state" means also "Closing the <close> variables". I agree with
Lorenzo that this should be explicitly stated. Maybe it is worth  also
to specify that the "Lua state" it refers to is just the "Main"
thread.

---

[1] The following code

```
local x = coroutine.wrap(function()
  do
    local A <close> = setmetatable({},{
      __close = function() print("A closed") end,
    })
  end
  local B <close> = setmetatable({},{
    __close = function() print("B closed") end,
  })
  coroutine.yield()
  os.exit(0, true)
  coroutine.yield()
end)

local C <close> = setmetatable({},{
  __close = function() print("C closed") end,
})

print("Call 1")
x()
print("Call 2")
-- os.exit(0, true)
x()
print("Call 3")
```

prints:

```
Call 1
A closed
Call 2
C closed
```
Reply | Threaded
Open this post in threaded view
|

Re: RFE: refman - adding mention of os.exit not closing variables with default arguments.

Robert Burke
In reply to this post by Gé Weijers
On Wed, Jun 24, 2020 at 10:45 AM Gé Weijers <[hidden email]> wrote:
> - coroutine.close does not work on threads that are not suspended or
> dead,  i.e. threads that are waiting for coroutine.resume to return.

This seems a bit hard to fix. If you put such a running thread that's
in coroutine.resume into a dead state, you'll have a hard time
yielding back into it. Is the problem here just about what happens
when you use os.exit in nested coroutines?
Reply | Threaded
Open this post in threaded view
|

Re: RFE: refman - adding mention of os.exit not closing variables with default arguments.

Lorenzo Donati-3
In reply to this post by Gé Weijers
On 24/06/2020 03:43, Gé Weijers wrote:

>> To-be-closed variables are a new feature and a safety- and data
>> integrity-related one so, IMO, users should be given extremely
>> clear information on what are the guarantees of the new mechanisms,
>> otherwise they could get a false sense of security and put nasty
>> bugs in their program.
>
> Another trap you can fall into is to put a local ... <close>
> variable inside a coroutine. There are a number of problems with
> that:
>
> - you have to explicitly call coroutine.close(thread) to do the
> variable cleanup if the coroutine is suspended (has yielded). -
> coroutine.close does not work on threads that are not suspended or
> dead,  i.e. threads that are waiting for coroutine.resume to return.
> - if you call os.exit(<code>, true) from a coroutine no __close
> metamethods will be called in any thread, not even the main one.
>

I'm not a big expert of coroutines, so I can't comment on the nuances of
the subject. Anyway it seems that this new "to-be-closed" variables
mechanism makes working with coroutines more awkward, at least if
someone wants to use TBC vars in coroutines.

> The behavior of os.exit is in my opinion a bit inconsistent, it
> unwinds the stack and closes to-be-closed variables in the main
> thread, but only if you call it from that main thread, and it does
> not do anything to other coroutines.

I wonder whether the whole mechanism is amenable of improvement in these
areas (i.e. guaranteed closing despite any mechanism that makes a
program terminate in orderly fashion) or there is a fundamental flaw in
the design that makes it impossible to handle some specific cases.

Although I've always wanted a clean, language-supported way to do RAII
in Lua, I'm still quite unsure if the current mechanism, as it is now,
is worth the hassle.

I didn't follow the 5.4 development closely, but it seems that the
mechanism doesn't provide unique features. I mean, you can still do RAII
without it, it just needs more programming discipline.

The whole point of the mechanism, as I see it, is that you could do RAII
in a much simpler way. But if it introduces other, subtler, issues then
I'm beginning to question its usefulness.

To be clearer: now (Lua 5.3) I can do RAII using more discipline and
sprinkling error handling code in key parts of the code. This approach
adds clutter to the code but the error handling code is explicit and clear.

In 5.4 I could reduce clutter by using TBC vars, but then I must add
some extra checks or mechanism to handle those exceptional cases when
TBC vars mechanism "fails" (and coroutine users are especially worse
off, it seems). Moreover, the new syntax is not that great.

All in all, I'm starting to think that the mechanism is not
well-polished yet. At least not enough to warrant a shift of paradigm.
But maybe I'm missing something.

I think I will stick to the older way of doing this. At least I know
better how things can go wrong. So probably I won't upgrade to 5.4 yet,
since the only new features that could made me switch were TBC vars.
Const vars are cool, but all by themselves are not groundbreaking enough.

I'll probably wait and see what other users think of the new features
once they begin putting them to good use.


>
> Gé
>



-- Lorenzo






Reply | Threaded
Open this post in threaded view
|

Re: RFE: refman - adding mention of os.exit not closing variables with default arguments.

Francisco Olarte
Lorenzo.

On Wed, Jun 24, 2020 at 1:02 PM Lorenzo Donati
<[hidden email]> wrote:
> On 24/06/2020 03:43, Gé Weijers wrote:
...
> > The behavior of os.exit is in my opinion a bit inconsistent, it
> > unwinds the stack and closes to-be-closed variables in the main
> > thread, but only if you call it from that main thread, and it does
> > not do anything to other coroutines.
> I wonder whether the whole mechanism is amenable of improvement in these
> areas (i.e. guaranteed closing despite any mechanism that makes a
> program terminate in orderly fashion) or there is a fundamental flaw in
> the design that makes it impossible to handle some specific cases.

You must not forget os.exit calls exit, and exit is a big hammer.

You do not normally call exit(3) on a heavily threaded program or from
deep within a call stack, unless you want problems, it is normally
done as a civilized abort.

> Although I've always wanted a clean, language-supported way to do RAII
> in Lua, I'm still quite unsure if the current mechanism, as it is now,
> is worth the hassle.

Look at the RAII poster child. C++. If you call os.exit it makes some
cleanups, but does not destroy automatic storage duration objects (
which IMHO are the most similar to lua toclose ones ). It closes some
things, destroys automatics ( which may be similar to lua __gc on
globals, which IIRC is invoked ), and goes out. And toclose is not
RAII, is try-with-resources, which AFAIK is what is normally done to
approach RAII in garbage-collected languages, when realizing not all
resources are as easy as memory.

> I didn't follow the 5.4 development closely, but it seems that the
> mechanism doesn't provide unique features. I mean, you can still do RAII
> without it, it just needs more programming discipline.

You can approach RAII. And you can perfectly manage resources in
assembler, C++ compilers do it. But the point on RAII is freeing you
from too much programming discipline, insuring that
"do_database_stuff(create_database_connection())" and similar stuff
just work. If you try to do it by adding a destroy() method to every
"object" you use and carefully using pcall anywhere you can get an
error, and adding destroy() calls everywhere,  you do not gain much
clarity, it's better to seek other ways.

I really do not use os.exit, all my lua is embeded, but it should not
be to difficult to dessign your objects in a way that they are
destroyed ( just overrwrite os.exit, add some atexit() support, do
something like having and idempotent destroy method, make your objects
register in a table for atexit, make __close call destroy and
deregister them, maybe use some handle idiom for easier clearing,
easy-peasy).

> The whole point of the mechanism, as I see it, is that you could do RAII
> in a much simpler way. But if it introduces other, subtler, issues then
> I'm beginning to question its usefulness.

Use it like try-with-resources, for things which need to be cleaned
when the program errors and unwinds, keep using resources which need
to be properly cleaned on os.exit in other way, it doesn't have to
solve all the problems, and it manages to solve some of them.

> To be clearer: now (Lua 5.3) I can do RAII using more discipline and
> sprinkling error handling code in key parts of the code. This approach
> adds clutter to the code but the error handling code is explicit and clear.

That is because it is not RAII, the point of RAII is AVOIDING the sprinkling.

> In 5.4 I could reduce clutter by using TBC vars, but then I must add
> some extra checks or mechanism to handle those exceptional cases when
> TBC vars mechanism "fails" (and coroutine users are especially worse
> off, it seems). Moreover, the new syntax is not that great.

You do not need to add. Move what you want to TBC, keep the rest as is.

> All in all, I'm starting to think that the mechanism is not
> well-polished yet. At least not enough to warrant a shift of paradigm.
> But maybe I'm missing something.

IMNSHO, you'll never be able to have RAII in a language like lua,
Java, etc.. where ( ignoring lua strings, number, ... ) everything is
a pointer in disguise and you cannot have value objects and it is
relying on garbage collection ( i.e., in perl every "object" is
typically a hash, managed with a pointer in disguise, but the scalar
variables where you store them have ref-counting semantics, so you
have something like C++ but you only are allowed to use make_shared()
to create objects and shared_ptr to ref them ( it has its own problems
with cycles, as C++ ) ).

I've found garbage collection is typically great when you must do
complex memory only structures, not that much elsewhere. I've till now
been fortunate to be able to put this kind of things in arenas when I
have to use them in C++, so no need to use Boehms or similar things.
But for "heavy" resources, I rely on RAII, potentially with
shared_ptrs, and I never, never, use std::exit from deep down, I just
raise my exit_exception and catch it in main().

Francisco Olarte.
Reply | Threaded
Open this post in threaded view
|

Re: RFE: refman - adding mention of os.exit not closing variables with default arguments.

Roberto Ierusalimschy
In reply to this post by Gé Weijers
> Another trap you can fall into is to put a local ... <close> variable
> inside a coroutine. There are a number of problems with that:
>
> - you have to explicitly call coroutine.close(thread) to do the
> variable cleanup if the coroutine is suspended (has yielded).
> - coroutine.close does not work on threads that are not suspended or
> dead,  i.e. threads that are waiting for coroutine.resume to return.
> - if you call os.exit(<code>, true) from a coroutine no __close
> metamethods will be called in any thread, not even the main one.
>
> The behavior of os.exit is in my opinion a bit inconsistent, it
> unwinds the stack and closes to-be-closed variables in the main
> thread, but only if you call it from that main thread, and it does not
> do anything to other coroutines.

Having finalizers to back up __close solves all these cases, where an
object goes out of existence without going out of scope. The manual
hints about this, but it could be more emphatic.

The goal of <close> inside coroutines + coroutine.close is to allow
orderly kills of threads/coroutines in Lua.

-- Roberto
Reply | Threaded
Open this post in threaded view
|

Re: RFE: refman - adding mention of os.exit not closing variables with default arguments.

Lorenzo Donati-3
In reply to this post by Francisco Olarte
On 24/06/2020 13:40, Francisco Olarte wrote:

> Lorenzo.
>
> On Wed, Jun 24, 2020 at 1:02 PM Lorenzo Donati
> <[hidden email]> wrote:
>> On 24/06/2020 03:43, Gé Weijers wrote:
> ...
>>> The behavior of os.exit is in my opinion a bit inconsistent, it
>>> unwinds the stack and closes to-be-closed variables in the main
>>> thread, but only if you call it from that main thread, and it
>>> does not do anything to other coroutines.
>> I wonder whether the whole mechanism is amenable of improvement in
>> these areas (i.e. guaranteed closing despite any mechanism that
>> makes a program terminate in orderly fashion) or there is a
>> fundamental flaw in the design that makes it impossible to handle
>> some specific cases.
>
> You must not forget os.exit calls exit, and exit is a big hammer.
>
> You do not normally call exit(3) on a heavily threaded program or
> from deep within a call stack, unless you want problems, it is
> normally done as a civilized abort.
>

I'm no big expert of multithreading, either cooperative or preemptive,
and I use C only to a limited extent.

However I thought C's `exit` was the right way to exit a program
(possibly together with `atexit` mechanism) if returning from main is
too cumbersome.

At least C's standard seems to think so:
file:///G:/root/main/core/C-CPP/doc/standards/cppreference.com/en/c/program/exit.html

I admit I've no experience with C big programs (say 5kLOC+; especially
multithreaded/GUI ones), so I don't know what's current practice.



>> Although I've always wanted a clean, language-supported way to do
>> RAII in Lua, I'm still quite unsure if the current mechanism, as it
>> is now, is worth the hassle.
>
> Look at the RAII poster child. C++. If you call os.exit it makes
> some cleanups, but does not destroy automatic storage duration
> objects ( which IMHO are the most similar to lua toclose ones ). It
> closes some things, destroys automatics ( which may be similar to lua
> __gc on globals, which IIRC is invoked ), and goes out.

You are right! I just checked here:

file:///G:/root/main/core/C-CPP/doc/standards/cppreference.com/en/cpp/utility/program/exit.html

In C++ std::exit doesn't unwind the stack. I didn't know that!

About year 2000 I had been using C++ intensively for about 3 years  and
never stumbled on that info. Mostly was number-crunching, data analysis
and offline simulation of TLC systems, so no interactive stuff.


> And toclose is not RAII, is try-with-resources, which AFAIK is what
> is normally done to approach RAII in garbage-collected languages,
> when realizing not all resources are as easy as memory.
>

I don't follow you here. Why do you say it is not RAII? To me RAIIin C++
is when you create a local object whose ctor allocates resources and
whose destructor deallocates them when the object goes out of scope or
the stack unwinds (returns, exceptions), thanks to C++ semantics.

Apart from not there being an exact parallel, because of different
mechanisms and semantics, it seems Lua TBC vars are just a way to
conceptually do RAII, i.e. deterministic automatic release of resources.

The key point being the resource release handler (__close metamethod) is
defined upfront when you acquire the resource, and not lexically scoped
as a catch block at the end.

Am I missing something about what you are saying?


>> I didn't follow the 5.4 development closely, but it seems that the
>> mechanism doesn't provide unique features. I mean, you can still do
>> RAII without it, it just needs more programming discipline.
>
> You can approach RAII. And you can perfectly manage resources in
> assembler, C++ compilers do it. But the point on RAII is freeing you
> from too much programming discipline, insuring that
> "do_database_stuff(create_database_connection())" and similar stuff
> just work. If you try to do it by adding a destroy() method to every
> "object" you use and carefully using pcall anywhere you can get an
> error, and adding destroy() calls everywhere,  you do not gain much
> clarity, it's better to seek other ways.
>
> I really do not use os.exit, all my lua is embeded, but it should
> not be to difficult to dessign your objects in a way that they are
> destroyed ( just overrwrite os.exit, add some atexit() support, do
> something like having and idempotent destroy method, make your
> objects register in a table for atexit, make __close call destroy
> and deregister them, maybe use some handle idiom for easier
> clearing, easy-peasy).
>
>> The whole point of the mechanism, as I see it, is that you could do
>> RAII in a much simpler way. But if it introduces other, subtler,
>> issues then I'm beginning to question its usefulness.
>
> Use it like try-with-resources, for things which need to be cleaned
> when the program errors and unwinds, keep using resources which need
> to be properly cleaned on os.exit in other way, it doesn't have to
> solve all the problems, and it manages to solve some of them.
>
>> To be clearer: now (Lua 5.3) I can do RAII using more discipline
>> and sprinkling error handling code in key parts of the code. This
>> approach adds clutter to the code but the error handling code is
>> explicit and clear.
>
> That is because it is not RAII, the point of RAII is AVOIDING the
> sprinkling.
>

You are right. My bad. I didn't mean to write RAII. I used RAII here to
mean "correct release of resources even on error conditions".

>> In 5.4 I could reduce clutter by using TBC vars, but then I must
>> add some extra checks or mechanism to handle those exceptional
>> cases when TBC vars mechanism "fails" (and coroutine users are
>> especially worse off, it seems). Moreover, the new syntax is not
>> that great.
>
> You do not need to add. Move what you want to TBC, keep the rest as
> is.
>
>> All in all, I'm starting to think that the mechanism is not
>> well-polished yet. At least not enough to warrant a shift of
>> paradigm. But maybe I'm missing something.
>
> IMNSHO, you'll never be able to have RAII in a language like lua,
> Java, etc.. where ( ignoring lua strings, number, ... ) everything
> is a pointer in disguise and you cannot have value objects and it is
> relying on garbage collection [...]

Again, here I lost you. What's the problem about objects not being on
the stack but being references? TBC vars have been created just for
that. I agree the semantics is not the same as C++, but the general idea
is the same:

1. (C++): Allocate a local object X of a class with a CTOR that
allocates the resources you need.
1. (Lua): Create a local TB vars that refers to an object X with a
__close metamethod that allocates the resource you need.


2. (C++): use the resources accessing them through helper methods of X.
2. (Lua): use the resources accessing them through helper methods of X.

3. (C++): when you exit the scope where the resources are used using X,
stack is unwound, and X destructor is called, which correctly releases
the resources.
3. (Lua): when you exit the scope where the resources are used using X,
X's __close metamethod is called, which correctly releases the resources.

The difference is that C++ keeps track at compile time of which
destructors have to be called for object on the stack. Lua keeps track
at runtime of which stack slot are associated with TBC vars and calls
the __close metamethod of the object the var is referring to, which
can't be changed since they are <const>. But the pattern is the same.
The key poing being /deterministic/ cleanup instead of relying on
finalizers, which are executed "randomly" under GC control.

With TBC vars GC doesn't come into play, if I got it right.
Am I missing something?

>
> I've found garbage collection is typically great when you must do
> complex memory only structures, not that much elsewhere. I've till
> now been fortunate to be able to put this kind of things in arenas
> when I have to use them in C++, so no need to use Boehms or similar
> things. But for "heavy" resources, I rely on RAII, potentially with
> shared_ptrs, and I never, never, use std::exit from deep down, I
> just raise my exit_exception and catch it in main().
>
> Francisco Olarte.
>

-- Lorenzo
Reply | Threaded
Open this post in threaded view
|

Re: RFE: refman - adding mention of os.exit not closing variables with default arguments.

Francisco Olarte
Lorenzo:

On Wed, Jun 24, 2020 at 5:29 PM Lorenzo Donati
<[hidden email]> wrote:
...
> I'm no big expert of multithreading, either cooperative or preemptive,
> and I use C only to a limited extent.
> However I thought C's `exit` was the right way to exit a program
> (possibly together with `atexit` mechanism) if returning from main is
> too cumbersome.

Well, C exit jumps out of the call stack and kills your process.
stdlib manages to have your FILE * flushed, OS frees your mem and
handles. But if you need to do cleanup things like status saving, you
need atexit. You use that when returning from main is cumbersome, but
normally you have some other methods ( like killing an event loop or
some thing like that )

> At least C's standard seems to think so:
> file:///G:/root/main/core/C-CPP/doc/standards/cppreference.com/en/c/program/exit.html

You notice this is a local reference ( windowish? ) to a file in your
machine? ( I now cpp reference tells that, and know it is a somehow
non leaking way to exit, but in complex programas your transactions
will not get commited and things like dat ).

> I admit I've no experience with C big programs (say 5kLOC+; especially
> multithreaded/GUI ones), so I don't know what's current practice.

It depends on the libs you use, many give you some ways for easier
exiting, or register their own cleanup handlers, but exit is really
hard.


> > Look at the RAII poster child. C++. If you call os.exit it makes
> > some cleanups, but does not destroy automatic storage duration
> > objects ( which IMHO are the most similar to lua toclose ones ). It
> > closes some things, destroys automatics ( which may be similar to lua
> > __gc on globals, which IIRC is invoked ), and goes out.
> You are right! I just checked here:
> file:///G:/root/main/core/C-CPP/doc/standards/cppreference.com/en/cpp/utility/program/exit.html
> In C++ std::exit doesn't unwind the stack. I didn't know that!

exit never unwinds the stack its purpose is exactly exiting without unwinding.

> About year 2000 I had been using C++ intensively for about 3 years  and
> never stumbled on that info. Mostly was number-crunching, data analysis
> and offline simulation of TLC systems, so no interactive stuff.

Those are generally easy on the resource side, I worked on those
things a lot and you normally could even abort.

> > And toclose is not RAII, is try-with-resources, which AFAIK is what
> > is normally done to approach RAII in garbage-collected languages,
> > when realizing not all resources are as easy as memory.
> I don't follow you here. Why do you say it is not RAII? To me RAIIin C++
> is when you create a local object whose ctor allocates resources and
> whose destructor deallocates them when the object goes out of scope or
> the stack unwinds (returns, exceptions), thanks to C++ semantics.

RAII in C++ means the moment a constructor get executed the destructor
will be called unless you've taken the precaution of using new.

You do not have to store the var in an especial construct, you can
also create it in a temporary expression, i.e., in C++ you can do
something like

db_connection(params).execute_query(my_query).get_resultset().print();

And the intermediate db_connection, query, resultset objects, if
appropiately managed, will vanish and be properly closed ( for these
things you need move semantics or advanced classes, in old school C++
you would propably have something like

resultset(query(connection(params),my_query)).print()

The thing is you do not have special toclose because you have value
objects, while in java/lua all your objects go into the free store,
you always use somethign like new. So you need syntax to manage them (
try/finally is not a language feature, it is a neccessity, java needs
that, C++ does not. Try with resources is try-finally on steroids. Lua
to-close is very similar to Java try with resources )

In C++ you can create a db connection or get a pointer to an allocated
one. How do you it dictates your management style. In Java or Lua you
always get a pointer, so you need toclose and try with resources if
you want local lifetime management.

> Apart from not there being an exact parallel, because of different
> mechanisms and semantics, it seems Lua TBC vars are just a way to
> conceptually do RAII, i.e. deterministic automatic release of resources.

They are a safer and more elegant way to do a try-finally.

> The key point being the resource release handler (__close metamethod) is
> defined upfront when you acquire the resource, and not lexically scoped
> as a catch block at the end.

That's just sugar, but yes, and sugar is important.

> Am I missing something about what you are saying?

Once you have a destruction and an initialization method, call the
dtor __gc, __close, close, destroy or ~object, you can strictly say
you are doing RAII.

But using RAII in languages without proper scope tracking of values is
a PITA. You get help with TWR, toclose, but you do not get the same
easy code as you have with C++. In C++ you have to explicitly ask for
a dynamically allocated object via new ( or indirectly ) to be able to
foget destroying it. In Java/lua you have to explicitly mark it via
toclose or rely on the garbage collector with its problems.

And the problem with exit is orthogonal to this. It's Thor's retun hammer.

> You are right. My bad. I didn't mean to write RAII. I used RAII here to
> mean "correct release of resources even on error conditions".

That's correct code. It can be done in nearly any language (some of
them bomb out, but they are rare)
....
> Again, here I lost you. What's the problem about objects not being on
> the stack but being references? TBC vars have been created just for
> that. I agree the semantics is not the same as C++, but the general idea
> is the same:
> 1. (C++): Allocate a local object X of a class with a CTOR that
> allocates the resources you need.
> 1. (Lua): Create a local TB vars that refers to an object X with a
> __close metamethod that allocates the resource you need.

C++ - declare the var, do not use new.
Lua - remember to use a to close.

What happens if you want the resource holding object just to pass it
to a method?

C++ - create it in the parm. list as you would for a var.
Lua - Create a to close, and potentially a block, for the sole purpose
of holding the reg around the call.

> With TBC vars GC doesn't come into play, if I got it right.
> Am I missing something?

No, you can do RAII in any language, it's just painful in some of
them, because you need to explicitly track them.
You cannot put <toclose> in a parameter list.
Also, you have no easy way to see how to track ownership ( i.e.,
imagine you need something fro share ownership. It can be done easily
passing shared__ptrs by value in C++, but althoguh shared_ptrs can be
easily done in lua via some ref-counting or linking and creative
__index manipulation they soon become cumbersome. You can get cleaner
code with toclose, but it is very limited because yo have to mark
every ownership point with local <toclose>.

The problem is I associate RAII with proper value objects, and I mix
it a bit. Nearly everything which you can do in C++ can be done with
creative use of toclose and scopes, but it will get complicated. I.e.,
to do something like filecopy(ofstream("dest"),ofstream("source"))"
you'll need to add a couple of local-toclose for the temporaries, plus
a do-end to avoid namespace pollution. It's not just variable lifetime
what the C++ compiler tracks. But you're right, something like FILE*
=fopen, very similar to lua io, is RAII too. The problem is I normally
want proper value lifetime tracking, which is hard to do without
values.

Francisco Olarte.
Reply | Threaded
Open this post in threaded view
|

Re: RFE: refman - adding mention of os.exit not closing variables with default arguments.

Gé Weijers
In reply to this post by Roberto Ierusalimschy
On Wed, Jun 24, 2020 at 7:26 AM Roberto Ierusalimschy <[hidden email]> wrote:
> Having finalizers to back up __close solves all these cases, where an
> object goes out of existence without going out of scope. The manual
> hints about this, but it could be more emphatic.
>

This will release resources such as file handles just fine. The nice thing about __close is that it's pretty deterministic, like a C++ destructor, the C# using() block, or the Golang "defer" statement. You know when it will be called, whereas __gc can be called whenever the VM or a C routine allocates memory, so you have to be very careful about touching anything outside of the object being garbage collected. The deterministic behavior makes __close useful for more than closing file handles and such.

> The goal of <close> inside coroutines + coroutine.close is to allow
> orderly kills of threads/coroutines in Lua.

It took me a little while to figure out a strategy that allows that orderly kill of any coroutine left behind by an error in a program.
Perhaps this is a subject for a new chapter in an updated edition of you Lua book😀

Thanks,


--
Reply | Threaded
Open this post in threaded view
|

Re: RFE: refman - adding mention of os.exit not closing variables with default arguments.

Sean Conner
In reply to this post by Lorenzo Donati-3
It was thus said that the Great Lorenzo Donati once stated:
>
> However I thought C's `exit` was the right way to exit a program
> (possibly together with `atexit` mechanism) if returning from main is
> too cumbersome.

  In C89, you have exit().  In C99, you have exit() and _Exit().  In C11,
you have exit(), _Exit() and quick_exit().  Here's what each does.

        exit()

                Run each function registered by atexit() in reverser order
                of registration (at least 32 such functions are supported).

                Write buffered data in open files, and close all open files.
                All files created with tmpfile() are removed.

                Return the status code to the operating system.

        _Exit()

                Return the status code to the operating sytem.

        quick_exit()

                Run each function registered by at_quick_exit() in reverse
                order of registration (at least 32 such functions are
                supported).

                Return the status code to the operating system.

  _Exit() does the least amount of work, exit() the most.  The only
resources that are cleaned up by the C standard library are files (and only
in the case of exit()); memory is not mentioned at all, and in my
experience, on non-virtual memory systems (like MS-DOS or AmigaOS 1.x) this
means it is leaked.

  Also, calling exit() is the same a doing a return from main---that is,
when you return from main, the process is as if you called exit().

  Why one would call quick_exit() or _Exit() is highly dependent upon the
operating system and nature of the program.

  -spc
Reply | Threaded
Open this post in threaded view
|

Re: RFE: refman - adding mention of os.exit not closing variables with default arguments.

Lorenzo Donati-3
In reply to this post by Francisco Olarte
On 24/06/2020 20:09, Francisco Olarte wrote:

> Lorenzo:
>
> On Wed, Jun 24, 2020 at 5:29 PM Lorenzo Donati
> <[hidden email]> wrote:
> ...
>> I'm no big expert of multithreading, either cooperative or preemptive,
>> and I use C only to a limited extent.
>> However I thought C's `exit` was the right way to exit a program
>> (possibly together with `atexit` mechanism) if returning from main is
>> too cumbersome.
>
> Well, C exit jumps out of the call stack and kills your process.
> stdlib manages to have your FILE * flushed, OS frees your mem and
> handles. But if you need to do cleanup things like status saving, you
> need atexit. You use that when returning from main is cumbersome, but
> normally you have some other methods ( like killing an event loop or
> some thing like that )
>
>> At least C's standard seems to think so:
>> file:///G:/root/main/core/C-CPP/doc/standards/cppreference.com/en/c/program/exit.html
>
> You notice this is a local reference ( windowish? ) to a file in your
> machine? ( I now cpp reference tells that, and know it is a somehow
> non leaking way to exit, but in complex programas your transactions
> will not get commited and things like dat ).
>

(*doh*)! I just had two browser tabs open (together with a dozen more),
one with a local copy of cppreference.com and the other with the online
site. It seems I copied the address from the wrong tab. Sorry!
For the archive, the correct link was meant to be:

https://en.cppreference.com/w/c/program/exit


>> I admit I've no experience with C big programs (say 5kLOC+; especially
>> multithreaded/GUI ones), so I don't know what's current practice.
>
> It depends on the libs you use, many give you some ways for easier
> exiting, or register their own cleanup handlers, but exit is really
> hard.
>
>
>>> Look at the RAII poster child. C++. If you call os.exit it makes
>>> some cleanups, but does not destroy automatic storage duration
>>> objects ( which IMHO are the most similar to lua toclose ones ). It
>>> closes some things, destroys automatics ( which may be similar to lua
>>> __gc on globals, which IIRC is invoked ), and goes out.
>> You are right! I just checked here:
>> file:///G:/root/main/core/C-CPP/doc/standards/cppreference.com/en/cpp/utility/program/exit.html
>> In C++ std::exit doesn't unwind the stack. I didn't know that!
>

Again, copied from wrong window. Correct link was meant to be:

https://en.cppreference.com/w/cpp/utility/program/exit

> exit never unwinds the stack its purpose is exactly exiting without unwinding.
>
>> About year 2000 I had been using C++ intensively for about 3 years  and
>> never stumbled on that info. Mostly was number-crunching, data analysis
>> and offline simulation of TLC systems, so no interactive stuff.
>
> Those are generally easy on the resource side, I worked on those
> things a lot and you normally could even abort.
>
>>> And toclose is not RAII, is try-with-resources, which AFAIK is what
>>> is normally done to approach RAII in garbage-collected languages,
>>> when realizing not all resources are as easy as memory.
>> I don't follow you here. Why do you say it is not RAII? To me RAIIin C++
>> is when you create a local object whose ctor allocates resources and
>> whose destructor deallocates them when the object goes out of scope or
>> the stack unwinds (returns, exceptions), thanks to C++ semantics.
>
> RAII in C++ means the moment a constructor get executed the destructor
> will be called unless you've taken the precaution of using new.
>
> You do not have to store the var in an especial construct, you can
> also create it in a temporary expression, i.e., in C++ you can do
> something like
>
> db_connection(params).execute_query(my_query).get_resultset().print();
>
> And the intermediate db_connection, query, resultset objects, if
> appropiately managed, will vanish and be properly closed ( for these
> things you need move semantics or advanced classes, in old school C++
> you would propably have something like
>
> resultset(query(connection(params),my_query)).print()
>

OK. That's also what I know. As long as an object is on the stack, the
dtor will be called and manage resources tear down.


> The thing is you do not have special toclose because you have value
> objects, while in java/lua all your objects go into the free store,
> you always use somethign like new. So you need syntax to manage them (
> try/finally is not a language feature, it is a neccessity, java needs
> that, C++ does not. Try with resources is try-finally on steroids. Lua
> to-close is very similar to Java try with resources )
>
> In C++ you can create a db connection or get a pointer to an allocated
> one. How do you it dictates your management style. In Java or Lua you
> always get a pointer, so you need toclose and try with resources if
> you want local lifetime management.
>
>> Apart from not there being an exact parallel, because of different
>> mechanisms and semantics, it seems Lua TBC vars are just a way to
>> conceptually do RAII, i.e. deterministic automatic release of resources.
>
> They are a safer and more elegant way to do a try-finally.
>
>> The key point being the resource release handler (__close metamethod) is
>> defined upfront when you acquire the resource, and not lexically scoped
>> as a catch block at the end.
>
> That's just sugar, but yes, and sugar is important.
>
>> Am I missing something about what you are saying?
>
> Once you have a destruction and an initialization method, call the
> dtor __gc, __close, close, destroy or ~object, you can strictly say
> you are doing RAII.
>
> But using RAII in languages without proper scope tracking of values is
> a PITA. You get help with TWR, toclose, but you do not get the same
> easy code as you have with C++. In C++ you have to explicitly ask for
> a dynamically allocated object via new ( or indirectly ) to be able to
> foget destroying it. In Java/lua you have to explicitly mark it via
> toclose or rely on the garbage collector with its problems.
>
> And the problem with exit is orthogonal to this. It's Thor's retun hammer.
>
>> You are right. My bad. I didn't mean to write RAII. I used RAII here to
>> mean "correct release of resources even on error conditions".
>
> That's correct code. It can be done in nearly any language (some of
> them bomb out, but they are rare)
> ....
>> Again, here I lost you. What's the problem about objects not being on
>> the stack but being references? TBC vars have been created just for
>> that. I agree the semantics is not the same as C++, but the general idea
>> is the same:
>> 1. (C++): Allocate a local object X of a class with a CTOR that
>> allocates the resources you need.
>> 1. (Lua): Create a local TB vars that refers to an object X with a
>> __close metamethod that allocates the resource you need.
>
> C++ - declare the var, do not use new.
> Lua - remember to use a to close.
>
> What happens if you want the resource holding object just to pass it
> to a method?
>

Now I think I'm getting your point (I hope).

The problem you seem to point out is resource ownership management. C++
objects, especially in newer versions with move semantics, can own
resources and be passed around if needed. So the lifetime of a resource
is linked to the lifetime of a specific object. If that object is on the
stack, exiting the scope of that object will cause its deterministic
destruction, and consequent automatic deterministic resource release.

In Lua, on the contrary, __toclose will be called only if the object
(the resource handler, we may say) is stored in a TBC variable, so the
cleanup is not linked to the lifetime of the object, but the scope of
the TBC vars.

This precludes passing resource handlers around AND having deterministic
cleanup, since passing them around will create aliased references to the
handlers, so the risk of __close being called more than one time
(whereas an object dtor in C++ cannot be called automatically more than
one time if the object is on the stack).
I can somewhat see you can probably get around some of these problems by
making __close keep track of how many times it has been called, and
maybe interact with __gc.

Mmmh. I now understand I can't get the whole picture with this __toclose
feature. I need to see more examples on how it could be used besides the
trivial case:

do
    local h <toclose> = allocate_resource(...)
    ...
    do_stuff_with_resource(h)
    ...
end -- here __toclose for h will be called and cleanup resource

I have some difficulties now to picture in my head more difficult
scenarios. Besides missing deterministic execution, __gc seems far more
easy to reason about.

> C++ - create it in the parm. list as you would for a var.
> Lua - Create a to close, and potentially a block, for the sole purpose
> of holding the reg around the call.
>
>> With TBC vars GC doesn't come into play, if I got it right.
>> Am I missing something?
>
> No, you can do RAII in any language, it's just painful in some of
> them, because you need to explicitly track them.
> You cannot put <toclose> in a parameter list.
> Also, you have no easy way to see how to track ownership ( i.e.,
> imagine you need something fro share ownership. It can be done easily
> passing shared__ptrs by value in C++, but althoguh shared_ptrs can be
> easily done in lua via some ref-counting or linking and creative
> __index manipulation they soon become cumbersome. You can get cleaner
> code with toclose, but it is very limited because yo have to mark
> every ownership point with local <toclose>.
>
> The problem is I associate RAII with proper value objects, and I mix
> it a bit. Nearly everything which you can do in C++ can be done with
> creative use of toclose and scopes, but it will get complicated. I.e.,
> to do something like filecopy(ofstream("dest"),ofstream("source"))"
> you'll need to add a couple of local-toclose for the temporaries, plus
> a do-end to avoid namespace pollution. It's not just variable lifetime
> what the C++ compiler tracks. But you're right, something like FILE*
> =fopen, very similar to lua io, is RAII too. The problem is I normally
> want proper value lifetime tracking, which is hard to do without
> values.
>
> Francisco Olarte.
>

Thanks for the explanation!

-- Lorenzo
Reply | Threaded
Open this post in threaded view
|

Re: RFE: refman - adding mention of os.exit not closing variables with default arguments.

Lorenzo Donati-3
In reply to this post by Francisco Olarte
On 24/06/2020 20:09, Francisco Olarte wrote:

> Lorenzo:
>
> On Wed, Jun 24, 2020 at 5:29 PM Lorenzo Donati
> <[hidden email]> wrote:
> ...
>> I'm no big expert of multithreading, either cooperative or preemptive,
>> and I use C only to a limited extent.
>> However I thought C's `exit` was the right way to exit a program
>> (possibly together with `atexit` mechanism) if returning from main is
>> too cumbersome.
>
> Well, C exit jumps out of the call stack and kills your process.
> stdlib manages to have your FILE * flushed, OS frees your mem and
> handles. But if you need to do cleanup things like status saving, you
> need atexit. You use that when returning from main is cumbersome, but
> normally you have some other methods ( like killing an event loop or
> some thing like that )
>
>> At least C's standard seems to think so:
>> file:///G:/root/main/core/C-CPP/doc/standards/cppreference.com/en/c/program/exit.html
>
> You notice this is a local reference ( windowish? ) to a file in your
> machine? ( I now cpp reference tells that, and know it is a somehow
> non leaking way to exit, but in complex programas your transactions
> will not get commited and things like dat ).
>

(*doh*)! I just had two browser tabs open (together with a dozen more),
one with a local copy of cppreference.com and the other with the online
site. It seems I copied the address from the wrong tab. Sorry!

For the archive, the correct link was meant to be:

https://en.cppreference.com/w/c/program/exit


>> I admit I've no experience with C big programs (say 5kLOC+; especially
>> multithreaded/GUI ones), so I don't know what's current practice.
>
> It depends on the libs you use, many give you some ways for easier
> exiting, or register their own cleanup handlers, but exit is really
> hard.
>
>
>>> Look at the RAII poster child. C++. If you call os.exit it makes
>>> some cleanups, but does not destroy automatic storage duration
>>> objects ( which IMHO are the most similar to lua toclose ones ). It
>>> closes some things, destroys automatics ( which may be similar to lua
>>> __gc on globals, which IIRC is invoked ), and goes out.
>> You are right! I just checked here:
>> file:///G:/root/main/core/C-CPP/doc/standards/cppreference.com/en/cpp/utility/program/exit.html
>> In C++ std::exit doesn't unwind the stack. I didn't know that!
>

Again, copied from wrong window. Correct link was meant to be:

https://en.cppreference.com/w/cpp/utility/program/exit

> exit never unwinds the stack its purpose is exactly exiting without unwinding.
>
>> About year 2000 I had been using C++ intensively for about 3 years  and
>> never stumbled on that info. Mostly was number-crunching, data analysis
>> and offline simulation of TLC systems, so no interactive stuff.
>
> Those are generally easy on the resource side, I worked on those
> things a lot and you normally could even abort.
>
>>> And toclose is not RAII, is try-with-resources, which AFAIK is what
>>> is normally done to approach RAII in garbage-collected languages,
>>> when realizing not all resources are as easy as memory.
>> I don't follow you here. Why do you say it is not RAII? To me RAIIin C++
>> is when you create a local object whose ctor allocates resources and
>> whose destructor deallocates them when the object goes out of scope or
>> the stack unwinds (returns, exceptions), thanks to C++ semantics.
>
> RAII in C++ means the moment a constructor get executed the destructor
> will be called unless you've taken the precaution of using new.
>
> You do not have to store the var in an especial construct, you can
> also create it in a temporary expression, i.e., in C++ you can do
> something like
>
> db_connection(params).execute_query(my_query).get_resultset().print();
>
> And the intermediate db_connection, query, resultset objects, if
> appropiately managed, will vanish and be properly closed ( for these
> things you need move semantics or advanced classes, in old school C++
> you would propably have something like
>
> resultset(query(connection(params),my_query)).print()
>

OK. That's also what I know. As long as an object is on the stack, the
dtor will be called and manage resources tear down.


> The thing is you do not have special toclose because you have value
> objects, while in java/lua all your objects go into the free store,
> you always use somethign like new. So you need syntax to manage them (
> try/finally is not a language feature, it is a neccessity, java needs
> that, C++ does not. Try with resources is try-finally on steroids. Lua
> to-close is very similar to Java try with resources )
>
> In C++ you can create a db connection or get a pointer to an allocated
> one. How do you it dictates your management style. In Java or Lua you
> always get a pointer, so you need toclose and try with resources if
> you want local lifetime management.
>
>> Apart from not there being an exact parallel, because of different
>> mechanisms and semantics, it seems Lua TBC vars are just a way to
>> conceptually do RAII, i.e. deterministic automatic release of resources.
>
> They are a safer and more elegant way to do a try-finally.
>
>> The key point being the resource release handler (__close metamethod) is
>> defined upfront when you acquire the resource, and not lexically scoped
>> as a catch block at the end.
>
> That's just sugar, but yes, and sugar is important.
>
>> Am I missing something about what you are saying?
>
> Once you have a destruction and an initialization method, call the
> dtor __gc, __close, close, destroy or ~object, you can strictly say
> you are doing RAII.
>
> But using RAII in languages without proper scope tracking of values is
> a PITA. You get help with TWR, toclose, but you do not get the same
> easy code as you have with C++. In C++ you have to explicitly ask for
> a dynamically allocated object via new ( or indirectly ) to be able to
> foget destroying it. In Java/lua you have to explicitly mark it via
> toclose or rely on the garbage collector with its problems.
>
> And the problem with exit is orthogonal to this. It's Thor's retun hammer.
>
>> You are right. My bad. I didn't mean to write RAII. I used RAII here to
>> mean "correct release of resources even on error conditions".
>
> That's correct code. It can be done in nearly any language (some of
> them bomb out, but they are rare)
> ....
>> Again, here I lost you. What's the problem about objects not being on
>> the stack but being references? TBC vars have been created just for
>> that. I agree the semantics is not the same as C++, but the general idea
>> is the same:
>> 1. (C++): Allocate a local object X of a class with a CTOR that
>> allocates the resources you need.
>> 1. (Lua): Create a local TB vars that refers to an object X with a
>> __close metamethod that allocates the resource you need.
>
> C++ - declare the var, do not use new.
> Lua - remember to use a to close.
>
> What happens if you want the resource holding object just to pass it
> to a method?
>

Now I think I'm getting your point (I hope).

The problem you seem to point out is resource ownership management. C++
objects, especially in newer versions with move semantics, can own
resources and be passed around if needed. So the lifetime of a resource
is linked to the lifetime of a specific object. If that object is on the
stack, exiting the scope of that object will cause its deterministic
destruction, and consequent automatic deterministic resource release.

In Lua, on the contrary, __close will be called only if the object (the
resource handler, we may say) is stored in a TBC variable, so the
cleanup is not linked to the lifetime of the object, but the scope of
the TBC vars.

This precludes passing resource handlers around AND having deterministic
cleanup, since passing them around will create aliased references to the
handlers, so the risk of __close being called more than one time
(whereas an object dtor in C++ cannot be called automatically more than
one time if the object is on the stack).
I can somewhat see you can probably get around some of these problems by
making __close keep track of how many times it has been called, and
maybe interact with __gc.

Mmmh. I now understand I can't still get the whole picture with this
feature. I need to see more examples on how it could be used besides the
trivial case:

do
    local h <close> = allocate_resource(...)
    ...
    do_stuff_with_resource(h)
    ...
end -- here __close for h will be called, freeing the resource

I have some difficulties now to picture in my head more difficult
scenarios. Besides missing deterministic execution, __gc seems far more
easy to reason about.

> C++ - create it in the parm. list as you would for a var.
> Lua - Create a to close, and potentially a block, for the sole purpose
> of holding the reg around the call.
>
>> With TBC vars GC doesn't come into play, if I got it right.
>> Am I missing something?
>
> No, you can do RAII in any language, it's just painful in some of
> them, because you need to explicitly track them.
> You cannot put <toclose> in a parameter list.
> Also, you have no easy way to see how to track ownership ( i.e.,
> imagine you need something fro share ownership. It can be done easily
> passing shared__ptrs by value in C++, but althoguh shared_ptrs can be
> easily done in lua via some ref-counting or linking and creative
> __index manipulation they soon become cumbersome. You can get cleaner
> code with toclose, but it is very limited because yo have to mark
> every ownership point with local <toclose>.
>
> The problem is I associate RAII with proper value objects, and I mix
> it a bit. Nearly everything which you can do in C++ can be done with
> creative use of toclose and scopes, but it will get complicated. I.e.,
> to do something like filecopy(ofstream("dest"),ofstream("source"))"
> you'll need to add a couple of local-toclose for the temporaries, plus
> a do-end to avoid namespace pollution. It's not just variable lifetime
> what the C++ compiler tracks. But you're right, something like FILE*
> =fopen, very similar to lua io, is RAII too. The problem is I normally
> want proper value lifetime tracking, which is hard to do without
> values.
>
> Francisco Olarte.
>

Thanks for the explanation!

-- Lorenzo
Reply | Threaded
Open this post in threaded view
|

Re: RFE: refman - adding mention of os.exit not closing variables with default arguments.

Lorenzo Donati-3
In reply to this post by Sean Conner
On 24/06/2020 23:00, Sean Conner wrote:

> It was thus said that the Great Lorenzo Donati once stated:
>>
>> However I thought C's `exit` was the right way to exit a program
>> (possibly together with `atexit` mechanism) if returning from main is
>> too cumbersome.
>
>   In C89, you have exit().  In C99, you have exit() and _Exit().  In C11,
> you have exit(), _Exit() and quick_exit().  Here's what each does.
>
> exit()
>
> Run each function registered by atexit() in reverser order
> of registration (at least 32 such functions are supported).
>
> Write buffered data in open files, and close all open files.
> All files created with tmpfile() are removed.
>
> Return the status code to the operating system.
>
> _Exit()
>
> Return the status code to the operating sytem.
>
> quick_exit()
>
> Run each function registered by at_quick_exit() in reverse
> order of registration (at least 32 such functions are
> supported).
>
> Return the status code to the operating system.
>
>   _Exit() does the least amount of work, exit() the most.  The only
> resources that are cleaned up by the C standard library are files (and only
> in the case of exit()); memory is not mentioned at all, and in my
> experience, on non-virtual memory systems (like MS-DOS or AmigaOS 1.x) this
> means it is leaked.
>
>   Also, calling exit() is the same a doing a return from main---that is,
> when you return from main, the process is as if you called exit().
>
>   Why one would call quick_exit() or _Exit() is highly dependent upon the
> operating system and nature of the program.
>
>   -spc
>
Yep. Thanks. I did know the existance of those functions. After the the
brief discussion with Francisco Olarte, I wonder whether all those
standard functions to perform exit are really useful.

It seems that exiting an application is quite complicated and requires
custom code that relies both on the os and the specific application.

I wonder how many people really use all this C standard facilities
instead of writing their own exit function.

-- Lorenzo





Reply | Threaded
Open this post in threaded view
|

Re: RFE: refman - adding mention of os.exit not closing variables with default arguments.

Coda Highland

On Thu, Jun 25, 2020 at 11:30 AM Lorenzo Donati <[hidden email]> wrote:
Yep. Thanks. I did know the existance of those functions. After the the
brief discussion with Francisco Olarte, I wonder whether all those
standard functions to perform exit are really useful.

It seems that exiting an application is quite complicated and requires
custom code that relies both on the os and the specific application.

I wonder how many people really use all this C standard facilities
instead of writing their own exit function.

-- Lorenzo

In general, the "standard" facilities are only useful as an emergency abort -- when your application has entered a state that can't be recovered from, you can use the exit() family of functions to at least try to terminate without making too much of a mess. Things that absolutely must be properly handled even during an emergency abort can then provide an atexit callback. Otherwise, you should always try to end the program by setting things up so that control flow bubbles up to main(), which then returns as normal. (As mentioned, typically this is done by flagging the event loop to stop looping.)

That said, most people don't write their own either. They use one of the countless event handling libraries (glib, libuv, the Windows message pump, etc.) to abstract out the nitty-gritty details.

In C++ you can throw an exception, which will unwind the stack properly on the way out, and if main() doesn't catch it, then it calls abort(). C has no standard equivalent.

/s/ Adam
Reply | Threaded
Open this post in threaded view
|

Re: RFE: refman - adding mention of os.exit not closing variables with default arguments.

Roberto Ierusalimschy
In reply to this post by Gé Weijers
> This will release resources such as file handles just fine. The nice thing
> about __close is that it's pretty deterministic, like a C++ destructor, the
> C# using() block, or the Golang "defer" statement. You know when it will be
> called, whereas __gc can be called whenever the VM or a C routine allocates
> memory, so you have to be very careful about touching anything outside of
> the object being garbage collected. The deterministic behavior makes
> __close useful for more than closing file handles and such.

I am not sure I see how calling __close in the cases we are discussing
is different from what you described. Calling __close when exiting a
program is also pretty much "non deterministic", as a program can call
os.exit anywhere. Moreover, there is no intrinsic order we could use to
close coroutines, so all these objects being closed together should also
be careful about what has been closed already. Dead coroutines pose
similar problems.

Probably it is not by chance that C++ does not call destructors for
objects in the stack when os.exit is called.

-- Roberto
Reply | Threaded
Open this post in threaded view
|

Re: RFE: refman - adding mention of os.exit not closing variables with default arguments.

Francisco Olarte
Roberto:

On Fri, Jun 26, 2020 at 7:56 PM Roberto Ierusalimschy
<[hidden email]> wrote:
> Probably it is not by chance that C++ does not call destructors for
> objects in the stack when os.exit is called.

For what I've been reading in the thread many people are not familiar
with C/C++ exit(3), so they seem to think exit is the recommended way
to get of the program.

One think, on the parallels between c** std:exit / lua
os.exit(x,true), i.e., closing the state.

We do not have atexit, but this is easy to do with a module and a
proxy, so not really needed.

C++, IIRC, destroys globals when exit() is called. os.exit(x,true)
closes the states, which is documented as "Destroys all objects in the
given Lua state (calling the corresponding garbage-collection
metamethods, if any)". I've always assumed this means ALL, I mean,
including objects referenced by local vars in modules, main thread
call stack and potentially suspended threads call stack. Is this
correct? ( I do not use exit but use lua_close profusely in embeded,
and have had not problem with missing __gc calls, but I'd like to know
if there is some corner case which I've not seen in the docs ( no
loops, resurrections or fancy things done in my code, but I'm in the
middle of a project which may call for destroying with suspended
threads, which I normally never do ) )

( note, I'm talking __gc, not __close, but my objects normally end up
calling and idempotent close() method anyway, which I call in the
normal flow, pretend to catch via toclose with 5.4 ( and avoid some
ugly xpcalls ), and will rely in __gc calling just for some serious
errors which warrant state destruction ).

Francisco Olarte.
Reply | Threaded
Open this post in threaded view
|

Re: RFE: refman - adding mention of os.exit not closing variables with default arguments.

Roberto Ierusalimschy
> [...] os.exit(x,true)
> closes the states, which is documented as "Destroys all objects in the
> given Lua state (calling the corresponding garbage-collection
> metamethods, if any)". I've always assumed this means ALL, I mean,
> including objects referenced by local vars in modules, main thread
> call stack and potentially suspended threads call stack. Is this
> correct?

The manual says "all objects", so your question boils down to whether
all means ALL. Yes, all means ALL, regardless capitalization.

-- Roberto
Reply | Threaded
Open this post in threaded view
|

Re: RFE: refman - adding mention of os.exit not closing variables with default arguments.

Lorenzo Donati-3
In reply to this post by Coda Highland
On 25/06/2020 19:56, Coda Highland wrote:

> On Thu, Jun 25, 2020 at 11:30 AM Lorenzo Donati <[hidden email]>
> wrote:
>
>> Yep. Thanks. I did know the existance of those functions. After the the
>> brief discussion with Francisco Olarte, I wonder whether all those
>> standard functions to perform exit are really useful.
>>
>> It seems that exiting an application is quite complicated and requires
>> custom code that relies both on the os and the specific application.
>>
>> I wonder how many people really use all this C standard facilities
>> instead of writing their own exit function.
>>
>> -- Lorenzo
>>
>
> In general, the "standard" facilities are only useful as an emergency abort
> -- when your application has entered a state that can't be recovered from,
> you can use the exit() family of functions to at least try to terminate
> without making too much of a mess. Things that absolutely must be properly
> handled even during an emergency abort can then provide an atexit callback.
> Otherwise, you should always try to end the program by setting things up so
> that control flow bubbles up to main(), which then returns as normal. (As
> mentioned, typically this is done by flagging the event loop to stop
> looping.)
>
> That said, most people don't write their own either. They use one of the
> countless event handling libraries (glib, libuv, the Windows message pump,
> etc.) to abstract out the nitty-gritty details.

Thanks for the information. I've seen this pattern in the past, but now
you confirm it's common and, as I see, recommended practice.

>
> In C++ you can throw an exception, which will unwind the stack properly on
> the way out, and if main() doesn't catch it, then it calls abort(). C has
> no standard equivalent.
>
> /s/ Adam
>

Mmmh. So does this mean that C applications have quite a hard time when
they need to quit in a clean way when executing some code with a deep
call stack, right?

Thanks,

-- Lorenzo
Reply | Threaded
Open this post in threaded view
|

Re: RFE: refman - adding mention of os.exit not closing variables with default arguments.

Gé Weijers


On Mon, Jun 29, 2020 at 11:32 AM Lorenzo Donati <[hidden email]> wrote:
>
> In C++ you can throw an exception, which will unwind the stack properly on
> the way out, and if main() doesn't catch it, then it calls abort(). C has
> no standard equivalent.
>
> /s/ Adam
>

Mmmh. So does this mean that C applications have quite a hard time when
they need to quit in a clean way when executing some code with a deep
call stack, right?


Exception handling is a double edged sword, you can cleanly unwind the stack but it may leave your program in a state you can't continue with, because your data structures are left in a state that's undefined. It can also introduce space leaks unless you follow certain rules. Experienced C++ programmers can use the language to write great code, but you tend to make a lot of mistakes before you become an experienced programmer.

Returning an error code and writing explicit error handling code at each level, the thing you typically do in C, may often work better. Google's Go language intentionally left exception handling out of the language.

--

12