snapshot of Lua 4.1 (work)

classic Classic list List threaded Threaded
29 messages Options
12
Reply | Threaded
Open this post in threaded view
|

snapshot of Lua 4.1 (work)

Luiz Henrique de Figueiredo
>The one wish I'd love to see granted is read-only access to the active
>code base - i.e. to *see* what your upcoming 4.1 is starting to look
>like.

Your wish is granted:
	http://www.tecgraf.puc-rio.br/lua/lua-4.1-work.tar.gz
Be careful what you wish :-)

Now, the catches:
- This is a snapshot. It might not even compile (it does, though).
- There is no documentation. The code is the documentation :-)
- Please do not ask questions about the code. Wait until an alpha version is
  out and see whether the questions are still valid.
- I'll try to keep this up-to-date, but I can't promise anything.

Lua 4.1 brings a new API function
  LUA_API lua_State *lua_newthread (lua_State *L, int stacksize);
States created with this share the global environment with L, but have separate
runtime data. (If L is NULL, lua_newthread returns a complete new state.)

There are also hooks for mutex: lua_lock(L) and lua_unlock(L), which are empty
by default.

We'd like to see implementations of coroutines and multithreads on top these.
--lhf

Reply | Threaded
Open this post in threaded view
|

Re: snapshot of Lua 4.1 (work)

Jean-Claude Wippler
Luiz Henrique de Figueiredo <[hidden email]> wrote:

>Your wish is granted:
>	http://www.tecgraf.puc-rio.br/lua/lua-4.1-work.tar.gz
>Be careful what you wish :-)

Thank you, sir!

>Now, the catches:
>- This is a snapshot. It might not even compile (it does, though).
>- There is no documentation. The code is the documentation :-)
>- Please do not ask questions about the code. Wait until an alpha version is
>  out and see whether the questions are still valid.
>- I'll try to keep this up-to-date, but I can't promise anything.

Of course.  This was not intended to cause you more work, but to be able
to find out more without having to ask how or when, which is what such a
"development snapshot" addresses.

>We'd like to see implementations of coroutines and multithreads on top these.

Aha, so a coroutine would be a lua_newthread call, tied to some C glue
code as closure, and returned to the original context as callable
userdata.  Very nice.

Is anyone doing this, or are you looking for a volunteer?
Oops, now I'm asking more questions...

-jcw


Reply | Threaded
Open this post in threaded view
|

Re: snapshot of Lua 4.1 (work)

Luiz Henrique de Figueiredo
In reply to this post by Luiz Henrique de Figueiredo
>Aha, so a coroutine would be a lua_newthread call, tied to some C glue
>code as closure, and returned to the original context as callable
>userdata.  Very nice.

Possibily, but there might be other solutions.
We think that coroutines in Lua should be provided by a Lua *library*.

>Is anyone doing this, or are you looking for a volunteer?

We, the Lua team, are not doing this right now, but we may do until 4.1 is out.

For the time being, perhaps the easiest thing would be to use Edgar Toernig's
coroutine library as a test. It would be great if someone out there did this.
See "coro" at http://lecker.essen.de/~froese/ .
--lhf

Reply | Threaded
Open this post in threaded view
|

Re: snapshot of Lua 4.1 (work)

Roberto Ierusalimschy
In reply to this post by Jean-Claude Wippler
> Is anyone doing this, or are you looking for a volunteer?

I have a primitive implementation of pthreads using these new facilities. I 
am not sure whether it is still working with this Lua code (I made it some 
time ago), but the main ideas are there. You must put the .h file as the 
LUA_USER_H; it redefines LUA_LOCK and LUA_UNLOCK (this is a first 
incompatibility; now those macros are written in lower case...). 
Moreover, there is still the possibility of some syncronization problems 
during garbage collection. Use it at your own risk!

-- Roberto

#include <pthread.h>

#define LUA_LOCK(L)	pthread_mutex_lock(&mutex)
#define LUA_UNLOCK(L)	pthread_mutex_unlock(&mutex)


extern pthread_mutex_t mutex;

void openthreads (lua_State *L);


#define LUA_USERINIT(L)		(openstdlibs(L), openthreads(L))
#include <assert.h>
#include <pthread.h>

#include "../lua.h"
#include "../lauxlib.h"


pthread_mutex_t mutex = PTHREAD_MUTEX_INITIALIZER;



static void *init_thread (void *arg) {
  lua_State *L = (lua_State *)arg;
  int n = lua_getn(L, 1);  /* number of arguments */
  int i;
  for (i=1; i<=n; i++)  /* push arguments */
    lua_rawgeti(L, 1, i);
  lua_remove(L, 1);  /* table is garbage now */
  pthread_detach(pthread_self());
  lua_call(L, n, 0);
  return NULL;
}


static int newthread (lua_State *L) {
  lua_State *NL;
  int ref;
  pthread_t thread;
  luaL_checktype(L, 1, LUA_TFUNCTION);
  luaL_checktype(L, 2, LUA_TTABLE);
  NL = lua_open(L, luaL_opt_int(L, 3, 0));
  if (NL == NULL)
    lua_error(L, "cannot create new stack");
  lua_settop(L, 2);  /* remove eventual stacksize */
  /* move table and function from L to NL */
  ref = lua_ref(L, 1);
  lua_getref(NL, ref);
  lua_unref(L, ref);  
  ref = lua_ref(L, 1);
  lua_getref(NL, ref);
  lua_unref(L, ref);  
  if (pthread_create(&thread, NULL, init_thread, NL) != 0)
    lua_error(L, "cannot create new thread");
  return 0;
}


static int newmutex (lua_State *L) {
  pthread_mutex_t *mtx =
            (pthread_mutex_t *)lua_newuserdata(L, sizeof(pthread_mutex_t));
  if (pthread_mutex_init(mtx, NULL) != 0)
    lua_error(L, "unable to create mutex");
  lua_settag(L, lua_type2tag(L, "pthread_Mutex"));
  return 1;
}


static int mdestroy (lua_State *L) {
  pthread_mutex_t *mtx = (pthread_mutex_t *)lua_touserdata(L, 1);
  pthread_mutex_destroy(mtx);
  return 0;
}


static int mlock (lua_State *L) {
  pthread_mutex_t *mtx =
            (pthread_mutex_t *)luaL_check_userdata(L, 1, "pthread_Mutex");
  if (pthread_mutex_lock(mtx) != 0)
    lua_error(L, "unable to lock mutex");
  return 0;
}


static int munlock (lua_State *L) {
  pthread_mutex_t *mtx =
            (pthread_mutex_t *)luaL_check_userdata(L, 1, "pthread_Mutex");
  if (pthread_mutex_unlock(mtx) != 0)
    lua_error(L, "unable to unlock mutex");
  return 0;
}


struct Cond {
  pthread_cond_t cond;  /* condition */
  pthread_mutex_t *mtx;  /* corresponding mutex */
  int mtxref;  /* reference to mutex (to avoid GC) */
};


static int newcond (lua_State *L) {
  struct Cond *cond = (struct Cond *)lua_newuserdata(L, sizeof(struct Cond));
  cond->mtx = (pthread_mutex_t *)luaL_check_userdata(L, 1, "pthread_Mutex");
  lua_pushvalue(L, 1);
  cond->mtxref = lua_ref(L, 1);  /* lock mutex */
  if (pthread_cond_init(&cond->cond, NULL) != 0)
    lua_error(L, "unable to create condition");
  lua_settag(L, lua_type2tag(L, "pthread_Condition"));
  return 1;
}


static int cwait (lua_State *L) {
  struct Cond *cond =
      (struct Cond *)luaL_check_userdata(L, 1, "pthread_Condition");
  if (pthread_cond_wait(&cond->cond, cond->mtx) != 0)
    lua_error(L, "unable to wait");
  return 0;
}


static int csignal (lua_State *L) {
  int status;
  struct Cond *cond =
      (struct Cond *)luaL_check_userdata(L, 1, "pthread_Condition");
  if (*luaL_opt_string(L, 2, "a") == 'a')
    status = pthread_cond_broadcast(&cond->cond);
  else
    status = pthread_cond_signal(&cond->cond);
  if (status != 0)
    lua_error(L, "unable to signal");
  return 0;
}


static int cdestroy (lua_State *L) {
  struct Cond *cond = (struct Cond *)lua_touserdata(L, 1);
  pthread_cond_destroy(&cond->cond);
  lua_unref(L, cond->mtxref);
  return 0;
}


static int gettab (lua_State *L) {
  lua_insert(L, 1);
  lua_gettable(L, 1);
  return 1;
}


static const struct luaL_reg lib[] = {
  {"newthread", newthread},
  {"newmutex", newmutex},
  {"newcond", newcond}
};


static const struct luaL_reg muops[] = {
  {"lock", mlock},
  {"unlock", munlock},
  {NULL, NULL}
};


static const struct luaL_reg cndops[] = {
  {"wait", cwait},
  {"signal", csignal},
  {NULL, NULL}
};


static void createmethods (lua_State *L, const struct luaL_reg *m, int tag) {
  lua_newtable(L);
  for (;m->name; m++) {
    lua_pushstring(L, m->name);
    lua_pushcfunction(L, m->func);
    lua_settable(L, -3);
  }
  lua_pushcclosure(L, gettab, 1);
  lua_settagmethod(L, tag, "gettable");
}


void openthreads (lua_State *L) {
  int mtx = lua_newtype(L, "pthread_Mutex", LUA_TUSERDATA);
  int cnd = lua_newtype(L, "pthread_Condition", LUA_TUSERDATA);
  lua_pushcfunction(L, mdestroy);
  lua_settagmethod(L, mtx, "gc");
  lua_pushcfunction(L, cdestroy);
  lua_settagmethod(L, cnd, "gc");
  createmethods(L, muops, mtx);
  createmethods(L, cndops, cnd);
  luaL_openl(L, lib);
}
Reply | Threaded
Open this post in threaded view
|

Re: snapshot of Lua 4.1 (work)

Edgar Toernig
In reply to this post by Luiz Henrique de Figueiredo
> Your wish is granted:
>         http://www.tecgraf.puc-rio.br/lua/lua-4.1-work.tar.gz

Thanks very much, ET.



Reply | Threaded
Open this post in threaded view
|

RE: snapshot of Lua 4.1 (work)

Eric Ries
In reply to this post by Luiz Henrique de Figueiredo
I am a little confused. I thought coroutines were going to be part of the
4.1 release? If not, I'll give the CORO library a shot, and see what I can
get it to do.

Has anyone had any luck running it on Windows either under Cygwin or MSVC++?

Eric

> -----Original Message-----
> From: [hidden email]
> [[hidden email] Behalf Of Luiz Henrique de
> Figueiredo
> Sent: Wednesday, May 09, 2001 12:35 PM
> To: Multiple recipients of list
> Subject: Re: snapshot of Lua 4.1 (work)
>
>
> >Aha, so a coroutine would be a lua_newthread call, tied to some C glue
> >code as closure, and returned to the original context as callable
> >userdata.  Very nice.
>
> Possibily, but there might be other solutions.
> We think that coroutines in Lua should be provided by a Lua *library*.
>
> >Is anyone doing this, or are you looking for a volunteer?
>
> We, the Lua team, are not doing this right now, but we may do
> until 4.1 is out.
>
> For the time being, perhaps the easiest thing would be to use
> Edgar Toernig's
> coroutine library as a test. It would be great if someone out
> there did this.
> See "coro" at http://lecker.essen.de/~froese/ .
> --lhf


Reply | Threaded
Open this post in threaded view
|

Re: snapshot of Lua 4.1 (work)

Edgar Toernig
In reply to this post by Jean-Claude Wippler
Jean-Claude Wippler wrote:
> >We'd like to see implementations of coroutines and multithreads on top these.
> 
> Is anyone doing this, or are you looking for a volunteer?

I'll definitely take a look.  In Sol I already have a coroutine library
based on longjmp.  But it requires knowledge about the structure of the
jmp_buf.  I recently found the POSIX functions getcontext/makecontext/
swapcontext.  I'll change Sol's coro-lib to depend on these functions
and add a prototype ucontext implementation for systems that do not have
these functions [1].  I'll then port this lib to Lua 4.1.

Ciao, ET.

[1] I actually don't know which systems offer these functions.  They
are part of the POSIX standard but afaics not required.  My system
at least does not have them (old Linux).  It would be nice if someone
could tell me which system support them.  Solaris?  Win?  Newer Linux?
Probably a look whether <ucontext.h> with prototypes for these functions
exists is enough...



Reply | Threaded
Open this post in threaded view
|

RE: snapshot of Lua 4.1 (work)

Luiz Henrique de Figueiredo
In reply to this post by Luiz Henrique de Figueiredo
>I am a little confused. I thought coroutines were going to be part of the
>4.1 release?

Lua 4.1 will bring *support* for coroutines and multithreading, possibily with
sample implementations. After all, this is our philosophy: provide the tools,
not the features...

>If not, I'll give the CORO library a shot, and see what I can get it to do.

Please do. Again, what we're looking for is a *library* for coroutines,
much in the same vein as the library for multithreading with pthreads that
Roberto posted today.
--lhf

Reply | Threaded
Open this post in threaded view
|

Re: snapshot of Lua 4.1 (work)

Jean-Claude Wippler
In reply to this post by Luiz Henrique de Figueiredo
Luiz Henrique de Figueiredo <[hidden email]> wrote:

There are (only) three identifiers which prevent merging all the lua
sources into one big one and compiling it without errors:

    check       defined in ldebug, conflicts with lparser
    read_number a static function in llex, conflicts with liolib
    next        defined in llex, conflicts with next in several places

If you change these three declarations to lcheck, lread_number, and
lnext, then the sources of Lua can be combined.  FWIW, I use a Tcl script
to merge C sources + headers, see http://www.equi4.com/lux/onesrc.tcl

Such a merged source file represents a complete Lua system, including std
libraries.  It's no rocket science, but it can simplify deployment.

-jcw


Reply | Threaded
Open this post in threaded view
|

Re: snapshot of Lua 4.1 (work)

Edgar Toernig
In reply to this post by Eric Ries
Eric Ries wrote:
> 
> I am a little confused. I thought coroutines were going to be part of the
> 4.1 release?

Coroutines cannot be implemented in ANSI C (at least not for Lua).

> If not, I'll give the CORO library a shot, and see what I can
> get it to do.

Well, it will not help a lot.  The corolib only supports X86 Linux and
FreeBSD.  I never got any patches for other architectures.

> Has anyone had any luck running it on Windows either under Cygwin or MSVC++?

Peter Wang sent me a patch for djgpp (basically s/mmap/malloc/) but the
build requires unix tools (sh, sed, ...).  He mentions that it also compiles
with Mingw32 (whatever that is *g*).  I did not made a new version of
libcoro with his patch yet.  (It's becoming silly: only 3 arches supported,
all X86, and the diffs are basically only 2-4 lines.)

If you want to play with coroutines with the Lua-4.1 snapshot you may
be better off with the scorolib.c from Sol (requires changes, Sol API
is different!).  But you get the basic Lua framework...

Ciao, ET.


Reply | Threaded
Open this post in threaded view
|

Re: snapshot of Lua 4.1 (work)

Carlos Cassino
In reply to this post by Luiz Henrique de Figueiredo
> [1] I actually don't know which systems offer these functions.  They
> are part of the POSIX standard but afaics not required.  My system
> at least does not have them (old Linux).  It would be nice if someone
> could tell me which system support them.  Solaris?  Win?  Newer Linux?
> Probably a look whether <ucontext.h> with prototypes for these functions
> exists is enough...

AIX 4.3: All functions available.
IRIX 5.3: All functions available.
SunOS 5.5: All functions available.

Linux Red Hat 6.1
-----------------

getcontext/setcontext/swapcontext/makecontext are all defined in
<ucontext.h>, although there are man pages only for getcontext and
setcontext and the linker says:

/tmp/ccfrjprH.o: In function `main':
/tmp/ccfrjprH.o(.text+0x13): warning: getcontext is not implemented and will always fail
/tmp/ccfrjprH.o(.text+0xe): warning: setcontext is not implemented and will always fail
/tmp/ccfrjprH.o(.text+0x4): warning: makecontext is not implemented and will always fail
/tmp/ccfrjprH.o(.text+0x9): warning: swapcontext is not implemented and will always fail

In other words, functions not available (yet?).

Hope that helps.

-- Cassino

Reply | Threaded
Open this post in threaded view
|

Re: snapshot of Lua 4.1 (work)

John Belmonte-2
In reply to this post by Luiz Henrique de Figueiredo
What I'm happy about most with this snapshot, more than any of the new
features, is the clean-up of the Lua number type.  This will eliminate much
of the hacking I have to do every time a new version is released.  I did
notice a dangling double however, in llimits.h:

    union L_Umaxalign { double d; void *s; long l; };

should probably be:

    union L_Umaxalign { lua_Number d; void *s; long l; };

However a similar issue, controlling what the implementation uses for a
"long", still needs some work.  I noticed that many of the local variables
that used to be longs without a good reason are now ints, which helps to
reduce the problem.  However I'd really like a single place where the
implementation's notion of a "long" is defined with a macro, and that macro
to be used consistently in the code.  For example I use a macro LLNG, and so
the Instruction typedef would become:

    typedef unsigned LLNG Instruction;

On my platform I need to replace all long's (64-bit) with int's (32-bit).
One reason is the memory savings, although this is mainly addressed by the
Instruction typedef that already exists.  The other reason is that my
compiler generates inefficient code for 64-bit operations, especially
conversion between long integer and floating point.

It's not much work, here are the current long references:

    include\lauxlib.h(62): #define luaL_check_long(L,n)
((long)luaL_check_number(L, n))
    include\lauxlib.h(64): #define luaL_opt_long(L,n,d)
((long)luaL_opt_number(L, n,d))
    src\llimits.h(49): typedef unsigned long lu_mem;
    src\llimits.h(52): typedef long ls_nstr;
    src\llimits.h(85): union L_Umaxalign { double d; void *s; long l; };
    src\llimits.h(95): typedef unsigned long Instruction;
    src\lundump.c(222): if ((long)f!=(long)tf)
    src\lib\lbaselib.c(107): unsigned long n;
    src\lib\liolib.c(378): long offset = luaL_opt_long(L, 3, 0);
    src\lib\lstrlib.c(21): typedef long sint32;

With that I could finally remove an item from my Lua warts list  ;)

Regards,
-John



Reply | Threaded
Open this post in threaded view
|

Coroutines [was: snapshot of Lua 4.1 (work)]

Jean-Claude Wippler
In reply to this post by Edgar Toernig
Edgar Toernig <[hidden email]> wrote:

[coroutines]
>I'll definitely take a look.  In Sol I already have a coroutine library
>based on longjmp.  But it requires knowledge about the structure of the
>jmp_buf.  I recently found the POSIX functions getcontext/makecontext/
>swapcontext.  I'll change Sol's coro-lib to depend on these functions
>and add a prototype ucontext implementation for systems that do not have
>these functions [1].  I'll then port this lib to Lua 4.1.

Ouch.  C-level co-routines?  My first reaction was: why not simply
implement them at the Lua level?  Oh, of course: Lua intermingles the C
call stack with its own... like Tcl and standard Python - and unlike
Stackless Python (which implements "calling" without nesting the C
stack).  SP's trick is to have the VM push and pop call frames, instead
of going through a C function which nests the VM.

For Lua, this would require luaD_call to be split into two, and all
places where it gets called as well.  Quite a bit of surgery, unfortunately.

That's a pity.  If Lua were to avoid the C stack for ordinary function-
call nesting, then coroutines (at the Lua-level) would be easy to
implement - no special C code, and portable.  Simply a closure which
remembers the new_thread pointer, and saving/restoring the current VM
instruction pointer on that stack for suspend/resume.

-jcw


Reply | Threaded
Open this post in threaded view
|

Re: snapshot of Lua 4.1 (work)

Jean-Claude Wippler
In reply to this post by Edgar Toernig
Edgar Toernig <[hidden email]> wrote:

>Coroutines cannot be implemented in ANSI C (at least not for Lua).

True, C-level coroutine cannot, but see my previous post.

> [...] The corolib only supports X86 Linux and
>FreeBSD.  I never got any patches for other architectures.

You might want to have a look again at Icon, which has been ported to
many architectures, and which supports both generators and coroutines:
    http://www.cs.arizona.edu/icon/ftp/packages/unix/
See src/common/ for context-switch assembly code (e.g. "rswitch.s").

-jcw


Reply | Threaded
Open this post in threaded view
|

Re: Coroutines [was: snapshot of Lua 4.1 (work)]

John Belmonte-2
In reply to this post by Jean-Claude Wippler
Hi jc,

> That's a pity.  If Lua were to avoid the C stack for ordinary function-
> call nesting, then coroutines (at the Lua-level) would be easy to
> implement - no special C code, and portable.  Simply a closure which
> remembers the new_thread pointer, and saving/restoring the current VM
> instruction pointer on that stack for suspend/resume.

This issue has already been hashed out.  Initially they planned to make 4.1
stackless.  lhf had written:

> >Another
> >solution that would be nice if implemented in Lua would be to make
> >lua stackless. This way coroutines could easily be implemented in a
> >cross-platform way.
>
> That's the way we plan to do it in 4.1.

But here was Roberto's final word on it:

> > Presumably the hard bit is making it stay stackless when it calls C;
>
> The problem is that there are lots of places where Lua can make a
> call to C. Any tagmethod can be a C function, and so most basic operations
> may call C. Currently we handle calls to C and to Lua in the same way.
> To change that would break a nice "simetry" in the code, and would need
> lots of changes in the code. (It could also compromise performance...)
>
> We had three goals in mind: restricted co-rotines (only Lua), generic
> co-routines, and multi-thread. Lua stackless would demand lots of work,
and
> would solve only the first goal; it would be useless for the other two.
> Multiple stacks solves the other two goals (with a small external help),
it
> is quite simple to implement, and has no impact in performance.

-John



Reply | Threaded
Open this post in threaded view
|

Re: Coroutines [was: snapshot of Lua 4.1 (work)]

Jean-Claude Wippler
John Belmonte <[hidden email]> wrote:

[coro's]
>This issue has already been hashed out.

Thanks for the info - I had not seen this discussion before.

>But here was Roberto's final word on it:
>
>> > Presumably the hard bit is making it stay stackless when it calls C;
>>
>> The problem is that there are lots of places where Lua can make a
>> call to C. Any tagmethod can be a C function, and so most basic operations
>> may call C. Currently we handle calls to C and to Lua in the same way.
>> To change that would break a nice "simetry" in the code, and would need
>> lots of changes in the code. (It could also compromise performance...)

Calls to C are not a problem.  Calls from C to Lua are.  How would the
symmetry in the code be affected?

>> We had three goals in mind: restricted co-rotines (only Lua), generic
>> co-routines, and multi-thread. Lua stackless would demand lots of work,
>> and  would solve only the first goal; it would be useless for the other
>> two.  Multiple stacks solves the other two goals (with a small external
>> help), it is quite simple to implement, and has no impact in performance.

The first goal is actually a major one: one way to use scripting, is to
code only small (i.e. atomic) pieces in C.  So C is a way to add
"primitives" to the scripting language (the other main use is to add
bindings to existing libs).  With that perspective, co-routines are not
needed in C: if some C logic requires the ability to suspend/resume, one
can try to recode the logic as two C functions and make the state
explicit (or go for multi-threading).  To put it differently: one could
achieve excellent parallelism with just Lua-level coroutines (async I/O +
"restricted" coro's would be an awesome combo, IMO).

Goal two comes at a considerable cost: portability, or rather lack of it.
 C-level stack switching requires machine code, and introduces the tricky
issue of preventing overflow in all stacks.  It would only be needed when
C code calls back into Lua which *then* suspends, which is not too likely
when bringing in existing C libraries, nor when C code consists of small
bits of functionality (e.g. speed-ups).  It should be possible to detect
and prevent the use of suspend / resume in such a Lua -> C -> Lua nested
case (just like tag methods prevent runaway recursion).

Goal three is not really related to the above: Lua uses no globals, so
threading in itself already comes for free.  The lock/unlock hooks make
it possible to share data between multiple threads.  This issue does not
require stack switching, and is in essence already solved in the 4.1-work
preview.

While complete stacklessness ("continuations") is no doubt very hard to
achieve, I'm not convinced that one could not chase down all luaD_call
spots and resolve the issues for the restricted coro case.  Tag methods
must be dealt with, but many appear to be tail calls.  These could be
made stackless by pushing the call onto the VM execution stack and
*returning* to the VM, instead of calling into one more level of C.

-jcw


Reply | Threaded
Open this post in threaded view
|

Re: snapshot of Lua 4.1 (work)

Paul Hsieh
In reply to this post by Luiz Henrique de Figueiredo
> Lua 4.1 brings a new API function
>   LUA_API lua_State *lua_newthread (lua_State *L, int stacksize);
> States created with this share the global environment with L, but have separate
> runtime data. (If L is NULL, lua_newthread returns a complete new state.)
> 
> There are also hooks for mutex: lua_lock(L) and lua_unlock(L), which are empty
> by default.

Well -- is the Lua state indivisible?  The problem with putting a single lock on the 
whole state is that is will very quickly become a bottleneck that exposes different 
threads to each other's implementations in terms of performance (i.e., there will be 
a big incentive to minimize global state access, even if the multiple threads don't 
otherwise interfere with each other.)  Also if you are performing read-only 
accesses, then in fact its ok to have multiple readers if there are no writers.  

So, in fact, you would prefer something like:

handle = lua_AcquireAccess (Lua_state, state_entry, access_mode)
lua_ReleaseAccess (handle)

I am not yet familliar enough with the implementation of Lua to know if all of these 
things are possible.  But you can specify this today and grow into them in the 
future on both sides: 1) In the first Lua implementation state_entry would always be 
0, or the start of the state or whatever and access_mode would always be equal to 
"READWRITE".  2) In the first C callback implementations lua_AcquireAccess would 
just grab a global mutex, and lua_ReleaseAccess would just release that global mutex.

> We'd like to see implementations of coroutines and multithreads on top these.

Hmmm ... my suspicion is that where people want this most is with multiple instances 
of Lua environments talking to some userdata implemented in C which also have client 
processes in the C environment (at least that's what *I* want.)  In which case the 
API you present (with or without my proposed modifications) should be sufficient for 
that.  I.e., people should be able to use real threads (not just coroutines) and use 
this.

Oh, BTW, this functionality is very cool.
--
Paul Hsieh
[hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: snapshot of Lua 4.1 (work)

Reuben Thomas-4
In reply to this post by Edgar Toernig
> [1] I actually don't know which systems offer these functions.  They
> are part of the POSIX standard but afaics not required.  My system
> at least does not have them (old Linux).  It would be nice if someone
> could tell me which system support them.  Solaris?  Win?  Newer Linux?

My Linux system (RedHat 7.1) has prototypes for all four, but manpages only
for getcontext and setcontext. So does RH 6.2. SunOS 5.7 has all four
functions documented..

-- 
http://sc3d.org/rrt/ | Careful Cyclists Approaching From Right


Reply | Threaded
Open this post in threaded view
|

Re: snapshot of Lua 4.1 (work)

Roberto Ierusalimschy
In reply to this post by John Belmonte-2
>     union L_Umaxalign { double d; void *s; long l; };
> 
> should probably be:
> 
>     union L_Umaxalign { lua_Number d; void *s; long l; };

This double is to ensure proper alignment for userdata and strings 
*outside* Lua. (Notice that strings will always be aligned, too.)

-- Roberto

Reply | Threaded
Open this post in threaded view
|

Re: Coroutines [was: snapshot of Lua 4.1 (work)]

J. Perkins-2
In reply to this post by Jean-Claude Wippler
I haven't had a chance to look at the snapshot yet, but Lua-level coroutines are all that I would need. I just want to have an -official- way to do what Bret Mogilefsky did in GF way back when. Run a Lua-thread until it blocks or until it have executed a fixed number of opcodes, and then switch to the next Lua thread. Calls into C are treated as one atomic operation. It sounds like lua_newthread will support that easily, can't wait to give it a try (though I'll probably wait for an official release, unfortunately I don't have time to mess around with it right now :p ) As long as the task switch always takes place while the thread is in the
Lua VM it should work fine.

I don't think C-level coroutine support is a good idea, myself. Programs that need it should
(IMO of course) use a separate C library.

Jason
379




Jean-Claude Wippler wrote:

Edgar Toernig <[hidden email]> wrote:

[coroutines]

I'll definitely take a look.  In Sol I already have a coroutine library
based on longjmp.  But it requires knowledge about the structure of the
jmp_buf.  I recently found the POSIX functions getcontext/makecontext/
swapcontext.  I'll change Sol's coro-lib to depend on these functions
and add a prototype ucontext implementation for systems that do not have
these functions [1].  I'll then port this lib to Lua 4.1.


Ouch.  C-level co-routines?  My first reaction was: why not simply
implement them at the Lua level?  Oh, of course: Lua intermingles the C
call stack with its own... like Tcl and standard Python - and unlike
Stackless Python (which implements "calling" without nesting the C
stack).  SP's trick is to have the VM push and pop call frames, instead
of going through a C function which nests the VM.

For Lua, this would require luaD_call to be split into two, and all
places where it gets called as well.  Quite a bit of surgery, unfortunately.

That's a pity.  If Lua were to avoid the C stack for ordinary function-
call nesting, then coroutines (at the Lua-level) would be easy to
implement - no special C code, and portable.  Simply a closure which
remembers the new_thread pointer, and saving/restoring the current VM
instruction pointer on that stack for suspend/resume.





12