Coroutine support in 4.1 / IPC mechanisms

classic Classic list List threaded Threaded
19 messages Options
Reply | Threaded
Open this post in threaded view
|

Coroutine support in 4.1 / IPC mechanisms

Richard Vandenberg
Hi all.. I'm new to this list, but have used lua since version 2. I only
recently came looking for more recent versions and was pleasantly
surprised by the development done since version 2.5. (Thanks guys :))

I know there has been some considerable talk about coroutines &
multitasking, and I'm wondering if 4.1 will include anything in the way
of IPC's between the tasks, or if that will be left to the coroutine
implementor. 

I've pretty much written everything I need to handle multiple Lua
environments talking to each other, but with 4.1 alpha slated for
release sometime soon, I'm curious if I should stop working on these
things and just wait for the release.

Thanks...

Rick

Reply | Threaded
Open this post in threaded view
|

Re: Coroutine support in 4.1 / IPC mechanisms

Luiz Henrique de Figueiredo
>Hi all.. I'm new to this list, but have used lua since version 2. I only
>recently came looking for more recent versions and was pleasantly
>surprised by the development done since version 2.5. (Thanks guys :))

Wow! And you missed versions 3.x...
Welcome to tag methods, varargs, anonymous functions, upvalues, now multiple
states! And of course the new API...

>I know there has been some considerable talk about coroutines &
>multitasking, and I'm wondering if 4.1 will include anything in the way
>of IPC's between the tasks, or if that will be left to the coroutine
>implementor. 

Lua 4.1 will include lua_newthread, which creates a new "thread" of Lua
execution, sharing the global state of an existing state. So, yes, there'll
be ways to share information and send messages from you "thread" to another,
but these will have to be programmed by the user and also care must be taken
to avoid concurrent updates to the global state. There will be compile-time
hooks for this.

>I've pretty much written everything I need to handle multiple Lua
>environments talking to each other, but with 4.1 alpha slated for
>release sometime soon, I'm curious if I should stop working on these
>things and just wait for the release.

Unfortunately, 4.1 alpha is not going to be released by the end of April.
Perhaps, by the end of May.

Here is a quick preview of what is probably coming out with 4.1 (no promises):

- suport for multithreads and coroutines
- suport for wchar (preparing for unicode)
- weak tables
- more general syntax for function definition and expressions:
  function a.b.c.d.e.f (...) end
  (a)=3; (print or write)(9)
- names for user created types
- secure binary dostring/dofile
- new functions (time/date, tmpfile, unwrap, require)
- positon capture in string matching
- better handling of userdata/tags
- faster VM (!)

--lhf

Reply | Threaded
Open this post in threaded view
|

Re: Coroutine support in 4.1 / IPC mechanisms

Edgar Toernig
Luiz Henrique de Figueiredo wrote:
>
> - more general syntax for function definition and expressions:
>   function a.b.c.d.e.f (...) end
>   (a)=3; (print or write)(9)

How is that going to work?  Will the ; be required in Lua 4.1?
Sol allows these kind of things too.  But the requirement still is,
the a statements begins either with a keyword or an identifier.

So this is valid

	a,(b or c).x,d = 1,2,3

but this is not

	(a or b).c = 1

because of the leading '('.  It could be misinterpreted i.e.

	f(1)
	(a or b).c = 1

-->	f(1)(a or b).c = 1
that is call f(1), returns a function, call this with (a or b), 
returns a table and assign to member c of it.

So the limitation "a statement _must_ begin with a keyword or
an identifier (or upvalue)" is still there.  How are you going
to solve this?

Btw, in Sol the parenthesized expressions have the additional side
effect that they generate an r-value (with one result).

So this is _not_ valid:
	a,(b),c = 1,2,3

And here it may be useful:
	f(a, b, (g(1)))
which will _always_ pass 3 args to f because the () around the g call
will generate a single r-value.

Ciao, ET.


PS: Btw, these changes makes the parser even smaller ;-)

Reply | Threaded
Open this post in threaded view
|

Re: Coroutine support in 4.1 / IPC mechanisms

Luiz Henrique de Figueiredo
In reply to this post by Richard Vandenberg
>>   (a)=3; (print or write)(9)
>
>How is that going to work?  Will the ; be required in Lua 4.1?

Yes, in this case the ';' will be required to remove the ambiguity.
I'm glad that ';' will now be useful (but not always required)!

>	(a or b).c = 1
>	a,(b),c = 1,2,3

These will be valid in 4.1.

>And here it may be useful:
>	f(a, b, (g(1)))
>which will _always_ pass 3 args to f because the () around the g call
>will generate a single r-value.

This will not be the semantics in 4.1.
--lhf

Reply | Threaded
Open this post in threaded view
|

Re: Coroutine support in 4.1 / IPC mechanisms

Edgar Toernig
Luiz Henrique de Figueiredo wrote:
> 
> >>   (a)=3; (print or write)(9)
> >
> >How is that going to work?  Will the ; be required in Lua 4.1?
> 
> Yes, in this case the ';' will be required to remove the ambiguity.
> I'm glad that ';' will now be useful (but not always required)!

Hmm...  up to now the ';' was really optional[1].  Now it makes a
difference.  If all cases where it is required would generate a
syntax error it would be ok.  But there are cases where both
interpretations are valid (i.e.  a=b ; (print)(9) ) and IMHO
that's not good.  It makes the language more fragile and complicated.
Maybe a standard identity function [1] (part of the baselib) would
be better.
 
> > [  (a)=3   ]
> 
> These will be valid in 4.1.

IMHO to let ()-expressions be l-values gives nothing new to the
language and just shows more limitations:

	(a) = 1  -- ok
	(b) = 1  -- ok
	(a and b) = 1  -- invalid
	(f()) = 1  -- invalid

Saying: '()' always generates a single value is IMHO cleaner.

Ciao, ET.

[1] I was even about to remove it from Sol completely.  But
remembering my first steps with Lua I kept it :-)

[2] function id(x) return x end



Reply | Threaded
Open this post in threaded view
|

Re: Coroutine support in 4.1 / IPC mechanisms

John Belmonte-2
> > >How is that going to work?  Will the ; be required in Lua 4.1?
> >
> > Yes, in this case the ';' will be required to remove the ambiguity.
> > I'm glad that ';' will now be useful (but not always required)!
>
> Hmm...  up to now the ';' was really optional[1].  Now it makes a
> difference.

I was just thinking the other day I should start using ; in Lua.  I keep
omitting them when I go back to C...



Reply | Threaded
Open this post in threaded view
|

Re: Coroutine support in 4.1 / IPC mechanisms

Jonathan Adamczewski
In reply to this post by Luiz Henrique de Figueiredo
> Here is a quick preview of what is probably coming out with 4.1 (no promises):
> 
> - suport for multithreads and coroutines
> - faster VM (!)

Yeee :)



-- 
Coffee should be served hot as hell, black as night and sweet as love.
   Turkish Proverb


Reply | Threaded
Open this post in threaded view
|

Re: Coroutine support in 4.1 / IPC mechanisms

Dennis Taylor
	Well, with all the talk of the new features of Lua 4.1, do we
have any vague guesses of approximate release dates to look forward to?
I, for one, would give my left arm for a good mechanism for coroutines
right now. :-)

_________________________________________________________________________
Dennis Taylor           "Anyone whose days are all the same and free from
[hidden email]    want inhabits eternity of a sort."  - Peter Hoeg
_________________________________________________________________________
   PGP Fingerprint: E8D6 9670 4FBD EEC3 6C6B  810B 2B30 E529 51BD 7B90

Reply | Threaded
Open this post in threaded view
|

Re: Coroutine support in 4.1 / IPC mechanisms

Michael T. Richter-2
In reply to this post by Luiz Henrique de Figueiredo
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

> Here is a quick preview of what is probably coming out with 4.1 (no
> promises): - suport for multithreads and coroutines

Woo hoo!

> - suport for wchar (preparing for unicode)

Double woo hoo!

> - weak tables
> - more general syntax for function definition and expressions:
>   function a.b.c.d.e.f (...) end
>   (a)=3; (print or write)(9)
> - names for user created types

Woo hoo!

> - secure binary dostring/dofile
> - new functions (time/date, tmpfile, unwrap, require)
> - positon capture in string matching
> - better handling of userdata/tags
> - faster VM (!)

Triple woo hoo!  (Speed is always good.)

- --
Michael T. Richter
"Be seeing you."

-----BEGIN PGP SIGNATURE-----
Version: PGP Personal Privacy 6.5.3

iQA/AwUBOt+6+rTM3QkE7U/oEQI2oQCeKbGS3akvBnCnw/ooKkgFgbmI9KgAoKGL
6OUahKNDmbHhmfBkNoz8wKHN
=0CQV
-----END PGP SIGNATURE-----



Reply | Threaded
Open this post in threaded view
|

RE: Coroutine support in 4.1 / IPC mechanisms

Philippe Lhoste
In reply to this post by Luiz Henrique de Figueiredo
> Here is a quick preview of what is probably coming out with 4.1
> (no promises):
>
> - suport for multithreads and coroutines
> - suport for wchar (preparing for unicode)
> - weak tables

What is it? Sorry, I am not a language guru (I am a nothing guru, actually
:-)

> - more general syntax for function definition and expressions:
>   function a.b.c.d.e.f (...) end
>   (a)=3; (print or write)(9)

I can't figure out what (print or write)(9) will do...

> - names for user created types
> - secure binary dostring/dofile
> - new functions (time/date, tmpfile, unwrap, require)

I would like to see binary operations: and or xor, perhaps shifts. This
would help to parse binary headers of files, among other things, and doesn't
cost much.

> - positon capture in string matching
> - better handling of userdata/tags
> - faster VM (!)

Regards.

--._.·´¯`·._.·´¯`·._.·´¯`·._.·´¯`·._.·´¯`·._.·´¯`·._.--
Philippe Lhoste (Paris -- France)
Professional programmer and amateur artist
http://jove.prohosting.com/~philho/
--´¯`·._.·´¯`·._.·´¯`·._.·´¯`·._.·´¯`·._.·´¯`·._.·´¯`--


Reply | Threaded
Open this post in threaded view
|

Re: Coroutine support in 4.1 / IPC mechanisms

Tore Lund
In reply to this post by Luiz Henrique de Figueiredo
Luiz Henrique de Figueiredo wrote:
> 
> Here is a quick preview of what is probably coming out with 4.1 (no promises):
> 
> - suport for multithreads and coroutines
> - suport for wchar (preparing for unicode)
> - weak tables
> - more general syntax for function definition and expressions:
>   function a.b.c.d.e.f (...) end
>   (a)=3; (print or write)(9)
> - names for user created types
> - secure binary dostring/dofile
> - new functions (time/date, tmpfile, unwrap, require)
> - positon capture in string matching
> - better handling of userdata/tags
> - faster VM (!)

What about the size of all this?  One reason why I am looking into Lua
is that it fits comfortably into a 16-bit SMALL memory model (64 KB code
+ 64 KB data).  I hope (core) Lua will not outgrow this model.

Judging from the archives it appears that Lua developers have put a good
deal of work into making a hassle-free 16-bit Lua.  (And it works great,
thank you.)  Then why not make it official by using something like
#ifdef USE16BIT her and there?  I believe it is only needed in 2 places,
but there might be others.  Having an "official" macro like USE16BIT
would be a good way to sort out such issues.

The same goes for the integer variety of Lua.  According to LTN002,

  This note has focused on reducing the amount of code added to the 
  application by the Lua library. Applications that need this
  will probably also prefer to use integers instead of floating-point
  numbers for the numbers in Lua. (Does a microwave oven need 
  floating-point?) This should be simple to do, as described in 
  lua/config, but the details will probably be discussed in another LTN.

This amounts to a semi-official blessing of an integer version.  I would
like to see it completely official with #ifdef USEINTEGER directives
where needed.  As it is, type double is used in many places where
LUA_NUM_TYPE could have been used instead, and some details concerning
strtol/strtod need to be fixed by hand.

Instead of several private libraries for integer operations there could
be a single lintlib.c.  And it would be reasonable to let the lexer of
the integer version understand hexadecimal and maybe binary numbers (not
sure who really needs octal nowadays).  I think this would make Lua more
attractive to many programmers who are oriented toward embedded or
systems programming.

Also, having 32-bit integers instead of 64-bit doubles is a great help
toward shrinking memory requirements.

After all, this is just a matter of officially blessing what is already
a *de facto* integer version of Lua.  I am not talking about adding
integer as an extra type.
-- 
    Tore


Reply | Threaded
Open this post in threaded view
|

Re: Coroutine support in 4.1 / IPC mechanisms

Luiz Henrique de Figueiredo
In reply to this post by Richard Vandenberg
>	Well, with all the talk of the new features of Lua 4.1, do we
>have any vague guesses of approximate release dates to look forward to?

Like I said, 4.1 alpha should be out by end of May.
--lhf

Reply | Threaded
Open this post in threaded view
|

RE: Coroutine support in 4.1 / IPC mechanisms

Luiz Henrique de Figueiredo
In reply to this post by Richard Vandenberg
>> - weak tables
>
>What is it?

Key and/or values in weak tables are subject to GC, ie, they may be collected.
The reference mechanisms will be implemented using a weak table in 4.1, but
there are others uses: see LTN 6.

This is another example of metamechanisms: instead of implementing weak
references, we implemented something more general... But it's left to you
Lua programmers around the world to find ways of exploiting it.

>I can't figure out what (print or write)(9) will do...

That was just an example. But it calls print or write, if print is not defined.

>I would like to see binary operations: and or xor, perhaps shifts. This
>would help to parse binary headers of files, among other things, and doesn't
>cost much.

There are libraries for this, aren't there?
--lhf

Reply | Threaded
Open this post in threaded view
|

Re: Coroutine support in 4.1 / IPC mechanisms

Luiz Henrique de Figueiredo
In reply to this post by Richard Vandenberg
>> Here is a quick preview of what is probably coming out with 4.1 (no promises):
>What about the size of all this?  One reason why I am looking into Lua
>is that it fits comfortably into a 16-bit SMALL memory model (64 KB code
>+ 64 KB data).  I hope (core) Lua will not outgrow this model.

Small size remains one of our design goals. We do not antecipate any large
growth in Lua's core. As far as I know, Lua 4.1 is much the size size of 4.0.

>Judging from the archives it appears that Lua developers have put a good
>deal of work into making a hassle-free 16-bit Lua.  (And it works great,
>thank you.)  Then why not make it official by using something like
>#ifdef USE16BIT her and there

We'll do something more general: you'll be able to compile Lua wth different
"personalities": integer, float, 16-bit, wchar, multithreaded, etc.,
by adding a user-defined header file luser.h.
The distribution will include some examples, including these ones.

--lhf

Reply | Threaded
Open this post in threaded view
|

Re: Coroutine support in 4.1 / IPC mechanisms

Tore Lund
Luiz Henrique de Figueiredo wrote:
> 
> We'll do something more general: you'll be able to compile Lua wth different
> "personalities": integer, float, 16-bit, wchar, multithreaded, etc.,
> by adding a user-defined header file luser.h.
> The distribution will include some examples, including these ones.

Marvelous!
-- 
    Tore


Reply | Threaded
Open this post in threaded view
|

RE: Coroutine support in 4.1 / IPC mechanisms

Philippe Lhoste
In reply to this post by Luiz Henrique de Figueiredo
> >> - weak tables
> >
> >What is it?
>
> Key and/or values in weak tables are subject to GC, ie, they may
> be collected.
> The reference mechanisms will be implemented using a weak table
> in 4.1, but
> there are others uses: see LTN 6.
>
> This is another example of metamechanisms: instead of implementing weak
> references, we implemented something more general... But it's left to you
> Lua programmers around the world to find ways of exploiting it.

Thank you.

> >I can't figure out what (print or write)(9) will do...
>
> That was just an example. But it calls print or write, if print
> is not defined.

OK, since I though that print will always be defined (which isn't true since
it resides in a library), I missed the point...

> >I would like to see binary operations: and or xor, perhaps shifts. This
> >would help to parse binary headers of files, among other things,
> and doesn't
> >cost much.
>
> There are libraries for this, aren't there?

Yes, but what I would like is to put these functions in the standard math
library, ie. in the official release. Ie. in the official lua.exe. So if I
use such a function in a (standalone) script, users won't have to fetch the
special lua.exe able to process it. These are quite standard functions, hard
to emulate in Lua only.

Unless the "require" mechanism makes my request pointless...

Regards.

--._.·´¯`·._.·´¯`·._.·´¯`·._.·´¯`·._.·´¯`·._.·´¯`·._.--
Philippe Lhoste (Paris -- France)
Professional programmer and amateur artist
http://jove.prohosting.com/~philho/
--´¯`·._.·´¯`·._.·´¯`·._.·´¯`·._.·´¯`·._.·´¯`·._.·´¯`--


Reply | Threaded
Open this post in threaded view
|

RE: Coroutine support in 4.1 / IPC mechanisms

Martin Hollis
In reply to this post by Luiz Henrique de Figueiredo
lhf wrote:
- suport for multithreads and coroutines

Co-routines: Excellent. At the least I expect all the game developers will be pleased. Which seems to be about half the list.



- suport for wchar (preparing for unicode)

I can imagine some people need to store more than ASCII in lua strings. I can even imagine that we might use this feature for game localization. Adding it seems the right thing to do... but please avoid bloat. Is there demand for this feature? Is UTF-8 the answer?



  (a)=3; (print or write)(9)

Increased semicolon reliance sounds fragile. And the benefit in this example seems small. I think lua already has a good selection of ways to shoot yourself in the foot.



We'll do something more general: you'll be able to compile Lua wth different
"personalities": integer, float, 16-bit, wchar, multithreaded, etc.,
by adding a user-defined header file luser.h.

I trust this will not involve #if spaghetti?

(Incidentally are you aware that 'luser' is a mild derogatory term for computer users with little technical knowledge?)



- weak tables
- more general syntax for function definition and expressions:
  function a.b.c.d.e.f (...) end

- names for user created types
- secure binary dostring/dofile
- new functions (time/date, tmpfile, unwrap, require)
- positon capture in string matching
- better handling of userdata/tags
- faster VM (!)


On the subject of adding features, I would like to say a few words:

Please continue to stubbornly resist the temptation to add features that benefit a few. I hope that lua 10 will be more powerful and simpler than lua 4. There are other languages that have many features.

In my view the soul of lua is power through simplicity.

M.


Reply | Threaded
Open this post in threaded view
|

RE: Coroutine support in 4.1 / IPC mechanisms

Samson-3
In reply to this post by Richard Vandenberg
>We'll do something more general: you'll be able to compile Lua wth
different
>"personalities": integer, float, 16-bit, wchar, multithreaded, etc.,
>by adding a user-defined header file luser.h.
>The distribution will include some examples, including these ones.

Will there be any way within scripts to determine the current 'personality'
?

Reply | Threaded
Open this post in threaded view
|

RE: Coroutine support in 4.1 / IPC mechanisms

Luiz Henrique de Figueiredo
In reply to this post by Richard Vandenberg
>>- suport for wchar (preparing for unicode)
>
>I can imagine some people need to store more than ASCII in lua strings. I 
>can even imagine that we might use this feature for game localization. 
>Adding it seems the right thing to do... but please avoid bloat. Is there 
>demand for this feature? Is UTF-8 the answer?

No bloat, unless you do use Unicode or whatever. Like I said, it's all a matter
of defining the right macros. The code has been changed to add macros for
basic types such as char and also aroung literal strings.

>>We'll do something more general: you'll be able to compile Lua wth different
>>"personalities": integer, float, 16-bit, wchar, multithreaded, etc.,
>>by adding a user-defined header file luser.h.
>
>I trust this will not involve #if spaghetti?

No spaghetti! See above.

>(Incidentally are you aware that 'luser' is a mild derogatory term for 
>computer users with little technical knowledge?)

Oops! On the other hand, such users will probably not be able to compile Lua
in any way other than plain in-the-box.

>In my view the soul of lua is power through simplicity.

Same here.
--lhf