Unicode

classic Classic list List threaded Threaded
52 messages Options
123
Reply | Threaded
Open this post in threaded view
|

Re: RTTI

Lenny Palozzi-4
On Sat, Feb 03, 2001 at 01:47:46PM -0600, Joshua Ritter wrote:
> 

{snip}

> I have run into a small snag however.. 
> All function tags callback to the same static int luacallback(lua_State* ls)
> function I have defined... what I need to get from here is:
> 
> A) the "class" of the table
> B) the _name_ of the field (function)
> 
> so a table of "type" cpClass with a function called setName.. called from
> Lua as:
> 
> class = cpClass:new()
> class:setName("whewp")
> 
> would end up in the callback static int luacallback(lua_State* ls)
> 
> How do I get the cpClass and the setName here so I can match the arguments
> and call the appropriate function?

You could use lua closures when you "register" your class and methods. When
those methods are called, the closure values that you assigned to them get pushed
on the stack. In your luacallback you pop and inspect the closure values to
determine what function to call.  I use a vector array of pointer to my
member functions, the indices of the array are my closure values. In my
callback I simply pop the closure value and index my vector array for the
function pointer.

-Lenny

Reply | Threaded
Open this post in threaded view
|

Re: co-routines [was Re: Unicode]

Roberto Ierusalimschy
In reply to this post by Edgar Toernig
> I can't remember why I didn't follow that way but at least one reason was, 
> that it takes to GC passes to collect all data from the threads (first to 
> collect the userdata that holds the state and another one for the objects 
> used by that thread). And it's slower. 

The only difference in the GC is that it has to traverse multiple stacks 
instead of only one. It is quite simple, and as efficient as it was (when 
there is only one stack). Notice that, in my implementation, a thread (or 
stack) is not a new type, and is not subjected to GC: The external code 
must destroy (close) it explicitily. (Of course, you can pack it inside a 
userdata, and set a GC tag method for that.) 


> I noticed two points: you need something to pass data from one state to 
> another. Your solution with lua_ref(L1)/lua_getref(L2) is IMHO not correct. 
> You may raise an error in the wrong state. 

You have to work really hard to get an error calling lua_getref. But I 
agree that it is not "polite" to do calls over a foreigner state. A 
"passvalue" function is already in my "to think" list (although I guess 
that for that case we need a "sendvalue" instead of a "receive" one). 


> Another point is that I have globals for each state.  At the moment
> (with the standard API) it seems impossible to change globals for
> other states without a new API function.

Each thread has its own "globals". lua_setglobals operate over the
given thread, so you can change globals as you like. (Again, you may
need to call that function over the "wrong" thread, but again you have
to work hard to get an error over lua_setglobals.)


> Sounds like the "Big Kernel Lock" in first Linux SMP kernels.  The problem
> was that it scaled very bad.  Basically you limit the use of Lua to one
> state at a time.

I don't think this is really a "Big Kernel Lock". Every time Lua calls C it 
unlocks, so even a loop like «while 1 do a=sin(i) end» will be preemptive. 
(OK, you cannot do busy waiting, like «while not a do end») And, as I said, 
you can register an empty call hook, so that every function call that Lua 
makes it also unlocks; or you can even register an empty line hook, so that 
no program can keep the lock for too long (with line hooks, you can even do 
busy waiting). The hooks are per/thread, so each thread can have different 
tradeoffs bettween preemptiveness and efficiency. 


> I guess, people not bothered by that could just as easy use coroutines ;-)

I also like coroutines, And, using a call hook as a dispatcher, we can get 
almost "real threads" with coroutines. But many people prefer real
trheads. But coroutines block the whole program in I/O operations (yes, you 
may use select, but not always, and not for output). Moreover (again, I 
agree this is sad) most OS offer an "official" implementation for threads, 
but not for coroutines.


> Don't you think that a handful of locks could work?  Especially the stack
> does not need a lock and I guess that's the most used location.

Even the stack needs lock, because another thread may be doing garbage 
collection. And the more locks you have, the bigger the chance of subtle 
errors; and the performance gets worst too (lock/unlock are not that 
cheap).

-- Roberto

Reply | Threaded
Open this post in threaded view
|

Re: co-routines [was Re: Unicode]

Edgar Toernig
[MULTIPLE STATES]

Roberto Ierusalimschy wrote:
>
> Notice that, in my implementation, a thread (or stack) is not a new type,
> and is not subjected to GC: The external code must destroy (close) it
> explicitily.

The same as of Sol-0.42.

> (Of course, you can pack it inside a userdata, and set a GC tag method for
> that.)

And this is exactly the case I meant.  Each coroutine is implemented as
a userdata object that holds the state.  In the first pass the userdata
is collect (but the state was still present), the gc-callback frees the
state and in the next pass the objects of the state are freed.

> > I noticed two points: you need something to pass data from one state to
> > another. Your solution with lua_ref(L1)/lua_getref(L2) is IMHO not correct.
> > You may raise an error in the wrong state.
>
> You have to work really hard to get an error calling lua_getref.

A Lua-stack overflow.  And the error will be deadly!

> But I agree that it is not "polite" to do calls over a foreigner state.

Not only "not polite".  Between two API calls (i.e. lua_getref(L2, x)
lua_setglobals(L2)) the other thread may run and you have just put him
some garbage on the stack...

> A "passvalue" function is already in my "to think" list (although I guess
> that for that case we need a "sendvalue" instead of a "receive" one).

I needed the sendvalue too.  But just during the setup phase where I'm sure
a stack overflow cannot happen (the stack is new and empty).  So I've
misused receivevalue.

> > Another point is that I have globals for each state.  At the moment
> > (with the standard API) it seems impossible to change globals for
> > other states without a new API function.
>
> Each thread has its own "globals". lua_setglobals operate over the
> given thread, so you can change globals as you like. (Again, you may
> need to call that function over the "wrong" thread, but again you have
> to work hard to get an error over lua_setglobals.)

Lua_setglobals cannot raise an error.  But you have to get the table
to the other state.  Same problem as above.  IMHO that's a generic
problem.  Maybe a function similar to luaD_runprotected is required to
perform a couple of operations atomically within the other state and it
will forward errors to the local state.



[LOCKING]

> > Sounds like the "Big Kernel Lock" in first Linux SMP kernels.  The problem
> > was that it scaled very bad.  Basically you limit the use of Lua to one
> > state at a time.
>
> I don't think this is really a "Big Kernel Lock". Every time Lua calls C it
> unlocks, so even a loop like «while 1 do a=sin(i) end» will be preemptive.
> (OK, you cannot do busy waiting, like «while not a do end») And, as I said,
> you can register an empty call hook, so that every function call that Lua
> makes it also unlocks; [...]

On SMP only 1 CPU may be within Lua.  On single CPU systems every thread
fights for the lock and you get a thread ping pong.  IMHO, it does not
scale.  But maybe most people just want pthreads and do not care about
scalability.

> I also like coroutines, And, using a call hook as a dispatcher, we can get
> almost "real threads" with coroutines.

As a bonus you get all this nasty locking stuff in the high level language :-)

> But coroutines block the whole program in I/O operations (yes, you
> may use select, but not always, and not for output).

Why not for output?  Especially in the defined environment you find in
Lua it would be reasonable (not in ANSI-C though).  And then you have
what most people call "real threads".

> Moreover (again, I agree this is sad) most OS offer an "official"
> implementation for threads, but not for coroutines.

And even more sad, the only (regular) OS I know that supports coroutines
(they call it different, though *g*) is the one I will never work with.

> Even the stack needs lock, because another thread may be doing garbage
> collection.

Uh, yes, the GC would be harder.  It wants to run exclusive in one thread
and wants to lock everything.  You would need something to synchronize all
threads...

> And the more locks you have, the bigger the chance of subtle
> errors; and the performance gets worst too (lock/unlock are not that
> cheap).

Guess why I wrote "Good luck in mutex hell" once ;-)  Afaik, locks
are reasonable fast as long as there's no contention.  But to get
everything right is IMHO kind of masochism ;-)

I guess, a lot of people are satisfied with the single lock.  I do
not need pthreads, only simple coroutines.  And these seem to work
well with the interface.

Ciao, ET.



Reply | Threaded
Open this post in threaded view
|

Re: co-routines [was Re: Unicode]

Roberto Ierusalimschy
> Not only "not polite".  Between two API calls (i.e. lua_getref(L2, x)
> lua_setglobals(L2)) the other thread may run and you have just put him
> some garbage on the stack...

This is harder still, as I have not started the other process when I call 
lua_getref. Afaiks, you cannot do anything reasonable in another thread's 
stack after it starts running (even with a function like receivevalue), 
unless if you implement some kind of randevous. You are not going to change 
another thread's global table on the fly... In my view, you create the new 
Lua stack, configure it (then you can use lua_getref, lua_setglobals, and 
others without problems), and then you start the new thread. After that, 
all communication must be through "official" channels (critical regions, 
semaphores, etc.). 


> Afaik, locks are reasonable fast as long as there's no contention.

In my Linux, just adding the macros with the locks slowdowns
"typical" Lua programs by a factor of 20% to 40% (depends how much
they call C). And this is only the main thread running, so there is
no contention.

-- Roberto

Reply | Threaded
Open this post in threaded view
|

Re: RTTI

Renato Cerqueira-2
In reply to this post by Joshua Ritter
Hi Joshua,

You may find these papers interesting:
http://www.tecgraf.puc-rio.br/luaorb/pub/doa99.ps.gz
http://www.tecgraf.puc-rio.br/luaorb/pub/iccl98.ps.gz

Regards,
Renato

-- 
------------------------------------------------------------------------
Renato Cerqueira, Ph.D.             email: [hidden email]
Research Staff Member                      [hidden email]
Tecgraf/PUC-Rio, Brazil             http://www.tecgraf.puc-rio.br/~rcerq

On leave at:
Systems Software Research Group     http://choices.cs.uiuc.edu
Department of Computer Science
University of Illinois at Urbana-Champaign
------------------------------------------------------------------------

Joshua Ritter wrote:
> 
> Ok.. I am pretty new to Lua...
> 
> I have an extensive RTTI information system built into my cpp class
> hierarchy.  This includes the variables that make up the class (for
> serialization primarily) and also the classes functions.. return
> type/parameter types/ and a functor (function pointer) to the associated
> function...
> 
> Following the example of how toLua generates bindings... I am teaching my
> system to automatically bind these classes to Lua... it is working
> fabulously...
> 
> I have run into a small snag however...
> 
> All function tags callback to the same static int luacallback(lua_State* ls)
> function I have defined... what I need to get from here is:
> 
> A) the "class" of the table
> B) the _name_ of the field (function)
> 
> so a table of "type" cpClass with a function called setName.. called from
> Lua as:
> 
> class = cpClass:new()
> class:setName("whewp")
> 
> would end up in the callback static int luacallback(lua_State* ls)
> 
> How do I get the cpClass and the setName here so I can match the arguments
> and call the appropriate function?
> 
> Thanks!
> -J

Reply | Threaded
Open this post in threaded view
|

Re: RTTI

Joshua Ritter
Thanks Renato...

I had a look into CORBA ... got pretty interested.. and then my head
_exploded_

It seems that this is very powerful method, though, also rather time
consuming... I was surprised at the amount of code in LuaOrb...

If I am mistaken on the complexity, I would like to know... what it seems to
me is that you generate some crazy IDL link between your C++ code and
LuaOrb... and that this process isn't _automated_ ?????

I really like the automated tolua program.. it gave me excellent results...
the problems I had being:

Having to keep seperate files (header,pkg) synced up...
Having to run an external program to generate the binding code
Having to adjust this automatically generated code (slightly)
If you wrap any amount of code using this method.. that is A LOT of code...
my dll was getting huge...

C++ is great if you are using only C++ it seems :)  Making a callback is
insanely difficult... Templates!  ARGH!  :)

Where I am currently at.. is I found a really cool C++ callback system (I
started with the RHickey link given on this list.. that got me so far but
not far enough)...

http://libsigc.sourceforge.net/link.html

I am writing a program to automatically generate all possible calling stubs
for all variable types... coupled with a custom "framestack".. this better
work!  After 6 years I don't want to move back to C.. :)

-J






----- Original Message -----
From: "Renato Cerqueira" <[hidden email]>
To: "Multiple recipients of list" <[hidden email]>
Sent: Tuesday, February 06, 2001 5:11 PM
Subject: Re: RTTI


> Hi Joshua,
>
> You may find these papers interesting:
> http://www.tecgraf.puc-rio.br/luaorb/pub/doa99.ps.gz
> http://www.tecgraf.puc-rio.br/luaorb/pub/iccl98.ps.gz
>
> Regards,
> Renato
>
> --
> ------------------------------------------------------------------------
> Renato Cerqueira, Ph.D.             email: [hidden email]
> Research Staff Member                      [hidden email]
> Tecgraf/PUC-Rio, Brazil             http://www.tecgraf.puc-rio.br/~rcerq
>
> On leave at:
> Systems Software Research Group     http://choices.cs.uiuc.edu
> Department of Computer Science
> University of Illinois at Urbana-Champaign
> ------------------------------------------------------------------------
>
> Joshua Ritter wrote:
> >
> > Ok.. I am pretty new to Lua...
> >
> > I have an extensive RTTI information system built into my cpp class
> > hierarchy.  This includes the variables that make up the class (for
> > serialization primarily) and also the classes functions.. return
> > type/parameter types/ and a functor (function pointer) to the associated
> > function...
> >
> > Following the example of how toLua generates bindings... I am teaching
my
> > system to automatically bind these classes to Lua... it is working
> > fabulously...
> >
> > I have run into a small snag however...
> >
> > All function tags callback to the same static int luacallback(lua_State*
ls)
> > function I have defined... what I need to get from here is:
> >
> > A) the "class" of the table
> > B) the _name_ of the field (function)
> >
> > so a table of "type" cpClass with a function called setName.. called
from
> > Lua as:
> >
> > class = cpClass:new()
> > class:setName("whewp")
> >
> > would end up in the callback static int luacallback(lua_State* ls)
> >
> > How do I get the cpClass and the setName here so I can match the
arguments
> > and call the appropriate function?
> >
> > Thanks!
> > -J
>


Reply | Threaded
Open this post in threaded view
|

Philosophy ;-)

Philippe Lhoste-2
In reply to this post by Roberto Ierusalimschy
First, thanks to Luiz for saying to me to subscribe to the TecGraf mailing
list, not the YahooGroups one. Receiving daily summaries is OK in lurk mode,
but to very suitable in interactive mode.

I have two points on licence issue I would like to talk. It is mostly
curiosity from me, in a quite theoritical level, as I don't intend to do
what I am asking (lack of time, and probably of skill).

1. Does your licence allows to use part of your code? I briefly though about
using your expression parser in another project, but I doubt it can be done
easily, as it must be deeply dependent of the whole project (ie. difficult
to isolate).

2. I like the philosophy of Lua: you have the source and you can make
proprietary changes, but if you change the syntax or behavior of the
language, don't call it Lua.
This is flexible and wise, as it avoid to have a lot of Lua dialects,
incompatibles. You can make a Lua dialect, but it is no longer Lua.
But does it work the other way around? I mean, if I write another Lua
interpreter (again, I don't have the skill to do it, and even less the
time!), which mimic exactly the Lua syntax and behavior, can I call it a Lua
interpreter?
After all, we may want to make a non-portable Lua interpreter, making
optimized system calls instead of using the C standard library, which can be
slower. Or even an assembly language interpreter! Or just rewrite it in C or
C++ for the fun of it...
Or write it in Java to allow running Lua program on browsers (hello Yindo
developpers!).
Or write it in Perl (!) to allow using Lua CGI scripts on servers like
ProHosting which allows only Perl scripts.
You get the idea.
There are many C compilers, from different sources. Interpreted languages
(Perl, Tcl, Python) have more monolithic sources. Just wondering why.
Probably because most of them (except Rebol) are open source, so they can be
improved without the need to make entirely new code.

A word about the (moderated) criticisms I saw here. I think the authors are
injustely accused of not being reactive about change suggestions. I believe
they don't work on Lua on full time, probably working on other projects. So
they may have not so much time. And time spent on heated discussions on this
mailing list is time not spent on Lua code...
And actually, they are quite helpfull on practical issues, so thank you very
much for your time and for this wonderful language.

Lua is managed in a satisfying manner. Authors do ear their users, and
doesn't hesitate to break compatibility (at least on the API level) to
improve the language, not hindered by legacy problems. Old programs can
still run on old interpreter, anyway. Still, they don't jump at all the
suggestions, otherwise the language would grow uncontrolled and would become
bloated. (My English isn't very good, I have some trouble expressing my
exact thoughts.)
A thin path, indeed.

BTW, Roberto, I am reading your book draft, and it is excellent. It is
exactly the tutorial I was looking for! Even my thick brain (I had to read
twice the chapter on upvalues and closures...) finishes to understand the
exposed concepts. I am waiting eagerly for the final version.
Your English is excellent, I have just found a typo, a '$Ta' instead of a
'a'. You probably already have corrected it.

Regards.

--._.·´¯`·._.·´¯`·._.·´¯`·._.·´¯`·._.·´¯`·._.·´¯`·._.--
Philippe Lhoste (Paris -- France)
Professional programmer and amateur artist
http://jove.prohosting.com/~philho/
--´¯`·._.·´¯`·._.·´¯`·._.·´¯`·._.·´¯`·._.·´¯`·._.·´¯`--


Reply | Threaded
Open this post in threaded view
|

Re: Philosophy ;-)

Luiz Henrique de Figueiredo
>1. Does your licence allows to use part of your code?

I think so, but I'm not sure myself.
I'll have to talk with the rest of the team and perhaps add a sentence to
the copyright notice.

>I briefly though about
>using your expression parser in another project, but I doubt it can be done
>easily, as it must be deeply dependent of the whole project (ie. difficult
>to isolate).

One *easy* way to reuse the expression parser is to use it in Lua,
as explained in the paper and in the seminar slides.
But if you want it in C, you'd probably find it easier to write your own,
perhaps in yacc. Early versions of Lua include a yacc grammar for Lua.
There's also "hoc", a simple language describe in "The UNIX Programming
Environment" and probably available in the internet (perhaps at the authors
sites). And many others very small languages (eg. etalk and bob).

>But does it work the other way around? I mean, if I write another Lua
>interpreter (again, I don't have the skill to do it, and even less the
>time!), which mimic exactly the Lua syntax and behavior, can I call it a Lua
>interpreter?

Yes, definitely! I'd love to see this as I think this would take Lua to
a different level as an influential language. Do Perl or Python or TCL have
more than one implementation? But Scheme does.

>A word about the (moderated) criticisms I saw here. I think the authors are
>injustely accused of not being reactive about change suggestions. I believe
>they don't work on Lua on full time, probably working on other projects. So
>they may have not so much time.

That's right, but Lua is a major concern for us and is probably Roberto's main
project (as he says in his web site). Roberto is definitely the leader of the
team -- he alone makes changes to the code and thinks about most changes to
Lua. I take care of luac, the web site, the mailing list, etc.  Waldemar takes
cares of tools and the use of Lua in TeCGraf. But we three have academic
careers that need close attention. (Roberto is the language expert; Waldemar
and I are mostly graphics people.) Despite other projects and papers to write,
we have frequent (and heated) discussions about the future of Lua, many times
motivated by postings to this list.

So, yes, we do hear what everyone says here, but not everything can be added
to Lua, some for technical reasons, some simply because of taste.
Plus lua-l has reached a size where other people can answer questions.
There are many knowledegable people here, and not only about Lua!

>And time spent on heated discussions on this
>mailing list is time not spent on Lua code...

Oh, yeah! With 120 messages just this month (that's one every 2 hours!),
it's hard to read them all closely, let alone answer them. (we had network
problems in the past couple of days and it was a slight relief that lua-l was
slient for a while... :-)

But again, we do take notice of what goes on here.
We just have to keep focused on what we want to do with Lua.
One thing is certain: we want it small and as simple as possible,
but not simpler (Einstein? :-).

That was a long message. Let's go back to Lua now :-).
--lhf

Reply | Threaded
Open this post in threaded view
|

Re: Philosophy ;-)

Cary O'Brien
[snip]

> >But does it work the other way around? I mean, if I write another Lua
> >interpreter (again, I don't have the skill to do it, and even less the
> >time!), which mimic exactly the Lua syntax and behavior, can I call it a Lua
> >interpreter?
> 
> Yes, definitely! I'd love to see this as I think this would take Lua to
> a different level as an influential language. Do Perl or Python or TCL have
> more than one implementation? But Scheme does.

FWIW...

There are currently (at least) two python implementations, on in C
(CPython, or 'regular' python) and Jython (nee JPython) which is
implemented in Java, and (I think) generates java bytecodes.

I think there is a Java implementation of Tcl called either Jacl or TCL
Blend.  I forget which one.

Another example is prolog -- there are many prolog interpreters -- in
fact I was searching for a good one for my daughter last night.

-- cary

Reply | Threaded
Open this post in threaded view
|

Natural Selection

Joshua Ritter
I am posting this for anyone else lurking out there.. debating on which
scripting language to use... *warning a brain dump follows*

I have been "investigating" Python and Lua for almost a year... finally,
some of the deadlines have lightened up, so I am in the process of actually
_doing_ something about all this...

Some reasons we are going with Lua:

We have non-programmers scripting .. Lua is the only choice as far as I am
concerned on this point... and I applaud the language design for it's
power... with lack of complexity... I would say though.. if you have
experienced programmers on a project.. and they are doing the scripting..
they may get to missing some features...

Ease of wrapper classes... I was just reminded about Python's extension
mechanism... a bit of a beast... well.. my classes are automatically exposed
to Lua... I have _one_ callback for functions, _one_ callback for a new
operator, _one_ callback for getting data, and _one_ callback for setting
data ... I used a little trickery here... using c++ callback templates I
generated calling stubs for all possible calling parameter combos (typecasts
from void)... I wrote a program to do this for me of course... all told
about 600k of source :)  I had to stop at 2 parameters(3 megs of source for
3 parameters)... _but_ I can add 3 parameter function templates by hand...
not a big deal as there are surprisingly few of those... also keeping in
mind that nonprogrammers can't deal with tons of parameters anyway... being
able to call ANY cpp member function from a void ptr is an INCREDIBLE
feeling of power.. I don't know if I can ever go back to normal coding
practices  :)

Wrapper classes continued... Lua helped make this wrapper method possible in
that it is so CONSISTANT.

I have seen benchmarks and Llua is quite fast... I have heard nothing to the
contrary...

I suppose these are the big three.. ease of learning for nonprogrammers and
programmers(hey some stuff you can do with Lua would be a chore in C++),
ease of wrapping for programmers, and execution speed...

And this isn't to mention memory footprints and everything else that goes
into this stuff!

-J
















Reply | Threaded
Open this post in threaded view
|

Stackless and coroutines and stuff

Joshua Ritter
In reply to this post by Cary O'Brien
I have been looking long and hard at stackless python... it isn't python
that I am interested in.. it is the execution model of stackless...

continuations, microthreads

I know that I can come up with an acceptable system outside of using
"microthreads"... from a design standpoint they would really simplify things
however.. and when I think of the people writing scripts modeling
independant behaviors I get to drooling...

The thought has crossed my mind to take Stackless... and call Lua code from
it...

Now wouldn't this be a bizarre experiment? :)

-J










Reply | Threaded
Open this post in threaded view
|

Re: Philosophy ;-)

Magnus Lie Hetland
In reply to this post by Cary O'Brien
> FWIW...
> 
> There are currently (at least) two python implementations, on in C
> (CPython, or 'regular' python) and Jython (nee JPython) which is
> implemented in Java, and (I think) generates java bytecodes.
> 
> I think there is a Java implementation of Tcl called either Jacl or TCL
> Blend.  I forget which one.
> 
> Another example is prolog -- there are many prolog interpreters -- in
> fact I was searching for a good one for my daughter last night.

FYI: I'm working on a Java implementation of Lua. It's been a while
since I've done any serious work on it, but I have the parser and
a big chunk of the interpreter engine finished... When (or if) I
have something that works, I will post a message here.

(See also http://www.jlua.org)

> 
> -- cary


--

  Magnus Lie Hetland      (magnus at hetland dot org)

 "Reality is what refuses to disappear when you stop
  believing in it"                 -- Philip K. Dick



123