On lua5 and embedding

classic Classic list List threaded Threaded
35 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Re: On lua5 and embedding

Björn De Meyer
Luiz Henrique de Figueiredo wrote:
> 
> It worries me that some people seem to have a negative impression of Lua 5
> so near to it being officially released. Someone even said that we have
> "seriously damaged Lua functionality"! Sure, Lua 5 is different from Lua 4.
> The metamethod scheme is not the same as the tagmethod scheme, but it is
> simpler and more flexible, even if sometimes you have to use proxy tables
> for doing some stuff that was easy in Lua 4. Lua 5 was the outcome of a long
> sequence of work versions of Lua 4.1 and has gone through alpha and now beta
> stages. I'd think major design flaws would be found by now :-(
> --lhf

I think the problem is that these new features are not well documented.
lua_boxpointer()/lua_unboxpointer() is a good API, however,
it's currently only documented here in the newsgroup. 

As for the __get and __set issue, proxy tables can
indeed be used, however, it would be nice if there was 
an example in the official distribution. And 
some documentation and explanation in the manual 
on how __index and __newindex work, and on how to 
set up a proxy table. A Lua API would even be nicer, but
I think better documentation is what Lua 5 needs now the most. 

 



-- 
"No one knows true heroes, for they speak not of their greatness." -- 
Daniel Remar.
Björn De Meyer 
[hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: On lua5 and embedding

Björn De Meyer
In reply to this post by Luiz Henrique de Figueiredo
Luiz Henrique de Figueiredo wrote:
> 
> >As for the __get and __set issue, proxy tables can
> >indeed be used, however, it would be nice if there was
> >an example in the official distribution.
> 
> See readonly.lua, trace-globals.lua, and undefined.lua in test/ .
> They use limited proxies (not totally empty tables), though.
> 

As those weren't to my liking, I wrote my own example
which follows below, which allows the metamethods
__get and __set to be used.  The idea is that you use 
the function makeproxy() directly after setting the 
metatable and use that proxy as a replacement for the 
real table. I haven't tested the other methods like 
__add, __sub, etc but it should also work for those metamethods.
Comments and improvements are welcome. I
would especially appreciate Jimmyp_gr's comments,
as he requested such functionality.

                                                                                                    
__proxy__index    = function
(t,i)                                                                  
local meta = getmetatable
(t)                                                                       
   if 
meta.__get                                                                                   
   then return
meta.__get(meta.target,i)                                                            
   else return meta.target[i];
end                                                                  
end                                                                                                 
                                                                                                    
__proxy__newindex    = function
(t,i,v)                                                             
local meta = getmetatable
(t)                                                                       
   if
meta.__set                                                                                    
   then
meta.__set(meta.target,i,v)                                                                 
   else meta.target[i] =
v                                                                          
  
end                                                                                              
end                                                                                                 

                                                                                  
function
makeproxy(tab)                                                                             
local proxy,
meta                                                                                   
meta =
getmetatable(tab)                                                                            
proxy = {  }; -- always
empty                                                                       
proxymeta = {
}                                                                                     
if
meta                                                                                             
  then for k, v in meta do 
rawset(proxymeta,k,v)                                                   
 
end                                                                                               
end                                                                                                 
-- copy the metatable, to get the
operators.                                                        
proxymeta.__index    =
__proxy__index                                                               
proxymeta.__newindex =
__proxy__newindex                                                            
-- override these
two.                                                                              
proxymeta.target =
tab;                                                                             
-- so we know whom we are
proxying!                                                                 
setmetatable(proxy,
proxymeta);                                                                     
return proxy;           
end                                                                                                 
                                                                                                    
-- example metatable, with the much desired __get and __set
methods.                                                                                                    
                                                                                                    
tabmeta =
{                                                                                         
__get = function(t,i) print "__get called!" return rawget(t, i)
end,                                
__set = function(t,i,v) print "__set called!" rawset(t, i,v)
end,                                   
__call = function() print "__call called!"
end,                                                     
__unm = function() print "__unm called!"
end                                                        
}                                                                                                   
                                                                                                    
mytab =
{                                                                                           
   first = "first
string"                                                                           
}                                                                                                   
                                                                                                    
setmetatable(mytab,tabmeta)                                                                         
mytab =
makeproxy(mytab)                                                                            
-- this is how "makeproxy" should be called.

-- some tests  
-- __get and __set from metatable should be
called.                                                                                                  

print(mytab.first)                                                                                  
mytab.first = "First string
again!"                                                                 
print(mytab.first) 
print(mytab.second)                                                                                 
mytab.second = "Second
line"                                                                        
print(mytab.second)                                                                                 
mytab.second = "Second line
again!"                                                                 
print(mytab.second)                                                                                 
mytab();                                                                                            
fake = -mytab;          



-- 
"No one knows true heroes, for they speak not of their greatness." -- 
Daniel Remar.
Björn De Meyer 
[hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: On lua5 and embedding

Thomas Wrensch-2
In reply to this post by Dimitris Papavasileiou
>>> [hidden email] 01/29/03 03:07 AM >>>
.. snipped real message, left comment...

Frequently the problem is not performance, but the perception people
have about performance.


Oh yeah. One thing I haven't seen is any real numbers on the overhead of
doing lookups. Here are a few that might help..

A little timing test I like to do to get a rough idea about the speed of
Lua on a machine is this one:

    function fact(n)
        if n<=1 then 
          return 1 
        else 
          return n*fact(n-1)
        end
    end
    do local t=os.clock()
       for i=1,1000000 do fact(10) end
       print(os.clock()-t)
    end

On my iBook 500MHz OSX 10.2 I get 8.48 seconds

If we modify this so factorial is done this way:

x = {}
function x:fact(n)
    if x<=1 then
      return 1
    else
      return n*self:fact(n-1)
    end
end

I modified the timing code to use x:fact(10) and the time was slightly
FASTER at 8.44 seconds.

Now make x into a metaclass and use the __index as a table trick:

y = {}
x.__index = x
setmetatable(y,x)

Change the timing code to use y:fact(10) and I get 11.77 seconds.
Definitely a slowdown, but not too bad.

I'd like to send you some timing for making __index a function, but I
have a class to teach in 4 minutes.

  - Tom

Reply | Threaded
Open this post in threaded view
|

Re: On lua5 and embedding

Luiz Henrique de Figueiredo
In reply to this post by Dimitris Papavasileiou
>Lua on a machine is this one:
>
>
>I modified the timing code to use x:fact(10) and the time was slightly
>FASTER at 8.44 seconds.

Probably because now you're indexing a local variable ("self")...

Try
    local function fact(n)

in the original code...
--lhf

Reply | Threaded
Open this post in threaded view
|

RE: On lua5 and embedding

Bilyk, Alex
In reply to this post by Dimitris Papavasileiou
This is a quick test with Lua4 vs. Lua5 (the script is at the very bottom). Both executables are in release built generated by VC6 under Win32@Win2000. Here are the outputs

Lua4 (local f) - the function f was declared as local
0.625 
0.75   
Lua5 (local f)
0.813
1.015
Lua4 (global f) - the function f was NOT declared as local
0.75  
0.75  
Lua5 (global f)
0.843 
1.032 

Aside from the fact that Lua4 is simply faster on function calls by up to 30% (.625 vs. 813), tag-method calls have the same performance as global function calls in Lua4. In Lua5, however, the meta-method has another 22% overhead over global function calls. The tag methods of Lua4 are 37% faster than meta-mehods in Lua5. 

Is the test good enough? If it is I am even more surprised than I had expected. I wonder how other platforms would measure on this.

Of course this test doesn't imply that real world applications will suffer from these differences. But it does illustrate my point and beyond. I would imagine that on any custom class hierarchy (inheritance chains) created with Lua5 the slowdown would be noticeable. If someone wanted to emulate "__get"/"__set" using proxy tables the difference to tag-methods performance would have been even greater.

AB
-----Original Message-----
From: Roberto Ierusalimschy [[hidden email]]
Sent: Wednesday, January 29, 2003 1:28 AM
To: Multiple recipients of list
Subject: Re: On lua5 and embedding 

> Basically, as long as one runs plain vanilla Lua5 scripts they will
> go pretty fast. But as soon as one starts using inheritance, call and
> other meta-methods on their objects that is when the meta-method
> overhead jumps in.

Can you give us your performance measures and details of your tests
(platform, etc.)? In our own tests, we did not detect any significant
difference between tag methods in Lua 4.0 and metamethods in Lua 5.0b.

-- Roberto

-- ====== "__call" / "function" event timing script ==============================
local f = function ()
--f = function ()
end

TAG = newtag()
settagmethod(TAG, "function", f)
local t = {}
settag(t, TAG)
--local mt = {__call = f}
--setmetatable(t, mt)

c = clock()
for i = 1, 100000 
do
   f()
   f()
   f()
   f()
   f()

   f()
   f()
   f()
   f()
   f()

   f()
   f()
   f()
   f()
   f()

   f()
   f()
   f()
   f()
   f()

   f()
   f()
   f()
   f()
   f()

   f()
   f()
   f()
   f()
   f()
end
print(clock()-c)

c = clock()
for i = 1, 100000 
do
   t()
   t()
   t()
   t()
   t()

   t()
   t()
   t()
   t()
   t()

   t()
   t()
   t()
   t()
   t()

   t()
   t()
   t()
   t()
   t()

   t()
   t()
   t()
   t()
   t()

   t()
   t()
   t()
   t()
   t()
end
print(clock()-c)


Reply | Threaded
Open this post in threaded view
|

Re: On lua5 and embedding

Thomas Wrensch-2
In reply to this post by Dimitris Papavasileiou
>>> [hidden email] 01/29/03 11:00 AM >>>
>>I modified the timing code to use x:fact(10) and the 
>>time was slightly FASTER at 8.44 seconds.

> Probably because now you're indexing a local variable 
> ("self")...
>
> Try
>     local function fact(n)
> 
> in the original code...
> --lhf

Actually, I think there's more to it. While 'self' is a  local variable,
it is a table itself, and 'fact' is looked up in that table.

Moreover, the 'self' table is an empty table. Each lookup of 'fact'
should require:

    lookup 'fact' in 'self' which fails
    check for a metatable, which exists
    lookup __index in the metatable
    since __index is a table, lookup 'fact' there

I assumed the small increase in speed was due to the size of the tables
involved. Or am I missing something?

   - Tom


Reply | Threaded
Open this post in threaded view
|

Re: On lua5 and embedding

Dimitris Papavasileiou
In reply to this post by Björn De Meyer
I checked out the proxy code.Interesting enough although I knew about
proxy tables from previous threads(and am currently using
them).Allright I definately was overreacting about the damaged
functionality part.It seems to me now that __get and __set in the
language itself may even be considered redundant if they can be
implemented using lua itself.Unless they are faster if done internally
but that remains to be seen... Now how about a unary plus method for
symmetry?:)



Reply | Threaded
Open this post in threaded view
|

Re: On lua5 and embedding

Andrzej Borsuk
In reply to this post by Luiz Henrique de Figueiredo
Hello,
Luiz Henrique de Figueiredo wrote:

You can "box" them inside a heavy userdata. C pointers that do not need GC
methods can be stored in light userdata.

For discussions of heavy x light userdata, search the lua-l archives for
lua_newuserdata. For instance:
  http://lua-users.org/lists/lua-l/2002-11/msg00062.html
  http://lua-users.org/lists/lua-l/2002-08/msg00162.html
After last discussion about boxing pointers I have wrote function to emulate old lua-4 behavior (lua_pushuserdata). It's based on your proposal about using weak tables and it was pretty easy to implement :-)

This function makes possible box the same pointer twice (or more) and then compare it from lua. If you assign metatable with __gc function you are able to use GC for any C data..

#define bos_unbox lua_unboxpointer
void bos_box(lua_State *L, void *data)
{
   lua_pushstring(L,"BOXER");            /* "BOXER" */
   lua_rawget(L,LUA_REGISTRYINDEX);      /*  BOXER? */
   if(lua_isnil(L,-1)) {
       lua_remove(L,-1);
       lua_newtable(L);                  /* {} */

       /* lua_setmode(L,-1, "kv"); */
       {
           lua_newtable(L);              /* {} meta */
           lua_pushstring(L, "__mode");  /* {} meta "__mode" */
           lua_pushstring(L, "kv");      /* {} meta "__mode" "kv" */
	    lua_settable(L, -3);          /* {} meta */
           lua_setmetatable(L,-2);       /* {} */
       }
lua_pushliteral(L,"BOXER");
       lua_pushvalue(L,-2);
       lua_settable(L,LUA_REGISTRYINDEX);
   }
lua_pushlightuserdata(L,data); /* BOXER, data */
   lua_gettable(L,-2);                   /* BOXER, udata? */
   if(lua_isnil(L,-1)) {
       lua_remove(L,-1);
       lua_boxpointer(L,data);           /* BOXER, udata */

       lua_pushlightuserdata(L,data);    /* BOXER, udata, data */
       lua_pushvalue(L, -2);             /* BOXER, udata, data, udata */
       lua_settable(L,-4);               /* BOXER, udata */
   }

   lua_remove(L,-2);                     /* udata */
}


Best Regards,
Andrzej Borsuk

ps.
Last time I was wrote that lua sollution is very limited, I was wrong! After some looking into Lua sources, I have changed my mind. Current Lua sollution is very clean and simple -- its much better than prevoius.

--
    _
   (;[   SPAM?
  \/>)   Proszę wysyłać na moje podstawowe konto
 `ab~^   [hidden email]



Reply | Threaded
Open this post in threaded view
|

Re: On lua5 and embedding

Roberto Ierusalimschy
In reply to this post by Bilyk, Alex
> This is a quick test with Lua4 vs. Lua5 (the script is at the very bottom).
> [...]

I did exactly the same tests on my machine (gcc 2.96/Linux 2.4):

Lua4 (local f) - the function f was declared as local
0.33
0.43
Lua5.0b (local f)
0.33
0.44
Lua4 (global f) - the function f was NOT declared as local
0.38
0.43
Lua5.0b (global f)
0.41
0.49

The largest difference was 14%. The difference in the case that used
tag methods with f local was 2%. (BTW, don't ask me why there is any
difference between f local or global when using metatables; the second
loop does not use the variable "f" at all! This only shows how tests can
be sensitive to details.)

Then, I changed the test a little, so that "f" makes a minimum of work:

  f = function ()
    return 2+3
  end

Now the second case (global f) results in

lua4.0
0.46
0.66
lua5.0b
0.44
0.56

(Again, don't ask me why the slowdown is not the same with and without
tag methods/metatables...)


> Aside from the fact that Lua4 is simply faster on function calls by up to 30%

This is one point I do not argue (although, as the tests sugest, this
difference is quite smaller when the function does anything). But this
slowdown is a consequence of coroutines, not of metatables. If someone
argues that we shold remove coroutines from the language so it would run
faster, then we have a very different scenario.

-- Roberto

Reply | Threaded
Open this post in threaded view
|

Re: On lua5 and embedding

C.S. Brooks
As a professional game developer, here's my 2 cents:

We're using Lua 5 extensively in a computer game here, and it really doesn't
seem to be having a negative impact on performance at all. The speed seems
more than adequate for the task of game programming, especially since the
C/C++ part of your code should really be doing the "heavy lifting" anyway.

The functionality added by coroutines more than makes up for any slight
performance cost. The real issue when you use a scripting language is saving
development time, not increasing performance, anyway. The parts of the code
we have written in lua are simply *not* where most of the cpu cycles are
spent in our game. (And we're writing *all* game-specific code in Lua.) And
coroutines have made the code fit the problem set so much better, and the
resulting code much shorter.

The only performance issue I think we might've run into is garbage
collection, and we were able to work around that by pooling the Lua objects
we were creating. (And that was so quick to code, I haven't really had time
to go back and verify that the gc was, in fact, the problem there.)

I feel like I've been reading an unusual amount of complaining by users on
the list lately, which I don't think is justified. Lua is, by any measure,
an incredible piece of work, and has saved us a *ton* of time on this
project, without a doubt, and made the development a lot more fun, to boot.
And if anyone is really unhappy, I'm sure you can get a full refund. ;)

Anyway, keep up the good work!

--
C.S. Brooks
Professor Fog's Workshop
http://www.fogsworkshop.com


----- Original Message -----
From: "Roberto Ierusalimschy" <[hidden email]>
>
> This is one point I do not argue (although, as the tests sugest, this
> difference is quite smaller when the function does anything). But this
> slowdown is a consequence of coroutines, not of metatables. If someone
> argues that we shold remove coroutines from the language so it would run
> faster, then we have a very different scenario.
>
> -- Roberto
>


Reply | Threaded
Open this post in threaded view
|

Re: On lua5 and embedding

Björn De Meyer
In reply to this post by Roberto Ierusalimschy
Roberto Ierusalimschy wrote:
/snip
> This is one point I do not argue (although, as the tests sugest, this
> difference is quite smaller when the function does anything). But this
> slowdown is a consequence of coroutines, not of metatables. If someone
> argues that we shold remove coroutines from the language so it would run
> faster, then we have a very different scenario.
> 
> -- Roberto

That's a slowdown I'll gladly accept. I have just discovered how
elegant coroutines are for event-oreinted programming... 
Still, maybe there are some people who'd like to disable 
their coroutines?  For those people, perhaps some 
#define LUA_NOCOROUTINES could be implemented to allow 
turning off the coroutines?  


-- 
"No one knows true heroes, for they speak not of their greatness." -- 
Daniel Remar.
Björn De Meyer 
[hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: On lua5 and embedding

Peter Hill-2
In reply to this post by Roberto Ierusalimschy
>> Aside from the fact that Lua4 is simply faster on function calls by up
>> to 30%

Roberto Ierusalimschy:
> This is one point I do not argue (although, as the tests sugest, this
> difference is quite smaller when the function does anything). But this
> slowdown is a consequence of coroutines, not of metatables. If someone
> argues that we shold remove coroutines from the language so it would run
> faster, then we have a very different scenario.

I (not being aware of how the internals of your co-routines are implemented)
am wondering why having coroutines now makes general Lua function calls
slower?

*cheers*
Peter Hill.



Reply | Threaded
Open this post in threaded view
|

Re: On lua5 and embedding

Roberto Ierusalimschy
> I (not being aware of how the internals of your co-routines are implemented)
> am wondering why having coroutines now makes general Lua function calls
> slower?

I guess the main reason is that we had to unroll recursive calls (in the
C code) into loops with an explicit stack. Several local variables in
luaV_execute were automatically saved/restored when that function called
itself recursively; now we must save/restore them manually. The internal
C stack is faster than any external stack that we can code (despite some
books trying to convince you otherwise).

Also, the call/return code is a little more complex, because there are
more different cases to handle (e.g. a "return" may be a real C return,
if the function was called from C, or a stack pop, if the function was
called from Lua).

-- Roberto

Reply | Threaded
Open this post in threaded view
|

Re: On lua5 and embedding

Peter Hill-2
In reply to this post by Björn De Meyer
Luiz Henrique de Figueiredo:
> It worries me that some people seem to have a negative impression of Lua 5
> so near to it being officially released. Someone even said that we have
> "seriously damaged Lua functionality"!

It's a PR problem. They need to know that Lua5 is not as different from Lua4
as it seems. They need to be reassured by having each change explained. For
example, lexical scoping looks (from the outside) like it would be much less
efficient than upvalues. An implementation description describing why it
isn't would be nice.

> Sure, Lua 5 is different from Lua 4. The metamethod scheme is not the same
> as the tagmethod scheme, but it is simpler and more flexible, even if
> sometimes you have to use proxy tables for doing some stuff that was easy
> in Lua 4. Lua 5 was the outcome of a long sequence of work versions of Lua
> 4.1 and has gone through alpha and now beta stages. I'd think major design
> flaws would be found by now :-(

Björn De Meyer:
> I think the problem is that these new features are not well documented.
> lua_boxpointer()/lua_unboxpointer() is a good API, however, it's currently
> only documented here in the newsgroup.

I agree. I feel there are a reasonable number of Lua users (probably the
majority) that may be wanting to try Lua4's soon-to-be replacement Lua5
without being active members of the mailing list. They've missed months
(years?) of discussion and are seeing the "new way" for the first time from
a purely Lua4 perspective.

There should be a "design notes" section of the manual, even (hmm, no,
*especially*) in the beta stage. It should list the changes with:

(a) examples & discussion on how the change relates to Lua4 equivalents
(especially taking into account typical preconceptions of someone who has
not been following the list). Eg, metatables look, at a casual glance, to be
very different from tags... but are actually almost exactly the same thing.
Eg, the new "for" statement looks to be very different but is just an
expansion of the old method.

(b) the rationale of *why* such a change was implemented (and considered
desirable), and how it might affect performance.

(c) migration paths, including both changes to source plus wrappers.

This should be in the manual... separate code examples are not enough.


Björn De Meyer:
> As for the __get and __set issue, proxy tables can indeed be used,
> however, it would be nice if there was an example in the official
> distribution. And some documentation and explanation in the manual on how
> __index and __newindex work, and on how to set up a proxy table. A Lua API
> would even be nicer, but I think better documentation is what Lua 5 needs
> now the most.

This is just one of many examples that such documentation should handle.


Joe Stewart:
> Has anyone made a 4 to 5 migration guide?
>
> Would there be interest in such a guide? As I convert my lua 4 scripts to
> run under lua 5, I'll be compiling my own list. I'd be happy to put it on
> lua-users as I go.

Sounds good to me. Try to include some rationales (not just examples) about
the changes. And if you format it in the same style as the official manual
it might even get included???

*cheers*
Peter Hill.



Reply | Threaded
Open this post in threaded view
|

Re: On lua5 and embedding

Enrico Colombini
On Saturday 01 February 2003 08:04, Peter Hill wrote:
> There should be a "design notes" section of the manual, even (hmm, no,
> *especially*) in the beta stage. It should list the changes with:

I think that a separate document ("Moving from Lua 4 to Lua 5") aimed at Lua 4 
programmers and explaining in detail new features, differences, pitfalls, 
performance tips (etc.) would do wonders for a faster adoption of lua 5.

I know very well that such a document is not easy or amusing to write, but it 
would be *really* useful. At least, for me it will certainly be.

  .Erix.

12