index tag method

classic Classic list List threaded Threaded
33 messages Options
12
Reply | Threaded
Open this post in threaded view
|

index tag method

John Belmonte-2
The index tag method is a sub-case of gettable and can be removed, reducing
the complexity of tag methods and slimming down luaV_gettable in the virtual
machine.  If the user wants to treat a previously nil  (or any other value
or type for that matter) index as a special case they can do it with
gettable, so why go as far as having this in the VM which slows down all
table reads?

At least for Sol this would make a lot of sense since 1) tag method
backwards compatibility was already tossed and 2) global reads go through
normal table access so the VM's table read is even more of a workhorse than
Lua.

-John



Reply | Threaded
Open this post in threaded view
|

Re: index tag method

Steve Dekorte-4
John Belmonte wrote:
> The index tag method is a sub-case of gettable and can be removed, reducing 
> the complexity of tag methods and slimming down luaV_gettable in the virtual 
> machine.  If the user wants to treat a previously nil  (or any other value 
> or type for that matter) index as a special case they can do it with 
> gettable, so why go as far as having this in the VM which slows down all 
> table reads? 

I think the most common use of a hook on table access is to implement
inheritance. In that case, you only care about look ups on missing table
elements. Having to handle all look ups just to catch the nils would be 
very slow, particularly if your hook is implemented in Lua. 

Steve



Reply | Threaded
Open this post in threaded view
|

Re: index tag method

Edgar Toernig
In reply to this post by John Belmonte-2
Hi,

John Belmonte wrote:
> 
> The index tag method is a sub-case of gettable and can be removed, reducing
> the complexity of tag methods and slimming down luaV_gettable in the virtual
> machine.  If the user wants to treat a previously nil  (or any other value
> or type for that matter) index as a special case they can do it with
> gettable, so why go as far as having this in the VM which slows down all
> table reads?
> 
> At least for Sol this would make a lot of sense since 1) tag method
> backwards compatibility was already tossed and 2) global reads go through
> normal table access so the VM's table read is even more of a workhorse than
> Lua.

Hmm...  And I've just added a "newindex" method to Sol ;-)  It's
similar to "index" but called from settable if a new element is
created.

IMHO these two methods are too useful for general Lua/Sol programming
(fallback methods/inheritance, default elements, caches, ...).  In
contrast set/getglobal is mostly used for debugging and set/gettable
for table simulation - nothing that I would use for a typical program-
ming task.  If it were just about minimalism you'd have to remove the
table operators (. : []), the for-statement, the constant initializers,
and so on.

About performance: Yes, Lua4/Sol is slightly slower at globals than
Lua3.2.  But I don't think you can measure a difference between Sol
and Lua4 in any real application (and even if you could, I'm not sure
who is faster :-).  I guess, removing the "index" test would give
below 1% difference in execution speed if you do nothing else than
table lookups.

Ciao, ET.

Reply | Threaded
Open this post in threaded view
|

Re: index tag method

John Belmonte-2
In reply to this post by Steve Dekorte-4
Steve Dekorte wrote:

> I think the most common use of a hook on table access is to implement
> inheritance. In that case, you only care about look ups on missing table
> elements. Having to handle all look ups just to catch the nils would be
> very slow, particularly if your hook is implemented in Lua.

I see your point about that being a common use.  Tag methods represent very
low level events (after all they are called from the VM), so my view is if
one even thinks the word "performance" you have to implement frequently used
hooks in C, not Lua.  So it's a question of how fast the VM can call a
C-side function.

Edgar Toernig wrote:
> IMHO these two methods are too useful for general Lua/Sol programming
> (fallback methods/inheritance, default elements, caches, ...).  In
> contrast set/getglobal is mostly used for debugging and set/gettable
> for table simulation - nothing that I would use for a typical program-
> ming task.

I think set/gettable are important for general use.  You mentioned wanting
to work on namespaces/modules... do you plan to implement them by adding to
Lua's syntax and semantics?  I see it being done without change to Lua
(except for removing set/getglobal which is backwards compatible), as
set/gettable tag methods coded in C.  But I think some work needs to be done
reducing the hook call overhead.  Removing the index tag method was one
idea... but it's not enough, not popular, and can't be done with backwards
compatibility.  Looking elsewhere...

Thanks,
-John



Reply | Threaded
Open this post in threaded view
|

Re: index tag method

Reuben Thomas-3
> namespaces/modules... do you plan to implement them by adding to
> Lua's syntax and semantics?

I think that modules should involve new syntax, preferably just a new
reserved word or two. Otherwise, I suspect they won't end up forcing
themselves on the programmer enough to be used (is this a problem in Perl?).

While it should certainly remain possible to write non-modular Lua (for
quick scripting), it should ideally work by making scripts a special case.
For example, in Haskell, the equivalent to C's main function is the Main
module (every program must have one). The Main module is the only one that
doesn't need a module declaration, so if the compiler finds a module without
a declaration, it assumes it's main.

This sort of defaulting gives backwards compatibility (with the exception of
new reserved words) while allowing the new syntax to give a definite feel to
modules when they are being used.

-- 
http://sc3d.org/rrt/ | plagiarism, n.  the mind burgles


Reply | Threaded
Open this post in threaded view
|

Re: index tag method

John Belmonte-2
Hi Reuben,

> I think that modules should involve new syntax, preferably just a new
> reserved word or two. Otherwise, I suspect they won't end up forcing
> themselves on the programmer enough to be used (is this a problem in
Perl?).
>
> While it should certainly remain possible to write non-modular Lua (for
> quick scripting), it should ideally work by making scripts a special case.

I think Python's way could be implemented with Lua's extensions (again with
improvements).  There is nothing special to do when writing modules.  The
globals of an imported file become the names within the module.

-John



Reply | Threaded
Open this post in threaded view
|

Re: index tag method

Reuben Thomas-3
> I think Python's way could be implemented with Lua's extensions (again with
> improvements).  There is nothing special to do when writing modules.  The
> globals of an imported file become the names within the module.

This certainly sounds like a good way to start (and means that there's at
least a backwards compatible version of modules). However, it can be useful
to say what you want to import from a given module (I suppose you can do
that too without extra syntax), and also to restrict what you export (i.e.
allow private globals).

All this can be done with a certain amount of self-discipline, but then
modules themselves are just a way of controlling the namespace; if all that
was needed was self-discipline we wouldn't be having this discussion in the
first place.

-- 
http://sc3d.org/rrt/ | humour, n.  unexpected recognition


Reply | Threaded
Open this post in threaded view
|

Re: index tag method

Edgar Toernig
In reply to this post by John Belmonte-2
Hi,

John Belmonte wrote:
> 
> I think set/gettable are important for general use.  You mentioned wanting
> to work on namespaces/modules... do you plan to implement them by adding to
> Lua's syntax and semantics?

Syntax: No.  I think that the elements present in Sol are sufficient.
Semantics: Yes.  IMHO some changes (methods) would make it better.

> I see it being done without change to Lua (except for removing set/getglobal
> which is backwards compatible), as set/gettable tag methods coded in C.

I don't think that modules have to be implemented in C.  This can be done
in Lua/Sol.  You fear the overhead.  And that's the point: I think that
the method concept of Sol would make it more efficient and cleaner than
any implementation with the current Lua API.

Concept: Modules exports one table (or more).  This table holds exported
functions and data objects (File.open, File.stdout, Socket.connect).  Objects
created by the module get attached method tables so that each objects may
be used to call further module functions (fp:write, sock:send, or fp:close
and sock:close).  The difference between Lua and Sol is that in Sol you
get all this as a basic (and efficient) concept of the language whereas in
Lua you have to do some (IHMO) ugly hacks to do that.  (For an implementation
example lock at the luasocket library for Lua and siolib.c for Sol.)

> But I think some work needs to be done reducing the hook call overhead.
> Removing the index tag method was one idea...

It's the actual call and execution of the tag method that generates an
overhead.  Not the check whether that method exists.  And index/newindex
- contrary to gettable/settable - help to avoid calls.

Ciao, ET.



Reply | Threaded
Open this post in threaded view
|

Re: modules (was: index tag method)

Edgar Toernig
In reply to this post by Reuben Thomas-3
Reuben Thomas wrote:
>
> All this can be done with a certain amount of self-discipline, but then
> modules themselves are just a way of controlling the namespace; if all that
> was needed was self-discipline we wouldn't be having this discussion in the
> first place.

I think that exactly this is what is required: A discipline on how to
write modules.  You cannot perform self-discipline if you don't know
what the discipline is ;-)  IMHO there has to be a definition of a
standard execution environment.  What can modules expect to exist, how
they are supposed to export their functionality, how are modules loaded
and used.  In languages like Modula this is build right into the language.
In Lua no such policies exist.  Everybody invents his own concept.  And
this has two major drawbacks: first, you "have to invent" which may be more
difficult than you want and then you may get problems of module interaction
because of different ideologies (conflicting use of shared resources, in-
compatible calling conventions, dependency handling, ...).

If Lua is simply used as a scripting extension of a larger project all
this is normally not required.  The project defines the environment.
But if you want to use it as a standalone extendable language some
policies are required.  And then part of the design is the question:
"we enforce this policy" or "it only works with self-discipline".
(I would always choose the later ;-)

Ciao, ET.



Reply | Threaded
Open this post in threaded view
|

Re: modules (was: index tag method)

Luiz Henrique de Figueiredo
>I think that exactly this is what is required: A discipline on how to
>write modules.

That's a good topic for a LTN...
--lhf

Reply | Threaded
Open this post in threaded view
|

Re: modules (was: index tag method)

Edgar Toernig
Luiz Henrique de Figueiredo wrote:
> 
> >I think that exactly this is what is required: A discipline on how to
> >write modules.
> 
> That's a good topic for a LTN...

First I have to develop a good one ;-)

Ciao, ET.

Reply | Threaded
Open this post in threaded view
|

Re: index tag method

John Belmonte-2
In reply to this post by Edgar Toernig
Hello,

Edgar Toernig wrote:
> > I think set/gettable are important for general use.  You mentioned
wanting
> > to work on namespaces/modules... do you plan to implement them by adding
to
> > Lua's syntax and semantics?
>
> Syntax: No.  I think that the elements present in Sol are sufficient.
> Semantics: Yes.  IMHO some changes (methods) would make it better.

I can't argue with that, but I'm trying to work out a practical solution for
Lua.  Until Lua becomes Sol or Sol becomes Lua, Sol being sufficient doesn't
help this particular task.

> Concept: Modules exports one table (or more).  This table holds exported
> functions and data objects (File.open, File.stdout, Socket.connect).

Already with that start I think a problem exists.  To code readable modules
(that is, without making a rats nest of upvalues) the module's table needs
to be a global.  To make a global you have to pick a name, and therefore
will have name clash problems.  To be fair Python has the same problem (what
if two modules have the same file name?).  However I'm thinking of a more
general system that can be used for namespaces also, in which case the
chance of name clash is much higher.

The way I'm thinking to get around this is with get/settable hooks, assuming
a Lua change where global accesses trigger those hooks.  First I'm working
on that Lua change which I believe is helpful in any case.  When that's done
I may take a stab at a module/namespace solution.

Anyway I think there are several ways to deal with these issues and it's
good that a few approaches are being explored.

-John



Reply | Threaded
Open this post in threaded view
|

Re: index tag method

Luiz Henrique de Figueiredo
In reply to this post by John Belmonte-2
>Until Lua becomes Sol or Sol becomes Lua

I don't think this is going to happen. :-)

>Already with that start I think a problem exists.  To code readable modules
>(that is, without making a rats nest of upvalues) the module's table needs
>to be a global.  To make a global you have to pick a name, and therefore
>will have name clash problems. 

Lua 4.0 introduced the REGISTRY, which can be used to avoid clashes.
Or, at least, to have a way to know whether someone already has taken the name
you want.

The registry is an ordinary Lua table that is available only to C (you get it
on the stack by lua_getregistry(L)). It is mainly used in libraries now,
but we hope it has others uses, and helps to create disciplines.
It's up to you to find how to do that...
--lhf

Reply | Threaded
Open this post in threaded view
|

Re: index tag method

Edgar Toernig
In reply to this post by John Belmonte-2
Hi,

John Belmonte wrote:
> 
> Edgar Toernig wrote:
> >
> > Syntax: No.  I think that the elements present in Sol are sufficient.
> > Semantics: Yes.  IMHO some changes (methods) would make it better.
> 
> I can't argue with that, but I'm trying to work out a practical solution for
> Lua.  Until Lua becomes Sol or Sol becomes Lua, Sol being sufficient doesn't
> help this particular task.

I doubt that Lua ever becomes Sol ;-)  But maybe they take some ideas.
About the "sufficient": s/present in Sol/present in Lua\/Sol/.  The few
syntactical changes in Sol only makes some things a little bit nicer,
mainly:

Foo = {
	function foo1() ... end
	function foo2() ... end
	...
}

That is normal function definitions within tables without the need for ','.

> > Concept: Modules exports one table (or more).  This table holds exported
> > functions and data objects (File.open, File.stdout, Socket.connect).
> 
> Already with that start I think a problem exists.

I disagree ;-)  Yes, you could do something like
	Foo = module "mod_foo_filename"
and move the policy to the user of the module.  I would call this gutless ;-)
You already have a namespace collision in the file name.  And what is if two
modules want to use the same third module?  Load it twice because noone knows
if anyone else has already loaded it?[1]  I think it's much easier for everyone
if you just say: mod_foo has the file name "foo_filename" and exports the
table "Foo".  Now every user of mod_foo knows how to access it, even the module
itself may refer to Foo.foo1.  Yes, that's policy.  And it requires some
discipline.  But at some place you _have_ to define such things.  If you don't
do it at the module layer you do it within your application and have interoper-
ation problems again.  (And problems reading others code ;-)

([1] You could do some management within the module load function to handle
the "more users" case.  But IMHO that results in too much policy.  It would
require a pretty strong module definition (like: a module has to export ex-
actly one table).  I would prefer a minimum policy that is still flexible.)

> To code readable modules (that is, without making a rats nest of upvalues)
> the module's table needs to be a global.  To make a global you have to pick
> a name, and therefore will have name clash problems.

It's right that writing static/private functions in Lua is not very nice.
You normally use locals and upvalues.  IMHO just playing around with the
global table during module load time will not work.  Maybe it can be made
more friendly by switching local/global default semantics?  Or by redefin-
ing the global scope to always be that that was active during load time?

I'm still thinking about that.  Up to now I stick with the local+upvalues
or simply put private data into the exported table too ;-)

> The way I'm thinking to get around this is with get/settable hooks, assuming
> a Lua change where global accesses trigger those hooks.

There are already get/settable (or even get/setglobal) hooks for globals!?!
*puzzled*

> First I'm working on that Lua change which I believe is helpful in any
> case.  When that's done I may take a stab at a module/namespace solution.

Just report your results/findings/ideas ;-)

> Anyway I think there are several ways to deal with these issues and it's
> good that a few approaches are being explored.

I fully agree!

Ciao, ET.



Reply | Threaded
Open this post in threaded view
|

Re: index tag method

John Belmonte-2
Edgar Toernig wrote:
> The few syntactical changes in Sol only makes some things a little
> bit nicer, mainly:
>
> Foo = {
> function foo1() ... end
> function foo2() ... end
> ...
> }
>
> That is normal function definitions within tables without the need for
','.

I don't think this is too useful.  Since access to Foo's names from within
foo1() and foo2() still have to be explicit (for example, Foo.foo1), I don't
mind being explicit about the function definition in the same way.  (After
changing the parser to accept multiple .'s in the function name I've been
much happier with this situation.  Any verdict from the Lua authors on that
change yet?)

Regarding the module discussion (and real-time GC, etc., etc.), it's easy
for both of us to shoot holes in things that don't exist.  If I design
something, and I have a feeling you are the same way, the result is going to
be solid or else I'll give it up along the way.

-John



Reply | Threaded
Open this post in threaded view
|

Re: modules (was: index tag method)

Reuben Thomas-3
In reply to this post by Edgar Toernig
> policies are required.  And then part of the design is the question:
> "we enforce this policy" or "it only works with self-discipline".
> (I would always choose the later ;-)

I would always choose the former! I don't believe that self-discipline works.

-- 
http://sc3d.org/rrt/ | certain, a.  insufficiently analysed




Reply | Threaded
Open this post in threaded view
|

Re: index tag method

Roberto Ierusalimschy
In reply to this post by John Belmonte-2
> Any verdict from the Lua authors on that change yet?

The syntax for function "names" in Lua 4.1 will be

         name { '.' name } [ ':' name ]

That is, zero or more '.name' components, followed by an optional ':name'.
This is fully compatible, and as someone (you?) pointed out, really
useful for objects inside modules.

-- Roberto

Reply | Threaded
Open this post in threaded view
|

Re: modules (was: index tag method)

Edgar Toernig
In reply to this post by Reuben Thomas-3
Reuben Thomas wrote:
> 
> > policies are required.  And then part of the design is the question:
> > "we enforce this policy" or "it only works with self-discipline".
> > (I would always choose the later ;-)
> 
> I would always choose the former! I don't believe that self-discipline works.

Different philosophy of life I guess ;-)  Although I didn't expect it at(??)
a Lua programmer :-)

Ciao, ET.



Reply | Threaded
Open this post in threaded view
|

RE: modules

Philippe Lhoste-2
In reply to this post by Edgar Toernig
Edgar wrote:
> If Lua is simply used as a scripting extension of a larger project all
> this is normally not required.  The project defines the environment.
> But if you want to use it as a standalone extendable language some
> policies are required.  And then part of the design is the question:
> "we enforce this policy" or "it only works with self-discipline".
> (I would always choose the later ;-)

There is a third way.
If you don't mind my paraphrasing, I will summarise in other words my view
on the subject:

- Lua can be used as an embedded scripting engine of an application, ie. its
code is integral part of the project.
That's the main aim of this language. It is often the option of games, where
the scripts are probably integral part of the application too, ie. the final
user don't have to know Lua is used here.
Of course, in this case of closed environment, libraries are easy to
integrate, and scripts know which libraries are available and will not try
to access other ones.

- Lua can be also used as a standalone language, like Perl, Python or Tcl.
In this case, you can get scripts that need a given set of libraries (eg.
sockets, regular expressions, etc.). Of course, you can get a copy of Lua
executable with the script. But that means you can end with a number of
various Lua exe. residing in various spaces of your hard disk(s). A waste of
space (although not much, Lua is compact), and a source of confusion. Plus
it kind of ruin the interest of having an interpreted language...

- The third way is to use Lua as a scripting engine of an application, but
keeping the Lua code outside the application, probably in a shared library
or DLL.
It is useful where the users can write their own scripts, like in a text
editor.
The main advantage of this scheme is that end users can upgrade their copy
of Lua (if the new one is compatible, of course) without waiting for the
good will of the creator of their program. So they can benefit of increased
speed, bug corrections, improved features, etc.
And if a system of library modules is available, they can even write scripts
doing stuff that the program creator didn't even imagined (like accessing
CVS, etc.).

So, yes we need a good way to add third party libraries to an existing Lua
executable without having to recompile it. The Perl example is quite good
here, as we have a lot of modules which you can grab as you need.

Regards.

--._.·´¯`·._.·´¯`·._.·´¯`·._.·´¯`·._.·´¯`·._.·´¯`·._.--
Philippe Lhoste (Paris -- France)
Professional programmer and amateur artist
http://jove.prohosting.com/~philho/
--´¯`·._.·´¯`·._.·´¯`·._.·´¯`·._.·´¯`·._.·´¯`·._.·´¯`--


Reply | Threaded
Open this post in threaded view
|

RE: modules

Luiz Henrique de Figueiredo
>So, yes we need a good way to add third party libraries to an existing Lua
>executable without having to recompile it.

I think loadlib can help here: http://www.tecgraf.puc-rio.br/~rborges/loadlib/
--lhf

12