Next Version of Lua?

143 messages
123456 ... 8
Open this post in threaded view
|

Re: Next Version of Lua?

 On Wed, Jun 10, 2009 at 1:59 PM, Luiz Henrique de Figueiredo<[hidden email]> >   > setmetatable(t,{ >            __call = function (x,y,z) return next(x,z) end >    }) > >    for k,v in t do print(k,v) end > > No parentheses needed. That is actually rather cute, but how would you do that for an ipairs-style iterator? steve d.
Open this post in threaded view
|

Re[2]: Next Version of Lua?

 In reply to this post by Duncan Cross Hello Duncan, Wednesday, June 10, 2009, 3:32:52 PM, you wrote: > I had an idea for an explicit syntax for this, that overloads the ... operator: > local tbl = {SixValues()..., 7, 8, 9} better table = table1 .. table2 but the next question will be - does we concat on pairs or ipairs? pitting both arrays and hashes to the same datatype strikes us here -- Best regards,  Bulat                            mailto:[hidden email]
Open this post in threaded view
|

Re: Next Version of Lua?

 In reply to this post by steve donovan On Wed, Jun 10, 2009 at 1:38 PM, steve donovan<[hidden email]> wrote: > On Wed, Jun 10, 2009 at 1:59 PM, Luiz Henrique de > Figueiredo<[hidden email]> >   > setmetatable(t,{ >>            __call = function (x,y,z) return next(x,z) end >>    }) >> >>    for k,v in t do print(k,v) end >> >> No parentheses needed. > > That is actually rather cute, but how would you do that for an > ipairs-style iterator? t = {"hello", "world", "from", "Lua"} setmetatable(t, {__call = function(t,v,i)   i = (i or 0) + 1   v = t[i]   if v then return i, v end end}) for k, v in t do print(k, v) end --> 1  hello --> 2  world --> 3  from --> 4  Lua
Open this post in threaded view
|

Re: Next Version of Lua?

 In reply to this post by steve donovan On Wed, Jun 10, 2009 at 1:38 PM, steve donovan<[hidden email]> wrote: > On Wed, Jun 10, 2009 at 1:59 PM, Luiz Henrique de > Figueiredo<[hidden email]> >   > setmetatable(t,{ >>            __call = function (x,y,z) return next(x,z) end >>    }) >> >>    for k,v in t do print(k,v) end >> >> No parentheses needed. > > That is actually rather cute, but how would you do that for an > ipairs-style iterator? Or, alternatively, using the undocumented behaviour of ipairs: ipairs_next = ipairs{} t = {"hello", "world", "from", "Lua"} setmetatable(t, {__call = function(t,_,i) return ipairs_next(t, i or 0) end}) for k, v in t do print(k, v) end
Open this post in threaded view
|

Re: Next Version of Lua?

 On Wed, Jun 10, 2009 at 2:49 PM, Peter Cawley<[hidden email]> wrote: > Or, alternatively, using the undocumented behaviour of ipairs: > > ipairs_next = ipairs{} > t = {"hello", "world", "from", "Lua"} > setmetatable(t, {__call = function(t,_,i) return ipairs_next(t, i or 0) end}) > for k, v in t do print(k, v) end Very elegant - actually, it is documented that the ipairs() function would return 'an iterator function' as its first result.
Open this post in threaded view
|

Re: Next Version of Lua?

 > Very elegant - actually, it is documented that the ipairs() function > would return 'an iterator function' as its first result. But not that the iterator function does not depend on the table...
Open this post in threaded view
|

Re: Re[2]: Next Version of Lua?

 In reply to this post by Bulat Ziganshin On Wed, Jun 10, 2009 at 2:40 PM, Bulat Ziganshin<[hidden email]> wrote: > table = table1 .. table2 > > but the next question will be - does we concat on pairs or ipairs? > pitting both arrays and hashes to the same datatype strikes us here Well, seen as sequences, it would be on ipairs. But it would be confusing if this happened for all tables, and unnecessary, since it's so simple to overload __concat appropriately.
Open this post in threaded view
|

RE: Next Version of Lua? (__iter metamethod)

 In reply to this post by Luiz Henrique de Figueiredo Ugg! Not a good choice, in my opinion! There is a "right choice" of iterator for different types of object and it should be contextually selected not having to remember whether to use pairs or ipairs. If you go down this way, we'll end up defining __pairs or __ipairs to give error messages if the wrong one is used! This is like saying the user has to use a different function for adding real and complex numbers and if they pick the wrong one they'll get a result, but a meaningless one. Apart from this consideration of principle, it is using two metamethods to do a half-assed job that could be properly done with just one! I cannot see why this is syntactically impossible. Just check first for a __iter when parsing the for statement, if not present, fall back to processing as currently. > -----Original Message----- > From: [hidden email] [mailto:lua- > [hidden email]] On Behalf Of Luiz Henrique de > Figueiredo > Sent: 10 June 2009 13:34 > To: Lua list > Subject: Re: Next Version of Lua? (__iter metamethod) > > > Could I push my luck by asking if there is a position on '__iter'? > > There'll be metamethods __pairs and __ipairs. See > http://lua-users.org/wiki/LuaFiveTwo
Open this post in threaded view
|

Re[4]: Next Version of Lua?

 In reply to this post by steve donovan Hello steve, Wednesday, June 10, 2009, 4:59:14 PM, you wrote: >> table = table1 .. table2 >> >> but the next question will be - does we concat on pairs or ipairs? >> pitting both arrays and hashes to the same datatype strikes us here > Well, seen as sequences, it would be on ipairs. the problem is that concatenation of hash tables is also useful -- Best regards,  Bulat                            mailto:[hidden email]
Open this post in threaded view
|

Re: Next Version of Lua? (__iter metamethod)

 In reply to this post by Luiz Henrique de Figueiredo On Wed, Jun 10, 2009 at 2:16 PM, John Hind<[hidden email]> wrote: > Ugg! Not a good choice, in my opinion! There is a "right choice" of iterator > for different types of object and it should be contextually selected not > having to remember whether to use pairs or ipairs. If you go down this way, > we'll end up defining __pairs or __ipairs to give error messages if the > wrong one is used! This is like saying the user has to use a different > function for adding real and complex numbers and if they pick the wrong one > they'll get a result, but a meaningless one. pairs and ipairs are "intended" for iterating over table-like objects. __pairs and __ipairs allow non-tables to behave like tables do in more situations. If you want to attach them to a more complex object, then simply set __pairs and __ipairs to the same function if you want to make pairs() and ipairs() both do the default iteration. For complex objects, it may not be obvious (to people other than the library author) what the default iteration is. In this case, it can be clearer if the complex object has members on it for iteration, like "for k,v in obj:children() do" or "for callback in obj:callbacks() do", thus making it explicit what part of the object is being iterated. On Wed, Jun 10, 2009 at 2:16 PM, John Hind<[hidden email]> wrote: > I cannot see why this is syntactically impossible. Just check first for a > __iter when parsing the for statement, if not present, fall back to > processing as currently. The parser doesn't know what type of value "t" is in "for k, v in t do", let alone if it has an __iter metamethod or not.
Open this post in threaded view
|

Re: Next Version of Lua?

 In reply to this post by Olivier Hamel 2009/6/10 Olivier Hamel <[hidden email]>: > Ah, I thank you for point this page out to me. I'll take this opportunity to > dump my suggestions/wishes/hopes for Lua 5.2. Basically Lua 5.1 is very > close to what I would consider the penultimate scripting language I'm using lua as a game scripting language and the only thing I'm missing is support for fast value types, i. e. user-created objects with the same semantics as lua_Numbers. Currently, if you want to create Complex or Vector3 type you have to do it as a table or a userdata. And if you do a serious ammount of processing on them, the ammount of garbage generated by simple arithmetic operations will soon put a significant strain on the system. (You could use a pool of such objects, but that would mean resorting to manual memory management with all its pains - especially when you are using it for something as simple as numbers.) A generational garbage collector could be an alternative to value types, I guess. Provided it is fast and smart enough to keep up with the arithmetic garbage. // Niklas
Open this post in threaded view
|

Re: Next Version of Lua?

 On Wed, Jun 10, 2009 at 2:55 PM, Niklas Frykholm<[hidden email]> wrote: > 2009/6/10 Olivier Hamel <[hidden email]>: > Currently, if you want to create Complex or Vector3 type you have to > do it as a table or a userdata. And if you do a serious ammount of > processing on them, the ammount of garbage generated by simple > arithmetic operations will soon put a significant strain on the > system. (You could use a pool of such objects, but that would mean > resorting to manual memory management with all its pains - especially > when you are using it for something as simple as numbers.) Perhaps I am misunderstanding, but if the pool is a table that has been set to have weak values (i.e. its metatable's __mode field is set to 'v'), you should not have to do any manual memory management - values that only exist in the pool will be eligible for garbage collection. -Duncan
Open this post in threaded view
|

RE: Next Version of Lua? (__iter metamethod)

 In reply to this post by Peter Cawley > pairs and ipairs are "intended" for iterating over table-like objects. > __pairs and __ipairs allow non-tables to behave like tables do in more > situations. If you want to attach them to a more complex object, then > simply set __pairs and __ipairs to the same function if you want to > make pairs() and ipairs() both do the default iteration. For complex > objects, it may not be obvious (to people other than the library > author) what the default iteration is. In this case, it can be clearer > if the complex object has members on it for iteration, like "for k,v > in obj:children() do" or "for callback in obj:callbacks() do", thus > making it explicit what part of the object is being iterated.   Looking at it fundamentally, any object that implements collection semantics benefits from iterator functionality. "pairs" is appropriate for dictionary-like collections (whether implemented using tables or otherwise) while "ipairs" is appropriate for "list" or "array" like collections (whether implemented using tables or otherwise). With basic Lua whether a table is "dictionary-like" or "list-like" is about how you assign the keys, but with true object orientation (using metatables) you will usually decide the structure of the table in the class design. OK sometimes an object requires more than one type of iterator, but rarely exactly "pairs" and "ipairs" - so it would be better to cope with this by allowing "__iter" to be parameterised (or as you suggest, using methods for non-default iterators). The Lua "String" type got an object oriented library in the last version, but think about how the same would have to be done for "Table". You would need to fork the Table type into two Classes- many of the functions in the library only work on "List-Like" tables so you'd want one metatable for those and another for "Generic" "Dictionary-like" tables. "pairs" would be the right iterator in the generic class and "ipairs" in the "List-Like" case. > The parser doesn't know what type of value "t" is in "for k, v in t > do", let alone if it has an __iter metamethod or not. OK, but the case is no different from any other "core" metamethod i.e. "__add". I admit I've never looked into the details, but if this metamethod has to be resolved at runtime so could the "__iter" metamethod be. Maybe that involves a new bytecode, but that is what new versions are for surely? I get that "__pairs" and "__ipairs" are "library metamethods" in the same sense as "__tostring" is, but this means they are rather trivial and could be implemented easily on the current language platform just by shelling the pairs and ipairs functions.
Open this post in threaded view
|

Re: Next Version of Lua?

 In reply to this post by Duncan Cross 2009/6/10 Duncan Cross <[hidden email]>: > On Wed, Jun 10, 2009 at 2:55 PM, Niklas Frykholm<[hidden email]> wrote: >> 2009/6/10 Olivier Hamel <[hidden email]>: >> Currently, if you want to create Complex or Vector3 type you have to >> do it as a table or a userdata. And if you do a serious ammount of >> processing on them, the ammount of garbage generated by simple >> arithmetic operations will soon put a significant strain on the >> system. (You could use a pool of such objects, but that would mean >> resorting to manual memory management with all its pains - especially >> when you are using it for something as simple as numbers.) > > Perhaps I am misunderstanding, but if the pool is a table that has > been set to have weak values (i.e. its metatable's __mode field is set > to 'v'), you should not have to do any manual memory management - > values that only exist in the pool will be eligible for garbage > collection. My description was a bit short, but the whole point of having a pool was to avoid generating garbage. (By using a free object from the pool, rather than creating a new one, whenever a new object is needed.) But this requires us to manually track which objects in the pool are free or not (i.e., manual memory management). If the objects in the pool are eligible for garbage collection, then the pool doesn't really buy us anything. We will generate the same ammount of garbage with the pool as without it. // Niklas
Open this post in threaded view
|

Re: Next Version of Lua?

 In reply to this post by Olivier Hamel On Jun 10, 2009, at 3:02 AM, Olivier Hamel wrote: > * (VM?) + Lexer: Have an explicitly defined handling of cases like   > {[3] = 3, 1, 2, 3} and do not adjust the return codes to one unless   > an operation is directly performed on it. Example: > local tbl = {ReturnSixIncrementalValuesFromZeroToSix(), 7, 8, 9} --   > this would equate to doing {1, 2, 3, 4, 5, 6, 7, 8, 9} > local tbl = {ReturnSixIncrementalValuesFromZeroToSix() + 10, 7, 8,   > 9} -- this would equate to doing {11, 7, 8, 9} > The alternate suggestion of using ';' to elaborate when to use   > multiple returns might be better (feedback on this please?)? I'll let others discuss the pros and cons -- one of the big cons being   compatibility -- but I will concur that it would be nice to have some   sort of splicing construction for tables if only to allow for   functions that returned nothing and didn't then insert a nil into the   table which then tells ipairs that it's the end of the table: ViewList = {         Button( "I'm always here" ),         Optional( TestCondition(), Button( "Condition Was True" ) )         Button( "I'm always here too" ) } The current work around for this is to have Optional return a "should   not be here value" and have a post-process remove such values. One can   also address splicing that way. But that doesn't feel quite as natural   when building structures. On the other hand, I don't know what syntax I would propose for this.   It probably needs to stand out, but the more it stands out the more it   risks not being used properly. Mark
Open this post in threaded view
|

Re: Next Version of Lua?

 In reply to this post by Olivier Hamel On Jun 10, 2009, at 3:02 AM, Olivier Hamel wrote: > * (VM?) + Lexer: Have an explicitly defined handling of cases like   > {[3] = 3, 1, 2, 3} and do not adjust the return codes to one unless   > an operation is directly performed on it. Example: > local tbl = {ReturnSixIncrementalValuesFromZeroToSix(), 7, 8, 9} --   > this would equate to doing {1, 2, 3, 4, 5, 6, 7, 8, 9} > local tbl = {ReturnSixIncrementalValuesFromZeroToSix() + 10, 7, 8,   > 9} -- this would equate to doing {11, 7, 8, 9} > The alternate suggestion of using ';' to elaborate when to use   > multiple returns might be better (feedback on this please?)? I'll let others discuss the pros and cons -- one of the big cons being   compatibility -- but I will concur that it would be nice to have some   sort of splicing construction for tables if only to allow for   functions that returned nothing and didn't then insert a nil into the   table which then tells ipairs that it's the end of the table: ViewList = {         Button( "I'm always here" ),         Optional( TestCondition(), Button( "Condition Was True" ) )         Button( "I'm always here too" ) } The current work around for this is to have Optional return a "should   not be here value" and have a post-process remove such values. One can   also address splicing that way. But that doesn't feel quite as natural   when building structures. On the other hand, I don't know what syntax I would propose for this.   It probably needs to stand out, but the more it stands out the more it   risks not being used properly. Mark
Open this post in threaded view
|

Re: Next Version of Lua?

 In reply to this post by Niklas Frykholm On Wed, Jun 10, 2009 at 3:42 PM, Niklas Frykholm<[hidden email]> wrote: > 2009/6/10 Duncan Cross <[hidden email]>: >> On Wed, Jun 10, 2009 at 2:55 PM, Niklas Frykholm<[hidden email]> wrote: >>> 2009/6/10 Olivier Hamel <[hidden email]>: >>> Currently, if you want to create Complex or Vector3 type you have to >>> do it as a table or a userdata. And if you do a serious ammount of >>> processing on them, the ammount of garbage generated by simple >>> arithmetic operations will soon put a significant strain on the >>> system. (You could use a pool of such objects, but that would mean >>> resorting to manual memory management with all its pains - especially >>> when you are using it for something as simple as numbers.) >> >> Perhaps I am misunderstanding, but if the pool is a table that has >> been set to have weak values (i.e. its metatable's __mode field is set >> to 'v'), you should not have to do any manual memory management - >> values that only exist in the pool will be eligible for garbage >> collection. > > My description was a bit short, but the whole point of having a pool > was to avoid generating garbage. (By using a free object from the > pool, rather than creating a new one, whenever a new object is > needed.) But this requires us to manually track which objects in the > pool are free or not (i.e., manual memory management). > > If the objects in the pool are eligible for garbage collection, then > the pool doesn't really buy us anything. We will generate the same > ammount of garbage with the pool as without it. > > // Niklas > Well, it depends - are you ending up with many many separate copies of exactly the same vector? My assumption of how the pool would work was that the key used to insert values into the pool would be a serialized version of the object (for example using 'luaL_pushlstring(L, (const char*)pvector, sizeof(VECTOR));' to generate the key, assuming your vector is a simple struct stored in a full userdata) and the pool would be queried by value to see if an existing vector object could be reused. But if that isn't the case, and the vectors you are dealing with are unique in the majority of cases, then sure, it doesn't help. (It also doesn't help if you need the vectors to be mutable rather than immutable objects.) -Duncan
Open this post in threaded view
|

Re: Next Version of Lua? (__iter metamethod)

 In reply to this post by John Hind John Hind wrote: > Ugg! Not a good choice, in my opinion! There is a "right choice" of iterator > for different types of object and it should be contextually selected not > having to remember whether to use pairs or ipairs. Not true. I often have multi-use tables, such that t[id] = t[n] = object. I can then lookup objects by id, or iterate them with ipairs. In this case, how would your code 'contextually select' the iterator ? > Apart from this consideration of principle, it is using two metamethods to > do a half-assed job that could be properly done with just one! Is the solution half-assed, or your understanding of the problem ?
Open this post in threaded view
|

Re: Next Version of Lua?

 In reply to this post by Niklas Frykholm The argument for having support for __iter is that if one starts to   build iterations via sequences of filters, maps, and so forth then   __iter allows one to avoid having to tack something on at the end to   actually convert to iteration form while at the same time avoiding the   performance costs of going through the __call metamethod lookup on   every loop iteration. The expense of doing this is the extra opcode at the beginning of a   for loop to see whether the first argument to the for loop has an   __iter metamethod and if so calling it. This arguably comes down to a question of how much Lua should embrace   an object-oriented approach to operations and how much it should stick   to a more procedural/functional approach. __iter suggests that we   think about for as taking iterable objects. While on the subject of iteration, at the expense of probably breaking   a number of iterator implementations, it seems like it would be nice   if one could signal done by having the iterator return nothing instead   of nil. That way one could build iterators that did in fact return   nil. I don't have a good example of usage handy, but imagine wanting   to return just the values in an array over a range. The work around   right now is to return an extra initial parameter and simply ignore it. In particular, here is a semi-desirable use case:         for v in ... do                 -- process varargs         end This doesn't iterate the varargs because it actually means use the ...   parameters as the arguments. Hence, we probably need some other syntax   anyway such as:         for v in iter( ... ) do                 -- process varargs         end It isn't a huge burden to have this also return an index which can   then be ignored, but the index might be seen as a distraction:         for _, v in iter( ... ) do                 -- process varargs         end With or without the change to support nil values from iterations, some   way to iterate (and for that matter index) ... would be really nice   since use of select tends to lead to quadratic time complexity. Mark