Lua 5.4 Tuples FTW!

classic Classic list List threaded Threaded
21 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Lua 5.4 Tuples FTW!

Duane Leslie
Hi,

A quick and dirty tuple implementation using the new multi-uservalue
userdata implementation.

Compile with `cc -shared -fPIC -o tuple.so ltuple.c`.

Use as follows:

Lua 5.4.0 (work1)  Copyright (C) 1994-2018 Lua.org, PUC-Rio
> tuple = require"tuple"
> function test(...) return tuple(...) end
> t = test(1,2,nil,4,nil)
> #t
5
> t[2], t[3], t[4]
2    nil    4
> for i,v in pairs(t) do print(i,v) end
1    1
2    2
3    nil
4    4
5    nil
>

Enjoy.

Regards,

Duane.

ltuple.c (2K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Lua 5.4 Tuples FTW!

Thomas Jericke
On 16.03.2018 00:37, Duane Leslie wrote:
Hi,

A quick and dirty tuple implementation using the new multi-uservalue
userdata implementation.
The problem I see, is that you don't get raw equality with any userdata implementation of tuples. So I cannot use them as table keys.
For example I cannot use tuples to index a matrix.

t = {}
t[tuple(2,2)] = 42
print(tuple(2,2)) -- prints nil

Also they are passed by reference and not by value. That can be confusing once you edit a value in that tuple.
--
Thomas
Reply | Threaded
Open this post in threaded view
|

Re: Lua 5.4 Tuples FTW!

William Ahern
On Fri, Mar 16, 2018 at 07:54:12AM +0100, Thomas Jericke wrote:

> On 16.03.2018 00:37, Duane Leslie wrote:
> > Hi,
> >
> > A quick and dirty tuple implementation using the new multi-uservalue
> > userdata implementation.
> The problem I see, is that you don't get raw equality with any userdata
> implementation of tuples. So I cannot use them as table keys.
> For example I cannot use tuples to index a matrix.
>
> t = {}
> t[tuple(2,2)] = 42
> print(tuple(2,2)) -- prints nil

You can memoize using an ephemeron table.

> Also they are passed by reference and not by value. That can be confusing
> once you edit a value in that tuple.


Reply | Threaded
Open this post in threaded view
|

__ipairs (was: Re: Lua 5.4 Tuples FTW!)

Andrew Gierth
In reply to this post by Duane Leslie
>>>>> "Duane" == Duane Leslie <[hidden email]> writes:

 Duane> Hi,

 Duane> A quick and dirty tuple implementation using the new
 Duane> multi-uservalue userdata implementation.

[...]

 >> for i,v in pairs(t) do print(i,v) end

A perfect example of why deprecating __ipairs was the wrong move. It
ought to be possible to make this code work with ipairs(t) as well, but
with no __ipairs metamethod it can't be done.

--
Andrew.

Reply | Threaded
Open this post in threaded view
|

Re: Lua 5.4 Tuples FTW!

Thomas Jericke
In reply to this post by William Ahern
On 16.03.2018 09:12, William Ahern wrote:
On Fri, Mar 16, 2018 at 07:54:12AM +0100, Thomas Jericke wrote:
On 16.03.2018 00:37, Duane Leslie wrote:
Hi,

A quick and dirty tuple implementation using the new multi-uservalue
userdata implementation.
The problem I see, is that you don't get raw equality with any userdata
implementation of tuples. So I cannot use them as table keys.
For example I cannot use tuples to index a matrix.

t = {}
t[tuple(2,2)] = 42
print(tuple(2,2)) -- prints nil
You can memoize using an ephemeron table.

Not saying that this doesn't work, but what would I use as the key of the ephemeron table? I still need to somehow "hash" or concatenation the tuple into some raw type. And then why I am asking myself, why don't just use tables all the way. Maybe there is a performance benefit?
--
Thomas
Reply | Threaded
Open this post in threaded view
|

Re: __ipairs

Soni "They/Them" L.
In reply to this post by Andrew Gierth


On 2018-03-16 05:56 AM, Andrew Gierth wrote:

>>>>>> "Duane" == Duane Leslie <[hidden email]> writes:
>   Duane> Hi,
>
>   Duane> A quick and dirty tuple implementation using the new
>   Duane> multi-uservalue userdata implementation.
>
> [...]
>
>   >> for i,v in pairs(t) do print(i,v) end
>
> A perfect example of why deprecating __ipairs was the wrong move. It
> ought to be possible to make this code work with ipairs(t) as well, but
> with no __ipairs metamethod it can't be done.
>

Uh, the only issue is that ipairs would "respect" nil, but otherwise it
does/should work.

But yes, being able to explicitly include nils in ipairs output, i.e.
with __ipairs, would be nice.

I don't think the Lua team wants to have both __ipairs and __index
respected by ipairs tho...

Do you know anything in Lua which respects *two* metamethods?

--
Disclaimer: these emails may be made public at any given time, with or without reason. If you don't agree with this, DO NOT REPLY.


Reply | Threaded
Open this post in threaded view
|

Re: __ipairs

Oliver Kroth

> Uh, the only issue is that ipairs would "respect" nil, but otherwise
> it does/should work.
>
> But yes, being able to explicitly include nils in ipairs output, i.e.
> with __ipairs, would be nice.
>
afaIk, getting nil from the iterator is what terminates the loop...

--
Oliver

Reply | Threaded
Open this post in threaded view
|

Re: __ipairs

Soni "They/Them" L.


On 2018-03-16 07:00 AM, Oliver Kroth wrote:
>
>> Uh, the only issue is that ipairs would "respect" nil, but otherwise
>> it does/should work.
>>
>> But yes, being able to explicitly include nils in ipairs output, i.e.
>> with __ipairs, would be nice.
>>
> afaIk, getting nil from the iterator is what terminates the loop...
>

ipairs uses nil as a loop end

e.g.

t = {1, 2, 3, nil, 4}

for k, v in ipairs(t) do print(k,v) end

--[[
1 1
2 2
3 3
--]]

When using __index, it still stops on the first nil. It doesn't respect
__len.

This means tuples without nil in them would iterate fine:

for k, v in ipairs(tuple(1, 2, 3)) do print(k,v) end

--[[
1 1
2 2
3 3
--]]

But tuples with nil would stop short.

--
Disclaimer: these emails may be made public at any given time, with or without reason. If you don't agree with this, DO NOT REPLY.


Reply | Threaded
Open this post in threaded view
|

Re: Lua 5.4 Tuples FTW!

Duane Leslie
In reply to this post by Duane Leslie
> On 16 Mar 2018, at 17:54, Thomas Jericke <[hidden email]> wrote:
>
> The problem I see, is that you don't get raw equality with any userdata implementation of tuples. So I cannot use them as table keys.

This can be solved using interning as is often done with strings, it just makes the constructor function more difficult and requires a `__gc` metamethod.

> Also they are passed by reference and not by value. That can be confusing once you edit a value in that tuple.

The passing method doesn't matter if the tuple is immutable (or copy-on-edit) so this isn't an issue.

Regards,

Duane.
Reply | Threaded
Open this post in threaded view
|

Re: __ipairs (was: Re: Lua 5.4 Tuples FTW!)

John Hind
In reply to this post by Andrew Gierth
>
> >>>> for i,v in pairs(t) do print(i,v) end
>
> >>A perfect example of why deprecating __ipairs was the wrong move. It
> >>ought to be possible to make this code work with ipairs(t) as well,
>but
> >>with no __ipairs metamethod it can't be done.
>
> >>Andrew.
Except of course by the simple expedient of replacing ipairs(t) with a
version
that does what you want it to do directly! Even, if you insist, with a
version that
uses a metamethod __ipairs! The metamethods on both pairs and ipairs
have
always baffled me with their utter triviality!

- John.


Reply | Threaded
Open this post in threaded view
|

Re: __ipairs (was: Re: Lua 5.4 Tuples FTW!)

Elias Hogstvedt
I don't get the point of those iterators either, but one possible argument for the operators would be external libraries taking in external objects using pairs or ipairs on them. Maybe I see pairs and ipairs as utility functions and not part of the syntax.

I think an operator like this would make more sense (although I probably wouldn't use it personally)

    local meta = {}
    meta.__index = meta

    function meta:__iterator(a, ...)
        local i = 0
        return function()
            i = i + 1
            return self.foo[i] + a
        end
    end

    local obj = setmetatable({}, meta)

    obj.foo = {1,2,3,4,5}

    for res in obj, 10 do
        print(res)
    end

    >>11
    >>12
    >>13
    >>14
    >>15
    >>16

On Fri, Mar 16, 2018 at 11:56 AM, John Hind <[hidden email]> wrote:

>>>> for i,v in pairs(t) do print(i,v) end

>>A perfect example of why deprecating __ipairs was the wrong move. It
>>ought to be possible to make this code work with ipairs(t) as well, but
>>with no __ipairs metamethod it can't be done.

>>Andrew.
Except of course by the simple expedient of replacing ipairs(t) with a version
that does what you want it to do directly! Even, if you insist, with a version that
uses a metamethod __ipairs! The metamethods on both pairs and ipairs have
always baffled me with their utter triviality!

- John.



Reply | Threaded
Open this post in threaded view
|

Re: Lua 5.4 Tuples FTW!

Thomas Jericke
In reply to this post by Duane Leslie
On 16.03.2018 11:11, Duane Leslie wrote:
On 16 Mar 2018, at 17:54, Thomas Jericke [hidden email] wrote:

The problem I see, is that you don't get raw equality with any userdata implementation of tuples. So I cannot use them as table keys.
This can be solved using interning as is often done with strings, it just makes the constructor function more difficult and requires a `__gc` metamethod.
Just.

Well I see how this can work, but then, this is nothing that could not be done with Lua 5.3 and tables.

Also they are passed by reference and not by value. That can be confusing once you edit a value in that tuple.
The passing method doesn't matter if the tuple is immutable (or copy-on-edit) so this isn't an issue.

Regards,

Duane.
I am actually struggling a little how I would implement a copy-on-edit. The __newindex metamethod gets the userdata and the field name and the new value as parameters. The userdata is passed as a reference, but if I am not completely wrong I cannot edit the original reference, I just get a copy of the reference which is then placed on the stack. If Lua wanted to support this, it would need to be possible that the __newindex method returns a value that replaces the original reference.I haven't tried this so I might be wrong here.

Anyway the resort to immutable will work for sure.
--
Thomas
Reply | Threaded
Open this post in threaded view
|

Re: __ipairs (was: Re: Lua 5.4 Tuples FTW!)

John Hind
In reply to this post by Elias Hogstvedt
Elias: You may be interested in my patch "Self-iterating Objects" here:

lua-users.org/wiki/LuaPowerPatches

I've been trying to sell this for years with little traction!

- John


Reply | Threaded
Open this post in threaded view
|

Re: __ipairs (was: Re: Lua 5.4 Tuples FTW!)

Hisham
In reply to this post by John Hind
On 16 March 2018 at 07:56, John Hind <[hidden email]> wrote:

>>
>> >>>> for i,v in pairs(t) do print(i,v) end
>>
>> >>A perfect example of why deprecating __ipairs was the wrong move. It
>> >>ought to be possible to make this code work with ipairs(t) as well, but
>> >>with no __ipairs metamethod it can't be done.
>>
>> >>Andrew.
>
> Except of course by the simple expedient of replacing ipairs(t) with a
> version
> that does what you want it to do directly! Even, if you insist, with a
> version that
> uses a metamethod __ipairs! The metamethods on both pairs and ipairs have
> always baffled me with their utter triviality!

When writing a library, you cannot assume you can monkey-patch other
people's standard libraries at will.

Not everyone writes Lua code in a situation where they can play God
with the environment.

-- Hisham

Reply | Threaded
Open this post in threaded view
|

Re: __ipairs (was: Re: Lua 5.4 Tuples FTW!)

John Hind
 >>Hisham wrote:

 >>When writing a library, you cannot assume you can monkey-patch other
 >>people's standard libraries at will.

 >>Not everyone writes Lua code in a situation where they can play God
 >>with the environment.

So provide an iterator factory in your library which does what you think
should be done:

for i, v in mytypeiterator(instanceofmytype) do print(i, v) end

Better still make it a method rather than a metamethod of your object
so:

for i, v in instanceofmytype:ipairs() do print(i, v) end

pairs and ipairs were never supposed to be syntax, they are one of any
number of possible implementations of iteration which were intended to
be extended by users and library authors.

Best of all, were my "Self Iterating Objects" patch
(lua-users.org/wiki/LuaPowerPatches) to be adopted, you could make the
iterator appropriate to the type a metamethod __iter of the type and
then your users can write the much more intuitive:

for i, v in instanceofmytype do print(i, v) end

- John

------ Original Message ------
From: "Hisham" <[hidden email]>
To: "John Hind" <[hidden email]>; "Lua mailing list"
<[hidden email]>
Subject: Re: __ipairs (was: Re: Lua 5.4 Tuples FTW!)

>On 16 March 2018 at 07:56, John Hind <[hidden email]> wrote:
>>>
>>> >>>> for i,v in pairs(t) do print(i,v) end
>>>
>>> >>A perfect example of why deprecating __ipairs was the wrong move.
>>>It
>>> >>ought to be possible to make this code work with ipairs(t) as well,
>>>but
>>> >>with no __ipairs metamethod it can't be done.
>>>
>>> >>Andrew.
>>
>>Except of course by the simple expedient of replacing ipairs(t) with a
>>version
>>that does what you want it to do directly! Even, if you insist, with a
>>version that
>>uses a metamethod __ipairs! The metamethods on both pairs and ipairs
>>have
>>always baffled me with their utter triviality!
>
>-- Hisham


Reply | Threaded
Open this post in threaded view
|

pairs and ipairs: what are they good for? (Was: Re: __ipairs)

Peter Aronoff
John Hind <[hidden email]> wrote:
> pairs and ipairs were never supposed to be syntax, they are one of any
> number of possible implementations of iteration which were intended to be
> extended by users and library authors.

I’m sorry, but just to be clear: are you saying that every module author
should create their own custom iteration? That seems like a huge stretch,
even if we all agree that it’s a strength of Lua that you *can* easily
write your own custom iterators.


P

--
We have not been faced with the need to satisfy someone else's
requirements, and for this freedom we are grateful.
    Dennis Ritchie and Ken Thompson, The UNIX Time-Sharing System

Reply | Threaded
Open this post in threaded view
|

Re: __ipairs (was: Re: Lua 5.4 Tuples FTW!)

Hisham
In reply to this post by John Hind
On 16 March 2018 at 12:38, John Hind <[hidden email]> wrote:

>>>Hisham wrote:
>
>>>When writing a library, you cannot assume you can monkey-patch other
>>>people's standard libraries at will.
>
>>>Not everyone writes Lua code in a situation where they can play God
>>>with the environment.
>
> So provide an iterator factory in your library which does what you think
> should be done:

By that logic, why have a __len metamethod if we can just provide
mytypelength(), or any of the other metamethods. :)

Having said that, I personally don't think a metamethod is necessary
for ipairs(), now that Lua (since 5.3) respects the other metamethods.
The protocol for ipairs() is restricted enough so that it should be
possible to produce, using the table metamethods, a behavior that is
customizable and yet respects the protocol expected by users of
ipairs() (ordered integer keys, value is never nil). Removing __ipairs
ensures that this protocol is never broken, so I think it's a good
call.

-- Hisham

Reply | Threaded
Open this post in threaded view
|

Re: pairs and ipairs: what are they good for? (Was: Re: __ipairs)

Hisham
In reply to this post by Peter Aronoff
On 16 March 2018 at 12:44, Peter Aronoff <[hidden email]> wrote:
> John Hind <[hidden email]> wrote:
>> pairs and ipairs were never supposed to be syntax, they are one of any
>> number of possible implementations of iteration which were intended to be
>> extended by users and library authors.
>
> I’m sorry, but just to be clear: are you saying that every module author
> should create their own custom iteration? That seems like a huge stretch,
> even if we all agree that it’s a strength of Lua that you *can* easily
> write your own custom iterators.

As a Lua user, this is what I expect from pairs and ipairs:

* pairs - for an indexable value (i.e. some v I can do v[x] on), give
me all possible x's, and their v[x]'s, with no order guarantees.
* ipairs - for an integer-indexable value (i.e. some v I can do v[i]
on), give me all values from v[1] up in order, breaking at the first
nil.

If I can make my custom object (metatable-powered userdata or
whatever) respect these two protocols, I'll make them pairs() and
ipairs() usable for them via metamethods. If I want any other behavior
(restricting or relaxing the above protocols), I will provide a
separate iterator.

For example: I'll use a sortedpairs() iterator instead of adding to a
table a __pairs metamethod that always returns keys in sorted order.
It's easier for whoever is reading the code.

Another example: one time when I bridged Java collections into Lua, I
gave users an each() iterator for iterating Java Vectors (that
returned only values and no keys), instead of making pairs() return
sorted elements or making ipairs() return a zero-based iteration (or
worse, returned different indices for values on the Lua and Java
sides).

-- Hisham

Reply | Threaded
Open this post in threaded view
|

Re: __ipairs (was: Re: Lua 5.4 Tuples FTW!)

John Hind
In reply to this post by Hisham
> >>By that logic, why have a __len metamethod if we can just provide
> >>mytypelength(), or any of the other metamethods. :)
>
> >>-- Hisham
That is illogical logic! __len adds functionality to an operator and
cannot be implemented in Lua without a metamethod. The Lua Reference
Manual implicitly recognises two categories of metamethods: those which
provide real functionality in the core are listed in the metatable
section including "__len" (and would include my proposed "__iter"). The
others, implemented entirely in library code, are only mentioned in the
library documentation.

I agree that I'd be hard pressed to identify any of the latter that pass
the Saint-Exupery test ("A designer knows that he has arrived at
perfection not when there is no longer anything to add, but when there
is no longer anything to take away.")

- John


Reply | Threaded
Open this post in threaded view
|

Re: Lua 5.4 Tuples FTW!

Duane Leslie
In reply to this post by Duane Leslie
> On 16 Mar 2018, at 23:54, Thomas Jericke <[hidden email]> wrote:
>
> Well I see how this can work, but then, this is nothing that could not be done with Lua 5.3 and tables.

Except that it was not easy to store `nil` previously.  But yes, tables are sufficiently powerful to do pretty much anything with some effort.

The effort required is not so bad if you 'cheat' and use the internal structure of the userdata, because I believe the uservalue array can just be treated as a byte string that uniquely identifies the tuple.  I'd have to play with this some more.

> I am actually struggling a little how I would implement a copy-on-edit. The __newindex metamethod gets the userdata and the field name and the new value as parameters.

I specifically would not implement `__newindex`, but rather use slicing and joining type operations since these semantically imply the return of a new object.  Not very efficient, but then tuples shouldn't be edited in general.

Regards,

Duane.
12