Strong tables in Lua 5.4

classic Classic list List threaded Threaded
46 messages Options
123
Reply | Threaded
Open this post in threaded view
|

Re: Strong tables in Lua 5.4

Sean Conner
It was thus said that the Great Sean Conner once stated:

> It was thus said that the Great Roberto Ierusalimschy once stated:
> > > Every `t[k] = f()` of every module and dependency (and their
> > > dependencies recursively) would have to be checked for memory leaks.
> > > And this won't be the typical Lua 5.x to Lua 5.(x+1) situation where
> > > you try out a dependency on a new Lua version and it doesn't) build
> > > because lua_dump now has an extra argument, or lua_objlen was renamed
> > > to lua_rawlen in a Lua/C module, or you get a crash with a stack trace
> > > in your testsuite because setfenv() doesn't exist anymore.
> >
> > I still think you are exaggerating the problem, but I may be wrong. I
> > would like to know how many real programs have the construction
> > t[k] = f() where f() returning 'nil' is not a bug. (You mentioned
> > that table.move could create that kind of thing. I asked for a real
> > scenario, but got no answer.)
> >
> > More concretely, I propose a little experiment: suppose that Lua raises
> > a run-time error for every assignment of nils to tables except in
> > explicit assignments with a constant nil (t[k]=nil). How frequently this
> > error will occurr without hinting a hidden bug in the code? (Please,
> > any answer should be about some piece of real, useful code where that
> > could happen.)
>
>   Less than a minute in, and I found a place what would break code I've
> written (for work no less!).
>
>   I have the following function:
>
> -- Okay, we store a coutine and arguments into a table for later
> -- running the it.
>
> function schedule(co,...)
>  RQUEUE[#RQUEUE + 1] = { co, ... }
> end
>
> then later down in the code I have the following:
>
> -- We've timed out on waiting for something.  signal the error by
> -- returning nil and and error of TIMEDOUT
>
> schedule(obj.co,nil,errno.ETIMEDOUT)

  I'm replying to this to say that I might have found a bug in that code and
that it works is a quirk of the implementation.  I went further down the
code to where I actually run the coroutine and lo, I found:

        local okay,err = coroutine.resume(unpack(obj))
        -- obj is a bad name but that's beside the point!

  The 'obj' table contains:

        { co , nil , errno.TIMEDOUT }

and I suppose because of the binary search nature of determining the length,
this works!  Wow!  I think I was concentrating on the Happy Path (TM) that
the undefined nature of length in the presence of holes skipped me entirely.

  -spc


Reply | Threaded
Open this post in threaded view
|

Re: Strong tables in Lua 5.4

Hisham
In reply to this post by Roberto Ierusalimschy
On 19 March 2018 at 16:08, Roberto Ierusalimschy <[hidden email]> wrote:

>> Every `t[k] = f()` of every module and dependency (and their
>> dependencies recursively) would have to be checked for memory leaks.
>> And this won't be the typical Lua 5.x to Lua 5.(x+1) situation where
>> you try out a dependency on a new Lua version and it doesn't) build
>> because lua_dump now has an extra argument, or lua_objlen was renamed
>> to lua_rawlen in a Lua/C module, or you get a crash with a stack trace
>> in your testsuite because setfenv() doesn't exist anymore.
>
> I still think you are exaggerating the problem, but I may be wrong. I
> would like to know how many real programs have the construction
> t[k] = f() where f() returning 'nil' is not a bug. (You mentioned
> that table.move could create that kind of thing. I asked for a real
> scenario, but got no answer.)

I sent the table.move example in another email.

A few more real-world examples:

In the example below, a field may not have a default value:
```
entity[k] = default_value(field)
```
>From https://github.com/Kong/kong/blob/next/kong/db/schema/init.lua#L670

Constructs of the form `t[k] = u[v]` are yet another variant to this.
As in the example below, you may be filling up entries of a table
using fallback values from another one which may or may not exist.
```
build.build_variables[variable] = rockspec.variables[variable]
```
>From https://github.com/luarocks/luarocks/blob/master/src/luarocks/build/make.lua#L75

And then there are the many cases like the ones that Tomás brought up
about optional arguments being stored in tables... the list goes on.

Obviously, I cannot answer you how _many_ programs are out there that
have these constructs. But at least this shows you how easy it is to
find such examples.

> More concretely, I propose a little experiment: suppose that Lua raises
> a run-time error for every assignment of nils to tables except in
> explicit assignments with a constant nil (t[k]=nil). How frequently this
> error will occurr without hinting a hidden bug in the code? (Please,
> any answer should be about some piece of real, useful code where that
> could happen.)

There is only one way to know: make a patched version of Lua with that
behavior and give it a try running some real-world Lua programs. :)

-- Hisham

Reply | Threaded
Open this post in threaded view
|

Re: Strong tables in Lua 5.4

Dirk Laurie-2
In reply to this post by Gregg Reynolds-2
2018-03-19 22:12 GMT+02:00 Gregg Reynolds <[hidden email]>:

>
>
> On Fri, Mar 16, 2018, 6:31 AM Dirk Laurie <[hidden email]> wrote:
>>
>> Define a "strong table" as one in which 't[k]=nil' does not remove the
>> key, it stores nil.
>
>
> Result: two types of tables. Bad idea.
>
> Non sunt multiplicanda entia sine necessitate
>
> (Occam's razor)

Well, there are already four types of table, three of which are
varieties of weak (weak keys, weak values, weak both). So Occam's
razor has already lost its keenness.

Reply | Threaded
Open this post in threaded view
|

Re: Strong tables in Lua 5.4

Javier Guerra Giraldez
In reply to this post by Roberto Ierusalimschy
On 19 March 2018 at 19:08, Roberto Ierusalimschy <[hidden email]> wrote:
> I still think you are exaggerating the problem, but I may be wrong. I
> would like to know how many real programs have the construction
> t[k] = f() where f() returning 'nil' is not a bug.

I do that often.  for example, when generating code from a parsed AST,
 I usually append stuff in a table, calling several functions, either
from a loop (while walking the AST) or explicitly calling several
different functions (when taking each part of the meaning of a
fragment)  on each call, i just do `o[#o+1] = f()`, if the function
returns nil, nothing is appended to the `o` table.

similarly, to remove elements from a list, the "regular" way is

local o = {}
for i, v in ipairs(t) do
  if wanted(v) then
    o[#o+1] = v
  end
end

but sometimes it's easier to do:

local o = {}
for i, v in ipairs(t) do
  o[#o+1] = wanted(v) and v or nil
end

especially when the 'wanted(v)' function already returns `v` (or a
translation of it) or nil


--
Javier

Reply | Threaded
Open this post in threaded view
|

Re: Strong tables in Lua 5.4

Thomas Jericke
In reply to this post by Hisham
On 19.03.2018 17:16, Hisham wrote:

Every attempt of smoothing out the incompatibility ("strong tables",
undef syntax hack, etc.), while well-intentioned, just adds terrible,
terrible pitfalls to the language (e.g. "Of course you may not pass a
strong table to old libraries").

-- Hisham

I assume you have read my post, as your example is an exact quote from my mail.
If my assumption is correct. Can you please elaborate what is so terrible, terrible about it?

Passing a "strong" table to a library that isn't able to cope with it will get you unpredictable results. That is not nice, but it is not too hard to avoid.
There are similar problems with tables that have metatables already. You can actually implement a "strong" table with Lua 5.3 using metatables, therefore the damage has already been done.

I actually still like to believe, that an opt-in solution (only those who need it must use it) is possible within the Lua 5 universe without the language having to take terrible, terrible damage.
--
Thomas
Reply | Threaded
Open this post in threaded view
|

Re: Strong tables in Lua 5.4

Egor Skriptunoff-2
In reply to this post by Roberto Ierusalimschy
On Mon, Mar 19, 2018 at 10:08 PM, Roberto Ierusalimschy wrote:
More concretely, I propose a little experiment: suppose that Lua raises
a run-time error for every assignment of nils to tables except in
explicit assignments with a constant nil (t[k]=nil). How frequently this
error will occurr without hinting a hidden bug in the code? (Please,
any answer should be about some piece of real, useful code where that
could happen.)


https://gist.github.com/Egor-Skriptunoff/1cf4070c7532db0f8ce0b875717801d8#file-integernumberinwords-lua-L33
Here a sequence of words is being created.
Depending on a parameter, an additional word must be inserted in the sequence:
"one hundred one" / "one hundred and one"
The following expression is used:
   table.insert(array, parameter_which_may_be_nil and "word_to_be_inserted")
When parameter is nil, then nothing is inserted.  This is intended logic, not a hidden bug.
Reply | Threaded
Open this post in threaded view
|

Re: Strong tables in Lua 5.4

Dibyendu Majumdar
In reply to this post by Dirk Laurie-2
On 19 March 2018 at 17:51, Dirk Laurie <[hidden email]> wrote:

> Lua 5.4 should have a constant of type nil called 'null' [1]. Not a
> new keyword, only a new predefined value.
>
> All the functionality aimed for with nil-in-tables can be achieved this way.
>
> - Being of type nil, 'null' by itself tests false.
> - Not being equal to nil, 'nil' is a non-hole.
> - Can be preassigned to global values (or a value in any table) so
> that __index can just signal an error.
>

Hi, I agree partly with your idea. Maybe Lua can have a well defined
singleton object call MISSING or NONE or UNDEFINED. Say it is a global
value which is actually just a Userdata.

Nil should remain as is; and semantics for Nil should remain as is.

Anyone wanting to put values that do not exist will simply set this to
MISSING (If that is the global name).

MISSING will evaluate to True not false as it is just another value,
albeit a special one.
MISSING will only be equal to another reference to MISSING and nothing else.

In statistics you often have missing values. This is hard to model in
Lua tables currently but above approach would work.

Unlike others I personally do not think a break in backwards
compatibility is at all desirable even for Lua 6.0. Introducing a
subtype of Nil or Nil in tables has the problem that all existing Lua
code relies upon the invariant that Nils do not appear when iterating.

Regards
Dibyendu

Reply | Threaded
Open this post in threaded view
|

Re: Strong tables in Lua 5.4

Sean Conner
It was thus said that the Great Dibyendu Majumdar once stated:

> On 19 March 2018 at 17:51, Dirk Laurie <[hidden email]> wrote:
> > Lua 5.4 should have a constant of type nil called 'null' [1]. Not a
> > new keyword, only a new predefined value.
> >
> > All the functionality aimed for with nil-in-tables can be achieved this way.
> >
> > - Being of type nil, 'null' by itself tests false.
> > - Not being equal to nil, 'nil' is a non-hole.
> > - Can be preassigned to global values (or a value in any table) so
> > that __index can just signal an error.
> >
>
> Hi, I agree partly with your idea. Maybe Lua can have a well defined
> singleton object call MISSING or NONE or UNDEFINED. Say it is a global
> value which is actually just a Userdata.

  Um ... Roberto just said that "false" was introduced into Lua *just for
this purpose*.

  -spc


Reply | Threaded
Open this post in threaded view
|

Re: Strong tables in Lua 5.4

Hisham
In reply to this post by Thomas Jericke
On 20 March 2018 at 10:27, Thomas Jericke <[hidden email]> wrote:

> On 19.03.2018 17:16, Hisham wrote:
>
>
> Every attempt of smoothing out the incompatibility ("strong tables",
> undef syntax hack, etc.), while well-intentioned, just adds terrible,
> terrible pitfalls to the language (e.g. "Of course you may not pass a
> strong table to old libraries").
>
> -- Hisham
>
> I assume you have read my post, as your example is an exact quote from my
> mail.
> If my assumption is correct. Can you please elaborate what is so terrible,
> terrible about it?
>
> Passing a "strong" table to a library that isn't able to cope with it will
> get you unpredictable results. That is not nice, but it is not too hard to
> avoid.
> There are similar problems with tables that have metatables already. You can
> actually implement a "strong" table with Lua 5.3 using metatables, therefore
> the damage has already been done.
>
> I actually still like to believe, that an opt-in solution (only those who
> need it must use it) is possible within the Lua 5 universe without the
> language having to take terrible, terrible damage.

First of all, I meant no offense. Lua (like every other language, I
guess) already has major pitfalls (#t being one of them). The pitfall
I was referring to would be the situation where something like {...}
creates a strong table in Lua 5.4+ and a normal table in Lua 5.3 and
below. _Fully opt-in_ strong tables, as you said, are doable in Lua
5.3 today already, and they would not cause terrible, terrible damage
at all, because their use would be explicit (like setting metatables).
As you said, those who need it, use it, and are aware of it.

But I when consider the proposal of strong tables, what I have in mind
is not only "nulls in JSON arrays" but the whole spectrum of problems
that -DLUA_NILINTABLE is aiming to solve, including things like {...}
and {f()} (i.e. the need for select()/table.pack()/table.unpack() in
the language). If constructs like this become strong tables by
default, then it will not fit the Lua 5 universe without bringing
hard-to-track compatibility issues.

(In short, I consider a source of hard-to-track compatibility issues
any proposal that requires a complete dependencies-deep code audit to
ensure that Lua 5.x code will remain working as intended without
introducing memory leaks.)

Each idea has pros and cons. Egor's proposal, keeping Lua 5 table
behavior by default but adding new syntax for tables that behave
differently, is an interesting one. Yes, it means that Lua 6 code
would not parse under Lua 5.x, but it would mean that all existing Lua
5 modules would run on Lua 6 the way their authors intended. Its major
con is that if {...#} means "pack all arguments into this table
including any nils" [1], then we'd get in a weird JavaScript-like
situation where "=== is the correct equality and == is the old one you
shouldn't use anymore but we can't take away", because {...#} would be
the one that "always works" (and {...} would become a pitfall in
itself).

-- Hisham

[1] via either Egor's original proposal of a user-settable length or
combining it with Dirk's proposal to make {...#} the strong-table
constructor

Reply | Threaded
Open this post in threaded view
|

Re: Strong tables in Lua 5.4

Hisham
In reply to this post by Sean Conner
On 20 March 2018 at 18:38, Sean Conner <[hidden email]> wrote:

> It was thus said that the Great Dibyendu Majumdar once stated:
>> On 19 March 2018 at 17:51, Dirk Laurie <[hidden email]> wrote:
>> > Lua 5.4 should have a constant of type nil called 'null' [1]. Not a
>> > new keyword, only a new predefined value.
>> >
>> > All the functionality aimed for with nil-in-tables can be achieved this way.
>> >
>> > - Being of type nil, 'null' by itself tests false.
>> > - Not being equal to nil, 'nil' is a non-hole.
>> > - Can be preassigned to global values (or a value in any table) so
>> > that __index can just signal an error.
>> >
>>
>> Hi, I agree partly with your idea. Maybe Lua can have a well defined
>> singleton object call MISSING or NONE or UNDEFINED. Say it is a global
>> value which is actually just a Userdata.
>
>   Um ... Roberto just said that "false" was introduced into Lua *just for
> this purpose*.

Which, with all due respect I obviously have towards Roberto, I think
it's a very odd choice and the wrong motivation for booleans. As a Lua
programmer, I love having first-class booleans in the language, but it
would never occur to me to use "false" them to represent "missing
value" -- false means false. And true is not useless in Lua, it means
true, and allows us to state our intent in an unambiguous and
interoperable way. Plus, if we didn't have booleans, we'd be
complaining not only about how to convert `null` to and from other
formats, but also `true` and `false`.

If I was to set out to fix the "holes in arrays/interop with JSON
etc." alone, then I think I would go with Dibyendu's idea and
introduce a new constant in the standard library, a lightuserdata
called NULL that contains a NULL C pointer. If the language predefined
NULL, I'm sure everyone would start using that as the convention for
converting to and from null (instead of our current situation where
each library declares its own json.null, ngx.null, etc.) This is a
sort of "solution by convention", akin to adding <stdbool.h> to C to
fix the lack of agreed-upon booleans.

However, the nils-in-tables proposal came up because the Lua Team is
clearly looking beyond this single practical problem that many of us
have and that was outlined in the previous paragraph. They are aiming
to solve that plus a number of quirks in the language (the things that
give birth to select(), table.pack() and table.unpack()) in one fell
swoop. I applaud them for going after the root cause.

But while many solutions can fix the former specific problem, with
various degrees of compatibility concerns — ranging from virtually
none (adding a NULL constant which can be easily emulated in 5.1-5.3)
to very high (anything that changes the default behavior of #t) —
addressing the issue of nils-in-tables in the proper general way (i.e.
make {...} really "store all arguments") cannot be done without a
considerable compatibility break.

-- Hisham

Reply | Threaded
Open this post in threaded view
|

Re: Strong tables in Lua 5.4

Dirk Laurie-2
In reply to this post by Hisham
2018-03-21 1:14 GMT+02:00 Hisham <[hidden email]>:

> [1] via either Egor's original proposal of a user-settable length or
> combining it with Dirk's proposal to make {...#} the strong-table
> constructor

That was not my proposal — it was proposed by "pocomane".

My suggestion was only: if we must have strong tables, do it on a
per-table basis using the existing __mode metafield rather than across
the board.

-- Dirk

Reply | Threaded
Open this post in threaded view
|

Re: Strong tables in Lua 5.4

Dirk Laurie-2
In reply to this post by Dibyendu Majumdar
2018-03-20 23:24 GMT+02:00 Dibyendu Majumdar <[hidden email]>:

> On 19 March 2018 at 17:51, Dirk Laurie <[hidden email]> wrote:
>> Lua 5.4 should have a constant of type nil called 'null' [1]. Not a
>> new keyword, only a new predefined value.
>>
>> All the functionality aimed for with nil-in-tables can be achieved this way.
>>
>> - Being of type nil, 'null' by itself tests false.
>> - Not being equal to nil, 'nil' is a non-hole.
>> - Can be preassigned to global values (or a value in any table) so
>> that __index can just signal an error.
>>
>
> Hi, I agree partly with your idea. Maybe Lua can have a well defined
> singleton object call MISSING or NONE or UNDEFINED. Say it is a global
> value which is actually just a Userdata.
>
> Nil should remain as is; and semantics for Nil should remain as is.
>
> Anyone wanting to put values that do not exist will simply set this to
> MISSING (If that is the global name).
>
> MISSING will evaluate to True not false as it is just another value,
> albeit a special one.

I am not dogmatic about it. I suggested it because the current
implementation at the C level in effect has a value of _type_ nil
different from the _value_ nil. No reason why it can't be say a light
userdata instead.

Maybe the name should rather start with an underscore, since Lua
programmers have been taught not to use such names for their own
purposes.

If _EMPTY has truth value 'true', backward compatiblity would be possible:

    if _EMPTY == nil then _EMPTY = function() return end end

One might argue that this would not be a new language feature at all.
Most JSON parsers that I know (e.g. 'cjson' and 'conman.json') do that
(using the name 'null'). But having _EMPTY available, and documented,
would make such subterfuges unnecessary.

Also, existing programs that accept user-written functions can, when
loaded, test for a non-nil _EMPTY and issue a warning or error.

Reply | Threaded
Open this post in threaded view
|

Re: Strong tables in Lua 5.4

Thomas Jericke
In reply to this post by Hisham
On 21.03.2018 00:14, Hisham wrote:
On 20 March 2018 at 10:27, Thomas Jericke [hidden email] wrote:
On 19.03.2018 17:16, Hisham wrote:


Every attempt of smoothing out the incompatibility ("strong tables",
undef syntax hack, etc.), while well-intentioned, just adds terrible,
terrible pitfalls to the language (e.g. "Of course you may not pass a
strong table to old libraries").

-- Hisham

I assume you have read my post, as your example is an exact quote from my
mail.
If my assumption is correct. Can you please elaborate what is so terrible,
terrible about it?

Passing a "strong" table to a library that isn't able to cope with it will
get you unpredictable results. That is not nice, but it is not too hard to
avoid.
There are similar problems with tables that have metatables already. You can
actually implement a "strong" table with Lua 5.3 using metatables, therefore
the damage has already been done.

I actually still like to believe, that an opt-in solution (only those who
need it must use it) is possible within the Lua 5 universe without the
language having to take terrible, terrible damage.
First of all, I meant no offense. Lua (like every other language, I
guess) already has major pitfalls (#t being one of them). The pitfall
I was referring to would be the situation where something like {...}
creates a strong table in Lua 5.4+ and a normal table in Lua 5.3 and
below. _Fully opt-in_ strong tables, as you said, are doable in Lua
5.3 today already, and they would not cause terrible, terrible damage
at all, because their use would be explicit (like setting metatables).
As you said, those who need it, use it, and are aware of it.

But I when consider the proposal of strong tables, what I have in mind
is not only "nulls in JSON arrays" but the whole spectrum of problems
that -DLUA_NILINTABLE is aiming to solve, including things like {...}
and {f()} (i.e. the need for select()/table.pack()/table.unpack() in
the language). If constructs like this become strong tables by
default, then it will not fit the Lua 5 universe without bringing
hard-to-track compatibility issues.

(In short, I consider a source of hard-to-track compatibility issues
any proposal that requires a complete dependencies-deep code audit to
ensure that Lua 5.x code will remain working as intended without
introducing memory leaks.)

Each idea has pros and cons. Egor's proposal, keeping Lua 5 table
behavior by default but adding new syntax for tables that behave
differently, is an interesting one. Yes, it means that Lua 6 code
would not parse under Lua 5.x, but it would mean that all existing Lua
5 modules would run on Lua 6 the way their authors intended. Its major
con is that if {...#} means "pack all arguments into this table
including any nils" [1], then we'd get in a weird JavaScript-like
situation where "=== is the correct equality and == is the old one you
shouldn't use anymore but we can't take away", because {...#} would be
the one that "always works" (and {...} would become a pitfall in
itself).

-- Hisham

[1] via either Egor's original proposal of a user-settable length or
combining it with Dirk's proposal to make {...#} the strong-table
constructor

I wasn't offended. Actually I had to smile at the use of the phrase "terrible, terrible".

I just think it's unfair to criticize the proposal of strong tables using examples that are actually not part of the original proposal.
That pretty much the definition of a straw man argument.

In your original post you said that *every* attempt to smooth out incompatibility will lead into pitfalls,
while actually the original proposal doesn't add new pitfalls.
All it does is to standardize something that is already possible within the current version of Lua.
And hopefully it will highly boost the performance of this pattern, as implementing strong tables with
metamethods need at least two extra tables IMO.

I pretty much agree on everything else that you said. That's why I don't like the undef idea.
I neither like the idea, that existing syntax should return strong tables.
Function redefining may be possible, as it is usually possible to search for function names like "pack" and "unpack" over a project easily.
But then again, it might be better to just introduce a table.strongpack that packs the arguments into a strong table.

I see no problem in adding a new syntax that creates strong tables, as long as it is possible to use the existing syntax as well.
That would mean, the programmer can decide himself if his code is backward compatible or not.
--
Thomas

Reply | Threaded
Open this post in threaded view
|

Re: Strong tables in Lua 5.4

Alexander Nasonov
In reply to this post by Dirk Laurie-2
Dirk Laurie wrote:
> I am not dogmatic about it. I suggested it because the current
> implementation at the C level in effect has a value of _type_ nil
> different from the _value_ nil. No reason why it can't be say a light
> userdata instead.

Sounds like you're introducing NULL (light userdata edition) ;-)

--
Alex

Reply | Threaded
Open this post in threaded view
|

Re: Strong tables in Lua 5.4

Thomas Jericke
In reply to this post by Hisham
On 21.03.2018 00:30, Hisham wrote:
On 20 March 2018 at 18:38, Sean Conner [hidden email] wrote:
It was thus said that the Great Dibyendu Majumdar once stated:
On 19 March 2018 at 17:51, Dirk Laurie [hidden email] wrote:
Lua 5.4 should have a constant of type nil called 'null' [1]. Not a
new keyword, only a new predefined value.

All the functionality aimed for with nil-in-tables can be achieved this way.

- Being of type nil, 'null' by itself tests false.
- Not being equal to nil, 'nil' is a non-hole.
- Can be preassigned to global values (or a value in any table) so
that __index can just signal an error.

Hi, I agree partly with your idea. Maybe Lua can have a well defined
singleton object call MISSING or NONE or UNDEFINED. Say it is a global
value which is actually just a Userdata.
  Um ... Roberto just said that "false" was introduced into Lua *just for
this purpose*.
Which, with all due respect I obviously have towards Roberto, I think
it's a very odd choice and the wrong motivation for booleans. As a Lua
programmer, I love having first-class booleans in the language, but it
would never occur to me to use "false" them to represent "missing
value" -- false means false. And true is not useless in Lua, it means
true, and allows us to state our intent in an unambiguous and
interoperable way. 
Couldn't agree more. False is to me a way to explicitly say "this value is not missing it is untrue" while nil is a missing/undefined value.
And following the discussion here, it seems that false didn't solve the storing nil into a table problem after all.
Either because it was not the right value for the job or because no value could fill that job. I am tending to the first answer.
--
Thomas
Reply | Threaded
Open this post in threaded view
|

Re: Strong tables in Lua 5.4

Dibyendu Majumdar
In reply to this post by Dibyendu Majumdar


On Tuesday, 20 March 2018, Dibyendu Majumdar <[hidden email]> wrote:

> On 19 March 2018 at 17:51, Dirk Laurie <[hidden email]> wrote:
>> Lua 5.4 should have a constant of type nil called 'null' [1]. Not a
>> new keyword, only a new predefined value.
>>
>> All the functionality aimed for with nil-in-tables can be achieved this way.
>>
>> - Being of type nil, 'null' by itself tests false.
>> - Not being equal to nil, 'nil' is a non-hole.
>> - Can be preassigned to global values (or a value in any table) so
>> that __index can just signal an error.
>>
>
> Hi, I agree partly with your idea. Maybe Lua can have a well defined
> singleton object call MISSING or NONE or UNDEFINED. Say it is a global
> value which is actually just a Userdata.
>
> Nil should remain as is; and semantics for Nil should remain as is.
>
> Anyone wanting to put values that do not exist will simply set this to
> MISSING (If that is the global name).
>
> MISSING will evaluate to True not false as it is just another value,
> albeit a special one.
> MISSING will only be equal to another reference to MISSING and nothing else.
>
> In statistics you often have missing values. This is hard to model in
> Lua tables currently but above approach would work.
>
> Unlike others I personally do not think a break in backwards
> compatibility is at all desirable even for Lua 6.0. Introducing a
> subtype of Nil or Nil in tables has the problem that all existing Lua
> code relies upon the invariant that Nils do not appear when iterating.
>

Btw just giving credit to
Sergey Zakharchenk who originally suggested something similar.
Reply | Threaded
Open this post in threaded view
|

Re: Strong tables in Lua 5.4

Sergey Zakharchenko

Hello,

Dibyendu Majumdar <[hidden email]>:
> Btw just giving credit to
> Sergey Zakharchenk who originally suggested something similar.

Can't take much credit for this either as it's just existing cjson, etc. practice. The missing pieces are not being false in a boolean context and possibly the type. Having its type be 'nil' might sound fine, or crazy, depending on how you look at it...

Best regards,

--
DoubleF

Reply | Threaded
Open this post in threaded view
|

Re: Strong tables in Lua 5.4

Roberto Ierusalimschy
In reply to this post by Egor Skriptunoff-2
Many thanks for the examples. All of them actually would raise errors in
my little experiment, but let us see them in the discussion about nils
in tables.

The example from Tomás, about optional parameters to functions with
named arguments, is fully compatible with nils in tables. (It is really
adding a nil to a table, just it makes no difference that the nil is not
there.)

In the examples from Hisham, I could not figure out whether we should see
the nils as actual values or "instructions" to not add something.
For instance, in the case of

  entity[k] = default_value(field)

does a nil mean no default or a nil as the default? Would it make any
difference to the program if the nil was added to the table?

The example from Javier, however, clearly uses a nil as an "instruction"
not to add something. So, it is directly incompatible with the nils in
tables idea.

The example from Egor let me wondering why not to write

  if parameter_which_may_be_nil then
     table.insert(array, "word_to_be_inserted")
  end

instead of

     table.insert(array, parameter_which_may_be_nil and "word_to_be_inserted")

It is clearer (in my view) and faster; ok, I grant the original uses less
characters :-). Anyway, it is another valid example that is directly
incompatible with the nils in tables idea.

-- Roberto

Reply | Threaded
Open this post in threaded view
|

Re: Strong tables in Lua 5.4

Hisham
On 21 March 2018 at 13:06, Roberto Ierusalimschy <[hidden email]> wrote:
> Many thanks for the examples. All of them actually would raise errors in
> my little experiment, but let us see them in the discussion about nils
> in tables.

I hope this shows how easy it is to find examples of the construct I
was talking about. :)

> In the examples from Hisham, I could not figure out whether we should see
> the nils as actual values or "instructions" to not add something.
> For instance, in the case of
>
>   entity[k] = default_value(field)
>
> does a nil mean no default or a nil as the default? Would it make any
> difference to the program if the nil was added to the table?

In both cases it is an instruction to not add something. In
default_value(), because we have to interact with the outside world
(converting to and from JSON), when the default is "null" we return it
as ngx.null.

And I hope this shows how non-trivial it is to audit working codebases
regarding their usage of nil. :)

-- Hisham

Reply | Threaded
Open this post in threaded view
|

Re: Strong tables in Lua 5.4

Italo Maia
So, does "missing"/"undefined" singleton for handling holes in tables have a verdict? Seems quite compatible and easy to handle in old/new code.

2018-03-21 14:10 GMT-03:00 Hisham <[hidden email]>:
On 21 March 2018 at 13:06, Roberto Ierusalimschy <[hidden email]> wrote:
> Many thanks for the examples. All of them actually would raise errors in
> my little experiment, but let us see them in the discussion about nils
> in tables.

I hope this shows how easy it is to find examples of the construct I
was talking about. :)

> In the examples from Hisham, I could not figure out whether we should see
> the nils as actual values or "instructions" to not add something.
> For instance, in the case of
>
>   entity[k] = default_value(field)
>
> does a nil mean no default or a nil as the default? Would it make any
> difference to the program if the nil was added to the table?

In both cases it is an instruction to not add something. In
default_value(), because we have to interact with the outside world
(converting to and from JSON), when the default is "null" we return it
as ngx.null.

And I hope this shows how non-trivial it is to audit working codebases
regarding their usage of nil. :)

-- Hisham




--
"A arrogância é a arma dos fracos."

===========================
Me. Italo Moreira Campelo Maia
Co-fundador do Grupo de Usuários Python do Ceará
Secretário ForHacker (fb.com/ForHackerSpace)
Desenvolvedor Full-Stack, Escritor, Empresário, Visionário
-----------------------------------------------------
Meu Livro, Site, Blog
===========================
123