[PATCH] setmetatable({}) causes error

classic Classic list List threaded Threaded
29 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Re: [PATCH] setmetatable({}) causes error

Andrew Starks-2

On Thu, Jul 13, 2017 at 11:32 Viacheslav Usov <[hidden email]> wrote:
On Thu, Jul 13, 2017 at 6:12 PM, Roberto Ierusalimschy <[hidden email]> wrote:

> I hope you do not expect that the previous sentence ("Parameters act as local variables that are initialized with the argument values") applies also to C functions.

In fact, at some point back in the past, that is exactly what I expected. I expected that an omitted argument would be seen as nil in a C function. I learned the actual behaviour only when I had to chase some subtle bugs introduced by my expectation. I did not like the experience. That is why I called that arcane and confusing.

Another problem is that your language above implies that we somehow must expect that setmetatable should behave like a "C function" and not a like a Lua function. Why? I do not think "because it is implemented in C is a sensible answer for someone who has no idea about the implementation of Lua. And why, indeed, should we expect that C functions behave differently? The difference is explained very compactly at the end of section 4.3, but it only makes sense for a C programmer, not for a Lua programmer (a pure Lua programmer would not even look there), who still sees there "virtual type LUA_TNONE, which behaves like a nil value". Does that imply that every user of Lua should understand the arcane details of Lua's C language API? I think that would be too much to expect.

> Similarly for the next sentence ("A vararg function [...] collects all extra arguments and supplies them to the function through a vararg expression").  So, I see no reason to expect that this particular sentence would apply to C functions.

In fact, the behaviour of C functions can be most easily understood by assuming they are always vararg. Even then the TNONE/TNIL difference remains, but at least the "adjustment" is taken out.

Cheers,
V.

Your argument is completely valid. To be fair, it's pretty simple to create a Lua function that errors when an argument is omitted:

(function(...) if select('#',...) <1 then error("no argument") end return "did stuff" end)() --errors

It's much easier to do this on the C side, or at least it doesn't require a function call to select. 

I think the reason that the choice was made in this way is more of a matter of design. The thinking might be that by making sure that you have to use nil explicitly, you are forced to be more verbose about your intent. 

If it was done that way simply because it's a C function, then I think that is a good argument to be made in favor of a change to the Lua library, imho. If it's a design question, then patch away and use it in good health!


-Andrew


Reply | Threaded
Open this post in threaded view
|

Lua arity tests (for sandbox creators) (was: Re: [PATCH] setmetatable({}) causes error)

Soni "They/Them" L.


On 2017-07-13 06:11 PM, Andrew Starks wrote:

>
> On Thu, Jul 13, 2017 at 11:32 Viacheslav Usov <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     On Thu, Jul 13, 2017 at 6:12 PM, Roberto Ierusalimschy
>     <[hidden email] <mailto:[hidden email]>> wrote:
>
>     > I hope you do not expect that the previous sentence ("Parameters
>     act as local variables that are initialized with the argument
>     values") applies also to C functions.
>
>     In fact, at some point back in the past, that is exactly what I
>     expected. I expected that an omitted argument would be seen as nil
>     in a C function. I learned the actual behaviour only when I had to
>     chase some subtle bugs introduced by my expectation. I did not
>     like the experience. That is why I called that arcane and confusing.
>
>     Another problem is that your language above implies that we
>     somehow must expect that setmetatable should behave like a "C
>     function" and not a like a Lua function. Why? I do not think
>     "because it is implemented in C is a sensible answer for someone
>     who has no idea about the implementation of Lua. And why, indeed,
>     should we expect that C functions behave differently? The
>     difference is explained very compactly at the end of section 4.3,
>     but it only makes sense for a C programmer, not for a Lua
>     programmer (a pure Lua programmer would not even look there), who
>     still sees there "virtual type LUA_TNONE, which behaves like a nil
>     value". Does that imply that every user of Lua should understand
>     the arcane details of Lua's C language API? I think that would be
>     too much to expect.
>
>     > Similarly for the next sentence ("A vararg function [...]
>     collects all extra arguments and supplies them to the function
>     through a vararg expression").  So, I see no reason to expect that
>     this particular sentence would apply to C functions.
>
>     In fact, the behaviour of C functions can be most easily
>     understood by assuming they are always vararg. Even then the
>     TNONE/TNIL difference remains, but at least the "adjustment" is
>     taken out.
>
>     Cheers,
>     V.
>
>
> Your argument is completely valid. To be fair, it's pretty simple to
> create a Lua function that errors when an argument is omitted:
>
> (function(...) if select('#',...) <1 then error("no argument") end
> return "did stuff" end)() --errors
>
> It's much easier to do this on the C side, or at least it doesn't
> require a function call to select.
>
> I think the reason that the choice was made in this way is more of a
> matter of design. The thinking might be that by making sure that you
> have to use nil explicitly, you are forced to be more verbose about
> your intent.
>
> If it was done that way simply because it's a C function, then I think
> that is a good argument to be made in favor of a change to the Lua
> library, imho. If it's a design question, then patch away and use it
> in good health!
>

Has anyone made a testsuite to test these arity errors? It'd be really
useful when making a pure-Lua sandbox/"alternative stdlib" (e.g.
luajit+ffi jitable I/O, luvit's require(), etc).

>
> -Andrew
>
>

--
Disclaimer: these emails may be made public at any given time, with or without reason. If you don't agree with this, DO NOT REPLY.


Reply | Threaded
Open this post in threaded view
|

Re: [PATCH] setmetatable({}) causes error

Roberto Ierusalimschy
In reply to this post by nobody
> Almost… ^,^  The easiest way to understand C functions from Lua is that
> they are always _vararg-only_ functions with no named parameters.  TNONE
> doesn't exist on the Lua side (it's an artifact of indexing beyond top
> of stack) but can be emulated by checking if the index is bigger than
> `select('#',...)`.
>
> So you can actually treat C functions as a feature-limited subset of Lua
> functions, which is easy to understand.

Being vararg does not seem to be "feature-limited", quite the
opposite. We can think as if all functions in Lua were vararg. The
syntax

  function foo (a,b,c)

can be considered basically a sugar for

  function foo (...)  local a,b,c = ...

and, therefore, a restriction over a more permissive vararg handling.

The main difference between Lua functions and C functions is that C
functions do not have this sugar available, and so it is easier (and
more efficient, and maybe more natural) for them to work directly on the
varargs. (A near equivalent to this sugar in C would be to start the C
function with a lua_settop.)

-- Roberto


Reply | Threaded
Open this post in threaded view
|

Re: [PATCH] setmetatable({}) causes error

Soni "They/Them" L.


On 2017-07-13 09:10 PM, Roberto Ierusalimschy wrote:

>> Almost… ^,^  The easiest way to understand C functions from Lua is that
>> they are always _vararg-only_ functions with no named parameters.  TNONE
>> doesn't exist on the Lua side (it's an artifact of indexing beyond top
>> of stack) but can be emulated by checking if the index is bigger than
>> `select('#',...)`.
>>
>> So you can actually treat C functions as a feature-limited subset of Lua
>> functions, which is easy to understand.
> Being vararg does not seem to be "feature-limited", quite the
> opposite. We can think as if all functions in Lua were vararg. The
> syntax
>
>    function foo (a,b,c)
>
> can be considered basically a sugar for
>
>    function foo (...)  local a,b,c = ...
>
> and, therefore, a restriction over a more permissive vararg handling.
>
> The main difference between Lua functions and C functions is that C
> functions do not have this sugar available, and so it is easier (and
> more efficient, and maybe more natural) for them to work directly on the
> varargs. (A near equivalent to this sugar in C would be to start the C
> function with a lua_settop.)
>
> -- Roberto
>
>

If functions had default values, NONE would be replaced with the given
default values, while nil would be used as is. By using a sentinel
value, it would be possible to tell none and nil apart without using
select(). (e.g. in a sandbox)

local sentinel = {}

function foo(v, t=sentinel)
   if t == sentinel then
     error("table or nil expected, got none")
   end
   return setmetatable(v, t)
end

--
Disclaimer: these emails may be made public at any given time, with or without reason. If you don't agree with this, DO NOT REPLY.


Reply | Threaded
Open this post in threaded view
|

Re: [PATCH] setmetatable({}) causes error

Dirk Laurie-2
In reply to this post by Andrew Starks-2
2017-07-13 23:11 GMT+02:00 Andrew Starks <[hidden email]>:

>
> On Thu, Jul 13, 2017 at 11:32 Viacheslav Usov <[hidden email]> wrote:
>>
>> On Thu, Jul 13, 2017 at 6:12 PM, Roberto Ierusalimschy
>> <[hidden email]> wrote:
>>
>> > I hope you do not expect that the previous sentence ("Parameters act as
>> > local variables that are initialized with the argument values") applies also
>> > to C functions.
>>
>> In fact, at some point back in the past, that is exactly what I expected.
>> I expected that an omitted argument would be seen as nil in a C function. I
>> learned the actual behaviour only when I had to chase some subtle bugs
>> introduced by my expectation. I did not like the experience. That is why I
>> called that arcane and confusing.
>>
>> Another problem is that your language above implies that we somehow must
>> expect that setmetatable should behave like a "C function" and not a like a
>> Lua function. Why? I do not think "because it is implemented in C is a
>> sensible answer for someone who has no idea about the implementation of Lua.
>> And why, indeed, should we expect that C functions behave differently? The
>> difference is explained very compactly at the end of section 4.3, but it
>> only makes sense for a C programmer, not for a Lua programmer (a pure Lua
>> programmer would not even look there), who still sees there "virtual type
>> LUA_TNONE, which behaves like a nil value". Does that imply that every user
>> of Lua should understand the arcane details of Lua's C language API? I think
>> that would be too much to expect.
>>
>> > Similarly for the next sentence ("A vararg function [...] collects all
>> > extra arguments and supplies them to the function through a vararg
>> > expression").  So, I see no reason to expect that this particular sentence
>> > would apply to C functions.
>>
>> In fact, the behaviour of C functions can be most easily understood by
>> assuming they are always vararg. Even then the TNONE/TNIL difference
>> remains, but at least the "adjustment" is taken out.
>>
>> Cheers,
>> V.
>
>
> Your argument is completely valid. To be fair, it's pretty simple to create
> a Lua function that errors when an argument is omitted:
>
> (function(...) if select('#',...) <1 then error("no argument") end return
> "did stuff" end)() --errors
>
> It's much easier to do this on the C side, or at least it doesn't require a
> function call to select.
>
> I think the reason that the choice was made in this way is more of a matter
> of design. The thinking might be that by making sure that you have to use
> nil explicitly, you are forced to be more verbose about your intent.
>
> If it was done that way simply because it's a C function, then I think that
> is a good argument to be made in favor of a change to the Lua library, imho.
> If it's a design question, then patch away and use it in good health!

As has been demonstrated, it is not a Lua/C peculiarity whether
a missing argument is treated as nil. You can do whatever you prefer
in either language.

Personally I think it is quite reasonable for anyone to implement
a function to have whatever API they like, as long as it is documented.

So, the argument that the manual has

     setmetatable (table, metatable)

and not `setmetatable (table[, metatable])` is to my mind a clincher.

It is totally futile to argue otherwise; it's not going to change. The tiny
handful among us who actually bother to read the manual thoroughly
define "expected behaviour" = "documented behaviour". We _expect_
Lua to give an error message in cases like this. We have come to
rely on it. Working on a patched Lua in which it is perfectly OK to
omit arguments documented as being compulsory would confuse _us_.

-- Dirk

Reply | Threaded
Open this post in threaded view
|

Re: [PATCH] setmetatable({}) causes error

Viacheslav Usov
In reply to this post by Roberto Ierusalimschy
On Fri, Jul 14, 2017 at 2:10 AM, Roberto Ierusalimschy <[hidden email]> wrote:

> The syntax
>
>   function foo (a,b,c)
>
> can be considered basically a sugar for
>
>   function foo (...)  local a,b,c = ...

There is more than sugar to that, because there is a distinction between a vararg and non-vararg function in Lua:

local function f1(...)
  print(select('#', ...))
end

local function f2(a, b)
  print(select('#', ...))
end

f1(1)
f2(1)

The above does not compile, producing "cannot use '...' outside a vararg function near '...'" in the body of f2.

And even if that compiled, then select(') would have to behave differently in those two cases, because it should skip non-vararg arguments.

The other difference, which is the subject of this thread, is that ordinary functions have the nil substitution as part of the language, while vararg functions are free to emulate that behaviour or do something else. In the pure Lua world, the emulation follows naturally, unless the programmer gets smart about counting the number of arguments with select(). In the C underworld, it is easier for the programmer to do "something else", because the extra "none" type makes that possible without counting the number of arguments. Even when this is not intended, this is a mistake that is easy to make (I have done that myself and seen others do, too).

This behaviour is confusing for Lua programmers when some functions, ostensibly documented as non-vararg, do not behave as ordinary Lua functions. For a Lua programmer it is frequently impossible or at least difficult to say whether a given function is implemented in C or in Lua, even if these differences are understood.

To make matters worse, advanced programmers, like you and LHF, find the C-function behaviour so natural that they assume everybody else understands and expects that. In this very thread, both of you silently assumed that setmetatable(), despite not being documented as vararg, is permitted to behave as if it were vararg. I think the disconnect between the expectations is what this thread is really about.


Cheers,
V.
Reply | Threaded
Open this post in threaded view
|

Re: [PATCH] setmetatable({}) causes error

Roberto Ierusalimschy
> On Fri, Jul 14, 2017 at 2:10 AM, Roberto Ierusalimschy <
> [hidden email]> wrote:
>
> > The syntax
> >
> >   function foo (a,b,c)
> >
> > can be considered basically a sugar for
> >
> >   function foo (...)  local a,b,c = ...
>
> There is more than sugar to that, because there is a distinction between a
> vararg and non-vararg function in Lua:

Yes. That is why there was a "basically" in my sentence... (In your
nitpicking, you forgot to mention that the debug information is also
different in the two versions :-)

Anyway, my point was that it is not "Lua" who does the adjustment, it
is the function itself. Adjustment is not a general property of the
language, but a property of non-vararg Lua functions, which do
the adjustment when they do the equivalent to that initial assignment.
(Not by change, the adjustment of parameters follows exactly the
rules for adjustment in assignment.)

-- Roberto

Reply | Threaded
Open this post in threaded view
|

Re: [PATCH] setmetatable({}) causes error

Viacheslav Usov
On Fri, Jul 14, 2017 at 2:54 PM, Roberto Ierusalimschy <[hidden email]> wrote:

> (In your nitpicking, you forgot to mention that the debug information is also different in the two versions :-)

I am not nitpicking and I am sorry to have made that impression. I used that to demonstrate that there was more than just a syntactical difference. In retrospect, I can say that perhaps it was not necessary because you did not mean there was only a syntactical difference.

> Anyway, my point was that it is not "Lua" who does the adjustment, it is the function itself. Adjustment is not a general property of the language, but a property of non-vararg Lua functions, which do the adjustment when they do the equivalent to that initial assignment.

Well, I still think this is a language feature, but we are debating terminology here.

As I said in the previous message, the disconnect is that what is documented as a non-vararg function, behaves as a vararg function. And the other disconnect is in your assumption that the readers of the manual should know better than to expect that the "standard library" functions behave like ordinary non-vararg Lua functions. The starter of this thread obviously had a different assumption, and so did I in the past.

Cheers,
V.
Reply | Threaded
Open this post in threaded view
|

Re: [PATCH] setmetatable({}) causes error

Roberto Ierusalimschy
> As I said in the previous message, the disconnect is that what is
> documented as a non-vararg function, behaves as a vararg function. And the
> other disconnect is in your assumption that the readers of the manual
> should know better than to expect that the "standard library" functions
> behave like ordinary non-vararg Lua functions. The starter of this thread
> obviously had a different assumption, and so did I in the past.

We intend to make the manual clearer about that.

-- Roberto

12