Returning nil vs returning nothing

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
16 messages Options
Reply | Threaded
Open this post in threaded view
|

Returning nil vs returning nothing

Grzegorz Krasoń
Function that returns nothing seems different than function that returns `nil`:

```lua
function f()
    return nil
end

function g()
end

print(f()) -- prints nil
print(g()) -- prints empty line
```

How to recognize this at the function call? For example how to implement an universal wrapper that would suppress all errors and still returns exactly the same thing as wrapped function in case of success?

```lua
function supperess(f, ...)
    success, result = pcall(f, ...)
    return result
end

print(supperess(f)) -- prints nil
print(supperess(g)) -- also prints nil, how to fix it?
```
Reply | Threaded
Open this post in threaded view
|

Re: Returning nil vs returning nothing

Viacheslav Usov
On Tue, Oct 27, 2020 at 8:26 PM Grzegorz Krasoń
<[hidden email]> wrote:

> How to recognize this at the function call? For example how to implement an universal wrapper that would suppress all errors and still returns exactly the same thing as wrapped function in case of success?

function ret(...)
    local n = select('#', ...)
    local t = table.pack(...)
    return table.unpack(t, 2, n)
end

function supperess(f, ...)
    return ret(pcall(f, ...))
end

print(supperess(f)) -- prints nil
print(supperess(g)) -- prints empty line

Cheers,
V.
Reply | Threaded
Open this post in threaded view
|

Re: Returning nil vs returning nothing

nobody
In reply to this post by Grzegorz Krasoń
(written without access to a Lua interpreter, bugs unlikely but possible)

   local function xpack( x, ... )
     return x, table.pack( ... )
   end
   function suppress( f, ... )
     local _, rets = xpack( pcall( f, ... ) )
     return table.unpack( rets, 1, rets.n )
   end

-- nobody

On 27/10/2020 20.25, Grzegorz Krasoń wrote:

> Function that returns nothing seems different than function that returns
> `nil`:
>
> ```lua
> function f()
>      return nil
> end
>
> function g()
> end
>
> print(f()) -- prints nil
> print(g()) -- prints empty line
> ```
>
> How to recognize this at the function call? For example how to implement
> an universal wrapper that would suppress all errors and still returns
> exactly the same thing as wrapped function in case of success?
>
> ```lua
> function supperess(f, ...)
>      success, result = pcall(f, ...)
>      return result
> end
>
> print(supperess(f)) -- prints nil
> print(supperess(g)) -- also prints nil, how to fix it?
> ```
Reply | Threaded
Open this post in threaded view
|

Re: Returning nil vs returning nothing

Grzegorz Krasoń
Both versions work like a charm. Thanks a million!

On Tue, 27 Oct 2020 at 20:47, nobody <[hidden email]> wrote:
(written without access to a Lua interpreter, bugs unlikely but possible)

   local function xpack( x, ... )
     return x, table.pack( ... )
   end
   function suppress( f, ... )
     local _, rets = xpack( pcall( f, ... ) )
     return table.unpack( rets, 1, rets.n )
   end

-- nobody

On 27/10/2020 20.25, Grzegorz Krasoń wrote:
> Function that returns nothing seems different than function that returns
> `nil`:
>
> ```lua
> function f()
>      return nil
> end
>
> function g()
> end
>
> print(f()) -- prints nil
> print(g()) -- prints empty line
> ```
>
> How to recognize this at the function call? For example how to implement
> an universal wrapper that would suppress all errors and still returns
> exactly the same thing as wrapped function in case of success?
>
> ```lua
> function supperess(f, ...)
>      success, result = pcall(f, ...)
>      return result
> end
>
> print(supperess(f)) -- prints nil
> print(supperess(g)) -- also prints nil, how to fix it?
> ```
Reply | Threaded
Open this post in threaded view
|

Re: Returning nil vs returning nothing

Andrew Gierth
In reply to this post by Grzegorz Krasoń
>>>>> "Grzegorz" == Grzegorz Krasoń <[hidden email]> writes:

 Grzegorz> How to recognize this at the function call? For example how
 Grzegorz> to implement an universal wrapper that would suppress all
 Grzegorz> errors and still returns exactly the same thing as wrapped
 Grzegorz> function in case of success?

The examples that use pack/unpack seem misguided since they're
allocating a table unnecessarily.

Instead, how about:

function suppress(func, ...)
  local function drop1(success, ...)
    if success then
      return ...
    else
      return
    end
  end
  return drop1(pcall(func, ...))
end

--
Andrew.
Reply | Threaded
Open this post in threaded view
|

Re: Returning nil vs returning nothing

Viacheslav Usov
On Tue, Oct 27, 2020 at 11:38 PM Andrew Gierth
<[hidden email]> wrote:

>
> >>>>> "Grzegorz" == Grzegorz Krasoń <[hidden email]> writes:
>
>  Grzegorz> How to recognize this at the function call? For example how
>  Grzegorz> to implement an universal wrapper that would suppress all
>  Grzegorz> errors and still returns exactly the same thing as wrapped
>  Grzegorz> function in case of success?
>
> The examples that use pack/unpack seem misguided since they're
> allocating a table unnecessarily.

Speaking of my example, it answered the two questions above, not just
the second one. PiL 3 recommends that tables are the general way to
handle variadic arguments. There was a discussion and some
benchmarking here four-five years ago, when it was found that one
should worry about the impact of table creation only when the actual
number of variadic arguments is 0, 1 or 2. With progressively more
variadic arguments, they are first on par and then (vastly) better
than other ways of handling variadic arguments.

It is true that in the answer to the second question tables are
redundant, but more generally they are the way to go.

For example, I do not think there is a straight-forward way to pass
all but the last return value, or more generally any that is not an
N-th return value, or pass no more than first N return values, even if
N is statically fixed. With tables this is trivial and performs well,
for N both static and dynamic.

Cheers,
V.
Reply | Threaded
Open this post in threaded view
|

Re: Returning nil vs returning nothing

Andrew Gierth
>>>>> "Viacheslav" == Viacheslav Usov <[hidden email]> writes:

 >> The examples that use pack/unpack seem misguided since they're
 >> allocating a table unnecessarily.

 Viacheslav> Speaking of my example, it answered the two questions
 Viacheslav> above, not just the second one. PiL 3 recommends that
 Viacheslav> tables are the general way to handle variadic arguments.
 Viacheslav> There was a discussion and some benchmarking here four-five
 Viacheslav> years ago, when it was found that one should worry about
 Viacheslav> the impact of table creation only when the actual number of
 Viacheslav> variadic arguments is 0, 1 or 2. With progressively more
 Viacheslav> variadic arguments, they are first on par and then (vastly)
 Viacheslav> better than other ways of handling variadic arguments.

Given:

local function foo1(f,...)
        local function drop1(s,...)
                if s then
                        return ...
                else
                        return
                end
        end
        drop1(f(...))
end

local function foo2(f,...)
        local r = table.pack(f(...))
        if r[1] then
                return table.unpack(r, 2, r.n)
        else
                return
        end
end

then averaged over a large number of calls, with 5 to 50 results in
the list, foo2 is around 3x slower than foo1. The proportional speed
difference does not seem to be narrowing for longer argument lists.
(For short lists the difference is narrower, but the non-table version
is always at least 2x faster.)

Now, for something other than "drop the first result" which happens to
be easy without needing a table, it might well be simpler to use a table
(dropping the last result, for example, would require a lot of
recursion).

--
Andrew.
Reply | Threaded
Open this post in threaded view
|

Re: Returning nil vs returning nothing

William Ahern
On Thu, Oct 29, 2020 at 12:20:54AM +0000, Andrew Gierth wrote:
<snip>
> Now, for something other than "drop the first result" which happens to
> be easy without needing a table, it might well be simpler to use a table
> (dropping the last result, for example, would require a lot of
> recursion).

Dropping trailing arguments does seem tricky (without the C API), but
dropping the first n arguments is as simple as `return select(n, ...)`.
Reply | Threaded
Open this post in threaded view
|

Re: Returning nil vs returning nothing

Viacheslav Usov
In reply to this post by Andrew Gierth
On Thu, Oct 29, 2020 at 1:21 AM Andrew Gierth
<[hidden email]> wrote:

> then averaged over a large number of calls, with 5 to 50 results in
> the list, foo2 is around 3x slower than foo1. The proportional speed
> difference does not seem to be narrowing for longer argument lists.

I am not sure whether this is trying to dispute what I said. I was
talking about the performance in the general case. I did say that in
the specific case that you are addressing tables were redundant.

This can be seen without benchmarking. The problem with variadic
arguments is that every function call with them is O(n), where n is
the number of arguments. Just the call, ignoring whatever the function
does. The same is true of return ... .

Each invocation of either foo1 or foo2 has a variadic argument call
and (essentially) a variadic argument return. On the other hand, this
is the entire functionality of foo1, while foo2 does more. So foo2
should be expected to be slower than foo1 no matter what n.

Cheers,
V.
Reply | Threaded
Open this post in threaded view
|

Re: Returning nil vs returning nothing

Roberto Ierusalimschy
> This can be seen without benchmarking. The problem with variadic
> arguments is that every function call with them is O(n), where n is
> the number of arguments. Just the call, ignoring whatever the function
> does. [...]

This is true for non-variadic functions as well, as Lua has to prepare
the n arguments to the function. I think most calls in most languages
follow that rule. (Inline functions that do not use its parameters would
be a counter-example.)

-- Roberto
Reply | Threaded
Open this post in threaded view
|

Re: Returning nil vs returning nothing

Viacheslav Usov
On Thu, Oct 29, 2020 at 2:53 PM Roberto Ierusalimschy
<[hidden email]> wrote:

> This is true for non-variadic functions as well, as Lua has to prepare
> the n arguments to the function.

Yes, my statement was trivial in its essence.

However, non-variadic functions are (usually, in most languages) not
called with a variable number of arguments, so one would not speak of
O(n) in that context.

Cheers,
V.
Reply | Threaded
Open this post in threaded view
|

Re: Returning nil vs returning nothing

Sam Putman
In reply to this post by Roberto Ierusalimschy


On Thu, Oct 29, 2020 at 3:53 AM Roberto Ierusalimschy <[hidden email]> wrote:
> This can be seen without benchmarking. The problem with variadic
> arguments is that every function call with them is O(n), where n is
> the number of arguments. Just the call, ignoring whatever the function
> does. [...]

This is true for non-variadic functions as well, as Lua has to prepare
the n arguments to the function. I think most calls in most languages
follow that rule. (Inline functions that do not use its parameters would
be a counter-example.)

-- Roberto

Is it in fact the case that variadic functions have additional overhead? That is,  calling `fn(a, b, c)` would be cheaper if this was defined as `function fn(a, b, c)`, rather than `function fn(...)` ?

I know there's more to this question than it appears, since one must *do* something with the arguments, and `{...}` would have different overhead from e.g. `select(1, ...)` and so on.

I had the impression that variadics carry some amount of overhead, but never benchmarked it.

For the sake of argument, we could compare:

function subber(...)
   return string.sub(...)
end

with:

function subber(str, start, finish)
    return string.sub(str, start, finish)
end

cheers,
-Sam
Reply | Threaded
Open this post in threaded view
|

Re: Returning nil vs returning nothing

Sean Conner
In reply to this post by Grzegorz Krasoń
It was thus said that the Great Grzegorz Krasoń once stated:

> Function that returns nothing seems different than function that returns
> `nil`:
>
> ```lua
> function f()
>     return nil
> end
>
> function g()
> end
>
> print(f()) -- prints nil
> print(g()) -- prints empty line
> ```
>
> How to recognize this at the function call? For example how to implement an
> universal wrapper that would suppress all errors and still returns exactly
> the same thing as wrapped function in case of success?

  I can't say I'm surprised that no one mentioned a C solution.  It's
actually trivial in C:

#include <lua.h>
#include <lauxlib.h>

static int supress(lua_State *L)
{
  lua_pcall(L,lua_gettop(L)-1,LUA_MULTRET,0);
  return lua_gettop(L);
}

int luaopen_supress(lua_State *L)
{
  lua_pushcfunction(L,supress);
  return 1;
}

And the script:

supress = require "supress"

function f()
    return nil
end

function g()
end

function h(...)
  return ...
end

print(f()) -- prints nil
print(g()) -- prints empty line
print(h(1,2,3,4,5)) -- prints parameters

print(supress(f)) -- prints nil
print(supress(g)) -- prints empty line
print(supress(h,1,2,3,4,5)) -- prints parameters

prints the following:

nil

1       2       3       4       5
nil

1       2       3       4       5

  I think the question I have is "why?"  Why do you need such functionality?
And what would you do when pcall() (or lua_pcall()) fails?

  -spc
Reply | Threaded
Open this post in threaded view
|

Re: Returning nil vs returning nothing

Roberto Ierusalimschy
In reply to this post by Sam Putman
> Is it in fact the case that variadic functions have additional overhead?
> That is,  calling `fn(a, b, c)` would be cheaper if this was defined as
> `function fn(a, b, c)`, rather than `function fn(...)` ?
>
> [...]
>
> I had the impression that variadics carry some amount of overhead, but
> never benchmarked it.
>
> For the sake of argument, we could compare:
>
> function subber(...)
>    return string.sub(...)
> end
>
> with:
>
> function subber(str, start, finish)
>     return string.sub(str, start, finish)
> end

Why don't you compare them?

-- Roberto
Reply | Threaded
Open this post in threaded view
|

Re: Returning nil vs returning nothing

Coda Highland
In reply to this post by Sam Putman
On Thu, Oct 29, 2020 at 3:48 PM Sam Putman <[hidden email]> wrote:
Is it in fact the case that variadic functions have additional overhead? That is,  calling `fn(a, b, c)` would be cheaper if this was defined as `function fn(a, b, c)`, rather than `function fn(...)` ?

You can reason about this if you think about it. Lua isn't a statically-typed language, and there's no mechanism for dispatching functions differently based on their parameter lists because the bytecode compiler doesn't even know what the function is at compile time. ALL functions are called by pushing the arguments onto the stack, so the invocation part of a variadic function call and a non-variadic function call should have the same performance.

On function startup, a non-variadic function actually has one point that could be more expensive than a variadic call: the non-variadic function will have to make sure that at least the minimum number of stack slots are filled, so it'll have to push nils if you didn't pass enough in. Of course a variadic function can have non-... arguments so it would have to do the same thing for those.

Lua function parameters are names assigned to stack slots. Since variadic parameters don't have names, you can only access their values by using a function (that is, select()). This is the primary source of overhead when working with variadic functions. If you're just passing them through, there's a small overhead involved in looping over the stack slots to copy them instead of copying a number of slots predetermined at compile time -- this is the O(n) thing that people have been talking about. That said, since there's no instruction dispatch necessary for this loop, it's a lot tighter than anything you could have implemented in Lua code itself, although it would still be more overhead than passing through a single table argument.

As Roberto said, you can benchmark it for yourself if you really want to know exactly how much of a difference there is.

/s/ Adam
Reply | Threaded
Open this post in threaded view
|

Re: Returning nil vs returning nothing

Sam Putman


On Sun, Nov 1, 2020 at 6:53 PM Coda Highland <[hidden email]> wrote:
On Thu, Oct 29, 2020 at 3:48 PM Sam Putman <[hidden email]> wrote:
Is it in fact the case that variadic functions have additional overhead? That is,  calling `fn(a, b, c)` would be cheaper if this was defined as `function fn(a, b, c)`, rather than `function fn(...)` ?

You can reason about this if you think about it. Lua isn't a statically-typed language, and there's no mechanism for dispatching functions differently based on their parameter lists because the bytecode compiler doesn't even know what the function is at compile time. ALL functions are called by pushing the arguments onto the stack, so the invocation part of a variadic function call and a non-variadic function call should have the same performance.

On function startup, a non-variadic function actually has one point that could be more expensive than a variadic call: the non-variadic function will have to make sure that at least the minimum number of stack slots are filled, so it'll have to push nils if you didn't pass enough in. Of course a variadic function can have non-... arguments so it would have to do the same thing for those.

Lua function parameters are names assigned to stack slots. Since variadic parameters don't have names, you can only access their values by using a function (that is, select()). This is the primary source of overhead when working with variadic functions. If you're just passing them through, there's a small overhead involved in looping over the stack slots to copy them instead of copying a number of slots predetermined at compile time -- this is the O(n) thing that people have been talking about. That said, since there's no instruction dispatch necessary for this loop, it's a lot tighter than anything you could have implemented in Lua code itself, although it would still be more overhead than passing through a single table argument.

As Roberto said, you can benchmark it for yourself if you really want to know exactly how much of a difference there is.

/s/ Adam

Thank you Adam. There is of course no substitute for profiling, I asked more to provoke conversation from those more knowledgeable about the internals than I.

"Variadic functions are slower than named parameters" was just a bit of lore I absorbed by osmosis, somewhere along the line. As it happens, I only have LuaJIT and Lua 5.2 running on my own system, and for the purposes of this list, 5.4 would be the useful benchmark, so I won't bother in this case. It's pretty clear that I can just use them when appropriate, and *maybe* revisit that if I ever spot one in a hot loop I'm optimizing.

Cheers,
-Sam