Strange way to determine whether it is Lua or C function

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
15 messages Options
Reply | Threaded
Open this post in threaded view
|

Strange way to determine whether it is Lua or C function

Egor Skriptunoff-2
Hi!

I have seen Lua 5.2+ code which uses the following approach
to distinguish C functions from Lua functions:

local function is_C_function(func)
  local x = coroutine.create(func)
  debug.sethook(x, function() x=debug.getlocal(2,2) coroutine.yield() end, "c")
  coroutine.resume(x)
  return not x
end

print(is_C_function(print))             -->  true
print(is_C_function(function() end))    -->  false

Why does this code work?

v
Reply | Threaded
Open this post in threaded view
|

Re: Strange way to determine whether it is Lua or C function

v
On Sun, 2020-08-23 at 09:44 +0300, Egor Skriptunoff wrote:
> local function is_C_function(func)
>   local x = coroutine.create(func)
>   debug.sethook(x, function() x=debug.getlocal(2,2) coroutine.yield()
> end, "c")
>   coroutine.resume(x)
>   return not x
> end
>
> Why does this code work?

I think it does the following:

1. Creates coroutine that will run specified function and sets hook on
"call" event.
2. Starts coroutine. This causes Lua to put called function's context
on stack and trigger hook.
3. Hook function writes the name of second local variable of second
function from top of the stack (one that triggered the hook and one
we're checking) to variable `x`. After this, it suspends execution,
returning control to `is_C_function`.
4. It returns if such variable doesn't exist.

The idea here is that C functions have at most one local and Lua
functions always have at least two. I'm not sure why this holds true,
but that's what script is assuming.
--
v <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Strange way to determine whether it is Lua or C function

Paul K-2
> The idea here is that C functions have at most one local and Lua
> functions always have at least two. I'm not sure why this holds true,
> but that's what script is assuming.

It looks like it checks the "line" parameter of the debug hook (it
will be the second function from the top), which is always `nil` for C
functions and not for Lua functions.

Paul.

On Sun, Aug 23, 2020 at 12:41 AM v <[hidden email]> wrote:

>
> On Sun, 2020-08-23 at 09:44 +0300, Egor Skriptunoff wrote:
> > local function is_C_function(func)
> >   local x = coroutine.create(func)
> >   debug.sethook(x, function() x=debug.getlocal(2,2) coroutine.yield()
> > end, "c")
> >   coroutine.resume(x)
> >   return not x
> > end
> >
> > Why does this code work?
>
> I think it does the following:
>
> 1. Creates coroutine that will run specified function and sets hook on
> "call" event.
> 2. Starts coroutine. This causes Lua to put called function's context
> on stack and trigger hook.
> 3. Hook function writes the name of second local variable of second
> function from top of the stack (one that triggered the hook and one
> we're checking) to variable `x`. After this, it suspends execution,
> returning control to `is_C_function`.
> 4. It returns if such variable doesn't exist.
>
> The idea here is that C functions have at most one local and Lua
> functions always have at least two. I'm not sure why this holds true,
> but that's what script is assuming.
> --
> v <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Strange way to determine whether it is Lua or C function

Viacheslav Usov
In reply to this post by Egor Skriptunoff-2
On Sun, Aug 23, 2020 at 8:45 AM Egor Skriptunoff
<[hidden email]> wrote:

> local function is_C_function(func)
>   local x = coroutine.create(func)
>   debug.sethook(x, function() x=debug.getlocal(2,2) coroutine.yield() end, "c")
>   coroutine.resume(x)
>   return not x
> end

The first argument to getlocal() selects the function that is the
subject of the test.

This is so because the "c" hook is invoked before the function is
called, yet the context (stack frame) of the function is already
established. So its stack frame is number 2.

Despite its name, getlocal() may return a register in the specified
stack frame. In a Lua function, there is always enough stack for two
registers, that is why getlocal(*, 2) will always return something for
a Lua function. This is assured by initializing the 'maxstacksize'
field of the function's "prototype" with 2. Why this is so, I do not
know.

On the other hand, for a C function, getlocal() for a non-zero level
only looks through the function's actual stack frame. It might seem
that it should be empty, because is_C_function() passes no arguments
to it, and the hook is called before the function itself has a chance
to execute; however, the invocation of a Lua hook function leaves an
artifact on the hooked function's stack frame, which is the hook
table. So getlocal(*, 1) would return something in any case, while
getlocal(>0, 2) returns nil for a C function called without arguments,
and which itself pushes nothing on the stack.

The latest condition is "naturally" ensured by the "c" hook.

The coroutine business ensures that the test subjects never actually run.

So the distinction is made by using an interplay of two implementation
details that are not intrinsically connected; moreover, at least for
one of them there is no really solid reason why it is so. Very
fragile.

Given that this technique already requires the debug library, I'd say
there is a far more regular and simpler way to achieve the goal.

Cheers,
V.
Reply | Threaded
Open this post in threaded view
|

Re: Strange way to determine whether it is Lua or C function

Egor Skriptunoff-2
In reply to this post by Paul K-2
On Sun, Aug 23, 2020 at 6:18 PM Paul K wrote:
It looks like it checks the "line" parameter of the debug hook (it
will be the second function from the top),

No.
The hook function is the first function from the top.
The second function from the top is the "func" (the function
passed as an argument for is_C_function).
It could be verified by inserting the following statement inside the hook
print("Stack level 2 is 'func':", debug.getinfo(2).func == func)

So, the code actually checks the parameters/registers of the "func" stack frame.
Reply | Threaded
Open this post in threaded view
|

Re: documentation bug: call hook [WAS: Strange way to determine whether it is Lua or C function]

Viacheslav Usov
In reply to this post by Viacheslav Usov
On Sun, Aug 23, 2020 at 5:26 PM Viacheslav Usov <[hidden email]> wrote:

> This is so because the "c" hook is invoked before the function is
> called, yet the context (stack frame) of the function is already
> established.

This is indeed so (in Lua 5.3 at least), contrarily to the
documentation that states: "The call hook: is called when the
interpreter calls a function. The hook is called just after Lua enters
the new function, before the function gets its arguments."

Example:

local debug = require 'debug'

debug.sethook(
    function()
        for i = 1, 9 do print(i, debug.getlocal(2, i)) end
    end,
    'c'
)

local function f(a, b, c) end

f('foo', 'bar', 'baz')

Output:

1       a       foo
2       b       bar
3       c       baz
4       (*temporary)    table: 0000016BA4367798
5       nil
6       nil
7       nil
8       nil
9       nil

So the arguments are definitely on stack and available for inspection.
It is actually good, because a call hook without access to arguments
would be much less useful. The meaning of "just after Lua enters the
new function" is subject to interpretation.

I suggest another description instead: "The call hook: is called when
the interpreter calls a function. The hook is called as if the new
function had just begun execution, with its stack frame (specifically,
arguments) observable from the hook."

Cheers,
V.
Reply | Threaded
Open this post in threaded view
|

documentation (?) bug: call hook yields [WAS: Re: Strange way to determine whether it is Lua or C function]

Viacheslav Usov
In reply to this post by Viacheslav Usov
On Sun, Aug 23, 2020 at 5:26 PM Viacheslav Usov <[hidden email]> wrote:

> The coroutine business ensures that the test subjects never actually run.

... while according to the documentation, it should not. Here is the
relevant line of code:

debug.sethook(x, function() x=debug.getlocal(2,2) coroutine.yield() end, "c")

This sets a call hook, which yields. Relevant documentation: "Hook
functions can yield under the following conditions: Only count and
line events can yield..." It is not stated what should happen if the
condition is not met, but in my tests (Lua 5.3) it seems most
certainly that the main thread resumes execution, while the hooked
function, either C or Lua, does not execute any longer.

So either the documentation or the behavior is wrong.

Cheers,
V.
Reply | Threaded
Open this post in threaded view
|

Re: Strange way to determine whether it is Lua or C function

Egor Skriptunoff-2
In reply to this post by Viacheslav Usov
On Sun, Aug 23, 2020 at 6:27 PM Viacheslav Usov wrote:
In a Lua function, there is always enough stack for two
registers
...
the invocation of a Lua hook function leaves an
artifact on the hooked function's stack frame

Thanks for the explanation.

 
Given that this technique already requires the debug library, I'd say
there is a far more regular and simpler way to achieve the goal.

The approach discussed here was found inside an obfuscated code.
Simplicity was not the goal, just the opposite.
Reply | Threaded
Open this post in threaded view
|

Re: documentation (?) bug: call hook yields [WAS: Re: Strange way to determine whether it is Lua or C function]

Egor Skriptunoff-2
In reply to this post by Viacheslav Usov
On Sun, Aug 23, 2020 at 8:45 PM Viacheslav Usov wrote:
debug.sethook(x, function() x=debug.getlocal(2,2) coroutine.yield() end, "c")

This sets a call hook, which yields. Relevant documentation: "Hook
functions can yield under the following conditions: Only count and
line events can yield..."

The hook does not successfully yield in this example.
"coroutine.yield()" raises an exception "attempt to yield across a C-call boundary".
The coroutine dies, and the main thread resumes execution.

 
So either the documentation or the behavior is wrong.

Both are correct: the call hook can not yield.
Reply | Threaded
Open this post in threaded view
|

Re: Strange way to determine whether it is Lua or C function

Paul K-2
In reply to this post by Egor Skriptunoff-2
Hi Egor,

> So, the code actually checks the parameters/registers of the "func" stack frame.

Correct; my guess was wrong. Thank you for the explanation.

Paul.

On Sun, Aug 23, 2020 at 9:20 AM Egor Skriptunoff
<[hidden email]> wrote:

>
> On Sun, Aug 23, 2020 at 6:18 PM Paul K wrote:
>>
>> It looks like it checks the "line" parameter of the debug hook (it
>> will be the second function from the top),
>
>
> No.
> The hook function is the first function from the top.
> The second function from the top is the "func" (the function
> passed as an argument for is_C_function).
> It could be verified by inserting the following statement inside the hook
> print("Stack level 2 is 'func':", debug.getinfo(2).func == func)
>
> So, the code actually checks the parameters/registers of the "func" stack frame.
Reply | Threaded
Open this post in threaded view
|

Re: documentation (?) bug: call hook yields [WAS: Re: Strange way to determine whether it is Lua or C function]

Viacheslav Usov
In reply to this post by Egor Skriptunoff-2
On Sun, Aug 23, 2020 at 10:23 PM Egor Skriptunoff
<[hidden email]> wrote:

> "coroutine.yield()" raises an exception "attempt to yield across a C-call boundary".
> The coroutine dies, and the main thread resumes execution.

Indeed, I overlooked that. So, no bug here. The only - small - thing
is that the message is the same when either C or Lua-based coroutine
is called, which looks weird in the latter case.

Cheers,
V.
Reply | Threaded
Open this post in threaded view
|

Re: Strange way to determine whether it is Lua or C function

Viacheslav Usov
In reply to this post by Viacheslav Usov
On Sun, Aug 23, 2020 at 5:26 PM Viacheslav Usov <[hidden email]> wrote:

> The coroutine business ensures that the test subjects never actually run.

As established in an erroneous spin-off bug thread, the yield results
in an error rather than a normal yield. Which suggests that coroutines
are not in fact a necessary means toward this end. Indeed, the
following code seems to achieve the same effect without coroutines:

local debug = require 'debug'

local function is_C_function(func)
  local x
  debug.sethook(
    function()
      if debug.getinfo(2, 'f').func == func then
        x = debug.getlocal(2, 2)
        error(0)
      end
    end,
    'c'
  )
  pcall(func)
  return not x
end
)

Cheers,
V.
Reply | Threaded
Open this post in threaded view
|

Re: Strange way to determine whether it is Lua or C function

Egor Skriptunoff-2
On Mon, Aug 24, 2020 at 10:50 AM Viacheslav Usov wrote:
the following code seems to achieve the same effect without coroutines

You forgot to unhook (to save and restore the original hook).
So, using a temporary coroutine does make sense here.
BTW, "error(0)" could be replaced with just "x()" :-)
 
Reply | Threaded
Open this post in threaded view
|

Re: Strange way to determine whether it is Lua or C function

Viacheslav Usov
On Mon, Aug 24, 2020 at 5:02 PM Egor Skriptunoff
<[hidden email]> wrote:

> So, using a temporary coroutine does make sense here.

I agree that the use of coroutines seems more elegant, and, fittingly,
it obscures what is going on. My intent was to show that it was not
necessary, which I think I did, albeit imperfectly.

Cheers,
V.
Reply | Threaded
Open this post in threaded view
|

Re: Strange way to determine whether it is Lua or C function

Egor Skriptunoff-2
On Mon, Aug 24, 2020 at 6:26 PM Viacheslav Usov wrote:
My intent was to show that [the use of coroutines] was not
necessary.

Yes, your observation is important.
The magic is in a call hook, not in a coroutine.