Looking for some help - add fenv and other 5.1 features to ravi-distro

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Looking for some help - add fenv and other 5.1 features to ravi-distro

Dibyendu Majumdar
Hi,

I was wondering if anyone would be able to help me add some Lua 5.1
compatibility features to Lua 5.3 in my Ravi-Distro package.

I include Lua 5.3.5 in the distro with these enhancements:

* The LuaJIT bit library is included
* All available 5.1 and 5.2 compatibility options are enabled,
including LUA_COMPAT_FLOATSTRING .

These compatibility options enable Lua 5.3 to run LuaJIT's dynasm tool
for instance.

However I would like to improve backwards compatibility with Lua 5.1
even further; in particular would like to add support for fenv(). But
I have never used Lua 5.1, and despite good intentions, haven't been
able to work on this aspect. I would very much appreciate any help.

Thanks and regards
Dibyendu

Reply | Threaded
Open this post in threaded view
|

Re: Looking for some help - add fenv and other 5.1 features to ravi-distro

Soni L.
it's rather impractical: there are huge semantic differences between how they work.

e.g.

local _ENV = ...
function foo()
_ENV = ...
end
function bar()
print(foo)
end

This is impossible in Lua 5.1.

On Tue, Nov 27, 2018, 19:56 Dibyendu Majumdar <[hidden email] wrote:
Hi,

I was wondering if anyone would be able to help me add some Lua 5.1
compatibility features to Lua 5.3 in my Ravi-Distro package.

I include Lua 5.3.5 in the distro with these enhancements:

* The LuaJIT bit library is included
* All available 5.1 and 5.2 compatibility options are enabled,
including LUA_COMPAT_FLOATSTRING .

These compatibility options enable Lua 5.3 to run LuaJIT's dynasm tool
for instance.

However I would like to improve backwards compatibility with Lua 5.1
even further; in particular would like to add support for fenv(). But
I have never used Lua 5.1, and despite good intentions, haven't been
able to work on this aspect. I would very much appreciate any help.

Thanks and regards
Dibyendu

Reply | Threaded
Open this post in threaded view
|

Re: Looking for some help - add fenv and other 5.1 features to ravi-distro

Sean Conner
In reply to this post by Dibyendu Majumdar
It was thus said that the Great Dibyendu Majumdar once stated:

> Hi,
>
> I was wondering if anyone would be able to help me add some Lua 5.1
> compatibility features to Lua 5.3 in my Ravi-Distro package.
>
> I include Lua 5.3.5 in the distro with these enhancements:
>
> * The LuaJIT bit library is included
> * All available 5.1 and 5.2 compatibility options are enabled,
> including LUA_COMPAT_FLOATSTRING .
>
> These compatibility options enable Lua 5.3 to run LuaJIT's dynasm tool
> for instance.
>
> However I would like to improve backwards compatibility with Lua 5.1
> even further; in particular would like to add support for fenv(). But
> I have never used Lua 5.1, and despite good intentions, haven't been
> able to work on this aspect. I would very much appreciate any help.

  It may be possible.  setfenv() is used to set an environment table to
either a function or userdata.  If it's a function, this table become the
global environment for the function; for a userdata, it's similar to
setuservalue().  Given that Lua 5.3, section 3.3.2 states:

        As such, chunks can define local variables, receive arguments, and
        return values. Moreover, such anonymous function is compiled as in
        the scope of an external local variable called _ENV (see §2.2). The
        resulting function always has _ENV as its only upvalue, even if it
        does not use that variable.

  So any code loaded via load() (or similar) will hvae _ENV as the first
upvalue, you can use setupvalue() to switch out the _ENV value currently
set.  Arbitrary functions might be a bit harder to locate the proper _ENV to
swap (if not outright impossible in some cases).

  setfenv() at least, the Lua function, not the C function, can also modify
a function in the call stack.  Per the description:

        Sets the environment to be used by the given function. f can be a
        Lua function or a number that specifies the function at that stack
        level: Level 1 is the function calling setfenv. setfenv returns the
        given function.

        As a special case, when f is 0 setfenv changes the environment of
        the running thread. In this case, setfenv returns no values.

  -spc (As always, the devil is in the details ... )

Reply | Threaded
Open this post in threaded view
|

Re: Looking for some help - add fenv and other 5.1 features to ravi-distro

Muh Muhten
On 11/27/18, Sean Conner <[hidden email]> wrote:
> (if not outright impossible in some cases).

In some cases, it is outright impossible, in the sense that deciding
which upvalue was _ENV is not a function of the code.

Consider the examples I give for
<https://gist.github.com/f469d25d375e4e742b74c7a938c58ac8#limitations>:
the only way to tell the difference is to actually check the upvalues,
and see that one of them is the same as the global table. Of course,
this is not guaranteed either if _ENV is redefined.

What I believe you *could* do is, on finding an upvalue which is the
global table, debug.upvaluejoin it to the first upvalue of a chunk (or
more directly, a throwaway C closure) constructed with the new
environment as its only upvalue. I have not tried it. It probably
breaks expected semantics in all sort of subtle ways.

It's probably for the best to accept that fenv semantics are not
implementable in terms of _ENV. This is kind of expectable, since part
of the rationale for the change was (paraphrasing) that the setfenv
mechanism was too powerful and more suited to the debug library.

Incidentally, the _ENV mechanism is essentially syntactic sugar and
afaict can be mechanically translated to just introducing a local
named _ENV at the beginning of a chunk and rewriting all would-be
global accesses to index _ENV. There's one tiny difference (it's now a
local, not an upvalue, of the main function) but that seems like a
corner case that will only come up if you use the debug library or
otherwise muck with internals. As such, it might be easier to retarget
the 5.3 compiler to the 5.1 VM(!) if you *need* this functionality.
Certainly 5.2.

Reply | Threaded
Open this post in threaded view
|

Re: Looking for some help - add fenv and other 5.1 features to ravi-distro

Tim Hill
In reply to this post by Sean Conner


On Nov 27, 2018, at 3:04 PM, Sean Conner <[hidden email]> wrote:

 So any code loaded via load() (or similar) will hvae _ENV as the first
upvalue, you can use setupvalue() to switch out the _ENV value currently
set.  Arbitrary functions might be a bit harder to locate the proper _ENV to
swap (if not outright impossible in some cases).

Sadly that’s not true. Any chunk that is COMPILED will honor this, but code that is obtained via string.dump() followed by load() often will not. Such a dumped function may or may not have _ENV as one of its upvalues, and even if it does, it may not be the first.

—Tim

Reply | Threaded
Open this post in threaded view
|

Re: Looking for some help - add fenv and other 5.1 features to ravi-distro

Sean Conner
It was thus said that the Great Tim Hill once stated:

>
>
> > On Nov 27, 2018, at 3:04 PM, Sean Conner <[hidden email]> wrote:
> >
> >  So any code loaded via load() (or similar) will hvae _ENV as the first
> > upvalue, you can use setupvalue() to switch out the _ENV value currently
> > set.  Arbitrary functions might be a bit harder to locate the proper _ENV to
> > swap (if not outright impossible in some cases).
>
> Sadly that’s not true. Any chunk that is COMPILED will honor this, but
> code that is obtained via string.dump() followed by load() often will not.
> Such a dumped function may or may not have _ENV as one of its upvalues,
> and even if it does, it may not be the first.

  I think there's a bug in the description for string.dump().  It states,
"[w]hen (re)loaded, those upvalues receive fresh instances containing nil."
But load() states "When you load a main chunk, the resulting function will
always have exactly one upvalue, the _ENV variable (see §2.2). However, when
you load a binary chunk created from a function (see string.dump), the
resulting function can have an arbitrary number of upvalues."

  Running this:

        local function upvals(name,f)
          local function upv(i)
            local n,v = debug.getupvalue(f,i)
            if v then
              print("",i,n,v)
              return upv(i+1)
            end
          end
         
          print(name)
          upv(1)
        end

        local a = 0
        local function foo(x)
          a = a + 1
          print(x)
          print(os.getenv(x))
          return a
        end

        upvals("foo",foo)

        local dumped = string.dump(foo)
        local reloaded = load(dumped)

        upvals("reloaded",reloaded)

  I get:

foo
        1       a       0
        2       _ENV    table: 0x89b5568
reloaded
        1       a       table: 0x89b5568

  So the first upvalue will have _ENV (in this case, the global _ENV), but
not the rest (also, the first upvalue for reloaded() should be a number, not
a table).

  -spc (Which presents yet another problem with simulating setfenv() in Lua
        5.2 or higher ... hmmm ... perhaps not a good idea after all)

Reply | Threaded
Open this post in threaded view
|

Re: Looking for some help - add fenv and other 5.1 features to ravi-distro

Muh Muhten
On 11/27/18, Sean Conner <[hidden email]> wrote:
>   I think there's a bug in the description for string.dump().  It states,
> "[w]hen (re)loaded, those upvalues receive fresh instances containing nil."
> But load() states "When you load a main chunk, the resulting function will
> always have exactly one upvalue, the _ENV variable (see §2.2). However,
> when
> you load a binary chunk created from a function (see string.dump), the
> resulting function can have an arbitrary number of upvalues."

Sure. When (re)loaded, the dumped function has fresh upvalues that
aren't the same as those or any other function, which are initially
nil. Then load() (and lua_load()) set the first upvalue.

... hm, perhaps load setting the first upvalue of functions that
aren't main chunks (linenumber != 0) just doesn't make much sense in
the first place? Or if the incompatibility there seems gratuitous
(maybe it can go into 5.4?), indeed, mention the first upvalue in the
description of string.dump.

Initially I expected lua_load to not set the first upvalue, but that
doesn't seem to be the case.