Symbol not found error using embedded Lua interpreter in Rust

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Symbol not found error using embedded Lua interpreter in Rust

Deepak Jois
I am trying to use [rust-lua53](https://github.com/jcmoyer/rust-lua53)
to execute Lua code. _rust-lua53_ already bundles the source code for
Lua 5.3.3, and provides some bindings on top of it.

It seems to run fine with regular Lua code, but when I try to load a
Lua C module, it gives a "Symbol not found" error.

The Lua code I am trying to execute is:

```
   print('hello')

    if not pcall(function() require('serpent') end) then
      print('error loading serpent')
    else
        spt = require('serpent')
        print('serpent successfully loaded')
    end

    local status, err = pcall(function() require('lfs') end)
    print(err)
```

which ends up giving the following output (on OS X, but I get
something similar on Linux):

```

hello
serpent successfully loaded
error loading module 'lfs' from file
'/Users/deepak/Downloads/lua53/lib/lua/5.3/lfs.so':
        dlopen(/Users/deepak/Downloads/lua53/lib/lua/5.3/lfs.so, 6):
Symbol not found: _lua_settable
  Referenced from: /Users/deepak/Downloads/lua53/lib/lua/5.3/lfs.so
  Expected in: flat namespace
 in /Users/deepak/Downloads/lua53/lib/lua/5.3/lfs.so
```

Could somebody help me understand where I can start troubleshooting
this? What does the Lua interpreter look for when trying to load a Lua
C module? Is there a specific way to link the Lua code inside of Rust
so that the symbols are available to the C module?

The full error report is here: https://github.com/jcmoyer/rust-lua53/issues/85

Thanks
Deepak

Reply | Threaded
Open this post in threaded view
|

Re: Symbol not found error using embedded Lua interpreter in Rust

steve donovan
On Wed, Jun 21, 2017 at 9:56 AM, Deepak Jois <[hidden email]> wrote:
> I am trying to use [rust-lua53](https://github.com/jcmoyer/rust-lua53)
> to execute Lua code. _rust-lua53_ already bundles the source code for
> Lua 5.3.3, and provides some bindings on top of it.

What is your lfs.so linked against?

If rust-lua53 links in Lua statically, then it might not re-export the
Lua symbols for the dynamic linking of lfs.

Another approach is to link lfs in the Lua build statically - it's
pretty straightforward.

Reply | Threaded
Open this post in threaded view
|

Re: Symbol not found error using embedded Lua interpreter in Rust

Deepak Jois
On Wed, Jun 21, 2017 at 1:34 PM, steve donovan
<[hidden email]> wrote:
>
> What is your lfs.so linked against?
>

I am not sure I fully understand the question. But on Linux, I
installed it using the LuaRocks utility. ldd reports this:

```
$ ldd  /home/deepak/lua53/lib/lua/5.3/lfs.so
        linux-vdso.so.1 =>  (0x00007fffe6cff000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f0cedf40000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f0cee600000)
```

> If rust-lua53 links in Lua statically, then it might not re-export the
> Lua symbols for the dynamic linking of lfs.
>

Yes, rust-lua53 links to Lua statically. I am not sure how to find out
if it re-exports the symbols.

However, the 'nm' utility on Linux reports this on the  executable
that I am using to invoke the Lua code:

```
$ nm -g target/debug/rlua | grep lua | wc -l
298

$ nm -g target/debug/rlua | grep lua_gettop
000000000000dd63 T lua_gettop
```

On Linux, the error is "undefined symbol: lua_gettop", but as you can
see the grep output seems to clearly show it as being available. So I
am not sure where the problem is exactly.

> Another approach is to link lfs in the Lua build statically - it's
> pretty straightforward.
>

Yes, that may be straightforward. But I am looking to build a Lua
interpreter inside Rust that has equivalent to functionality to the
standard Lua interpreter, including the ability to load arbirtary Lua
C modules. I later plan to extend it with other stuff.

Deepak

Reply | Threaded
Open this post in threaded view
|

Re: Symbol not found error using embedded Lua interpreter in Rust

Michal Kottman
On Jun 21, 2017 10:20 AM, "Deepak Jois" <[hidden email]> wrote:
$ nm -g target/debug/rlua | grep lua_gettop
000000000000dd63 T lua_gettop
```

On Linux, the error is "undefined symbol: lua_gettop", but as you can
see the grep output seems to clearly show it as being available. So I
am not sure where the problem is exactly.

A long time ago I had a similar issue with Python (was it with Lunatic Python?) and the issue was that the default setting in Python was RTLD_LOCAL (see 'man dlopen'). Fortunately Python allows you to change the defaults [1]. Maybe you need to do something similar for Rust.

Reply | Threaded
Open this post in threaded view
|

Re: Symbol not found error using embedded Lua interpreter in Rust

Deepak Jois
In reply to this post by Deepak Jois
Thanks for the suggestions.

On Wed, Jun 21, 2017 at 1:26 PM, Deepak Jois <[hidden email]> wrote:
> Is there a specific way to link the Lua code inside of Rust
> so that the symbols are available to the C module?

FWIW, I managed to solve this by linking the Rust program with the
right flags. I had to pass a flag -rdynamic.

I just found the answer by doing a lot of trial and error, and
research. I can’t claim I understand all the mysteries of linking. I
wonder if this sort of info is already there on some Lua wiki. If not,
I would add it to an appropriate place if somebody can point it out to
me.

Some helpful links for anybody who might end up in the archives
searching for the exact same problem:

* https://stackoverflow.com/questions/21279036/what-is-clangs-equivalent-to-rdynamic-gcc-flag

* http://keitaito.com/blog/2017/01/02/exploring-compiler-options.html

Reply | Threaded
Open this post in threaded view
|

Yet another user confused by the *program* exporting symbols (was Re: Symbol not found error using embedded Lua interpreter in Rust)

Jay Carlson
Note that the below is not relevant to any systems more embedded than a router; they won't care because they won't use ELF SVR4-style shared libraries.

IMO, the most compelling argument for the odd way upstream prefers to build modules is that the main program does not have to be position-independent. On some architectures, there is a large penalty for PIC, especially on i386 implementations. (For starters, you lose a very scarce register.) Using /usr/bin/lua as liblua.so as well helped get Lua to the top of the programming language shootout.

We're just about at the end of the i386 architecture. All the other platforms which could use shared libraries are not so register-starved. In the "PC" space, almost everything is amd64, and amd64 is much better for position-independent code. 

In addition, the preferred way to build liblua.so these days is to only export the public Lua API. This allows the linker to optimize internal references to private symbols. 

I would run benchmarks for you in my museum, but I would need some benchmarks to run. Any pointers or volunteers?

Jay

(Overhead could be reduced more if external symbols did not affect how liblua.so refers to the liblua.so symbols. This breaks LD_PRELOAD among other things, but it's about morally equivalent to exporting symbols from the main program.)


On Thu, Jun 22, 2017, 00:32 Deepak Jois <[hidden email]> wrote:
Thanks for the suggestions.

On Wed, Jun 21, 2017 at 1:26 PM, Deepak Jois <[hidden email]> wrote:
> Is there a specific way to link the Lua code inside of Rust
> so that the symbols are available to the C module?

FWIW, I managed to solve this by linking the Rust program with the
right flags. I had to pass a flag -rdynamic.

I just found the answer by doing a lot of trial and error, and
research. I can’t claim I understand all the mysteries of linking. I
wonder if this sort of info is already there on some Lua wiki. If not,
I would add it to an appropriate place if somebody can point it out to
me.

Some helpful links for anybody who might end up in the archives
searching for the exact same problem:

* https://stackoverflow.com/questions/21279036/what-is-clangs-equivalent-to-rdynamic-gcc-flag

* http://keitaito.com/blog/2017/01/02/exploring-compiler-options.html