require("some_dll") with a statically linked Lua 5.2.4

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

require("some_dll") with a statically linked Lua 5.2.4

ThePhD
I've run into a peculiar case while compiling and running my test suite for Lua and sol2 using Lua 5.2.4. In my tests, I build a lua_CFunction-containing DLL from source called "my_object". I also build Lua from source in various versions (everything from LuaJIT to Lua 5.3.4).

When I do require("my_object"), it works just fine on all platforms.... EXCEPT on Windows, when I compile Lua 5.2.4 as a static Library, and link that directly into the "my_object" and consuming executable. Code using require("my_object") fails with a runtime error, and the pushed error is a lightuserdata I cannot parse so I can't get any descriptive error out of Lua.

Compiling Lua 5.2.4 as a DLL and linking against the Lua DLL works just fine for the code.

Compiling Lua 5.3/5.1 statically or dynamically and linking against that for the DLL is also fine.

I don't know if anyone else has a Windows computer, but maybe I've just done something wrong? The example code is down here: https://github.com/ThePhD/sol2/tree/develop/examples/require_dll_example

my_object.cpp is built into a dynamic lib, require_from_dll.cpp is built into an executable, both link to Lua of whatever flavor I'm running the tests on.


Reply | Threaded
Open this post in threaded view
|

Re: require("some_dll") with a statically linked Lua 5.2.4

Russell Haley
Sorry for the top post. Are you building with Visual Studio? If so, it sounds like one of your build configurations might be a little different than the others?  From the configuration manager you can select manage configurations and in the drop down there should be an option for all configurations.  I'd double check the project isn't building a DLL by accident. 

My VS experience so far: I've had to set all configurations and go through each properties screen of each project to ensure they all align. WinLua currently only supports 5.1 and 5.3 so I've never tested against 5.2, nor have I tested my static libraries properly. WinLua builds a static and a dynamic library in separate projects in the one solution.

For What It's Worth,

Russ
Sent from my BlackBerry 10 smartphone on the Virgin Mobile network.
From: ThePhD
Sent: Sunday, February 11, 2018 6:24 AM
To: Lua mailing list
Reply To: Lua mailing list
Subject: require("some_dll") with a statically linked Lua 5.2.4

I've run into a peculiar case while compiling and running my test suite for Lua and sol2 using Lua 5.2.4. In my tests, I build a lua_CFunction-containing DLL from source called "my_object". I also build Lua from source in various versions (everything from LuaJIT to Lua 5.3.4).

When I do require("my_object"), it works just fine on all platforms.... EXCEPT on Windows, when I compile Lua 5.2.4 as a static Library, and link that directly into the "my_object" and consuming executable. Code using require("my_object") fails with a runtime error, and the pushed error is a lightuserdata I cannot parse so I can't get any descriptive error out of Lua.

Compiling Lua 5.2.4 as a DLL and linking against the Lua DLL works just fine for the code.

Compiling Lua 5.3/5.1 statically or dynamically and linking against that for the DLL is also fine.

I don't know if anyone else has a Windows computer, but maybe I've just done something wrong? The example code is down here: https://github.com/ThePhD/sol2/tree/develop/examples/require_dll_example

my_object.cpp is built into a dynamic lib, require_from_dll.cpp is built into an executable, both link to Lua of whatever flavor I'm running the tests on.



Reply | Threaded
Open this post in threaded view
|

Re: require("some_dll") with a statically linked Lua 5.2.4

ThePhD
The problem does only manifest with Visual Studio 2017. When I build using MinGW's GCC 7.1.0 (or 7.2.0 or 6.3.0 or any others) the code works just fine. I suspect this might have something to do with Lua 5.2.4's config and how it loads a DLL. I will probably have to build Lua 5.2.4 in Debug mode and then through the code as it is being required to see what breaks and get back to everyone here.

On Sun, Feb 11, 2018 at 11:47 AM, Russell Haley <[hidden email]> wrote:
Sorry for the top post. Are you building with Visual Studio? If so, it sounds like one of your build configurations might be a little different than the others?  From the configuration manager you can select manage configurations and in the drop down there should be an option for all configurations.  I'd double check the project isn't building a DLL by accident. 

My VS experience so far: I've had to set all configurations and go through each properties screen of each project to ensure they all align. WinLua currently only supports 5.1 and 5.3 so I've never tested against 5.2, nor have I tested my static libraries properly. WinLua builds a static and a dynamic library in separate projects in the one solution.

For What It's Worth,

Russ
Sent from my BlackBerry 10 smartphone on the Virgin Mobile network.
From: ThePhD
Sent: Sunday, February 11, 2018 6:24 AM
To: Lua mailing list
Reply To: Lua mailing list
Subject: require("some_dll") with a statically linked Lua 5.2.4

I've run into a peculiar case while compiling and running my test suite for Lua and sol2 using Lua 5.2.4. In my tests, I build a lua_CFunction-containing DLL from source called "my_object". I also build Lua from source in various versions (everything from LuaJIT to Lua 5.3.4).

When I do require("my_object"), it works just fine on all platforms.... EXCEPT on Windows, when I compile Lua 5.2.4 as a static Library, and link that directly into the "my_object" and consuming executable. Code using require("my_object") fails with a runtime error, and the pushed error is a lightuserdata I cannot parse so I can't get any descriptive error out of Lua.

Compiling Lua 5.2.4 as a DLL and linking against the Lua DLL works just fine for the code.

Compiling Lua 5.3/5.1 statically or dynamically and linking against that for the DLL is also fine.

I don't know if anyone else has a Windows computer, but maybe I've just done something wrong? The example code is down here: https://github.com/ThePhD/sol2/tree/develop/examples/require_dll_example

my_object.cpp is built into a dynamic lib, require_from_dll.cpp is built into an executable, both link to Lua of whatever flavor I'm running the tests on.




Reply | Threaded
Open this post in threaded view
|

Re: require("some_dll") with a statically linked Lua 5.2.4

ThePhD
Found it.


The problem:
It's similar to the same problem I had on GCC/Linux and MinGW builds, except that compiler presented it to me as a linker error rather than a runtime failure that I couldn't quite understand.

The error is tossed from luaL_checkversion (luaL_checkversion_ is what it points to in lauxlib.c, 5.2.4 source tree):

LUALIB_API void luaL_checkversion_ (lua_State *L, lua_Number ver) {
  const lua_Number *v = lua_version(L); // uh oh...
  if (v != lua_version(NULL)) // --------------- HERE!!!
    luaL_error(L, "multiple Lua VMs detected");


Linking the same static library in a Windows Executable and a Windows DLL means that the static library has the same code dumped out twice. This means there are 2 versions of the code, and the check being done here places a number inside of its final compiled target to uniquely identify the Lua VM to the binary it was compiled into (whether that was from being statically linked or if it was baked into a DLL).

I'm not sure what Lua 5.1 didn't have or Lua 5.3 changed/introduced to make it work, but this is the gist of the problem.


Explanation and Solutions:
On GCC/Linux and MinGW, trying to link 2 libraries like this ultimately produces a failure at link-time, stating that I have multiply-defined symbols attempting to be packed into my executable. There's also a separate problem of when you only try to link one on later versions of GCC, where it will perform an optimization where it strips out unused symbols. So, if I were to link the static lib into the DLL only and then hope that all of Lua's functions were available when linking the DLL into the Executable, it would complain about undefined references to functions it couldn't find (because the Linker optimizes for space by default and cuts unused functions).

On GCC/Linux, the solution is to link the library only once in the DLL, and surround it with the flags -Wl,-whole-archive {lua library name} -Wl,-no-whole-archive.

On Windows, they just privatize the static library into each executable / dynamic library's virtual memory space and load them separately. Supposedly, I have the ability to do much the same as GCC with the flag /WHOLEARCHIVE[:libname], where in this case I would put the static library name of Lua there.

Hopefully, this helps somebody else who comes across this.




On Mon, Feb 12, 2018 at 9:37 PM, ThePhD <[hidden email]> wrote:
The problem does only manifest with Visual Studio 2017. When I build using MinGW's GCC 7.1.0 (or 7.2.0 or 6.3.0 or any others) the code works just fine. I suspect this might have something to do with Lua 5.2.4's config and how it loads a DLL. I will probably have to build Lua 5.2.4 in Debug mode and then through the code as it is being required to see what breaks and get back to everyone here.

On Sun, Feb 11, 2018 at 11:47 AM, Russell Haley <[hidden email]> wrote:
Sorry for the top post. Are you building with Visual Studio? If so, it sounds like one of your build configurations might be a little different than the others?  From the configuration manager you can select manage configurations and in the drop down there should be an option for all configurations.  I'd double check the project isn't building a DLL by accident. 

My VS experience so far: I've had to set all configurations and go through each properties screen of each project to ensure they all align. WinLua currently only supports 5.1 and 5.3 so I've never tested against 5.2, nor have I tested my static libraries properly. WinLua builds a static and a dynamic library in separate projects in the one solution.

For What It's Worth,

Russ
Sent from my BlackBerry 10 smartphone on the Virgin Mobile network.
From: ThePhD
Sent: Sunday, February 11, 2018 6:24 AM
To: Lua mailing list
Reply To: Lua mailing list
Subject: require("some_dll") with a statically linked Lua 5.2.4

I've run into a peculiar case while compiling and running my test suite for Lua and sol2 using Lua 5.2.4. In my tests, I build a lua_CFunction-containing DLL from source called "my_object". I also build Lua from source in various versions (everything from LuaJIT to Lua 5.3.4).

When I do require("my_object"), it works just fine on all platforms.... EXCEPT on Windows, when I compile Lua 5.2.4 as a static Library, and link that directly into the "my_object" and consuming executable. Code using require("my_object") fails with a runtime error, and the pushed error is a lightuserdata I cannot parse so I can't get any descriptive error out of Lua.

Compiling Lua 5.2.4 as a DLL and linking against the Lua DLL works just fine for the code.

Compiling Lua 5.3/5.1 statically or dynamically and linking against that for the DLL is also fine.

I don't know if anyone else has a Windows computer, but maybe I've just done something wrong? The example code is down here: https://github.com/ThePhD/sol2/tree/develop/examples/require_dll_example

my_object.cpp is built into a dynamic lib, require_from_dll.cpp is built into an executable, both link to Lua of whatever flavor I'm running the tests on.