How to use a c module with embedded lua on windows

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

How to use a c module with embedded lua on windows

Marcus Mason
Hi, I'm trying to build lpeg to use with lua embedded in a windows application. Everytime I try to load my dll into the application, I get "multiple vm versions detected". I checked and the host application has linked statically to lua 5.3.1 (I'm also linking statically against lua 5.3.1). Any wisdom on building c libraries for this kind of setup?
Reply | Threaded
Open this post in threaded view
|

Re: How to use a c module with embedded lua on windows

Andrew Gierth
>>>>> "Marcus" == Marcus Mason <[hidden email]> writes:

 Marcus> Hi, I'm trying to build lpeg to use with lua embedded in a
 Marcus> windows application. Everytime I try to load my dll into the
 Marcus> application, I get "multiple vm versions detected". I checked
 Marcus> and the host application has linked statically to lua 5.3.1
 Marcus> (I'm also linking statically against lua 5.3.1). Any wisdom on
 Marcus> building c libraries for this kind of setup?

The host application has to export its Lua API symbols and the dll has
to link against that export library, it can't use its own embedded copy
of the code.

--
Andrew.

Reply | Threaded
Open this post in threaded view
|

Re: How to use a c module with embedded lua on windows

Marcus Mason
Thanks for the reply. The host app does not export its lua API symbols. I had seen some kind of proxy dll on the lua wiki as a workaround but I'm not sure it's updated and/or relevent to my case?

On Sat, Jun 22, 2019 at 10:34 PM Andrew Gierth <[hidden email]> wrote:
>>>>> "Marcus" == Marcus Mason <[hidden email]> writes:

 Marcus> Hi, I'm trying to build lpeg to use with lua embedded in a
 Marcus> windows application. Everytime I try to load my dll into the
 Marcus> application, I get "multiple vm versions detected". I checked
 Marcus> and the host application has linked statically to lua 5.3.1
 Marcus> (I'm also linking statically against lua 5.3.1). Any wisdom on
 Marcus> building c libraries for this kind of setup?

The host application has to export its Lua API symbols and the dll has
to link against that export library, it can't use its own embedded copy
of the code.

--
Andrew.

Reply | Threaded
Open this post in threaded view
|

Re: How to use a c module with embedded lua on windows

Francisco Olarte
Marcus:

On Sun, Jun 23, 2019 at 3:06 AM Marcus Mason <[hidden email]> wrote:
> Thanks for the reply. The host app does not export its lua API symbols. I had seen some kind of proxy dll on the lua wiki as a workaround but I'm not sure it's updated and/or relevent to my case?

Are you sure it does not? The fact that something DETECTS multiple VM
version hints at two copies being exported, and from what you told it
seem host links statically and export and you link statically and
export, hence the collision. I'm not familiar with windows, but there
are surely some tools for listing exported symbols for it.

Francisco Olarte.

Reply | Threaded
Open this post in threaded view
|

Re: How to use a c module with embedded lua on windows

Andrew Gierth
>>>>> "Francisco" == Francisco Olarte <[hidden email]> writes:

 >> Thanks for the reply. The host app does not export its lua API
 >> symbols. I had seen some kind of proxy dll on the lua wiki as a
 >> workaround but I'm not sure it's updated and/or relevent to my case?

 Francisco> Are you sure it does not? The fact that something DETECTS
 Francisco> multiple VM version hints at two copies being exported,

No, because the detection system doesn't work like that.

Prior to 5.4 (which abandons this particular check), the check (in
luaL_checkversion) for multiple copies of the interpreter works like
this: it checks that the lua_version() function which the current module
is calling is the same copy of the function as the one which was called
from lua_newstate() when creating the passed-in Lua state. It does this
by using the address of a static variable defined in that function.

However, the Lua core does try to get its API symbols exported by
declaring them as DLLEXPORT (which expands to suitable attribute
declarations unless overridden). With an export library, linking a
module against the host app would be easy, but since I don't do windows
myself I have no idea if you can generate this after the fact, or work
around the absense of it (maybe with a .DEF file? just guessing here).

--
Andrew.

Reply | Threaded
Open this post in threaded view
|

Re: How to use a c module with embedded lua on windows

Paul K-2
Hi Marcus,

> The host app does not export its lua API symbols. I had seen some kind of proxy dll on the lua wiki as a workaround but I'm not sure it's updated and/or relevent to my case?

As Andrew wrote, you do need to have the symbols exported if you want
to use the proxy DLL (it basically forwards the calls to the
appropriate addresses in the executable). See my SO answer and the
threads linked there for details:
https://stackoverflow.com/questions/31639483/lua-multiple-vms-detected-while-trying-to-add-extension-for-statically-linke/31639643#31639643.
I've used them with Lua 5.1 and Lua 5.2 and it shouldn't be a problem
to make one for Lua 5.3 if needed.

Another option is to compile everything statically, but if you want to
use external modules that are compiled against a Lua DLL, then the
proxy DLL is your only option.

Paul.

On Sun, Jun 23, 2019 at 4:46 PM Andrew Gierth
<[hidden email]> wrote:

>
> >>>>> "Francisco" == Francisco Olarte <[hidden email]> writes:
>
>  >> Thanks for the reply. The host app does not export its lua API
>  >> symbols. I had seen some kind of proxy dll on the lua wiki as a
>  >> workaround but I'm not sure it's updated and/or relevent to my case?
>
>  Francisco> Are you sure it does not? The fact that something DETECTS
>  Francisco> multiple VM version hints at two copies being exported,
>
> No, because the detection system doesn't work like that.
>
> Prior to 5.4 (which abandons this particular check), the check (in
> luaL_checkversion) for multiple copies of the interpreter works like
> this: it checks that the lua_version() function which the current module
> is calling is the same copy of the function as the one which was called
> from lua_newstate() when creating the passed-in Lua state. It does this
> by using the address of a static variable defined in that function.
>
> However, the Lua core does try to get its API symbols exported by
> declaring them as DLLEXPORT (which expands to suitable attribute
> declarations unless overridden). With an export library, linking a
> module against the host app would be easy, but since I don't do windows
> myself I have no idea if you can generate this after the fact, or work
> around the absense of it (maybe with a .DEF file? just guessing here).
>
> --
> Andrew.
>

Reply | Threaded
Open this post in threaded view
|

Re: How to use a c module with embedded lua on windows

Lorenzo Donati-3
In reply to this post by Francisco Olarte
On 23/06/2019 20:32, Francisco Olarte wrote:

> Marcus:
>
> On Sun, Jun 23, 2019 at 3:06 AM Marcus Mason <[hidden email]> wrote:
>> Thanks for the reply. The host app does not export its lua API symbols. I had seen some kind of proxy dll on the lua wiki as a workaround but I'm not sure it's updated and/or relevent to my case?
>
> Are you sure it does not? The fact that something DETECTS multiple VM
> version hints at two copies being exported, and from what you told it
> seem host links statically and export and you link statically and
> export, hence the collision. I'm not familiar with windows, but there
> are surely some tools for listing exported symbols for it.
>
> Francisco Olarte.
>
>
Since no-one has mentioned it, it is worth saying that Dependency Walker
[1] is a nice freeware tool for Windows that allows checking which
symbols are exported/imported by executables and DLLs.

Cheers!

-- Lorenzo


[1] http://www.dependencywalker.com/

Reply | Threaded
Open this post in threaded view
|

Re: How to use a c module with embedded lua on windows

Marcus Mason
I used dumpbin /exports to look at what was being exported by what. Cynically and curiously enough the luaL_newlib macro was the only thing calling the version check so I just expanded it and removed that call and tentatively I think it successfully loads now, however I'm sure that call was added for a reason. 

On Mon, 24 Jun 2019, 11:16 Lorenzo Donati, <[hidden email]> wrote:
On 23/06/2019 20:32, Francisco Olarte wrote:
> Marcus:
>
> On Sun, Jun 23, 2019 at 3:06 AM Marcus Mason <[hidden email]> wrote:
>> Thanks for the reply. The host app does not export its lua API symbols. I had seen some kind of proxy dll on the lua wiki as a workaround but I'm not sure it's updated and/or relevent to my case?
>
> Are you sure it does not? The fact that something DETECTS multiple VM
> version hints at two copies being exported, and from what you told it
> seem host links statically and export and you link statically and
> export, hence the collision. I'm not familiar with windows, but there
> are surely some tools for listing exported symbols for it.
>
> Francisco Olarte.
>
>
Since no-one has mentioned it, it is worth saying that Dependency Walker
[1] is a nice freeware tool for Windows that allows checking which
symbols are exported/imported by executables and DLLs.

Cheers!

-- Lorenzo


[1] http://www.dependencywalker.com/

Reply | Threaded
Open this post in threaded view
|

Re: How to use a c module with embedded lua on windows

Francisco Olarte
In reply to this post by Lorenzo Donati-3
On Mon, Jun 24, 2019 at 12:16 PM Lorenzo Donati
<[hidden email]> wrote:
...
> Since no-one has mentioned it, it is worth saying that Dependency Walker
> [1] is a nice freeware tool for Windows that allows checking which
> symbols are exported/imported by executables and DLLs.

In fact I think I used it in the nineties! ( and having a look at your
provided link I might well have ). Must  have done something right to
survive that long.

Francisco Olarte

Reply | Threaded
Open this post in threaded view
|

Re: How to use a c module with embedded lua on windows

Francisco Olarte
In reply to this post by Andrew Gierth
Andrew...

On Mon, Jun 24, 2019 at 1:46 AM Andrew Gierth
<[hidden email]> wrote:
...
>  Francisco> Are you sure it does not? The fact that something DETECTS
>  Francisco> multiple VM version hints at two copies being exported,
> No, because the detection system doesn't work like that.
> Prior to 5.4 (which abandons this particular check), the check (in
> luaL_checkversion) for multiple copies of the interpreter works like
> this: it checks that the lua_version() function which the current module
> is calling is the same copy of the function as the one which was called
> from lua_newstate() when creating the passed-in Lua state. It does this
> by using the address of a static variable defined in that function.

Mmmm, seems tricky. But I haven't been able to understand dynamic
loading since CP/M.

Isn't this thing a bit asymetric ( I mean, you have two copies, one
detects duality the other does not ( but works right ) ) ?

Francisco Olarte.