Running Lua from within a Window Program

classic Classic list List threaded Threaded
26 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Running Lua from within a Window Program

Leinen, Rick

Greetings,

 

Can mingw use the existing lua53.dll created for command line Lua execution in a program it creates?  In other words, can I call Lua from my C program created with mingw using the lua53.dll or do I have to compile the Lua source files with the program?

 

I’m an embedded programmer, not a windows programmer.

 

Thanks,

 

Rick

 

 

Reply | Threaded
Open this post in threaded view
|

Re: Running Lua from within a Window Program

Erik Hougaard
On 19-09-2016 17:01, Leinen, Rick wrote:
Can mingw use the existing lua53.dll created for command line Lua execution in a program it creates?  In other words, can I call Lua from my C program created with mingw using the lua53.dll or do I have to compile the Lua source files with the program?
Yes, that's the beauty with DLLs :)

lua53.dll is a Win32 DLL, and lua.exe is also a "Windows program", just the console type. If you create a windowing Windows program, that is still just a Win32 application and the DLL works the exact same way from that.

If your Windows Program is dotNET, you can use one of many wrappers to ease the usage.

The only trouble you can get into, is if your DLL is 32bit and the Windows Program is 64bit, then you need a Lua53.dll compiled for 64bit (a 64bit process cannot load a 32bit DLL).

/Erik
Reply | Threaded
Open this post in threaded view
|

Re: Running Lua from within a Window Program

steve donovan
On Tue, Sep 20, 2016 at 4:39 AM, Erik Hougaard <[hidden email]> wrote:
> The only trouble you can get into, is if your DLL is 32bit and the Windows
> Program is 64bit, then you need a Lua53.dll compiled for 64bit (a 64bit
> process cannot load a 32bit DLL).

The other thing that's important: the DLL must also be compiled with
mingw so that both the program and the DLL are using the same runtime.

Reply | Threaded
Open this post in threaded view
|

Re: Running Lua from within a Window Program

Thijs Schreijer

> On 20 Sep 2016, at 08:11, steve donovan <[hidden email]> wrote:
>
> On Tue, Sep 20, 2016 at 4:39 AM, Erik Hougaard <[hidden email]> wrote:
>> The only trouble you can get into, is if your DLL is 32bit and the Windows
>> Program is 64bit, then you need a Lua53.dll compiled for 64bit (a 64bit
>> process cannot load a 32bit DLL).
>
> The other thing that's important: the DLL must also be compiled with
> mingw so that both the program and the DLL are using the same runtime.
>

This  ^^^   actually is the most important caveat. 64 vs 32 bit you will notice immediately, yet having the wrong runtime, will give you hard to trace sporadic failures.

Thijs
Reply | Threaded
Open this post in threaded view
|

Re: Running Lua from within a Window Program

Shmuel Zeigerman
On 20/09/2016 10:19, Thijs Schreijer wrote:

>> On 20 Sep 2016, at 08:11, steve donovan <[hidden email]> wrote:
>>
>> On Tue, Sep 20, 2016 at 4:39 AM, Erik Hougaard <[hidden email]> wrote:
>>> The only trouble you can get into, is if your DLL is 32bit and the Windows
>>> Program is 64bit, then you need a Lua53.dll compiled for 64bit (a 64bit
>>> process cannot load a 32bit DLL).
>> The other thing that's important: the DLL must also be compiled with
>> mingw so that both the program and the DLL are using the same runtime.
>>
> This  ^^^   actually is the most important caveat. 64 vs 32 bit you will notice immediately, yet having the wrong runtime, will give you hard to trace sporadic failures.
>

As Lua APIs do not use run-time-specific types (such as FILE*), and
neither require malloc/free to be called from different sides of an app,
why different runtimes should be an issue?
I use mingw-built lua51.dll with Borland C++ Builder 5.0 applications
for years without problems.
Please give an example where one could have a problem.

--
Shmuel


Reply | Threaded
Open this post in threaded view
|

Re: Running Lua from within a Window Program

Viacheslav Usov
On Tue, Sep 20, 2016 at 9:55 AM, Shmuel Zeigerman <[hidden email]> wrote:

> As Lua APIs do not use run-time-specific types (such as FILE*), and neither require malloc/free to be called from different sides of an app, why different runtimes should be an issue?

The C language has a bunch of implementation-defined types and macros. A particular runtime will expect a particular set of definitions for those. For example, the jmp_buf type that is required to call setjmp and longjmp is specified to be an array of an unspecified size; it is expressly unspecified whether setjmp is a macro or an identifier with external linkage. Thus any code using setjmp/longjmp compiled for one runtime can be incompatible with another runtime.

Lua uses setjmp/longjmp to raise errors, so it is susceptible to such incompatibilities. Lua can also be compiled using the C++ throw/catch mechanism for that purpose, but that is even more runtime-dependent.

Cheers,
V.

Reply | Threaded
Open this post in threaded view
|

Re: Running Lua from within a Window Program

Christian N.
Am 20.09.2016 um 14:37 schrieb Viacheslav Usov:
> Lua uses setjmp/longjmp to raise errors, so it is susceptible to such
> incompatibilities. Lua can also be compiled using the C++ throw/catch
> mechanism for that purpose, but that is even more runtime-dependent.
>
> Cheers,
> V.
>

I thought of setjmp/longjmp too but the setjmp (pcall) and longjmp
(error) both happen in the same Lua DLL. So as long as longjmp is not
implemented in a way that relies on the stack frames between
setjmp/longjmp to have a particular format (e.g. when implemented using
exception handling), that should work.

‒ Christian

Reply | Threaded
Open this post in threaded view
|

Re: Running Lua from within a Window Program

Thijs Schreijer
In reply to this post by Viacheslav Usov

On 20 Sep 2016, at 14:37, Viacheslav Usov <[hidden email]> wrote:

On Tue, Sep 20, 2016 at 9:55 AM, Shmuel Zeigerman <[hidden email]> wrote:

> As Lua APIs do not use run-time-specific types (such as FILE*), and neither require malloc/free to be called from different sides of an app, why different runtimes should be an issue?

The C language has a bunch of implementation-defined types and macros. A particular runtime will expect a particular set of definitions for those. For example, the jmp_buf type that is required to call setjmp and longjmp is specified to be an array of an unspecified size; it is expressly unspecified whether setjmp is a macro or an identifier with external linkage. Thus any code using setjmp/longjmp compiled for one runtime can be incompatible with another runtime.

Lua uses setjmp/longjmp to raise errors, so it is susceptible to such incompatibilities. Lua can also be compiled using the C++ throw/catch mechanism for that purpose, but that is even more runtime-dependent.

Cheers,
V.



The most common problem is passing objects around (file handles etc.) their internal structures have changed in between versions and hence can cause trouble.


Thijs
Reply | Threaded
Open this post in threaded view
|

Re: Running Lua from within a Window Program

Viacheslav Usov
In reply to this post by Christian N.
On Tue, Sep 20, 2016 at 2:42 PM, Christian N. <[hidden email]> wrote:

> I thought of setjmp/longjmp too but the setjmp (pcall) and longjmp (error) both happen in the same Lua DLL.

That does not matter. The problem in question is Lua built for runtime A but is executed with runtime B. In runtime A, the size of jmp_buf is a; in runtime B the size is b, and it may be possible that a != b. Then, if a < b, runtime B's setjmp called by Lua will cause a buffer overrun, or, if a > b, runtime B's longjmp will not restore some of the state required by Lua compiled for runtime A.

Secondly, setjmp, being a macro in either A or B or both, may do something not compatible with the other runtime's longjmp even if a == b.

Cheers,
V.
Reply | Threaded
Open this post in threaded view
|

Re: Running Lua from within a Window Program

Viacheslav Usov
In reply to this post by Thijs Schreijer
On Tue, Sep 20, 2016 at 2:45 PM, Thijs Schreijer <[hidden email]> wrote:

> The most common problem is passing objects around (file handles etc.) their internal structures have changed in between versions and hence can cause trouble.

To which Shmuel Zeigerman objected, saying that Lua's API does not use any such objects, which, I think, is correct.

Cheers,
V.
Reply | Threaded
Open this post in threaded view
|

Re: Running Lua from within a Window Program

Christian N.
In reply to this post by Viacheslav Usov
Am 20.09.2016 um 15:01 schrieb Viacheslav Usov:

> On Tue, Sep 20, 2016 at 2:42 PM, Christian N. <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>> I thought of setjmp/longjmp too but the setjmp (pcall) and longjmp
> (error) both happen in the same Lua DLL.
>
> That does not matter. The problem in question is Lua built for runtime A
> but is executed with runtime B. In runtime A, the size of jmp_buf is a;
> in runtime B the size is b, and it may be possible that a != b. Then, if
> a < b, runtime B's setjmp called by Lua will cause a buffer overrun, or,
> if a > b, runtime B's longjmp will not restore some of the state
> required by Lua compiled for runtime A.
>
> Secondly, setjmp, being a macro in either A or B or both, may do
> something not compatible with the other runtime's longjmp even if a == b.
>
> Cheers,
> V.

Of course if a binary was compiled against CRT A but is dynamically
linked against CRT B then all bets are off.

But normally incompatible runtimes will have different DLL names
(msvcr110.dll, msvcr120.dll, etc.). Thus when you compile a DLL against
CRT A and your executable against CRT B then both CRTs will be loaded at
runtime and the DLL and the main executable will both use their own ones
independently.

‒ Christian

Reply | Threaded
Open this post in threaded view
|

Re: Running Lua from within a Window Program

Viacheslav Usov
On Tue, Sep 20, 2016 at 3:12 PM, Christian N. <[hidden email]> wrote:

> Of course if a binary was compiled against CRT A but is dynamically linked against CRT B then all bets are off.

That is essentially what I tried to illustrate.

> But normally incompatible runtimes will have different DLL names (msvcr110.dll, msvcr120.dll, etc.).

In the case of MinGW, they will not. It always links to the DLL named msvcrt.dll, which is not even supposed to be linked to except by the OS's own components [1].

Cheers,
V.

Reply | Threaded
Open this post in threaded view
|

Re: Running Lua from within a Window Program

steve donovan
On Tue, Sep 20, 2016 at 3:28 PM, Viacheslav Usov <[hidden email]> wrote:
> msvcrt.dll, which is not even supposed to be linked to except by the OS's
> own components [1].

And yet, it's so terribly convenient ;)  MS runtimes seem to change
every year, and they are irritating to distribute.

Reply | Threaded
Open this post in threaded view
|

Re: Running Lua from within a Window Program

Christian N.
In reply to this post by Viacheslav Usov
Am 20.09.2016 um 15:28 schrieb Viacheslav Usov:

>> But normally incompatible runtimes will have different DLL names
>> (msvcr110.dll, msvcr120.dll, etc.).
>
> In the case of MinGW, they will not. It always links to the DLL named
> msvcrt.dll, which is not even supposed to be linked to except by the
> OS's own components [1].
>
> Cheers,
> V.
>
> [1] https://blogs.msdn.microsoft.com/oldnewthing/20140411-00/?p=1273

I fail to see how this is a problem. If I link my MSVC-compiled program
with a MinGW-compiled lua53.dll, I might get the following dependency
tree (swap msvcrt.dll with msvcr120.dll for the reverse case):

   myprogram.exe
   ├───lua53.dll
   │   └───msvcrt.dll
   └───msvcr120.dll

Two runtimes are loaded independently, as described before.

‒ Christian

Reply | Threaded
Open this post in threaded view
|

Re: Running Lua from within a Window Program

Viacheslav Usov
On Tue, Sep 20, 2016 at 4:01 PM, Christian N. <[hidden email]> wrote:

> I fail to see how this is a problem. If I link my MSVC-compiled program with a MinGW-compiled lua53.dll, I might get the following dependency tree (swap msvcrt.dll with msvcr120.dll for the reverse case):

First, the reliance on msvcrt.dll is a problem with MinGW in general, not just with Lua built with MinGW.

Second, a problem with using different runtimes is that those runtimes may each assume they are the only runtime, with disastrous consequences.

Third, there is the problem of passing "objects" created in one runtime to another one, as mentioned by others earlier, even though I do not think it affects Lua.

Fourth, multiple libraries that work well with one runtime may fail to work when they use multiple runtimes even in salutations not covered by #3, because they will not have a common global state in their runtimes. E.g., if some C code wants to communicate with some Lua code via the environment, this may or may not work unpredictably if runtimes are allowed to be different.

Cheers,
V.
Reply | Threaded
Open this post in threaded view
|

Re: Running Lua from within a Window Program

Philipp Janda
In reply to this post by Viacheslav Usov
Am 20.09.2016 um 15:11 schröbte Viacheslav Usov:
> On Tue, Sep 20, 2016 at 2:45 PM, Thijs Schreijer <[hidden email]>
> wrote:
>
>> The most common problem is passing objects around (file handles etc.)
> their internal structures have changed in between versions and hence can
> cause trouble.
>
> To which Shmuel Zeigerman objected, saying that Lua's API does not use any
> such objects, which, I think, is correct.

Lua has an io library (which usually is bundled with the Lua API and
thus linked to the same runtime), and some popular third-party modules
(e.g. luaposix and luafilesystem) can work with Lua file handles.

>
> Cheers,
> V.
>

Philipp




Reply | Threaded
Open this post in threaded view
|

Re: Running Lua from within a Window Program

Thijs Schreijer
In reply to this post by Christian N.

> On 20 Sep 2016, at 15:12, Christian N. <[hidden email]> wrote:
>
> Am 20.09.2016 um 15:01 schrieb Viacheslav Usov:
>> On Tue, Sep 20, 2016 at 2:42 PM, Christian N. <[hidden email]
>> <mailto:[hidden email]>> wrote:
>>
>>> I thought of setjmp/longjmp too but the setjmp (pcall) and longjmp
>> (error) both happen in the same Lua DLL.
>>
>> That does not matter. The problem in question is Lua built for runtime A
>> but is executed with runtime B. In runtime A, the size of jmp_buf is a;
>> in runtime B the size is b, and it may be possible that a != b. Then, if
>> a < b, runtime B's setjmp called by Lua will cause a buffer overrun, or,
>> if a > b, runtime B's longjmp will not restore some of the state
>> required by Lua compiled for runtime A.
>>
>> Secondly, setjmp, being a macro in either A or B or both, may do
>> something not compatible with the other runtime's longjmp even if a == b.
>>
>> Cheers,
>> V.
>
> Of course if a binary was compiled against CRT A but is dynamically linked against CRT B then all bets are off.
>
> But normally incompatible runtimes will have different DLL names (msvcr110.dll, msvcr120.dll, etc.). Thus when you compile a DLL against CRT A and your executable against CRT B then both CRTs will be loaded at runtime and the DLL and the main executable will both use their own ones independently.
>
> ‒ Christian
>

This is simply not true, read up on the MSDN link I send earlier in the thread, where Microsoft discusses the potential problems of using multiple runtime (don’t take my word for it, better take theirs, they wrote the stuff).

Repeating; mixing runtime ‘seems’ to work, until it doesn’t. You’ll get sporadic and hard to trace crashes. Search the archives, plenty of others went before you…

Thijs
Reply | Threaded
Open this post in threaded view
|

Re: Running Lua from within a Window Program

Viacheslav Usov
In reply to this post by Philipp Janda
On Wed, Sep 21, 2016 at 12:07 AM, Philipp Janda <[hidden email]> wrote:

> Lua has an io library (which usually is bundled with the Lua API and thus linked to the same runtime), and some popular third-party modules (e.g. luaposix and luafilesystem) can work with Lua file handles.

Technically, it is those libraries, not Lua, which are responsible for this. Practically, the difference is slim. I think there is enough rationale in this thread to conclude that mixing runtimes is a very bad idea.

Cheers,
V.
Reply | Threaded
Open this post in threaded view
|

Mixing CRTs on windows (was: Re: Running Lua from within a Window Program)

Christian N.
In reply to this post by Thijs Schreijer
Am 21.09.2016 um 01:33 schrieb Thijs Schreijer:

>
>> On 20 Sep 2016, at 15:12, Christian N. <[hidden email]> wrote:
>> Of course if a binary was compiled against CRT A but is dynamically linked against CRT B then all bets are off.
>>
>> But normally incompatible runtimes will have different DLL names (msvcr110.dll, msvcr120.dll, etc.). Thus when you compile a DLL against CRT A and your executable against CRT B then both CRTs will be loaded at runtime and the DLL and the main executable will both use their own ones independently.
>>
>> ‒ Christian
>>
>
> This is simply not true, read up on the MSDN link I send earlier in the thread, where Microsoft discusses the potential problems of using multiple runtime (don’t take my word for it, better take theirs, they wrote the stuff).
>
> Repeating; mixing runtime ‘seems’ to work, until it doesn’t. You’ll get sporadic and hard to trace crashes. Search the archives, plenty of others went before you…
>
> Thijs
>


First, even though I argue that in the case of Lua, there is no problem
with mixed CRTs, I do so mostly on theoretical grounds (though I did
have success mixing a small MSVC 2015 executable and the MSVC 2013 Lua
DLL). I do not recommend mixing CRTs if at all possible.

But I have read your MSDN link
(https://msdn.microsoft.com/en-us/library/ms235460.aspx) and if
anything, it seems to support my arguments:

 > Each copy of the CRT library has a separate and distinct state.

In detail:

 > When you pass C Run-time (CRT) objects such as file handles, locales,
 > and environment variables into or out of a DLL unexpected behavior
 > can occur if the DLL, as well as the files calling into the DLL, use
 > different copies of the CRT libraries.

Since Lua 5.3, there is indeed a (single) place where this happens: A
luaL_Stream has a FILE* member. Also communicating with Lua via
environment variables and locales may or may not be possible across CRTs
(I think that was already mentioned in this thread). That alone is
already good reason not to mix CRTs.

 > A related problem can occur when you allocate memory (either
 > explicitly with new or malloc, or implicitly with strdup,
 > strstreambuf::str, and so on) and then pass a pointer across a DLL
 > boundary to be freed.

This is not the case for Lua thanks to the usage of a single lua_Alloc
function that manages all memory (but beware calling lua_setallocf!).

If you interpret the "if" in the following sentence as "if and only if":

 > If you design your DLL so that it passes CRT objects across the
 > boundary or allocates memory and expects it to be freed outside the
 > DLL, you restrict the DLL users to use the same copy of the CRT
 > library as the DLL.

then this is even documented as working. Which would make sense when you
consider things like ActiveX, COM, OLE which are binary standards (but
more or less a superset of a library-less C ABI) designed to work even
across language boundaries. E.g. I can see in my running explorer.exe:
- ucrtbase.dll (VC 14)
- msvcr120.dll
- msvcr90.dll
- msvcrt.dll (this one seems to be loaded into every process)

Shell plugins and the like that work across different windows versions
would not be possible if only one CRT could be loaded per process. I
thus conclude that a CRT may not assume that it is the only one in a
process (on Windows).


Tangetially related: With the ucrtbase.dll since Windows 10 / VS 2015,
the operating system becomes a CRT delivery channel again (cf. the
oldnewthing blog article mentioned previously in this thread). This CRT
is announced to be stable across major VS versions
(https://blogs.msdn.microsoft.com/vcblog/2015/03/03/introducing-the-universal-crt/)
and will thus hopefully tame the Windows CRT jungle.

‒ Christian

Reply | Threaded
Open this post in threaded view
|

Re: Running Lua from within a Window Program

Jerome Vuarand
In reply to this post by Christian N.
2016-09-20 15:01 GMT+01:00 Christian N. <[hidden email]>:

> Am 20.09.2016 um 15:28 schrieb Viacheslav Usov:
>>>
>>> But normally incompatible runtimes will have different DLL names
>>> (msvcr110.dll, msvcr120.dll, etc.).
>>
>>
>> In the case of MinGW, they will not. It always links to the DLL named
>> msvcrt.dll, which is not even supposed to be linked to except by the
>> OS's own components [1].
>>
>> Cheers,
>> V.
>>
>> [1] https://blogs.msdn.microsoft.com/oldnewthing/20140411-00/?p=1273
>
>
> I fail to see how this is a problem. If I link my MSVC-compiled program with
> a MinGW-compiled lua53.dll, I might get the following dependency tree (swap
> msvcrt.dll with msvcr120.dll for the reverse case):
>
>   myprogram.exe
>   ├───lua53.dll
>   │   └───msvcrt.dll
>   └───msvcr120.dll
>
> Two runtimes are loaded independently, as described before.

C runtimes are not just a bunch of clean C functions using only stack
memory. Some might expect their main wrapper to have been called to
initialize some globals and will break if the exe was linked to
something else. Some might use thread-local storage in different ways,
and overwrite each other's memory.

But even with two nice and clean runtimes, the Lua 5.3 API has a FILE*
in luaL_Stream (I was under the impression that older Lua version had
some FILE* too, but I can't find them now). If your program creates
some of these, or if binary modules access some of Lua's own streams,
you need to make sure every party involved agree on what FILE*
pointers point at.

12