LoadLibrary and 5.2

classic Classic list List threaded Threaded
30 messages Options
12
Reply | Threaded
Open this post in threaded view
|

LoadLibrary and 5.2

Antonio Scuri
 Hi all,

 Some time ago was suggested to change LoadLibrary to LoadLibraryEx then use
the LOAD_WITH_ALTERED_SEARCH_PATH flag. This would benefit the Lua for
Windows distribution so it would not have to change the PATH when installed
and we can have a standalone Lua for Windows distribution, I mean running on
a pen drive without the need of any installation.

  This is necessary because Lua for Windows uses a "clibs" subfolder for all
DLLs. The use of the "clibs" subfolder gives Lua for Windows a more
organized installation.

  My question is, can we have that change in 5.2? It still makes sense?

Thanks,
Scuri


Reply | Threaded
Open this post in threaded view
|

Re: LoadLibrary and 5.2

David Burgess-3
Sometime ago some powerful arguments for no change were placed on the
list. I believe the arguments are still valid.
DB

Antonio Scuri wrote:

>  Hi all,
>
>  Some time ago was suggested to change LoadLibrary to LoadLibraryEx then use
> the LOAD_WITH_ALTERED_SEARCH_PATH flag. This would benefit the Lua for
> Windows distribution so it would not have to change the PATH when installed
> and we can have a standalone Lua for Windows distribution, I mean running on
> a pen drive without the need of any installation.
>
>   This is necessary because Lua for Windows uses a "clibs" subfolder for all
> DLLs. The use of the "clibs" subfolder gives Lua for Windows a more
> organized installation.
>
>   My question is, can we have that change in 5.2? It still makes sense?
>
> Thanks,
> Scuri
>
>
Reply | Threaded
Open this post in threaded view
|

Re: LoadLibrary and 5.2

Theodor-Iulian Ciobanu
On Tue, 12 Jan 2010 08:36:30 +1100
David Burgess <[hidden email]> wrote:

> Sometime ago some powerful arguments for no change were placed on the
> list. I believe the arguments are still valid.

These were the arguments I found so far:
http://lua-users.org/lists/lua-l/2008-05/msg00397.html
http://lua-users.org/lists/lua-l/2008-08/msg00618.html

But maybe I haven't searched the archive deep enough because I'd like
to see this change being made as well - I've been using it for some
time for myself.

--
Theo
Reply | Threaded
Open this post in threaded view
|

Re: LoadLibrary and 5.2

David Burgess-3
Answering you query ther are a lot more posts from Lua 5.0, onwards
methinks.

At the risk of the discussion starting all over again...

The current strategy (I believe) is that Lua loads DLLs from a known set
of locations which is in effect controlled by package.cpath which can be
controlled by the environment variable LUA_CPATH. The default value is
"!\\" which expands to the executable directory path.

The strength of this approach is that Lua DLL loads are by default loads
from an explicit path, this is multiple orders of magnitude faster than
allowing windows to search for a DLL (by just using a file name). This
is also compatible with Microsofts recommendation of where to put your
application DLLs (WRT the executable) , <QUOTE>It is good practice to
install application DLLs in the same directory that contains the
application</QUOTE>ref:
http://msdn.microsoft.com/en-us/library/ms682600%28VS.85%29.aspx.

Again to quote MS from
http://msdn.microsoft.com/en-us/library/ms682586%28VS.85%29.aspx, "Note
that the standard search strategy and the alternate search strategy
specified by LoadLibraryEx with LOAD_WITH_ALTERED_SEARCH_PATH differ in
just one way: The standard search begins in the calling application's
directory, and the alternate search begins in the directory of the
executable module that LoadLibraryEx is loading."

This means that with the way loadlib.c currently works the only time
when LOAD_WITH_ALTERED_SEARCH_PATH would have an effect is when your
executable and DLLs are in different directories and one has modified
LUA_CDIR and/or LUA_CPATH to *not* use absolute path loads.

MS have change the search strategy with different OSs and indeed with
service packs, as it stands Lua has been immune to this hackery.

Of the requests for change that I have read it would seem that the
solution is not to use LOAD_WITH_ALTERED_SEARCH_PATH but to change or
add a path to the default package.cpath.

In choosing the best solution I note that DLL redirection and
manifests/assemblies complicate the Windows scene.

So maybe the solution to keep more people happy is to set the
package.cpath default to something like:

".\\?.dll;"  LUA_CDIR"?.dll;" LUA_CDIR"\dll\?.dll;" LUA_CDIR"loadall.dll"

or my preference is to use absolute paths

LUA_CDIR"?.dll;" LUA_CDIR"\dll\?.dll;" LUA_CDIR"loadall.dll"


Finally a question, does anyone have a Windows SxS assembly for a Lua 5
DLL using VS2008?

DB

Theodor-Iulian Ciobanu wrote:

> On Tue, 12 Jan 2010 08:36:30 +1100
> David Burgess <[hidden email]> wrote:
>
>> Sometime ago some powerful arguments for no change were placed on the
>> list. I believe the arguments are still valid.
>
> These were the arguments I found so far:
> http://lua-users.org/lists/lua-l/2008-05/msg00397.html
> http://lua-users.org/lists/lua-l/2008-08/msg00618.html
>
> But maybe I haven't searched the archive deep enough because I'd like
> to see this change being made as well - I've been using it for some
> time for myself.
>
Reply | Threaded
Open this post in threaded view
|

RE: LoadLibrary and 5.2

Antonio Scuri
  Hi David,

  I understand what you are saying but the main motivation for the change
was NOT to find modules, it was to find DLLs that the modules depend on and
that are not Lua modules.

  For example, iuplua depends on iup. When iuplua is required the iup DLL
must also be found. And this is NOT handled by the package.cpath. Lua for
Windows has lots of these DLLs that are NOT modules. That's the only reason
the Lua for Windows installation must change the PATH so these DLLs can be
found in the "clibs" subfolder.

  Is there any other solution to this problem despite changing LoadLibrary?

Thanks,
scuri


> -----Original Message-----
> From: [hidden email] [mailto:lua-
> [hidden email]] On Behalf Of David Burgess
> Sent: terça-feira, 12 de janeiro de 2010 09:11
> To: Lua list
> Subject: Re: LoadLibrary and 5.2
>
> Answering you query ther are a lot more posts from Lua 5.0, onwards
> methinks.
>
> At the risk of the discussion starting all over again...
>
> The current strategy (I believe) is that Lua loads DLLs from a known
> set
> of locations which is in effect controlled by package.cpath which can
> be
> controlled by the environment variable LUA_CPATH. The default value is
> "!\\" which expands to the executable directory path.
>
> The strength of this approach is that Lua DLL loads are by default
> loads
> from an explicit path, this is multiple orders of magnitude faster than
> allowing windows to search for a DLL (by just using a file name). This
> is also compatible with Microsofts recommendation of where to put your
> application DLLs (WRT the executable) , <QUOTE>It is good practice to
> install application DLLs in the same directory that contains the
> application</QUOTE>ref:
> http://msdn.microsoft.com/en-us/library/ms682600%28VS.85%29.aspx.
>
> Again to quote MS from
> http://msdn.microsoft.com/en-us/library/ms682586%28VS.85%29.aspx, "Note
> that the standard search strategy and the alternate search strategy
> specified by LoadLibraryEx with LOAD_WITH_ALTERED_SEARCH_PATH differ in
> just one way: The standard search begins in the calling application's
> directory, and the alternate search begins in the directory of the
> executable module that LoadLibraryEx is loading."
>
> This means that with the way loadlib.c currently works the only time
> when LOAD_WITH_ALTERED_SEARCH_PATH would have an effect is when your
> executable and DLLs are in different directories and one has modified
> LUA_CDIR and/or LUA_CPATH to *not* use absolute path loads.
>
> MS have change the search strategy with different OSs and indeed with
> service packs, as it stands Lua has been immune to this hackery.
>
> Of the requests for change that I have read it would seem that the
> solution is not to use LOAD_WITH_ALTERED_SEARCH_PATH but to change or
> add a path to the default package.cpath.
>
> In choosing the best solution I note that DLL redirection and
> manifests/assemblies complicate the Windows scene.
>
> So maybe the solution to keep more people happy is to set the
> package.cpath default to something like:
>
> ".\\?.dll;"  LUA_CDIR"?.dll;" LUA_CDIR"\dll\?.dll;"
> LUA_CDIR"loadall.dll"
>
> or my preference is to use absolute paths
>
> LUA_CDIR"?.dll;" LUA_CDIR"\dll\?.dll;" LUA_CDIR"loadall.dll"
>
>
> Finally a question, does anyone have a Windows SxS assembly for a Lua 5
> DLL using VS2008?
>
> DB
>
> Theodor-Iulian Ciobanu wrote:
> > On Tue, 12 Jan 2010 08:36:30 +1100
> > David Burgess <[hidden email]> wrote:
> >
> >> Sometime ago some powerful arguments for no change were placed on
> the
> >> list. I believe the arguments are still valid.
> >
> > These were the arguments I found so far:
> > http://lua-users.org/lists/lua-l/2008-05/msg00397.html
> > http://lua-users.org/lists/lua-l/2008-08/msg00618.html
> >
> > But maybe I haven't searched the archive deep enough because I'd like
> > to see this change being made as well - I've been using it for some
> > time for myself.
> >

Reply | Threaded
Open this post in threaded view
|

Re: LoadLibrary and 5.2

David Burgess-3
So your issue is with package.loadlib rather than require() ?

Antonio Scuri wrote:

>   Hi David,
>
>   I understand what you are saying but the main motivation for the change
> was NOT to find modules, it was to find DLLs that the modules depend on and
> that are not Lua modules.
>
>   For example, iuplua depends on iup. When iuplua is required the iup DLL
> must also be found. And this is NOT handled by the package.cpath. Lua for
> Windows has lots of these DLLs that are NOT modules. That's the only reason
> the Lua for Windows installation must change the PATH so these DLLs can be
> found in the "clibs" subfolder.
>
>   Is there any other solution to this problem despite changing LoadLibrary?
>
> Thanks,
> scuri
>
>
>> -----Original Message-----
>> From: [hidden email] [mailto:lua-
>> [hidden email]] On Behalf Of David Burgess
>> Sent: terça-feira, 12 de janeiro de 2010 09:11
>> To: Lua list
>> Subject: Re: LoadLibrary and 5.2
>>
>> Answering you query ther are a lot more posts from Lua 5.0, onwards
>> methinks.
>>
>> At the risk of the discussion starting all over again...
>>
>> The current strategy (I believe) is that Lua loads DLLs from a known
>> set
>> of locations which is in effect controlled by package.cpath which can
>> be
>> controlled by the environment variable LUA_CPATH. The default value is
>> "!\\" which expands to the executable directory path.
>>
>> The strength of this approach is that Lua DLL loads are by default
>> loads
>> from an explicit path, this is multiple orders of magnitude faster than
>> allowing windows to search for a DLL (by just using a file name). This
>> is also compatible with Microsofts recommendation of where to put your
>> application DLLs (WRT the executable) , <QUOTE>It is good practice to
>> install application DLLs in the same directory that contains the
>> application</QUOTE>ref:
>> http://msdn.microsoft.com/en-us/library/ms682600%28VS.85%29.aspx.
>>
>> Again to quote MS from
>> http://msdn.microsoft.com/en-us/library/ms682586%28VS.85%29.aspx, "Note
>> that the standard search strategy and the alternate search strategy
>> specified by LoadLibraryEx with LOAD_WITH_ALTERED_SEARCH_PATH differ in
>> just one way: The standard search begins in the calling application's
>> directory, and the alternate search begins in the directory of the
>> executable module that LoadLibraryEx is loading."
>>
>> This means that with the way loadlib.c currently works the only time
>> when LOAD_WITH_ALTERED_SEARCH_PATH would have an effect is when your
>> executable and DLLs are in different directories and one has modified
>> LUA_CDIR and/or LUA_CPATH to *not* use absolute path loads.
>>
>> MS have change the search strategy with different OSs and indeed with
>> service packs, as it stands Lua has been immune to this hackery.
>>
>> Of the requests for change that I have read it would seem that the
>> solution is not to use LOAD_WITH_ALTERED_SEARCH_PATH but to change or
>> add a path to the default package.cpath.
>>
>> In choosing the best solution I note that DLL redirection and
>> manifests/assemblies complicate the Windows scene.
>>
>> So maybe the solution to keep more people happy is to set the
>> package.cpath default to something like:
>>
>> ".\\?.dll;"  LUA_CDIR"?.dll;" LUA_CDIR"\dll\?.dll;"
>> LUA_CDIR"loadall.dll"
>>
>> or my preference is to use absolute paths
>>
>> LUA_CDIR"?.dll;" LUA_CDIR"\dll\?.dll;" LUA_CDIR"loadall.dll"
>>
>>
>> Finally a question, does anyone have a Windows SxS assembly for a Lua 5
>> DLL using VS2008?
>>
>> DB
>>
>> Theodor-Iulian Ciobanu wrote:
>>> On Tue, 12 Jan 2010 08:36:30 +1100
>>> David Burgess <[hidden email]> wrote:
>>>
>>>> Sometime ago some powerful arguments for no change were placed on
>> the
>>>> list. I believe the arguments are still valid.
>>> These were the arguments I found so far:
>>> http://lua-users.org/lists/lua-l/2008-05/msg00397.html
>>> http://lua-users.org/lists/lua-l/2008-08/msg00618.html
>>>
>>> But maybe I haven't searched the archive deep enough because I'd like
>>> to see this change being made as well - I've been using it for some
>>> time for myself.
>>>
>

Reply | Threaded
Open this post in threaded view
|

Re: LoadLibrary and 5.2

David Burgess-3
In reply to this post by Antonio Scuri
Just so I got this problem right we have -

luadir -
   - C module dir A which contains a C module DLL
     - subordinate DLLs  which contain DLLs that A is dependent upon
   - C module dir B which contains a C module DLL
     - subordinate DLLs  which contain DLLs that B is dependent upon
   - C module dir C which contains a C module DLL
     - subordinate DLLs  which contain DLLs that C is dependent upon

Yes?
DB
Antonio Scuri wrote:

>   Hi David,
>
>   I understand what you are saying but the main motivation for the change
> was NOT to find modules, it was to find DLLs that the modules depend on and
> that are not Lua modules.
>
>   For example, iuplua depends on iup. When iuplua is required the iup DLL
> must also be found. And this is NOT handled by the package.cpath. Lua for
> Windows has lots of these DLLs that are NOT modules. That's the only reason
> the Lua for Windows installation must change the PATH so these DLLs can be
> found in the "clibs" subfolder.
>
>   Is there any other solution to this problem despite changing LoadLibrary?
>
> Thanks,
> scuri
>
>
>> -----Original Message-----
>> From: [hidden email] [mailto:lua-
>> [hidden email]] On Behalf Of David Burgess
>> Sent: terça-feira, 12 de janeiro de 2010 09:11
>> To: Lua list
>> Subject: Re: LoadLibrary and 5.2
>>
>> Answering you query ther are a lot more posts from Lua 5.0, onwards
>> methinks.
>>
>> At the risk of the discussion starting all over again...
>>
>> The current strategy (I believe) is that Lua loads DLLs from a known
>> set
>> of locations which is in effect controlled by package.cpath which can
>> be
>> controlled by the environment variable LUA_CPATH. The default value is
>> "!\\" which expands to the executable directory path.
>>
>> The strength of this approach is that Lua DLL loads are by default
>> loads
>> from an explicit path, this is multiple orders of magnitude faster than
>> allowing windows to search for a DLL (by just using a file name). This
>> is also compatible with Microsofts recommendation of where to put your
>> application DLLs (WRT the executable) , <QUOTE>It is good practice to
>> install application DLLs in the same directory that contains the
>> application</QUOTE>ref:
>> http://msdn.microsoft.com/en-us/library/ms682600%28VS.85%29.aspx.
>>
>> Again to quote MS from
>> http://msdn.microsoft.com/en-us/library/ms682586%28VS.85%29.aspx, "Note
>> that the standard search strategy and the alternate search strategy
>> specified by LoadLibraryEx with LOAD_WITH_ALTERED_SEARCH_PATH differ in
>> just one way: The standard search begins in the calling application's
>> directory, and the alternate search begins in the directory of the
>> executable module that LoadLibraryEx is loading."
>>
>> This means that with the way loadlib.c currently works the only time
>> when LOAD_WITH_ALTERED_SEARCH_PATH would have an effect is when your
>> executable and DLLs are in different directories and one has modified
>> LUA_CDIR and/or LUA_CPATH to *not* use absolute path loads.
>>
>> MS have change the search strategy with different OSs and indeed with
>> service packs, as it stands Lua has been immune to this hackery.
>>
>> Of the requests for change that I have read it would seem that the
>> solution is not to use LOAD_WITH_ALTERED_SEARCH_PATH but to change or
>> add a path to the default package.cpath.
>>
>> In choosing the best solution I note that DLL redirection and
>> manifests/assemblies complicate the Windows scene.
>>
>> So maybe the solution to keep more people happy is to set the
>> package.cpath default to something like:
>>
>> ".\\?.dll;"  LUA_CDIR"?.dll;" LUA_CDIR"\dll\?.dll;"
>> LUA_CDIR"loadall.dll"
>>
>> or my preference is to use absolute paths
>>
>> LUA_CDIR"?.dll;" LUA_CDIR"\dll\?.dll;" LUA_CDIR"loadall.dll"
>>
>>
>> Finally a question, does anyone have a Windows SxS assembly for a Lua 5
>> DLL using VS2008?
>>
>> DB
>>
>> Theodor-Iulian Ciobanu wrote:
>>> On Tue, 12 Jan 2010 08:36:30 +1100
>>> David Burgess <[hidden email]> wrote:
>>>
>>>> Sometime ago some powerful arguments for no change were placed on
>> the
>>>> list. I believe the arguments are still valid.
>>> These were the arguments I found so far:
>>> http://lua-users.org/lists/lua-l/2008-05/msg00397.html
>>> http://lua-users.org/lists/lua-l/2008-08/msg00618.html
>>>
>>> But maybe I haven't searched the archive deep enough because I'd like
>>> to see this change being made as well - I've been using it for some
>>> time for myself.
>>>
>

Reply | Threaded
Open this post in threaded view
|

Re: LoadLibrary and 5.2

Peter Cawley
In reply to this post by David Burgess-3
On Tue, Jan 12, 2010 at 1:17 PM, David Burgess <[hidden email]> wrote:
> So your issue is with package.loadlib rather than require() ?
I do not think so.

Assume that we are distributing some executable which uses LuaCURL,
and loads it via require"curl". One could distribute all the required
files alongside each other:
my-app/my-lua-app.exe
my-app/curl.dll (the Lua glue around the CURL library)
my-app/libcurl.dll (the actual CURL library, placed in a separate DLL
due to being a different license)

Say that one wants to move Lua extension DLLs to their own directory.
It would be nice to add the extensions directory to package.cpath, and
then do something like:
my-app/my-lua-app.exe
my-app/extensions/curl.dll
my-app/extensions/libcurl.dll

However this will not currently work. require"curl" will find
my-app/extensions/curl.dll, but curl.dll depends upon libcurl.dll, and
the Windows DLL loader has no knowledge of package.cpath, so it will
try (and fail) to load my-app/libcurl.dll (as the default DLL search
order looks in the directory of the EXE, rather than the directory of
the dependee DLL being loaded). For things to work, either
LOAD_WITH_ALTERED_SEARCH_PATH needs to be used, or dependant DLLs
would need to be placed in my-app/ rather than my-app/extensions/.
Reply | Threaded
Open this post in threaded view
|

RE: LoadLibrary and 5.2

Antonio Scuri
In reply to this post by David Burgess-3
  In fact it affects require also. My issue is directly related with
LoadLibrary not being able to find secondary DLLs for the module it loads in
a given subfolder.

  If we have to move all DLLs that are not modules to the executable folder
in Lua for Windows then the clibs subfolder kind of lose its purpose and the
main folder will be a mess.

Best,
scuri

> So your issue is with package.loadlib rather than require() ?
>
> Antonio Scuri wrote:
> >   Hi David,
> >
> >   I understand what you are saying but the main motivation for the
> change
> > was NOT to find modules, it was to find DLLs that the modules depend
> on and
> > that are not Lua modules.
> >
> >   For example, iuplua depends on iup. When iuplua is required the iup
> DLL
> > must also be found. And this is NOT handled by the package.cpath. Lua
> for
> > Windows has lots of these DLLs that are NOT modules. That's the only
> reason
> > the Lua for Windows installation must change the PATH so these DLLs
> can be
> > found in the "clibs" subfolder.
> >
> >   Is there any other solution to this problem despite changing
> LoadLibrary?
> >
> > Thanks,
> > scuri
> >
> >
> >> -----Original Message-----
> >> From: [hidden email] [mailto:lua-
> >> [hidden email]] On Behalf Of David Burgess
> >> Sent: terça-feira, 12 de janeiro de 2010 09:11
> >> To: Lua list
> >> Subject: Re: LoadLibrary and 5.2
> >>
> >> Answering you query ther are a lot more posts from Lua 5.0, onwards
> >> methinks.
> >>
> >> At the risk of the discussion starting all over again...
> >>
> >> The current strategy (I believe) is that Lua loads DLLs from a known
> >> set
> >> of locations which is in effect controlled by package.cpath which
> can
> >> be
> >> controlled by the environment variable LUA_CPATH. The default value
> is
> >> "!\\" which expands to the executable directory path.
> >>
> >> The strength of this approach is that Lua DLL loads are by default
> >> loads
> >> from an explicit path, this is multiple orders of magnitude faster
> than
> >> allowing windows to search for a DLL (by just using a file name).
> This
> >> is also compatible with Microsofts recommendation of where to put
> your
> >> application DLLs (WRT the executable) , <QUOTE>It is good practice
> to
> >> install application DLLs in the same directory that contains the
> >> application</QUOTE>ref:
> >> http://msdn.microsoft.com/en-us/library/ms682600%28VS.85%29.aspx.
> >>
> >> Again to quote MS from
> >> http://msdn.microsoft.com/en-us/library/ms682586%28VS.85%29.aspx,
> "Note
> >> that the standard search strategy and the alternate search strategy
> >> specified by LoadLibraryEx with LOAD_WITH_ALTERED_SEARCH_PATH differ
> in
> >> just one way: The standard search begins in the calling
> application's
> >> directory, and the alternate search begins in the directory of the
> >> executable module that LoadLibraryEx is loading."
> >>
> >> This means that with the way loadlib.c currently works the only time
> >> when LOAD_WITH_ALTERED_SEARCH_PATH would have an effect is when your
> >> executable and DLLs are in different directories and one has
> modified
> >> LUA_CDIR and/or LUA_CPATH to *not* use absolute path loads.
> >>
> >> MS have change the search strategy with different OSs and indeed
> with
> >> service packs, as it stands Lua has been immune to this hackery.
> >>
> >> Of the requests for change that I have read it would seem that the
> >> solution is not to use LOAD_WITH_ALTERED_SEARCH_PATH but to change
> or
> >> add a path to the default package.cpath.
> >>
> >> In choosing the best solution I note that DLL redirection and
> >> manifests/assemblies complicate the Windows scene.
> >>
> >> So maybe the solution to keep more people happy is to set the
> >> package.cpath default to something like:
> >>
> >> ".\\?.dll;"  LUA_CDIR"?.dll;" LUA_CDIR"\dll\?.dll;"
> >> LUA_CDIR"loadall.dll"
> >>
> >> or my preference is to use absolute paths
> >>
> >> LUA_CDIR"?.dll;" LUA_CDIR"\dll\?.dll;" LUA_CDIR"loadall.dll"
> >>
> >>
> >> Finally a question, does anyone have a Windows SxS assembly for a
> Lua 5
> >> DLL using VS2008?
> >>
> >> DB
> >>
> >> Theodor-Iulian Ciobanu wrote:
> >>> On Tue, 12 Jan 2010 08:36:30 +1100
> >>> David Burgess <[hidden email]> wrote:
> >>>
> >>>> Sometime ago some powerful arguments for no change were placed on
> >> the
> >>>> list. I believe the arguments are still valid.
> >>> These were the arguments I found so far:
> >>> http://lua-users.org/lists/lua-l/2008-05/msg00397.html
> >>> http://lua-users.org/lists/lua-l/2008-08/msg00618.html
> >>>
> >>> But maybe I haven't searched the archive deep enough because I'd
> like
> >>> to see this change being made as well - I've been using it for some
> >>> time for myself.
> >>>
> >

Reply | Threaded
Open this post in threaded view
|

RE: LoadLibrary and 5.2

Antonio Scuri
In reply to this post by David Burgess-3
  In Lua for Windows there is only one subdir called clibs where all DLLs
are placed.

Best,
scuri

> -----Original Message-----
> From: [hidden email] [mailto:lua-
> [hidden email]] On Behalf Of David Burgess
> Sent: terça-feira, 12 de janeiro de 2010 11:29
> To: Lua list
> Subject: Re: LoadLibrary and 5.2
>
> Just so I got this problem right we have -
>
> luadir -
>    - C module dir A which contains a C module DLL
>      - subordinate DLLs  which contain DLLs that A is dependent upon
>    - C module dir B which contains a C module DLL
>      - subordinate DLLs  which contain DLLs that B is dependent upon
>    - C module dir C which contains a C module DLL
>      - subordinate DLLs  which contain DLLs that C is dependent upon
>
> Yes?
> DB
> Antonio Scuri wrote:
> >   Hi David,
> >
> >   I understand what you are saying but the main motivation for the
> change
> > was NOT to find modules, it was to find DLLs that the modules depend
> on and
> > that are not Lua modules.
> >
> >   For example, iuplua depends on iup. When iuplua is required the iup
> DLL
> > must also be found. And this is NOT handled by the package.cpath. Lua
> for
> > Windows has lots of these DLLs that are NOT modules. That's the only
> reason
> > the Lua for Windows installation must change the PATH so these DLLs
> can be
> > found in the "clibs" subfolder.
> >
> >   Is there any other solution to this problem despite changing
> LoadLibrary?
> >
> > Thanks,
> > scuri
> >
> >
> >> -----Original Message-----
> >> From: [hidden email] [mailto:lua-
> >> [hidden email]] On Behalf Of David Burgess
> >> Sent: terça-feira, 12 de janeiro de 2010 09:11
> >> To: Lua list
> >> Subject: Re: LoadLibrary and 5.2
> >>
> >> Answering you query ther are a lot more posts from Lua 5.0, onwards
> >> methinks.
> >>
> >> At the risk of the discussion starting all over again...
> >>
> >> The current strategy (I believe) is that Lua loads DLLs from a known
> >> set
> >> of locations which is in effect controlled by package.cpath which
> can
> >> be
> >> controlled by the environment variable LUA_CPATH. The default value
> is
> >> "!\\" which expands to the executable directory path.
> >>
> >> The strength of this approach is that Lua DLL loads are by default
> >> loads
> >> from an explicit path, this is multiple orders of magnitude faster
> than
> >> allowing windows to search for a DLL (by just using a file name).
> This
> >> is also compatible with Microsofts recommendation of where to put
> your
> >> application DLLs (WRT the executable) , <QUOTE>It is good practice
> to
> >> install application DLLs in the same directory that contains the
> >> application</QUOTE>ref:
> >> http://msdn.microsoft.com/en-us/library/ms682600%28VS.85%29.aspx.
> >>
> >> Again to quote MS from
> >> http://msdn.microsoft.com/en-us/library/ms682586%28VS.85%29.aspx,
> "Note
> >> that the standard search strategy and the alternate search strategy
> >> specified by LoadLibraryEx with LOAD_WITH_ALTERED_SEARCH_PATH differ
> in
> >> just one way: The standard search begins in the calling
> application's
> >> directory, and the alternate search begins in the directory of the
> >> executable module that LoadLibraryEx is loading."
> >>
> >> This means that with the way loadlib.c currently works the only time
> >> when LOAD_WITH_ALTERED_SEARCH_PATH would have an effect is when your
> >> executable and DLLs are in different directories and one has
> modified
> >> LUA_CDIR and/or LUA_CPATH to *not* use absolute path loads.
> >>
> >> MS have change the search strategy with different OSs and indeed
> with
> >> service packs, as it stands Lua has been immune to this hackery.
> >>
> >> Of the requests for change that I have read it would seem that the
> >> solution is not to use LOAD_WITH_ALTERED_SEARCH_PATH but to change
> or
> >> add a path to the default package.cpath.
> >>
> >> In choosing the best solution I note that DLL redirection and
> >> manifests/assemblies complicate the Windows scene.
> >>
> >> So maybe the solution to keep more people happy is to set the
> >> package.cpath default to something like:
> >>
> >> ".\\?.dll;"  LUA_CDIR"?.dll;" LUA_CDIR"\dll\?.dll;"
> >> LUA_CDIR"loadall.dll"
> >>
> >> or my preference is to use absolute paths
> >>
> >> LUA_CDIR"?.dll;" LUA_CDIR"\dll\?.dll;" LUA_CDIR"loadall.dll"
> >>
> >>
> >> Finally a question, does anyone have a Windows SxS assembly for a
> Lua 5
> >> DLL using VS2008?
> >>
> >> DB
> >>
> >> Theodor-Iulian Ciobanu wrote:
> >>> On Tue, 12 Jan 2010 08:36:30 +1100
> >>> David Burgess <[hidden email]> wrote:
> >>>
> >>>> Sometime ago some powerful arguments for no change were placed on
> >> the
> >>>> list. I believe the arguments are still valid.
> >>> These were the arguments I found so far:
> >>> http://lua-users.org/lists/lua-l/2008-05/msg00397.html
> >>> http://lua-users.org/lists/lua-l/2008-08/msg00618.html
> >>>
> >>> But maybe I haven't searched the archive deep enough because I'd
> like
> >>> to see this change being made as well - I've been using it for some
> >>> time for myself.
> >>>
> >

Reply | Threaded
Open this post in threaded view
|

Re: LoadLibrary and 5.2

David Burgess-3
In reply to this post by Antonio Scuri
I will sleep on it.

Antonio Scuri wrote:

>   Hi David,
>
>   I understand what you are saying but the main motivation for the change
> was NOT to find modules, it was to find DLLs that the modules depend on and
> that are not Lua modules.
>
>   For example, iuplua depends on iup. When iuplua is required the iup DLL
> must also be found. And this is NOT handled by the package.cpath. Lua for
> Windows has lots of these DLLs that are NOT modules. That's the only reason
> the Lua for Windows installation must change the PATH so these DLLs can be
> found in the "clibs" subfolder.
>
>   Is there any other solution to this problem despite changing LoadLibrary?
>
> Thanks,
> scuri
>
>
>> -----Original Message-----
>> From: [hidden email] [mailto:lua-
>> [hidden email]] On Behalf Of David Burgess
>> Sent: terça-feira, 12 de janeiro de 2010 09:11
>> To: Lua list
>> Subject: Re: LoadLibrary and 5.2
>>
>> Answering you query ther are a lot more posts from Lua 5.0, onwards
>> methinks.
>>
>> At the risk of the discussion starting all over again...
>>
>> The current strategy (I believe) is that Lua loads DLLs from a known
>> set
>> of locations which is in effect controlled by package.cpath which can
>> be
>> controlled by the environment variable LUA_CPATH. The default value is
>> "!\\" which expands to the executable directory path.
>>
>> The strength of this approach is that Lua DLL loads are by default
>> loads
>> from an explicit path, this is multiple orders of magnitude faster than
>> allowing windows to search for a DLL (by just using a file name). This
>> is also compatible with Microsofts recommendation of where to put your
>> application DLLs (WRT the executable) , <QUOTE>It is good practice to
>> install application DLLs in the same directory that contains the
>> application</QUOTE>ref:
>> http://msdn.microsoft.com/en-us/library/ms682600%28VS.85%29.aspx.
>>
>> Again to quote MS from
>> http://msdn.microsoft.com/en-us/library/ms682586%28VS.85%29.aspx, "Note
>> that the standard search strategy and the alternate search strategy
>> specified by LoadLibraryEx with LOAD_WITH_ALTERED_SEARCH_PATH differ in
>> just one way: The standard search begins in the calling application's
>> directory, and the alternate search begins in the directory of the
>> executable module that LoadLibraryEx is loading."
>>
>> This means that with the way loadlib.c currently works the only time
>> when LOAD_WITH_ALTERED_SEARCH_PATH would have an effect is when your
>> executable and DLLs are in different directories and one has modified
>> LUA_CDIR and/or LUA_CPATH to *not* use absolute path loads.
>>
>> MS have change the search strategy with different OSs and indeed with
>> service packs, as it stands Lua has been immune to this hackery.
>>
>> Of the requests for change that I have read it would seem that the
>> solution is not to use LOAD_WITH_ALTERED_SEARCH_PATH but to change or
>> add a path to the default package.cpath.
>>
>> In choosing the best solution I note that DLL redirection and
>> manifests/assemblies complicate the Windows scene.
>>
>> So maybe the solution to keep more people happy is to set the
>> package.cpath default to something like:
>>
>> ".\\?.dll;"  LUA_CDIR"?.dll;" LUA_CDIR"\dll\?.dll;"
>> LUA_CDIR"loadall.dll"
>>
>> or my preference is to use absolute paths
>>
>> LUA_CDIR"?.dll;" LUA_CDIR"\dll\?.dll;" LUA_CDIR"loadall.dll"
>>
>>
>> Finally a question, does anyone have a Windows SxS assembly for a Lua 5
>> DLL using VS2008?
>>
>> DB
>>
>> Theodor-Iulian Ciobanu wrote:
>>> On Tue, 12 Jan 2010 08:36:30 +1100
>>> David Burgess <[hidden email]> wrote:
>>>
>>>> Sometime ago some powerful arguments for no change were placed on
>> the
>>>> list. I believe the arguments are still valid.
>>> These were the arguments I found so far:
>>> http://lua-users.org/lists/lua-l/2008-05/msg00397.html
>>> http://lua-users.org/lists/lua-l/2008-08/msg00618.html
>>>
>>> But maybe I haven't searched the archive deep enough because I'd like
>>> to see this change being made as well - I've been using it for some
>>> time for myself.
>>>
>

Reply | Threaded
Open this post in threaded view
|

Re: LoadLibrary and 5.2

RJP Computing
In reply to this post by Antonio Scuri
On Tue, Jan 12, 2010 at 9:00 AM, Antonio Scuri <[hidden email]> wrote:
 In Lua for Windows there is only one subdir called clibs where all DLLs
are placed.

Best,
scuri

I completely agree. This has forces LfW to need to edit Windows PATH. I think this is not great and we would like to not need to mess with a users machine.

-- 

Regards,
Ryan
Reply | Threaded
Open this post in threaded view
|

Re: LoadLibrary and 5.2

Jerome Vuarand
2010/1/12 RJP Computing <[hidden email]>:

> On Tue, Jan 12, 2010 at 9:00 AM, Antonio Scuri <[hidden email]>
> wrote:
>>
>>  In Lua for Windows there is only one subdir called clibs where all DLLs
>> are placed.
>>
>> Best,
>> scuri
>
> I completely agree. This has forces LfW to need to edit Windows PATH. I
> think this is not great and we would like to not need to mess with a users
> machine.

The modification you're asking for doesn't make Lua better, it makes
it more suited to your application (LfW). But changing the PATH
environment variable is not the only alternative.

In your situation I'd rather patch the lua.exe interpreter so that it
replaces the default C module searchers with a custom version that use
LOAD_WITH_ALTERED_SEARCH_PATH. The Lua dll stays intact, should it be
used by other applications (that may or may not use C modules from the
LfW install). Only the interpreter application has to know about the
specific way to load modules, and with a custom searcher that's
exactly the case.

And if you don't want to patch Lua, it's still possible to put that
searcher substition code in a LUA_INIT environment variable.
Reply | Threaded
Open this post in threaded view
|

Re: LoadLibrary and 5.2

Cosmin Apreutesei
In reply to this post by Antonio Scuri
>  I understand what you are saying but the main motivation for the change
> was NOT to find modules, it was to find DLLs that the modules depend on and
> that are not Lua modules.
>
>  For example, iuplua depends on iup. When iuplua is required the iup DLL
> must also be found. And this is NOT handled by the package.cpath. Lua for
> Windows has lots of these DLLs that are NOT modules. That's the only reason
> the Lua for Windows installation must change the PATH so these DLLs can be
> found in the "clibs" subfolder.
>
>  Is there any other solution to this problem despite changing LoadLibrary?


As usual M$ solve it very late in the game... Here's a snippet from my
win32 test suite of fbclient:


--bind SetDllDirectory from kernel32.dll (available in WinXP SP1+ and Vista)
--it's the official way in Windows to load a bunch of related dlls
from a directory of your chosing.
local alien = require'alien'
local kernel32 = alien.load('kernel32')
local ok,err = xpcall(function()
  kernel32.SetDllDirectoryA:types{ABI='stdcall',ret='int','string'}
end,debug.traceback)
if not ok then
  error ([[
Sorry but you need a kernel32.dll with SetDllDirectoryA() (WinXP SP1+ or Vista)
to run the automated test suite on Windows.
The error was: ]]..err)
end

function set_dll_directory(dir)
  assert(kernel32.SetDllDirectoryA(dir) ~= 0, 'SetDllDirectoryA() error')
end
Reply | Threaded
Open this post in threaded view
|

RE: LoadLibrary and 5.2

Antonio Scuri
In reply to this post by Jerome Vuarand
> The modification you're asking for doesn't make Lua better, it makes
> it more suited to your application (LfW). But changing the PATH
> environment variable is not the only alternative.

  I do think it makes Lua better because LfW is more than a simple "application". It is a very popular Lua distribution for general use. But if there are alternatives I would like to know them.

 
> In your situation I'd rather patch the lua.exe interpreter so that it
> replaces the default C module searchers with a custom version that use
> LOAD_WITH_ALTERED_SEARCH_PATH. The Lua dll stays intact, should it be
> used by other applications (that may or may not use C modules from the
> LfW install). Only the interpreter application has to know about the
> specific way to load modules, and with a custom searcher that's
> exactly the case.

  How we do that?

 
> And if you don't want to patch Lua, it's still possible to put that
> searcher substition code in a LUA_INIT environment variable.

  No we can't do this, it will not solve the problem of dependent DLLs. Because that DLLs are searched by the system, not by Lua.

Best,
Scuri


Reply | Threaded
Open this post in threaded view
|

RE: LoadLibrary and 5.2

Antonio Scuri
In reply to this post by Cosmin Apreutesei
  So you are saying that one possible solution is to call SetDllDirectory on the Lua interpreter?

  Checking its documentation it looks promising. The only drawback is that it needs Service Pack 1 in Windows XP.

Best,
scuri

> -----Original Message-----
> From: [hidden email] [mailto:lua-
> [hidden email]] On Behalf Of Cosmin Apreutesei
> Sent: terça-feira, 12 de janeiro de 2010 12:38
> To: Lua list
> Subject: Re: LoadLibrary and 5.2
>
> >  I understand what you are saying but the main motivation for the
> change
> > was NOT to find modules, it was to find DLLs that the modules depend
> on and
> > that are not Lua modules.
> >
> >  For example, iuplua depends on iup. When iuplua is required the iup
> DLL
> > must also be found. And this is NOT handled by the package.cpath. Lua
> for
> > Windows has lots of these DLLs that are NOT modules. That's the only
> reason
> > the Lua for Windows installation must change the PATH so these DLLs
> can be
> > found in the "clibs" subfolder.
> >
> >  Is there any other solution to this problem despite changing
> LoadLibrary?
>
>
> As usual M$ solve it very late in the game... Here's a snippet from my
> win32 test suite of fbclient:
>
>
> --bind SetDllDirectory from kernel32.dll (available in WinXP SP1+ and
> Vista)
> --it's the official way in Windows to load a bunch of related dlls
> from a directory of your chosing.
> local alien = require'alien'
> local kernel32 = alien.load('kernel32')
> local ok,err = xpcall(function()
>   kernel32.SetDllDirectoryA:types{ABI='stdcall',ret='int','string'}
> end,debug.traceback)
> if not ok then
>   error ([[
> Sorry but you need a kernel32.dll with SetDllDirectoryA() (WinXP SP1+
> or Vista)
> to run the automated test suite on Windows.
> The error was: ]]..err)
> end
>
> function set_dll_directory(dir)
>   assert(kernel32.SetDllDirectoryA(dir) ~= 0, 'SetDllDirectoryA()
> error')
> end

Reply | Threaded
Open this post in threaded view
|

Re: LoadLibrary and 5.2

Ignacio Burgueño
On 12/01/2010 14:57, Antonio Scuri wrote:
>    So you are saying that one possible solution is to call SetDllDirectory on the Lua interpreter?
>
>    Checking its documentation it looks promising. The only drawback is that it needs Service Pack 1 in Windows XP.
>

Another drawback is that it is a process-wide change. It will cause
trouble if we have more than one Lua VM per process.


Reply | Threaded
Open this post in threaded view
|

Re: LoadLibrary and 5.2

Thomas Harning Jr.
In reply to this post by Antonio Scuri
On Tue, Jan 12, 2010 at 11:57 AM, Antonio Scuri
<[hidden email]> wrote:
>  So you are saying that one possible solution is to call SetDllDirectory on the Lua interpreter?
>
>  Checking its documentation it looks promising. The only drawback is that it needs Service Pack 1 in Windows XP.
Another possible solution is to somehow track down the dependent
libraries, load them w/ package.loadlib to get the DLL loaded in the
namespace... you don't even need to know a valid function in the
library from what it looks like... since once a library is loaded, it
looks like it will never be unloaded, unless you remove it from the
registry (indexed by LIBPREFIX .. path).

The basic algorithm for Windows dependent library loading is this:
 * If a library with the same base filename is already loaded (ex:
MyLibrary.dll) use that
 * Search for appropriate library in the search paths
... it doesn't matter if the MyLibrary.dll is the 'right' one, since
it just uses the base name... no path involved.

the package.loadlib trick would get the dependent library into the
namespace, then Windows would use that and increment the use count
since a DLL depended on it.


Use case:
 * for each library module X depends on
   * package.loadlib(library, "dontcare")
 * load module X
 * for each library module X depends on
  * debug.getregistry()[PREFIX .. library] = nil

Nasty... but a possible solution to an ugly problem...

--
Thomas Harning Jr.
Reply | Threaded
Open this post in threaded view
|

Re: LoadLibrary and 5.2

Ignacio Burgueño
In reply to this post by Antonio Scuri
On 12/01/2010 14:51, Antonio Scuri wrote:
>> The modification you're asking for doesn't make Lua better, it makes
>> it more suited to your application (LfW). But changing the PATH
>> environment variable is not the only alternative.
>
>    I do think it makes Lua better because LfW is more than a simple "application". It is a very popular Lua distribution for general use. But if there are alternatives I would like to know them.

Or LuaRocks, for the same reasons.

>
>
>> In your situation I'd rather patch the lua.exe interpreter so that it
>> replaces the default C module searchers with a custom version that use
>> LOAD_WITH_ALTERED_SEARCH_PATH. The Lua dll stays intact, should it be
>> used by other applications (that may or may not use C modules from the
>> LfW install). Only the interpreter application has to know about the
>> specific way to load modules, and with a custom searcher that's
>> exactly the case.
>
>    How we do that?

Maybe the solution would imply having a module that would place our
custom loader in package.loaders.

require "alteredpath"

That module will roughly do something like:

table.insert(package.loaders, 1, function(modname)
   local path, err = package.searchpath(modname, package.cpath)
   if not path then deal_with_error end
   load_module_with_altered_search_path(path)
   -- etc
end)


The issue I have with this code is the index '1'. I'd like to insert
this loader before the C loader. How do I know where is it? If only
package.c_loader were available so one could look for it in the table...

Regards,
Ignacio


Reply | Threaded
Open this post in threaded view
|

Re: LoadLibrary and 5.2

Theodor-Iulian Ciobanu
In reply to this post by Antonio Scuri
On Tue, 12 Jan 2010 11:57:41 -0200
"Antonio Scuri" <[hidden email]> wrote:

>   If we have to move all DLLs that are not modules to the executable
> folder in Lua for Windows then the clibs subfolder kind of lose its
> purpose and the main folder will be a mess.

Not only will it be a mess, but it might cause problems with other
applications. I have the Lua folder in PATH. Having these libraries
next to lua.exe means they would also be available to LoadLibrary. For
me it meant wget.exe not working anymore because it was loading the
wrong ssleay32.dll. This was easily fixed, but I can imagine worse case
scenarios.
12