Bundling several Lua modules in one file

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Bundling several Lua modules in one file

Matt Campbell-4
Hello all:

For the past few versions, Python has been able to import modules directly from ZIP archives. This may be done to take advantage of ZIP compression or to ease distribution and installation by reducing the number of files to install. So has anyone implemented a feature like this for Lua? I know the Lua package system enables implementation of features like this outside the Lua core. If no one else has done it, then I will, but I thought I should check first.

Thanks,
Matt

Reply | Threaded
Open this post in threaded view
|

Re: Bundling several Lua modules in one file

Ico Doornekamp

* On 2008-06-02 Matt Campbell <[hidden email]> wrote  :

> Hello all:
>
> For the past few versions, Python has been able to import modules  
> directly from ZIP archives.  This may be done to take advantage of ZIP  
> compression or to ease distribution and installation by reducing the  
> number of files to install.  So has anyone implemented a feature like  
> this for Lua?  I know the Lua package system enables implementation of  
> features like this outside the Lua core.  If no one else has done it,  
> then I will, but I thought I should check first.

I implemented something like this some time ago, quite trivially. Just
create a loader for your archive format (I used .tar.gz) and add it to the
package.loaders table.

-- 
:wq
^X^Cy^K^X^C^C^C^C

Reply | Threaded
Open this post in threaded view
|

Re: Bundling several Lua modules in one file

Andreas Krinke
Ico Doornekamp schrieb:

I implemented something like this some time ago, quite trivially. Just
create a loader for your archive format (I used .tar.gz) and add it to the
package.loaders table.

Is it possible to load binary modules (*.so and *.dll) this way without extracting them to a location where the normal loader finds them?

--
ak


Reply | Threaded
Open this post in threaded view
|

Re: Bundling several Lua modules in one file

Luís Eduardo Jason Santos-2
Hi,

We have been working a project called LAR [1] that is part of the Kepler Project. Recently, we have put some brain into this discussion and found no portable/comfortable way to load binary modules without extracting them.

LAR is a virtual file system for Lua using ZIP compression. It has been developed quite a long ago, and have never achieved beta status. The code in the CVS is functional, but could use some extra hands.

[1] http://luaforge.net/projects/lar/

On Tue, Jun 3, 2008 at 10:20 AM, Andreas Krinke <[hidden email]> wrote:
Ico Doornekamp schrieb:


I implemented something like this some time ago, quite trivially. Just
create a loader for your archive format (I used .tar.gz) and add it to the
package.loaders table.

Is it possible to load binary modules (*.so and *.dll) this way without extracting them to a location where the normal loader finds them?

--
ak





--
Luís Eduardo Jason Santos
Reply | Threaded
Open this post in threaded view
|

Re: Bundling several Lua modules in one file

Andreas Krinke
Luís Eduardo Jason Santos schrieb:

We have been working a project called LAR [1] that is part of the Kepler Project. Recently, we have put some brain into this discussion and found no portable/comfortable way to load binary modules without extracting them.

LAR is a virtual file system for Lua using ZIP compression. It has been developed quite a long ago, and have never achieved beta status. The code in the CVS is functional, but could use some extra hands.

[1] http://luaforge.net/projects/lar/

Thanks a lot for this info. Currently I am using PhysFS [1] for my little engine [2]. PhysFS provides a whole virtual file system and supports several compression formats. But it's nice to know there is something more lightweight to achieve at least a part of this functionality. Unfortunately this doesn't solve the binary module loading.

[1] http://icculus.org/physfs/
[2] http://scrupp.sourceforge.net

--
ak


Reply | Threaded
Open this post in threaded view
|

Re: Bundling several Lua modules in one file

Jerome Vuarand
In reply to this post by Andreas Krinke
2008/6/3 Andreas Krinke <[hidden email]>:
> Ico Doornekamp schrieb:
>
>> I implemented something like this some time ago, quite trivially. Just
>> create a loader for your archive format (I used .tar.gz) and add it to the
>> package.loaders table.
>
> Is it possible to load binary modules (*.so and *.dll) this way without
> extracting them to a location where the normal loader finds them?

AFAIK it is not possible to have Windows load a dll from memory. It is
of course possible to do the loading and dynamic linking manually, but
that's not an easy task. I don't know the situation for shared objects
on Linux or other operating systems.

Reply | Threaded
Open this post in threaded view
|

Re: Bundling several Lua modules in one file

Daniel Quintela
In reply to this post by Andreas Krinke
Andreas Krinke escribió:
Luís Eduardo Jason Santos schrieb:

We have been working a project called LAR [1] that is part of the Kepler Project. Recently, we have put some brain into this discussion and found no portable/comfortable way to load binary modules without extracting them.

LAR is a virtual file system for Lua using ZIP compression. It has been developed quite a long ago, and have never achieved beta status. The code in the CVS is functional, but could use some extra hands.

[1] http://luaforge.net/projects/lar/

Thanks a lot for this info. Currently I am using PhysFS [1] for my little engine [2]. PhysFS provides a whole virtual file system and supports several compression formats. But it's nice to know there is something more lightweight to achieve at least a part of this functionality. Unfortunately this doesn't solve the binary module loading.

[1] http://icculus.org/physfs/
[2] http://scrupp.sourceforge.net

The Kurmi project (kurmi.luaforge.net) has support for dofile and loadlib from zip files (or executables with a trailing zip).

It implements a transparent extractToTemp+load and automatic delete of temp files at exit.

It was developed as a Lua based install system, but it can be adapted to any use.

But... it is 5.0 based... I'm sorry about that... :-(

DQ


Attachment: smime.p7s
Description: S/MIME Cryptographic Signature

Reply | Threaded
Open this post in threaded view
|

Re: Bundling several Lua modules in one file

Luís Eduardo Jason Santos-2
In reply to this post by Jerome Vuarand
It is possible, but very difficult and highly unadvisable.
Huge limitations on windows memory management nowadays.

IMHO, extract-to-temp-and-load-from
-there is the way to go.


On Tue, Jun 3, 2008 at 5:13 PM, Jérôme Vuarand <[hidden email]> wrote:
2008/6/3 Andreas Krinke <[hidden email]>:
> Ico Doornekamp schrieb:
>
>> I implemented something like this some time ago, quite trivially. Just
>> create a loader for your archive format (I used .tar.gz) and add it to the
>> package.loaders table.
>
> Is it possible to load binary modules (*.so and *.dll) this way without
> extracting them to a location where the normal loader finds them?

AFAIK it is not possible to have Windows load a dll from memory. It is
of course possible to do the loading and dynamic linking manually, but
that's not an easy task. I don't know the situation for shared objects
on Linux or other operating systems.




--
Luís Eduardo Jason Santos