Distributing Lua packages to Windows machines

classic Classic list List threaded Threaded
19 messages Options
Reply | Threaded
Open this post in threaded view
|

Distributing Lua packages to Windows machines

mchalkley
I know this is a sore subject to begin with, and is probaby on the
"forbidden topics for Mondays" list (or should be), but I'm afraid I
have to do it anyway.

The issue is that a server/application monitoring program I've written
in Lua has become very useful (that's the good news), which now
necessitates deploying it to a bunch (about 75) of our customer's
Windows servers (that's the bad news). The program requires lfs,
socket, sigar, and bit. To my knowledge, sigar only works with Lua 5.1.

To have a completely standalone application within a single directory,
this is what I've gotten down to:

<MonitorDir>:
lua.exe
lua51.dll
bit.dll
lfs.dll
liblua.dll
sigar.dll
libsigar.dll
socket.lua
mime.lua
Monitor.lua

socket - a directory containing:
core.dll

mime - a directory containing:
core.dll

With the above configuration, I can use Windows Task Manager to run
the monitor program and it works just fine.

Ideally, I'd like to run it with LuaJIT, but for now, I'd be thrilled
to just get the above into a single windows exe that could be packed
with UPX/zipped/etc.

My objective is to decrease the number of files necessary to deploy
and also to keep folks from mucking about with the code and screwing
up something, so compressing/obfuscating/something along those lines
would be necessary.

Is there currently a "best way" to do this?

Mark


Reply | Threaded
Open this post in threaded view
|

Re: Distributing Lua packages to Windows machines

Sean Conner
It was thus said that the Great [hidden email] once stated:
> I know this is a sore subject to begin with, and is probaby on the
> "forbidden topics for Mondays" list (or should be), but I'm afraid I
> have to do it anyway.

  [ snip ]

> My objective is to decrease the number of files necessary to deploy
> and also to keep folks from mucking about with the code and screwing
> up something, so compressing/obfuscating/something along those lines
> would be necessary.
>
> Is there currently a "best way" to do this?

  I don't know of any program to construct a single executable, but here's
how I do it at work (similar problem, but instead of multiple Windows boxes
it's multiple Unix boxes)---I compiled everything into a single executable.

  The main C program is very simple---it's just a single function (main())
that creates a Lua state, calls an initialization function (see below), then
loads the main script (in your case, it looks like it would be
"Monitor.lua") and executes it.

  The "initialization function" [1] modifies the stock Lua module loader to
look for builtin modules stored (or linked) into the executable (and it
would include Monitor.lua in your case).  The C modules are statically
linked into the executable.  The Lua modules are run through a tool [2] that
converts them into a C file or an object file [3] which are then included
into the final executable.  

  Yeah, it was a chore writing the Makefile to build all this, but now that
it's done, I don't have to worry about it anymore.  But I only recommend
this path if you are comfortable with compiling, linking and writing
makefiles.  There might be an automatic way to do this under Windows, but I
not sure what it is, and it would probably do something like this anyway.

  -spc (Hope this helps some ... )

[1] http://boston.conman.org/2013/03/23.1

[2] Just do a search for 'bin2c' and you'll find plenty.

[3] I first compile the Lua modules with luac, then compress using zlib
        to decrease the size of the executable.  I have found that they
        compress even better if I don't use luac first, but I've been too
        lazy to fix the Makefile.

        I then modified the preloader code to decompress the module when
        loading.  Note---only the modules written in Lua are compressed.

Reply | Threaded
Open this post in threaded view
|

Re: Distributing Lua packages to Windows machines

Nagaev Boris
In reply to this post by mchalkley
On Mon, Mar 2, 2015 at 6:12 PM,  <[hidden email]> wrote:

> I know this is a sore subject to begin with, and is probaby on the
> "forbidden topics for Mondays" list (or should be), but I'm afraid I
> have to do it anyway.
>
> The issue is that a server/application monitoring program I've written
> in Lua has become very useful (that's the good news), which now
> necessitates deploying it to a bunch (about 75) of our customer's
> Windows servers (that's the bad news). The program requires lfs,
> socket, sigar, and bit. To my knowledge, sigar only works with Lua 5.1.
>
> To have a completely standalone application within a single directory,
> this is what I've gotten down to:
>
> <MonitorDir>:
> lua.exe
> lua51.dll
> bit.dll
> lfs.dll
> liblua.dll
> sigar.dll
> libsigar.dll
> socket.lua
> mime.lua
> Monitor.lua
>
> socket - a directory containing:
> core.dll
>
> mime - a directory containing:
> core.dll
>
> With the above configuration, I can use Windows Task Manager to run
> the monitor program and it works just fine.
>
> Ideally, I'd like to run it with LuaJIT, but for now, I'd be thrilled
> to just get the above into a single windows exe that could be packed
> with UPX/zipped/etc.
>
> My objective is to decrease the number of files necessary to deploy
> and also to keep folks from mucking about with the code and screwing
> up something, so compressing/obfuscating/something along those lines
> would be necessary.
>
> Is there currently a "best way" to do this?
>
> Mark
>
>

I use MXE [1] to build a static executable for Windows. There are MXE
packages: Lua (version 5.2) and LuaJIT. Use one of them as a static
library. Program bin2c.lua [2] can be used to include your Lua file
into C file as a string. In C create Lua state, load Lua code from the
string and execute it. And finally statically link the executable with
modules used getting all-in-one EXE file.

> and also to keep folks from mucking about with the code and screwing
> up something, so compressing/obfuscating/something along those lines
> would be necessary.
Converting Lua file to byte-code before distributing or embedding it
is time-cost trade off. A normal user can't convert byte-code back to
Lua. A hacker can disable almost any kind of protection.

I've created Lua rock "luacryptor" [3], which takes a Lua module and
converts it to C module. It also has an option to convert to byte-code
before encrypting. All Lua functions are encrypted with a password in
the C module. A function is decrypted each time being called. Do not
judge strictly, it is my first Lua rock.

 [1] http://mxe.cc/
 [2] http://lua-users.org/files/wiki_insecure/users/MarkEdgar/bin2c.lua
 [3] https://github.com/starius/luacryptor

Best regards,
Boris Nagaev

Reply | Threaded
Open this post in threaded view
|

Re: Distributing Lua packages to Windows machines

Philipp Janda
In reply to this post by mchalkley
Am 02.03.2015 um 19:12 schröbte [hidden email]:
> [...]
>
> My objective is to decrease the number of files necessary to deploy
> and also to keep folks from mucking about with the code and screwing
> up something, so compressing/obfuscating/something along those lines
> would be necessary.
>
> Is there currently a "best way" to do this?

Not necessarily the "best way", but I have successfully used (a slightly
modified) eee[1] to package an IUP application on Windows a while ago.

>
> Mark
>

Philipp

   [1]:  http://www.erikveen.dds.nl/eee/index.html



Reply | Threaded
Open this post in threaded view
|

Re: Distributing Lua packages to Windows machines

steve donovan
In reply to this post by mchalkley
On Mon, Mar 2, 2015 at 8:12 PM,  <[hidden email]> wrote:
> I know this is a sore subject to begin with, and is probaby on the
> "forbidden topics for Mondays" list (or should be), but I'm afraid I
> have to do it anyway.

It is now Tuesday so we can safely discuss this.  The simplest way is
to use lhf's srlua to glue lua.exe and Monitor.lua together as an
executable.

Windows people are used to an application sitting in its own directory
with its dependencies. This way at least no one can fiddle with the
script in the mistaken belief that they can improve it.

steve d.

Reply | Threaded
Open this post in threaded view
|

Re: Distributing Lua packages to Windows machines

aryajur
In reply to this post by Philipp Janda
I use srlua[1] to pack my application for distribution as an exe. You can build srlua with Lua 5.1, 5.2 or 5.3 and also include the relevant windows manifest to take advantage of XP/Vista visual effects if you have any GUI programming in Lua. You can also include a application icon. srlua takes and embeds 1 lua file. So if your code spans multiple lua file as mine does I use the many2one [2] utility to make a single Lua file and combine it all the Lua files into 1. That is a small Lua script which is very simple conceptually and easily understandable.

After using many2One and then srlua I just have a single executable and then I distribute that with all the required dll files. I think it is possible to include all the dll's in the srlua compilation so that in the end you are just left with 1 exe but I haven't done that myself.

Regards,
Milind

On Mon, Mar 2, 2015 at 2:58 PM, Philipp Janda <[hidden email]> wrote:
Am 02.03.2015 um 19:12 schröbte [hidden email]:
[...]

My objective is to decrease the number of files necessary to deploy
and also to keep folks from mucking about with the code and screwing
up something, so compressing/obfuscating/something along those lines
would be necessary.

Is there currently a "best way" to do this?

Not necessarily the "best way", but I have successfully used (a slightly modified) eee[1] to package an IUP application on Windows a while ago.


Mark


Philipp

  [1]:  http://www.erikveen.dds.nl/eee/index.html




Reply | Threaded
Open this post in threaded view
|

Re: Distributing Lua packages to Windows machines

Luiz Henrique de Figueiredo
> srlua takes and embeds 1 lua file.

That is correct but that Lua file can be the output of luac, and luac
does combine several files into one. However, luac does not handle
requiring bundled files. You could write a replacement for require that
does that. You can also use my luac.lua which can be told to mark some
bundled file as pre-required. However, luac.lua is not portable and is
only available for Lua 5.1 and in 32-bit platforms.

Reply | Threaded
Open this post in threaded view
|

Re: Distributing Lua packages to Windows machines

David Krause

Not Lua:
In my opinion the best way to deploy software on windows is with an installer.
I would suggest to take a look at NSIS ( http://nsis.sourceforge.net/Main_Page ).
With NSIS you could pack everything nicely into one file, install everything silently and even generate (installable) patch diffs with vpatch
( http://nsis.sourceforge.net/VPatch_plug-in )
The "downside" is that you still have the to deploy the whole directory (but create nice desktop links to start your application).

David Krause


Reply | Threaded
Open this post in threaded view
|

Re: Distributing Lua packages to Windows machines

mchalkley
In reply to this post by Luiz Henrique de Figueiredo


Tuesday, March 3, 2015, 6:22:56 AM, you wrote:

>> srlua takes and embeds 1 lua file.

> That is correct but that Lua file can be the output of luac, and luac
> does combine several files into one. However, luac does not handle
> requiring bundled files. You could write a replacement for require that
> does that. You can also use my luac.lua which can be told to mark some
> bundled file as pre-required. However, luac.lua is not portable and is
> only available for Lua 5.1 and in 32-bit platforms.

I really appreciate all the responses and ideas, both via the list and
directly.

I'm going over all of them to see what will work best for me...

Thanks again!

Mark


Reply | Threaded
Open this post in threaded view
|

Re: Distributing Lua packages to Windows machines

Peter Drahoš

I really appreciate all the responses and ideas, both via the list and
directly.

I'm going over all of them to see what will work best for me...

Thanks again!

Mark


Adding one more option to the topic.
You can simply pack your application directory into a single executable using apps such as Cameyo[1]. I did this before with LuaDist binaries[2] for multiple machines and windows versions without issues.

pd


signature.asc (859 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Distributing Lua packages to Windows machines

Renaud Kern
Hello

I use  InnoScript[1] to distribute my lua programs on Windows. I usually include Lua, Luarocks and the rocks the programs need. It's free and there is some decent documentation. There is a kind of IDE also [2]  that might help.

Does anyone tried the "chocolatey" package manager [3] ?

Best,

Renaud

2015-03-03 16:51 GMT+01:00 Peter Drahoš <[hidden email]>:

I really appreciate all the responses and ideas, both via the list and
directly.

I'm going over all of them to see what will work best for me...

Thanks again!

Mark


Adding one more option to the topic.
You can simply pack your application directory into a single executable using apps such as Cameyo[1]. I did this before with LuaDist binaries[2] for multiple machines and windows versions without issues.

pd


Reply | Threaded
Open this post in threaded view
|

Re: Distributing Lua packages to Windows machines

Javier Guerra Giraldez
In reply to this post by mchalkley
On Mon, Mar 2, 2015 at 1:12 PM,  <[hidden email]> wrote:
> Ideally, I'd like to run it with LuaJIT, but for now, I'd be thrilled
> to just get the above into a single windows exe that could be packed
> with UPX/zipped/etc.


If you're already using LuaJIT, it can save the bytecode as an .obj
that can be linked with your C application[1].  LuaJIT includes a
slight modification to require() that loads those linked bytecodes
just as if it were plain Lua files.


[1]: http://luajit.org/running.html  check the -b option

--
Javier

Reply | Threaded
Open this post in threaded view
|

Re: Distributing Lua packages to Windows machines

aryajur
In reply to this post by Luiz Henrique de Figueiredo


On Tue, Mar 3, 2015 at 3:22 AM, Luiz Henrique de Figueiredo <[hidden email]> wrote:
> srlua takes and embeds 1 lua file.
 
However, luac does not handle
requiring bundled files. You could write a replacement for require that
does that.

That is exactly what the many2one script does after adding the required lua files as string chunks in the main lua file. 

Reply | Threaded
Open this post in threaded view
|

Re: Distributing Lua packages to Windows machines

Jerome Vuarand
In reply to this post by mchalkley
2015-03-02 18:12 GMT+00:00  <[hidden email]>:
> My objective is to decrease the number of files necessary to deploy
> and also to keep folks from mucking about with the code and screwing
> up something, so compressing/obfuscating/something along those lines
> would be necessary.
>
> Is there currently a "best way" to do this?

I don't think there's a best way, but there are many solutions.

What I've done in the past is write a custom launcher binary. All the
Lua modules and the main script were embedded as PE resources inside
the executable. All the binary modules were statically linked (but
with the luaopen_modname entry points still exported). The C main
function would be similar from the one in lua.c, except it would
replace the Lua and C searchers/loaders with a couple a custom ones.
One would look for luaopen_modname functions in the current executable
instead of separate DLLs, and the other would look for Lua modules in
resources. Then it would load a resource with a predefined name as the
main script.

One advantage of doing it that way is that you can edit an exe file
resources without recompiling it. So as long as the set of binary
modules you use stays the same, you can reuse the single binary for
multiple programs.

The code (undocumented and maybe outdated) is available online if
you're interested: https://bitbucket.org/doub/launcher and
https://bitbucket.org/doub/launcher2

Reply | Threaded
Open this post in threaded view
|

Re: Distributing Lua packages to Windows machines

Samir Tine
In reply to this post by mchalkley
<mchalkley <at> mail.com> writes:

>
> I know this is a sore subject to
begin with, and is probaby on the
> "forbidden topics for Mondays" list
(or should be), but I'm afraid I
> have to do it anyway.
>
> The issue is that a
server/application monitoring program
I've written
> in Lua has become very useful (that's
the good news), which now
> necessitates deploying it to a bunch
(about 75) of our customer's
> Windows servers (that's the bad
news). The program requires lfs,
> socket, sigar, and bit. To my
knowledge, sigar only works with Lua
5.1.
>
> To have a completely standalone
application within a single directory,

> this is what I've gotten down to:
>
> <MonitorDir>:
> lua.exe
> lua51.dll
> bit.dll
> lfs.dll
> liblua.dll
> sigar.dll
> libsigar.dll
> socket.lua
> mime.lua
> Monitor.lua
>
> socket - a directory containing:
> core.dll
>
> mime - a directory containing:
> core.dll
>
> With the above configuration, I can
use Windows Task Manager to run
> the monitor program and it works just
fine.
>
> Ideally, I'd like to run it with
LuaJIT, but for now, I'd be thrilled
> to just get the above into a single
windows exe that could be packed
> with UPX/zipped/etc.
>
> My objective is to decrease the
number of files necessary to deploy
> and also to keep folks from mucking
about with the code and screwing
> up something, so
compressing/obfuscating/something along
those lines
> would be necessary.
>
> Is there currently a "best way" to do
this?
>
> Mark
>
>

Hi Mark,

I'm working on ljc, a tool that ease
the deployement of lua programs to
standalone executables on Windows and
Linux. The first considered-alpha
version is available. Take a look at
http://www.ljcompiler.com
Hope it helps.

Sam



Reply | Threaded
Open this post in threaded view
|

Re: Distributing Lua packages to Windows machines

mchalkley


Saturday, March 7, 2015, 9:03:00 AM, you wrote:

> <mchalkley <at> mail.com> writes:

>>
>> I know this is a sore subject to
> begin with, and is probaby on the
>> "forbidden topics for Mondays" list
> (or should be), but I'm afraid I
>> have to do it anyway.
>>
>> The issue is that a
> server/application monitoring program
> I've written
>> in Lua has become very useful (that's
> the good news), which now
>> necessitates deploying it to a bunch
> (about 75) of our customer's
>> Windows servers (that's the bad
> news). The program requires lfs,
>> socket, sigar, and bit. To my
> knowledge, sigar only works with Lua
> 5.1.
>>
>> To have a completely standalone
> application within a single directory,
>> this is what I've gotten down to:
>>
>> <MonitorDir>:
>> lua.exe
>> lua51.dll
>> bit.dll
>> lfs.dll
>> liblua.dll
>> sigar.dll
>> libsigar.dll
>> socket.lua
>> mime.lua
>> Monitor.lua
>>
>> socket - a directory containing:
>> core.dll
>>
>> mime - a directory containing:
>> core.dll
>>
>> With the above configuration, I can
> use Windows Task Manager to run
>> the monitor program and it works just
> fine.
>>
>> Ideally, I'd like to run it with
> LuaJIT, but for now, I'd be thrilled
>> to just get the above into a single
> windows exe that could be packed
>> with UPX/zipped/etc.
>>
>> My objective is to decrease the
> number of files necessary to deploy
>> and also to keep folks from mucking
> about with the code and screwing
>> up something, so
> compressing/obfuscating/something along
> those lines
>> would be necessary.
>>
>> Is there currently a "best way" to do
> this?
>>          ,
>> Mark
>>
>>

> Hi Mark,

> I'm working on ljc, a tool that ease
> the deployement of lua programs to
> standalone executables on Windows and
> Linux. The first considered-alpha
> version is available. Take a look at
> http://www.ljcompiler.com
> Hope it helps.

> Sam

Very interesting, Sam!  Does it work with Lua 5.1?

Thanks,

Mark


Reply | Threaded
Open this post in threaded view
|

Re: Distributing Lua packages to Windows machines

Samir Tine

> > Hi Mark,
>
> > I'm working on ljc, a tool that
ease
> > the deployement of lua programs to
> > standalone executables on Windows
and
> > Linux. The first considered-alpha
> > version is available. Take a look
at
> > http://www.ljcompiler.com
> > Hope it helps.
>
> > Sam
>
> Very interesting, Sam!  Does it work
with Lua 5.1?
>
> Thanks,
>
> Mark
>
>

Because ljc is based on LuaJIT, yes
it's compatible with Lua 5.1

Expect a new release very soon, with
lot of bugfixes and new features.

Sam


Reply | Threaded
Open this post in threaded view
|

Re: Distributing Lua packages to Windows machines

Tony Papadimitriou
Just so you know, the current version did not work for me (Win7).  I tried
it with a very-very simple app.  The exe crashed without any signs of even
partial execution of the script.

-----Original Message-----
From: Samir Tine
Sent: Monday, March 09, 2015 12:40 AM
To: [hidden email]
Subject: Re: Distributing Lua packages to Windows machines


> > Hi Mark,
>
> > I'm working on ljc, a tool that
ease
> > the deployement of lua programs to
> > standalone executables on Windows
and
> > Linux. The first considered-alpha
> > version is available. Take a look
at
> > http://www.ljcompiler.com
> > Hope it helps.
>
> > Sam
>
> Very interesting, Sam!  Does it work
with Lua 5.1?
>
> Thanks,
>
> Mark
>
>

Because ljc is based on LuaJIT, yes
it's compatible with Lua 5.1

Expect a new release very soon, with
lot of bugfixes and new features.

Sam



Reply | Threaded
Open this post in threaded view
|

Re: Distributing Lua packages to Windows machines

Samir Tine
 <tonyp <at> acm.org> writes:

>
> Just so you know, the current version did not work for me (Win7).  I tried
> it with a very-very simple app.  The exe crashed without any signs of even
> partial execution of the script.


Thanx for the report Tony. Msvcrt.dll hell must explain your crash.
Should be fixed in the next release.

Sam