Minimal Lua 5.x installations

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

Minimal Lua 5.x installations

Benjamin TERRIER
Hi,

I am looking to embed Lua into an small embedded system.

I have found the Technical Notes LTN 1 and LTN 2 about how to remove
the lexer and parser to reduce the size of the binary.

However, these notes are for Lua 3 and 4 and I have not found anything
similar for Lua 5.
The only thing I have found is the last question in the FAQ on the Wiki:

> Lua code can be stored and loaded in a pre-compiled binary form which can be generated by its standard compiler tool, luac.
> The binary form is more memory-efficient than source text, but also allows another possibility for the interpreter: the parsing/compiling
> code can be removed altogether. This reduces the size of the already small Lua core to about 40K. Lua programs in binary form can
> also be encoded as C strings and easily stored in the host's source code. It is therefore convenient to use Lua in devices that lack
> a filesystem.

So this strategy seems to be still possible to do, but I am not sure
that the guidelines of LTN 2 are still meaningful.
For instance `lua_dobuffer()` has been removed and the replacement is
to use `luaL_loadbuffer()` which accepts both text and binary.
I have not found a function that can load/run only binaries. So, I am
wondering if the parser is more linked into the library (and harder to
remove) in Lua 5.

To conclude: is there a guide to remove the parser from Lua 5?
Is there a better function than `luaL_loadbuffer()` to run binaries?

Thanks for your help

Regards,

Benjamin

Reply | Threaded
Open this post in threaded view
|

Re: Minimal Lua 5.x installations

Sean Conner
It was thus said that the Great Benjamin TERRIER once stated:
>
> To conclude: is there a guide to remove the parser from Lua 5?
> Is there a better function than `luaL_loadbuffer()` to run binaries?

  In the Lua 5.1 distribution, there's the file etc/noparser.c with
instructions on its use.  That could be a useful base upon which to build if
you need to use a version of Lua later than 5.1.

  -spc


Reply | Threaded
Open this post in threaded view
|

Re: Minimal Lua 5.x installations

Jonathan Goble
In reply to this post by Benjamin TERRIER
On Sat, Jul 6, 2019 at 6:03 PM Benjamin TERRIER <[hidden email]> wrote:
>
> For instance `lua_dobuffer()` has been removed and the replacement is
> to use `luaL_loadbuffer()` which accepts both text and binary.
> I have not found a function that can load/run only binaries. So, I am
> wondering if the parser is more linked into the library (and harder to
> remove) in Lua 5.
>
> To conclude: is there a guide to remove the parser from Lua 5?
> Is there a better function than `luaL_loadbuffer()` to run binaries?

First, you need to specify the Lua version more precisely, as Lua does
not use semantic versioning and different 5.x versions have
incompatibilities with each other, including breaking C API changes.
5.1 and 5.3 are the most commonly used from my understanding; 5.1 is
officially end-of-life but commonly used for compatibility with LuaJIT
(a third-party project built on 5.1), and 5.3 is the latest stable
version. (5.4 is in alpha right now.) The remainder of this email
assumes you are referring to 5.3.

Let's look at the reference manual for 5.3 [1]:

- `luaL_loadbuffer` is stated to be equivalent to `luaL_loadbufferx`
with mode `NULL`.
- `luaL_loadbufferx` says that it uses `lua_load`, and mode is a
string with the same behavior as in `lua_load`.
- `lua_load` refers to the Lua standard library function `load` for
the use of the mode string.
- Lua `load` function says that mode can be "t" for text chunks only,
"b" for binary chunks only, or "bt" for either.

So for your question, "Is there a better function than
`luaL_loadbuffer()` to run binaries?", the answer is
`luaL_loadbufferx()` with the mode argument set to "b".

[1] https://www.lua.org/manual/5.3/manual.html

Reply | Threaded
Open this post in threaded view
|

Re: Minimal Lua 5.x installations

Luiz Henrique de Figueiredo
In reply to this post by Benjamin TERRIER
> To conclude: is there a guide to remove the parser from Lua 5?

See http://www.lua.org/extras/5.3/noparser.c

Reply | Threaded
Open this post in threaded view
|

Re: Minimal Lua 5.x installations

Benjamin TERRIER
In reply to this post by Jonathan Goble
> So for your question, "Is there a better function than
> `luaL_loadbuffer()` to run binaries?", the answer is
> `luaL_loadbufferx()` with the mode argument set to "b".

Thank you.

>   In the Lua 5.1 distribution, there's the file etc/noparser.c with
> instructions on its use.  That could be a useful base upon which to build if
> you need to use a version of Lua later than 5.1.

> > To conclude: is there a guide to remove the parser from Lua 5?
>
> See http://www.lua.org/extras/5.3/noparser.c

I was able to use the noparser.c file from 5.1.
Since I am using 5.3.5, I had to change the signature of the parser
function, but it works perfectly.

I saw the file fro 5.3 just when I was done ^^

Thank you all.

Reply | Threaded
Open this post in threaded view
|

Re: Minimal Lua 5.x installations

Russell Haley


On Sat, Jul 6, 2019 at 3:55 PM Benjamin TERRIER <[hidden email]> wrote:
> So for your question, "Is there a better function than
> `luaL_loadbuffer()` to run binaries?", the answer is
> `luaL_loadbufferx()` with the mode argument set to "b".

Thank you.

>   In the Lua 5.1 distribution, there's the file etc/noparser.c with
> instructions on its use.  That could be a useful base upon which to build if
> you need to use a version of Lua later than 5.1.

> > To conclude: is there a guide to remove the parser from Lua 5?
>
> See http://www.lua.org/extras/5.3/noparser.c

I was able to use the noparser.c file from 5.1.
Since I am using 5.3.5, I had to change the signature of the parser
function, but it works perfectly.

I saw the file fro 5.3 just when I was done ^^

Thank you all.

Can you get Lua on something with 1 MB storage and 256kb of Ram like a Nordicsemi chip?

Or is that strictly eLua territory?

I was playing with a TI Lanchpad that had those specs too: http://www.ti.com/tool/cc3220sf-launchxl
It was using FreeRTOS and the chip has a fairly complete posix API and a built in socket library. I would have liked to have a Lua interpreter running in a thread.

Russ 
Reply | Threaded
Open this post in threaded view
|

Re: Minimal Lua 5.x installations

Benjamin TERRIER
Le dim. 7 juil. 2019 à 06:39, Russell Haley <[hidden email]> a écrit :

>
> Can you get Lua on something with 1 MB storage and 256kb of Ram like a Nordicsemi chip?
> https://www.nordicsemi.com/Products/Low-power-short-range-wireless/nRF52840
>
> Or is that strictly eLua territory?
>
> I was playing with a TI Lanchpad that had those specs too: http://www.ti.com/tool/cc3220sf-launchxl
> It was using FreeRTOS and the chip has a fairly complete posix API and a built in socket library. I would have liked to have a Lua interpreter running in a thread.
>
> Russ

I am trying to do something similar.
Compiling Lua without parser, dump and undump, it increases the size
of my final binary by 41 KB.
With the libraries base, bit32, coroutine, math, string and table, I
have a total increase of 83 KB.

So you should be good with the 1 MB storage. I still have to check the
RAM consumption and the size of Lua bytecode.

Regards

Benjamin

Reply | Threaded
Open this post in threaded view
|

Re: Minimal Lua 5.x installations

Luiz Henrique de Figueiredo
> Compiling Lua without parser, dump and undump, it increases the size
> of my final binary by 41 KB.

It *increases* the size of the binary?

Reply | Threaded
Open this post in threaded view
|

Re: Minimal Lua 5.x installations

Sean Conner
It was thus said that the Great Luiz Henrique de Figueiredo once stated:
> > Compiling Lua without parser, dump and undump, it increases the size
> > of my final binary by 41 KB.
>
> It *increases* the size of the binary?

  Adding Lua, minus those parts, increased his application by 41K.  I assume
that Lua with those parts would be a larger increase.

  -spc


Reply | Threaded
Open this post in threaded view
|

Re: Minimal Lua 5.x installations

Flemming Madsen-3
In reply to this post by Benjamin TERRIER
You should be able to do a fair bit with 256kB RAM. I have a nontrivial app layer implemented in roughly 3 kLOC of lua with a full Lua 5.3 + some bindings on an ESP32 and it consumes just over 100kB. This is not a data heavy app though, mostly business logic.

/Flemming

On Tue, 9 Jul 2019 at 17:39, Benjamin TERRIER <[hidden email]> wrote:
Le dim. 7 juil. 2019 à 06:39, Russell Haley <[hidden email]> a écrit :
>
> Can you get Lua on something with 1 MB storage and 256kb of Ram like a Nordicsemi chip?
> https://www.nordicsemi.com/Products/Low-power-short-range-wireless/nRF52840
>
> Or is that strictly eLua territory?
>
> I was playing with a TI Lanchpad that had those specs too: http://www.ti.com/tool/cc3220sf-launchxl
> It was using FreeRTOS and the chip has a fairly complete posix API and a built in socket library. I would have liked to have a Lua interpreter running in a thread.
>
> Russ

I am trying to do something similar.
Compiling Lua without parser, dump and undump, it increases the size
of my final binary by 41 KB.
With the libraries base, bit32, coroutine, math, string and table, I
have a total increase of 83 KB.

So you should be good with the 1 MB storage. I still have to check the
RAM consumption and the size of Lua bytecode.

Regards

Benjamin

Reply | Threaded
Open this post in threaded view
|

Re: Minimal Lua 5.x installations

Flemming Madsen-3
Correcting myself: Actual heap usage for the Lua interpreter as reported by collectgarbage('count') is 180 kB after initialization. Would still fit though ..

On Wed, 10 Jul 2019 at 22:25, Flemming Madsen <[hidden email]> wrote:
You should be able to do a fair bit with 256kB RAM. I have a nontrivial app layer implemented in roughly 3 kLOC of lua with a full Lua 5.3 + some bindings on an ESP32 and it consumes just over 100kB. This is not a data heavy app though, mostly business logic.

/Flemming

On Tue, 9 Jul 2019 at 17:39, Benjamin TERRIER <[hidden email]> wrote:
Le dim. 7 juil. 2019 à 06:39, Russell Haley <[hidden email]> a écrit :
>
> Can you get Lua on something with 1 MB storage and 256kb of Ram like a Nordicsemi chip?
> https://www.nordicsemi.com/Products/Low-power-short-range-wireless/nRF52840
>
> Or is that strictly eLua territory?
>
> I was playing with a TI Lanchpad that had those specs too: http://www.ti.com/tool/cc3220sf-launchxl
> It was using FreeRTOS and the chip has a fairly complete posix API and a built in socket library. I would have liked to have a Lua interpreter running in a thread.
>
> Russ

I am trying to do something similar.
Compiling Lua without parser, dump and undump, it increases the size
of my final binary by 41 KB.
With the libraries base, bit32, coroutine, math, string and table, I
have a total increase of 83 KB.

So you should be good with the 1 MB storage. I still have to check the
RAM consumption and the size of Lua bytecode.

Regards

Benjamin