Is it possible to pre-allocate a table with a specific size(i.e. pre-allocating the max memory that may be used by a table)?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Is it possible to pre-allocate a table with a specific size(i.e. pre-allocating the max memory that may be used by a table)?

孙世龙 sunshilong
Hi, list

Is it possible to pre-allocate a table with a specific size(i.e.
pre-allocating the max memory that may be used by a table)?

As per the book(Programming in Lua 4th Edition, page 37), which says that:
To represent a conventional array or a list, we simply use a table
with integer keys. There is neither a way
nor a need to declare a size.

Could somebody shed some light on this question?
I would be grateful to have some help with this question.

Best wishes
Sunshlong
Reply | Threaded
Open this post in threaded view
|

Re: Is it possible to pre-allocate a table with a specific size(i.e. pre-allocating the max memory that may be used by a table)?

Ranier Vilela-2
Em dom., 11 de out. de 2020 às 04:58, 孙世龙 sunshilong <[hidden email]> escreveu:
Hi, list

Is it possible to pre-allocate a table with a specific size(i.e.
pre-allocating the max memory that may be used by a table)?

As per the book(Programming in Lua 4th Edition, page 37), which says that:
To represent a conventional array or a list, we simply use a table
with integer keys. There is neither a way
nor a need to declare a size.
As far as I know, there's no way.
Even the Lua C API is limited by int (MAX_INT).

regards,
Ranier Vilela
Reply | Threaded
Open this post in threaded view
|

Re: Is it possible to pre-allocate a table with a specific size(i.e. pre-allocating the max memory that may be used by a table)?

Jonathan Goble
On Sun, Oct 11, 2020 at 6:20 AM Ranier Vilela <[hidden email]> wrote:
Em dom., 11 de out. de 2020 às 04:58, 孙世龙 sunshilong <[hidden email]> escreveu:
Hi, list

Is it possible to pre-allocate a table with a specific size(i.e.
pre-allocating the max memory that may be used by a table)?

As per the book(Programming in Lua 4th Edition, page 37), which says that:
To represent a conventional array or a list, we simply use a table
with integer keys. There is neither a way
nor a need to declare a size.
As far as I know, there's no way.
Even the Lua C API is limited by int (MAX_INT).

In the C API, there is lua_createtable [1]:

"Parameter narr is a hint for how many elements the table will have as a sequence; parameter nrec is a hint for how many other elements the table will have. Lua may use these hints to preallocate memory for the new table. This preallocation may help performance when you know in advance how many elements the table will have."

This is not exposed on the Lua side, though it would be trivial to write a small extension module to make it available.

Reply | Threaded
Open this post in threaded view
|

Re: Is it possible to pre-allocate a table with a specific size(i.e. pre-allocating the max memory that may be used by a table)?

Philippe Verdy-2
I think this should not be exposed, unless you tie your Lua program to a specific implementation and version. This is basically used only in the current implementation just to allow fast startup of the existing engine but this API should not even be considered "stable" and is only needed for internal use. Any later version of Lua could change it, providing other tuning parameters (and for now there's no official stable API to tune a specific implementation, it would require IMHO defining a new type for tuning parameters, using a set of key/value property pairs, with keys inside some namespace targeting a specific class of implementations. These will only be used as hints, and not requirements, the implementations are then free to ignore these parameters.


Le dim. 11 oct. 2020 à 17:24, Jonathan Goble <[hidden email]> a écrit :
On Sun, Oct 11, 2020 at 6:20 AM Ranier Vilela <[hidden email]> wrote:
Em dom., 11 de out. de 2020 às 04:58, 孙世龙 sunshilong <[hidden email]> escreveu:
Hi, list

Is it possible to pre-allocate a table with a specific size(i.e.
pre-allocating the max memory that may be used by a table)?

As per the book(Programming in Lua 4th Edition, page 37), which says that:
To represent a conventional array or a list, we simply use a table
with integer keys. There is neither a way
nor a need to declare a size.
As far as I know, there's no way.
Even the Lua C API is limited by int (MAX_INT).

In the C API, there is lua_createtable [1]:

"Parameter narr is a hint for how many elements the table will have as a sequence; parameter nrec is a hint for how many other elements the table will have. Lua may use these hints to preallocate memory for the new table. This preallocation may help performance when you know in advance how many elements the table will have."

This is not exposed on the Lua side, though it would be trivial to write a small extension module to make it available.

Reply | Threaded
Open this post in threaded view
|

Re: Is it possible to pre-allocate a table with a specific size(i.e. pre-allocating the max memory that may be used by a table)?

孙世龙 sunshilong
In reply to this post by Jonathan Goble
>In the C API, there is lua_createtable
>This is not exposed on the Lua side, though it would be trivial to write a small extension module to make it available.
So, it's possible to expose such a interface by ourselves? Right?

On Sun, Oct 11, 2020 at 11:24 PM Jonathan Goble <[hidden email]> wrote:

>
> On Sun, Oct 11, 2020 at 6:20 AM Ranier Vilela <[hidden email]> wrote:
>>
>> Em dom., 11 de out. de 2020 às 04:58, 孙世龙 sunshilong <[hidden email]> escreveu:
>>>
>>> Hi, list
>>>
>>> Is it possible to pre-allocate a table with a specific size(i.e.
>>> pre-allocating the max memory that may be used by a table)?
>>>
>>> As per the book(Programming in Lua 4th Edition, page 37), which says that:
>>> To represent a conventional array or a list, we simply use a table
>>> with integer keys. There is neither a way
>>> nor a need to declare a size.
>>
>> As far as I know, there's no way.
>> Even the Lua C API is limited by int (MAX_INT).
>
>
> In the C API, there is lua_createtable [1]:
>
> "Parameter narr is a hint for how many elements the table will have as a sequence; parameter nrec is a hint for how many other elements the table will have. Lua may use these hints to preallocate memory for the new table. This preallocation may help performance when you know in advance how many elements the table will have."
>
> This is not exposed on the Lua side, though it would be trivial to write a small extension module to make it available.
>
> [1] https://www.lua.org/manual/5.4/manual.html#lua_createtable
Reply | Threaded
Open this post in threaded view
|

Re: Is it possible to pre-allocate a table with a specific size(i.e. pre-allocating the max memory that may be used by a table)?

Jonathan Goble
On Sun, Oct 11, 2020 at 9:53 PM 孙世龙 sunshilong <[hidden email]> wrote:
>In the C API, there is lua_createtable
>This is not exposed on the Lua side, though it would be trivial to write a small extension module to make it available.
So, it's possible to expose such a interface by ourselves? Right?

Yes, and in fact I wrote the code for it in 5 minutes (having not touched the C API in several months), wrote a rockspec for easily building it in another 5, and tested it.

createtable.c:

#include "lua.h"
#include "lauxlib.h"

int myCreateTable(lua_State* L) {
    int narr = luaL_checkinteger(L, 1);
    int nrec = luaL_optinteger(L, 2, 0);
    lua_createtable(L, narr, nrec);
    return 1;
}

LUA_API int luaopen_createtable(lua_State* L) {
    lua_pushcfunction(L, myCreateTable);
    return 1;
}



createtable-0.1.0-1.rockspec:

package = "createtable"
version = "0.1.0-1"

source = {
    url = "none (use luarocks make)",
}

build = {
    type = "builtin",
    modules = {
        createtable = "createtable.c"
    }
}



Place both files together in an empty directory and run "luarocks make" (assuming you have LuaRocks installed and configured). Alternatively, for an embedding case you can build this into the Lua sources themselves (it should be simple to add to the standard table library).

(I release the code above into the public domain via the CC0 license: https://creativecommons.org/publicdomain/zero/1.0/)
Reply | Threaded
Open this post in threaded view
|

Re: Is it possible to pre-allocate a table with a specific size(i.e. pre-allocating the max memory that may be used by a table)?

DarkWiiPlayer
In reply to this post by 孙世龙 sunshilong
In LuaJIT you can use `table.new` to create a table with a pre-allocated
array- and hash-part. Otherwise, it's only possible using the C API, but
I already built a small module to expose that API to Lua:
https://luarocks.org/modules/darkwiiplayer/table_new

On 11/10/2020 09:58, 孙世龙 sunshilong wrote:

> Hi, list
>
> Is it possible to pre-allocate a table with a specific size(i.e.
> pre-allocating the max memory that may be used by a table)?
>
> As per the book(Programming in Lua 4th Edition, page 37), which says that:
> To represent a conventional array or a list, we simply use a table
> with integer keys. There is neither a way
> nor a need to declare a size.
>
> Could somebody shed some light on this question?
> I would be grateful to have some help with this question.
>
> Best wishes
> Sunshlong


signature.asc (849 bytes) Download Attachment