Memory allocation of arrays

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Memory allocation of arrays

Rodrigo Azevedo
Dear,

The memory allocation of "array" part of tables increases by a factor
of 2, as we can see from the program below

mem = function() return collectgarbage"count"/1048576 end -- Gbyte
t = {}
c = mem()
print("MEM",c)
count = 1
while c < 4 do
  t[count] = c
  c = mem()
  if c > 1.5*t[#t] then print(c) end
  count = count + 1
end
print("MEM",mem())

but this is very annoying when we are using many Lua states trying to
use each byte of many GB with very huge tables (many GB) concurrently.

But the code

f = function(n) return string.format("%d",n) end
t = nil collectgarbage()
t = {}
c = mem()
print("MEM",c)
count = 1
while c < 4 do
  t[f(count)] = c
  c = mem()
  if c > 1.5*t[f(count)] then print(c) end
  count = count + 1
end
print("MEM",mem())

does not have this annoying behavior, showing a more straight memory allocation.

Then, this seems to be an "implementation detail" that I would want to avoid.

Is this possible? How?

Thanks!
--
Rodrigo Azevedo Moreira da Silva

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Memory allocation of arrays

Duncan Cross
On Fri, Jul 21, 2017 at 12:21 PM, Rodrigo Azevedo <[hidden email]> wrote:
> Then, this seems to be an "implementation detail" that I would want to avoid.
>
> Is this possible? How?

The C API has a lua_createtable() function [1] allows you to specify
the size of the array part on table creation using the "narr"
parameter. This functionality isn't normally exposed to the Lua side,
and it does mean you have to know the exact length you want before you
begin populating the table, but if those are not problems for you then
this looks like your best opportunity.

[1] https://www.lua.org/manual/5.3/manual.html#lua_createtable

-Duncan

Loading...