constant table overflow

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

constant table overflow

Michael Rose
-- File: test_limited_constants.lua
--
-- This example demonstrates, that 'luajit' at the moment
-- has limited capabilities to read larger data files in
-- LUA format.
--
-- Creating the test file:
--
-- > lua[jit] test_limited_constants.lua
--
-- Creates the file 'limited_constants.lua', which is a
-- trivial data file for an 'xy' table. Just as an example.
-- If somone wants to use LUA as a DSL substitute, such
-- kind of usage seems in principal very useful, even when
-- the data is intermixed with large parts of string data or
-- numerical data. Imagine data from measurement systems
-- for example. Now the tests:
--
-- > lua5.1.4 limited_constants.lua  (n = 270000)
-- ] lua: constant table overflow
--
-- > luajit2.0.0-beta6 limited_constants.lua (n = 70000)
-- ] luajit: limited_constants.lua:70006: function at line 1 has more than 65536 constants
--
-- > lua5.2.0-alpha limited_constants.lua (n = 10000000) limited_constants.lua
-- ] 9999995    9999996
-- ] 9999996    9999997
-- ] 9999997    9999998
-- ] 9999998    9999999
-- ] 9999999    10000000
-- ] 10000000    10000001
-- OK!!! (Not testet for higher n).
--
-- The reason is given by the fact, that lua5.1.4 has an upper constraint for indices
-- into constant tables from compiled functions of 2^18-1=262143 (6 bits opcode, 8 bits
-- A-register and 18 bits for index in BC-register). lua5.2.0-alpha has introduced an
-- extra opcode which is used at places where indices into the constant table of compiled
-- functions are too high to fit. This gives room for indices up to
-- 2^26-1=67108863 (6 bits opcode, 26 bits for large index).
-- Though 'luajit' has max hashtable and array sizes of at least 2^26, all opcodes
-- for accesses to the constant table of functions are limited to 16 bit values
-- for register D in the bytecodes.

n = 10000000
f = io.open("limited_constants.lua","w")
f:write("function f()\n")
f:write("tt = {\n")
for k=1,n do
  f:write(" {x=",k,",y=",k+1,"},\n")
end
f:write("}\n")
f:write("return tt\nend\n")
f:write("tt = f()\n")
f:write("for k=",n-5,",",n," do print(tt[k].x,tt[k].y) end\n")
f:close()

-- Kind regards,
--   Michael
___________________________________________________________
Schon gehört? WEB.DE hat einen genialen Phishing-Filter in die
Toolbar eingebaut! http://produkte.web.de/go/toolbar

Reply | Threaded
Open this post in threaded view
|

Re: constant table overflow

Patrick Donnelly
Does this email have a point? Also, it's Lua not LUA [1].

[1] http://www.lua.org/about.html#name

--
- Patrick Donnelly

Reply | Threaded
Open this post in threaded view
|

Re: constant table overflow

Dimiter 'malkia' Stanev
In reply to this post by Michael Rose
Well I think these limitations are okay, if they help luajit.

In what real cases one would be using so much constants?

On 5/6/2011 9:14 AM, Michael Rose wrote:

> -- File: test_limited_constants.lua
> --
> -- This example demonstrates, that 'luajit' at the moment
> -- has limited capabilities to read larger data files in
> -- LUA format.
> --
> -- Creating the test file:
> --
> -- >  lua[jit] test_limited_constants.lua
> --
> -- Creates the file 'limited_constants.lua', which is a
> -- trivial data file for an 'xy' table. Just as an example.
> -- If somone wants to use LUA as a DSL substitute, such
> -- kind of usage seems in principal very useful, even when
> -- the data is intermixed with large parts of string data or
> -- numerical data. Imagine data from measurement systems
> -- for example. Now the tests:
> --
> -- >  lua5.1.4 limited_constants.lua  (n = 270000)
> -- ] lua: constant table overflow
> --
> -- >  luajit2.0.0-beta6 limited_constants.lua (n = 70000)
> -- ] luajit: limited_constants.lua:70006: function at line 1 has more than 65536 constants
> --
> -- >  lua5.2.0-alpha limited_constants.lua (n = 10000000) limited_constants.lua
> -- ] 9999995    9999996
> -- ] 9999996    9999997
> -- ] 9999997    9999998
> -- ] 9999998    9999999
> -- ] 9999999    10000000
> -- ] 10000000    10000001
> -- OK!!! (Not testet for higher n).
> --
> -- The reason is given by the fact, that lua5.1.4 has an upper constraint for indices
> -- into constant tables from compiled functions of 2^18-1=262143 (6 bits opcode, 8 bits
> -- A-register and 18 bits for index in BC-register). lua5.2.0-alpha has introduced an
> -- extra opcode which is used at places where indices into the constant table of compiled
> -- functions are too high to fit. This gives room for indices up to
> -- 2^26-1=67108863 (6 bits opcode, 26 bits for large index).
> -- Though 'luajit' has max hashtable and array sizes of at least 2^26, all opcodes
> -- for accesses to the constant table of functions are limited to 16 bit values
> -- for register D in the bytecodes.
>
> n = 10000000
> f = io.open("limited_constants.lua","w")
> f:write("function f()\n")
> f:write("tt = {\n")
> for k=1,n do
>    f:write(" {x=",k,",y=",k+1,"},\n")
> end
> f:write("}\n")
> f:write("return tt\nend\n")
> f:write("tt = f()\n")
> f:write("for k=",n-5,",",n," do print(tt[k].x,tt[k].y) end\n")
> f:close()
>
> -- Kind regards,
> --   Michael
> ___________________________________________________________
> Schon gehört? WEB.DE hat einen genialen Phishing-Filter in die
> Toolbar eingebaut! http://produkte.web.de/go/toolbar
>
>


Reply | Threaded
Open this post in threaded view
|

Re: constant table overflow

Miles Bader-2
"Dimiter \"malkia\" Stanev" <[hidden email]> writes:
> Well I think these limitations are okay, if they help luajit.

[Why do they "help luajit"?]

> In what real cases one would be using so much constants?

Loading large files full of data...

I've definitely run into this limitation in real life.

Lua 5.2 fixes it.

-Miles

--
「すっごい」と呟いてる。「へんてこなもんばっかり」
「そんなにへんてこ?」
「へんてこへんてこ」
そう言われると見たくなってしまう。