Bugs and questions

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Bugs and questions

Mauro Vezzosi
Hi all

Introduction:
I'm using Lua as a stand-alone language for the MSDOS and the Nokia 9110
Communicator (a GSM phone with PDA functions; its operating system runs 
on top of Embedded DOS-ROM which is mostly compatible with MS-DOS). For 
this reason, I compile Lua for a 16 bit environment and my compiler 
consider the Int C type composed by 16 bits.

Bugs and notes:
1) In lmem.h the macros
#define luaM_newvector(n,t)      ((t *)luaM_malloc((n)*sizeof(t)))
#define luaM_reallocvector(v,n,t)     ((v)=(t *)luaM_realloc(v,(n)
*sizeof(t)))
can pass a wrong size value to luaM_realloc() if Int is composed by 16 
bits and "n" is big enough.
For example, this program hangs at the 2136th loop, when Lua grows the
memory allocated for the table "t":
t = { }
n = 0
while 1 do
        t[n] = n
        n = n + 1
        print(n)
end

IMO the macros should contains a cast to unsigned long:
#define luaM_newvector(n,t)      ((t *)luaM_malloc((unsigned
long)(n)*sizeof(t)))
#define luaM_reallocvector(v,n,t)     ((v)=(t *)luaM_realloc(v,(unsigned
long)(n)*sizeof(t)))

I have't made a complete search, but I think that the following 
functions can have the same 16 bit related problem. Please note that I 
just had a look at this source codes without study them in depth 
(particularly for the 4.0), so could be wrong:

- lbuffer.c->openspace(): if (L->Mbuffnext+(size) > L->Mbuffsize) ...
- lbuffer.c->Openspace():
  - 3.2: size += EXTRABUFF;
    For example:
    a = strrep("A", 16380)
    b = strrep("B", 16380)
    c = a .. b
    gives correctly "lua error: internal array bigger than `int' limit
" because in lmem.c->luaM_growaux() the lines
   unsigned long newn = nelems+inc;
   if (newn >= limit) lua_error(errormsg);
   have the values of
   unsigned long (4294934552) = (0)+(-32744);
   if ((4294934552) >= (32765)) lua_error(errormsg);
   ("inc" has the value of Openspace()'s "size")
   Is it the right behaviour or only fortune ?
  - 4.0: L->Mbuffsize = (L->Mbuffnext+size+EXTRABUFF)*2;
2 e 4.0 (should be very rare):
  Closure *c = (Closure*)luaM_malloc(sizeof(Closure)+nelems*sizeof
(TObject));
- lundump.c->LoadCode():
  - 3.2: Byte* b=luaM_malloc(size+PAD);
  - 4.0 (I don't know if this line can gives problems or not):
    LoadBlock(L,tf->code,size*sizeof(*tf->code),Z);

2) Lua 4.0:
  - lparser.c->listfields(): checklimit(ls, n, 
MAXARG_A*LFIELDS_PER_FLUSH, "...") gives "'*' : integral constant 
overflow; result truncated" on my 16 bit compiler, because MAXARG_A is 
2^17-1 and LFIELDS_PER_FLUSH is 64 and their product is greater than 
32767.
  - lbuiltin.c->luaB_predefine() with the macro DEBUG defined:
luaB_opentests() gives "unresolved external" (there is also the 
function's prototype at line 44). IIRC, this warning was already posted 
but related to luac and without DEBUG.
  - In manual.html at "8 - Lua Stand-alone" is missing the explanation 
of the -s argument.

3) In interactive mode, after entering the line (for examples)
   a, b = read("*n", "*n")
   Lua displays 2 prompts ("> >").

Questions:
1) Sometimes I need to return the elements of a table as single 
variables. For examples:
function read(...)
        dosomething()
        local retlist = call(%read, arg, "p")
        dosomethingelse()
        return tunpack(retlist)  [see forward for tunpack()]
end
a, b = read("*n", "*n")

To do this, I wrote this function:
function tunpack(tbl)
        if (tbl == nil) or (tbl.n == nil) or (tbl.n == 0) then return 
end
        local tbl1 = tbl[1]
        tremove(tbl, 1)
        return tbl1, tunpack(tbl)
end

Is there a better way to do this ? (I would like to avoid the 
recursion's overhead and to destroy the table)

2) At the moment, I also use tunpack() to return a variable number of
values. For example:
function example(flags)
        local ret = { }
        if strfind(flags, "A") then tinsert(ret, dosomething()) end
        if strfind(flags, "B") then tinsert(ret, dosomethingelse()) end
        ...
        return tunpack(ret)
end
a, b


Is there a better way to do this instead of using a table ?

3) Is it possible to avoid the assignment of some arguments in the "="
operator ? For example something like:
nil, a, nil, b = dosomething()
, a, , b = dosomething()

I could write
a, a, a, b = dosomething()
but this involves always 4 assignments even if I need only the last one.


I still have a couple of questions about memory management to reduce and
recover the memory allocation but I don't want to bother you too much 
with this email.

Sorry for this long email and for my wrong English.

TIA
Best regards

Mauro


Reply | Threaded
Open this post in threaded view
|

Re: Bugs and questions

Luiz Henrique de Figueiredo
>From: "mvezzosi@libero."<[hidden email]>

>I compile Lua for a 16 bit environment and my compiler 
>consider the Int C type composed by 16 bits.

We tried to avoid any potential problems for 16-bit systems, but we do not
have a real 16 bit environment to test it. So, thanks for the comments.
You'll see what can be done.

>1) In lmem.h the macros
>#define luaM_newvector(n,t)      ((t *)luaM_malloc((n)*sizeof(t)))
>#define luaM_reallocvector(v,n,t)     ((v)=(t *)luaM_realloc(v,(n)
>*sizeof(t)))
>can pass a wrong size value to luaM_realloc() if Int is composed by 16 
>bits and "n" is big enough.

luaM_malloc is a macro that calls luaM_realloc, whose declaration is

	void *luaM_realloc (void *block, unsigned long size);

So, there should be no problems, unless sizeof returns int.

>IMO the macros should contains a cast to unsigned long:

This makes sense, but malloc is defined as

	void *malloc(size_t size);

and so if sizeof returns int then casting to unsigned long wouldn't help.
Perhaps a cast to size_t works.
Could you try this on your example?

>  - lbuiltin.c->luaB_predefine() with the macro DEBUG defined:
>luaB_opentests() gives "unresolved external" (there is also the 
>function's prototype at line 44).

luaB_opentests is only used here, for testing.
But we are considering distributing it too.

>IIRC, this warning was already posted but related to luac and without DEBUG.

I don't recall this.

>3) In interactive mode, after entering the line (for examples)
>   a, b = read("*n", "*n")
>   Lua displays 2 prompts ("> >").

This is ok. The problem is that the lua interpreter is reading from stdin too,
and so is "read", of course. So, they are competing for the same buffer and
messing things.  But I don't know why it displays 2 prompts.

>1) Sometimes I need to return the elements of a table as single 
>variables. For examples:
>function read(...)
>        dosomething()
>        local retlist = call(%read, arg, "p")
>        dosomethingelse()
>        return tunpack(retlist)  [see forward for tunpack()]
>end
>a, b = read("*n", "*n")

"read" already does that. I guess your point it the dosomething() and
dosomethingelse(). I hope that dosomethingelse() uses retlist :-)

>Is there a better way to do this ? (I would like to avoid the 
>recursion's overhead and to destroy the table)

Not that I know. :-(

>3) Is it possible to avoid the assignment of some arguments in the "="
>operator ?

No. The way we usually do this in our own programs is by using "_":

local _,a,_,b = dosomething()

Usually, dosomething is strfind :-)

--lhf

Reply | Threaded
Open this post in threaded view
|

Re: Bugs and questions

Roberto Ierusalimschy
In reply to this post by Mauro Vezzosi
> Is there a better way to do this ? (I would like to avoid the 
> recursion's overhead and to destroy the table)

I don't know how to avoid the recursion, but you can keep the table intact:

  function unpack (t, i)
    i = i or 1
    if (i <= getn(t)) then
      return t[i], unpack(t, i+1)
    end
  end

-- Roberto

Reply | Threaded
Open this post in threaded view
|

Re: Bugs and questions

Alan Watson-2
In reply to this post by Luiz Henrique de Figueiredo
> So, there should be no problems, unless sizeof returns int.

I should not be. The type of the integer constant formed by
the sizeof operator is size_t, and size_t must be an
unsigned integral type (sections 6.3.3.4 and 7.1.6 of the
ISO standard).

Regards,

Alan
-- 
Dr Alan Watson
Instituto de Astronomía UNAM

Reply | Threaded
Open this post in threaded view
|

RE: Bugs and questions

Luiz Carlos de Castro Silveira Filho
In reply to this post by Roberto Ierusalimschy
try this example.
i don?t know if concatenating all these strings is faster than recursion, but...

tbl = {'param1', 'param2', 'param3', 'param4', 'param5'}

_unpack = nil
function unpack(t)
local unpack_str

	for i=1,getn(t) do
		if (unpack_str) then
			unpack_str = unpack_str..", _unpack["..i.."]"
		else
			unpack_str = "_unpack["..i.."]"
		end
	end
	_unpack = t
	return dostring("return "..unpack_str)
end



a, b, c, d, e = unpack(tbl)
print(a, b, c, d, e)

-----Original Message-----
From: [hidden email] [[hidden email]
Behalf Of Roberto Ierusalimschy
Sent: Friday, May 12, 2000 3:13 PM
To: Multiple recipients of list
Subject: Re: Bugs and questions 


> Is there a better way to do this ? (I would like to avoid the 
> recursion's overhead and to destroy the table)

I don't know how to avoid the recursion, but you can keep the table intact:

  function unpack (t, i)
    i = i or 1
    if (i <= getn(t)) then
      return t[i], unpack(t, i+1)
    end
  end

-- Roberto

Reply | Threaded
Open this post in threaded view
|

Re: Bugs and questions

Luiz Henrique de Figueiredo
In reply to this post by Mauro Vezzosi
>From: Alan Watson <[hidden email]>
>
lhf> So, there should be no problems, unless sizeof returns int.
>
>I should not be. The type of the integer constant formed by
>the sizeof operator is size_t, and size_t must be an
>unsigned integral type (sections 6.3.3.4 and 7.1.6 of the
>ISO standard).

I meant, unless sizeof(size_t)=2.
--lhf

Reply | Threaded
Open this post in threaded view
|

Re: Bugs and questions

Alan Watson-2
> I meant, unless sizeof(size_t)=2.

Okay.

However, unless I'm misunderstanding something, there's not
much you can do about the original problem, as if size_t is
only 16 bits then you can't grow a table to be more than
64k. There are similar limits for when size_t is 32 bits,
but I don't think many people run into 4 GB tables.

The only solution I see is keeping track of sizes internally
as unsigned longs and, if need be, allocating tables in
multiple chunks. Again, this would only delay the problem
until the size of the table reached ULONG_MAX.

Regards,

Alan
-- 
Dr Alan Watson
Instituto de Astronomía UNAM

Reply | Threaded
Open this post in threaded view
|

Re: Bugs and questions

Luiz Henrique de Figueiredo
In reply to this post by Mauro Vezzosi
>However, unless I'm misunderstanding something, there's not
>much you can do about the original problem, as if size_t is
>only 16 bits then you can't grow a table to be more than
>64k. There are similar limits for when size_t is 32 bits,
>but I don't think many people run into 4 GB tables.
>
>The only solution I see is keeping track of sizes internally
>as unsigned longs and, if need be, allocating tables in
>multiple chunks. Again, this would only delay the problem

Yes, I think this sums it up very nicely. Thanks.
16-bit systems have memory allocation restrictions and that's it.
--lhf