[ANN] Lua 5.3.0 (rc1) now available

classic Classic list List threaded Threaded
47 messages Options
123
Reply | Threaded
Open this post in threaded view
|

Re: Negative table index

Tom N Harris
On Monday, December 22, 2014 01:13:12 PM Roberto Ierusalimschy wrote:
> The expression 'math.maxinteger*2+1' evaluates to -1 (by the rules
> of arithmetic overflows) before the function is called. The calls
> then become
>
>   table.move({"a","b","c"},1,3, -1,{})
>   table.move(t,1,-1, -1,{})
>

I was incorrectly thinking that table indexes were unsigned. The issue then is
overlapping copies when the sign changes. It's not that the source is
negative, but that the destination wraps around. What about this check?

    n = e - f;
    luaL_argcheck(L, t+n >= t, 4, "destination index overflows");
    for (i = 0; i <= n; i++)

(skipping the overlap check for brevity)

--
tom <[hidden email]>

Reply | Threaded
Open this post in threaded view
|

Re: Negative table index

Roberto Ierusalimschy
> [...] What about this check?
>
>     n = e - f;
>     luaL_argcheck(L, t+n >= t, 4, "destination index overflows");
>     for (i = 0; i <= n; i++)

A good compiler can throw away the test 't+n>=t': It knows that 'n'
is non negative, because of a previous 'if e >= f', and it can assume
that signed arithmetic operations do not overflow (as overflows have
undefined behavior in the standard).

(Your code also has a problem when 'n' is maxinteger, because then the
for loop will never end.)

-- Roberto

Reply | Threaded
Open this post in threaded view
|

Re: Negative table index

Tom N Harris
On Tuesday, December 23, 2014 09:14:59 AM Roberto Ierusalimschy wrote:
> A good compiler can throw away the test 't+n>=t': It knows that 'n'
> is non negative, because of a previous 'if e >= f', and it can assume
> that signed arithmetic operations do not overflow (as overflows have
> undefined behavior in the standard).
>
> (Your code also has a problem when 'n' is maxinteger, because then the
> for loop will never end.)

Overflow of signed integers remains my least favorite part of C. A reason to
abhor undefined behavior. Which is why I'm nervous about the state of negative
indexes in table library functions.

Anyway, since there won't be negative overflow the check is still just
INT_MAX - t < e - f. Unless there is some reason other than overflow of t that
needs to be considered.

--
tom <[hidden email]>

Reply | Threaded
Open this post in threaded view
|

Re: Negative table index

Roberto Ierusalimschy
> Anyway, since there won't be negative overflow the check is still just
> INT_MAX - t < e - f. Unless there is some reason other than overflow
> of t that needs to be considered.

'INT_MAX - t' will overflow for any negative 't' and therefore invalidate
the check. I am out now.

-- Roberto

Reply | Threaded
Open this post in threaded view
|

Re: Negative table index

Dirk Laurie-2
In reply to this post by Tom N Harris
2014-12-23 21:11 GMT+02:00 Tom N Harris <[hidden email]>:

> Overflow of signed integers remains my least favorite part of C.

Actually a mere counterpart of machine instructions on most CPUs.

Reply | Threaded
Open this post in threaded view
|

Re: Negative table index

Tom N Harris
In reply to this post by Roberto Ierusalimschy
On Tuesday, December 23, 2014 05:27:23 PM Roberto Ierusalimschy wrote:
> 'INT_MAX - t' will overflow for any negative 't' and therefore invalidate
> the check. I am out now.
>

I did warn you that I have difficulty with signed overflow.

The question I had was if there was a problem other than wrapping of the
destination. The answer is "yes". 'n = e - f + 1' can overflow with f<=0.
That's why checking just the destination isn't enough.

One of the things I was missing that I see now is Lua integer arithmetic is
done over unsigned integers so there's no undefined overflow. That's why
't[math.maxinteger+1] = "x"' will work. (And internally, tables store all
integers as unsigned.) But you can still run into trouble when using library
functions that add lua_Integer. 'table.move(t,1,2,math.maxinteger)' is
undefined. Would it be helpful to calculate 'n' as unsigned? Would it be much
slower or complex? I also need to refresh my memory about mixed-sign
arithmetic. (I think it promotes to unsigned.) table.unpack uses Unsigned.

tinsert:
  lua_Integer e = aux_getn(L, 1, &ta) + 1;

Is there a condition somewhere else that prevents this from overflowing? Other
than the machine would have run out of memory first. Looks like the only check
is MAXASIZE which is 1u<<number_of_bits_in_int.

I think it's unfortunate that not allowing negative indexes prevents uses that
wouldn't cause overflow. But we're living in the gray area of table arrays
that aren't strictly defined sequences. This is also something I shouldn't
have waited for the RC versions to think about.

--
tom <[hidden email]>

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Lua 5.3.0 (rc1) now available

Edward Berner
In reply to this post by Roberto Ierusalimschy
On 12/19/2014 3:37 AM, Roberto Ierusalimschy wrote:

>> (Since you asked for testing of various platforms and compilers...)
>>
>> Compiling with Open Watcom 1.9 produces some warnings.  I have not
>> investigated any of the warnings.  I didn't use the makefile, I just
>> moved luac.c out of the way and compiled *.c.  For comparison,
>> compiling Lua 5.2.3 does not report any warnings.
>>
>> $ owcc -Wall -Wextra -o lua *.c
>> lcode.c(766): Warning! W200: 'a' has been referenced but never
>> assigned a value
>> lcode.c(766): Warning! W200: 'b' has been referenced but never
>> assigned a value
>> ldo.c(709): Warning! W124: Comparison result always 0
>> lgc.c(762): Warning! W124: Comparison result always 0
>> llex.c(63): Warning! W124: Comparison result always 0
>> llex.c(176): Warning! W124: Comparison result always 0
>> lstate.c(246): Warning! W124: Comparison result always 0
>> lzio.c(73): Warning! W124: Comparison result always 0
> Thanks for the feedback.
>
> -- Roberto
>

And 5.3.0-rc2 compiles in Open Watcom with no errors or warnings. Thanks!

--
Edward Berner


123