Computation error in tonumber?

classic Classic list List threaded Threaded
6 messages Options
gs
Reply | Threaded
Open this post in threaded view
|

Computation error in tonumber?

gs
I am relatively new to Lua, so please bear with me.

On Lua 5.2:
> io.write(string.format("%d\n", tonumber("5c7d528b825b090", 16)));
416535861419946112


On Lua 5.3:
>  io.write(string.format("%d\n", tonumber("5c7d528b825b090", 16)));
416535861419946128

The latter is the correct answer.

(1) Am I doing something wrong in Lua 5.2. I am relatively new to Lua, and did not see an explicit mention of this being  fixed in 5.3 , so, I am not sure if there is an actual bug in 5.2.
(2) For some reason, I can't upgrade to 5.3 (due to policies outside my control). Is there a workaround to get the right answer in 5.2?

Thanks

Reply | Threaded
Open this post in threaded view
|

Re: Computation error in tonumber?

Coda Highland
On Mon, Sep 26, 2016 at 9:04 AM, gs <[hidden email]> wrote:

> I am relatively new to Lua, so please bear with me.
>
> On Lua 5.2:
>> io.write(string.format("%d\n", tonumber("5c7d528b825b090", 16)));
> 416535861419946112
>
>
> On Lua 5.3:
>>  io.write(string.format("%d\n", tonumber("5c7d528b825b090", 16)));
> 416535861419946128
>
> The latter is the correct answer.
>
> (1) Am I doing something wrong in Lua 5.2. I am relatively new to Lua, and
> did not see an explicit mention of this being  fixed in 5.3 , so, I am not
> sure if there is an actual bug in 5.2.
> (2) For some reason, I can't upgrade to 5.3 (due to policies outside my
> control). Is there a workaround to get the right answer in 5.2?
>
> Thanks
>

Lua 5.2 doesn't have 64-bit integer support; its precision caps out at 53 bits.

/s/ Adam

Reply | Threaded
Open this post in threaded view
|

Re: Computation error in tonumber?

Nagaev Boris
In reply to this post by gs
On Mon, Sep 26, 2016 at 5:04 PM, gs <[hidden email]> wrote:

> I am relatively new to Lua, so please bear with me.
>
> On Lua 5.2:
>> io.write(string.format("%d\n", tonumber("5c7d528b825b090", 16)));
> 416535861419946112
>
>
> On Lua 5.3:
>>  io.write(string.format("%d\n", tonumber("5c7d528b825b090", 16)));
> 416535861419946128
>
> The latter is the correct answer.
>
> (1) Am I doing something wrong in Lua 5.2. I am relatively new to Lua, and
> did not see an explicit mention of this being  fixed in 5.3 , so, I am not
> sure if there is an actual bug in 5.2.
> (2) For some reason, I can't upgrade to 5.3 (due to policies outside my
> control). Is there a workaround to get the right answer in 5.2?
>
> Thanks
>

Lua 5.2 uses floating point number (C's double) for numbers.
Lua 5.3 uses 64-bit integers as well as floating point numbers.

--


Best regards,
Boris Nagaev

Reply | Threaded
Open this post in threaded view
|

Re: Computation error in tonumber?

Ką Mykolas
I wonder if there is an easy way of checking if the number does not exceed the precision cap BEFORE converting to some base...

On Mon, Sep 26, 2016 at 7:14 PM, Nagaev Boris <[hidden email]> wrote:
On Mon, Sep 26, 2016 at 5:04 PM, gs <[hidden email]> wrote:
> I am relatively new to Lua, so please bear with me.
>
> On Lua 5.2:
>> io.write(string.format("%d\n", tonumber("5c7d528b825b090", 16)));
> 416535861419946112
>
>
> On Lua 5.3:
>>  io.write(string.format("%d\n", tonumber("5c7d528b825b090", 16)));
> 416535861419946128
>
> The latter is the correct answer.
>
> (1) Am I doing something wrong in Lua 5.2. I am relatively new to Lua, and
> did not see an explicit mention of this being  fixed in 5.3 , so, I am not
> sure if there is an actual bug in 5.2.
> (2) For some reason, I can't upgrade to 5.3 (due to policies outside my
> control). Is there a workaround to get the right answer in 5.2?
>
> Thanks
>

Lua 5.2 uses floating point number (C's double) for numbers.
Lua 5.3 uses 64-bit integers as well as floating point numbers.

--


Best regards,
Boris Nagaev


Reply | Threaded
Open this post in threaded view
|

Re: Computation error in tonumber?

Martin
On 16-09-26 09:41 AM, Ką Mykolas wrote:
> I wonder if there is an easy way of checking if the number does not
> exceed the precision cap BEFORE converting to some base...
math.frexp. Second result is bits needed.

> math.frexp(tonumber("5c7d528b825b090", 16))
0.7225745373914 59

> math.frexp(tonumber("FFFF", 16))
0.99998474121094 16

In lua5.3 this function is somewhy deprecated (but still available).

Sorry for late reply.

Reply | Threaded
Open this post in threaded view
|

Re: Computation error in tonumber?

Dirk Laurie-2
2016-09-29 11:15 GMT+02:00 Martin <[hidden email]>:

> On 16-09-26 09:41 AM, Ką Mykolas wrote:
>> I wonder if there is an easy way of checking if the number does not
>> exceed the precision cap BEFORE converting to some base...
> math.frexp. Second result is bits needed.
>
>> math.frexp(tonumber("5c7d528b825b090", 16))
> 0.7225745373914 59
>
>> math.frexp(tonumber("FFFF", 16))
> 0.99998474121094        16
>
> In lua5.3 this function is somewhy deprecated (but still available).

Somewhy --> search the list archives for the phrase "conceptual
integrity". E.g.

   http://lua-users.org/lists/lua-l/2014-07/msg00387.html