string format help

classic Classic list List threaded Threaded
24 messages Options
12
Reply | Threaded
Open this post in threaded view
|

string format help

James Chang
Hi Everyone,

I'm relatively new to lua, and I'm wondering if there is a fix for this on 5.2.1:

> print(string.format('%d', 2^63))
9223372036854775807 [[ = 2^63 - 1 ]]

This should throw an error instead of printing a number, right?

The catch is that I'm being stuck working with 5.2 and a big-endian processor. I'm upgrading to 5.3.0 to see if it's fixed, and I run into another error while running the testsuite:

It checks:
assert(tonumber(string.format("%A", 0x1fffff.0)) == 0X1.FFFFFP+20)

Then gives the error:
$(@D)/src/lua: strings.lua:231: malformed number near '0x1fffff.0'
stack traceback:
       [C]: in function 'dofile'
       all.lua:153: in main chunk
       [C]: in ?

Is the malformed number here supposed to fail, or am I missing something?

Thanks,

James Chang
Reply | Threaded
Open this post in threaded view
|

Re: string format help

Daurnimator
On 17 June 2015 at 08:41, James Chang <[hidden email]> wrote:
> $(@D)/src/lua: strings.lua:231: malformed number near '0x1fffff.0'
> stack traceback:
>        [C]: in function 'dofile'
>        all.lua:153: in main chunk
>        [C]: in ?
>
> Is the malformed number here supposed to fail, or am I missing something?

Sounds like your libc's `strtod` routine doesn't handle hex floating
point numbers.
What arch + distro are you using?

Reply | Threaded
Open this post in threaded view
|

Re: string format help

James Chang


On Tue, Jun 16, 2015 at 4:40 PM, Daurnimator <[hidden email]> wrote:
On 17 June 2015 at 08:41, James Chang <[hidden email]> wrote:
> $(@D)/src/lua: strings.lua:231: malformed number near '0x1fffff.0'
> stack traceback:
>        [C]: in function 'dofile'
>        all.lua:153: in main chunk
>        [C]: in ?
>
> Is the malformed number here supposed to fail, or am I missing something?

Sounds like your libc's `strtod` routine doesn't handle hex floating
point numbers.
What arch + distro are you using?

I'm using solaris, both x86 and sparc (big-endian).  

Reply | Threaded
Open this post in threaded view
|

Re: string format help

Daurnimator
On 17 June 2015 at 09:56, James Chang <[hidden email]> wrote:
> On Tue, Jun 16, 2015 at 4:40 PM, Daurnimator <[hidden email]> wrote:
>> Sounds like your libc's `strtod` routine doesn't handle hex floating
>> point numbers.
>> What arch + distro are you using?
>
>
> I'm using solaris, both x86 and sparc (big-endian).
>

I found this post which seems to confirm your issue:
http://grokbase.com/t/perl/perl5-porters/1482874b1r/perl-122219-support-hexadecimal-floats#20140805amjbq5ccdqf65fqz4qxkssysf4

> In Solaris 10, strtod must be in "c99 mode" for the hexfloats to be recognized. (strtold is always in this mode). The "c99 mode' is achieved by using "c99" as the Solaris Studio compiler (driver), instead of "cc".
> In Solaris 9 (or earlier), there is no support for hexfloats. (Not blaming Solaris in particular: I'm pretty certain many older OS releases will be similarly C99-unsupportive.)
> If one is not using Solaris Studio cc (something beginning with g, maybe), one can live dangerously and explicitly link in either of /usr/lib/{32,64}/values-xpg6.o and get the "c99 strtod". Dangerous living because probably many other things get "upgraded", too.

Reply | Threaded
Open this post in threaded view
|

Re: string format help

James Chang
On Tue, Jun 16, 2015 at 5:10 PM, Daurnimator <[hidden email]> wrote:
On 17 June 2015 at 09:56, James Chang <[hidden email]> wrote:
> On Tue, Jun 16, 2015 at 4:40 PM, Daurnimator <[hidden email]> wrote:
>> Sounds like your libc's `strtod` routine doesn't handle hex floating
>> point numbers.
>> What arch + distro are you using?
>
>
> I'm using solaris, both x86 and sparc (big-endian).
>

I found this post which seems to confirm your issue:
http://grokbase.com/t/perl/perl5-porters/1482874b1r/perl-122219-support-hexadecimal-floats#20140805amjbq5ccdqf65fqz4qxkssysf4

> In Solaris 10, strtod must be in "c99 mode" for the hexfloats to be recognized. (strtold is always in this mode). The "c99 mode' is achieved by using "c99" as the Solaris Studio compiler (driver), instead of "cc".
> In Solaris 9 (or earlier), there is no support for hexfloats. (Not blaming Solaris in particular: I'm pretty certain many older OS releases will be similarly C99-unsupportive.)
> If one is not using Solaris Studio cc (something beginning with g, maybe), one can live dangerously and explicitly link in either of /usr/lib/{32,64}/values-xpg6.o and get the "c99 strtod". Dangerous living because probably many other things get "upgraded", too.

I am using c99, but now I'm trying to upgrade to 5.3.1 and am getting:

$(@D)/src/lua: db.lua:385: assertion failed!
stack traceback:
       [C]: in function 'assert'
       db.lua:385: in hook '?'
       db.lua:377: in local 'foo'
       db.lua:392: in main chunk
       (...tail calls...)
       all.lua:154: in main chunk
       [C]: in ?
>>> closing state <<<


WEIRD

James

Reply | Threaded
Open this post in threaded view
|

Re: string format help

Roberto Ierusalimschy
> I am using c99, but now I'm trying to upgrade to 5.3.1 and am getting:
>
> $(@D)/src/lua: db.lua:385: assertion failed!
> stack traceback:
>        [C]: in function 'assert'
>        db.lua:385: in hook '?'
>        db.lua:377: in local 'foo'
>        db.lua:392: in main chunk
>        (...tail calls...)
>        all.lua:154: in main chunk
>        [C]: in ?
> >>> closing state <<<

Are you using the correct version of tests? (Tests for 5.3.1 are
different from those for 5.3.0, in particular around this assertion.)

-- Roberto

Reply | Threaded
Open this post in threaded view
|

Re: string format help

James Chang


On Wed, Jun 17, 2015 at 2:17 PM, Roberto Ierusalimschy <[hidden email]> wrote:
> I am using c99, but now I'm trying to upgrade to 5.3.1 and am getting:
>
> $(@D)/src/lua: db.lua:385: assertion failed!
> stack traceback:
>        [C]: in function 'assert'
>        db.lua:385: in hook '?'
>        db.lua:377: in local 'foo'
>        db.lua:392: in main chunk
>        (...tail calls...)
>        all.lua:154: in main chunk
>        [C]: in ?
> >>> closing state <<<

Are you using the correct version of tests? (Tests for 5.3.1 are
different from those for 5.3.0, in particular around this assertion.)

I'm using 5.3.1.
 
-- Roberto


Reply | Threaded
Open this post in threaded view
|

Re: string format help

Roberto Ierusalimschy
In reply to this post by Roberto Ierusalimschy
> > I am using c99, but now I'm trying to upgrade to 5.3.1 and am getting:
> >
> > $(@D)/src/lua: db.lua:385: assertion failed!
> > stack traceback:
> >        [C]: in function 'assert'
> >        db.lua:385: in hook '?'
> >        db.lua:377: in local 'foo'
> >        db.lua:392: in main chunk
> >        (...tail calls...)
> >        all.lua:154: in main chunk
> >        [C]: in ?
> > >>> closing state <<<
>
> Are you using the correct version of tests? (Tests for 5.3.1 are
> different from those for 5.3.0, in particular around this assertion.)

Forget it. (The difference is that tests for 5.3.0 do not have that
assertion :-)

-- Roberto

Reply | Threaded
Open this post in threaded view
|

Re: string format help

James Chang
In reply to this post by James Chang


On Tue, Jun 16, 2015 at 3:41 PM, James Chang <[hidden email]> wrote:
Hi Everyone,

I'm relatively new to lua, and I'm wondering if there is a fix for this on 5.2.1:

> print(string.format('%d', 2^63))
9223372036854775807 [[ = 2^63 - 1 ]]

Anybody have a clue why this happens? Doesn't seem to be string.format related.
 

This should throw an error instead of printing a number, right?

The catch is that I'm being stuck working with 5.2 and a big-endian processor. I'm upgrading to 5.3.0 to see if it's fixed, and I run into another error while running the testsuite:

It checks:
assert(tonumber(string.format("%A", 0x1fffff.0)) == 0X1.FFFFFP+20)

Then gives the error:
$(@D)/src/lua: strings.lua:231: malformed number near '0x1fffff.0'
stack traceback:
       [C]: in function 'dofile'
       all.lua:153: in main chunk
       [C]: in ?

Is the malformed number here supposed to fail, or am I missing something?

Thanks,

James Chang

Reply | Threaded
Open this post in threaded view
|

Re: string format help

Javier Guerra Giraldez
On Thu, Jun 18, 2015 at 12:20 PM, James Chang <[hidden email]> wrote:
>>
>>
>> > print(string.format('%d', 2^63))
>> 9223372036854775807 [[ = 2^63 - 1 ]]


previously to Lua 5.3, all numbers in Lua were stored and managed as
double float quantities (unless you tweak and compile Lua yourself).
that means scientific format numbers: a mantissa and an exponent.
double floats have 52 bit mantissas, meaning that you only get integer
precision up to 2^52.  beyond that, you can still express big
quantities, but you lose precision.

--
Javier

Reply | Threaded
Open this post in threaded view
|

Re: string format help

Luiz Henrique de Figueiredo
In reply to this post by James Chang
> > I'm relatively new to lua, and I'm wondering if there is a fix for this on
> > 5.2.1:
> >
> > > print(string.format('%d', 2^63))
> > 9223372036854775807 [[ = 2^63 - 1 ]]
> >
>
> Anybody have a clue why this happens? Doesn't seem to be string.format
> related.

One fix is use Lua 5.2.2 or later, which at least complains:

   Lua 5.2.4  Copyright (C) 1994-2015 Lua.org, PUC-Rio
   > print(string.format('%d', 2^63))
   stdin:1: bad argument #2 to 'format' (not a number in proper range)

Reply | Threaded
Open this post in threaded view
|

Re: string format help

James Chang


On Thu, Jun 18, 2015 at 10:32 AM, Luiz Henrique de Figueiredo <[hidden email]> wrote:
> > I'm relatively new to lua, and I'm wondering if there is a fix for this on
> > 5.2.1:
> >
> > > print(string.format('%d', 2^63))
> > 9223372036854775807 [[ = 2^63 - 1 ]]
> >
>
> Anybody have a clue why this happens? Doesn't seem to be string.format
> related.

One fix is use Lua 5.2.2 or later, which at least complains:

   Lua 5.2.4  Copyright (C) 1994-2015 Lua.org, PUC-Rio
   > print(string.format('%d', 2^63))
   stdin:1: bad argument #2 to 'format' (not a number in proper range)

It already does that on x86 on 5.2.1, but not on sparc. Both are running solaris.
Reply | Threaded
Open this post in threaded view
|

Re: string format help

Roberto Ierusalimschy
> > One fix is use Lua 5.2.2 or later, which at least complains:
> >
> >    Lua 5.2.4  Copyright (C) 1994-2015 Lua.org, PUC-Rio
> >    > print(string.format('%d', 2^63))
> >    stdin:1: bad argument #2 to 'format' (not a number in proper range)
> >
> > It already does that on x86 on 5.2.1, but not on sparc. Both are running
> solaris.

The check relies on non-standard C. It converts the double to long long
and then converts ib back to double, seeing whether the result is equal
the original. If so, it assumes that long long can safely represent
the number. C gives no garanties about what happens when it converts
an out-of-bound double (e.g., 2^63) to long long. Probably that is the
reason why it works on x86 but not on Sparc.

-- Roberto

Reply | Threaded
Open this post in threaded view
|

Re: string format help

James Chang
What regressions are the developers trying to catch with this test?

assert(not pcall(string.format, "%d", 2^63))

-- James

On Thu, Jun 18, 2015 at 12:17 PM, Roberto Ierusalimschy <[hidden email]> wrote:
> > One fix is use Lua 5.2.2 or later, which at least complains:
> >
> >    Lua 5.2.4  Copyright (C) 1994-2015 Lua.org, PUC-Rio
> >    > print(string.format('%d', 2^63))
> >    stdin:1: bad argument #2 to 'format' (not a number in proper range)
> >
> > It already does that on x86 on 5.2.1, but not on sparc. Both are running
> solaris.

The check relies on non-standard C. It converts the double to long long
and then converts ib back to double, seeing whether the result is equal
the original. If so, it assumes that long long can safely represent
the number. C gives no garanties about what happens when it converts
an out-of-bound double (e.g., 2^63) to long long. Probably that is the
reason why it works on x86 but not on Sparc.

-- Roberto


Reply | Threaded
Open this post in threaded view
|

Re: string format help

Philipp Janda
Am 26.06.2015 um 01:04 schröbte James Chang:
> What regressions are the developers trying to catch with this test?
>
> assert(not pcall(string.format, "%d", 2^63))

A signed 64-bit integer usually can represent numbers from -2^63 up to
+2^63-1. 2^63 is just out of range.

>
> -- James

Philipp


>
> On Thu, Jun 18, 2015 at 12:17 PM, Roberto Ierusalimschy <
> [hidden email]> wrote:
>
>>>> One fix is use Lua 5.2.2 or later, which at least complains:
>>>>
>>>>     Lua 5.2.4  Copyright (C) 1994-2015 Lua.org, PUC-Rio
>>>>     > print(string.format('%d', 2^63))
>>>>     stdin:1: bad argument #2 to 'format' (not a number in proper range)
>>>>
>>>> It already does that on x86 on 5.2.1, but not on sparc. Both are
>> running
>>> solaris.
>>
>> The check relies on non-standard C. It converts the double to long long
>> and then converts ib back to double, seeing whether the result is equal
>> the original. If so, it assumes that long long can safely represent
>> the number. C gives no garanties about what happens when it converts
>> an out-of-bound double (e.g., 2^63) to long long. Probably that is the
>> reason why it works on x86 but not on Sparc.
>>
>> -- Roberto
>>
>>
>



Reply | Threaded
Open this post in threaded view
|

Re: string format help

James Chang


On Thu, Jun 25, 2015 at 4:31 PM, Philipp Janda <[hidden email]> wrote:
Am 26.06.2015 um 01:04 schröbte James Chang:
What regressions are the developers trying to catch with this test?

assert(not pcall(string.format, "%d", 2^63))

A signed 64-bit integer usually can represent numbers from -2^63 up to +2^63-1. 2^63 is just out of range.

Correct, but this is more of a meta(?) question. Why include this if lua is intended for x86 chipsets if the behavior for signed long long is always this way? 


-- James

Philipp




On Thu, Jun 18, 2015 at 12:17 PM, Roberto Ierusalimschy <
[hidden email]> wrote:

One fix is use Lua 5.2.2 or later, which at least complains:

    Lua 5.2.4  Copyright (C) 1994-2015 Lua.org, PUC-Rio
    > print(string.format('%d', 2^63))
    stdin:1: bad argument #2 to 'format' (not a number in proper range)

It already does that on x86 on 5.2.1, but not on sparc. Both are
running
solaris.

The check relies on non-standard C. It converts the double to long long
and then converts ib back to double, seeing whether the result is equal
the original. If so, it assumes that long long can safely represent
the number. C gives no garanties about what happens when it converts
an out-of-bound double (e.g., 2^63) to long long. Probably that is the
reason why it works on x86 but not on Sparc.

-- Roberto







Reply | Threaded
Open this post in threaded view
|

Re: string format help

Daurnimator
On 26 June 2015 at 10:22, James Chang <[hidden email]> wrote:

>
>
> On Thu, Jun 25, 2015 at 4:31 PM, Philipp Janda <[hidden email]> wrote:
>>
>> Am 26.06.2015 um 01:04 schröbte James Chang:
>>>
>>> What regressions are the developers trying to catch with this test?
>>>
>>> assert(not pcall(string.format, "%d", 2^63))
>>
>>
>> A signed 64-bit integer usually can represent numbers from -2^63 up to
>> +2^63-1. 2^63 is just out of range.
>
>
> Correct, but this is more of a meta(?) question. Why include this if lua is
> intended for x86 chipsets if the behavior for signed long long is always
> this way?

The test is to make sure that an error is thrown. i.e.

> string.format("%d", 2^63)
stdin:1: bad argument #2 to 'format' (number has no integer representation)
stack traceback:
    [C]: in function 'string.format'
    stdin:1: in main chunk
    [C]: in ?

Reply | Threaded
Open this post in threaded view
|

Re: string format help

Roberto Ierusalimschy
In reply to this post by James Chang
> Correct, but this is more of a meta(?) question. Why include this if lua is
> intended for x86 chipsets if the behavior for signed long long is always
> this way?

What do you mean by "Lua is intended for x86 chipsets"?

-- Roberto

Reply | Threaded
Open this post in threaded view
|

Re: string format help

James Chang
Apologies, let me reword that. In my limited exposure to lua, it seems that most people use it on intel chips. Sparc and solaris seem to round differently from x86 at least, which is what produces the discrepancy between these two test results. Pedantic question: Is there anything else we can conclude from this test working/breaking other than "okay you can't use numbers that are out of range"? I am pretty sure it was not written with sparc/solaris in mind.

-- James

On Fri, Jun 26, 2015 at 5:17 AM, Roberto Ierusalimschy <[hidden email]> wrote:
> Correct, but this is more of a meta(?) question. Why include this if lua is
> intended for x86 chipsets if the behavior for signed long long is always
> this way?

What do you mean by "Lua is intended for x86 chipsets"?

-- Roberto


Reply | Threaded
Open this post in threaded view
|

Re: string format help

Coda Highland
On Fri, Jun 26, 2015 at 9:39 AM, James Chang <[hidden email]> wrote:
> Apologies, let me reword that. In my limited exposure to lua, it seems that
> most people use it on intel chips. Sparc and solaris seem to round
> differently from x86 at least, which is what produces the discrepancy
> between these two test results. Pedantic question: Is there anything else we
> can conclude from this test working/breaking other than "okay you can't use
> numbers that are out of range"? I am pretty sure it was not written with
> sparc/solaris in mind.
>
> -- James

I feel reasonably certain that this is exactly the conclusion that
this test is supposed to draw. Why would you expect more?

/s/ Adam

P.S. Don't top-post; put your comments at the bottom of your e-mail replies.

12