LUA_COMPAT_FLOATSTRING

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

LUA_COMPAT_FLOATSTRING

Matthias Kluwe
Hi!

When converting a number object to a string, luaO_tostring gets called,
which adds a "float mark" to floating point numbers that come out of
sprint looking like an int, to the effect that "0" becomes "0.0".

This can be turned off by defining LUA_COMPAT_FLOATSTRING.
luaconf.h says: "This macro is not on by default even in compatibility
mode, because this is not really an incompatibility."

I slightly disagree, as it introduces an incompatibility when using
different locales for output.

This example (expecting decimal comma representation)

    print( os.setlocale 'german' )
    print( 0.0, 0.1 )

outputs

    German_Germany.1252
    0       0,1

using Lua 5.2.4 and

    German_Germany.1252
    0.0     0,1

with Lua 5.3.0.

I had to learn that the hard way yesterday when my output containing
(floating point) zeros got severely borked. I *know* that using
unformatted output after switching to 5.3 means asking for some trouble,
but adding hard-coded '.0' to the number does not look like a sensible
default to me.

Regards,
Matthias

Reply | Threaded
Open this post in threaded view
|

Re: LUA_COMPAT_FLOATSTRING

Nagaev Boris
On Tue, Mar 31, 2015 at 6:34 PM, Matthias Kluwe <[hidden email]> wrote:

> Hi!
>
> When converting a number object to a string, luaO_tostring gets called,
> which adds a "float mark" to floating point numbers that come out of
> sprint looking like an int, to the effect that "0" becomes "0.0".
>
> This can be turned off by defining LUA_COMPAT_FLOATSTRING.
> luaconf.h says: "This macro is not on by default even in compatibility
> mode, because this is not really an incompatibility."
>
> I slightly disagree, as it introduces an incompatibility when using
> different locales for output.
>
> This example (expecting decimal comma representation)
>
>     print( os.setlocale 'german' )
>     print( 0.0, 0.1 )
>
> outputs
>
>     German_Germany.1252
>     0       0,1
>
> using Lua 5.2.4 and
>
>     German_Germany.1252
>     0.0     0,1
>
> with Lua 5.3.0.
>
> I had to learn that the hard way yesterday when my output containing
> (floating point) zeros got severely borked. I *know* that using
> unformatted output after switching to 5.3 means asking for some trouble,
> but adding hard-coded '.0' to the number does not look like a sensible
> default to me.
>
> Regards,
> Matthias
>

Hello,

Lua 5.3 manual says following [1]:

> The conversion of a float to a string now adds a .0 suffix to the result if it looks like an integer. (For instance, the float 2.0 will be printed as 2.0, not as 2.) You should always use an explicit format when you need a specific format for numbers.
> (Formally this is not an incompatibility, because Lua does not specify how numbers are formatted as strings, but some programs assumed a specific format.)

If I understand correctly, it was not documented in Lua < 5.3, how
numbers are converted to strings. That is why this change is not
considered to be an incompatibility. When undefined behavior becomes
defined behavior, it is not called an incompatibility.

[1] http://www.lua.org/manual/5.3/manual.html#8.1

Best regards,
Boris Nagaev

Reply | Threaded
Open this post in threaded view
|

Re: LUA_COMPAT_FLOATSTRING

Coda Highland
On Tue, Mar 31, 2015 at 12:59 PM, Nagaev Boris <[hidden email]> wrote:

> On Tue, Mar 31, 2015 at 6:34 PM, Matthias Kluwe <[hidden email]> wrote:
>> Hi!
>>
>> When converting a number object to a string, luaO_tostring gets called,
>> which adds a "float mark" to floating point numbers that come out of
>> sprint looking like an int, to the effect that "0" becomes "0.0".
>>
>> This can be turned off by defining LUA_COMPAT_FLOATSTRING.
>> luaconf.h says: "This macro is not on by default even in compatibility
>> mode, because this is not really an incompatibility."
>>
>> I slightly disagree, as it introduces an incompatibility when using
>> different locales for output.
>>
>> This example (expecting decimal comma representation)
>>
>>     print( os.setlocale 'german' )
>>     print( 0.0, 0.1 )
>>
>> outputs
>>
>>     German_Germany.1252
>>     0       0,1
>>
>> using Lua 5.2.4 and
>>
>>     German_Germany.1252
>>     0.0     0,1
>>
>> with Lua 5.3.0.
>>
>> I had to learn that the hard way yesterday when my output containing
>> (floating point) zeros got severely borked. I *know* that using
>> unformatted output after switching to 5.3 means asking for some trouble,
>> but adding hard-coded '.0' to the number does not look like a sensible
>> default to me.
>>
>> Regards,
>> Matthias
>>
>
> Hello,
>
> Lua 5.3 manual says following [1]:
>
>> The conversion of a float to a string now adds a .0 suffix to the result if it looks like an integer. (For instance, the float 2.0 will be printed as 2.0, not as 2.) You should always use an explicit format when you need a specific format for numbers.
>> (Formally this is not an incompatibility, because Lua does not specify how numbers are formatted as strings, but some programs assumed a specific format.)
>
> If I understand correctly, it was not documented in Lua < 5.3, how
> numbers are converted to strings. That is why this change is not
> considered to be an incompatibility. When undefined behavior becomes
> defined behavior, it is not called an incompatibility.
>
> [1] http://www.lua.org/manual/5.3/manual.html#8.1
>
> Best regards,
> Boris Nagaev
>

It could be considered an error if this code asserts under any circumstances:

local x = 1.0
local y = tonumber(tostring(x))
assert(x == y, "roundtrip failed for forced suffix")
local x = 0.1
local y = tonumber(tostring(x))
assert(x == y, "roundtrip failed for normal suffix")

/s/ Adam

Reply | Threaded
Open this post in threaded view
|

Re: LUA_COMPAT_FLOATSTRING

Matthias Kluwe
In reply to this post by Nagaev Boris
Hi!

Nagaev Boris <bnagaev <at> gmail.com> writes:

> On Tue, Mar 31, 2015 at 6:34 PM, Matthias Kluwe <mkluwe <at> gmail.com> wrote:
> > [...]
> >
> >     print( os.setlocale 'german' )
> >     print( 0.0, 0.1 )
> >
> > outputs
> >
> >     German_Germany.1252
> >     0       0,1
> >
> > using Lua 5.2.4 and
> >
> >     German_Germany.1252
> >     0.0     0,1
> >
> > with Lua 5.3.0.
> >
> > [...]
>
> Lua 5.3 manual says following [1]:
>
> The conversion of a float to a string now adds a .0 suffix to the result
> if it looks like an integer. (For instance, the float 2.0 will be printed
> as 2.0, not as 2.) You should always use an explicit format when you need
> a specific format for numbers. (Formally this is not an incompatibility,
> because Lua does not specify how numbers are formatted as strings, but
> some programs assumed a specific format.)

Ah, yes, I must admit that I did not take the manual too literally here
skipping through it when Lua 5.3 was released some time ago. I did not
expect that ".0" was meant to be read literally irrespectice of the locale
currently in effect.

> If I understand correctly, it was not documented in Lua < 5.3, how
> numbers are converted to strings. That is why this change is not
> considered to be an incompatibility. When undefined behavior becomes
> defined behavior, it is not called an incompatibility.

I agree that the behaviour is defined now, so that's not an incompatibility,
*technically*. But the behaviour is somewhat awkward, given the problem that
relatively simple code such as

    table.concat( { 1, 0.0 }, ';' )

for generating CSV data stopped working for me.

Regards,
Matthias



Reply | Threaded
Open this post in threaded view
|

Re: LUA_COMPAT_FLOATSTRING

Matthias Kluwe
In reply to this post by Coda Highland
Hi!

Coda Highland <chighland <at> gmail.com> writes:

> On Tue, Mar 31, 2015 at 12:59 PM, Nagaev Boris <bnagaev <at> gmail.com> wrote:
> > On Tue, Mar 31, 2015 at 6:34 PM, Matthias Kluwe <mkluwe <at> gmail.com>
wrote:

> >>
> >> When converting a number object to a string, luaO_tostring gets called,
> >> which adds a "float mark" to floating point numbers that come out of
> >> sprint looking like an int, to the effect that "0" becomes "0.0".
>
> It could be considered an error if this code asserts under any circumstances:
>
> local x = 1.0
> local y = tonumber(tostring(x))
> assert(x == y, "roundtrip failed for forced suffix")

Yes, that's the problem put in a nutshell. Thank you for that, it will go
right into my set of local test cases.

Regards,
Matthias