Missing feature in 5.3: LUA_FLOAT_NONE for crippled C libraries

classic Classic list List threaded Threaded
17 messages Options
Reply | Threaded
Open this post in threaded view
|

Missing feature in 5.3: LUA_FLOAT_NONE for crippled C libraries

Real Time Logic
Although Lua 5.3 strictly follows C89 and C99, many embedded environments come with crippled C libraries that make it impossible to use floating point operations. Changing the number type from double to "integer only" was easy in previous versions of Lua since only the configuration file needed to be changed. This no longer seems to be the case since the C code directly uses many floating point library functions and macro constants(C89/C99). We have tried to port Lua 5.3.1 to some of these embedded environments, and we have run into major issues that cannot be resolved without major changes to the C code. I therefore propose that you add an option (which can be set via the configuration file) that removes any use of floating point from Lua.

Examples:

Compiling for Cortex M7 using the IAR compiler worked (C99 compatible floating point library).
Compiling for ColdFire using the CodeWarrior compiler failed (crippled C89 floating point library).

In addition to issues with exotic compilers/libraries, some Real Time Operating Systems do not save the floating point registers on context switches, which makes it impossible to use floating point with CPU's that include hardware floating point operations. In others, one must specifically enable it. For example, in VxWorks, floating point registers are not saved unless a specific option is set when the thread is created.
Reply | Threaded
Open this post in threaded view
|

Re: Missing feature in 5.3: LUA_FLOAT_NONE for crippled C libraries

Lourival Vieira Neto
On Thu, Oct 15, 2015 at 2:23 PM, Real Time <[hidden email]> wrote:
>I therefore propose that you add an option (which can be set via the
> configuration file) that removes any use of floating point from Lua.

You might want to take a look at [1]. However, I still need to fix few
bugs found by Guilherme Salazar during this year GSoC and update to
5.3.1.

[1] https://github.com/lneto/lua/tree/no-float
--
Lourival Vieira Neto

Reply | Threaded
Open this post in threaded view
|

Re: Missing feature in 5.3: LUA_FLOAT_NONE for crippled C libraries

Real Time Logic

Thanks you for this information, however, I do not think this is an option for various reasons. Such a fundamental feature should be part of the main branch.

Roberto, I hope you add "integer only" as a compile time option in the next version.

-Wilfred




You might want to take a look at [1]. However, I still need to fix few
bugs found by Guilherme Salazar during this year GSoC and update to
5.3.1.

[1] https://github.com/lneto/lua/tree/no-float
--
Lourival Vieira Neto


Reply | Threaded
Open this post in threaded view
|

Re: Missing feature in 5.3: LUA_FLOAT_NONE for crippled C libraries

Andrew Cagney
In reply to this post by Real Time Logic
To support your argument, NetBSD seems to have hacked their version
(used #ifdef _KERNEL) so that float isn't required.

On 15 October 2015 at 13:23, Real Time <[hidden email]> wrote:
> In addition to issues with exotic compilers/libraries, some Real Time
> Operating Systems do not save the floating point registers on context
> switches, which makes it impossible to use floating point with CPU's that
> include hardware floating point operations. In others, one must specifically
> enable it. For example, in VxWorks, floating point registers are not saved
> unless a specific option is set when the thread is created.

Reply | Threaded
Open this post in threaded view
|

Re: Missing feature in 5.3: LUA_FLOAT_NONE for crippled C libraries

Marc Balmer
that's true, but not because the compiler is crippled, rather because there is no floating point math available in the kernel in general.

> Am 17.10.2015 um 15:44 schrieb Andrew Cagney <[hidden email]>:
>
> To support your argument, NetBSD seems to have hacked their version
> (used #ifdef _KERNEL) so that float isn't required.
>
>> On 15 October 2015 at 13:23, Real Time <[hidden email]> wrote:
>> In addition to issues with exotic compilers/libraries, some Real Time
>> Operating Systems do not save the floating point registers on context
>> switches, which makes it impossible to use floating point with CPU's that
>> include hardware floating point operations. In others, one must specifically
>> enable it. For example, in VxWorks, floating point registers are not saved
>> unless a specific option is set when the thread is created.
>

Reply | Threaded
Open this post in threaded view
|

Re: Missing feature in 5.3: LUA_FLOAT_NONE for crippled C libraries

Lourival Vieira Neto
In reply to this post by Andrew Cagney
On Sat, Oct 17, 2015 at 10:44 AM, Andrew Cagney <[hidden email]> wrote:
> To support your argument, NetBSD seems to have hacked their version
> (used #ifdef _KERNEL) so that float isn't required.

Pretty much the same patch posted before =p. It just hasn't
kernel-specific stuff. And I don't think it supports that Lua *must*
have this "feature". (It would be good though.)

--
Lourival Vieira Neto

Reply | Threaded
Open this post in threaded view
|

Re: Missing feature in 5.3: LUA_FLOAT_NONE for crippled C libraries

Lourival Vieira Neto
In reply to this post by Real Time Logic
On Fri, Oct 16, 2015 at 4:43 PM, Real Time <[hidden email]> wrote:
> Thanks you for this information, however, I do not think this is an option
> for various reasons. Such a fundamental feature should be part of the main
> branch.
>
> Roberto, I hope you add "integer only" as a compile time option in the next
> version.

Many software distributions modify Lua (or any other free program), I
can't see why this is not an option. IMHO, demanding *your* own
requirements to be part of a *third-party* software that is not an
option.

--
Lourival Vieira Neto

Reply | Threaded
Open this post in threaded view
|

Re: Missing feature in 5.3: LUA_FLOAT_NONE for crippled C libraries

Luiz Henrique de Figueiredo
In reply to this post by Lourival Vieira Neto
> You might want to take a look at [1]. However, I still need to fix few
> bugs found by Guilherme Salazar during this year GSoC and update to
> 5.3.1.
>
> [1] https://github.com/lneto/lua/tree/no-float

This is extensive surgery, not particularly hard to do but boring and
error prone to do for each Lua release.

I wondered how far one would go by simply adding
        #define double long
at the top of luaconf.h and fixing a few library calls...

I did just that for Lua 5.3.1 and got just a few compilation errors,
mostly about LUA_NUMBER_FMT being wrong for long and for implicit
conversion of double values to long in lmathlib.c (which would really
need to be changed).

Bottom line: it seems to be straightforward to add support for
LUA_FLOAT_NONE in luaconf.h.

What am I missing?

Reply | Threaded
Open this post in threaded view
|

Re: Missing feature in 5.3: LUA_FLOAT_NONE for crippled C libraries

Real Time Logic
The problem is that many (standard) floating point functions are used in the C code. These function calls cannot be removed by modifying the configuration file. Also, many math constants are used, which may not be defined in the math library being used. The "fixing a few library calls..." is not that easy and as you say, it's error prone.

IMHO, the best solution would be to have support for "integer only" as an option in the main Lua branch by for example using ifdefs in the C code.

-Wilfred




This is extensive surgery, not particularly hard to do but boring and
error prone to do for each Lua release.

I wondered how far one would go by simply adding
        #define double long
at the top of luaconf.h and fixing a few library calls...

I did just that for Lua 5.3.1 and got just a few compilation errors,
mostly about LUA_NUMBER_FMT being wrong for long and for implicit
conversion of double values to long in lmathlib.c (which would really
need to be changed).

Bottom line: it seems to be straightforward to add support for
LUA_FLOAT_NONE in luaconf.h.

What am I missing?


Reply | Threaded
Open this post in threaded view
|

Re: Missing feature in 5.3: LUA_FLOAT_NONE for crippled C libraries

Lourival Vieira Neto
In reply to this post by Luiz Henrique de Figueiredo
On Tue, Oct 20, 2015 at 3:09 PM, Luiz Henrique de Figueiredo
<[hidden email]> wrote:
>> You might want to take a look at [1]. However, I still need to fix few
>> bugs found by Guilherme Salazar during this year GSoC and update to
>> 5.3.1.
>>
>> [1] https://github.com/lneto/lua/tree/no-float
>
> This is extensive surgery, not particularly hard to do but boring and
> error prone to do for each Lua release.

Indeed.

> I wondered how far one would go by simply adding
>         #define double long
> at the top of luaconf.h and fixing a few library calls...
>
> I did just that for Lua 5.3.1 and got just a few compilation errors,
> mostly about LUA_NUMBER_FMT being wrong for long and for implicit
> conversion of double values to long in lmathlib.c (which would really
> need to be changed).
>
> Bottom line: it seems to be straightforward to add support for
> LUA_FLOAT_NONE in luaconf.h.
>
> What am I missing?

The main issue for me is that the subtype "float" is not completely
removed using this approach. For example, "1" + 0 results in float
subtype and 1 / 0 results in SIGFLOAT.
--
Lourival Vieira Neto

Reply | Threaded
Open this post in threaded view
|

Re: Missing feature in 5.3: LUA_FLOAT_NONE for crippled C libraries

Roberto Ierusalimschy
In reply to this post by Real Time Logic
> The problem is that many (standard) floating point functions are used in
> the C code. These function calls cannot be removed by modifying the
> configuration file. Also, many math constants are used, which may not be
> defined in the math library being used. The "fixing a few library calls..."
> is not that easy and as you say, it's error prone.

It would be more useful if you give concrete data, instead of generic
claims. Which/where floating point functions are used in the code? Which
constants? Why they are difficult to be corrected through the
configuration file?


> IMHO, the best solution would be to have support for "integer only" as an
> option in the main Lua branch by for example using ifdefs in the C code.

Of course, the best solution for you is to get your specific needs
fulfilled (that seems to be an univeral truth). The best solution for us
may involve some other metric.

-- Roberto

Reply | Threaded
Open this post in threaded view
|

Re: Missing feature in 5.3: LUA_FLOAT_NONE for crippled C libraries

Real Time Logic
> It would be more useful if you give concrete data, instead of generic
> claims. Which/where floating point functions are used in the code? Which
> constants? Why they are difficult to be corrected through the
> configuration file?

I have included the complete error log below. The errors I get are
related to standard float functions such as floor, freexp, etc. I see
you wrap all of these calls in macro l_mathop, but I do not see how I
can cirumvent the use of the floating point library calls by modifying
l_mathop (i.e. by only modifying the config file). You can also see from
the log below the errors I get on math contants such as HUGE_VAL,
DBL_MANT_DIG, etc..


> Of course, the best solution for you is to get your specific needs
> fulfilled (that seems to be an univeral truth). The best solution for us
> may involve some other metric.
>
> -- Roberto

Yes, I do understand Roberto, but I would think a "non float" option
would be very common feature that fits many Lua use cases.

-Wilfred

The errors:

(Note: the missing prototype errors are not from not including the
correct header file. The functions simply do not exist in the C library).

### mwccmcf Compiler:
#    File: R:\src\plugins\lua\lobject.c
# -------------------------------------
#     112:     case LUA_OPPOW: return  ((void)L, l_mathop(pow)( v1, v2));
#   Error:                                                    ^
#   function has no prototype
### mwccmcf Compiler:
#     113:  (l_mathop(floor)((( v1)/( v2))))
#   Error:                  ^
#   function has no prototype
### mwccmcf Compiler:
#     117:        { ( m) = l_mathop(fmod)( v1, v2); if (( m)*( v2) < 0)
( m) += ( v2); };
#   Error:                               ^
#   function has no prototype
### mwccmcf Compiler:
#     241:   return l_mathop(ldexp)(r, e);
#   Error:          ^^^^^^^^
#   function has no prototype
### mwccmcf Compiler:
#     255:     *result =  strtod((s), ( &endptr));
#   Error:                      ^
#   function has no prototype



### mwccmcf Compiler:
#    File: R:\src\plugins\lua\lstrlib.c
# -------------------------------------
#     826:   lua_Number dd = l_mathop(floor)(x);  /* get integer part
from 'x' */
#   Error:                   ^^^^^^^^
#   function has no prototype
### mwccmcf Compiler:
#     834:   if (x != x || x == HUGE_VAL || x == -HUGE_VAL)  /* inf or
NaN? */
#   Error:                      ^^^^^^^^
#   undefined identifier 'HUGE_VAL'
### mwccmcf Compiler:
#     843:     lua_Number m = l_mathop(frexp)(x, &e);  /* 'x' fraction
and exponent */
#   Error:                    ^^^^^^^^
#   function has no prototype
### mwccmcf Compiler:
#     850: (DBL_MANT_DIG)
#   Error:              ^
#   undefined identifier 'DBL_MANT_DIG'
### mwccmcf Compiler:
#     851: (DBL_MANT_DIG)
#   Error:              ^
#   undefined identifier 'DBL_MANT_DIG'
### mwccmcf Compiler:
#     971:  (DBL_MAX_10_EXP)
#   Error:                 ^
#   undefined identifier 'DBL_MAX_10_EXP'

Errors caused tool to abort.
### mwccmcf Compiler:
#    File: R:\src\plugins\lua\ltable.c
# ------------------------------------
#     102:   n = l_mathop(frexp)(n, &i) * -cast_num(INT_MIN);
#   Error:       ^^^^^^^^
#   function has no prototype



### mwccmcf Compiler:
#    File: R:\src\plugins\lua\lvm.c
# ---------------------------------
#      99:     lua_Number f =  (l_mathop(floor)(n));
#   Error:                                     ^
#   function has no prototype
### mwccmcf Compiler:
#     270: (DBL_MANT_DIG)
#   Error:              ^
#   undefined identifier 'DBL_MANT_DIG'
### mwccmcf Compiler:
#     289: (DBL_MANT_DIG)
#   Error:              ^
#   undefined identifier 'DBL_MANT_DIG'
### mwccmcf Compiler:
#     951:            { ( m) = l_mathop(fmod)( nb, nc); if (( m)*( nc) <
0) ( m) += ( nc); };
#   Error:                                   ^
#   function has no prototype
### mwccmcf Compiler:
#     966: ); val_(io).n=( ((void)L, (floor((( nb)/( nc))))));
settt_(io, LUA_TNUMFLT); };
#   Error:                                 ^
#   function has no prototype
### mwccmcf Compiler:
#     976: (ra); val_(io).n=( ((void)L, pow( nb, nc))); settt_(io,
LUA_TNUMFLT); };
#   Error:                                 ^
#   function has no prototype


Reply | Threaded
Open this post in threaded view
|

Re: Missing feature in 5.3: LUA_FLOAT_NONE for crippled C libraries

Roberto Ierusalimschy
I may be missing something, but the following definitions in 'luaconf.h'
seem to do a good job of elliminating floats from Lua. A few caveats:

- Depending how you add these definitions to your luaconf.h, you will
have to undefine the original definition.

- I assume you removed the math lib (a one-line change in linit.c).

- There are some quirks; most of them I think were present also in
older versions of Lua converted to int:
  * exponentiation results in zero (we should provide our own
    implementation for integer exponentiation)
  * string.format for floats (%a, %f, %g, etc.) give weird results

-- Roberto

/* -----------------------------------------------------------------*/
#define LUA_NUMBER LUA_INTEGER
#define LUAI_UACNUMBER LUAI_UACINT
#define LUA_NUMBER_FRMLEN LUA_INTEGER_FRMLEN
#define LUA_NUMBER_FMT LUA_INTEGER_FMT

#define luai_numpow(L,a,b) 0  /* we should provide an implementation... */
#define luai_nummod(L,a,b,m) ((m) = luaV_mod(L, (a), (b)))
#define luai_numdiv(L,a,b) luaV_div(L, a, b)
#define luai_numidiv(L,a,b) luaV_div(L, a, b)

#define l_floor(x) (x)

#define l_mathop(op) X X X /* ensure it is not being used */

#define LUA_COMPAT_FLOATSTRING

#define lua_str2number(s,p) (*(p) = (char*)(s), 0)

#define l_hashfloat(n) (n)

#define l_mathlim(x) (LUAI_##x)
#define LUAI_MANT_DIG 63
#define LUAI_MAX_10_EXP 10 /* arbitrary */

#define lua_numbertointeger(n,p) (*(p)=(n), 1)

Reply | Threaded
Open this post in threaded view
|

Re: Missing feature in 5.3: LUA_FLOAT_NONE for crippled C libraries

Dirk Laurie-2
2015-10-21 15:21 GMT+02:00 Roberto Ierusalimschy <[hidden email]>:

> I may be missing something, but the following definitions in 'luaconf.h'
> seem to do a good job of elliminating floats from Lua. A few caveats:

I think the OP agrees that it is not hard, but is trying to push the notion
that it is a sufficiently common Lua variation to justify being switched
on with just a single -D. Imagine a cardiac pacemaker that is scriptable
in Lua, that sort of thing.

Reply | Threaded
Open this post in threaded view
|

Re: Missing feature in 5.3: LUA_FLOAT_NONE for crippled C libraries

Roberto Ierusalimschy
> 2015-10-21 15:21 GMT+02:00 Roberto Ierusalimschy <[hidden email]>:
>
> > I may be missing something, but the following definitions in 'luaconf.h'
> > seem to do a good job of elliminating floats from Lua. A few caveats:
>
> I think the OP agrees that it is not hard, but is trying to push the notion
> that it is a sufficiently common Lua variation to justify being switched
> on with just a single -D. Imagine a cardiac pacemaker that is scriptable
> in Lua, that sort of thing.

The OP started his original request with this text:

    Changing the number type from double to "integer only" was easy in
    previous versions of Lua since only the configuration file needed to be
    changed.

This gave me the impression that the previous situation was "good
enough" for him, so I was mainly arguing that things are more or less
as they were.

-- Roberto

Reply | Threaded
Open this post in threaded view
|

Re: Missing feature in 5.3: LUA_FLOAT_NONE for crippled C libraries

Rena
In reply to this post by Dirk Laurie-2

On Oct 21, 2015 10:22 AM, "Dirk Laurie" <[hidden email]> wrote:
>
> 2015-10-21 15:21 GMT+02:00 Roberto Ierusalimschy <[hidden email]>:
>
> > I may be missing something, but the following definitions in 'luaconf.h'
> > seem to do a good job of elliminating floats from Lua. A few caveats:
>
> I think the OP agrees that it is not hard, but is trying to push the notion
> that it is a sufficiently common Lua variation to justify being switched
> on with just a single -D. Imagine a cardiac pacemaker that is scriptable
> in Lua, that sort of thing.
>
I'd rather not! No offense to Lua, but it's not certified for use in safety-critical operations like that. Those kinds of things really should have very minimal software written in heavily reviewed and well-formatted MISRA C. The more code, the more potential for bugs. Just ask Toyota...

</offtopic>

There are plenty of embedded (non-safety-critical) systems where an integer-only Lua might be useful, though. For example, some Nintendo DS games used such a modified version of Lua 5.1, since it lacked an FPU and was too slow to feasibly simulate one in software. (Not to be confused with the 3DS which does have an FPU.)

Such systems are becoming more common now too with cheap microcontrollers reaching the point where running Lua on them is feasible.

Reply | Threaded
Open this post in threaded view
|

Re: Missing feature in 5.3: LUA_FLOAT_NONE for crippled C libraries

Real Time Logic
In reply to this post by Roberto Ierusalimschy
Roberto,

Based on the config settings you provided and by performing some minimal
changes to the C code, I got it to compile. Preliminary tests show that
it is working.

Thanks,
Wilfred