Must "attempt to get length of a number value" be an error?

classic Classic list List threaded Threaded
42 messages Options
123
Reply | Threaded
Open this post in threaded view
|

Re: Must "attempt to get length of a number value" be an error?

Coda Highland


On Thu, May 9, 2019 at 11:43 AM Dirk Laurie <[hidden email]> wrote:
Op Do. 9 Mei 2019 om 15:45 het Coda Highland <[hidden email]> geskryf:

> But the #n == 8 comment was never meant as anything more than a flippant remark.

My original motivation (not divulged in the post) was that invoking a
function to get the absolute value of a number is really a very
cumbersome way for something that can be done by the C ternary
operator.Therefore I looked around for a unary operator not yet
defined for type "number" and found that __len is the only candidate.

It's a ten-line patch to lvm.c:
534a535,544
>     case LUA_TNUMINT: {
>       lua_Integer n=ivalue(rb);
>       setivalue(ra,n<0?-n:n);
>       return;
>     }
>     case LUA_TNUMFLT: {
>       lua_Number x=fltvalue(rb);
>       setfltvalue(ra,x<0?-x:x);
>       return;
>     }

Then I suppose it is both amusing and appropriate that I came up with #n = abs(n) as well. :P

Not sure I think it's all that clumsy to use a function, though.

/s/ Adam 
Reply | Threaded
Open this post in threaded view
|

Re: Must "attempt to get length of a number value" be an error?

Sean Conner
In reply to this post by Egor Skriptunoff-2
It was thus said that the Great Egor Skriptunoff once stated:

> On Thu, May 9, 2019 at 7:59 PM Egor Skriptunoff wrote:
>
> > On Thu, May 9, 2019 at 7:43 PM Dirk Laurie wrote:
> >
> >>
> >> It's a ten-line patch to lvm.c:
> >> 534a535,544
> >> >     case LUA_TNUMINT: {
> >> >       lua_Integer n=ivalue(rb);
> >> >       setivalue(ra,n<0?-n:n);
> >> >       return;
> >> >     }
> >>
> >
> > Ten lines are not enough.
> > What result would you expect for INT_MIN ?
> >
>
> BTW, math.abs(1<<63) in both Lua 5.3 and 5.4 returns negative value.
> That's a bug.

  That's also a bug in most CPUs (those that are 2's complement), although a
number will set the overflow flag [1] to signal the condision [2].

  -spc

[1] 6809, 8080, Z80, 68k series, x86 series do; other CPUs might as
        well.

[2] The VAX doesn't.  Also, there's no integer instruction on the MIPS
        that does negation.  There are two obvious sequences that will do
        negation, but only one will trap on overflow.

Reply | Threaded
Open this post in threaded view
|

Re: Must "attempt to get length of a number value" be an error?

Egor Skriptunoff-2
On Thu, May 9, 2019 at 11:49 PM Sean Conner wrote:
It was thus said that the Great Egor Skriptunoff once stated:
>
> BTW, math.abs(1<<63) in both Lua 5.3 and 5.4 returns negative value.
> That's a bug.

  That's also a bug in most CPUs (those that are 2's complement), although a
number will set the overflow flag [1] to signal the condision [2].


CPUs are making all arithmetic modulo their word (2^32), this is how the CPU operations are declared in CPU architecture manual.
So, it's not a bug in CPU.

Integer operators in Lua are explicitly declared modulo 2^64 (Lua manual section 3.4.1 - Arithmetic operators), so the following is OK:
assert(-INT_MIN==INT_MIN)

But Lua math library is supposed to give mathematically correct results.
math.abs(INT_MIN) should return correct (float) value.
The similar issue has already been successfully solved in Lua: function tonumber returns result having most suitable numeric subtype.
Reply | Threaded
Open this post in threaded view
|

Re: Must "attempt to get length of a number value" be an error?

Francisco Olarte
In reply to this post by Dirk Laurie-2
On Thu, May 9, 2019 at 6:43 PM Dirk Laurie <[hidden email]> wrote:
...
> My original motivation (not divulged in the post) was that invoking a
> function to get the absolute value of a number is really a very
> cumbersome way for something that can be done by the C ternary
> operator.Therefore I looked around for a unary operator not yet
> defined for type "number" and found that __len is the only candidate.

So, in C you use (x<0)?-x:x ( or a variation ) instead of just abs(3)
( or cousins ) ?

IMHO abs(x) is easier to read, hard to get wrong. I try to use it
(shorter easier than the ternary, and compilers optimize it quite well
since the nineties, IIRC microsoft dit it when generating 8086 code ).

And for just a little more keying ( # needs an special shitf in
spanish keyboards :-(  ) I would prefer it in lua to # )

Also, if you like the ternary, isnt "(x<0) and -x or x" one of the
places where lua can simulate the ternary without problems?

Is # for abs the norm in some kind of math notation?  ( really curious
on this, I haven't read much math lately )

Francisco Olarte.

Reply | Threaded
Open this post in threaded view
|

Re: Must "attempt to get length of a number value" be an error?

Sean Conner
In reply to this post by Egor Skriptunoff-2
It was thus said that the Great Egor Skriptunoff once stated:

> On Thu, May 9, 2019 at 11:49 PM Sean Conner wrote:
>
> > It was thus said that the Great Egor Skriptunoff once stated:
> > >
> > > BTW, math.abs(1<<63) in both Lua 5.3 and 5.4 returns negative value.
> > > That's a bug.
> >
> >   That's also a bug in most CPUs (those that are 2's complement), although
> > a
> > number will set the overflow flag [1] to signal the condision [2].
> >
> >
> CPUs are making all arithmetic modulo their word (2^32), this is how the
> CPU operations are declared in CPU architecture manual.
> So, it's not a bug in CPU.
>
> Integer operators in Lua are explicitly declared modulo 2^64 (Lua manual
> section 3.4.1 - Arithmetic operators), so the following is OK:
> assert(-INT_MIN==INT_MIN)

  Actually, GCC disagrees:

        #include <limits.h>
        #include <assert.h>
       
        int main(void)
        {
          assert(-INT_MIN == INT_MIN);
          return 0;
        }

        [spc]lucy:/tmp>gcc x.c
        x.c: In function `Main':
        x.c:7: warning: integer overflow in expression
        [spc]lucy:/tmp>

  Which is fine, because that's undefined behavior in C.  From the C99
standard, 7.20.6.1.2:

        The abs, labs, and llabs functions compute the absolute value of an
        integer j.  If the result cannot be represented, the behavior is
        undefined.[265]

        [265] The absolute value of the most negative number cannot be
        represented in two's complement.

(Okay, so the code actually compiles, and the programs run without aborting,
but technically speaking, the program above is invoking undefined behavior.)

  Now, quoting section 3.4.1 of the Lua manual:

        With the exception of exponentiation and float division, the
        arithmetic operators work as follows: If both operands are integers,
        the operation is performed over integers and the result is an
        integer. ...

        In case of overflows in integer arithmetic, all operations wrap
        around, according to the usual rules of two-complement arithmetic.
        (In other words, they return the unique representable integer that
        is equal modulo 264 to the mathematical result.)

And indeed:

        Lua 5.3.4  Copyright (C) 1994-2017 Lua.org, PUC-Rio
        > assert(-math.mininteger == math.mininteger)
        true

> But Lua *math* library is supposed to give mathematically correct results.
> math.abs(INT_MIN) should return correct (float) value.
> The similar issue has already been successfully solved in Lua: function
> *tonumber* returns result having most suitable numeric subtype.

  Finally, quoting section 6.7 of the Lua manual:

        This library provides basic mathematical functions. It provides all
        its functions and constants inside the table math. Functions with
        the annotation "integer/float" give integer results for integer
        arguments and float results for float (or mixed) arguments. Rounding
        functions (math.ceil, math.floor, and math.modf) return an integer
        when the result fits in the range of an integer, or a float
        otherwise.

        math.abs(x)
                Returns the absolute valud of x. (integer/float)

  Nowhere does it state that the Lua *math* library is supposed to give
mathematically correct results, unless there's some part of the manual I'm
missing (the only type the term "correct" is applied to a function in the
math module is for math.atan()).

  So Lua (5.3) is basically handling things as C would, only it avoids the
whole "invoking undefined behavior" by calculating the absolute value of an
integer using unsigned arithmetic:

        static int math_abs (lua_State *L) {
          if (lua_isinteger(L, 1)) {
            lua_Integer n = lua_tointeger(L, 1);
            if (n < 0) n = (lua_Integer)(0u - (lua_Unsigned)n);
            lua_pushinteger(L, n);
          }
          else
            lua_pushnumber(L, l_mathop(fabs)(luaL_checknumber(L, 1)));
          return 1;
        }

  So is the bug in the documentation, or the behavior?

  -spc


Reply | Threaded
Open this post in threaded view
|

Re: Must "attempt to get length of a number value" be an error?

Philippe Verdy


Le ven. 10 mai 2019 à 16:08, Sean Conner <[hidden email]> a écrit :
It was thus said that the Great Egor Skriptunoff once stated:
> On Thu, May 9, 2019 at 11:49 PM Sean Conner wrote:
>
> > It was thus said that the Great Egor Skriptunoff once stated:
> > >
> > > BTW, math.abs(1<<63) in both Lua 5.3 and 5.4 returns negative value.
> > > That's a bug.
> >
> >   That's also a bug in most CPUs (those that are 2's complement), although
> > a
> > number will set the overflow flag [1] to signal the condision [2].
> >
> >
> CPUs are making all arithmetic modulo their word (2^32), this is how the
> CPU operations are declared in CPU architecture manual.
> So, it's not a bug in CPU.
>
> Integer operators in Lua are explicitly declared modulo 2^64 (Lua manual
> section 3.4.1 - Arithmetic operators), so the following is OK:
> assert(-INT_MIN==INT_MIN)

  Actually, GCC disagrees:

        #include <limits.h>
        #include <assert.h>

        int main(void)
        {
          assert(-INT_MIN == INT_MIN);
          return 0;
        }

        [spc]lucy:/tmp>gcc x.c
        x.c: In function `Main':
        x.c:7: warning: integer overflow in expression
        [spc]lucy:/tmp>

  Which is fine, because that's undefined behavior in C.  From the C99
standard, 7.20.6.1.2:

        The abs, labs, and llabs functions compute the absolute value of an
        integer j.  If the result cannot be represented, the behavior is
        undefined.[265]

        [265] The absolute value of the most negative number cannot be
        represented in two's complement.

(Okay, so the code actually compiles, and the programs run without aborting,
but technically speaking, the program above is invoking undefined behavior.)

  Now, quoting section 3.4.1 of the Lua manual:

        With the exception of exponentiation and float division, the
        arithmetic operators work as follows: If both operands are integers,
        the operation is performed over integers and the result is an
        integer. ...

        In case of overflows in integer arithmetic, all operations wrap
        around, according to the usual rules of two-complement arithmetic.
        (In other words, they return the unique representable integer that
        is equal modulo 264 to the mathematical result.)

This statement in the doc CANNOT be true, otherwise we are unable to MULTIPLY arbitrary pairs of integers : in case of integer overflow, a floatting point value MUST be returned, because there's a single Lua "number" datatype for both integers and non-integers. Subtypes of numbers, are just internal optimisation hints, but such exceptions have to be handled to change the internal subtype.

And taking the opposite of an integer is simply a multiplication of that integer by integer -1 (and it should be).

I think this is a severe bug of the Lua doc ! So I also agree that assert(-INT_MIN==INT_MIN) should really cause an error, even if it does not (in which case this is an implementation bug, fogetting to check that case)



Reply | Threaded
Open this post in threaded view
|

Re: Must "attempt to get length of a number value" be an error?

Philippe Verdy
As well, if you add 1 to the MAX_INT value in Lua should not wrap around.

It's a fact that Lua implementations tried to optimize the operations on its "number" type to avoid using floatting points as much as possible, so that it could use the integer arithmeric of C or C++. But the number type is larger in range and precision than that. In some cases, Lua integers cannot fit the native integers in C and will still become floatting points (possibly with loss of precision and rounding for the lowest bits of large absolute values).

Optimization of operations on Lua numbers is then tricky, and implementations should care about it (and should raise overflows only in case of overflows of floatting points with a too large exponent, or at least returning signed "infinite" values which are also valid Lua number values).

If Lua code wants to wrap around, I really think that Lua should provide a scoping declaration or annotation for that (we've already spoken about weak references with "*toclose" which now requires such annotation). It's high time for Lua to think about generalizing a syntax for annotations (not just for declarations of "*toclose" variables, but also for expressions) and it's another argument for changing "*toclose" into "@toclose".

I still propose the syntax "@ANNOTATION_ID" or "@{ ANNOTATION_ID, parameters...}" for these annotations, to be used before a declared name or before a subexpression in parentheses, rather than "*ANNOTATION_ID" just before a declared name.


Le ven. 10 mai 2019 à 19:56, Philippe Verdy <[hidden email]> a écrit :


Le ven. 10 mai 2019 à 16:08, Sean Conner <[hidden email]> a écrit :
It was thus said that the Great Egor Skriptunoff once stated:
> On Thu, May 9, 2019 at 11:49 PM Sean Conner wrote:
>
> > It was thus said that the Great Egor Skriptunoff once stated:
> > >
> > > BTW, math.abs(1<<63) in both Lua 5.3 and 5.4 returns negative value.
> > > That's a bug.
> >
> >   That's also a bug in most CPUs (those that are 2's complement), although
> > a
> > number will set the overflow flag [1] to signal the condision [2].
> >
> >
> CPUs are making all arithmetic modulo their word (2^32), this is how the
> CPU operations are declared in CPU architecture manual.
> So, it's not a bug in CPU.
>
> Integer operators in Lua are explicitly declared modulo 2^64 (Lua manual
> section 3.4.1 - Arithmetic operators), so the following is OK:
> assert(-INT_MIN==INT_MIN)

  Actually, GCC disagrees:

        #include <limits.h>
        #include <assert.h>

        int main(void)
        {
          assert(-INT_MIN == INT_MIN);
          return 0;
        }

        [spc]lucy:/tmp>gcc x.c
        x.c: In function `Main':
        x.c:7: warning: integer overflow in expression
        [spc]lucy:/tmp>

  Which is fine, because that's undefined behavior in C.  From the C99
standard, 7.20.6.1.2:

        The abs, labs, and llabs functions compute the absolute value of an
        integer j.  If the result cannot be represented, the behavior is
        undefined.[265]

        [265] The absolute value of the most negative number cannot be
        represented in two's complement.

(Okay, so the code actually compiles, and the programs run without aborting,
but technically speaking, the program above is invoking undefined behavior.)

  Now, quoting section 3.4.1 of the Lua manual:

        With the exception of exponentiation and float division, the
        arithmetic operators work as follows: If both operands are integers,
        the operation is performed over integers and the result is an
        integer. ...

        In case of overflows in integer arithmetic, all operations wrap
        around, according to the usual rules of two-complement arithmetic.
        (In other words, they return the unique representable integer that
        is equal modulo 264 to the mathematical result.)

This statement in the doc CANNOT be true, otherwise we are unable to MULTIPLY arbitrary pairs of integers : in case of integer overflow, a floatting point value MUST be returned, because there's a single Lua "number" datatype for both integers and non-integers. Subtypes of numbers, are just internal optimisation hints, but such exceptions have to be handled to change the internal subtype.

And taking the opposite of an integer is simply a multiplication of that integer by integer -1 (and it should be).

I think this is a severe bug of the Lua doc ! So I also agree that assert(-INT_MIN==INT_MIN) should really cause an error, even if it does not (in which case this is an implementation bug, fogetting to check that case)



Reply | Threaded
Open this post in threaded view
|

Re: Must "attempt to get length of a number value" be an error?

Egor Skriptunoff-2
In reply to this post by Sean Conner
On Fri, May 10, 2019 at 5:08 PM Sean Conner wrote:

> But Lua *math* library is supposed to give mathematically correct results.
> math.abs(INT_MIN) should return correct (float) value.
> The similar issue has already been successfully solved in Lua: function
> *tonumber* returns result having most suitable numeric subtype.

  Finally, quoting section 6.7 of the Lua manual:

        This library provides basic mathematical functions. It provides all
        its functions and constants inside the table math. Functions with
        the annotation "integer/float" give integer results for integer
        arguments and float results for float (or mixed) arguments. Rounding
        functions (math.ceil, math.floor, and math.modf) return an integer
        when the result fits in the range of an integer, or a float
        otherwise.

        math.abs(x)
                Returns the absolute valud of x. (integer/float)


According to Lua manual, math.abs() must return the absolute value of its argument.
Obviously, math.abs(math.mininteger) not equals to the absolute value.
That's a bug.

"bug" means that behavior does not match documentation.
Either behavior should be changed (to return correct result)
or documentation should be changed (to claim math.abs as returning result modulo 2^64 instead of the correct result)
Reply | Threaded
Open this post in threaded view
|

math.abs(math.mininteger) (Was: Must "attempt to get length of a number value" be an error?

Dirk Laurie-2
Op Sa. 11 Mei 2019 om 01:38 het Egor Skriptunoff
<[hidden email]> geskryf:

> According to Lua manual, math.abs() must return the absolute value of its argument.
> Obviously, math.abs(math.mininteger) not equals to the absolute value.
> That's a bug.
>
> "bug" means that behavior does not match documentation.
> Either behavior should be changed (to return correct result)
> or documentation should be changed (to claim math.abs as returning result modulo 2^64 instead of the correct result)

In view of the fact that math.abs(math.mininteger) _can_ be
represented exactly as a lua_Number, I must agree with you.

I tried coding this case. It is suprisingly tricky.

fprintf(stderr,"n=%lli x=%llx  <≤=≥>:%i%i%i%i%i\n",n,n,n<0,n<=0,n==0,n>=0,n>0);
      if (n<0) n=-n;
fprintf(stderr,"n=%lli x=%llx  <≤=≥>:%i%i%i%i%i\n",n,n,n<0,n<=0,n==0,n>=0,n>0);

produces (GNU compiler):

n=-9223372036854775808 x=8000000000000000  <≤=≥>:11000
n=-9223372036854775808 x=8000000000000000  <≤=≥>:01010

I.e. x=-math.mininteger satisfies x<=0 and x<=0 but not x==0. Curious,
and not to be relied on, according to a 2012 paper on overflow in C
http://www.cs.utah.edu/~regehr/papers/overflow12.pdf which says that
the result of -INT_MIN is undefined in 2-s complement,

The correct coding must test for the edge case and dispose of it
before handing the general case.

      lua_Integer n=ivalue(rb);
      if (n==LUA_MININTEGER) setfltvalue(ra,-(lua_Number)(n))
      else if (n>=0) setivalue(ra,n)
      else setivalue(ra,-n);

Reply | Threaded
Open this post in threaded view
|

Re: Must "attempt to get length of a number value" be an error?

Roberto Ierusalimschy
In reply to this post by Egor Skriptunoff-2
> >         [...] . Functions with
> >         the annotation "integer/float" give integer results for integer
> >         arguments and float results for float (or mixed) arguments.
> >         [...]
> >         math.abs(x)
> >                 Returns the absolute valud of x. (integer/float)
>
> According to Lua manual, math.abs() must return the absolute value of its
> argument.
> Obviously, math.abs(math.mininteger) not equals to the absolute value.
> That's a bug.
>
> "bug" means that behavior does not match documentation.
> Either behavior should be changed (to return correct result)
> or documentation should be changed (to claim math.abs as returning result
> modulo 2^64 instead of the correct result)

The manual has incompatible requirements. As it is clear in your
message, the manual requires that 'math.abs(math.mininteger)' returns
an integer. So, returning a float would fix a bug to create another
(behavior *still* does not match documentation). So, one solution is
to change the documentation; the other is to change the behavior *and*
the documentation.

-- Roberto

Reply | Threaded
Open this post in threaded view
|

Re: Must "attempt to get length of a number value" be an error?

Egor Skriptunoff-2
On Sat, May 11, 2019 at 5:34 PM Roberto Ierusalimschy wrote:
the manual requires that 'math.abs(math.mininteger)' returns
an integer.



The following requirement in the manual
"integer results for integer arguments and float results for float (or mixed) arguments"
might be replaced with
"the value returned by a math function has the most suitable numeric subtype"

Actually, the rule of "most suitable subtype" is already implemented for tonumber, math.floor and math.ceil

Reply | Threaded
Open this post in threaded view
|

Re: math.abs(math.mininteger) (Was: Must "attempt to get length of a number value" be an error?

Philippe Verdy
In reply to this post by Dirk Laurie-2


Le sam. 11 mai 2019 à 09:15, Dirk Laurie <[hidden email]> a écrit :
Op Sa. 11 Mei 2019 om 01:38 het Egor Skriptunoff
<[hidden email]> geskryf:

> According to Lua manual, math.abs() must return the absolute value of its argument.
> Obviously, math.abs(math.mininteger) not equals to the absolute value.
> That's a bug.
>
> "bug" means that behavior does not match documentation.
> Either behavior should be changed (to return correct result)
> or documentation should be changed (to claim math.abs as returning result modulo 2^64 instead of the correct result)

In view of the fact that math.abs(math.mininteger) _can_ be
represented exactly as a lua_Number, I must agree with you.

I tried coding this case. It is suprisingly tricky.

fprintf(stderr,"n=%lli x=%llx  <≤=≥>:%i%i%i%i%i\n",n,n,n<0,n<=0,n==0,n>=0,n>0);
      if (n<0) n=-n;
fprintf(stderr,"n=%lli x=%llx  <≤=≥>:%i%i%i%i%i\n",n,n,n<0,n<=0,n==0,n>=0,n>0);

produces (GNU compiler):

n=-9223372036854775808 x=8000000000000000  <≤=≥>:11000
n=-9223372036854775808 x=8000000000000000  <≤=≥>:01010

What you show here is a bug of the C compiler in its optimizer: given that the printed values "n=... x..." are the same before and after negating, but different when you perform tests on n in other parameters of the C "fprintf" function, you have here the effect of an unexpected caching for the value of n, and the order of evaluation of parameter values, which is not consistant (the optimizer got too far when comparing n with zero, or precomputed these tests before the "if(n<0)..." instruction and then passed the wrong value)

I don't know how you compiled it, but if you got "<≤=≥>:11000" in the first line and "<≤=≥>:01010" on the second line, the value of n displayed in "n=..., x=..." MUST be different.
Or you did not declare "n" as a C integer but using some bogous custom class instead (which overrides some operators but not all: this custom type is not fully comparable)...

I tried just compiling your example using only an "int n;" or "long n;" declaration (and all other basic integer types of various size), and no custom class at all (or other incorrect macros transcluded from your platform headers, possibly patched by you), and I get consistant results, even with GCC, in C like in C++ and various other dialects, and as well in Java, J#, C#...

So it's very likely that your test was not performed on standard C/C++ integer types or that your host platform is incorrectly installed (incorrectly patched "standard headers): check your installation, or reinstall GCC in a clean environment (including all its standard headers, and the headers of your Linux kernel version). If needed, create a new Linux machine to isolate it: there are good distributions whose goals is not to support a complete graphical UI, but just to support a basic shell console (with almost no services, no browsers, no audio/video/multimedia, no web integration, no X11, only a single local filesystem and a telnet or serial session adapter) just used to install and run compilers (suitable for rebui_lding a new kernel image for example).

Or you have a severe bug in your CPU (or its loaded microfirmware) !

Before compiling your Lua implementation on your platform, you should run the safety tests to make sure you get consistant results. Otherwise your Lua implementation will report the same bugs at run time: its the basic work you need to do on your platform to make sure it satisfies some constraints (and if not, you'll not to add a compatibility layer of patches: it is possibly in that step that you defined a custom type for some part of the Lua host adapter, but without scoping it to only the cases where these patches are really needed, and what you did had unexpected effects on other parts of the code and you did not check it).

I think that a Lua builder should have a standard set of safety tests that it must satisfy, with dependencies that must be satisfied by order of priority (any other bugs occuring later because of a high priority test that failed, must NOT be corrected by a patch at that later stage, but on prior stage you forgot to check).

Even GCC itself is compiled with a set of tests that must be met in order, before going on and declaring that compilation succeeded. The same should be true for compiling your Lua engine.



Reply | Threaded
Open this post in threaded view
|

Re: math.abs(math.mininteger) (Was: Must "attempt to get length of a number value" be an error?

Andrew Gierth
>>>>> "Philippe" == Philippe Verdy <[hidden email]> writes:

 >> fprintf(stderr,"n=%lli x=%llx
 >> <≤=≥>:%i%i%i%i%i\n",n,n,n<0,n<=0,n==0,n>=0,n>0);
 >> if (n<0) n=-n;
 >> fprintf(stderr,"n=%lli x=%llx
 >> <≤=≥>:%i%i%i%i%i\n",n,n,n<0,n<=0,n==0,n>=0,n>0);
 >>
 >> produces (GNU compiler):
 >>
 >> n=-9223372036854775808 x=8000000000000000  <≤=≥>:11000
 >> n=-9223372036854775808 x=8000000000000000  <≤=≥>:01010

 Philippe> What you show here is a bug of the C compiler in its
 Philippe> optimizer

Possibly unfortunately, this is not a bug. The program invokes undefined
behavior when it does n=-n on the the value shown; the compiler is
entitled to produce any output at all as a result.

In this case it's fairly obvious what happened: after seeing the line

if (n<0) n=-n;

the compiler is entitled to assume that n>=0 is true regardless of the
prior value of n, since only undefined behavior could cause that not to
be the case and the compiler is allowed to assume that undefined
behavior never occurs. So in the second fprintf, the compiler can
optimize (n>=0) to a constant 1, and (n<0) to a constant 0, while still
computing the remaining tests.

 Philippe> I tried just compiling your example using only an "int n;" or
 Philippe> "long n;" declaration (and all other basic integer types of
 Philippe> various size), and no custom class at all (or other incorrect
 Philippe> macros transcluded from your platform headers, possibly
 Philippe> patched by you), and I get consistant results, even with GCC,
 Philippe> in C like in C++ and various other dialects, and as well in
 Philippe> Java, J#, C#...

When dealing with modern compilers you can't just take a fragment of
code like the above and expect to reproduce the results; you need a
complete compilable program. It matters for example whether "n" is a
compile-time constant, because if it is, the compiler will do all the
evaluations at compile time and not need to make the (n>=0) assumption.

Here is an example program to reproduce the issue:

#include <stdio.h>
#include <limits.h>

static void tst(long n)
{
    fprintf(stderr,"n=%li x=%lx  <≤=≥>:%i%i%i%i%i\n",n,n,n<0,n<=0,n==0,n>=0,n>0);
    if (n<0) n=-n;
    fprintf(stderr,"n=%li x=%lx  <≤=≥>:%i%i%i%i%i\n",n,n,n<0,n<=0,n==0,n>=0,n>0);
}

int main(int argc, char **argv)
{
    tst(argc > 0 ? LONG_MIN : 1);
    return 0;
}

The use of argc here is to force the value not to be compile-time
constant. (The asm output is a bit more readable if you force tst() not
to be inlined, but I didn't include that for portability reasons.) This
shows the issue when compiled with either gcc8 or clang8 at any
optimization level above 0, unless you use -fwrapv.

Interesting changes you can do to the above program to see what the
optimizer does:

1. change to tst(LONG_MIN) and the unexpected output disappears, because
   now the compiler is doing all the tests at compile time.

2. change to tst(argc > 0 ? LONG_MIN : 0) and the output becomes even
   more strange in a different way (explanation left as an exercise for
   the reader)

--
Andrew.

Reply | Threaded
Open this post in threaded view
|

Re: math.abs(math.mininteger) (Was: Must "attempt to get length of a number value" be an error?

Philippe Verdy


Le dim. 12 mai 2019 à 07:38, Andrew Gierth <[hidden email]> a écrit :
>>>>> "Philippe" == Philippe Verdy <[hidden email]> writes:

 >> fprintf(stderr,"n=%lli x=%llx
 >> <≤=≥>:%i%i%i%i%i\n",n,n,n<0,n<=0,n==0,n>=0,n>0);
 >> if (n<0) n=-n;
 >> fprintf(stderr,"n=%lli x=%llx
 >> <≤=≥>:%i%i%i%i%i\n",n,n,n<0,n<=0,n==0,n>=0,n>0);
 >>
 >> produces (GNU compiler):
 >>
 >> n=-9223372036854775808 x=8000000000000000  <≤=≥>:11000
 >> n=-9223372036854775808 x=8000000000000000  <≤=≥>:01010

 Philippe> What you show here is a bug of the C compiler in its
 Philippe> optimizer

Possibly unfortunately, this is not a bug. The program invokes undefined
behavior when it does n=-n on the the value shown; the compiler is
entitled to produce any output at all as a result.

In this case it's fairly obvious what happened: after seeing the line

if (n<0) n=-n;

the compiler is entitled to assume that n>=0 is true regardless of the
prior value of n, since only undefined behavior could cause that not to
be the case and the compiler is allowed to assume that undefined
behavior never occurs. So in the second fprintf, the compiler can
optimize (n>=0) to a constant 1, and (n<0) to a constant 0, while still
computing the remaining tests.

Such compiler assumption is wrong and it's clearly a bug in its optimizer , trying to infer constant values from code whose evaluation is clearly not constant here, that line does not mean that the result in n is necessarily positive. so it cannot assume that n>=0 after this line (even if C indicates that the result is undefined, then the further tests of n<0 and n>=0 must still be computed: an undefined value has no *constant* sign).

But with your code I do not get the same result as you, so I bet you use a bad version of GCC with this bug. I tried your code with several C or C++ compilers, none of them exhibit this, and the bits shown after "<≤=≥>:" are consistant.

Even if integers were natively implemented using floatting points, and the result of n=-n (when n is int_min) was an "NaN" value, which is not comparable (in any test) to 0 (or any value including another NaN), then the test bits you'd get would be "<≤=≥>:00000". In that case on such theoretical platform, the expressions (!(n<0)) and (n>=0) are not equivalent, and the compiler cannot also make any assumption and has to compte all tests.

Finally, not just the compiler version is important here: you may have used some advanced (but experimental) optimizing options on its command line to permit such unsafe assumptions (trying to infer constant from subexpressions or statements, and then propagating these constant to further reduce the code generation after them). I've tried to compile your code with optimization enabled (-O) and do not reproduce it, so it's possible you used some "long option" (--keyword).

Note that your code may not experiment this bug if there's a function call between the "if()" statement and the "fprintf" statement (whose parameters must be computed and passed before performing the call to that function... which may also be inlined using some compiler-specific intrinsic instead of performing an actual function call with parameters pushed on the stack). So changing 

    if (n<0) n=-n;
    fprintf(stderr,"n=%lli x=%llx <≤=≥>:%i%i%i%i%i\n",n,n,n<0,n<=0,n==0,n>=0,n>0);

just to:

    if (n<0) n=-n;
    fprintf(stderr,"n=%lli x=%llx",n,n)
    fprintf(stderr,"<≤=≥>:%i%i%i%i%i\n",n<0,n<=0,n==0,n>=0,n>0);

may not exhibit this bug on your platform (this also depends on where the variable "n" is declared: on the stack or as a global, or if the compiler can infer it is not aliased and the local variable is not modified by the first function call, acting as a "memory barrier").

My opinion is that your GCC compiler version has a bug or you used some unsupported options or non-standard/untested patch, or your Linux distribution is not reliable and delivers you very badly compiled products: change your Linux update source or switch to a reliable distribution.
Reply | Threaded
Open this post in threaded view
|

Re: math.abs(math.mininteger) (Was: Must "attempt to get length of a number value" be an error?

Lorenzo Donati-3
On 12/05/2019 12:08, Philippe Verdy wrote:

> Le dim. 12 mai 2019 à 07:38, Andrew Gierth <[hidden email]> a
> écrit :
>
>>>>>>> "Philippe" == Philippe Verdy <[hidden email]> writes:
>>
>>  >> fprintf(stderr,"n=%lli x=%llx
>>  >> <≤=≥>:%i%i%i%i%i\n",n,n,n<0,n<=0,n==0,n>=0,n>0);
>>  >> if (n<0) n=-n;
>>  >> fprintf(stderr,"n=%lli x=%llx
>>  >> <≤=≥>:%i%i%i%i%i\n",n,n,n<0,n<=0,n==0,n>=0,n>0);
>>  >>
>>  >> produces (GNU compiler):
>>  >>
>>  >> n=-9223372036854775808 x=8000000000000000  <≤=≥>:11000
>>  >> n=-9223372036854775808 x=8000000000000000  <≤=≥>:01010
>>
>>  Philippe> What you show here is a bug of the C compiler in its
>>  Philippe> optimizer
>>
>> Possibly unfortunately, this is not a bug. The program invokes undefined
>> behavior when it does n=-n on the the value shown; the compiler is
>> entitled to produce any output at all as a result.
>>
>> In this case it's fairly obvious what happened: after seeing the line
>>
>> if (n<0) n=-n;
>>
>> the compiler is entitled to assume that n>=0 is true regardless of the
>> prior value of n, since only undefined behavior could cause that not to
>> be the case and the compiler is allowed to assume that undefined
>> behavior never occurs. So in the second fprintf, the compiler can
>> optimize (n>=0) to a constant 1, and (n<0) to a constant 0, while still
>> computing the remaining tests.
>>
>
> Such compiler assumption is wrong and it's clearly a bug in its optimizer ,
> trying to infer constant values from code whose evaluation is clearly not
> constant here, that line does not mean that the result in n is necessarily
> positive. so it cannot assume that n>=0 after this line (even if C
> indicates that the result is undefined, then the further tests of n<0 and
> n>=0 must still be computed: an undefined value has no *constant* sign).
>


[snip]

"the result is undefined..." and "... and undefined value..."



Sorry, but it seems that you are not aware of what "undefined behavior"
(UB) means in C. There is no "undefined result" and neither "undefined
value".

That jargon is not generic CS jargon, but it is a concept mandated by
the standard.

Basically, once your program triggers UB even in a single spot in a
gazillion lines of code, the whole gazillion lines are garbage.

Quoting from section 3.4.3 of C99 draft standard (N1256), emphasis mine:

==========================================================
3.4.3
1 undefined behavior

behavior, upon use of a nonportable or erroneous program construct or of
erroneous data, for which this International Standard imposes /no
requirements/

2
NOTE Possible undefined behavior ranges from /ignoring the situation
completely/ with /unpredictable results/, to behaving during translation
or program execution in a documented manner characteristic of the
environment (with or without the issuance of a diagnostic message), to
terminating a translation or execution (with the issuance of a
diagnostic message).

3
EXAMPLE An example of undefined behavior is the behavior on integer
overflow.

==========================================================

So there is no requirement for any conforming implementation to do
anything sensible when UB is invoked.

Note also that it is not /the value/ that it is undefined. Any program
construct that invokes undefined behavior renders /the whole program/
garbage, from the POV of a conforming implementation.

To stress that ANYTHING could happen, it is a common joke among C
programmers to say that if you trigger undefined behavior you get
"demons coming out of your nose", usually abbreviated to "you get nasal
demons".

An interesting set of articles from the blog of John Regehr (Professor
of Computer Science, University of Utah, USA) that may shed some light
on the rationale behind UB:

https://blog.regehr.org/archives/213


Cheers!

-- Lorenzo






Reply | Threaded
Open this post in threaded view
|

Re: math.abs(math.mininteger) (Was: Must "attempt to get length of a number value" be an error?

Philippe Verdy
Interesting article : it finally shows that compilers are made by people that are interested only in compilers and prure performance, and don't even care about making it useful for any language.

Now if C99 allows this, then C99 is not a language defined for programmers, jsut for the interests of compiler makers. It just kills the language and makes it completely useless (and malicious) to any one: programmers, and final users (which are not even given any right to know what is in the program they buy or use, these programs are also best to categorize are malwares).

In summary, C is a malware to eradicate, unless it is implemented by replacing all undefined behaviors from the so called "standard" by defined behavior.

For the simple case of (INT_MAX+1), it can be deterministic on all platform by saying its result must be INT_MIN (disregard the concept of 2-complements which implies a binary representation). And on most machines, this is what the native opcodes do, so it has no cost. But compilers cannot misbehave and pretend this is undefined behavior allowing them to *silently* remove instructions and treating them as no-ops (ignoring completely the explicit developer's desires, as if the developer was stupid and only the compiler knows better what to do: at least the compiler should emit a warning that programmers can see, and the compiler and language specification must provide a way to avoid this situation).

As well (INT_MIN-1) must be INT_MAX.

For (1 << n), it must be defined (as 0) if 2^n is larger than INT_MAX (yes it implies a cost for compilers: on x86, the value of n is silently masked by considering only the 5 lowest bits of n for 32 bit operations, or 6 lowest bits for 64-bit operations, this only depends on the opcode used, i.e. the bitsize of the specified register for the SHL instruction). But in the C language, these considerations about native opcodes and native register sizes is not relevant, and it's up to compilers to allocate the proper registers and generate the instructions according to the language specifications, adding additional code where needed to get the expected result, so compilers should then emit an AND masking instruction before emitting the SHL.

What is the real cost of these check ? None. C is a now very old language, the minor performance costs was completely erased by massive gains of performance: the small impact it has is temporary and cancels rapidly.

Are compiler makers serious ? Don't they care about the huge costs they generate worldwide and the massive security attacks we see today ?

Those compiler makers should be fired and given NO vote to language standardizers: fire them from all standard bodies, don't hear what they want to pass !

Placing "undefined behavior" in any language standard is simply stupid (and lazy !). There may be cases where this was unexpectedly forgotten, but the best to do is to update the standard, look at existing "compliant" implementations and study what will be the best defined behavior to integrate in the standard. That's what is done (hopefully) for Java or ECMAscript/Javascript (which is now eradicating Typescript completely). Doing that will possibly make some existing "compliant" implementations suddenly non-compliant.

But at least developers will get an opportunity to update their compiler tools with more recent versions: even if the generated code is not modified (for compatibiliy reason), at least the new compilers will emit trackable warnings that will allow developers to fix their source code to behave as they expected once compiled and tested by them, and then run by final users that must be able to trust the developers.

For now the C standard is not made for any one. Its goals are stupid. The whole team of this standard body (most of them from ANSI) should be fired (or another standard body should take ove the goal of redefining it). Everyone in the computing industry will be interested and now every one in the world uses computers and are interested: this goal is no longer technical, it has become a fundamental human right to absolutely preserve, but voluntarily ignored by the existing standard body which behaves like devil.

Le dim. 12 mai 2019 à 12:41, Lorenzo Donati <[hidden email]> a écrit :
On 12/05/2019 12:08, Philippe Verdy wrote:
> Le dim. 12 mai 2019 à 07:38, Andrew Gierth <[hidden email]> a
> écrit :
>
>>>>>>> "Philippe" == Philippe Verdy <[hidden email]> writes:
>>
>>  >> fprintf(stderr,"n=%lli x=%llx
>>  >> <≤=≥>:%i%i%i%i%i\n",n,n,n<0,n<=0,n==0,n>=0,n>0);
>>  >> if (n<0) n=-n;
>>  >> fprintf(stderr,"n=%lli x=%llx
>>  >> <≤=≥>:%i%i%i%i%i\n",n,n,n<0,n<=0,n==0,n>=0,n>0);
>>  >>
>>  >> produces (GNU compiler):
>>  >>
>>  >> n=-9223372036854775808 x=8000000000000000  <≤=≥>:11000
>>  >> n=-9223372036854775808 x=8000000000000000  <≤=≥>:01010
>>
>>  Philippe> What you show here is a bug of the C compiler in its
>>  Philippe> optimizer
>>
>> Possibly unfortunately, this is not a bug. The program invokes undefined
>> behavior when it does n=-n on the the value shown; the compiler is
>> entitled to produce any output at all as a result.
>>
>> In this case it's fairly obvious what happened: after seeing the line
>>
>> if (n<0) n=-n;
>>
>> the compiler is entitled to assume that n>=0 is true regardless of the
>> prior value of n, since only undefined behavior could cause that not to
>> be the case and the compiler is allowed to assume that undefined
>> behavior never occurs. So in the second fprintf, the compiler can
>> optimize (n>=0) to a constant 1, and (n<0) to a constant 0, while still
>> computing the remaining tests.
>>
>
> Such compiler assumption is wrong and it's clearly a bug in its optimizer ,
> trying to infer constant values from code whose evaluation is clearly not
> constant here, that line does not mean that the result in n is necessarily
> positive. so it cannot assume that n>=0 after this line (even if C
> indicates that the result is undefined, then the further tests of n<0 and
> n>=0 must still be computed: an undefined value has no *constant* sign).
>


[snip]

"the result is undefined..." and "... and undefined value..."



Sorry, but it seems that you are not aware of what "undefined behavior"
(UB) means in C. There is no "undefined result" and neither "undefined
value".

That jargon is not generic CS jargon, but it is a concept mandated by
the standard.

Basically, once your program triggers UB even in a single spot in a
gazillion lines of code, the whole gazillion lines are garbage.

Quoting from section 3.4.3 of C99 draft standard (N1256), emphasis mine:

==========================================================
3.4.3
1 undefined behavior

behavior, upon use of a nonportable or erroneous program construct or of
erroneous data, for which this International Standard imposes /no
requirements/

2
NOTE Possible undefined behavior ranges from /ignoring the situation
completely/ with /unpredictable results/, to behaving during translation
or program execution in a documented manner characteristic of the
environment (with or without the issuance of a diagnostic message), to
terminating a translation or execution (with the issuance of a
diagnostic message).

3
EXAMPLE An example of undefined behavior is the behavior on integer
overflow.

==========================================================

So there is no requirement for any conforming implementation to do
anything sensible when UB is invoked.

Note also that it is not /the value/ that it is undefined. Any program
construct that invokes undefined behavior renders /the whole program/
garbage, from the POV of a conforming implementation.

To stress that ANYTHING could happen, it is a common joke among C
programmers to say that if you trigger undefined behavior you get
"demons coming out of your nose", usually abbreviated to "you get nasal
demons".

An interesting set of articles from the blog of John Regehr (Professor
of Computer Science, University of Utah, USA) that may shed some light
on the rationale behind UB:

https://blog.regehr.org/archives/213


Cheers!

-- Lorenzo






Reply | Threaded
Open this post in threaded view
|

Re: math.abs(math.mininteger) (Was: Must "attempt to get length of a number value" be an error?

Andrew Gierth
In reply to this post by Philippe Verdy
>>>>> "Philippe" == Philippe Verdy <[hidden email]> writes:

 >> Possibly unfortunately, this is not a bug. The program invokes
 >> undefined behavior when it does n=-n on the the value shown; the
 >> compiler is entitled to produce any output at all as a result.
 >>
 >> In this case it's fairly obvious what happened: after seeing the line
 >>
 >> if (n<0) n=-n;
 >>
 >> the compiler is entitled to assume that n>=0 is true regardless of
 >> the prior value of n, since only undefined behavior could cause that
 >> not to be the case and the compiler is allowed to assume that
 >> undefined behavior never occurs. So in the second fprintf, the
 >> compiler can optimize (n>=0) to a constant 1, and (n<0) to a
 >> constant 0, while still computing the remaining tests.

 Philippe> Such compiler assumption is wrong

The language standards disagree. Now you might choose to disagree with
those standards, but they are what they are.

 Philippe> But with your code I do not get the same result as you,

What exactly did you compile and how, and with what?

I don't keep a large supply of compilers on hand, but putting the code
on godbolt and looking at the output for various compiler versions, it's
clear that the optimization I described above is in effect for gcc
versions back to at least 4.9, and clang from 7.0 onwards (notably,
clang 6.0 does _not_ do it). See for example line 46 of the assembler
output of https://godbolt.org/z/vE8qCi where the "pushq $1" shows that
a constant 1 was pushed for the result of the >= test.

It _looks_ like neither icc nor msvc do this optimization (but I'm not
sure I got sufficient optimization flags in, I don't use either of those
myself).

 Philippe> Finally, not just the compiler version is important here: you
 Philippe> may have used some advanced (but experimental) optimizing
 Philippe> options on its command line

For both clang and gcc a simple -O is sufficient to show the issue.
Obviously if optimization is entirely disabled, the problem does not
occur.

 Philippe> My opinion is that your GCC compiler version has a bug or you
 Philippe> used some unsupported options or non-standard/untested patch,
 Philippe> or your Linux distribution is not reliable and delivers you
 Philippe> very badly compiled products: change your Linux update source
 Philippe> or switch to a reliable distribution.

Since I'm not even using linux, and the issue is easily demonstrated
with compiler comparison tools, your "opinion" here is based on false
assumptions and therefore worthless.

--
Andrew.

Reply | Threaded
Open this post in threaded view
|

Re: math.abs(math.mininteger) (Was: Must "attempt to get length of a number value" be an error?

Philippe Verdy
My opinion is worthwile, because you spoke about GCC and I correctly assumed you used Linux for it.

(there are very few projects on Windows that use GCC on Windows, given that MSVC is free for use now (even if its licence is not free/libre and its source not open), and well supported. And there are many other compilers on Windows. Most projects for Windows use Microsoft of Intel compilers (Intel compilers can be used as a backend integrated in the Dev Studio, which itself is now also free, but Intel compilers are among the best optimizing ones, but their licence is quite costly).

GCC on Windows is quite badly supported, it's compiled intself in an informal process like Mingw, which has lot of issues. And now that Windows 10 supports a native Linux environment (from Ubuntu), I think that when you speak about GCC on Windows, actually you use GCC on Ubuntu for Windows (that Microsoft initially called "Bash for Windows".. but its name is changing regularly, even if Microsoft really admits that its development is shared between Microsoft and Ubuntu that have cooperated to make adjustments in the Windows kernel to make better support of the Linux APIs, improve the security and compatiblity, including in core components like the console host, multithreading and scheduling, memory models, event queues, and filesystem support... for various things, this lead to patches being developped for Linux itself to debug some of its components and reduce the overhead of hypervisors like Hyper-V or others, that are now also better supporting Linux VMs and Windows VM concurrently, with these VMs being also easier to migrate between hypervisors based on Linux, Windows, or other systems).

With virtualization of OSes in rapid progress now, the underlying OS will no longer matter (except for performance or security goals). But today, security is a major need because of severe impacts of bugs (and performance is no longer a problem, and even storage space or memory resources is less critical: we have tons of them at cheap price). All developers now want compielrs that help them secure their code. A lot of them have abandoned C, and prefered working with much more managed or scripting languages (Java, Python, Javascript, even PHP) because they offer wide advantages (and are much easier to deploy and scale on many more platforms, rather than being "optimized" for a static architecture: JIT compilers, or install-time compilers now frequently offer better performance at run time on the final target host than statically compiled C code which may be perfect on a given machien, but not optimized at all on another: this explains the success of Java, C#, Javascript; the other reason is the mobile development, for Java-based VMs on Android and iOS).

And there is now a valid use of C compilers that no longer target a native machine code, but a virtual "bytecode" machine: the program will be recompiled to native on the final target host at deployment time, or with JIT compilers. And the final performance is good everywhere.

Le dim. 12 mai 2019 à 19:33, Andrew Gierth <[hidden email]> a écrit :
>>>>> "Philippe" == Philippe Verdy <[hidden email]> writes:

 >> Possibly unfortunately, this is not a bug. The program invokes
 >> undefined behavior when it does n=-n on the the value shown; the
 >> compiler is entitled to produce any output at all as a result.
 >>
 >> In this case it's fairly obvious what happened: after seeing the line
 >>
 >> if (n<0) n=-n;
 >>
 >> the compiler is entitled to assume that n>=0 is true regardless of
 >> the prior value of n, since only undefined behavior could cause that
 >> not to be the case and the compiler is allowed to assume that
 >> undefined behavior never occurs. So in the second fprintf, the
 >> compiler can optimize (n>=0) to a constant 1, and (n<0) to a
 >> constant 0, while still computing the remaining tests.

 Philippe> Such compiler assumption is wrong

The language standards disagree. Now you might choose to disagree with
those standards, but they are what they are.

 Philippe> But with your code I do not get the same result as you,

What exactly did you compile and how, and with what?

I don't keep a large supply of compilers on hand, but putting the code
on godbolt and looking at the output for various compiler versions, it's
clear that the optimization I described above is in effect for gcc
versions back to at least 4.9, and clang from 7.0 onwards (notably,
clang 6.0 does _not_ do it). See for example line 46 of the assembler
output of https://godbolt.org/z/vE8qCi where the "pushq $1" shows that
a constant 1 was pushed for the result of the >= test.

It _looks_ like neither icc nor msvc do this optimization (but I'm not
sure I got sufficient optimization flags in, I don't use either of those
myself).

 Philippe> Finally, not just the compiler version is important here: you
 Philippe> may have used some advanced (but experimental) optimizing
 Philippe> options on its command line

For both clang and gcc a simple -O is sufficient to show the issue.
Obviously if optimization is entirely disabled, the problem does not
occur.

 Philippe> My opinion is that your GCC compiler version has a bug or you
 Philippe> used some unsupported options or non-standard/untested patch,
 Philippe> or your Linux distribution is not reliable and delivers you
 Philippe> very badly compiled products: change your Linux update source
 Philippe> or switch to a reliable distribution.

Since I'm not even using linux, and the issue is easily demonstrated
with compiler comparison tools, your "opinion" here is based on false
assumptions and therefore worthless.

--
Andrew.

Reply | Threaded
Open this post in threaded view
|

Re: math.abs(math.mininteger) (Was: Must "attempt to get length of a number value" be an error?

Lorenzo Donati-3
In reply to this post by Philippe Verdy
On 12/05/2019 18:37, Philippe Verdy wrote:

> Interesting article : it finally shows that compilers are made by people
> that are interested only in compilers and prure performance, and don't even
> care about making it useful for any language.
>
> Now if C99 allows this, then C99 is not a language defined for programmers,
> jsut for the interests of compiler makers. It just kills the language and
> makes it completely useless (and malicious) to any one: programmers, and
> final users (which are not even given any right to know what is in the
> program they buy or use, these programs are also best to categorize are
> malwares).
>
> In summary, C is a malware to eradicate, unless it is implemented by
> replacing all undefined behaviors from the so called "standard" by defined
> behavior.
>

[snip]


I'm not a big fan of C (and C++ either: C++ it has almost the same
definition of UB as C), but I had to reconsider my views in the last 5
years, since I had to cope with microcontroller (MCU) software development.

I agree that, in an ideal world, no sane high level application should
be developed in C (or C++) because they are intrinsically unsafe, thanks
to UB. C++ mitigates that a little by providing high level constructs
that a programmer can use to avoid the most common cases where a mistake
can cause UB (e.g. dynamic memory management).

There could be, and probably there are, better, and safer, alternatives
for high level application development that don't sacrifice performance.

The problem is that at this stage there is no real alternative in the
following cases:

(1). You must heavily interact with OS API.

(2). You must write system-level software (e.g. device drivers).

(3). You must write SW for the bare metal (no OS, like on MCUs or
DSP-Digital Signal Processors).

(4). You must absolutely squeeze the last drop of performance from a
machine (without resorting to assembly), e.g. lots of heavy math
computations.

(5). Economics: the sheer number of /well tested/ code lines written in
C and C++ are a huge investment that firms are not going to replace
unless they are compelled by reasons that have the same "economic weight".


(1) and (2) stems from the fact that no OS I'm aware of is written in a
standardized language different from C/C++. So even if an OS provides an
API for other languages, the C/C++ API is going to be /the/ real native
API (which would be the only way to do really anything the OS allows you
to do, as far as API operations go).


(3) AFAIK C/C++ are the only widely used standardized languages that
allows assembly code to interface nicely (through compiler extensions)
with embedded assembly code. This is essential for low-level programming
to access specific CPU/MCU features that cannot be represented in C/C++
code (eg. switching memory banks, accessing bit-level instructions, etc.).

Moreover, it seems that writing a C compiler is not that hard, so a
CPU/MCU manufacturer can easily come up with a decent C compiler fairly
quickly once they develop a new architecture (in the high-end CPU world
we are accustomed to a handful of architectures, but in embedded world
there are dozens of architectures with thousands of different MCU
models, from 8bit to 64bits, not counting DSP chips!!!).

If you want to go OSS it's even easier: you write a new back-end for GCC
and you have a good C optimizing compiler without much effort (see GCC
for AVR architecture, for example)!

In the embedded world there are also examples of "mini C"
implementations, i.e. stripped-down, non-standard, C dialects, for which
a barebone compiler is even easier to write!

I know of only three "real" attempts to build a system language that has
the same range of applications as C/C++ and the same performance and are
safer at the same time: Go, Rust and Ada. I said "real" because (AFAIK)
they are backed by either big money or real effort and have gained
enough momentum to be, at least remotely, a viable alternative.

I know almost nothing about Rust, I've just read their FAQ some time
ago, just for curiosity, so I cannot comment further.

I had a stab at Go some years ago, but I think it failed its "mission"
to replace C as a system language. It is a nice language, but the
biggest drawback is that its runtime support system (AFAIR) is huge
(hundreds of MBs, IIRC), so that rules out using it on most embedded
systems. I don't know how it is now, since I haven't followed its
evolution, but I doubt they ever tried to target the non-PC world.

The third attempt, which was somewhat successful, was Ada. A language
sponsored by USA Department of Defense (DoD). At a certain point in time
(IIRC in the 90s) DoD mandated that any software that would run on its
systems (from database SW to avionics firmware) be written in Ada.

I read that after about 10 years they had to reconsider that position:
the cost of military SW development had skyrocketed because of that
constraint. They realized that it cost less to write SW in C/C++ and
then put money in more testing and safety frameworks. In fact it turned
out that an expert Ada programmer was extremely expensive to hire and
also to train (AFAIK Ada is a very complex language and it is extremely
fussy) compared to a C programmer of the same expertise. Moreover, they
were much harder to find (who writes SW in Ada outside USA
government?!?). And in addition to that the supporting tools for Ada
were worse or non-existent.


The last point, (5), is the most compelling though: economics. Sadly
economics trumps engineering almost always in the real world (as the DoD
Ada example shows). Companies are pumping billions of dollars into
software written in C/C+, so it makes sense for them to continue to
support their tools.

I've been said that there are some companies that still support their
old SW written in Cobol because the sheer amount of debugging/testing
put into that programs (banking SW) is worth millions of dollars and the
code base is now virtually bug-free and they are happy to overpay the
now rare Cobol programmers just to do some enhancement here and there.

Add to all this that C is an easy language to learn (alas, it's also
easy to learn wrong!). It's like a jackhammer: it is fairly easy to
understand how it works, so anyone could try to use it. And like a
jackhammer, it takes time to learn how to use it correctly without
risking of smashing your own feet!

So, in the end, I don't think we will see any real, practical
alternative to C any time soon (I'd say, sadly).

I'd love to hear something about Rust from people on this list, though.
Has someone used it, especially if it is usable in embedded world, and
how it compares to C.

Cheers!

-- Lorenzo





Reply | Threaded
Open this post in threaded view
|

Re: math.abs(math.mininteger) (Was: Must "attempt to get length of a number value" be an error?

Will Crowder
On Mon, 2019-05-13 at 09:49 +0200, Lorenzo Donati wrote:

[snip]
I'm not a big fan of C (and C++ either: C++ it has almost the same 
definition of UB as C), but I had to reconsider my views in the last 5 
years, since I had to cope with microcontroller (MCU) software development.

I agree that, in an ideal world, no sane high level application should 
be developed in C (or C++) because they are intrinsically unsafe, thanks 
to UB. C++ mitigates that a little by providing high level constructs 
that a programmer can use to avoid the most common cases where a mistake 
can cause UB (e.g. dynamic memory management).
C in the right hands (e.g., folks who understand language standards and how to avoid or allow for undefined behavior) is incredibly useful for high level applications, especially where performance is an issue.  It's a matter of understanding the tools you're using.  And in fact, many of the "higher" level languages have implementations written in C.  Does that mean it would be "insane" to code a "high level application" in any language whose underlying implementation was written in C?  I'd think not.

Will

123