[ANN] Ravi-Distro 0.5 alpha release

classic Classic list List threaded Threaded
26 messages Options
12
Reply | Threaded
Open this post in threaded view
|

[ANN] Ravi-Distro 0.5 alpha release

Dibyendu Majumdar
Ravi-Distro (https://github.com/dibyendumajumdar/ravi-distro) aims to
be a small and focused distribution of Lua 5.3 (www.lua.org) and Ravi
5.3 (https://github.com/dibyendumajumdar/ravi). In this release binary
distributions of Ravi and Lua 5.3 for Windows 64-bit are provided. The
packages included in this release are:

* lua 5.3.5
* Ravi with OMR JIT backend (updated!)
* luafilesystem
* lua-protobuf (new!)
* luasocket
* lpeglabel
* torch7
* torch7-nn
* torch7-optim
* torch7-autograd
* torch7-cephes
* penlight
* moses
* lua-cjson
* luaffi
* luv (libuv binding)
* nj (JIT engine based on Eclipse OMR)

Please visit the release page below for further information.

https://github.com/dibyendumajumdar/ravi-distro/releases

All feedback welcome!

Thanks and Regards
Dibyendu

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Ravi-Distro 0.5 alpha release

Egor Skriptunoff-2
On Mon, Nov 12, 2018 at 12:36 AM Dibyendu Majumdar wrote:
In this release binary
distributions of Ravi and Lua 5.3 for Windows 64-bit are provided.
https://github.com/dibyendumajumdar/ravi-distro/releases
All feedback welcome!

I'm expecting to find ravi-distro-0.5-windows-64bit.zip, but this file is missing.

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Ravi-Distro 0.5 alpha release

Dibyendu Majumdar
On Tue, 13 Nov 2018 at 20:11, Egor Skriptunoff
<[hidden email]> wrote:

>
> On Mon, Nov 12, 2018 at 12:36 AM Dibyendu Majumdar wrote:
>>
>> In this release binary
>> distributions of Ravi and Lua 5.3 for Windows 64-bit are provided.
>> https://github.com/dibyendumajumdar/ravi-distro/releases
>> All feedback welcome!
>
>
> I'm expecting to find ravi-distro-0.5-windows-64bit.zip, but this file is missing.
>

Yes, apologies will upload tonight.

Thanks and Regards
Dibyendu

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Ravi-Distro 0.5 alpha release

Dibyendu Majumdar
In reply to this post by Egor Skriptunoff-2
On Tue, 13 Nov 2018 at 20:11, Egor Skriptunoff
<[hidden email]> wrote:

>
> On Mon, Nov 12, 2018 at 12:36 AM Dibyendu Majumdar wrote:
>>
>> In this release binary
>> distributions of Ravi and Lua 5.3 for Windows 64-bit are provided.
>> https://github.com/dibyendumajumdar/ravi-distro/releases
>> All feedback welcome!
>
>
> I'm expecting to find ravi-distro-0.5-windows-64bit.zip, but this file is missing.
>

Hi, this is now available. The reason for the delay was that I found
that the release build was crashing a few tests. I believe  the
failures are related to the general problem with stack unwinding when
longjmp is used in a Win64 process; it requires some special actions
by the compiler. However the JIT engines do not tend to set this up
properly. Please let me know if you have noticed such failures.

Thanks and Regards
Dibyendu

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Ravi-Distro 0.5 alpha release

Egor Skriptunoff-2
On Wed, Nov 14, 2018 at 4:34 AM Dibyendu Majumdar wrote:
>
> I'm expecting to find ravi-distro-0.5-windows-64bit.zip, but this file is missing.
>

Hi, this is now available.


OK, let's test it!

1)
A bug: correct Lua 5.3 program doesn't run on Ravi.
   C:\Software\ravi\bin>ravi -e"for x = 1, 2 do x = 42.0 end"
   ravi: (command line):1: Invalid assignment: integer expected near 'end'
 
2)
Ravi shows some warnings in JIT mode.  (Nevertheless, the program works correctly despite of the warnings)
   buffer:2113:1: warning: constant 4023233417 is so big it is long long
   buffer:2113:1: warning: decimal constant 4023233417 is between LONG_MAX and ULONG_MAX. For C99 that means long long, C90 compilers are very likely to produce unsigned long (and a warning) here
   buffer:2115:1: warning: constant 2562383102 is so big it is long long
   buffer:2115:1: warning: decimal constant 2562383102 is between LONG_MAX and ULONG_MAX. For C99 that means long long, C90 compilers are very likely to produce unsigned long (and a warning) here
   buffer:2119:1: warning: constant 3285377520 is so big it is long long
   buffer:2119:1: warning: decimal constant 3285377520 is between LONG_MAX and ULONG_MAX. For C99 that means long long, C90 compilers are very likely to produce unsigned long (and a warning) here
   buffer:2190:1: warning: constant 4294967296 is so big it is long long
   buffer:2370:1: warning: constant 4294967297 is so big it is long long
   buffer:2946:1: warning: constant 1099511627776 is so big it is long long
 
3)
Most frustrating problem: compiled functions run slower than not compiled (what am I doing wrong?).
Simple example (compiled function takes 28 seconds, non-compiled 13 seconds):
 
   C:\Software\ravi\bin>type program.lua
      local function f()
         local tm = os.clock()
         local o = 0
         for j = 1, 1e4 do
            local x = 0
            for k = 1, 1e5 do
               x = x ~ (j + k)
            end
            o = o | x
         end
         print("   Result = "..o)
         print("   CPU time = "..(os.clock() - tm))
      end
      print"Benchmarking non-compiled function"
      f()
      print"Compiling the function"
      assert(ravi.compile(f))
      print"Benchmarking compiled function"
      f()
 
   C:\Software\ravi\bin>ravi program.lua
      Benchmarking non-compiled function
         Result = 114656
         CPU time = 13.425
      Compiling the function
      Benchmarking compiled function
         Result = 114656
         CPU time = 28.568

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Ravi-Distro 0.5 alpha release

Dibyendu Majumdar
On Wed, 14 Nov 2018 at 19:55, Egor Skriptunoff
<[hidden email]> wrote:
>
>
> OK, let's test it!
>
> 1)
> A bug: correct Lua 5.3 program doesn't run on Ravi.
>    C:\Software\ravi\bin>ravi -e"for x = 1, 2 do x = 42.0 end"
>    ravi: (command line):1: Invalid assignment: integer expected near 'end'
>

In this case Ravi is deciding that x is an integer type and hence
doesn't like that you are assigning a floating point value.

> 2)
> Ravi shows some warnings in JIT mode.  (Nevertheless, the program works correctly despite of the warnings)
>    buffer:2113:1: warning: constant 4023233417 is so big it is long long
>    buffer:2113:1: warning: decimal constant 4023233417 is between LONG_MAX and ULONG_MAX. For C99 that means long long, C90 compilers are very likely to produce unsigned long (and a warning) here
>    buffer:2115:1: warning: constant 2562383102 is so big it is long long
>    buffer:2115:1: warning: decimal constant 2562383102 is between LONG_MAX and ULONG_MAX. For C99 that means long long, C90 compilers are very likely to produce unsigned long (and a warning) here
>    buffer:2119:1: warning: constant 3285377520 is so big it is long long
>    buffer:2119:1: warning: decimal constant 3285377520 is between LONG_MAX and ULONG_MAX. For C99 that means long long, C90 compilers are very likely to produce unsigned long (and a warning) here
>    buffer:2190:1: warning: constant 4294967296 is so big it is long long
>    buffer:2370:1: warning: constant 4294967297 is so big it is long long
>    buffer:2946:1: warning: constant 1099511627776 is so big it is long long

Yes sorry about that - this is coming from the C parser - I will
switch off this warning.

>
> 3)
> Most frustrating problem: compiled functions run slower than not compiled (what am I doing wrong?).
> Simple example (compiled function takes 28 seconds, non-compiled 13 seconds):
>
>    C:\Software\ravi\bin>type program.lua
>       local function f()
>          local tm = os.clock()
>          local o = 0
>          for j = 1, 1e4 do
>             local x = 0
>             for k = 1, 1e5 do
>                x = x ~ (j + k)
>             end
>             o = o | x
>          end
>          print("   Result = "..o)
>          print("   CPU time = "..(os.clock() - tm))
>       end
>       print"Benchmarking non-compiled function"
>       f()
>       print"Compiling the function"
>       assert(ravi.compile(f))
>       print"Benchmarking compiled function"
>       f()
>
>    C:\Software\ravi\bin>ravi program.lua
>       Benchmarking non-compiled function
>          Result = 114656
>          CPU time = 13.425
>       Compiling the function
>       Benchmarking compiled function
>          Result = 114656
>          CPU time = 28.568
>

Okay - there are a few things here.
Firstly the JIT backend in ravi-distro is not inlining normal Lua
arithmetic / bitwise operators. Partly because I haven't got around to
doing this, and partly because I feel that inlining normal Lua ops
results in code bloat as each op has several branches.

I have changed your program as follows to help Ravi:

local function f()
   local tm = os.clock()
   local o: integer = 0
   for j = 1, 10000 do
      local x: integer = 0
      for k = 1, 100000 do
         x = x ~ (j + k)
      end
      o = o | x
   end
   print("   Result = "..o)
   print("   CPU time = "..(os.clock() - tm))
end
print"Benchmarking non-compiled function"
f()
print"Compiling the function"
-- ravi.dumplua(f)
assert(ravi.compile(f))
print"Benchmarking compiled function"
f()

Timings on Mac OSX:

Original code interpreted: 9.063722
Above code interpreted: 6.832995
For some reason the above modified version fails to JIT compile with
the OMR JIT backend; I need to investigate this.
Using LLVM backend original: 6.995143
Using LLVM backend above: 0.230904

So once type annotations are used, the LLVM backend can really improve
performance. The OMR backend should too, except that it is failing to
compile the code for some reason.

I realise that you may not want to annotate your code with types ... I
can try to amend your SHA2 code with types for Ravi if you like. It
will be a good piece of benchmark code for Ravi.

Thank you very much for sharing your findings - it really helps.

Regards
Dibyendu

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Ravi-Distro 0.5 alpha release

Dibyendu Majumdar
On Wed, 14 Nov 2018 at 20:55, Dibyendu Majumdar <[hidden email]> wrote:

>
> I have changed your program as follows to help Ravi:
>
> local function f()
>    local tm = os.clock()
>    local o: integer = 0
>    for j = 1, 10000 do
>       local x: integer = 0
>       for k = 1, 100000 do
>          x = x ~ (j + k)
>       end
>       o = o | x
>    end
>    print("   Result = "..o)
>    print("   CPU time = "..(os.clock() - tm))
> end

> For some reason the above modified version fails to JIT compile with
> the OMR JIT backend; I need to investigate this.

Okay its because in the OMR backend (included in Ravi Distro) I
haven't yet implemented the JITing of bitwise Ravi extension opcodes.
The standard Lua bitwise opcodes are handled via function calls in the
OMR backend, so they degrade performance.
The LLVM backend does handle all of them, although the Lua opcodes do
not tend to optimize well due to excessive branching.

I will look at adding the bitwise opcodes to the OMR backend.

Regards
Dibyendu

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Ravi-Distro 0.5 alpha release

Dibyendu Majumdar
In reply to this post by Egor Skriptunoff-2
On Wed, 14 Nov 2018 at 19:55, Egor Skriptunoff
<[hidden email]> wrote:
>       local function f()
>          local tm = os.clock()
>          local o = 0
>          for j = 1, 1e4 do
>             local x = 0
>             for k = 1, 1e5 do
>                x = x ~ (j + k)
>             end

What's interesting is that Lua's parser think 1e5 is floating point
... whereas 10000 is integer.
That's why I had to change the limit to 10000 in the type annotated version.

Regards

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Ravi-Distro 0.5 alpha release

Dibyendu Majumdar
In reply to this post by Egor Skriptunoff-2
On Wed, 14 Nov 2018 at 19:55, Egor Skriptunoff
<[hidden email]> wrote:

>    C:\Software\ravi\bin>type program.lua
>       local function f()
>          local tm = os.clock()
>          local o = 0
>          for j = 1, 1e4 do
>             local x = 0
>             for k = 1, 1e5 do
>                x = x ~ (j + k)
>             end
>             o = o | x
>          end
>          print("   Result = "..o)
>          print("   CPU time = "..(os.clock() - tm))
>       end
>       print"Benchmarking non-compiled function"
>       f()
>       print"Compiling the function"
>       assert(ravi.compile(f))
>       print"Benchmarking compiled function"
>       f()
>

Here are LLVM results on my Windows machine:
C:\work\github\ravi\ravi-tests>ravi egor_test.lua
Benchmarking non-compiled function
   Result = 114656
   CPU time = 11.709
Compiling the function
Benchmarking compiled function
   Result = 114656
   CPU time = 6.633

C:\work\github\ravi\ravi-tests>ravi egor_test_typed.lua
Benchmarking non-compiled function
   Result = 114656
   CPU time = 9.832
Compiling the function
Benchmarking compiled function
   Result = 114656
   CPU time = 0.311

I have uploaded a distribution that uses LLVM backend - please see
https://github.com/dibyendumajumdar/ravi-distro/releases

What you can see from above, and my experience is that, JIT compiling
plain Lua code will make code 2x faster at best; to get big
improvements  type annotations are needed.

Regards
Dibyendu

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Ravi-Distro 0.5 alpha release

Egor Skriptunoff-2
In reply to this post by Dibyendu Majumdar
On Wed, Nov 14, 2018 at 11:55 PM Dibyendu Majumdar wrote:
I realise that you may not want to annotate your code with types ... I
can try to amend your SHA2 code with types for Ravi if you like. It
will be a good piece of benchmark code for Ravi.
 
Yes, I'd like you to amend SHA2 with new branch of implementation for Ravi.
:-)


Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Ravi-Distro 0.5 alpha release

Philippe Verdy
In reply to this post by Dibyendu Majumdar


Le mer. 14 nov. 2018 à 21:55, Dibyendu Majumdar <[hidden email]> a écrit :
On Wed, 14 Nov 2018 at 19:55, Egor Skriptunoff
<[hidden email]> wrote:
>
>
> OK, let's test it!
>
> 1)
> A bug: correct Lua 5.3 program doesn't run on Ravi.
>    C:\Software\ravi\bin>ravi -e"for x = 1, 2 do x = 42.0 end"
>    ravi: (command line):1: Invalid assignment: integer expected near 'end'
>

In this case Ravi is deciding that x is an integer type and hence
doesn't like that you are assigning a floating point value.

That's because it attempts to optimize the integer loop unsafely: when compiling x=42.0, which changes the loop variable, it forgets to evaluate the loop condition (x<=42) which would normally imply that this assignment to x causes a break to the loop; but as this x variable is local to the loop and this assigned value is then not used after breaking, the assignment has no other effect than breaking the loop, so this code would be nearly like "for x = 1, 2 do break end" i.e. it will do nothing if compiled properly; if the code is interpreted, the for loop will be executed only once (local x=1 when entering the first loop which sets x=42.0, and exiting immediately which deletes the x variable).
There is code where modifying the loop control variable is perfectly valid, even if in general this is done by assigning a value of the same type.

A simple loop of the form "for x = a,b do... end" does not even require that "a" and "b" be the same type, it just needs that they are comparable ; the type assigned to "x" and incremented is the type of "a", not "b", the loop will compare "x <= b" after incrementing "x" which is the type of "a" (the type of the control variable "x" is not necessarily an "integer", actually in Lua it is just a "number");
the optimization to use integers has to be checked: you can optimize the compilation of "b" to an integer (if "a" is an integer) by compiling "for x = a,b do... end" as "for x = a,int(ceil(b)) do... end", then by compiling the inner assignment "x = 42.0" as "x=int(ceil(42.0)", i.e. "x=42".

But beware of the valid range of your optimized integers: Lua numbers have a wider range and allows a "for x=a,b" control variable to be any "number"; given the precision needed for incrementations to be working, the "a", and "b" numbers must be strictly between -2^53 and 2^53-1 (if "number" is compiled as an 64-bit IEEE "double precision floatting point") or between -2^24 and 2^24 (if "number" is compiled as an 32-bit IEEE "single precision floatting point"), otherwise the loop will be infinite. If your optimized code uses just a 32-bit integer, it may overflow when entering the loop or before reaching the end of the loop, or the end value "b" of the loop could be incorrectly truncated to be lower than "a" and the loop will never execute at all!

Writing  "for x = (a),(b) do (...) end" is only syntaxic sugar for: "do local x = (a); while x < (b) do (...); x = x + 1 end end"

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Ravi-Distro 0.5 alpha release

Philippe Verdy
Le jeu. 15 nov. 2018 à 12:54, Philippe Verdy <[hidden email]> a écrit :
Writing  "for x = (a),(b) do (...) end" is only syntaxic sugar for: "do local x = (a); while x < (b) do (...); x = x + 1 end end"
More exactly it is the equivalent of:  "do local x, __end__ = (a), (b); while x <=   __end__   do (...); x = x + 1 end end"
because (b) is evaluated only once before the first checking of the condition and then entering the first loop (there was also a missing "=" in the condition)

A compiler may check the type of the value (a) assigned to the control variable (x), if it is known (constant), but then it must:
* coerce the value (b) assigned to "__end__" to the same datatype using ceiling (not floor);
* check that the coercion of (b) in "local x,__end__ = (a),coerce(b)" does not cause an overflow (if this occurs, the loop cannot be optimized using integers, it has to use a double floatting point for the control variable...);
* coerce the incrementation "x = x + 1", as "x =  successor(coerce(x))", if (a) was an integer type, so that the condition "x <= __end__" remains correct;
* check the possible integer overflow of this incrementation (if overflow occurs, this must break the loop immediately: an overflow can occur here only if "successor(__end__)" cannot be warrantied to be inside the valid range of the chosen datatype, or if the current value of "x" which was changed in the middle of the loop is outside the inclusive integer range from (a) to (b));
* compile the code inside the loop so that any reference to "x" will "uncoerce" its chosen integer type to the full range of "number" when needed (notably when using the value of "x" in function calls, but as well in simple expressions like "x*x"...)

A compiler may also want to unwind such loops if the code inside the loop is small, and if the difference between (a) and (b) is known (both (a) and (b) are constants) and small (generally not more than 4, but this may depend on the code size inside the loop; this is complex to do if that Lua code inside the loop contains other execution controls, requiring the generation of labels for generated jumps into the generated opcodes, or if the compiler makes special optimizations for exception handling, i.e. to optimize invokations in Lua code to "pcall(function,...)").
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Ravi-Distro 0.5 alpha release

Philippe Verdy
it should also be noted that the Lua specification states that "you should never change the value of the control variable: The effect of such changes is unpredictable. If you want to break a forloop before its normal termination, use break."

What this means is that changing this value "may or may not" have a side effect on the number of loops executed: a compiler may validly create an iterator from (a) to (b) inclusively, and then the enumerator will return values into the local variable "x", ignoring and overwriting any previously assigned value inside the loop.

So the code "for x=1,3 do x=x+.5; print(x, ' ') end" may print either:
- "1.5 2.5 3.5" (using a prebuilt iterator and ignoring changes to the control variable, i.e. using a stateless loop) or
- "1.5 3" (taking the current value of the control variable into account, i.e. using a stateful loop).

As Lua is elusive about the expected behavior, the compiler should emit a strong warning for such assignments to local control variables (I think that compilers should behave the best using the first behavior using a stateless loop).

My opinion is that Lua should have been defined so that only the first (stateless) behavior is valid, and so "for x=(a),(b) do (...) end" should always behave like "for x in foriter(a, b) do (...) end"


Le jeu. 15 nov. 2018 à 13:57, Philippe Verdy <[hidden email]> a écrit :
Le jeu. 15 nov. 2018 à 12:54, Philippe Verdy <[hidden email]> a écrit :
Writing  "for x = (a),(b) do (...) end" is only syntaxic sugar for: "do local x = (a); while x < (b) do (...); x = x + 1 end end"
More exactly it is the equivalent of:  "do local x, __end__ = (a), (b); while x <=   __end__   do (...); x = x + 1 end end"
because (b) is evaluated only once before the first checking of the condition and then entering the first loop (there was also a missing "=" in the condition)

A compiler may check the type of the value (a) assigned to the control variable (x), if it is known (constant), but then it must:
* coerce the value (b) assigned to "__end__" to the same datatype using ceiling (not floor);
* check that the coercion of (b) in "local x,__end__ = (a),coerce(b)" does not cause an overflow (if this occurs, the loop cannot be optimized using integers, it has to use a double floatting point for the control variable...);
* coerce the incrementation "x = x + 1", as "x =  successor(coerce(x))", if (a) was an integer type, so that the condition "x <= __end__" remains correct;
* check the possible integer overflow of this incrementation (if overflow occurs, this must break the loop immediately: an overflow can occur here only if "successor(__end__)" cannot be warrantied to be inside the valid range of the chosen datatype, or if the current value of "x" which was changed in the middle of the loop is outside the inclusive integer range from (a) to (b));
* compile the code inside the loop so that any reference to "x" will "uncoerce" its chosen integer type to the full range of "number" when needed (notably when using the value of "x" in function calls, but as well in simple expressions like "x*x"...)

A compiler may also want to unwind such loops if the code inside the loop is small, and if the difference between (a) and (b) is known (both (a) and (b) are constants) and small (generally not more than 4, but this may depend on the code size inside the loop; this is complex to do if that Lua code inside the loop contains other execution controls, requiring the generation of labels for generated jumps into the generated opcodes, or if the compiler makes special optimizations for exception handling, i.e. to optimize invokations in Lua code to "pcall(function,...)").
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Ravi-Distro 0.5 alpha release

Dirk Laurie-2
Op Do. 15 Nov. 2018 om 15:23 het Philippe Verdy <[hidden email]> geskryf:

> it should also be noted that the Lua specification states that "you should never change the value of the control variable: The effect of such changes is unpredictable. If you want to break a forloop before its normal termination, use break."

I cannot find this quotation in the Lua 5.3 Referebce Manual.

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Ravi-Distro 0.5 alpha release

Albert Chan
In reply to this post by Philippe Verdy
On Nov 15, 2018, at 8:23 AM, Philippe Verdy <[hidden email]> wrote:

> My opinion is that Lua should have been defined so that only the first (stateless) behavior is valid, and so "for x=(a),(b) do (...) end" should always behave like "for x in foriter(a, b) do (...) end"

Does any version of Lua *not* do this way ?
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Ravi-Distro 0.5 alpha release

Philippe Verdy
Not all, according to the Lua doc in section 4.3.4 (Numeric for), the behavior is unspecified. I bet that the interpreter mode uses the 2nd approach which is simpler, but may give different results.
This is stated at end of this page:

Le jeu. 15 nov. 2018 à 14:46, Albert Chan <[hidden email]> a écrit :
On Nov 15, 2018, at 8:23 AM, Philippe Verdy <[hidden email]> wrote:

> My opinion is that Lua should have been defined so that only the first (stateless) behavior is valid, and so "for x=(a),(b) do (...) end" should always behave like "for x in foriter(a, b) do (...) end"

Does any version of Lua *not* do this way ?
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Ravi-Distro 0.5 alpha release

Philippe Verdy
What is even worse is that the same page states that " Second, the control variable is a local variable automatically declared by the for statement and is visible only inside the loop".
Which means that the local variable (visible inside the loop) and the control variable (for the loop itself, which should be completely hidden in the iterator, so that the local variable in the loop only gets a *copy* of the value in the hidden control variable of the iterator) are confused in that statement as if they were the same (which is an error in my opinion).

I don't know which approach is valid, the doc is clearly ambiguous.

The behavior should be specified clearly in the doc by showing, like in my example:
  for x=1,3 do x = x+0.5; print(x, ' ') end
that it MUST print "1.5 2.5 3.5", and NOT "1.5 3".

This stateless approach will then clearly ease the implementation of all compilers and interpreters, with predictable results, and no need to track in the source code of the inner loop if a "control variable" is assigned (this should have no effect at all on the number of loops executed, so that the number of loops is fully predictable before the first loop starts, and only the "break" instruction can interrupt the loops before the iterator reaches the ending condition).

As long as this behavior is unspecified, a compiler should emit a strong warning, if ever it finds inside the loop an assignment to the "local variable" declared by the "for" statement, which is not necessarily the control variable of the hidden iterator.

I'm not sure that there's really an iterator object instantiated to contain the control variable, the maximum value to compare with the control variable, and the stepping value; the doc just indicates that there are two hidden variables for the end value and the stepping value, and a compiler may also want to avoid creating a real iterator object, and would prefer just generating 1 hidden variables (or 2 with the stepping value), leaving the control variable directly usable as the local variable of the loop). With the stateless approach, a compiler would need to generate 2 hidden variables (or 3 with the stepping value) separately from the local variable visible inside the loop code).

When the stepping value is negative, the completion test is reversed: the "end value" is a minimal value so the condition is "while __control__>= __end__ do..." instead of ""while __control__<= __end__ do...", so there are actually two kinds of numeric iterators generated (forward or backward).

The doc is not clear about the behavior of a numeric for loop, when the optional stepping value is zero: "for i=1,10,0 do...end" cannot be determined to be a forward or backward iterator, but logically this is an infinite loop, given that the values in the two first parameters (1,10) are both inclusive.

Le jeu. 15 nov. 2018 à 17:03, Philippe Verdy <[hidden email]> a écrit :
Not all, according to the Lua doc in section 4.3.4 (Numeric for), the behavior is unspecified. I bet that the interpreter mode uses the 2nd approach which is simpler, but may give different results.
This is stated at end of this page:

Le jeu. 15 nov. 2018 à 14:46, Albert Chan <[hidden email]> a écrit :
On Nov 15, 2018, at 8:23 AM, Philippe Verdy <[hidden email]> wrote:

> My opinion is that Lua should have been defined so that only the first (stateless) behavior is valid, and so "for x=(a),(b) do (...) end" should always behave like "for x in foriter(a, b) do (...) end"

Does any version of Lua *not* do this way ?
Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Ravi-Distro 0.5 alpha release

Francisco Olarte
Philippe:

I'm a little lost in the discussion, due to he bottom quotes and
similars.  You seem to complain of unspecified behaviour of numeric
for loops in PIL and friends, and...

On Thu, Nov 15, 2018 at 5:38 PM, Philippe Verdy <[hidden email]> wrote:

> What is even worse is that the same page states that " Second, the control
> variable is a local variable automatically declared by the for statement and
> is visible only inside the loop".
> Which means that the local variable (visible inside the loop) and the
> control variable (for the loop itself, which should be completely hidden in
> the iterator, so that the local variable in the loop only gets a *copy* of
> the value in the hidden control variable of the iterator) are confused in
> that statement as if they were the same (which is an error in my opinion).
>
> I don't know which approach is valid, the doc is clearly ambiguous.
>
> The behavior should be specified clearly in the doc by showing, like in my
> example:
>   for x=1,3 do x = x+0.5; print(x, ' ') end
> that it MUST print "1.5 2.5 3.5", and NOT "1.5 3".

I've read the ref manual ( which IMHO is the place to go for this kind
of things ) ( https://www.lua.org/manual/5.3/manual.html#3.3.5 )

and It gives some equivalent code which seems to clearly define both
the above-quote loop behaviour and the 0-increment case discussed
below.

> This stateless approach will then clearly ease the implementation of all
> compilers and interpreters, with predictable results, and no need to track
> in the source code of the inner loop if a "control variable" is assigned
> (this should have no effect at all on the number of loops executed, so that
> the number of loops is fully predictable before the first loop starts, and
> only the "break" instruction can interrupt the loops before the iterator
> reaches the ending condition).

The code in ref manual for 5.3 seems to define exactly this behaviour.

Francisco Olarte

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Ravi-Distro 0.5 alpha release

Andrew Gierth
In reply to this post by Philippe Verdy
>>>>> "Philippe" == Philippe Verdy <[hidden email]> writes:

 Philippe> What is even worse is that the same page states

Why are you reading the online copy of PiL, which is for a long-obsolete
version of Lua, rather than the _actual documentation_?

https://www.lua.org/manual/5.3/manual.html#3.3.5

This makes it perfectly clear that:

 - the actual loop iteration variables are hidden and not accessible

 - any changes to the visible loop variable in the loop body have no
   effect beyond the end of the current iteration

 Philippe> The doc is not clear about the behavior of a numeric for
 Philippe> loop, when the optional stepping value is zero

The actual manual (as linked above) is precisely clear about this.

--
Andrew.

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] Ravi-Distro 0.5 alpha release

Egor Skriptunoff-2
In reply to this post by Dibyendu Majumdar
On Thu, Nov 15, 2018 at 1:32 AM Dibyendu Majumdar wrote:
>          for j = 1, 1e4 do

That's why I had to change the limit to 10000 in the type annotated version.

 
In Lua 5.3, in numeric "for"-loop, if both "initial value" and "step" are integers, then loop variable will have only integer values, even if "limit" is float.
 
for j = 1, math.pi, 2 do
   print(j, math.type(j))
end
 
So, when parsing "for j = 1, 1e4 do", Ravi should conclude that "j" is integer, similar to how Lua 5.3 deduces type of loop variable.
This way there would be no reason to replace 1e4 with 10000

12