Iterator construction

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Iterator construction

astient
Hi,

I'm stuck in my understanding of the way an iterator works. I found already some explanations in the lua doc but I don't understand why when calling a function by the "in" command the behavior of the variable in the function is different (ie: defined as local but behaves as global) :

-----------------------------------------------------------
   local table={1,2,3}           
   
   function iterator(table)
       local y = 0                             
       return function() y=y+1 print(y) return table[y] end     
   end
     
   -- 1) function iterator called by "for":  
     
         for i = 1,3 do
             iterator(table)()
         end
                 
    -- 2) function iterator call by "in":
     
          for i in iterator(table) do end
-------------------------------------------------------------          
   
The 2 loops call the same function, but in the first case the local variable y is not retained (which is the normal behavior of local variable as i understand it), so the value of y stays at 1.

But in the second case, the variable y is retained by the "magic" of "in" command and thus can be iterated from 1 to 3, although it is the same function.

If it possible to explain it simply (i'm not very skilled in lua so far) could you please explain me what happens really when using the "in" magic. Is this a way to make the local variable (like y here) of a function provisionally global, so that it can retains its value during the different calls to it, and when the "in loop" is finished it is taken back local again ?

Maybe this is done by assigning this local variable to a global one that is defined provisionally during the use of the "in loop" (and delete at the end of the loop), and when looping in fact you use this global variable instead of the local one ?

Also i found a particular syntax for iterator's use that i don't understand and I didn't find documentation on it :

-----------------------------------------------
function square(iteratorMaxCount,tot)
   local currentNumber=tot
      if currentNumber<iteratorMaxCount then
         currentNumber = currentNumber+1
         return currentNumber, currentNumber*currentNumber
      end
end

for i,n in square,3,0 do
  print(i,n)
end
----------------------------------------------

What means this particular way to call the function square: "square,3,0" ?
apparently this is not equivalent to "square(3,0)" as this syntax make an error.


Thank you very much for your help

Reply | Threaded
Open this post in threaded view
|

Re: Iterator construction

Michael Welsh Duggan
<[hidden email]> writes:

> I'm stuck in my understanding of the way an iterator works. I found
> already some explanations in the lua doc but I don't understand why
> when calling a function by the "in" command the behavior of the
> variable in the function is different (ie: defined as local but
> behaves as global) :
>
> -----------------------------------------------------------
> local table={1,2,3}
>
> function iterator(table)
> local y = 0
> return function() y=y+1 print(y) return table[y] end
> end
>
> -- 1) function iterator called by "for":
>
> for i = 1,3 do
> iterator(table)()
> end
>
> -- 2) function iterator call by "in":
>
> for i in iterator(table) do end
> -------------------------------------------------------------
>
> The 2 loops call the same function, but in the first case the local
> variable y is not retained (which is the normal behavior of local
> variable as i understand it), so the value of y stays at 1.
>
> But in the second case, the variable y is retained by the "magic" of
> "in" command and thus can be iterated from 1 to 3, although it is the
> same function.
>
> If it possible to explain it simply (i'm not very skilled in lua so
> far) could you please explain me what happens really when using the
> "in" magic. Is this a way to make the local variable (like y here) of
> a function provisionally global, so that it can retains its value
> during the different calls to it, and when the "in loop" is finished
> it is taken back local again ?

Simplest explanation is this.  In the (1) case, you are creating a new
iterator each time though the loop, in which its y starts at zero.  In
the (2) case, it creates the iterator function once, and then uses each
time through the loop.  (The function named 'iterator' gets called once,
the function 'iterator' returns is called once per loop.)  To make the
(1) case similar, it should be written like this:

x = iterator(table)
for i = 1, 3 do
   x()
end

--
Michael Welsh Duggan
([hidden email])

Reply | Threaded
Open this post in threaded view
|

Re: Iterator construction

Javier Guerra Giraldez
In reply to this post by astient
On Wed, Aug 13, 2014 at 8:22 AM,  <[hidden email]> wrote:
> The 2 loops call the same function, but in the first case the local variable
> y is not retained (which is the normal behavior of local variable as i
> understand it), so the value of y stays at 1.

the first case:

         for i = 1,3 do
             iterator(table)()
         end

calls "iterator(table)" three times, and after each one, it calls the
returned function once.

in your iterator() function you define "local y=0", so the variable is
defined and assigned each time iterator() is called.  the inner
function sees that value but only once, since it's called only once
before iterator() is called again and redefines y.


> But in the second case, the variable y is retained by the "magic" of "in"
> command and thus can be iterated from 1 to 3, although it is the same
> function.


the "in" syntax isn't magic, just a little complex.  it evaluates the
expression between the "in" and "do" keywords and expects to get up to
three values which are then used to manage the iteration.

in your second case:

          for i in iterator(table) do end

the expression "iterator(table)" is called _once_ to initiate the
loop, and then it gets the inner function, two other values (nil in
this case) to drive the loop, so the inner function gets called
several times and it sees the increasing values of "y"

from the docs:

"""
A for statement like

     for var_1, ···, var_n in explist do block end
is equivalent to the code:

     do
       local f, s, var = explist
       while true do
         local var_1, ···, var_n = f(s, var)
         if var_1 == nil then break end
         var = var_1
         block
       end
     end
"""
(where f, s and var internal variables, invisible to Lua code)

so, your one-line loop is transformed into

     do
       local f, s, var = iterator(table)
       while true do
         local i = f(s, var)
         if i == nil then break end
         var = i
         -- empty block
       end
     end

you can see the only call to iterator() is outside the loop, but the
local pseudovariable f is set to your inner function and then it's
called several times.

--
Javier

Reply | Threaded
Open this post in threaded view
|

Re: Iterator construction

Mason Mackaman
In reply to this post by astient
I would read this: http://www.lua.org/manual/5.2/manual.html#3.3.5. That’s how I learned how the generic for loop works.
On Aug 13, 2014, at 8:22 AM, [hidden email] wrote:

Hi,

I'm stuck in my understanding of the way an iterator works. I found already some explanations in the lua doc but I don't understand why when calling a function by the "in" command the behavior of the variable in the function is different (ie: defined as local but behaves as global) :

-----------------------------------------------------------
   local table={1,2,3}           
   
   function iterator(table)
       local y = 0                             
       return function() y=y+1 print(y) return table[y] end     
   end
     
   -- 1) function iterator called by "for":  
     
         for i = 1,3 do
             iterator(table)()
         end
                 
    -- 2) function iterator call by "in":
     
          for i in iterator(table) do end
-------------------------------------------------------------          
   
The 2 loops call the same function, but in the first case the local variable y is not retained (which is the normal behavior of local variable as i understand it), so the value of y stays at 1.

But in the second case, the variable y is retained by the "magic" of "in" command and thus can be iterated from 1 to 3, although it is the same function.

If it possible to explain it simply (i'm not very skilled in lua so far) could you please explain me what happens really when using the "in" magic. Is this a way to make the local variable (like y here) of a function provisionally global, so that it can retains its value during the different calls to it, and when the "in loop" is finished it is taken back local again ?

Maybe this is done by assigning this local variable to a global one that is defined provisionally during the use of the "in loop" (and delete at the end of the loop), and when looping in fact you use this global variable instead of the local one ?

Also i found a particular syntax for iterator's use that i don't understand and I didn't find documentation on it :

-----------------------------------------------
function square(iteratorMaxCount,tot)
   local currentNumber=tot
      if currentNumber<iteratorMaxCount then
         currentNumber = currentNumber+1
         return currentNumber, currentNumber*currentNumber
      end
end

for i,n in square,3,0 do
  print(i,n)
end
----------------------------------------------

What means this particular way to call the function square: "square,3,0" ?
apparently this is not equivalent to "square(3,0)" as this syntax make an error.


Thank you very much for your help


Reply | Threaded
Open this post in threaded view
|

Re: Iterator construction

Mason Mackaman
To add on to what I said, you really need to know the basics of BNF notation, which is used to show how the syntax works in the manual. They talk about it here http://www.lua.org/manual/5.2/manual.html#3. One thing they don’t say, which you can pick up from how it’s used, is ::= means equivalent to. For example, block::={stat} means that a block is 0 or more stat[ments]s. Not trying to assume you don’t already know this, but wrote it incase you didn’t.
On Aug 13, 2014, at 11:19 AM, Mason Mackaman <[hidden email]> wrote:

I would read this: http://www.lua.org/manual/5.2/manual.html#3.3.5. That’s how I learned how the generic for loop works.
On Aug 13, 2014, at 8:22 AM, [hidden email] wrote:

Hi,

I'm stuck in my understanding of the way an iterator works. I found already some explanations in the lua doc but I don't understand why when calling a function by the "in" command the behavior of the variable in the function is different (ie: defined as local but behaves as global) :

-----------------------------------------------------------
   local table={1,2,3}           
   
   function iterator(table)
       local y = 0                             
       return function() y=y+1 print(y) return table[y] end     
   end
     
   -- 1) function iterator called by "for":  
     
         for i = 1,3 do
             iterator(table)()
         end
                 
    -- 2) function iterator call by "in":
     
          for i in iterator(table) do end
-------------------------------------------------------------          
   
The 2 loops call the same function, but in the first case the local variable y is not retained (which is the normal behavior of local variable as i understand it), so the value of y stays at 1.

But in the second case, the variable y is retained by the "magic" of "in" command and thus can be iterated from 1 to 3, although it is the same function.

If it possible to explain it simply (i'm not very skilled in lua so far) could you please explain me what happens really when using the "in" magic. Is this a way to make the local variable (like y here) of a function provisionally global, so that it can retains its value during the different calls to it, and when the "in loop" is finished it is taken back local again ?

Maybe this is done by assigning this local variable to a global one that is defined provisionally during the use of the "in loop" (and delete at the end of the loop), and when looping in fact you use this global variable instead of the local one ?

Also i found a particular syntax for iterator's use that i don't understand and I didn't find documentation on it :

-----------------------------------------------
function square(iteratorMaxCount,tot)
   local currentNumber=tot
      if currentNumber<iteratorMaxCount then
         currentNumber = currentNumber+1
         return currentNumber, currentNumber*currentNumber
      end
end

for i,n in square,3,0 do
  print(i,n)
end
----------------------------------------------

What means this particular way to call the function square: "square,3,0" ?
apparently this is not equivalent to "square(3,0)" as this syntax make an error.


Thank you very much for your help



Reply | Threaded
Open this post in threaded view
|

Re: Iterator construction

Mason Mackaman
Also things separated between vertical bars ‘|’ means you pick one of them and a string between apostrophes mean that literal string. I think that covers everything used in the manual.
On Aug 13, 2014, at 11:26 AM, Mason Mackaman <[hidden email]> wrote:

To add on to what I said, you really need to know the basics of BNF notation, which is used to show how the syntax works in the manual. They talk about it here http://www.lua.org/manual/5.2/manual.html#3. One thing they don’t say, which you can pick up from how it’s used, is ::= means equivalent to. For example, block::={stat} means that a block is 0 or more stat[ments]s. Not trying to assume you don’t already know this, but wrote it incase you didn’t.
On Aug 13, 2014, at 11:19 AM, Mason Mackaman <[hidden email]> wrote:

I would read this: http://www.lua.org/manual/5.2/manual.html#3.3.5. That’s how I learned how the generic for loop works.
On Aug 13, 2014, at 8:22 AM, [hidden email] wrote:

Hi,

I'm stuck in my understanding of the way an iterator works. I found already some explanations in the lua doc but I don't understand why when calling a function by the "in" command the behavior of the variable in the function is different (ie: defined as local but behaves as global) :

-----------------------------------------------------------
   local table={1,2,3}           
   
   function iterator(table)
       local y = 0                             
       return function() y=y+1 print(y) return table[y] end     
   end
     
   -- 1) function iterator called by "for":  
     
         for i = 1,3 do
             iterator(table)()
         end
                 
    -- 2) function iterator call by "in":
     
          for i in iterator(table) do end
-------------------------------------------------------------          
   
The 2 loops call the same function, but in the first case the local variable y is not retained (which is the normal behavior of local variable as i understand it), so the value of y stays at 1.

But in the second case, the variable y is retained by the "magic" of "in" command and thus can be iterated from 1 to 3, although it is the same function.

If it possible to explain it simply (i'm not very skilled in lua so far) could you please explain me what happens really when using the "in" magic. Is this a way to make the local variable (like y here) of a function provisionally global, so that it can retains its value during the different calls to it, and when the "in loop" is finished it is taken back local again ?

Maybe this is done by assigning this local variable to a global one that is defined provisionally during the use of the "in loop" (and delete at the end of the loop), and when looping in fact you use this global variable instead of the local one ?

Also i found a particular syntax for iterator's use that i don't understand and I didn't find documentation on it :

-----------------------------------------------
function square(iteratorMaxCount,tot)
   local currentNumber=tot
      if currentNumber<iteratorMaxCount then
         currentNumber = currentNumber+1
         return currentNumber, currentNumber*currentNumber
      end
end

for i,n in square,3,0 do
  print(i,n)
end
----------------------------------------------

What means this particular way to call the function square: "square,3,0" ?
apparently this is not equivalent to "square(3,0)" as this syntax make an error.


Thank you very much for your help




Reply | Threaded
Open this post in threaded view
|

Re: Iterator construction

astient
In reply to this post by Javier Guerra Giraldez
Thank you all for your explanations, i understand i was misleading but it is more clear now.


De: "Javier Guerra Giraldez" <[hidden email]>
À: "Lua mailing list" <[hidden email]>
Envoyé: Mercredi 13 Août 2014 17:45:19
Objet: Re: Iterator construction

On Wed, Aug 13, 2014 at 8:22 AM,  <[hidden email]> wrote:
> The 2 loops call the same function, but in the first case the local variable
> y is not retained (which is the normal behavior of local variable as i
> understand it), so the value of y stays at 1.

the first case:

         for i = 1,3 do
             iterator(table)()
         end

calls "iterator(table)" three times, and after each one, it calls the
returned function once.

in your iterator() function you define "local y=0", so the variable is
defined and assigned each time iterator() is called.  the inner
function sees that value but only once, since it's called only once
before iterator() is called again and redefines y.


> But in the second case, the variable y is retained by the "magic" of "in"
> command and thus can be iterated from 1 to 3, although it is the same
> function.


the "in" syntax isn't magic, just a little complex.  it evaluates the
expression between the "in" and "do" keywords and expects to get up to
three values which are then used to manage the iteration.

in your second case:

          for i in iterator(table) do end

the expression "iterator(table)" is called _once_ to initiate the
loop, and then it gets the inner function, two other values (nil in
this case) to drive the loop, so the inner function gets called
several times and it sees the increasing values of "y"

from the docs:

"""
A for statement like

     for var_1, ···, var_n in explist do block end
is equivalent to the code:

     do
       local f, s, var = explist
       while true do
         local var_1, ···, var_n = f(s, var)
         if var_1 == nil then break end
         var = var_1
         block
       end
     end
"""
(where f, s and var internal variables, invisible to Lua code)

so, your one-line loop is transformed into

     do
       local f, s, var = iterator(table)
       while true do
         local i = f(s, var)
         if i == nil then break end
         var = i
         -- empty block
       end
     end

you can see the only call to iterator() is outside the loop, but the
local pseudovariable f is set to your inner function and then it's
called several times.

--
Javier