proposal for change in for protocol

classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|

proposal for change in for protocol

Cosmin Apreutesei
Hi,

What would you think of a change in the for protocol like this:

     do
       local f, s, var_1, ···, var_n = explist --keep all init values
       while true do
         local var_1, ···, var_n = f(s, var_1, ···, var_n)  --pass all
values back to f
         if var_1 == nil then break end
         block
       end
     end

Pros:
- allow one to write heapless stateless iterators like values(t) that
can be used in tight loops - I suspect the idiom for _,v in ipairs(t)
is because _ is the state that currently cannot be hidden.

Cons:
- functions that take iterators as arguments and are written like eg.
collect(f,s,v) instead of collect(f,...) will have to adjust but only
to work with new iterators that have more state variables
- there's a subtle semantic change: the hidden variables not
explicitly bound in the loop won't be gc'ed until the next iteration,
just like var_1 and s (can't imagine this breaking any code though).

What do you think? Hope I'm not missing an obvious show-stopper and
spamming for nothing.

Reply | Threaded
Open this post in threaded view
|

Re: proposal for change in for protocol

Duncan Cross
On Tue, Apr 3, 2012 at 12:29 PM, Cosmin Apreutesei
<[hidden email]> wrote:
> Hi,
>
> What would you think of a change in the for protocol like this:
>
>     do
>       local f, s, var_1, ···, var_n = explist --keep all init values
>       while true do
>         local var_1, ···, var_n = f(s, var_1, ···, var_n)

I think the main problem is here - f() can return any number of
values, not just the same number "n" as the number of initial values.
The "local var_1, ... var_n" list inside the while loop would not be
like a normal set of local variables, as the pseudocode implies, but
instead would have to be a dynamic allocation (similar to the '...'
varargs structure) to accommodate what f() chooses to return. This can
probably be done in a clever way, but I suspect it would still cancel
out the gains you hope to make by allowing more state values.

(Also, I think you made a mistake in your pseudocode that means f()
would only ever be called with the initial state values, not the most
recent ones.)

>         if var_1 == nil then break end
>         block
>       end
>     end
>
> Pros:
> - allow one to write heapless stateless iterators like values(t) that
> can be used in tight loops - I suspect the idiom for _,v in ipairs(t)
> is because _ is the state that currently cannot be hidden.

That seems like a bad example - in my experience, the index value in
an ipairs loop is useful more often than not.

-Duncan

Reply | Threaded
Open this post in threaded view
|

Re: proposal for change in for protocol

Miles Bader-2
Duncan Cross <[hidden email]> writes:
> That seems like a bad example - in my experience, the index value in
> an ipairs loop is useful more often than not.

I dunno, I seem to ignore the key in awful lot of for loops...

Given the prevalence of "iterate over the values only" in many
languages, it is a bit odd that Lua doesn't have it.

-Miles

--
Egotist, n. A person of low taste, more interested in himself than in me.

Reply | Threaded
Open this post in threaded view
|

Re: proposal for change in for protocol

Rena
On Tue, Apr 3, 2012 at 07:47, Miles Bader <[hidden email]> wrote:

> Duncan Cross <[hidden email]> writes:
>> That seems like a bad example - in my experience, the index value in
>> an ipairs loop is useful more often than not.
>
> I dunno, I seem to ignore the key in awful lot of for loops...
>
> Given the prevalence of "iterate over the values only" in many
> languages, it is a bit odd that Lua doesn't have it.
>
> -Miles
>
> --
> Egotist, n. A person of low taste, more interested in himself than in me.
>

Well, Lua does have it - it's ipairs(). You just discard the first
return value if you don't need it.

--
Sent from my toaster.

Reply | Threaded
Open this post in threaded view
|

Re: proposal for change in for protocol

Miles Bader-2
Rena <[hidden email]> writes:

>>> That seems like a bad example - in my experience, the index value in
>>> an ipairs loop is useful more often than not.
>>
>> I dunno, I seem to ignore the key in awful lot of for loops...
>>
>> Given the prevalence of "iterate over the values only" in many
>> languages, it is a bit odd that Lua doesn't have it.
>
> Well, Lua does have it - it's ipairs(). You just discard the first
> return value if you don't need it.

Er, no, that's my point:  I often use ipairs or pairs like that, but it
feels awkward.  It would be nicer to not have a dummy variable.

-Miles

--
Back, n. That part of your friend which it is your privilege to contemplate in
your adversity.

Reply | Threaded
Open this post in threaded view
|

Re: proposal for change in for protocol

Rebel Neurofog
> Er, no, that's my point:  I often use ipairs or pairs like that, but it
> feels awkward.  It would be nicer to not have a dummy variable.

There is special name of _ for dummy variable:
for _, value in ipairs (...) do ... end

There's no special reason to add specific function iterating only over values.

Reply | Threaded
Open this post in threaded view
|

Re: proposal for change in for protocol

Dirk Laurie-2
In reply to this post by Miles Bader-2
> Given the prevalence of "iterate over the values only" in many
> languages, it is a bit odd that Lua doesn't have it.
>
As always, Lua provides the soup and the spoon, but you have
to do your own feeding.

ivalues = function(tbl);
   local k=0
   return function() k=k+1; return tbl[k] end
end

Reply | Threaded
Open this post in threaded view
|

Re: proposal for change in for protocol

Dirk Laurie-2
In reply to this post by Rebel Neurofog
>
> There is special name of _ for dummy variable:
> for _, value in ipairs (...) do ... end
>
That's a perfectly ordinary name, not special.  it's merely
a good habit to reserve that name for a read-never variable.

Reply | Threaded
Open this post in threaded view
|

Re: proposal for change in for protocol

Miles Bader-2
In reply to this post by Rebel Neurofog
Rebel Neurofog <[hidden email]> writes:
>> Er, no, that's my point:  I often use ipairs or pairs like that, but it
>> feels awkward.  It would be nicer to not have a dummy variable.
>
> There is special name of _ for dummy variable:
> for _, value in ipairs (...) do ... end

It's still a dummy variable.  It still feels awkward.

> There's no special reason to add specific function iterating only over values.

Er, great, you have a different opinion... no prob with that.
But I think it feels awkward.

Mind you, I do think it's probably not worth changing the Lua
iteration API just to get rid of this little nit...

-miles

--
XML is like violence.  If it doesn't solve your problem, you're not
using enough of it.

Reply | Threaded
Open this post in threaded view
|

Re: proposal for change in for protocol

Miles Bader-2
In reply to this post by Dirk Laurie-2
Dirk Laurie <[hidden email]> writes:

>> Given the prevalence of "iterate over the values only" in many
>> languages, it is a bit odd that Lua doesn't have it.
>
> As always, Lua provides the soup and the spoon, but you have
> to do your own feeding.
>
> ivalues = function(tbl);
>    local k=0
>    return function() k=k+1; return tbl[k] end
> end

Sure, but in practice, one probably doesn't want to pay the price of a
closure just for that...

-miles

--
"Suppose we've chosen the wrong god. Every time we go to church we're
just making him madder and madder." -- Homer Simpson

Reply | Threaded
Open this post in threaded view
|

Re: proposal for change in for protocol

Tony Finch
In reply to this post by Cosmin Apreutesei
Cosmin Apreutesei <[hidden email]> wrote:
>
>      do
>        local f, s, var_1, ···, var_n = explist --keep all init values
>        while true do
>          local var_1, ···, var_n = f(s, var_1, ···, var_n)  --pass all values back to f
>          if var_1 == nil then break end
>          block
>        end
>      end

This can't work in a straightforward way. At the moment, the Lua compiler
knows the stack depth at all points in the program, as a static property
of the program. This allows local variables and temporary values to
map to fixed stack locations, and these locations are written into the
argument fields of the bytecode instructions.

This fixed stack depth property does not hold for the body of your
proposed for loop.

In Lua, variable stack depth only occurs at function call and return. When
the reference manual talks about "adjusting" lists of values, this is
basically equivalent to restoring the stack to the correct statically
known depth.

So to make your proposal work on anything like the current Lua
implementation, you would need to make the loop body into a closure, which
completely negates your aim of eliminating an allocation.

Tony.
--
f.anthony.n.finch  <[hidden email]>  http://dotat.at/
Bailey, Fair Isle, Faeroes: Northeast backing northwest 5 or 6, occasionally 7
at first in Fair Isle, decreasing 4 for a time. Moderate or rough,
occasionally very rough at first in Fair Isle. Wintry showers. Moderate or
good.
Reply | Threaded
Open this post in threaded view
|

Re: proposal for change in for protocol

Cosmin Apreutesei
> This fixed stack depth property does not hold for the body of your
> proposed for loop.

So it could only work with a fixed number of state vars for the entire
loop. Could these stack slots be counted upon evaluation of explist
and then the same number of variables be sent back to f ? Or would
that still be a pain to implement? I'm just trying to see if it's
feasible to squeeze in 1 or 2 more state vars in there, which would be
plenty enough to make some iterators heapless.

Reply | Threaded
Open this post in threaded view
|

Re: proposal for change in for protocol

Steve Litt
In reply to this post by Dirk Laurie-2
On Tue, 3 Apr 2012 16:30:09 +0200
Dirk Laurie <[hidden email]> wrote:

> >
> > There is special name of _ for dummy variable:
> > for _, value in ipairs (...) do ... end
> >
> That's a perfectly ordinary name, not special.  it's merely
> a good habit to reserve that name for a read-never variable.
>

Thanks Dirk,

I didn't know that, and always thought _ was a reserved word.

Now that I know it's an idiom for a read-never variable, everything
makes sense.

SteveT

Reply | Threaded
Open this post in threaded view
|

Re: proposal for change in for protocol

Tony Finch
In reply to this post by Cosmin Apreutesei
Cosmin Apreutesei <[hidden email]> wrote:
>
> So it could only work with a fixed number of state vars for the entire
> loop. Could these stack slots be counted upon evaluation of explist
> and then the same number of variables be sent back to f ? Or would
> that still be a pain to implement?

Sorry, I now realise I made a mistake. The extra state variables you were
thinking of are actually visible in the source, so the compiler can know
how many stack slots are required. This means the programmer has to know
how many state variable are needed, and can't leave out the ones that
aren't used in the loop body.

Tony.
--
f.anthony.n.finch  <[hidden email]>  http://dotat.at/
German Bight: East or northeast 5 to 7. Moderate, occasionally rough in west.
Rain or sleet then mainly fair. Good, occasionally poor.