[proposal] close file on io.lines("filename", "*a")()

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

[proposal] close file on io.lines("filename", "*a")()

Soni "They/Them" L.
since lua doesn't have refcounting and some ppl wanna do basically
io.open(file):read("*a") I wanna propose io.lines("filename", "*a")()
(or alternatively io.lines("filename", "*a", "*l")) should close the
file as if the iterator ran out.


in other words if the first value isn't nil then check if the last value
is nil or the last argument is "*a" and if it is then close the file.

Reply | Threaded
Open this post in threaded view
|

Re: [proposal] close file on io.lines("filename", "*a")()

nobody
On 30/07/2019 16.42, Soni "They/Them" L. wrote:
> since lua doesn't have refcounting and some ppl wanna do basically
> io.open(file):read("*a") I wanna propose io.lines("filename", "*a")()
> (or alternatively io.lines("filename", "*a", "*l")) should close the
> file as if the iterator ran out.

function readfile( name )
   local <toclose> f, err = io.open( name, "r" )
   if not f then  return nil, err  end
   return f:read "*a"
end

(or something like that) should work, right?

Sometimes it actually makes sense to read '*a' and then seek back and
read(/write) again…

-- nobody

Reply | Threaded
Open this post in threaded view
|

Re: [proposal] close file on io.lines("filename", "*a")()

Soni "They/Them" L.


On 2019-07-30 11:49 a.m., nobody wrote:

> On 30/07/2019 16.42, Soni "They/Them" L. wrote:
>> since lua doesn't have refcounting and some ppl wanna do basically
>> io.open(file):read("*a") I wanna propose io.lines("filename", "*a")()
>> (or alternatively io.lines("filename", "*a", "*l")) should close the
>> file as if the iterator ran out.
>
> function readfile( name )
>   local <toclose> f, err = io.open( name, "r" )
>   if not f then  return nil, err  end
>   return f:read "*a"
> end
>
> (or something like that) should work, right?
>
> Sometimes it actually makes sense to read '*a' and then seek back and
> read(/write) again…
>
> -- nobody
>

sometimes you don't wanna leak file descriptors which you can only have
about 1000 or so per user or something I might be off by an order of
magnitude or so but you get the idea.

Reply | Threaded
Open this post in threaded view
|

[bug] for l in io.lines( f, "*a" ) do … loops forever (was: Re: [proposal] close file on io.lines("filename", "*a")())

nobody
[bug: see bottom block separated by '-----']

On 30/07/2019 16.54, Soni "They/Them" L. wrote:

> On 2019-07-30 11:49 a.m., nobody wrote:
>> On 30/07/2019 16.42, Soni "They/Them" L. wrote:
>>> since lua doesn't have refcounting and some ppl wanna do basically
>>> io.open(file):read("*a") I wanna propose io.lines("filename", "*a")()
>>> (or alternatively io.lines("filename", "*a", "*l")) should close the
>>> file as if the iterator ran out.
>>
>> function readfile( name )
>>   local <toclose> f, err = io.open( name, "r" )
>>   if not f then  return nil, err  end
>>   return f:read "*a"
>> end
>>
>> (or something like that) should work, right?
>>
>> Sometimes it actually makes sense to read '*a' and then seek back
>> and read(/write) again…
>
> sometimes you don't wanna leak file descriptors which you can only
> have about 1000 or so per user or something I might be off by an
> order of magnitude or so but you get the idea.

Which is why you (locally) "build" the language that you need to solve
your problem…  (People usually call that "writing helper functions", but
thinking about this in terms of (efficient) communication (with both
humans and the computer) is a very useful perspective.)

Changing file:read "*a" to auto-close the file would be guaranteed to
break code.  Not all files are unchanging data, e.g. for log files it
totally makes sense to read everything and then wait for more stuff to
come in.  For stuff like "status bitmaps" it also makes sense to read
everything and then update/re-write select positions in the file (that
was opened as "r+".)  Etc.…

For io.lines, I'm not sure if that would be a problem…  but it's a `for`
iterator, which technically is a triple (next,state,position).  Relying
on the implementation detail that state/position are unused just so you
don't need to write a (very small) helper functions is not a good idea.

-----

When experimenting with this, I actually noticed a bug:

The manual says:

> When the iterator function detects the end of file, it returns no
> values (to finish the loop) and automatically closes the file.
but actually

for l in io.lines( "foo.txt", "*a" ) do  print( l )  end

or equivalently

f = io.open "foo.txt"
for l in f:lines( "*a" ) do  print( l )  end

will loop forever (tested in 5.3 and 5.4(git)).

(And if that's fixed so that file:lines( "*a" ) return nil at EOF, then
io.lines will also finish and close the file… so that should get Soni
the behavior that they want?)

-- nobody

Reply | Threaded
Open this post in threaded view
|

Re: [bug] for l in io.lines( f, "*a" ) do … loops forever

Soni "They/Them" L.


On 2019-07-30 12:43 p.m., nobody wrote:

> [bug: see bottom block separated by '-----']
>
> On 30/07/2019 16.54, Soni "They/Them" L. wrote:
>> On 2019-07-30 11:49 a.m., nobody wrote:
>>> On 30/07/2019 16.42, Soni "They/Them" L. wrote:
>>>> since lua doesn't have refcounting and some ppl wanna do basically
>>>> io.open(file):read("*a") I wanna propose io.lines("filename", "*a")()
>>>> (or alternatively io.lines("filename", "*a", "*l")) should close the
>>>> file as if the iterator ran out.
>>>
>>> function readfile( name )
>>>   local <toclose> f, err = io.open( name, "r" )
>>>   if not f then  return nil, err  end
>>>   return f:read "*a"
>>> end
>>>
>>> (or something like that) should work, right?
>>>
>>> Sometimes it actually makes sense to read '*a' and then seek back
>>> and read(/write) again…
>>
>> sometimes you don't wanna leak file descriptors which you can only
>> have about 1000 or so per user or something I might be off by an
>> order of magnitude or so but you get the idea.
>
> Which is why you (locally) "build" the language that you need to solve
> your problem…  (People usually call that "writing helper functions", but
> thinking about this in terms of (efficient) communication (with both
> humans and the computer) is a very useful perspective.)
>
> Changing file:read "*a" to auto-close the file would be guaranteed to
> break code.  Not all files are unchanging data, e.g. for log files it
> totally makes sense to read everything and then wait for more stuff to
> come in.  For stuff like "status bitmaps" it also makes sense to read
> everything and then update/re-write select positions in the file (that
> was opened as "r+".)  Etc.…
>
> For io.lines, I'm not sure if that would be a problem…  but it's a `for`
> iterator, which technically is a triple (next,state,position). Relying
> on the implementation detail that state/position are unused just so you
> don't need to write a (very small) helper functions is not a good idea.

It returns an iterator function, as per the manual. It's not a triple,
just one function.

>
> -----
>
> When experimenting with this, I actually noticed a bug:
>
> The manual says:
>
>> When the iterator function detects the end of file, it returns no
>> values (to finish the loop) and automatically closes the file.
> but actually
>
> for l in io.lines( "foo.txt", "*a" ) do  print( l )  end
>
> or equivalently
>
> f = io.open "foo.txt"
> for l in f:lines( "*a" ) do  print( l )  end
>
> will loop forever (tested in 5.3 and 5.4(git)).
>
> (And if that's fixed so that file:lines( "*a" ) return nil at EOF, then
> io.lines will also finish and close the file… so that should get Soni
> the behavior that they want?)
>
> -- nobody
>


Reply | Threaded
Open this post in threaded view
|

Re: [proposal] close file on io.lines("filename", "*a")()

Sean Conner
In reply to this post by Soni "They/Them" L.
It was thus said that the Great Soni They/Them L. once stated:
> since lua doesn't have refcounting and some ppl wanna do basically
> io.open(file):read("*a") I wanna propose io.lines("filename", "*a")()
> (or alternatively io.lines("filename", "*a", "*l")) should close the
> file as if the iterator ran out.

  Lua 5.4 already does that.

[spc]lucy:/tmp>lua-54
Lua 5.4.0  Copyright (C) 1994-2019 Lua.org, PUC-Rio
> print(io.lines("tabs"))
function: 0x8b39900 nil nil file (0x8b410f8)

  That final value marks the descriptor for closing when the iterator is
done.  If you want to automatically close the file after reading the
contents:

        local data do
          local <toclose> f = io.open(myfile)
          data              = f:read("a")
        end

        -- f is closed, and data has the contents.

  -spc

Reply | Threaded
Open this post in threaded view
|

Re: [bug] for l in io.lines( f, "*a" ) do … loops forever (was: Re: [proposal] close file on io.lines("filename", "*a")())

Roberto Ierusalimschy
In reply to this post by nobody
> When experimenting with this, I actually noticed a bug:
>
> The manual says:
>
> >When the iterator function detects the end of file, it returns no
> >values (to finish the loop) and automatically closes the file.
> but actually
>
> for l in io.lines( "foo.txt", "*a" ) do  print( l )  end
>
> [...]
>
> will loop forever (tested in 5.3 and 5.4(git)).

We should fix the manual, not the implementation. The manual is clear
that the iterator function works like file:read, and file:read never
fails (returns nil) with the "*a" format.

Thanks for the report.

-- Roberto

Reply | Threaded
Open this post in threaded view
|

Re: [proposal] close file on io.lines("filename", "*a")()

Soni "They/Them" L.
In reply to this post by Sean Conner


On 2019-07-30 2:12 p.m., Sean Conner wrote:

> It was thus said that the Great Soni They/Them L. once stated:
> > since lua doesn't have refcounting and some ppl wanna do basically
> > io.open(file):read("*a") I wanna propose io.lines("filename", "*a")()
> > (or alternatively io.lines("filename", "*a", "*l")) should close the
> > file as if the iterator ran out.
>
>    Lua 5.4 already does that.
>
> [spc]lucy:/tmp>lua-54
> Lua 5.4.0  Copyright (C) 1994-2019 Lua.org, PUC-Rio
> > print(io.lines("tabs"))
> function: 0x8b39900 nil nil file (0x8b410f8)
>
>    That final value marks the descriptor for closing when the iterator is
> done.  If you want to automatically close the file after reading the
> contents:
>
> local data do
>  local <toclose> f = io.open(myfile)
>  data              = f:read("a")
> end
>
> -- f is closed, and data has the contents.
>
>    -spc
>

no, sorry, I think you misunderstood.

io.lines("filename", "*a")()

note the call at the end. this reads the whole file. ideally it'd also
close it.

compare to:

io.open("filename"):read("*a")

Reply | Threaded
Open this post in threaded view
|

Re: [proposal] close file on io.lines("filename", "*a")()

Sergey Kovalev
вт, 30 июл. 2019 г. в 20:44, Soni "They/Them" L. <[hidden email]>:

> compare to:
> io.open("filename"):read("*a")

You can write simple function

function file_get_contents(name,mode)
  local f,r=io.open(name,mode or "rb")
  if f then r=f:read"a" else error(r) end
  f:close()
  return r
end

And read whole file just in one call file_get_contents

mydata=file_get_contents "myfile"