luasocket and selecting from files.

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

luasocket and selecting from files.

Jorge Visca
Hi everyone,

I'm trying to use luasocket's select() to read from files, under Linux.
As far as can see, it is possible now trough judicious use of setfd()
(and dirty()?). But I do not see where i can get that fd from (io.* uses
a opaque thing for files).
Is there any working code out there doing this?

Reply | Threaded
Open this post in threaded view
|

Re: luasocket and selecting from files.

Sean Conner
It was thus said that the Great Jorge once stated:
> Hi everyone,
>
> I'm trying to use luasocket's select() to read from files, under Linux.

  From files, like '/usr/local/include/lua.h' or from any type of device
that uses a file handle?  I only ask this because using select() on an
actual file (like '/usr/local/include/lua.h') will *always* be readable and
writable---i.e. there's just no use in using select() for files [1] as Linux
does some aggressive memory caching.

  select() [2] works much better for sockets and character devices like
/dev/tty.

> As far as can see, it is possible now trough judicious use of setfd()
> (and dirty()?). But I do not see where i can get that fd from (io.* uses
> a opaque thing for files).

  To get the underlying file descriptor from a FILE *, you need to use the C
function fileno().

> Is there any working code out there doing this?

  For files?  I don't know, but it would be similar with files (as sockets
are file handles).  

  -spc

[1] I think this is even true for network file sytems like NFS but I
        don't have a way to experiment with this right now.

[2] If you are using Linux, epoll() is a much better API for this type
        of thing.

Reply | Threaded
Open this post in threaded view
|

Re: luasocket and selecting from files.

Jorge Visca
On 01/07/15 04:29, Sean Conner wrote:
> From files, like '/usr/local/include/lua.h' or from any type of device
> that uses a file handle? I only ask this because using select() on an
> actual file (like '/usr/local/include/lua.h') will *always* be
> readable and writable---i.e. there's just no use in using select() for
> files [1] as Linux does some aggressive memory caching. select() [2]
> works much better for sockets and character devices like /dev/tty.

Yes, is for device files that block reads when there is no data. And for
plain files too.

> To get the underlying file descriptor from a FILE *, you need to use
> the C function fileno().
I see... Then luasocket can not be used on it's own for this purpose, I
need another chunk o C code somewhere. I my environment luasocket is the
only external library available, so some rethinking is needed.

>> Is there any working code out there doing this?
>    For files?  I don't know, but it would be similar with files (as sockets
> are file handles).

I was thinking code preparing a file handle for serving with luasocket.
There seems to be some weird dancing involved with creating a connected
socket, replacing its fd, then invalidating something setting a fd of
-1, etc.

Thanks,

Jorge


Reply | Threaded
Open this post in threaded view
|

Re: luasocket and selecting from files.

Bernd Eggink
In reply to this post by Jorge Visca
On 01.07.2015 01:05, Jorge wrote:
> Hi everyone,
>
> I'm trying to use luasocket's select() to read from files, under Linux.
> As far as can see, it is possible now trough judicious use of setfd()
> (and dirty()?). But I do not see where i can get that fd from (io.* uses
> a opaque thing for files).

luaposix has a function fileno(file) wich does this.

- Bernd



Reply | Threaded
Open this post in threaded view
|

Re: luasocket and selecting from files.

Sean Conner
In reply to this post by Jorge Visca
It was thus said that the Great Jorge once stated:

> On 01/07/15 04:29, Sean Conner wrote:
> >From files, like '/usr/local/include/lua.h' or from any type of device
> >that uses a file handle? I only ask this because using select() on an
> >actual file (like '/usr/local/include/lua.h') will *always* be
> >readable and writable---i.e. there's just no use in using select() for
> >files [1] as Linux does some aggressive memory caching. select() [2]
> >works much better for sockets and character devices like /dev/tty.
>
> Yes, is for device files that block reads when there is no data. And for
> plain files too.

  Using select() on a real file (like "/usr/include/stdio.h") will *always*
*always* *always* return ready for both *reading and writing*.  There's no
real point to using select() on a real file because of that behavior.  Even
if you give select() an infinite timeout, it will *always* return ready if
you give it a file descriptor to a real file.

  For character devices (like "/dev/tty") and sockets, select() works as
expected.

> >>Is there any working code out there doing this?
> >   For files?  I don't know, but it would be similar with files (as sockets
> >are file handles).
>
> I was thinking code preparing a file handle for serving with luasocket.
> There seems to be some weird dancing involved with creating a connected
> socket, replacing its fd, then invalidating something setting a fd of
> -1, etc.

  The problem you are having is that luasocket was written to deal with
sockets, not files.  You are trying to force it to so something it was not
written to deal with.  If, perhaps, you describe what you are trying to
accomplish, someone can point you in a better direction.

  -spc


Reply | Threaded
Open this post in threaded view
|

Re: luasocket and selecting from files.

Jorge Visca
On 01/07/15 11:43, Sean Conner wrote:

> It was thus said that the Great Jorge once stated:
>> On 01/07/15 04:29, Sean Conner wrote:
>> >From files, like '/usr/local/include/lua.h' or from any type of device
>>> that uses a file handle? I only ask this because using select() on an
>>> actual file (like '/usr/local/include/lua.h') will *always* be
>>> readable and writable---i.e. there's just no use in using select() for
>>> files [1] as Linux does some aggressive memory caching. select() [2]
>>> works much better for sockets and character devices like /dev/tty.
>> Yes, is for device files that block reads when there is no data. And for
>> plain files too.
>    Using select() on a real file (like "/usr/include/stdio.h") will *always*
> *always* *always* return ready for both *reading and writing*.  There's no
> real point to using select() on a real file because of that behavior.  Even
> if you give select() an infinite timeout, it will *always* return ready if
> you give it a file descriptor to a real file.
>
>    For character devices (like "/dev/tty") and sockets, select() works as
> expected.

Errrr, yes, that was I said, I need to read from device files. Yes, I
know select() works on them. I only was trying to use luasocket's
select() for that.

>
>> I was thinking code preparing a file handle for serving with luasocket.
>> There seems to be some weird dancing involved with creating a connected
>> socket, replacing its fd, then invalidating something setting a fd of
>> -1, etc.
>    The problem you are having is that luasocket was written to deal with
> sockets, not files.  You are trying to force it to so something it was not
> written to deal with.  If, perhaps, you describe what you are trying to
> accomplish, someone can point you in a better direction.

Yes, I gather that. But, luasocket allegedly allows to feed an arbitrary
fd to a socket object, and is supposed to work when done that [1] . It's
just that didn't find anyone doing that, and it seems a critical piece
is missing (the fileno).

What I'm doing is a I/O driven single-thread concurrent application,
that must process data coming from sockets and from device files
(serials, mouse, etc.) IT already works when using nixio as library,
which allows to handle sockets and files as equals, but now I was trying
to do the same on pure luasocket.

Jorge

[1] http://lua-users.org/lists/lua-l/2006-10/msg00125.html

Reply | Threaded
Open this post in threaded view
|

Re: luasocket and selecting from files.

Jorge Visca
In reply to this post by Bernd Eggink
On 01/07/15 13:28, Bernd Eggink wrote:
> luaposix has a function fileno(file) wich does this.
>

Thanks for the pointer. Nevertheless, pull luaposix as dependence in
only to use a hack over luasocket looks weird... Why not use luaposix's
select() then?

Jorge

Reply | Threaded
Open this post in threaded view
|

RE: luasocket and selecting from files.

Thijs Schreijer
In reply to this post by Jorge Visca
> >> Is there any working code out there doing this?
> >    For files?  I don't know, but it would be similar with files (as
> sockets
> > are file handles).
>
> I was thinking code preparing a file handle for serving with luasocket.
> There seems to be some weird dancing involved with creating a connected
> socket, replacing its fd, then invalidating something setting a fd of
> -1, etc.
>

LuaSec does this, but its result is still a handle for a socket. But it has some magic in setting the fd and invalidating the original.

Hth
Thijs


> Thanks,
>
> Jorge
>


Reply | Threaded
Open this post in threaded view
|

Re: luasocket and selecting from files.

Ignacio Burgueño-2
In reply to this post by Jorge Visca
On Wed, Jul 1, 2015 at 8:00 AM, Jorge <[hidden email]> wrote:
On 01/07/15 13:28, Bernd Eggink wrote:
luaposix has a function fileno(file) wich does this.


Thanks for the pointer. Nevertheless, pull luaposix as dependence in only to use a hack over luasocket looks weird... Why not use luaposix's select() then?

What about using Alien [1] to declare a binding to that function? (fileno) It is one more dependency, yes, but it might be lighter than luaposix (don't know, really).

Regards,
Ignacio

 
Reply | Threaded
Open this post in threaded view
|

Re: luasocket and selecting from files.

Jorge Visca
In reply to this post by Thijs Schreijer
On 01/07/15 14:33, Thijs Schreijer wrote:
>> I was thinking code preparing a file handle for serving with luasocket.
>> There seems to be some weird dancing involved with creating a connected
>> socket, replacing its fd, then invalidating something setting a fd of
>> -1, etc.
>>
> LuaSec does this, but its result is still a handle for a socket. But it has some magic in setting the fd and invalidating the original.

Now I get it, the invalidating is done to the keep the original socket
from closing the migrated fd. OTOH, as you mention LuaSec does not need
to deal with files (and it has a C component, so they could sneak a
fileno() binding if needed).

Very useful example anyway, thanks.

Reply | Threaded
Open this post in threaded view
|

Re: luasocket and selecting from files.

Jorge Visca
In reply to this post by Ignacio Burgueño-2
On 01/07/15 19:36, Ignacio Burgueño wrote:
>
> What about using Alien [1] to declare a binding to that function?
> (fileno) It is one more dependency, yes, but it might be lighter than
> luaposix (don't know, really).
>
> Regards,
> Ignacio
>
> [1]: https://luarocks.org/modules/mascarenhas/alien

I'll have to research this more... One of the requirment is to run under
NS-3 network simulator's DCE (direct code execution), a framework that
provides a environment to run native applications inside the simulator.
So, the environment is not a real Linux, but a cutdown look-a-like [1].
It's not clear to me that Alien can work there.

Thanks for the idea,

Jorge

[1]
https://www.nsnam.org/docs/dce/manual/html/dce-user-tech.html#api-coverage