Lua and networking - possibly basic question

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

Lua and networking - possibly basic question

Steve Kemp
  I've been embedding Lua 5.0 into a lot of local applications 
 recently and find it very nice to work with.

  For some software I'm working with I'm looking to add basic
 networking facilities, similar to the primitives people
 are used to :

	connect
	read
	write

  (I have no need for the bind() or accept() primitives.)

  I'm comfortable with binding C/C++ functions to Lua and
 working with the language nicely.  Currently my problem
 is a lack of direct mapping between Lua types and C types.

  The standard network read() call, essentially, works like this:

	read( socket, buffer, bufferLength );

  This doesn't fit in nicely with Lua's string objects which
 is the most obvious fit on the Lua side.

  So I'm struggling to understand how to approach implementation
 I want to allow users to be able to do:

	socket = connect( "host", port );
	send( socket, "GET / HTTP/1.0\n\n" );
	header = read( socket );
	data = read( socket );
	close( socket );

  Clearly there is a failure here.  I can implement C/C++ reading
 function as:

	void readDataUntilTerminator( socket, terminator )
	{
	   char *data = NULL

	   while( read ( .. ) ) 
	   {
	      data = realloc( new Size );
	      if ( data contains termintor )
		return;
	   }
	}

  This would allow callers to say "read until the expected response
 occurs" - however it leaves a nasty DOS condition if the expected
 result fails to occur.

  How do other people approach this problem?  (In composing this
 mail I did a few searchs and see the existance of LuaSocket - this
 might help me, if I can embed it within my code.  Too soon to tell,
 but my natural reaction is to think that is overkill ...)

Steve
--
http://www.steve.org.uk/

Reply | Threaded
Open this post in threaded view
|

Re: Lua and networking - possibly basic question

Torsten Karwoth
Hello,

>   So I'm struggling to understand how to approach implementation
>  I want to allow users to be able to do:
>
> 	socket = connect( "host", port );
> 	send( socket, "GET / HTTP/1.0\n\n" );
> 	header = read( socket );
> 	data = read( socket );
> 	close( socket );
>
[...]
>   How do other people approach this problem?  (In composing this
>  mail I did a few searchs and see the existance of LuaSocket - this
>  might help me, if I can embed it within my code.  Too soon to tell,
>  but my natural reaction is to think that is overkill ...)

Currently i'm embedding lua as an scripting language in an irc bot
based on the 'energymech'. For scripting purposes i made a 'Socket'
library which uses the bots 'infrastructure' for socket handling.
This socket code doesn't support binary data.

Currently implemented socket-functions for scripts are:

      n = Socket.Connect (s:host, n:port, s:function [, s:vhost])
      n = Socket.Listen (n:port, s:function [, b:perm])
          Socket.Close (n:sockID [, b:flag])
   n, b = Socket.Status (n:sockID)
   b, s = Socket.Write (n:sockID, s:message)
      s = Socket.Read (n:sockID)

to open an incoming (listening) socket you would write:

  lsock = Socket.Listen(40001, "mySockFunc", true);
  if not lsock then [error opening listener] end;

(the 'true' flag keeps the listener socket open, this means every
client which connects will create a new socket id). The function
"mySockFunc" will be called if a client connects, disconnects
or sends data, with three parameters: The sockID, a (message)
string, and a status code:

-- this function will just reply the incoming message
-- back to the sender until we receive 'CIAO':
function mySockFunc(sid, msg, stat)
    if stat < 0 then
        Socket.Close(sid)
        return
    elseif stat > 0 then
        Socket.Write(sid,"Welcome!")
        return
    end
    -- stat is 0 here
    if not msg then return end
    Socket.Write(sid, msg) -- send the message back
    if string.upper(msg) == "CIAO" then
        Socket.Close(sid)
        return
    end
end

so you just can "telnet <host> 40001" and type CIAO to disconnect ;)

If you're interested i can prepare an archive with the documentation
and source code (however, there will be changes to the final version).

Bye,
  Torsten

Reply | Threaded
Open this post in threaded view
|

Re: Lua and networking - possibly basic question

Adrian Sietsma
In reply to this post by Steve Kemp
Steve Kemp wrote:
I've been embedding Lua 5.0 into a lot of local applications recently and find it very nice to work with.

  For some software I'm working with I'm looking to add basic
 networking facilities, similar to the primitives people
 are used to :

	connect
	read
	write

  (I have no need for the bind() or accept() primitives.)

...

  How do other people approach this problem?  (In composing this
 mail I did a few searchs and see the existance of LuaSocket - this
 might help me, if I can embed it within my code.  Too soon to tell,
 but my natural reaction is to think that is overkill ...)


use LuaSocket. You probably won't need http, ftp etc; but why re-invent the wheel ? (you can always leave out the unused bits).

even if you are driven to roll your own socket implementation, you should still read the luasocket c source to see how Diego did it.

Adrian
(i have a co-routine threaded http proxy built on LuaSocket, and it is 100% reliable for my usage case).



Reply | Threaded
Open this post in threaded view
|

Re: Lua and networking - possibly basic question

D Burgess-4
Yes, I would agree with Adrian. LuaSocket is rather well tested (unless you
want to do it again). It is also quick enough for most needs. And yes basic
LuaSocket is light.

David B.

On 10/27/05, Adrian Sietsma <[hidden email]> wrote:
> Steve Kemp wrote:
> >   I've been embedding Lua 5.0 into a lot of local applications
> >  recently and find it very nice to work with.
> >
> >   For some software I'm working with I'm looking to add basic
> >  networking facilities, similar to the primitives people
> >  are used to :
> >
> >       connect
> >       read
> >       write
> >
> >   (I have no need for the bind() or accept() primitives.)
> >
> ...
> >
> >   How do other people approach this problem?  (In composing this
> >  mail I did a few searchs and see the existance of LuaSocket - this
> >  might help me, if I can embed it within my code.  Too soon to tell,
> >  but my natural reaction is to think that is overkill ...)
> >
>
> use LuaSocket. You probably won't need http, ftp etc; but why re-invent the
> wheel ? (you can always leave out the unused bits).
>
> even if you are driven to roll your own socket implementation, you should
> still read the luasocket c source to see how Diego did it.
>
> Adrian
> (i have a co-routine threaded http proxy built on LuaSocket, and it is 100%
> reliable for my usage case).
>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: Lua and networking - possibly basic question

Steve Kemp
  THanks to all the people that pointed me to luasocket, whilst
 I didn't end up using it in the end it was a useful thing to
 examine.

  I've now got a basic C implmentation of socketing operations
 extremely simplistic and only covering the cases that I need:

	accept() bind() connect() read() write() close();

  (To be honest this is actually more than I need).

  As a proof of concept I put together a *basic* HTTP server
 skellington and a matching client.

	http://www.steve.org.uk/Software/lua-httpd/

  I think this demonstrates that the code works sufficient
 for my future purposes.

Steve
-- 
http://www.steve.org.uk/

Reply | Threaded
Open this post in threaded view
|

Re: Lua and networking - possibly basic question

Alex Queiroz
Hallo,

On 28/10/05, Steve Kemp <[hidden email]> wrote:
>
>   As a proof of concept I put together a *basic* HTTP server
>  skellington and a matching client.
>
>         http://www.steve.org.uk/Software/lua-httpd/
>

    In function pRead of libhttpd.c:

----
    n = read(sockfd,buffer,sizeof(buffer)-1);


    lua_pushnumber(L, n );
    lua_pushstring(L, buffer );
----

Are you sure the buffer is always null-terminated?

--
-alex
http://www.ventonegro.org/


Reply | Threaded
Open this post in threaded view
|

Re: Lua and networking - possibly basic question

Alex Queiroz
Hallo,

On 29/10/05, Alex Queiroz <[hidden email]> wrote:
>
> Are you sure the buffer is always null-terminated?
>

    Ops, my mistake. I did not see the line  "memset( buffer, '\0',
sizeof(buffer));" before.

--
-alex
http://www.ventonegro.org/


Reply | Threaded
Open this post in threaded view
|

Re: Lua and networking - possibly basic question

Steve Kemp
In reply to this post by Alex Queiroz
On Sat, Oct 29, 2005 at 03:56:35PM -0300, Alex Queiroz wrote:

>     In function pRead of libhttpd.c:
> 
> ----
>     n = read(sockfd,buffer,sizeof(buffer)-1);
> 
> 
>     lua_pushnumber(L, n );
>     lua_pushstring(L, buffer );
> ----
> 
> Are you sure the buffer is always null-terminated?
> 

  I think I can be pretty sure that the buffer will have a 
 trailing null.  Looking at the code:

    /* Buffer we read() into */
    char buffer[4096];
    memset( buffer, '\0', sizeof(buffer));

    ..

    n = read(sockfd,buffer,sizeof(buffer)-1);

  1.  Create a buffer 4096 bytes long.
  2.  Set it full of '\0'.
  3.  Read no more than 4095 bytes into it:  (sizeof(buffer)-1);

  So there will *always* be at least one trailing NULL on the end
 of the buffer.

  (This from the CVS repository - some things changed, including the 
 size of the read buffer, but the memset/sizeof(-1) is the same in
 the code you quoted).

  A more interesting question is what to do if the data read contains
 NULLs.  So far that hasn't been a problem, but I'm not sure without
 checking whether Lua strings are NULL-clean..

Steve
--

Reply | Threaded
Open this post in threaded view
|

Re: Lua and networking - possibly basic question

Ashwin Hirschi-2

    In function pRead of libhttpd.c:

----
    n = read(sockfd,buffer,sizeof(buffer)-1);


    lua_pushnumber(L, n );
    lua_pushstring(L, buffer );

  A more interesting question is what to do if the data read contains
 NULLs.  So far that hasn't been a problem, but I'm not sure without
 checking whether Lua strings are NULL-clean..

Lua strings may contain 0x00 characters.

One way to improve the code returning the result would be to simply replace it with:

	lua_pushlstring(L, buffer, n);

This would get you on your way to enable sending images, for instance [something the current approach won't handle...].

You might want the check return value from the read() call as well.

Ashwin.
--
no signature is a signature.

Reply | Threaded
Open this post in threaded view
|

Re: Lua and networking - possibly basic question

Ashwin Hirschi-2
In reply to this post by Steve Kemp

And while I'm at it: you can improve the write section by replacing the

	length = strlen( data );

by a call to lua_strlen. That'll take embedded zeroes into account and will give the true length of the data (enabling you to send binary data, for images and such).

All this (and much more) can easily be gleaned from the Lua 5 reference manual. In this case the API section would be especially noteworthy:

	http://www.lua.org/manual/5.0/manual.html#3

I'd consider it required reading, well worth the little time needed to absord it.

Ashwin.
--
no signature is a signature.

Reply | Threaded
Open this post in threaded view
|

Re: Lua and networking - possibly basic question

Steve Kemp
In reply to this post by Ashwin Hirschi-2
On Sat, Oct 29, 2005 at 09:24:05PM +0200, Ashwin Hirschi wrote:

> Lua strings may contain 0x00 characters.

  Great, that is reassuring.

> One way to improve the code returning the result would be to simply replace 
> it with:
> 
> 	lua_pushlstring(L, buffer, n);
  
  Thanks, I'm still pretty new to interfacng with Lua and haven't fully
 explored the functionas available.

> This would get you on your way to enable sending images, for instance 
> [something the current approach won't handle...].

  I can send images already ...

> You might want the check return value from the read() call as well.

  Noted, thanks for your help.

Steve
--