LuaSocket: No way to protect against fuzzing attacks?

classic Classic list List threaded Threaded
24 messages Options
12
Reply | Threaded
Open this post in threaded view
|

LuaSocket: No way to protect against fuzzing attacks?

Rena
I've been working on a simple HTTP server using LuaSocket. I wanted to
defend against "fuzzing" attacks, where an attacker sends endless
streams of junk (often at very slow rates) as HTTP headers, never
ending the line or request header, thus tying up the server's
resources as it sits endlessly waiting for/buffering header strings
for a request that never completes.

For example, an attacker might send an endless request line:
sock:send('GET /'); while true do sock:send('x') end
or a request with some endless header line:
sock:send('GET / HTTP/1.0\r\nX-Endless-String: '); while true do
sock:send('x') end

The obvious way to defend against this attack would be to reject the
request and close the connection if any line is too long.
Unfortunately LuaSocket doesn't appear to provide any way to do this,
other than (slowly!) reading one byte at a time. It provides only
three receive options:
1) Read until connection is closed (or timeout). Not useful for HTTP,
since the connection needs to remain open to send the response.
2) Read until an end-of-line character. Not helpful in this scenario,
since the line never ends.
3) Read a specified number of bytes.

Option 3 seems like the answer, but there's no way to read *up to* a
certain number of bytes. If I specify 32, and the client only sends
16, the read will block forever or time out (and will *not* return
those 16 bytes). So, the server would have to read one byte at a time,
or else fail to accept legitimate requests. Of course this is terrible
for performance.

To properly defend against this attack, the server needs to be able to either:
1) Peek into the receive buffer, and don't actually read the data
(removing it from the buffer) until a line break appears, rejecting
the connection if too much data is buffered without a line break, or:
2) Read all data received in the next x seconds or the next n bytes,
whichever comes first. (Bonus if "or until the next line break" is
also an option.) The server can then buffer lines in the same manner
as option 1.

Unfortunately, LuaSocket doesn't seem to offer methods to do either of
these, thus leaving every server using it vulnerable to such attacks.

--
Sent from my toaster.

Reply | Threaded
Open this post in threaded view
|

Re: LuaSocket: No way to protect against fuzzing attacks?

Peter Pimley
Could you repeatedly call option 2 (up to newline) but with a timeout
of zero?  It'll return the partial result, which you can buffer up
until you get the actual end-of-line.

Reply | Threaded
Open this post in threaded view
|

Re: LuaSocket: No way to protect against fuzzing attacks?

Sean Conner
In reply to this post by Rena
It was thus said that the Great HyperHacker once stated:

>
> Option 3 seems like the answer, but there's no way to read *up to* a
> certain number of bytes. If I specify 32, and the client only sends
> 16, the read will block forever or time out (and will *not* return
> those 16 bytes). So, the server would have to read one byte at a time,
> or else fail to accept legitimate requests. Of course this is terrible
> for performance.
>
> To properly defend against this attack, the server needs to be able to either:
> 1) Peek into the receive buffer, and don't actually read the data
> (removing it from the buffer) until a line break appears, rejecting
> the connection if too much data is buffered without a line break, or:
> 2) Read all data received in the next x seconds or the next n bytes,
> whichever comes first. (Bonus if "or until the next line break" is
> also an option.) The server can then buffer lines in the same manner
> as option 1.
>
> Unfortunately, LuaSocket doesn't seem to offer methods to do either of
> these, thus leaving every server using it vulnerable to such attacks.

  Unfortunately, the low level socket interface doesn't offer said methods
either (at least under Unix; I have no idea how Windows handle sockets at
the lower levels).  The three low level (read: this is what is ultimately
called) functions to receive data from a socket are:

        read (int fd,void *buf,size_t count);
        recv (int fd,void *buf,size_t count,int flags);
        recvfrom(int fd,void *buf,size_t count,int flags,struct sockaddr *addr,socklen_t *addrlen);

  The first parameter is the socket connection, the next two describe the
destination to read the data into and how much; flags modify what type of
data to read and usually this paramater is 0 but could be, at least under
Linux:

        MSG_PEEK read incoming data, but the data returns is still
                        treated as unread
        MSG_OOB retreive out of band data.  Protocol specific.
        MSG_WAITALL wait for all the data specified to be returned

  The last two parameters for recvfrom() fill in the remote address.

  Your possible answers still have issues:

1) peeking into the buffer is still subject to denial of service attack.  As
an attacker, all I need to do is:

        while(1)
        {
          write(yourwebserver,"x",1);
          sleep(1);
        }

And not only will you never see a line return, but I've tied up one
connection for a *very* long time until you consider I've sent a "very long
line".  Also, using the MSG_PEEK for this means memory is wasted in kernel
space buffering and thus may be a bad thing, depending upon the operating
system in question.

2) is probably your best bet, but you need to provide the proper buffering,
because at the lowest level, there is no "line buffering" of socket
connections.

  LuaSocket appears to be a wrapper around the recv() and recvfrom() calls
(at least for Unix).  These functions, as you may have noticed, do not
timeout on their own.  For timeouts, you have several options:

        1) use the alarm() function to interrupt the recv()/recvfrom() call.
        I do not recommend this route as it deals with signal handlers,
        which are horrendous to get right.

        2) select() or poll() (or more esoteric variations on these,
        depending upon the operating system).  The more esoteric variations
        tend to have better performance, but aren't portable from system to
        system (Linux has epoll(), Solaris uses /dev/poll, *BSD has
        kqueue()), but failing that, you want to use poll() and only if you
        have to, select().

  Yes, doing the line buffering is a pain.  I haven't used LuaSockets enough
to say if it does the buffering for you, but a quick search of the code does
show buffer_meth_receive() checking for a "*l", which leads me to believe it
may do such buffering, but you'll have to check the documentation (or the
code itself) to make sure.

  I can outline the code you'll need, but you'll need to fill in some
details.  It will look something like (system calls I'd use in C are
designated as system.something()):

        -- connection is established
        -- connection is sock
        lastread = system.gettimeofday() -- returns current time to microsecond
        readset = { sock }

        line = ""
        numreads = 0

        while true do
          rc = system.select(readset,nil,nil,TIMEOUT)
          if rc == 0 then
            handle_timeout()
          elseif rc < 0 then
            handle_error()
          end

          -- socket is read to read
          data = system.recv(sock,0) -- read data from socket, no flags
          if data:find("\n") then
            -- return line .. data, minus the portion past the '\n'
            -- which needs to be saved for the next line
            -- also, you can check here to see if the line is too long,
            -- or contains malicious data or what have you
          end
          line = line .. data
          numreads = numreads + 1
          if numreads > THRESHHOLD then
            -- we've made too many calls to recv(), handle
          end
          now = system.gettimeofday()
          if (now - lastread) > TOOMUCHTIME then
            -- or you could go by how long it's taken to reach this point
            -- in time.  
          end
          lastread = now
        end

  Code is untested pseudocode, but it's approximately what you need to do.
Lord knows I've written enough code (in assembly and C) like this in the
past.

  -spc (All abstractions leak, by the way ... )



Reply | Threaded
Open this post in threaded view
|

Re: LuaSocket: No way to protect against fuzzing attacks?

Sean Conner
In reply to this post by Peter Pimley
It was thus said that the Great Peter Pimley once stated:
> Could you repeatedly call option 2 (up to newline) but with a timeout
> of zero?  It'll return the partial result, which you can buffer up
> until you get the actual end-of-line.

  Yes, but performance will drop like a whale that suddenly finds itself
popped into existence at 30,000 feet (or a bowl of petunias, take your
pick) with a large amount traffic.  

  -spc (Used systems where code was written like that---it wasn't fun)


Reply | Threaded
Open this post in threaded view
|

RE: LuaSocket: No way to protect against fuzzing attacks?

Dean Sellers
In reply to this post by Rena
> Unfortunately LuaSocket doesn't appear to provide any way to do this,
> other than (slowly!) reading one byte at a time.

I have been writing a TCP binding for a proprietry protocol for an
embeded device. It is basically an <STX> <ETX> framed message. The
device that I am getting data from may fragment the message across a few
TCP packets. There is a maximum message length defined.

In doing this I had a look at the lua socket source, as I had a similar
issue to you. I wanted a method that was 'return up to n bytes'. It
seemed though that byte by byte reading isn't too much of a penalty. The
kernel delivers a full tcp packet that lua socket buffers. Your (the
user's) read-a-byte calls just return bytes from the buffer that lua
socket already has, unless it is empty in which case read() is called on
the operating system.

At least this was how the unix side worked, my code will always run on
an embedded nix device so I didn't delve into the windows source.

So after a bit of testing my algorithm reads byte by byte until a
message is formed or the length is exceeded. The performance is
certainly acceptable for a system where 20-100 byte messages are
delivered at 25Hz.

In saying this it would be reasonably easy to add a 'return up to n
bytes' method to luasocket, I just figured for the small performance hit
I would stay with a more portable solution.

--

Reply | Threaded
Open this post in threaded view
|

Re: LuaSocket: No way to protect against fuzzing attacks?

Rena
In reply to this post by Sean Conner
On Tue, Oct 11, 2011 at 02:37, Peter Pimley <[hidden email]> wrote:
> Could you repeatedly call option 2 (up to newline) but with a timeout
> of zero?  It'll return the partial result, which you can buffer up
> until you get the actual end-of-line.
>
>

Hmm, according to the manual, receive() should return the partial
result after the error message, but I'm getting inconsistent results.
If I use *l, it always returns the full result, but *l strips the line
break character, so I can't tell if the line has actually ended. If I
use *a or a large number (e.g. 1024), it never returns any result. If
I use a small number (e.g. 16), it returns partial results only in
some cases. It looks as if the last n bytes never get returned - some
sort of bug?


On Tue, Oct 11, 2011 at 02:56, Sean Conner <[hidden email]> wrote:

>  Unfortunately, the low level socket interface doesn't offer said methods
> either (at least under Unix; I have no idea how Windows handle sockets at
> the lower levels).  The three low level (read: this is what is ultimately
> called) functions to receive data from a socket are:
>
>        read    (int fd,void *buf,size_t count);
>        recv    (int fd,void *buf,size_t count,int flags);
>        recvfrom(int fd,void *buf,size_t count,int flags,struct sockaddr *addr,socklen_t *addrlen);
>
>  The first parameter is the socket connection, the next two describe the
> destination to read the data into and how much; flags modify what type of
> data to read and usually this paramater is 0 but could be, at least under
> Linux:
>
>        MSG_PEEK        read incoming data, but the data returns is still
>                        treated as unread
>        MSG_OOB         retreive out of band data.  Protocol specific.
>        MSG_WAITALL     wait for all the data specified to be returned
>
>  The last two parameters for recvfrom() fill in the remote address.
>
>  Your possible answers still have issues:
>
> 1) peeking into the buffer is still subject to denial of service attack.  As
> an attacker, all I need to do is:
>
>        while(1)
>        {
>          write(yourwebserver,"x",1);
>          sleep(1);
>        }
>
> And not only will you never see a line return, but I've tied up one
> connection for a *very* long time until you consider I've sent a "very long
> line".  Also, using the MSG_PEEK for this means memory is wasted in kernel
> space buffering and thus may be a bad thing, depending upon the operating
> system in question.
>

MSG_PEEK would do the trick nicely, and such "slow fuzzing" attacks
are trivially defeated by rejecting any requests that take too long to
fully receive. (If your connection is slow enough that even a simple
HTTP request takes so long, you're probably not going to receive the
reply any time soon either...)

--
Sent from my toaster.

Reply | Threaded
Open this post in threaded view
|

Re: LuaSocket: No way to protect against fuzzing attacks?

Peter Pimley
On 11 October 2011 11:07, HyperHacker <[hidden email]> wrote:

> Hmm, according to the manual, receive() should return the partial
> result after the error message, but I'm getting inconsistent results.
> If I use *l, it always returns the full result, but *l strips the line
> break character, so I can't tell if the line has actually ended

If the line did actually end then it'll be the first result from
receive().  If a timeout occured then the results will be nil,
"timeout" and then the partial line.

Sean Connor pointed out that repeated calls with a zero timeout will
burn performance.  Perhaps you could use a timeout that starts off as
the maximum reasonable request time, and gradually decreases after
each successful call.

Reply | Threaded
Open this post in threaded view
|

Re: LuaSocket: No way to protect against fuzzing attacks?

Tony Finch
In reply to this post by Rena
HyperHacker <[hidden email]> wrote:

> 2) Read all data received in the next x seconds or the next n bytes,
> whichever comes first.

s:settimeout()

Tony.
--
f.anthony.n.finch  <[hidden email]>  http://dotat.at/
Hebrides, Bailey: Northwest veering southeast 4 or 5, increasing 6 at times.
Rough, occasionally very rough at first in Hebrides. Rain or showers. Good,
occasionally moderate.

Reply | Threaded
Open this post in threaded view
|

Re: LuaSocket: No way to protect against fuzzing attacks?

Javier Guerra Giraldez
In reply to this post by Sean Conner
On Tue, Oct 11, 2011 at 4:05 AM, Sean Conner <[hidden email]> wrote:
> It was thus said that the Great Peter Pimley once stated:
>> Could you repeatedly call option 2 (up to newline) but with a timeout
>> of zero?  It'll return the partial result, which you can buffer up
>> until you get the actual end-of-line.
>
>  Yes, but performance will drop like a whale that suddenly finds itself
> popped into existence at 30,000 feet (or a bowl of petunias, take your
> pick) with a large amount traffic.

not if you only read when you already know there's data to read.


>  -spc (Used systems where code was written like that---it wasn't fun)

Copas and Xavante are written like that --- some people think they're fun :-)


--
Javier

Reply | Threaded
Open this post in threaded view
|

Re: LuaSocket: No way to protect against fuzzing attacks?

Stefan Reich
On Tue, Oct 11, 2011 at 2:23 PM, Javier Guerra Giraldez
<[hidden email]> wrote:
> Copas and Xavante are written like that --- some people think they're fun :-)

Yeah, I think this is totally a solvable problem.

Use socket.select to wait for data. Then read from all sockets with
data in the queue. Simple as that. You can't get stalled that way, no
matter how slowly someone sends you data.

On top of that, just drop any connection as soon as it appears not to
be worthwhile. (Takes too long to deliver data, or too many headers.)

Doesn't that solve all your "fuzzing attack" problems?

[I didn't know that it is called fuzzing btw. Is that a well-known
term this side of the great beltway? :)]

We should put up some HTTP servers and try to fuzz them.

(Well, if the spirit of this list was not disencouraging actual
collaboration, we would...)

Cheers,
Stefan

Reply | Threaded
Open this post in threaded view
|

Re: LuaSocket: No way to protect against fuzzing attacks?

Petite Abeille
In reply to this post by Rena

On Oct 11, 2011, at 9:37 AM, HyperHacker wrote:

> I've been working on a simple HTTP server using LuaSocket. I wanted to
> defend against "fuzzing" attacks, where an attacker sends endless
> streams of junk (often at very slow rates) as HTTP headers, never
> ending the line or request header, thus tying up the server's
> resources as it sits endlessly waiting for/buffering header strings
> for a request that never completes.

ulimit + timelimit?

[1] http://compute.cnr.berkeley.edu/cgi-bin/man-cgi?ulimit+2
[2] http://devel.ringlet.net/sysutils/timelimit/


Reply | Threaded
Open this post in threaded view
|

Re: LuaSocket: No way to protect against fuzzing attacks?

Sam Roberts
In reply to this post by Rena
On Tue, Oct 11, 2011 at 12:37 AM, HyperHacker <[hidden email]> wrote:
> To properly defend against this attack, the server needs to be able to either:
...
> 2) Read all data received in the next x seconds or the next n bytes,
> whichever comes first. (Bonus if "or until the next line break" is
> also an option.) The server can then buffer lines in the same manner
> as option 1.

If you :settimeout(t>0), then call receive(n/"*l"), and the timeout
occurs before n bytes/a line
has been read, you should get

  nil, "timeout", partial

as the result.

> Unfortunately, LuaSocket doesn't seem to offer methods to do either of
> these, thus leaving every server using it vulnerable to such attacks.

I think luasocket provides the required functionality, though you
might not be able to take advantage
of some of its high-level buffering.

If you can reliably reproduce it not behaving as documented or
describe why it is impossible to use, I might have time to fix it (I'm
collecting patches at https://github.com/sam-github/luasocket).

luasocket's buffering is convenient, but not necessarily what you want
for a server. And a multiplexing
server will need to set the timeouts to 0 on all sockets, and to use select().

If you do that, you will need to buffer the data yourself, and
evaluate your conditions for determining when a peer is misbehaving.
You'll have to track time associated with each connection, and buffer
the data for each connection yourself.

Apache, as I recall, has  a short timeout in which it requires seeing
some data, then a longer one to recv the complete head of the request.

Your buffering scheme can use the prefix argument to :receive(), if it
wishes, or use a table as a queue of segments, and concat them with
table.concat().

You might find the code below useful to explore some of these
behaviours, though it doesn't multiplex using select. As a start, try
calling it like:

./read-safely 4567 "*l" 4

with a client:

(sleep 10; echo -n helo; sleep 10; echo bye; sleep 10) | nc localhost 4567

Cheers,
Sam

------ read-safely -----
#!/usr/bin/env lua

require"socket"

if arg[1] == "-h" then
    print("usage: "..arg[0].." port pattern timeout")
    os.exit(1)
end

port = tonumber(arg[1]) or 0
pattern = arg[2] or 1024
timeout = arg[3] or 10

print("port", port, "pattern", pattern, "timeout", timeout)

serv = assert(socket.tcp())
assert(serv:setoption("reuseaddr", true))
assert(serv:bind("*", port))
assert(serv:listen())

print("at", serv:getsockname())

sock = assert(serv:accept())

print("from", sock:getpeername())

assert(sock:settimeout(timeout))

while true do
    mesg, emsg, part = sock:receive(pattern)

    if emsg then print("error", emsg) end

    assert(mesg or emsg == "timeout", emsg)

    if mesg then
        print("recv mesg", #mesg, "<"..mesg..">")
    elseif #part > 0 then
        print("recv part", #part, "<"..part..">")
    end
end

Reply | Threaded
Open this post in threaded view
|

Re: LuaSocket: No way to protect against fuzzing attacks?

Sam Roberts
In reply to this post by Dean Sellers
On Tue, Oct 11, 2011 at 2:47 AM, Dean Sellers <[hidden email]> wrote:
> At least this was how the unix side worked, my code will always run on
> an embedded nix device so I didn't delve into the windows source.

Windows is the same.

> So after a bit of testing my algorithm reads byte by byte until a
> message is formed or the length is exceeded. The performance is
> certainly acceptable for a system where 20-100 byte messages are
> delivered at 25Hz.

You might not have noticed that receive() will return a partial result after
the "timeout".

But as you say, if you are looking for a specific end byte,
byte-by-byte is reasonable. I did something like this for a serial
link protocol recently, where we needed to scan the input bytes for a
start-byte before starting to read a packet (I extended luasocket to
support serial devices). Also, table.concat or the prefix arg to
:receive() can get you away from the exponential cost of string tail
concatenation.

> In saying this it would be reasonably easy to add a 'return up to n
> bytes' method to luasocket, I just figured for the small performance hit
> I would stay with a more portable solution.

This is easy to write yourself in lua, see :read():

https://github.com/sam-github/swirl/blob/master/lua/sockext.lua

Its particularly useful for transparent proxies, where the code doing
the reading doesn't know what to expect from the network, it just
wants to pass on data as it becomes available. I might add this to my
luasocket fork sometime, unless somebody else provides a patch first.

Cheers,
Sam

Reply | Threaded
Open this post in threaded view
|

RE: LuaSocket: No way to protect against fuzzing attacks?

Dean Sellers
I did notice the partial result return, but like HyperHacker, I couldn't
make it work reliably. Sometimes I would get no partial result?

As you point out though for this protocol byte by byte makes sense, in
fact this protocol was originally designed as a serial line one. That is
probably why it seems similar.

I do use the prefix argument to receive, which works well.

One of my jobs (in the near future) is to add serial support to the lua
API I am writing, so if you are willing to share I would love to see
your implementation :-)

(As an aside when looking through the luasocket code I was impressed
with how easy serial support looked to add, at least for *nix.)

Dean Sellers applications engineer  
rinstrum smart weighing  
Ph: +61 7 3216 7166  Fax: +61 7 3216 6211    
[hidden email]  
41 Success St, Acacia Ridge, QLD, 4110
 
www.rinstrum.com

> -----Original Message-----
> From: [hidden email]
> [mailto:[hidden email]] On Behalf Of Sam Roberts
> Sent: Wednesday, 12 October 2011 5:46 AM
> To: Lua mailing list
> Subject: Re: LuaSocket: No way to protect against fuzzing attacks?
>
>
> On Tue, Oct 11, 2011 at 2:47 AM, Dean Sellers
> <[hidden email]> wrote:
> > At least this was how the unix side worked, my code will
> always run on
> > an embedded nix device so I didn't delve into the windows source.
>
> Windows is the same.
>
> > So after a bit of testing my algorithm reads byte by byte until a
> > message is formed or the length is exceeded. The performance is
> > certainly acceptable for a system where 20-100 byte messages are
> > delivered at 25Hz.
>
> You might not have noticed that receive() will return a
> partial result after
> the "timeout".
>
> But as you say, if you are looking for a specific end byte,
> byte-by-byte is reasonable. I did something like this for a serial
> link protocol recently, where we needed to scan the input bytes for a
> start-byte before starting to read a packet (I extended luasocket to
> support serial devices). Also, table.concat or the prefix arg to
> :receive() can get you away from the exponential cost of string tail
> concatenation.
>
> > In saying this it would be reasonably easy to add a 'return up to n
> > bytes' method to luasocket, I just figured for the small
> performance hit
> > I would stay with a more portable solution.
>
> This is easy to write yourself in lua, see :read():
>
> https://github.com/sam-github/swirl/blob/master/lua/sockext.lua
>
> Its particularly useful for transparent proxies, where the code doing
> the reading doesn't know what to expect from the network, it just
> wants to pass on data as it becomes available. I might add this to my
> luasocket fork sometime, unless somebody else provides a patch first.
>
> Cheers,
> Sam
>
>
> --
> This message was scanned by ESVA and is believed to be clean.
> Click here to report this message as spam.
> http://mailscanner.rinstrum.com.au/cgi-bin/learn-msg.cgi?id=02
B9B27DFB.DB40E



Reply | Threaded
Open this post in threaded view
|

Re: LuaSocket: No way to protect against fuzzing attacks?

Sam Roberts
On Tue, Oct 11, 2011 at 4:25 PM, Dean Sellers <[hidden email]> wrote:
> I did notice the partial result return, but like HyperHacker, I couldn't
> make it work reliably. Sometimes I would get no partial result?

Never seen that, but I remain interested to know if somebody can show
me a reproduction.

> As you point out though for this protocol byte by byte makes sense, in
> fact this protocol was originally designed as a serial line one. That is
> probably why it seems similar.

What's the protocol?

> One of my jobs (in the near future) is to add serial support to the lua
> API I am writing, so if you are willing to share I would love to see
> your implementation :-)

https://github.com/sam-github/luasocket

specifically:

https://github.com/sam-github/luasocket/commit/ed892af39ee334bcd732a8ca9dd8cad267fc15aa

Code is based off unix.c, too bad there isn't a better way to avoid
the code duplication, but
I don't want to disturb the luasocket code base too much, I still have
hopes Diego will merge at least the obvious
bug fixes, if not the features.

Using a serial device also requires setting its modes, for that I have
bindings to termios, that I should be able
to put up on github, too. Of course, you can always set those with stty.

Cheers,
Sam

Reply | Threaded
Open this post in threaded view
|

Re: LuaSocket: No way to protect against fuzzing attacks?

Rena
On Tue, Oct 11, 2011 at 18:25, Sam Roberts <[hidden email]> wrote:

> On Tue, Oct 11, 2011 at 4:25 PM, Dean Sellers <[hidden email]> wrote:
>> I did notice the partial result return, but like HyperHacker, I couldn't
>> make it work reliably. Sometimes I would get no partial result?
>
> Never seen that, but I remain interested to know if somebody can show
> me a reproduction.
>
>> As you point out though for this protocol byte by byte makes sense, in
>> fact this protocol was originally designed as a serial line one. That is
>> probably why it seems similar.
>
> What's the protocol?
>
>> One of my jobs (in the near future) is to add serial support to the lua
>> API I am writing, so if you are willing to share I would love to see
>> your implementation :-)
>
> https://github.com/sam-github/luasocket
>
> specifically:
>
> https://github.com/sam-github/luasocket/commit/ed892af39ee334bcd732a8ca9dd8cad267fc15aa
>
> Code is based off unix.c, too bad there isn't a better way to avoid
> the code duplication, but
> I don't want to disturb the luasocket code base too much, I still have
> hopes Diego will merge at least the obvious
> bug fixes, if not the features.
>
> Using a serial device also requires setting its modes, for that I have
> bindings to termios, that I should be able
> to put up on github, too. Of course, you can always set those with stty.
>
> Cheers,
> Sam
>
>

I can't seem to reproduce the missing-last-bytes issue except in my
server, which is rather large and complex for a testcase and is using
threading and the acceptfd patch. (I did test acceptfd in a smaller
script as well.) Who knows why that's happening.

Anyway using *l I do get the partial result after the 'timeout'
message, so it looks like it is possible (although a bit ugly) to
buffer lines in my own applications and reject them if they're too
long (or too numerous, or arriving too slowly, etc). Thanks for
pointing that out (whomever did that!) :-)

--
Sent from my toaster.

Reply | Threaded
Open this post in threaded view
|

Re: LuaSocket: No way to protect against fuzzing attacks?

Sam Roberts
On Tue, Oct 11, 2011 at 10:36 PM, HyperHacker <[hidden email]> wrote:
> I can't seem to reproduce the missing-last-bytes issue except in my
> server, which is rather large and complex for a testcase and is using
> threading and the acceptfd patch. (I did test acceptfd in a smaller
> script as well.) Who knows why that's happening.

What's the acceptfd patch?

Instrument the code with some cutting edge technology, printf, in the
buffer sections to allow you to see what's happening, if you want to
track this down. Make sure your threads aren't eating each other's
data, and its not the CR/LFs that you are missing.

Cheers,
Sam

Reply | Threaded
Open this post in threaded view
|

Re: LuaSocket: No way to protect against fuzzing attacks?

Matthew Wild
On 12 October 2011 17:16, Sam Roberts <[hidden email]> wrote:
> On Tue, Oct 11, 2011 at 10:36 PM, HyperHacker <[hidden email]> wrote:
>> I can't seem to reproduce the missing-last-bytes issue except in my
>> server, which is rather large and complex for a testcase and is using
>> threading and the acceptfd patch. (I did test acceptfd in a smaller
>> script as well.) Who knows why that's happening.
>
> What's the acceptfd patch?

It allows creating a LuaSocket object from an fd. Would be very
useful, but not provided in standard LuaSocket because of portability
concerns (ie. Windows).

Regards,
Matthew

Reply | Threaded
Open this post in threaded view
|

Re: LuaSocket: No way to protect against fuzzing attacks?

Petite Abeille

On Oct 12, 2011, at 7:42 PM, Matthew Wild wrote:

>> What's the acceptfd patch?
>
> It allows creating a LuaSocket object from an fd. Would be very
> useful, but not provided in standard LuaSocket because of portability
> concerns (ie. Windows).

http://www.net-core.org/39/lua/patching-luasocket-to-make-it-compatible
http://www.net-core.org/dl/luasocket-2.0.2-acceptfd.patch

Reply | Threaded
Open this post in threaded view
|

Re: LuaSocket: No way to protect against fuzzing attacks?

Rena
In reply to this post by Sam Roberts
On Wed, Oct 12, 2011 at 10:16, Sam Roberts <[hidden email]> wrote:

> On Tue, Oct 11, 2011 at 10:36 PM, HyperHacker <[hidden email]> wrote:
>> I can't seem to reproduce the missing-last-bytes issue except in my
>> server, which is rather large and complex for a testcase and is using
>> threading and the acceptfd patch. (I did test acceptfd in a smaller
>> script as well.) Who knows why that's happening.
>
> What's the acceptfd patch?
>
> Instrument the code with some cutting edge technology, printf, in the
> buffer sections to allow you to see what's happening, if you want to
> track this down. Make sure your threads aren't eating each other's
> data, and its not the CR/LFs that you are missing.
>
> Cheers,
> Sam
>
>

Well, each socket is only ever used by one thread.The design is pretty simple:
-Main thread accepts connections using acceptfd() (returns a file
descriptor number instead of a socket object), starts a thread, and
passes it the fd
-Child thread opens a socket from the fd, handles the request, and closes it
The child thread is basically doing sock = socket.tcp(fd);
sock:receive(n) for some n, and the last n bytes get dropped. Since
the socket isn't touched by any other thread, and all other socket I/O
(including using *l instead in the same place) works as expected, the
only potential causes I can see are some issue in LuaSocket (possibly
introduced by the patch) or some quite bizarre race condition breaking
something somewhere.

--
Sent from my toaster.

12