Pooling of strings is good

classic Classic list List threaded Threaded
103 messages Options
123456
Reply | Threaded
Open this post in threaded view
|

Speed of Lua's immutable strings vs buffers (split from Re: Pooling of strings is good)

Sean Conner
It was thus said that the Great Coroutines once stated:
> On Mon, Aug 25, 2014 at 1:14 PM, Roberto Ierusalimschy
> <[hidden email]> wrote:
> >> Earlier in this long and unproductive email chain [...]
> >
> > On which you created more than 40% of the messages...
>
> Good, so you understand my disappointment -- having repeatedly
> clarified what I'm looking to do for those who didn't read.  Now that
> everybody is caught up we can start being productive ^__^

  For relative values of "productive."

  Seeing how I'm probably responsible for the other 40% of the messages, and
to be "more productive" I spent the past two days writing code to measure
the overhead of using Lua's immutable strings vs. buffers.  

  The thesis:  using Lua's immutable strings for saving network buffers is
more inefficient than using a reusable buffer.

  Coroutines:  it is more effieicent to reuse a buffer, so let's rework Lua
completely so we have both mutable and immutable strings.

  Me: it's not worth the effort, and mutable strings can cause more problems
than the benefits.

  So, the code.  It's all here:

        https://github.com/spc476/lua-net-speed

  I have a program in C that sends packets as fast as possible to the
destination.  All the packets are the same size (but the size is
configurable on the command line) and the program runs for a set amount of
time.  The results are from running the following command line (run for 10
seconds, use a packet size of 1450 bytes):

        ./blast -a 127.0.0.1 -p 22222 -t 10 -s 1450

on a 2.6GHz dual-core 32-bit pentium running Linux.  The code presented is
compiled using no optimization (just gcc); the Lua interpreter is the sock
compile from PUC (this is Lua 5.1 by the way---I'm not set up to use Lua
5.2). While I have a C version and a Lua version of the same program (the C
version is all C, except it passes the packet to a Lua function) I only have
a Lua version of the "buffer userdata" but that was enough to show an
interesting result.

  The function being run for each packet is (something.lua):

        counts = {}
        for i = 0 , 255 do
          counts[i] = 0
        end

        function main(packet)
          local max = #packet
          for i = 1 , #packet do
            local c = packet:byte(i)
            counts[c] = counts[c] + 1
          end
        end

  The intent was to do some processing over the packet.  Immutable strings
have the byte() method defined, and for the buffer userdata, I implemented
the same function (in fact, using the same code from PUC Lua, lifted 99%
straight---the only change was from using a const char * to a userdata) for
the buffer userdata so the same code could be used for both versions.  The
programs used to receive the data are written in Lua, and are suck.lua
(immutable strings) and suckb.lua (userdata buffer).  I ran each three times
and am presenting the results for each run.

        Using Lua immutable strings:
        Packets sent Packets recived Memory used in K
        by blast.c by suck.lua by suck.lua

        1442627 12003 67
        1442889 12090 85
        1440660 12063 54

I know, it looks horrible---1.4 million packets sent, 12,000 processed?
Buffers surely must do better, right?

        Using a buffer userdata:
        Packets sent Packets received Memory used in K
        by blast.c by suckb.lua by suckb.lua

        1440542 7612 57
        1441593 8306 91
        1447008 8278 87

  I am willing to conceed an error in my methodology, but the ball is now in
Coroutines' court to show where I am in error.

  On a related note---for drastic changes like introducing mutable strings
to Lua, it should be up to the presenter to do the work, to do the
implementation (even if it does duplicate code) to show the idea is sound.
Working code trumps wishes and laments.  I applaud Jan Behrens' work on
ipairs because he's taking the time to implement his ideas.

  -spc


Reply | Threaded
Open this post in threaded view
|

Re: Pooling of strings is good

Robert Virding-2
In reply to this post by Coroutines
Got into this discussion late.

If you have pooling of strings how can you safely use them for dynamic data, for example from input, if you want the system to survive for any length of time? Or have I misunderstood what is meant by pooling?

Robert



On 25 August 2014 22:29, Coroutines <[hidden email]> wrote:
On Mon, Aug 25, 2014 at 1:14 PM, Roberto Ierusalimschy
<[hidden email]> wrote:
>> Earlier in this long and unproductive email chain [...]
>
> On which you created more than 40% of the messages...

Good, so you understand my disappointment -- having repeatedly
clarified what I'm looking to do for those who didn't read.  Now that
everybody is caught up we can start being productive ^__^


Reply | Threaded
Open this post in threaded view
|

Re: Pooling of strings is good

Coroutines
On Mon, Aug 25, 2014 at 3:07 PM, Robert Virding <[hidden email]> wrote:
> Got into this discussion late.
>
> If you have pooling of strings how can you safely use them for dynamic data,
> for example from input, if you want the system to survive for any length of
> time? Or have I misunderstood what is meant by pooling?

Hmm, please elaborate :>  We've been talking about pooled strings in
the sense that once a string is created -- it does not change.  Not
only that, but it is checked for equality against the existing strings
Lua has already "seen"/interned.  When a string is created a hash is
computed based on the contents of that string.  So when you compare
'cat' == 'dog' Lua is internally doing a comparison of the hash of
"cat" to the hash of "dog" -- but it also may be as simple as a
pointer comparison.  (I don't know for sure in every instance)

Reply | Threaded
Open this post in threaded view
|

Re: Pooling of strings is good

Robert Virding-2
OK, this is what I understood by pooling.

What I was wondering is that if you have a system where each new string is interned then the string table will keep growing with new strings. This does make for very fast checking of string equality. However, if you have dynamically created strings, for example data coming into the system from the outside, then the string table will grow and eventually crash the system.

While I definitely agree that strings should be immutable, immutable data is good, this means that working with strings will dynamically create new strings all the time which will exacerbate the problem. Or are they removed by the garbage collector?

We have the same problem in Erlang with atoms which are interned and dynamically creating atoms in an uncontrolled way will eventually crash the system. The solution there is to avoid dynamically creating atom but use binaries/string instead which are not interned.

This is why I was wondering how Lua attacked this problem.

Robert



On 26 August 2014 00:19, Coroutines <[hidden email]> wrote:
On Mon, Aug 25, 2014 at 3:07 PM, Robert Virding <[hidden email]> wrote:
> Got into this discussion late.
>
> If you have pooling of strings how can you safely use them for dynamic data,
> for example from input, if you want the system to survive for any length of
> time? Or have I misunderstood what is meant by pooling?

Hmm, please elaborate :>  We've been talking about pooled strings in
the sense that once a string is created -- it does not change.  Not
only that, but it is checked for equality against the existing strings
Lua has already "seen"/interned.  When a string is created a hash is
computed based on the contents of that string.  So when you compare
'cat' == 'dog' Lua is internally doing a comparison of the hash of
"cat" to the hash of "dog" -- but it also may be as simple as a
pointer comparison.  (I don't know for sure in every instance)


Reply | Threaded
Open this post in threaded view
|

Re: Speed of Lua's immutable strings vs buffers (split from Re: Pooling of strings is good)

Coroutines
In reply to this post by Sean Conner
On Mon, Aug 25, 2014 at 3:02 PM, Sean Conner <[hidden email]> wrote:

>   So, the code.  It's all here:
>
>         https://github.com/spc476/lua-net-speed

Hmm.

Okay, first I want to say that I really appreciate you going through
all the effort to bootstrap this comparison.

I think we may have a misunderstanding though -- my objective was
mostly to save on reallocating the same buffer, as I cannot mutate it
once it becomes a string.  I think there might be a potential for this
to be faster by always having a fixed-sized buffer allocated to copy
out to a string (if needed), but mostly this was about avoiding
duplicating the buffer that gets recv()'d into :>  I am still
interested in this comparson on speed, however.

>   I am willing to conceed an error in my methodology, but the ball is now in
> Coroutines' court to show where I am in error.

I *want* to disagree with how this benchmark is done because even if
the networking portion of this comparison is done correctly it's using
your socket library and I have my own -- and others have luasocket.
It would take some time to properly critique all 3 and even set up all
3 for this benchmark to get a fairer view of things.

I'm going to try to take a day to look at all of this but I want to
point out that in suck.lua `data' is a local defined within the block
-- it is created from the :recv() call on every iteration.  In
suckb.lua `data' is a global declared on line 61 -- I think the global
lookup as `data' is passed to :recv() might have some cost in this.  I
don't believe this should make very much of a difference -- if these
results are to be believed I'm shocked you can get away with
allocating such a large buffer for the string-returning :recv() so
often.

Besides that, I think your recv()/recvb() functions could be "faster"
for both/overall if they avoided allocating for the peer address
userdata/sockaddr -- you can get the remote peer information with
getpeername().  As I'm sure you know, recv/recvfrom/recvmsg are
functions that are expected to be called quite often -- allocating for
the peer sockaddr is bad (imo) on every call to them.  I also think
it's odd that recv() calls recvfrom(), but there should be no
difference in the results.

You did a fair benchmark iterating over every :byte() in the buffer
and string -- I think this is also a fair benchmark in terms of
"real-world" usage even if I feel it should have just been focused on
duplicating and running the string library over a mutating userdata.
I would just like to point out that I am trying to use string.find()
or string.match() on the string/buffer I get from :recv() -- this
would be my common usage.  string.find() search through the string or
buffer/userdata faster than doing it byte-by-byte from Lua -- how I
use string.match() would typically create smaller strings from the
"recv buffer" (whatever it is), but locating what to substring would
go faster than traversing the buffer from Lua (the pattern-matching C
stuff would do it for me).

>   On a related note---for drastic changes like introducing mutable strings
> to Lua, it should be up to the presenter to do the work, to do the
> implementation (even if it does duplicate code) to show the idea is sound.
> Working code trumps wishes and laments.  I applaud Jan Behrens' work on
> ipairs because he's taking the time to implement his ideas.

Twist that dagger a little deeper, eh? :p  Introducing a buffer type
and metatable interface to it was not my idea.  I was hoping to learn
from upstream how I might trick Lua's typechecking functions into
believing userdata were strings and getting it to safely access the
->buf portion of the userdata (rather than the similar data member in
the TString struct).

Reply | Threaded
Open this post in threaded view
|

Re: Pooling of strings is good

Jay Carlson
In reply to this post by Coroutines
On Aug 25, 2014, at 6:19 PM, Coroutines <[hidden email]> wrote:

> When a string is created a hash is computed based on the contents of that string.

In 5.2.1 and later, strings longer than LUAI_MAXSHORTLEN (default 40) are not interned on creation.
Reply | Threaded
Open this post in threaded view
|

Re: Pooling of strings is good

Coroutines
In reply to this post by Robert Virding-2
On Mon, Aug 25, 2014 at 3:45 PM, Robert Virding <[hidden email]> wrote:

> While I definitely agree that strings should be immutable, immutable data is
> good, this means that working with strings will dynamically create new
> strings all the time which will exacerbate the problem. Or are they removed
> by the garbage collector?

Typically people don't leave around a lot of references to unique data
they don't plan to process again later -- I imagine the GC saves them,
yes :>

> We have the same problem in Erlang with atoms which are interned and
> dynamically creating atoms in an uncontrolled way will eventually crash the
> system. The solution there is to avoid dynamically creating atom but use
> binaries/string instead which are not interned.

I am not sure why creating many unique atoms crashes the system but
using binary/strings would not.  Surely deduplicated atoms would avoid
out-of-memory situations better than duplicated binary/strings.

What I'm talking about here is I know the buffer I recv() into will
usually be unique, mutates often, and has a short lifespan -- it would
seem appropriate for the buffer to be mutable (represented as
userdata).  What I've been talking about is trying to trick Lua into
thinking userdata is really a string, and accessing it safely (looking
at the data portion of userdata (which doesn't start in the same place
as the data portion of a Lua string)).

Reply | Threaded
Open this post in threaded view
|

Re: Pooling of strings is good

Coroutines
In reply to this post by Jay Carlson
On Mon, Aug 25, 2014 at 4:07 PM, Jay Carlson <[hidden email]> wrote:

> In 5.2.1 and later, strings longer than LUAI_MAXSHORTLEN (default 40) are not interned on creation.

Oh goodie, we can all go home now :D

...naah just kidding, I still need a way to present mutable userdata
to functions expecting strings without actually being strings :p

Reply | Threaded
Open this post in threaded view
|

Re: Speed of Lua's immutable strings vs buffers (split from Re: Pooling of strings is good)

Sean Conner
In reply to this post by Coroutines
It was thus said that the Great Coroutines once stated:

> On Mon, Aug 25, 2014 at 3:02 PM, Sean Conner <[hidden email]> wrote:
>
> >   So, the code.  It's all here:
> >
> >         https://github.com/spc476/lua-net-speed
>
> Hmm.
>
> Okay, first I want to say that I really appreciate you going through
> all the effort to bootstrap this comparison.
>
> I think we may have a misunderstanding though -- my objective was
> mostly to save on reallocating the same buffer, as I cannot mutate it
> once it becomes a string.  I think there might be a potential for this
> to be faster by always having a fixed-sized buffer allocated to copy
> out to a string (if needed), but mostly this was about avoiding
> duplicating the buffer that gets recv()'d into :>  I am still
> interested in this comparson on speed, however.

  The suckb.lua script does allocate the buffer once, and it is of a fixed
size.  The __len() function returns the currently used amount of the buffer.

> >   I am willing to conceed an error in my methodology, but the ball is
> > now in Coroutines' court to show where I am in error.
>
> I *want* to disagree with how this benchmark is done because even if
> the networking portion of this comparison is done correctly it's using
> your socket library and I have my own -- and others have luasocket.
> It would take some time to properly critique all 3 and even set up all
> 3 for this benchmark to get a fairer view of things.
>
> I'm going to try to take a day to look at all of this but I want to
> point out that in suck.lua `data' is a local defined within the block
> -- it is created from the :recv() call on every iteration.  In
> suckb.lua `data' is a global declared on line 61 -- I think the global
> lookup as `data' is passed to :recv() might have some cost in this.  I
> don't believe this should make very much of a difference -- if these
> results are to be believed I'm shocked you can get away with
> allocating such a large buffer for the string-returning :recv() so
> often.

  Fair criticism, and fixed in the repo.  I hoisted the definitions out of
the main loop, and for suckb.lua, I made sure that data was declared local.
I reran the tests but the results were similar as before.

> Besides that, I think your recv()/recvb() functions could be "faster"
> for both/overall if they avoided allocating for the peer address
> userdata/sockaddr -- you can get the remote peer information with
> getpeername().  As I'm sure you know, recv/recvfrom/recvmsg are
> functions that are expected to be called quite often -- allocating for
> the peer sockaddr is bad (imo) on every call to them.  I also think
> it's odd that recv() calls recvfrom(), but there should be no
> difference in the results.

  There are four functions you can use to read data from a socket:

        read()
        recv()
        recvfrom()
        recvmsg()

  read() is fine for TCP and for "connected" UDP sockets [1], but for
unconnected TCP (or even packets for other IP protocols like OSPF) you can't
use read().  That's one reason I rejected using read().

  Of the three left, recv() is similar to recvfrom(), but drops the remote
address, and recvfrom() can be written in terms of recvmsg() but with a bit
more work, so from that standpoint, I decided to just use recvfrom(), but
name the routine "sock:recv()" on the Lua side.  I mostly work with
unconnected UDP, so having sock:recv() push the address is required.  It
would be easy enough to comment out that part of the code to test, but I
doubt it'll give any significant performance boost (but I'm willing to be
surprised).

  But that's the reasoning behind the way I wrote the code.

> You did a fair benchmark iterating over every :byte() in the buffer
> and string -- I think this is also a fair benchmark in terms of
> "real-world" usage even if I feel it should have just been focused on
> duplicating and running the string library over a mutating userdata.

  The data has to come from somewhere.  Another approach would be to read
"/dev/urandom", but I felt like doing this via the network would be more in
tune towards the type of work you are doing with Lua.

> >   On a related note---for drastic changes like introducing mutable strings
> > to Lua, it should be up to the presenter to do the work, to do the
> > implementation (even if it does duplicate code) to show the idea is sound.
> > Working code trumps wishes and laments.  I applaud Jan Behrens' work on
> > ipairs because he's taking the time to implement his ideas.
>
> Twist that dagger a little deeper, eh? :p  

  It's a fair cop 8-P

> Introducing a buffer type and metatable interface to it was not my idea. I
> was hoping to learn from upstream how I might trick Lua's typechecking
> functions into believing userdata were strings and getting it to safely
> access the ->buf portion of the userdata (rather than the similar data
> member in the TString struct).

  Harder than it sounds.  One thougt I did have was to "tweak" the type byte
in the common header from LUA_TUSERDATA to LUA_TSTRING, but then I looked at
the defintions of both strings and userdata:

typedef union TString {
  L_Umaxalign dummy;  /* ensures maximum alignment for strings */
  struct {
    CommonHeader;
    lu_byte reserved;
    unsigned int hash;
    size_t len;
  } tsv;
} TString;

typedef union Udata {
  L_Umaxalign dummy;  /* ensures maximum alignment for ocal' udata */
  struct {
    CommonHeader;
    struct Table *metatable;
    struct Table *env;
    size_t len;
  } uv;
} Udata;

  You *MIGHT* get away with that on a 32-bit system where sizeof(pointer) ==
sizeof(int) == sizeof(lu_byte + padding) and nothing touches the hash,
metatable or env fields of either structure.  But on a 64-bit system, I
woudn't count on sizeof(tsv) (from TString) being equal to sizeof(uv) (from
Udata).

  -spc

[1] A call to connect() on a UDP socket associates a remote/local pair
        of addresses to the socket.  The downside is that you can only
        receive packets from the remote address you called connect() on.

Reply | Threaded
Open this post in threaded view
|

Re: Pooling of strings is good

William Ahern
In reply to this post by Robert Virding-2
On Tue, Aug 26, 2014 at 12:45:54AM +0200, Robert Virding wrote:
> OK, this is what I understood by pooling.
>
> What I was wondering is that if you have a system where each new string is
> interned then the string table will keep growing with new strings. This
> does make for very fast checking of string equality. However, if you have
> dynamically created strings, for example data coming into the system from
> the outside, then the string table will grow and eventually crash the
> system.

Strings are garbage collected just like any other object. All string
interning does it make sure that there exists only one copy of a string in
the VM. So in every conceivable scenario Lua will use no more memory than
with non-interned strings, and typically much less memory, even in the case
when you're generating many temporary strings.

In Java strings are immutable so that they can be interned as an
optimization. All compile-time constant strings in Java are interned.

Memory bloat can be an issue in Lua (and Java!) not because of string
interning, but because of immutable strings. This is less of an issue in Lua
than Java because in Lua all strings are interned, which minimizes the
number of unique string objects. But much like Java the idiomatic
"workaround" when it does become a problem is to implement a buffer object,
e.g. Java's StringBuffer.

Lua 5.2 added a generational collector, which could help alleviate excessive
garbage strings by quickly collecting unused strings. However, in reality
nobody had much of an issue with bloat in this regard, at least not so much
that anybody bothered to speak up to prevent the generational collector from
being removed from 5.3.

Reply | Threaded
Open this post in threaded view
|

Re: Speed of Lua's immutable strings vs buffers (split from Re: Pooling of strings is good)

Coroutines
In reply to this post by Sean Conner
On Mon, Aug 25, 2014 at 4:38 PM, Sean Conner <[hidden email]> wrote:

>   The suckb.lua script does allocate the buffer once, and it is of a fixed
> size.  The __len() function returns the currently used amount of the buffer.

Oh I know this -- I just thought your comparison was based on how
quickly you can write to the buffer and process it meaningfully (which
is a fair benchmark), but when I said "efficiently" I meant there are
times when you might be so constrained by memory that you cannot
afford to duplicate the recv() buffer by duplicating it into a string
you can then process/parse.  Conceptually, being able to read and
mutate userdata directly without creating "string clones" seems more
efficient.  I still want to take a closer look at this benchmark
though -- IT"S NOT OVER MY DEAR SEAN! BUWHAHAHAHAHAHAhahhahaha.  haha.
ha.

>   Fair criticism, and fixed in the repo.  I hoisted the definitions out of
> the main loop, and for suckb.lua, I made sure that data was declared local.
> I reran the tests but the results were similar as before.

Blah, I didn't think it would account for much -- but I still hoped
for a magic difference :-)

>   There are four functions you can use to read data from a socket:
>
>         read()
>         recv()
>         recvfrom()
>         recvmsg()
>
>   read() is fine for TCP and for "connected" UDP sockets [1], but for
> unconnected TCP (or even packets for other IP protocols like OSPF) you can't
> use read().  That's one reason I rejected using read().

Secret:  You can associate the remote peer (sockaddr) with a UDP
socket by calling connect() -- after that I believe you can call
read() like normal.  The man page says you can only connect() once for
TCP sockets, but you can call connect() multiple times for UDP
sockets.  This results in the kernel associating the remote peer with
that socket in the networking stack -- outwardly it doesn't seem to
have any effect.  People think because you only have an fd (an int)
that nothing happened -- buwhahaha bsd socket juju.  I know for sure
is how you can use recv() instead of recvfrom() on a UDP socket.

>   Harder than it sounds.  One thougt I did have was to "tweak" the type byte
> in the common header from LUA_TUSERDATA to LUA_TSTRING, but then I looked at
> the defintions of both strings and userdata:
>
> typedef union TString {
>   L_Umaxalign dummy;  /* ensures maximum alignment for strings */
>   struct {
>     CommonHeader;
>     lu_byte reserved;
>     unsigned int hash;
>     size_t len;
>   } tsv;
> } TString;
>
> typedef union Udata {
>   L_Umaxalign dummy;  /* ensures maximum alignment for ocal' udata */
>   struct {
>     CommonHeader;
>     struct Table *metatable;
>     struct Table *env;
>     size_t len;
>   } uv;
> } Udata;
>
>   You *MIGHT* get away with that on a 32-bit system where sizeof(pointer) ==
> sizeof(int) == sizeof(lu_byte + padding) and nothing touches the hash,
> metatable or env fields of either structure.  But on a 64-bit system, I
> woudn't count on sizeof(tsv) (from TString) being equal to sizeof(uv) (from
> Udata).

Hmm this will take some time for me to understand :>  Thanks for
finding the relevant bits for me <3

Reply | Threaded
Open this post in threaded view
|

Re: Speed of Lua's immutable strings vs buffers (split from Re: Pooling of strings is good)

Sean Conner
It was thus said that the Great Coroutines once stated:

> On Mon, Aug 25, 2014 at 4:38 PM, Sean Conner <[hidden email]> wrote:
>
> >   There are four functions you can use to read data from a socket:
> >
> >         read()
> >         recv()
> >         recvfrom()
> >         recvmsg()
> >
> >   read() is fine for TCP and for "connected" UDP sockets [1], but for
> > unconnected TCP (or even packets for other IP protocols like OSPF) you can't
> > use read().  That's one reason I rejected using read().
>
> Secret:  You can associate the remote peer (sockaddr) with a UDP
> socket by calling connect() -- after that I believe you can call
> read() like normal.

  Sure, on the client side.  On the server side, there is no UDP equivalent
of listen().  Yes, you can use recv() only on the server side, but then you
remove any chance of sending back a reply (note:  this applies to UDP).  

> >   Harder than it sounds.  One thougt I did have was to "tweak" the type byte
> > in the common header from LUA_TUSERDATA to LUA_TSTRING, but then I looked at
> > the defintions of both strings and userdata:
> >
> > typedef union TString {
> >   L_Umaxalign dummy;  /* ensures maximum alignment for strings */
> >   struct {
> >     CommonHeader;
> >     lu_byte reserved;
> >     unsigned int hash;
> >     size_t len;
> >   } tsv;
> > } TString;
> >
> > typedef union Udata {
> >   L_Umaxalign dummy;  /* ensures maximum alignment for ocal' udata */
> >   struct {
> >     CommonHeader;
> >     struct Table *metatable;
> >     struct Table *env;
> >     size_t len;
> >   } uv;
> > } Udata;
> >
> >   You *MIGHT* get away with that on a 32-bit system where sizeof(pointer) ==
> > sizeof(int) == sizeof(lu_byte + padding) and nothing touches the hash,
> > metatable or env fields of either structure.  But on a 64-bit system, I
> > woudn't count on sizeof(tsv) (from TString) being equal to sizeof(uv) (from
> > Udata).
>
> Hmm this will take some time for me to understand :>  Thanks for
> finding the relevant bits for me <3

  You mean you didn't even bother to do *this*?  <headdesk>

  -spc

Reply | Threaded
Open this post in threaded view
|

Re: Speed of Lua's immutable strings vs buffers (split from Re: Pooling of strings is good)

Coroutines
On Mon, Aug 25, 2014 at 5:03 PM, Sean Conner <[hidden email]> wrote:

>   Sure, on the client side.  On the server side, there is no UDP equivalent
> of listen().  Yes, you can use recv() only on the server side, but then you
> remove any chance of sending back a reply (note:  this applies to UDP).

I believe this is still done with connect() to write()/send() to the
associated peer.

Reply | Threaded
Open this post in threaded view
|

Re: Speed of Lua's immutable strings vs buffers (split from Re: Pooling of strings is good)

Sean Conner
It was thus said that the Great Coroutines once stated:
> On Mon, Aug 25, 2014 at 5:03 PM, Sean Conner <[hidden email]> wrote:
>
> >   Sure, on the client side.  On the server side, there is no UDP equivalent
> > of listen().  Yes, you can use recv() only on the server side, but then you
> > remove any chance of sending back a reply (note:  this applies to UDP).
>
> I believe this is still done with connect() to write()/send() to the
> associated peer.

  UDP is connectionless.  You have to recvfrom() to get the remote address
before you can call connect(), which now binds that socket to that one
remote address.  

  -spc


Reply | Threaded
Open this post in threaded view
|

Re: Speed of Lua's immutable strings vs buffers (split from Re: Pooling of strings is good)

Coroutines
On Mon, Aug 25, 2014 at 5:40 PM, Sean Conner <[hidden email]> wrote:

>   UDP is connectionless.  You have to recvfrom() to get the remote address
> before you can call connect(), which now binds that socket to that one
> remote address.

Oh that is true -- I was talking about where either side knows "who"
the peer will be.  Can't know that if you're a timeserver or
something...

Reply | Threaded
Open this post in threaded view
|

Re: Speed of Lua's immutable strings vs buffers (split from Re: Pooling of strings is good)

Sean Conner
In reply to this post by Coroutines
It was thus said that the Great Coroutines once stated:
>
> Besides that, I think your recv()/recvb() functions could be "faster"
> for both/overall if they avoided allocating for the peer address
> userdata/sockaddr -- you can get the remote peer information with
> getpeername().  As I'm sure you know, recv/recvfrom/recvmsg are
> functions that are expected to be called quite often -- allocating for
> the peer sockaddr is bad (imo) on every call to them.  I also think
> it's odd that recv() calls recvfrom(), but there should be no
> difference in the results.

  I made the change from using recvfrom() to recv() and thus, avoid
allocating space for the peer address for each packet received, and reran
the test.  No difference in the results.  I did not check in the changes
(they're easy enough to do if you want to test yourself).

  -spc


Reply | Threaded
Open this post in threaded view
|

Re: Speed of Lua's immutable strings vs buffers (split from Re: Pooling of strings is good)

William Ahern
In reply to this post by Coroutines
On Mon, Aug 25, 2014 at 04:54:53PM -0700, Coroutines wrote:
> On Mon, Aug 25, 2014 at 4:38 PM, Sean Conner <[hidden email]> wrote:
<snip>

> >   read() is fine for TCP and for "connected" UDP sockets [1], but for
> > unconnected TCP (or even packets for other IP protocols like OSPF) you can't
> > use read().  That's one reason I rejected using read().
>
> Secret:  You can associate the remote peer (sockaddr) with a UDP
> socket by calling connect() -- after that I believe you can call
> read() like normal.  The man page says you can only connect() once for
> TCP sockets, but you can call connect() multiple times for UDP
> sockets.  This results in the kernel associating the remote peer with
> that socket in the networking stack -- outwardly it doesn't seem to
> have any effect.  People think because you only have an fd (an int)
> that nothing happened -- buwhahaha bsd socket juju.  I know for sure
> is how you can use recv() instead of recvfrom() on a UDP socket.
>

This is broken on Linux, although I'm not sure when this problem was
introduced. I brought it up on the netdev mailing list last year and nobody
seemed to care.

If you try to reassociate a socket from a loopback address to a non-loopback
address then connect will fail. The problem resides in the kernel handling
of loopback device binding. I couldn't figure it out but somebody else on
comp.unix.programmer tracked it down. It actually only happens if the
_first_ bind was to a loopback address.

I originally stumbled upon this issue with my DNS resolver. In the resolver
I would simply reassociate the UDP socket using connect. However, if
somebody had an /etc/resolv.conf like

        nameserver 127.0.0.1
        nameserver 8.8.8.8

then it would return an error on the rare occassion it failed over to
8.8.8.8. You can see the behavior using the example code I've pasted at the
end of the message.

$ ./udp 127.0.0.1 8.8.8.8
connect(127.0.0.1): OK
connect(8.8.8.8): Invalid argument

$ ./udp 8.8.8.8 127.0.0.1
connect(8.8.8.8): OK
connect(127.0.0.1): OK

I've only seen this behavior on Linux. On every other systems I've tried--OS
X, *BSD, Solaris, AIX--you get the expected behavior.

The "fix" is to first unbind the socket by connecting it to AF_UNSPEC.
However, keep in mind that some systems (e.g. OS X) will return an error
when calling connect with an AF_UNSPEC address faily. So you either have to
do this using platform-specific logic, or ignore any failure when
"unbinding" the socket (or both, because I'm not sure which kernel version
or patch introduced this behavior).

Here's sample test code:

#include <stdio.h>
#include <string.h>
#include <errno.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <err.h>

int main(int argc, char *argv[]) {
  int i, fd;
  struct sockaddr_in sin;

  if (-1 == (fd = socket(AF_INET, SOCK_DGRAM, PF_UNSPEC)))
    err(1, "socket");

  for (i = 1; i < argc; i++) {
    sin.sin_family = AF_INET;
    sin.sin_port = htons(53);
    sin.sin_addr.s_addr = inet_addr(argv[i]);

    if (0 != connect(fd, (void *)&sin, sizeof sin))
      printf("connect(%s): %s\n", inet_ntoa(sin.sin_addr), strerror(errno));
    else
      printf("connect(%s): OK\n", inet_ntoa(sin.sin_addr));
  }

  return 0;
}

Reply | Threaded
Open this post in threaded view
|

Re: Speed of Lua's immutable strings vs buffers (split from Re: Pooling of strings is good)

Coroutines
On Mon, Aug 25, 2014 at 6:52 PM, William Ahern
<[hidden email]> wrote:

> On Mon, Aug 25, 2014 at 04:54:53PM -0700, Coroutines wrote:
>> On Mon, Aug 25, 2014 at 4:38 PM, Sean Conner <[hidden email]> wrote:
> <snip>
>> >   read() is fine for TCP and for "connected" UDP sockets [1], but for
>> > unconnected TCP (or even packets for other IP protocols like OSPF) you can't
>> > use read().  That's one reason I rejected using read().
>>
>> Secret:  You can associate the remote peer (sockaddr) with a UDP
>> socket by calling connect() -- after that I believe you can call
>> read() like normal.  The man page says you can only connect() once for
>> TCP sockets, but you can call connect() multiple times for UDP
>> sockets.  This results in the kernel associating the remote peer with
>> that socket in the networking stack -- outwardly it doesn't seem to
>> have any effect.  People think because you only have an fd (an int)
>> that nothing happened -- buwhahaha bsd socket juju.  I know for sure
>> is how you can use recv() instead of recvfrom() on a UDP socket.
>>
>
> This is broken on Linux, although I'm not sure when this problem was
> introduced. I brought it up on the netdev mailing list last year and nobody
> seemed to care.
>
> If you try to reassociate a socket from a loopback address to a non-loopback
> address then connect will fail. The problem resides in the kernel handling
> of loopback device binding. I couldn't figure it out but somebody else on
> comp.unix.programmer tracked it down. It actually only happens if the
> _first_ bind was to a loopback address.
>
> I originally stumbled upon this issue with my DNS resolver. In the resolver
> I would simply reassociate the UDP socket using connect. However, if
> somebody had an /etc/resolv.conf like
>
>         nameserver 127.0.0.1
>         nameserver 8.8.8.8
>
> then it would return an error on the rare occassion it failed over to
> 8.8.8.8. You can see the behavior using the example code I've pasted at the
> end of the message.
>
> $ ./udp 127.0.0.1 8.8.8.8
> connect(127.0.0.1): OK
> connect(8.8.8.8): Invalid argument
>
> $ ./udp 8.8.8.8 127.0.0.1
> connect(8.8.8.8): OK
> connect(127.0.0.1): OK
>
> I've only seen this behavior on Linux. On every other systems I've tried--OS
> X, *BSD, Solaris, AIX--you get the expected behavior.
>
> The "fix" is to first unbind the socket by connecting it to AF_UNSPEC.
> However, keep in mind that some systems (e.g. OS X) will return an error
> when calling connect with an AF_UNSPEC address faily. So you either have to
> do this using platform-specific logic, or ignore any failure when
> "unbinding" the socket (or both, because I'm not sure which kernel version
> or patch introduced this behavior).
>
> Here's sample test code:
>
> #include <stdio.h>
> #include <string.h>
> #include <errno.h>
> #include <sys/types.h>
> #include <sys/socket.h>
> #include <netinet/in.h>
> #include <arpa/inet.h>
> #include <err.h>
>
> int main(int argc, char *argv[]) {
>   int i, fd;
>   struct sockaddr_in sin;
>
>   if (-1 == (fd = socket(AF_INET, SOCK_DGRAM, PF_UNSPEC)))
>     err(1, "socket");
>
>   for (i = 1; i < argc; i++) {
>     sin.sin_family = AF_INET;
>     sin.sin_port = htons(53);
>     sin.sin_addr.s_addr = inet_addr(argv[i]);
>
>     if (0 != connect(fd, (void *)&sin, sizeof sin))
>       printf("connect(%s): %s\n", inet_ntoa(sin.sin_addr), strerror(errno));
>     else
>       printf("connect(%s): OK\n", inet_ntoa(sin.sin_addr));
>   }
>
>   return 0;
> }
>

Where have you been all my life?

Reply | Threaded
Open this post in threaded view
|

Re: Pooling of strings is good

Jorge Visca
In reply to this post by Coda Highland
On 25/08/14 09:15, Coda Highland wrote:
> (The lesson in Apple's SSL bug was "don't copy-paste code," which is
> much more broadly relevant.)

That's actually a criterion i've been munching as a posible default for
all the pesky "style" questions: "does it make copy&paste of code more
robust/safe, or not?".

You can apply that to stuff like to whether to use begin-end or "{}"
even when there's a single line, or where to declare variables (just
before first use or at the beginning of scope), or whether a bunch of
code should be factored out to a function.


Jorge

Reply | Threaded
Open this post in threaded view
|

Re: Speed of Lua's immutable strings vs buffers (split from Re: Pooling of strings is good)

Axel Kittenberger
In reply to this post by Coroutines
co> Twist that dagger a little deeper, eh? :p  Introducing a buffer type
co> and metatable interface to it was not my idea.  I was hoping to learn
co> from upstream how I might trick Lua's typechecking functions into
co> believing userdata were strings and getting it to safely access the
co> ->buf portion of the userdata (rather than the similar data member in
co> the TString struct).

You cannot expect in any way a C function to understand an additional unit type without modifying (or replacing) it.

sean> I applaud Jan Behrens' work on
sean> ipairs because he's taking the time to implement his ideas.

Back the days where the list was heavily loaded with arguments about #-operator and broken sequences ("arrays with holes") a rule was concluded: if you make a suggestion and expect the core team to take their time with it, take your time and provide a patch with your suggestion.

123456