scalability

classic Classic list List threaded Threaded
26 messages Options
12
Reply | Threaded
Open this post in threaded view
|

scalability

Phoenix Sol
Looking into libev integration, I found this:

"""
[Kepler-Project] libev + lua
gary ng
Wed Oct 22 03:05:56 GMT+2 2008

There is one floating around and I have also done that myself. Though my not so scientific test seems to suggest that the good old select() isn't that much worse unless we are talking about 1000+ connections and that may not be the target platform for lua(memory etc.)
"""

Can someone please explain this sentiment? What is he talking about? (Gary, are you in here?)

Memory fragmentation?  What is the 'etc.'? Stability problems or something?

My primary interest is in massively concurrent, massively scalable application development in the 'cloud environment' (EC2 specifically, for now).

Is lua a suitable language for this target domain?

Thanks,
Phoenix Sol
Reply | Threaded
Open this post in threaded view
|

Re: scalability

KHMan
Phoenix Sol wrote:

> Looking into libev integration, I found this:
>
> """
> [Kepler-Project] libev + lua *gary ng*
> /Wed Oct 22 03:05:56 GMT+2 2008
>
> /There is one floating around and I have also done that myself. Though
> my not so scientific test seems to suggest that the good old select()
> isn't that much worse unless we are talking about 1000+ connections and
> that may not be the target platform for lua(memory etc.)
> """
>
> Can someone please explain this sentiment? What is he talking about?
> (Gary, are you in here?)

Probably:
http://en.wikipedia.org/wiki/Thundering_herd_problem

or, more accurately:
http://www.kegel.com/c10k.html

> [snip]
> My primary interest is in massively concurrent, massively scalable
> application development in the 'cloud environment' (EC2 specifically,
> for now).
>
> Is lua a suitable language for this target domain?

Depends. Any programming language is suitable, given the correct
circumstances. And what application? Developer needs? Business
needs? Say if you want to run rendering via photon-tracing in a
cloud, obviously for performance you'd go for a language that
executes in native code at some level. Or you want development
speed. And so on, and so forth, yada yada yada... We can have a
1000-post thread and there will be no clear position. I can sell
you A and then switch and sell you B and both positions would be
strong. What matters more is your analysis given your
specifications. Is this for business or for a dissertation?

--
Cheers,
Kein-Hong Man (esq.)
Kuala Lumpur, Malaysia

Reply | Threaded
Open this post in threaded view
|

Re: scalability

Phoenix Sol

Probably:
http://en.wikipedia.org/wiki/Thundering_herd_problem

or, more accurately:
http://www.kegel.com/c10k.html

Sure, I've read those; But I'm afraid I don't know quite what you mean. I am wondering if there are any specific limitations of lua related to handling massive numbers of concurrent socket connections.
 
My primary interest is in massively concurrent, massively scalable application development in the 'cloud environment' (EC2 specifically, for now).

Is lua a suitable language for this target domain?

Depends. Any programming language is suitable, given the correct circumstances. And what application? Developer needs? Business needs? Say if you want to run rendering via photon-tracing in a cloud, obviously for performance you'd go for a language that executes in native code at some level. Or you want development speed. And so on, and so forth, yada yada yada... We can have a 1000-post thread and there will be no clear position. I can sell you A and then switch and sell you B and both positions would be strong. What matters more is your analysis given your specifications. Is this for business or for a dissertation?

Okay, good point; I'm talking about commercial 'web application' and 'web service' development... think 'social networking', 'chat', etc.

Thanks,
Phoenix Sol
Reply | Threaded
Open this post in threaded view
|

Re: scalability

KHMan
Phoenix Sol wrote:

>
>     Probably:
>     http://en.wikipedia.org/wiki/Thundering_herd_problem
>
>     or, more accurately:
>     http://www.kegel.com/c10k.html
>
> Sure, I've read those; But I'm afraid I don't know quite what you mean.
> I am wondering if there are any specific limitations of lua related to
> handling massive numbers of concurrent socket connections.

I'll leave this to those who use the said library to answer, since
the most useful titbits would be some actual performance figures.
I don't believe there are specific limitations that Lua imposes...

> [snip snip]
> Okay, good point; I'm talking about commercial 'web application' and
> 'web service' development... think 'social networking', 'chat', etc.

Interesting... The more fashionable tout things like Erlang and
Scala, yet Wikipedia works fine with mediawiki and the Internet
Archive works fine with I-don't-know-what. Seems like a lot of
salesmanship or evangelizing going on. Good luck... :-)

--
Cheers,
Kein-Hong Man (esq.)
Kuala Lumpur, Malaysia
Reply | Threaded
Open this post in threaded view
|

Re: scalability

Phoenix Sol
Interesting... The more fashionable tout things like Erlang and Scala, yet Wikipedia works fine with mediawiki and the Internet Archive works fine with I-don't-know-what. Seems like a lot of salesmanship or evangelizing going on. Good luck... :-)

I believe it's increasingly important to consider 'the electric meter'. And besides that, I'm pretty obsessed with the idea of making things as fast and efficient as possible with an interpreted language.

I suppose luck can't hurt either, KHMan; thanks ;-)

Phoenix Sol
Reply | Threaded
Open this post in threaded view
|

Re: scalability

Javier Guerra Giraldez
In reply to this post by Phoenix Sol
Phoenix Sol wrote:

>  Looking into libev integration, I found this:
>
> """
> [Kepler-Project] libev + lua *gary ng*
> *Wed Oct 22 03:05:56 GMT+2 2008
>
> *There is one floating around and I have also done that myself. Though my
> not so scientific test seems to suggest that the good old select() isn't
> that much worse unless we are talking about 1000+ connections and that may
> not be the target platform for lua(memory etc.)
> """
>
> Can someone please explain this sentiment? What is he talking about? (Gary,
> are you in here?)

what i read here is that select() isn't so bad for small number of connections, but more than that needs another approach (any of the fashionable event-based APIs)

LuaSocket is based on select(), BTW.

--
Javier
Reply | Threaded
Open this post in threaded view
|

Re: scalability

gary ng
In reply to this post by Phoenix Sol



--- On Sun, 4/26/09, Phoenix Sol <[hidden email]> wrote:
> Okay, good point; I'm talking about commercial 'web
> application' and 'web service' development...
> think 'social networking', 'chat', etc.
>
I believe that depends more on how to 'scale out' rather than scale within a single process(what libevent wants to solve) or machine.

Almost all the containers(like apache, lighttpd etc.) can already handle the role of dispatching(massive connections) quite efficiently. I see no point implementing that in lua or other languages unless your need is very specific.


     
Reply | Threaded
Open this post in threaded view
|

Re: scalability

Tobias Kieslich
On Sun, 26 Apr 2009, gary ng wrote:
> I believe that depends more on how to 'scale out' rather than scale within a single process(what libevent wants to solve) or machine.
>
> Almost all the containers(like apache, lighttpd etc.) can already handle the role of dispatching(massive connections) quite efficiently. I see no point implementing that in lua or other languages unless your need is very specific.
>
I like to second that, when you hit 1,000+ connections, the sockets are
the least of your problem. Serving businesslogic to 1,000 clients blows
up a single server anyway and at that point you will have to look at
horizontal scaling. That means you will have to run multiple application
servers proxied by something that can handle 1,000 + connections light
lighttpd, nginx ... the usual suspects. It's probably not a good idea to
do that in Lua and server business logic from the same machine.

        -Tobias
Reply | Threaded
Open this post in threaded view
|

Re: scalability

Louis Mamakos

On Apr 26, 2009, at 5:05 PM, Tobias Kieslich wrote:

> On Sun, 26 Apr 2009, gary ng wrote:
>> I believe that depends more on how to 'scale out' rather than scale  
>> within a single process(what libevent wants to solve) or machine.
>>
>> Almost all the containers(like apache, lighttpd etc.) can already  
>> handle the role of dispatching(massive connections) quite  
>> efficiently. I see no point implementing that in lua or other  
>> languages unless your need is very specific.
>>
> I like to second that, when you hit 1,000+ connections, the sockets  
> are
> the least of your problem. Serving businesslogic to 1,000 clients  
> blows
> up a single server anyway and at that point you will have to look at
> horizontal scaling. That means you will have to run multiple  
> application
> servers proxied by something that can handle 1,000 + connections light
> lighttpd, nginx ... the usual suspects. It's probably not a good  
> idea to
> do that in Lua and server business logic from the same machine.
>
> -Tobias
>

There are other protocols and applications other than HTTP to web  
servers.  I don't
understand, for instance, how I'd use lighttpd to build an XMPP server  
that needs
to manage a large number of TCP connections.  Likewise, "business  
logic" need not
be a heavyweight operation that hits a database, or even a disk arm.  
An XMPP
server might have lightweight per-connection state in memory for a few  
thousand
sessions.

Using an alternative to select(), like kqueue() is a huge win because  
you don't
have to continually pass an indication of interest back and forth  
across the
kernel system call interface for each invocation.  A small change like  
this can
avoid a lot of complexity in your system in managing multiple  
instances of
an application.

louie

Reply | Threaded
Open this post in threaded view
|

Re: scalability

gary ng
In reply to this post by Phoenix Sol

--- On Mon, 4/27/09, Louis Mamakos <[hidden email]> wrote:
>
> There are other protocols and applications other than HTTP
> to web servers ...
The OP was talking about web service etc.

>
> Using an alternative to select(), like kqueue() is a huge
> win because you don't
> have to continually pass an indication of interest back and
> forth across the
> kernel system call interface for each invocation.  A
> small change like this can
> avoid a lot of complexity in your system in managing
> multiple instances of
> an application.
>
may be, may be not. I found writing binding for libevent to be much more complex than copas using standard select() and I faintly remembered that memory usage per-connection seems to be larger too.

Should I really need to manage such a large number of connections, I would opt for erlang as that seems to be designed from the outset for this kind of application.



Reply | Threaded
Open this post in threaded view
|

Re: scalability

Tobias Kieslich
In reply to this post by Louis Mamakos
On Sun, 26 Apr 2009, Louis Mamakos wrote:

>
> There are other protocols and applications other than HTTP to web  
> servers.  I don't
> understand, for instance, how I'd use lighttpd to build an XMPP server  
> that needs
> to manage a large number of TCP connections.  Likewise, "business logic"
> need not
> be a heavyweight operation that hits a database, or even a disk arm.  An
> XMPP
> server might have lightweight per-connection state in memory for a few  
> thousand
> sessions.
>
> Using an alternative to select(), like kqueue() is a huge win because  
> you don't
> have to continually pass an indication of interest back and forth across
> the
> kernel system call interface for each invocation.  A small change like  
> this can
> avoid a lot of complexity in your system in managing multiple instances
> of
> an application.

Fair enough, my example was web specific. The idea of using load
balancers to scale vertically is still valid for  a lot of scenarios,
especially when business logic outweighs the processiong time for
connection handling by a long shot. My reply was mainly related to the
original post which I think addressed web application development.

After all, for many of these scenarios select does a better job than
threading or forking (even though that always depends on the particular
task) but if select would have been good enough for everything,
/dev/poll, epoll, kqueue etc. would never have been developed.

        -T
Reply | Threaded
Open this post in threaded view
|

Re: scalability

Asko Kauppi

I had a message on SPServer about this, but it seems to have been lost.

In any way, SPServer is a C++ framework for a server, and we've  
succesfully integrated it with Lua (pretty trivial).

Don't know if that is of any help, though.

-asko



Tobias Kieslich kirjoitti 27.4.2009 kello 18:27:

> On Sun, 26 Apr 2009, Louis Mamakos wrote:
>>
>> There are other protocols and applications other than HTTP to web
>> servers.  I don't
>> understand, for instance, how I'd use lighttpd to build an XMPP  
>> server
>> that needs
>> to manage a large number of TCP connections.  Likewise, "business  
>> logic"
>> need not
>> be a heavyweight operation that hits a database, or even a disk  
>> arm.  An
>> XMPP
>> server might have lightweight per-connection state in memory for a  
>> few
>> thousand
>> sessions.
>>
>> Using an alternative to select(), like kqueue() is a huge win because
>> you don't
>> have to continually pass an indication of interest back and forth  
>> across
>> the
>> kernel system call interface for each invocation.  A small change  
>> like
>> this can
>> avoid a lot of complexity in your system in managing multiple  
>> instances
>> of
>> an application.
>
> Fair enough, my example was web specific. The idea of using load
> balancers to scale vertically is still valid for  a lot of scenarios,
> especially when business logic outweighs the processiong time for
> connection handling by a long shot. My reply was mainly related to the
> original post which I think addressed web application development.
>
> After all, for many of these scenarios select does a better job than
> threading or forking (even though that always depends on the  
> particular
> task) but if select would have been good enough for everything,
> /dev/poll, epoll, kqueue etc. would never have been developed.
>
> -T

Reply | Threaded
Open this post in threaded view
|

Re: scalability

Ignacio Burgueño
Asko Kauppi wrote:
>
> I had a message on SPServer about this, but it seems to have been lost.
>
> In any way, SPServer is a C++ framework for a server, and we've
> succesfully integrated it with Lua (pretty trivial).
>
> Don't know if that is of any help, though.
>
>

I'm curious about this. Any chance to take a look at that code?
Regards,
Ignacio


Reply | Threaded
Open this post in threaded view
|

Re: scalability

Phoenix Sol
Google is your friend, Ignacio. I see it's hosted at code.google.com/p/spserver/.

Thanks for sharing, Asko :)

Phoenix Sol


2009/4/28 Ignacio Burgueño <[hidden email]>
Asko Kauppi wrote:

I had a message on SPServer about this, but it seems to have been lost.

In any way, SPServer is a C++ framework for a server, and we've succesfully integrated it with Lua (pretty trivial).

Don't know if that is of any help, though.



I'm curious about this. Any chance to take a look at that code?
Regards,
Ignacio



Reply | Threaded
Open this post in threaded view
|

Re: scalability

Asko Kauppi

Thanks, I did indeed forget the URL.

The benefit of using SPServer is that the server basically becomes a  
single executable, which can have necessary Lua codes "baked in".  
Ours is able to take queries via the URL and present meteorological  
output calculated via those custom scripts.

I will be giving a presentation about the particular use in Holland in  
June, so I believe at least the presentation can be made public then.

-asko


Phoenix Sol kirjoitti 28.4.2009 kello 21:25:

> Google is your friend, Ignacio. I see it's hosted at code.google.com/
> p/spserver/.
>
> Thanks for sharing, Asko :)
>
> Phoenix Sol
>
>
> 2009/4/28 Ignacio Burgueño <[hidden email]>
> Asko Kauppi wrote:
>
> I had a message on SPServer about this, but it seems to have been  
> lost.
>
> In any way, SPServer is a C++ framework for a server, and we've  
> succesfully integrated it with Lua (pretty trivial).
>
> Don't know if that is of any help, though.
>
>
>
> I'm curious about this. Any chance to take a look at that code?
> Regards,
> Ignacio
>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: scalability

Ignacio Burgueño
In reply to this post by Phoenix Sol
Phoenix Sol wrote:
> Google is your friend, Ignacio. I see it's hosted at
> code.google.com/p/spserver/ <http://code.google.com/p/spserver/>.
>
> Thanks for sharing, Asko :)
>
> Phoenix Sol
>

I meant the binding code Asko was talking about, not spserver itself.
However, from Asko's message it seems that he's not in the position of
disclosing that code, right?

Regards,
Ignacio
Reply | Threaded
Open this post in threaded view
|

Re: scalability

Wim Couwenberg
In reply to this post by Asko Kauppi
> I will be giving a presentation about the particular use in Holland in June,

Which event is that?

Bye,
Wim
Reply | Threaded
Open this post in threaded view
|

Re: scalability

Phoenix Sol
In reply to this post by Ignacio Burgueño
Please forgive me, Ignacio; I was in error, and what I said was very insulting.

Phoenix Sol


2009/4/28 Ignacio Burgueño <[hidden email]>
Phoenix Sol wrote:
Google is your friend, Ignacio. I see it's hosted at code.google.com/p/spserver/ <http://code.google.com/p/spserver/>.


Thanks for sharing, Asko :)

Phoenix Sol


I meant the binding code Asko was talking about, not spserver itself. However, from Asko's message it seems that he's not in the position of disclosing that code, right?

Regards,
Ignacio

Reply | Threaded
Open this post in threaded view
|

Re: scalability

Ivan-Assen Ivanov
In reply to this post by Phoenix Sol
> Is lua a suitable language for this target domain?

We've successfully deployed Lua in a production environment serving
hundreds of users from a single PC (a run-of-the-mill Core 2 Duo with
2 GB RAM, running Windows 2008 Server).
During our stress tests, we saw it handle it serve high-single-digits
thousands of clients. It is running smoothly, without crashes or
memory leaks, for months now; the machine has been rebooted once in
that period for external to our code reasons.

The select() code in luasocket becomes a problem before Lua itself
does; we made some small fixes to make it more Windows-friendly (we
provided our changes to Diego Nehab, the author of luasocket, I can
also send the small patch to anyone interested in serving 100+ clients
on Windows). Also, on Windows, you need to compile with a #define
before including the system headers to listen on thousands of ports.

We ran into some kind of soft performance limit with listening with
select() to too many sockets in the same process - on the order of
thousands. Our architecture is meant to "scale out", and we simply ran
more instances of our server code on the same machine, each serving
e.g. 1000 clients - this worked around the problem.

The workload consists of deserializing incoming packets, updating
in-memory data structures (Lua tables, of course), occasional
cryptographical computations (handled via libgmp in C), resending the
data to other clients, and occasionally writing out journal and log
files.

More performance can probably be extracted if luasocket is implemented
via "more navite" OS-specific mechanisms, e.g. IOCP on Windows.

My gut feeling is that nowadays it's quite possible to achieve the
goal of the C10K paper, that is, to serve 10k clients from a single
machine - in Lua. Don't let the small problem of having to serve
thousands of clients drive you into using an inferior language!

(Also, don't believe the "you can't really run servers on Windows"
conventional wisdom :-) )

Best regards,
Ivan-Assen Ivanov
CTO, Haemimont Games
Reply | Threaded
Open this post in threaded view
|

Re: scalability

Phoenix Sol
Your vignette is much appreciated, Ivan; thank-you :)

I would love to hear more testimonials like this, if anyone else can share them.

I am certainly still interested in using libev with lua, but it seems less important than just *using lua*, period. I know that the consensus (of those better educated than myself ;-) is probably right on: that 'business logic' imposes more of a performance limit than the event loop used. And it's becoming clear that lua applications ought to scale ('up' as well as 'out') very well using 'plain old select'...

I'm really excited about this. Thanks a heap, lua community =)

Phoenix Sol


On Tue, Apr 28, 2009 at 3:28 PM, Ivan-Assen Ivanov <[hidden email]> wrote:
> Is lua a suitable language for this target domain?

We've successfully deployed Lua in a production environment serving 

hundreds of users from a single PC (a run-of-the-mill Core 2 Duo with
2 GB RAM, running Windows 2008 Server).
During our stress tests, we saw it handle it serve high-single-digits
thousands of clients. It is running smoothly, without crashes or
memory leaks, for months now; the machine has been rebooted once in
that period for external to our code reasons.

The select() code in luasocket becomes a problem before Lua itself
does; we made some small fixes to make it more Windows-friendly (we
provided our changes to Diego Nehab, the author of luasocket, I can
also send the small patch to anyone interested in serving 100+ clients
on Windows). Also, on Windows, you need to compile with a #define
before including the system headers to listen on thousands of ports.

We ran into some kind of soft performance limit with listening with
select() to too many sockets in the same process - on the order of
thousands. Our architecture is meant to "scale out", and we simply ran
more instances of our server code on the same machine, each serving
e.g. 1000 clients - this worked around the problem.

The workload consists of deserializing incoming packets, updating
in-memory data structures (Lua tables, of course), occasional
cryptographical computations (handled via libgmp in C), resending the
data to other clients, and occasionally writing out journal and log
files.

More performance can probably be extracted if luasocket is implemented
via "more navite" OS-specific mechanisms, e.g. IOCP on Windows.

My gut feeling is that nowadays it's quite possible to achieve the
goal of the C10K paper, that is, to serve 10k clients from a single
machine - in Lua. Don't let the small problem of having to serve
thousands of clients drive you into using an inferior language!

(Also, don't believe the "you can't really run servers on Windows"
conventional wisdom :-) )

Best regards,
Ivan-Assen Ivanov
CTO, Haemimont Games

12