Is it possible that port node.js to Lua?

classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|

Is it possible that port node.js to Lua?

sagasw
I find node.js is very interesting, it seems very useful for high
performance server programming.
But I check the code of node.js, it all writes with C code, and
node.js almost write everything from scratch.

Now I have a question. Lua should be able to use in server too, and
could use libevent/libev as I know,
How about the performance that use Lua libevent compare with node.js?
Is it possible that port node.js to Lua? or just use node.js with Lua?

Many thanks,

------------------------------------------
blog: http://sunxiunan.com/
C++, Lua, living in Dalian
http://twitter.com/sagasw
------------------------------------------

Reply | Threaded
Open this post in threaded view
|

Re: Is it possible that port node.js to Lua?

Matthew Wild
On 17 September 2010 05:03, sagasw <[hidden email]> wrote:
> I find node.js is very interesting, it seems very useful for high
> performance server programming.

I agree, I've used it occasionally. However most of the time I do the
same kind of thing in Lua.

> But I check the code of node.js, it all writes with C code, and
> node.js almost write everything from scratch.
>

Actually it's also quite a bit of plain Javascript code too, and
JITted through the V8 engine. Note well... V8 is not LuaJIT :)

Node gets a lot of hype because it's using Javascript, one of the most
widespread languages there is at the moment - very popular among the
"Web 2.0" and startup crowd especially. It brings this group of people
performant server-side development, which before Node would have been
a concept that sounded pretty far-fetched to them.

> Now I have a question. Lua should be able to use in server too, and
> could use libevent/libev as I know,
> How about the performance that use Lua libevent compare with node.js?
> Is it possible that port node.js to Lua? or just use node.js with Lua?
>

I wouldn't port it directly. I did have the idea that it would be nice
to call into Lua from Node, but I think that's a different issue.

A couple of years back I started the Prosody XMPP server project.
Obviously we're very interested in performance (primarily
latency/throughput and RAM usage). Over time we've developed a set of
libraries at Prosody's core that are applicable to network servers
(and often clients), and we're always optimising them and what not.
It's happened that we now have APIs that do just about anything Node
does (and a few more, like internally joining connections with a
fixed-size intermediate buffer - useful in proxying). I'm also not at
all displeased with the performance, and while you can never have
enough speed, I'm quite satisfied. I've never compared the performance
to Node - I should port the Prosody core to Javascript and test it :)

Whenever I start a new project I now end up pulling in many of these
libraries from Prosody, and I've thought for a while that it would be
good to release them separately. As usual time has stood in the way,
but if there's enough interest and poking then perhaps I'd get around
to it. In the meantime feel free to poke around
http://hg.prosody.im/trunk (particularly net/server*.lua, the core
libs that originally came from the author of the luadch project).

Feel free to contact me off-list if you have any questions.

Matthew

Reply | Threaded
Open this post in threaded view
|

Re: Is it possible that port node.js to Lua?

Scott Vokes
In reply to this post by sagasw
On Fri, Sep 17, 2010 at 12:03 AM, sagasw <[hidden email]> wrote:
> Now I have a question. Lua should be able to use in server too, and
> could use libevent/libev as I know,
> How about the performance that use Lua libevent compare with node.js?
> Is it possible that port node.js to Lua? or just use node.js with Lua?
I was working on a similar project (which I started before I was aware
of node), but put it on the back burner for a while because I decided
I'd rather just use Erlang for high-performance async servers and
leave Lua to what it does best. :) (I'm also not a web developer - it
was a side project that grew out of a distributed filesystem I'm
working on.)

I will probably get back to it sooner or later, though - I have a
mostly-complete LPEG grammar for parsing HTML, a wrapper for libev
(which includes almost all the watcher types, unlike the one on
github), and separate backends for LuaSocket's select (doesn't scale
well, but very portable and still quite fast) and the libev wrapper
(not 100% complete yet). At the very least, I hope to get the libev
wrapper posted soon.

Scott

Reply | Threaded
Open this post in threaded view
|

Re: Is it possible that port node.js to Lua?

Matthew Wild
On 20 September 2010 15:05, Scott Vokes <[hidden email]> wrote:
> On Fri, Sep 17, 2010 at 12:03 AM, sagasw <[hidden email]> wrote:
>> Now I have a question. Lua should be able to use in server too, and
>> could use libevent/libev as I know,
>> How about the performance that use Lua libevent compare with node.js?
>> Is it possible that port node.js to Lua? or just use node.js with Lua?
> I was working on a similar project (which I started before I was aware
> of node), but put it on the back burner for a while because I decided
> I'd rather just use Erlang for high-performance async servers and
> leave Lua to what it does best. :)

As much as I admire Erlang's pioneering into concurrent language design,
our Lua XMPP server beats an Erlang XMPP server in a number of
benchmarks - perhaps peoples' concepts of what "Lua does best" ought
to change :)

> I will probably get back to it sooner or later, though - I have a
> mostly-complete LPEG grammar for parsing HTML, a wrapper for libev
> (which includes almost all the watcher types, unlike the one on
> github), and separate backends for LuaSocket's select (doesn't scale
> well, but very portable and still quite fast) and the libev wrapper
> (not 100% complete yet). At the very least, I hope to get the libev
> wrapper posted soon.
>

More activity in this area is a good thing I think - I'm convinced Lua has
a lot of potential that isn't being used to the full.

That said I'm currently using libevent, though I'm pondering a libev backend
written in C that is more integrated with our server logic. Currently during our
event loop we are very frequently crossing from Lua<->C and I think there's
a lot we could gain from having it more one side or the other. Until LuaJIT's
FFI library is unveiled I think that means more of the low-level network code
in C for now.

Matthew

Reply | Threaded
Open this post in threaded view
|

Re: Is it possible that port node.js to Lua?

Brian Maher-3
In reply to this post by Scott Vokes
Hi Scott,

Sorry I didn't notice this thread earlier.  I'm the author of the
libev binding posted on github.  I would be pleased to accept patches
if you have a fork of the libev wrapper I wrote.  I was planning on
eventually getting around to doing all the watcher types, but I wanted
to be sure to give them sufficient testing and attention.  The libev
binding was also tricky to get right so that I had no memory leaks,
I'm already on the 3rd rewrite.  Note that Brian Downing just recently
added support for idle watchers, so it is an active project :-).

I'm also interested in the node.js project and what it could mean to
the Lua community.  In particular, I wrote a binding to the ry's
http-parser which is used by node.js.  My eventual goal is to create a
webserver that had a main event loop for processing requests, but then
used lua lanes for doing CPU intensive tasks.  There are probably
1,000's of webserver implementations, but the design criteria I'm
going for are:

  * Easy to debug, easy to profile.
  * Ability to sandbox (run potentially malicious code, set CPU limits).
  * Ability to optimize I/O when aggregating content from several
back-end services.
  * Ability to handle 1,000s of simultaneous idle requests (for
polling applications).
  * Ability to load multiple versions of the same library so that
software upgrades can be done on a per session basis.

I'm doing all of this for fun, not profit, which means I'm taking my time :-).

Cheers,
-Brian

On Mon, Sep 20, 2010 at 7:05 AM, Scott Vokes <[hidden email]> wrote:

> On Fri, Sep 17, 2010 at 12:03 AM, sagasw <[hidden email]> wrote:
>> Now I have a question. Lua should be able to use in server too, and
>> could use libevent/libev as I know,
>> How about the performance that use Lua libevent compare with node.js?
>> Is it possible that port node.js to Lua? or just use node.js with Lua?
> I was working on a similar project (which I started before I was aware
> of node), but put it on the back burner for a while because I decided
> I'd rather just use Erlang for high-performance async servers and
> leave Lua to what it does best. :) (I'm also not a web developer - it
> was a side project that grew out of a distributed filesystem I'm
> working on.)
>
> I will probably get back to it sooner or later, though - I have a
> mostly-complete LPEG grammar for parsing HTML, a wrapper for libev
> (which includes almost all the watcher types, unlike the one on
> github), and separate backends for LuaSocket's select (doesn't scale
> well, but very portable and still quite fast) and the libev wrapper
> (not 100% complete yet). At the very least, I hope to get the libev
> wrapper posted soon.
>
> Scott
>
>



--
Brian Maher >> Glory to God <<

Reply | Threaded
Open this post in threaded view
|

Re: Is it possible that port node.js to Lua?

Pierre-Yves Gérardy
On Wed, Oct 6, 2010 at 23:46, Brian Maher <[hidden email]> wrote:
Hi Scott,
 * Easy to debug, easy to profile.
 * Ability to sandbox (run potentially malicious code, set CPU limits).
 * Ability to optimize I/O when aggregating content from several
back-end services.
 * Ability to handle 1,000s of simultaneous idle requests (for
polling applications).
 * Ability to load multiple versions of the same library so that
software upgrades can be done on a per session basis.

Mongrel2 by Zed Shaw seems to fulfill most if not all of these criteria.

It is based on the same HTTP parser as the Ruby HTTP server of the same name, but the rest of the backend is language agnostic, and Lua bindings have already been written.


Regards,
-- Pierre-Yves
Reply | Threaded
Open this post in threaded view
|

Re: Is it possible that port node.js to Lua?

Ignacio Burgueño
In reply to this post by Brian Maher-3
I'm too working on a port of Node.js to Lua at the moment. It will be called LuaNode or something like that, probably.

I'm mostly done with the TCP / HTTP stuff so I hope to release something "usable" soon (however, when I grep my code for "TODO" or "FIXME", the count is still much high :D )

I have almost the same design criteria as Brian (though not going for sandboxes nor multiple versions), but I'm using Boost::Asio instead of libevent or similar libraries, due to lack of experience with those and the need to this whole thing to run in Windows too.

Regards,
Ignacio
Reply | Threaded
Open this post in threaded view
|

Re: Is it possible that port node.js to Lua?

dcharno
In reply to this post by Pierre-Yves Gérardy
On 10/07/2010 06:45 AM, Pierre-Yves Gérardy wrote:

> On Wed, Oct 6, 2010 at 23:46, Brian Maher <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     Hi Scott,
>       * Easy to debug, easy to profile.
>       * Ability to sandbox (run potentially malicious code, set CPU limits).
>       * Ability to optimize I/O when aggregating content from several
>     back-end services.
>       * Ability to handle 1,000s of simultaneous idle requests (for
>     polling applications).
>       * Ability to load multiple versions of the same library so that
>     software upgrades can be done on a per session basis.
>
>
> Mongrel2 by Zed Shaw seems to fulfill most if not all of these criteria.
>
> It is based on the same HTTP parser as the Ruby HTTP server of the same
> name, but the rest of the backend is language agnostic, and Lua bindings
> have already been written.
>
> http://mongrel2.org/home

But, it requires ruby, doesn't it?


Reply | Threaded
Open this post in threaded view
|

Re: Is it possible that port node.js to Lua?

Jonathan Castello-2
On Fri, Oct 8, 2010 at 8:47 PM, dcharno <[hidden email]> wrote:
> But, it requires ruby, doesn't it?

Not at all. You're thinking of Mongrel, its predecessor. Mongrel2 has
language-agnosticism as one of its design goals.

~Jonathan

Reply | Threaded
Open this post in threaded view
|

Re: Is it possible that port node.js to Lua?

Pierre Chapuis

"Jonathan Castello" <[hidden email]> a écrit :
>On Fri, Oct 8, 2010 at 8:47 PM, dcharno <[hidden email]> wrote:
>> But, it requires ruby, doesn't it?

>Not at all. You're thinking of Mongrel, its predecessor. Mongrel2 has
>language-agnosticism as one of its design goals.

Yes, it used to require Python for configuration tools but they have been ported to plain C now.

It is a very interesting piece of software and some of its devs use Lua. The only warning I would give is that their versioning scheme is not what you would expect: the project is very young and I consider 1.0 far from usable in production.

Reply | Threaded
Open this post in threaded view
|

Re: Is it possible that port node.js to Lua?

dcharno
On 10/09/2010 08:09 AM, Pierre Chapuis wrote:

>
> "Jonathan Castello"<[hidden email]>  a écrit :
>> On Fri, Oct 8, 2010 at 8:47 PM, dcharno<[hidden email]>  wrote:
>>> But, it requires ruby, doesn't it?
>
>> Not at all. You're thinking of Mongrel, its predecessor. Mongrel2 has
>> language-agnosticism as one of its design goals.
>
> Yes, it used to require Python for configuration tools but they have been ported to plain C now.
>
> It is a very interesting piece of software and some of its devs use Lua. The only warning I would give is that their versioning scheme is not what you would expect: the project is very young and I consider 1.0 far from usable in production.

Thank you for the advice.  The devs use Fossil which I always find
difficult to navigate and understand.  And it seems quite new.

I'm looking for something that can do comet and this seems to fit the
bill.  It was interesting to see the use of ZeroMQ which I looked at
recently.



Reply | Threaded
Open this post in threaded view
|

Re: Is it possible that port node.js to Lua?

Josh Simmons
As for mongrel2...

Getting the code through fossil is trivial, but be warned that trunk is often unstable and you're probably better off with the latest release. (http://mongrel2.org/home#download)

The Lua handler API is very green, I've not got around to actually doing anything much with it yet, but I'm interested in seeing and helping people actually do that. So feel free to hit me up via email or github or #mongrel2 on freenode if you have any queries or issues and I'll do what I can to get them sorted.

Here are some appropriate links anyway:
http://github.com/jsimmons/mongrel2-lua my mongrel2 handler API for Lua.
http://github.com/norman/mongrel2_wsapi Norman's mongrel2 wsapi adapter using aforementioned API.

On Sat, Oct 9, 2010 at 11:45 PM, dcharno <[hidden email]> wrote:
On 10/09/2010 08:09 AM, Pierre Chapuis wrote:

"Jonathan Castello"<[hidden email]>  a écrit :
On Fri, Oct 8, 2010 at 8:47 PM, dcharno<[hidden email]>  wrote:
But, it requires ruby, doesn't it?

Not at all. You're thinking of Mongrel, its predecessor. Mongrel2 has
language-agnosticism as one of its design goals.

Yes, it used to require Python for configuration tools but they have been ported to plain C now.

It is a very interesting piece of software and some of its devs use Lua. The only warning I would give is that their versioning scheme is not what you would expect: the project is very young and I consider 1.0 far from usable in production.

Thank you for the advice.  The devs use Fossil which I always find difficult to navigate and understand.  And it seems quite new.

I'm looking for something that can do comet and this seems to fit the bill.  It was interesting to see the use of ZeroMQ which I looked at recently.




Reply | Threaded
Open this post in threaded view
|

Re: Is it possible that port node.js to Lua?

Ryan Phillips
In reply to this post by sagasw
sagasw <[hidden email]> said:
> I find node.js is very interesting, it seems very useful for high
> performance server programming.
> But I check the code of node.js, it all writes with C code, and
> node.js almost write everything from scratch.
>
> Now I have a question. Lua should be able to use in server too, and
> could use libevent/libev as I know,
> How about the performance that use Lua libevent compare with node.js?
> Is it possible that port node.js to Lua? or just use node.js with Lua?

Reconnoiter [1,2,3] (BSD licensed) takes an interesting approach with an
asynchronous backend that uses Lua and coroutines to perform similar
functionality that nodejs provides. However, the modules are written
more like a synchronous architecture.

I offer this merely as a data point.

Regards,
Ryan

[1] http://labs.omniti.com/labs/reconnoiter
[2] https://labs.omniti.com/reconnoiter/tags/urskek/src/modules-lua/noit/
[3] https://labs.omniti.com/reconnoiter/tags/urskek/src/modules/lua_noit.c

Reply | Threaded
Open this post in threaded view
|

Re: Is it possible that port node.js to Lua?

Ryan Phillips
In reply to this post by sagasw
sagasw <[hidden email]> said:
> I find node.js is very interesting, it seems very useful for high
> performance server programming.
> But I check the code of node.js, it all writes with C code, and
> node.js almost write everything from scratch.
>
> Now I have a question. Lua should be able to use in server too, and
> could use libevent/libev as I know,
> How about the performance that use Lua libevent compare with node.js?
> Is it possible that port node.js to Lua? or just use node.js with Lua?

I just came across this post [1] about NodeJS and a Lua eventing framework
called SXELua. I do not see any code posted though.

Regards,
Ryan

[1] http://simonhf.wordpress.com/2010/10/13/node-js-versus-lua-hello-world/