multiple lua process in a memory tight situation, would shared lib help ?

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

multiple lua process in a memory tight situation, would shared lib help ?

gary ng
Hi,

I am trying to squeeze lua into my WRT54G which has
only 16M RAM. What I want to do is to run multiple lua
http up/download process using lua socket.

I am wondering if lua+liblua() would result in less
overall memory use.

Oh, it is running linux 2.4.20 with uclibc.


 
____________________________________________________________________________________
Any questions? Get answers on any topic at www.Answers.yahoo.com.  Try it now.

Reply | Threaded
Open this post in threaded view
|

Re: multiple lua process in a memory tight situation, would shared lib help ?

Glenn edgar
The fastest web services in pyhton was twisted.  I believe twisted did not allocate multiple processess or threads.  Suggest a lua frame work like the python twisted environment be set up.  Twisted is a network application framework written in Python, uses a fast, asynchronous, event-driven model instead of multiple processes or threads. See the link http://www.onlamp.com/pub/a/python/2003/4/24/pythonnews.html

On 2/15/07, gary ng <[hidden email]> wrote:
Hi,

I am trying to squeeze lua into my WRT54G which has
only 16M RAM. What I want to do is to run multiple lua
http up/download process using lua socket.

I am wondering if lua+liblua() would result in less
overall memory use.

Oh, it is running linux 2.4.20 with uclibc.



____________________________________________________________________________________
Any questions? Get answers on any topic at www.Answers.yahoo.com.  Try it now.

Reply | Threaded
Open this post in threaded view
|

Re: multiple lua process in a memory tight situation, would shared lib help ?

gary ng
--- Glenn Edgar Admin <[hidden email]>
wrote:

>Suggest a lua frame work like the python twisted
>environment be set up.  

Thanks for the suggestion.

Initially, I want to make it a coroutine based
system(may be copas). But I run into a problem which I
don't know how to solve(thus the multi-process
approach). I am writing a relay service which needs
lazy write on the receiving end(return to the client
as "done" asap) then on the real send/retreive end
doing the real work which can take quite some time.
And the HTTP server in question don't support stream
mode. I don't want any of these worker thread stalled
the whole thing till it is done.


 
____________________________________________________________________________________
Cheap talk?
Check out Yahoo! Messenger's low PC-to-Phone call rates.
http://voice.yahoo.com

Reply | Threaded
Open this post in threaded view
|

Re: multiple lua process in a memory tight situation, would shared lib help ?

Glenn edgar
The async mode of twisted solved this problem.  Many data base servers use this approach too.  Multiple process are very expensive, CGI is based on this principle and most web servers went away from this technique many years ago.  Suggest linux threads if you do not want to deal with async processes.  That way process creation overhead is not a problem.

On 2/15/07, gary ng <[hidden email]> wrote:

--- Glenn Edgar Admin <[hidden email]>
wrote:

>Suggest a lua frame work like the python twisted
>environment be set up.

Thanks for the suggestion.

Initially, I want to make it a coroutine based
system(may be copas). But I run into a problem which I
don't know how to solve(thus the multi-process
approach). I am writing a relay service which needs
lazy write on the receiving end(return to the client
as "done" asap) then on the real send/retreive end
doing the real work which can take quite some time.
And the HTTP server in question don't support stream
mode. I don't want any of these worker thread stalled
the whole thing till it is done.



____________________________________________________________________________________
Cheap talk?
Check out Yahoo! Messenger's low PC-to-Phone call rates.
http://voice.yahoo.com

Reply | Threaded
Open this post in threaded view
|

Re: multiple lua process in a memory tight situation, would shared lib help ?

Javier Guerra Giraldez
In reply to this post by gary ng
On Thursday 15 February 2007, gary ng wrote:
> Initially, I want to make it a coroutine based
> system(may be copas). But I run into a problem which I
> don't know how to solve(thus the multi-process
> approach). I am writing a relay service which needs
> lazy write on the receiving end(return to the client
> as "done" asap) then on the real send/retreive end
> doing the real work which can take quite some time.
> And the HTTP server in question don't support stream
> mode. I don't want any of these worker thread stalled
> the whole thing till it is done.

if you do your delayed write in chunks you could still use copas and not stall 
the whole thing.

or, if you'd like to use threads, try the HelperThreadsToolkit 
(http://luaforge.net/projects/helper-threads/).  it has a copas-like 
scheduler that runs Lua code cooperatively (with coroutines), but manages a 
variable number of helper threads to execute HTT-enabled C code.  i have 
sample (but usable) modules for TCP and file I/O.

i was able to use them to write a Xavante-like web server; if you recall, one 
of the big differences between Apache 1.3 and 2.0 was its cooperative 
scheduling with "worker threads" that do most of the work.  HTT does a 
similar thing

-- 
Javier

Attachment: pgpeFVyStY3UT.pgp
Description: PGP signature

Reply | Threaded
Open this post in threaded view
|

Re: multiple lua process in a memory tight situation, would shared lib help ?

gary ng
--- Javier Guerra <[hidden email]> wrote:
> or, if you'd like to use threads, try the
> HelperThreadsToolkit 
>(http://luaforge.net/projects/helper-threads/).  it
> has a copas-like scheduler that runs Lua code 
>cooperatively (with coroutines), but manages a 
> variable number of helper threads to execute
> HTT-enabled C code.  i have sample (but usable) 
>modules for TCP and file I/O.
Thanks. Unfortunately, I am already running out of
space to add anymore so's to the flash. I can only use
what is available, so either coroutine or multi-process.


 
____________________________________________________________________________________
Looking for earth-friendly autos? 
Browse Top Cars by "Green Rating" at Yahoo! Autos' Green Center.
http://autos.yahoo.com/green_center/

Reply | Threaded
Open this post in threaded view
|

Re: multiple lua process in a memory tight situation, would shared lib help ?

D Burgess-4
LuaSocket can be made work with coroutines and LuaSocket.
Comsider mailing Adrian Sietsma (on this list). I know he has a
performance HTTP proxy using LuaSocket and coroutines.
I am sure he could help you.

db

On 2/16/07, gary ng <[hidden email]> wrote:

--- Javier Guerra <[hidden email]> wrote:
> or, if you'd like to use threads, try the
> HelperThreadsToolkit
>(http://luaforge.net/projects/helper-threads/).  it
> has a copas-like scheduler that runs Lua code
>cooperatively (with coroutines), but manages a
> variable number of helper threads to execute
> HTT-enabled C code.  i have sample (but usable)
>modules for TCP and file I/O.
Thanks. Unfortunately, I am already running out of
space to add anymore so's to the flash. I can only use
what is available, so either coroutine or multi-process.



____________________________________________________________________________________
Looking for earth-friendly autos?
Browse Top Cars by "Green Rating" at Yahoo! Autos' Green Center.
http://autos.yahoo.com/green_center/