Lua need 1M memory?

classic Classic list List threaded Threaded
17 messages Options
Reply | Threaded
Open this post in threaded view
|

Lua need 1M memory?

jason zhang-4
Hi,
I build a Lua5.1 interpreter in Linux, without any library. From the /proc/pid/status,
It show that the Lua interpreter use about 1M RSS memory. Is this the minimum memory usage of Lua?
Is there anyway to reduce it?
 
Regards,
Jason
 
SRC-GUI1
Tel:86-10-88366185-866
Reply | Threaded
Open this post in threaded view
|

Re: Lua need 1M memory?

chong tsong
hi Jason,

I think it might not be lua interpreter's problem, but up to the
platform and C compiler.

If no lua library is loaded, lua interpreter only use 920KB mem on my machine.
If io is loaded, the number changed to 940KB -- only 20KB more

but the simplest helloword program written in C will take 876KB.

If you are using some embeded system, maybe you can just have a try.
And please kindly post your result if possible :-)


Regards,

Tsong Chong

在 06-11-14,jason zhang<[hidden email]> 写道:


Hi,
I build a Lua5.1 interpreter in Linux, without any library. From the
/proc/pid/status,
It show that the Lua interpreter use about 1M RSS memory. Is this the
minimum memory usage of Lua?
Is there anyway to reduce it?

Regards,
Jason

SRC-GUI1
Tel:86-10-88366185-866


Reply | Threaded
Open this post in threaded view
|

Re: Lua need 1M memory?

Glenn edgar
In reply to this post by jason zhang-4
I have used lua in various embedded environments, including embedded linux,
windriver, and ucos.  The environment is C.  The rom space averages around
100K and I use a standalone heap for lua.  The size of the stand alone heap
I have used is around 100K.  Useful work can be done with this size of
heap.  May have to manually envoke garbage collector in applications.

Still in many embedded enviroments a 100K of RAM is not available due to
power and cost reasons. I am working on constructs where useful stuff could
be done with a 32K or 16K heap.

Bottom line Lua can run in a very small environment.


Glenn Edgar


>hi Jason,
>
>I think it might not be lua interpreter's problem, but up to the
>platform and C compiler.
>
>If no lua library is loaded, lua interpreter only use 920KB mem on my
machine.
>If io is loaded, the number changed to 940KB -- only 20KB more
>
>but the simplest helloword program written in C will take 876KB.
>
>If you are using some embeded system, maybe you can just have a try.
>And please kindly post your result if possible :-)
>
>
>
>Regards,
>
>Tsong Chong
>
>&#22312; 06-11-14&#65292;jason zhang<[hidden email]>
&#20889;&#36947;&#65306;

>
>
> Hi,
> I build a Lua5.1 interpreter in Linux, without any library. From the
> /proc/pid/status,
> It show that the Lua interpreter use about 1M RSS memory. Is this the
> minimum memory usage of Lua?
> Is there anyway to reduce it?
>
> Regards,
> Jason
>
> SRC-GUI1
> Tel:86-10-88366185-866 


--------------------------------------------------------------------
mail2web - Check your email from the web at
http://mail2web.com/ .




Reply | Threaded
Open this post in threaded view
|

Re: Lua need 1M memory?

Mike Pall-67
In reply to this post by jason zhang-4
Hi,

jason zhang wrote:
> I build a Lua5.1 interpreter in Linux, without any library. From the /proc/pid/status,
> It show that the Lua interpreter use about 1M RSS memory. Is this the minimum memory usage of Lua?

No. You are counting libc, libm, libdl, /lib/ld overhead, too.
And most of this is shared with other processes. Even the Lua
part is mostly shared code/data. You can get the heap allocation
with collectgarbage("count") (in Kilobytes).

You want to have a look at:

  # Shows the static code/data size:
  size `which lua`

  # The first few lines are due to Lua, the rest is libc et al.
  # The libc heap is usually overallocated and only a few K are committed.
  lua -e 'os.execute("cat /proc/$PPID/maps")'

Summary:

* read-only (shared) code/data: 40K-200K, depending on compiler
  options (-Os vs. -O3) and whether the parser is included.
  This is for x86 code; somewhat more for most other (RISC) CPUs.

* read-write heap: 18K with standard libs, as low as 1K without.

Bye,
     Mike

Reply | Threaded
Open this post in threaded view
|

Re: Lua need 1M memory?

Leo Razoumov
On 11/14/06, Mike Pall <[hidden email]> wrote:
 [..snip..]
* read-only (shared) code/data: 40K-200K, depending on compiler
  options (-Os vs. -O3) and whether the parser is included.
  This is for x86 code; somewhat more for most other (RISC) CPUs.

* read-write heap: 18K with standard libs, as low as 1K without.

Bye,
     Mike


How did you measure read-write heap?

--Leo--

Reply | Threaded
Open this post in threaded view
|

Re: Lua need 1M memory?

Mike Pall-67
Hi,

Leo Razoumov wrote:
> >* read-write heap: 18K with standard libs, as low as 1K without.
> 
> How did you measure read-write heap?

collectgarbage("count")    [ formerly known as gcinfo() ]

Bye,
     Mike

Reply | Threaded
Open this post in threaded view
|

Re: Lua need 1M memory?

jason zhang-4
In reply to this post by Mike Pall-67
Thank  you.
I think the overhead for shared library is "vmLib". "vmRSS" means the physical memory the process is using.
Am I right?


----- Original Message ----- 
From: "Mike Pall" <[hidden email]>
To: "Lua list" <[hidden email]>
Sent: Tuesday, November 14, 2006 8:08 PM
Subject: Re: Lua need 1M memory?


> Hi,
> 
> jason zhang wrote:
>> I build a Lua5.1 interpreter in Linux, without any library. From the /proc/pid/status,
>> It show that the Lua interpreter use about 1M RSS memory. Is this the minimum memory usage of Lua?
> 
> No. You are counting libc, libm, libdl, /lib/ld overhead, too.
> And most of this is shared with other processes. Even the Lua
> part is mostly shared code/data. You can get the heap allocation
> with collectgarbage("count") (in Kilobytes).
> 
> You want to have a look at:
> 
>  # Shows the static code/data size:
>  size `which lua`
> 
>  # The first few lines are due to Lua, the rest is libc et al.
>  # The libc heap is usually overallocated and only a few K are committed.
>  lua -e 'os.execute("cat /proc/$PPID/maps")'
> 
> Summary:
> 
> * read-only (shared) code/data: 40K-200K, depending on compiler
>  options (-Os vs. -O3) and whether the parser is included.
>  This is for x86 code; somewhat more for most other (RISC) CPUs.
> 
> * read-write heap: 18K with standard libs, as low as 1K without.
> 
> Bye,
>     Mike
>


Reply | Threaded
Open this post in threaded view
|

Re: Lua need 1M memory?

jason zhang-4
In reply to this post by chong tsong
Thanks.
The simplest hello world C program use about 300K RSS memory.
Others memory is for C runtime library. While Lua interpreter 
use 1M RSS memory. 

----- Original Message ----- 
From: "tsong chong" <[hidden email]>
To: "Lua list" <[hidden email]>
Sent: Tuesday, November 14, 2006 6:59 PM
Subject: Re: Lua need 1M memory?


> hi Jason,
> 
> I think it might not be lua interpreter's problem, but up to the
> platform and C compiler.
> 
> If no lua library is loaded, lua interpreter only use 920KB mem on my machine.
> If io is loaded, the number changed to 940KB -- only 20KB more
> 
> but the simplest helloword program written in C will take 876KB.
> 
> If you are using some embeded system, maybe you can just have a try.
> And please kindly post your result if possible :-)
> 
> 
> Regards,
> 
> Tsong Chong
> 
> 在 06-11-14,jason zhang<[hidden email]> 写道:
>>
>>
>> Hi,
>> I build a Lua5.1 interpreter in Linux, without any library. From the
>> /proc/pid/status,
>> It show that the Lua interpreter use about 1M RSS memory. Is this the
>> minimum memory usage of Lua?
>> Is there anyway to reduce it?
>>
>> Regards,
>> Jason
>>
>> SRC-GUI1
>> Tel:86-10-88366185-866
>


Reply | Threaded
Open this post in threaded view
|

Re: Lua need 1M memory?

Mike Pall-67
In reply to this post by jason zhang-4
Hi,

jason zhang wrote:
> I think the overhead for shared library is "vmLib". "vmRSS"
> means the physical memory the process is using.
> Am I right?

Well, RSS counts shared and non-shared pages ... There is no
single, definitive indicator for the "memory use of a process" in
a modern OS with a complex virtual memory subsystem.

A somewhat related read:
  http://www.mozilla.org/projects/footprint/footprint-guide.html

> The simplest hello world C program use about 300K RSS memory.
> Others memory is for C runtime library. While Lua interpreter
> use 1M RSS memory.

Try:
  ldd `which lua`

You may discover that Lua is linked with libreadline which drags
in quite a few other libraries. Don't forget about libm and
libdl, too. This and the fact that Lua initialization calls a
handful libc functions explains the difference. The latter adds
quite a few (shared) pages of libc to the RSS. The main memory
hogs are probably the libc memory allocator and stdio. Don't
blame this on Lua ...

Thankfully Lua allows you to drop most of these dependencies (if
you really need to). But I wouldn't bother on a desktop class
machine, because most of it is already in memory, anyway.

Bye,
     Mike

Reply | Threaded
Open this post in threaded view
|

Re: Lua need 1M memory?

jason zhang-4
Hi Mike,
Thank you. Can lua be compiled with uclibc?

Regards,
Jason

----- Original Message ----- 
From: "Mike Pall" <[hidden email]>
To: "Lua list" <[hidden email]>
Sent: Wednesday, November 15, 2006 9:57 AM
Subject: Re: Lua need 1M memory?


> Hi,
> 
> jason zhang wrote:
>> I think the overhead for shared library is "vmLib". "vmRSS"
>> means the physical memory the process is using.
>> Am I right?
> 
> Well, RSS counts shared and non-shared pages ... There is no
> single, definitive indicator for the "memory use of a process" in
> a modern OS with a complex virtual memory subsystem.
> 
> A somewhat related read:
>  http://www.mozilla.org/projects/footprint/footprint-guide.html
> 
>> The simplest hello world C program use about 300K RSS memory.
>> Others memory is for C runtime library. While Lua interpreter
>> use 1M RSS memory.
> 
> Try:
>  ldd `which lua`
> 
> You may discover that Lua is linked with libreadline which drags
> in quite a few other libraries. Don't forget about libm and
> libdl, too. This and the fact that Lua initialization calls a
> handful libc functions explains the difference. The latter adds
> quite a few (shared) pages of libc to the RSS. The main memory
> hogs are probably the libc memory allocator and stdio. Don't
> blame this on Lua ...
> 
> Thankfully Lua allows you to drop most of these dependencies (if
> you really need to). But I wouldn't bother on a desktop class
> machine, because most of it is already in memory, anyway.
> 
> Bye,
>     Mike
>


Reply | Threaded
Open this post in threaded view
|

Re: Lua need 1M memory?

Bennett Todd
2006-11-15T02:33:50 jason zhang:
> Can lua be compiled with uclibc?

Easily. Nothing to it, even. Here's the Bent Linux[1] spec for lua,
no special dodges needed to accomodate uClibc:

pkg lua-5.1.1

url http://www.lua.org/ftp/lua-5.1.1.tar.gz

build \
    tar xzf lua-5.1.1.tar.gz
    cd lua-5.1.1
    make INSTALL_TOP=/usr INSTALL_MAN=/usr/share/man/man1 linux
    make INSTALL_TOP=$BPM_ROOT/usr INSTALL_MAN=$BPM_ROOT/usr/share/man/man1 install
    strip $BPM_ROOT/usr/bin/*

-Bennett

[1] <URL:http://bent.latency.net/bent/>, statically-linked,
    uclibc-based distro w/o any dynamic loading support, nor any
    i18n, nor any GUI. "Unix as I came to love her in the early
    '80s, when both she and I were a lot slimmer and healthier"

Attachment: pgpqcdhAlFc07.pgp
Description: PGP signature

Reply | Threaded
Open this post in threaded view
|

Re: Lua need 1M memory?

jason zhang-4
Thank you so much.
----- Original Message ----- 
From: "Bennett Todd" <[hidden email]>
To: "Lua list" <[hidden email]>
Sent: Wednesday, November 15, 2006 10:51 PM
Subject: Re: Lua need 1M memory?



Reply | Threaded
Open this post in threaded view
|

how to declare a module like "CGILua.urlcode"

jason zhang-4
Hi,
I am porting the CGILua to SHTTPD (Simple httpd), but I got error when I load 
the CGILua.lua, when it execute the <require "CGILua.urlcode"> line. I know CGILua
has no problem and I run it successfully in IIS.  I must missed something.
So I write some simple script for test:

test.lua :
require "test.cgi"

cgi.lua
module "test.cgi"

when I excute the test.lua, the interpreter report error like below:

lua51: test.lua:6: module 'test.cgi' not found:
        no field package.preload['test.cgi']
        no file '.\test\cgi.lua'
        no file 'D:\prestudy\shttpd-1.33\lua\test\cgi.lua'
        no file 'D:\prestudy\shttpd-1.33\lua\test\cgi\init.lua'
        no file 'D:\prestudy\shttpd-1.33\test\cgi.lua'
        no file 'D:\prestudy\shttpd-1.33\test\cgi\init.lua'
        no file '.\test\cgi.dll'
        no file 'D:\prestudy\shttpd-1.33\test\cgi.dll'
        no file 'D:\prestudy\shttpd-1.33\loadall.dll'
        no file '.\test.dll'
        no file 'D:\prestudy\shttpd-1.33\test.dll'
        no file 'D:\prestudy\shttpd-1.33\loadall.dll'
stack traceback:
        [C]: in function 'require'
        test.lua:6: in main chunk
        [C]: ?

So, I create a sub directory named "test" and move the "cgi.lua" into it.
Then it works. But the CGILua don't work this way, all it's files are in the 
same way.

Could you please tell me what's wrong?

Regards,
Jason


Reply | Threaded
Open this post in threaded view
|

Re: how to declare a module like "CGILua.urlcode"

Tomas-14
So, I create a sub directory named "test" and move the "cgi.lua" into it.
	The error was raised by require, which could not find
the file (you named it "cgi.lua", but required the name "test.cgi").
While moved to the directory "test", `require' was able to find it
(check the reference manual for the behavior of `require").

Then it works. But the CGILua don't work this way, all it's files are in the
same way.

Could you please tell me what's wrong?
	I didn't understand what you mean with "CGILua don't work
this way"...

	Regards,
		Tomas

Reply | Threaded
Open this post in threaded view
|

Re: how to declare a module like "CGILua.urlcode"

jason zhang-4
Thank you. I understand.

----- Original Message ----- 
From: "Tomas" <[hidden email]>
To: "Lua list" <[hidden email]>
Sent: Thursday, November 16, 2006 8:24 PM
Subject: Re: how to declare a module like "CGILua.urlcode"


>> So, I create a sub directory named "test" and move the "cgi.lua" into it.
>  The error was raised by require, which could not find
> the file (you named it "cgi.lua", but required the name "test.cgi").
> While moved to the directory "test", `require' was able to find it
> (check the reference manual for the behavior of `require").
> 
>> Then it works. But the CGILua don't work this way, all it's files are in the
>> same way.
>>
>> Could you please tell me what's wrong?
>  I didn't understand what you mean with "CGILua don't work
> this way"...
> 
>  Regards,
>  Tomas
>


Reply | Threaded
Open this post in threaded view
|

Re: Lua need 1M memory?

Leo Razoumov
In reply to this post by jason zhang-4
On 11/15/06, jason zhang <[hidden email]> wrote:
Thank you so much.
----- Original Message -----
From: "Bennett Todd" <[hidden email]>
To: "Lua list" <[hidden email]>
Sent: Wednesday, November 15, 2006 10:51 PM
Subject: Re: Lua need 1M memory?


I used another method to check how much physical memory (RSS memory in
Linux) will be consumed by an additional Lua state with all standard
libraries loaded.
It turns out that it is just about 20KB (19.6KB in my tests on Linux).

Here is what I did. I took simple Lua stand-alone interpreter from
PiL2 (Listing 24.1, page 219) and augmented it with N dummy Lua
states. N is given as the only command line argument to the program.
The source code of "luai" is attached to this message. A dummy state
is created with luaL_newstate and all the standard libraries are
loaded by means of luaL_openlibs. Then, I ran it with different number
of "dummy" Lua states, let say "luai 0" and "luai 100" and checked RSS
memory usage on Linux in each case. It gave me 19.6KB of resident
physical memory per Lua state.

I think, this is remarkably low overhead. I can create 100 pthreads
with individual Lua states in them at the memory cost of only 2MB.

--Leo--
#include <string.h>
#include <stdlib.h>
#include <stdio.h>
#include <lua.h>
#include <lauxlib.h>
#include <lualib.h>

/* The only command line argument is the number of dummy Lua states N to create.  
 * Total number of Lua states in this program is (N+1)
*/


char g_buff[1024]= {0};

int main(int argc, char* argv[])
{
  char prompt[16]= {0};
  int error, i, N;
  lua_State *l= NULL;
  lua_State *L = luaL_newstate();  /* opens Lua */
  luaL_openlibs(L);                /* opens the standard libraries */

  if( argc != 2) 
    { 
      fputs("Usage:\n    sh$ luai N\nwhere N is a number of dummy Lua states\n",
            stderr);
      exit(1);
    }
  N= atoi( argv[1]);
  sprintf(prompt, "L%d> ", N+1);

  for(i=0; i<N; i++)
    {
      l= luaL_newstate();
      luaL_openlibs(l);
    }

  while(1)
    {
      fputs(prompt, stdout);
      fflush( stdout);
      if( fgets(g_buff, sizeof(g_buff), stdin) == NULL){ break;}
      /* fputs(g_buff, stdout); */
      error = luaL_loadbuffer(L, g_buff, strlen(g_buff), "line") ||
        lua_pcall(L, 0, 0, 0);
      if( error){
        fprintf(stderr, "%s", lua_tostring(L, -1));
        lua_pop(L, 1);  /* pop error message from the stack */
      }
    }
  lua_close(L);
  return 0;
}

/*EoF*/
Reply | Threaded
Open this post in threaded view
|

Re: Lua need 1M memory?

jason zhang-4
Thank you very much!

----- Original Message ----- 
From: "Leo Razoumov" <[hidden email]>
To: "Lua list" <[hidden email]>
Sent: Sunday, November 26, 2006 7:59 PM
Subject: Re: Lua need 1M memory?


> On 11/15/06, jason zhang <[hidden email]> wrote:
>> Thank you so much.
>> ----- Original Message -----
>> From: "Bennett Todd" <[hidden email]>
>> To: "Lua list" <[hidden email]>
>> Sent: Wednesday, November 15, 2006 10:51 PM
>> Subject: Re: Lua need 1M memory?
>>
> 
> I used another method to check how much physical memory (RSS memory in
> Linux) will be consumed by an additional Lua state with all standard
> libraries loaded.
> It turns out that it is just about 20KB (19.6KB in my tests on Linux).
> 
> Here is what I did. I took simple Lua stand-alone interpreter from
> PiL2 (Listing 24.1, page 219) and augmented it with N dummy Lua
> states. N is given as the only command line argument to the program.
> The source code of "luai" is attached to this message. A dummy state
> is created with luaL_newstate and all the standard libraries are
> loaded by means of luaL_openlibs. Then, I ran it with different number
> of "dummy" Lua states, let say "luai 0" and "luai 100" and checked RSS
> memory usage on Linux in each case. It gave me 19.6KB of resident
> physical memory per Lua state.
> 
> I think, this is remarkably low overhead. I can create 100 pthreads
> with individual Lua states in them at the memory cost of only 2MB.
> 
> --Leo--
>