Memory fragmentation?

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Memory fragmentation?

Matt Campbell-2
Hello:

Is memory fragmentation a common problem with Lua? I'm particularly interested in what people have observed on Windows, and any solutions that have been developed. Thanks.

--
Matt Campbell
Lead Programmer
Serotek Corporation
www.freedombox.info
"The Accessibility Anywhere People"


Reply | Threaded
Open this post in threaded view
|

Re: Memory fragmentation?

D Burgess-4
Since Lua 3.2 has not been a problem that I have had on WIndows.
I have played with using HeapAlloc / VirtualAlloc and friends
with little benefit. realloc() (RTL issues aside) seems just fine.

The thing that goes for you with Lua on Windows is that
apart from startup you are unlikely to perform small
(fragmenting) allocations/deallocations.
I presume you are using the MSVCRT.dll.

DB

Matt Campbell wrote:
> Hello:
>
> Is memory fragmentation a common problem with Lua?  I'm particularly
> interested in what people have observed on Windows, and any solutions
> that have been developed.  Thanks.
>
> --
> Matt Campbell
> Lead Programmer
> Serotek Corporation
> www.freedombox.info
> "The Accessibility Anywhere People"
>
>


Reply | Threaded
Open this post in threaded view
|

Re: Memory fragmentation?

Matt Campbell-2
What is it about Lua and/or Windows that makes small, fragmenting allocations and deallocations unlikely after startup? I'm using the MSVC runtime library, but statically linked.

--
Matt Campbell
Lead Programmer
Serotek Corporation
www.freedombox.info
"The Accessibility Anywhere People"


Reply | Threaded
Open this post in threaded view
|

Re: Memory fragmentation?

D Burgess-4
Life has changed a bit with 5.1, if I may quote from the manual

<quote>
The garbage-collector pause controls how long the collector waits
before starting a new cycle. Larger values make the collector less
aggressive. Values smaller than 1 mean the collector will not wait to
start a new cycle. A value of 2 means that the collector waits for the
total memory in use to double before starting a new cycle.

The step multiplier controls the relative speed of the collector
relative to memory allocation. Larger values make the collector more
aggressive but also increases the size of each incremental step.
Values smaller than 1 make the collector too slow and may result in
the collector never finishing a cycle. The default, 2, means that the
collector runs at "twice" the speed of memory allocation.
</quote>

So, we get

> =gcinfo()
23
> a=string.rep(" ",5000)
> =gcinfo()
34
> a=nil
> =gcinfo()
35
> a=string.rep(" ",5000)
> =gcinfo()
35
>

Without tracing my bet is that the last string.rep did not invoke
realloc(). You can tune the default GC parameters in LUACONF.H
and control the GC using collectgarbage(). I guess it varies
depending on the type of app that you have, but the current
GC behaviour is gentle with allocations on my application set.

It can be helpful to add a logfile trace (or OutputDebugString() )
to lua_alloc so you can see exactly when your app causes allocs
and frees.

I have no idea what causes the rash of lua_alloc() calls at lua_open().


DB

 Matt Campbell wrote:
> What is it about Lua and/or Windows that makes small, fragmenting
> allocations and deallocations unlikely after startup?  I'm using the
> MSVC runtime library, but statically linked.


Reply | Threaded
Open this post in threaded view
|

Re: Memory fragmentation?

Wim Couwenberg-2
In reply to this post by Matt Campbell-2
> Is memory fragmentation a common problem with Lua?  I'm particularly
> interested in what people have observed on Windows, and any solutions
> that have been developed.  Thanks.

We experienced heavy fragmentation in a Lua 4 app running on Win98,
built with Borland CBuilder.  That problem went away with Doug Lea's
dlmalloc memory manager.  I didn't specifically investigate
fragmentation behaviour in Lua 5.x apps or more recent Win versions.
(The newer Win allocator should be more up to speed with dlmalloc,
judging by the MSDN documentation anyway.)

--
Wim