LuaJIT and coroutines

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

LuaJIT and coroutines

Adrian Sietsma
Does LuaJIT have a (designed) limit on the number of coroutines it supports ?

I get an out of memory error (from Lua) at about the 5,000 coro mark
(created, none running).
collectgarbage reports about 3mb of memory used, before it fails.

The same program works fine at 10,000 coro's under Lua 5.1

OS - Windows XP

I don't have a simple test case yet.

Adrian
Reply | Threaded
Open this post in threaded view
|

Re: LuaJIT and coroutines

Mike Pall-4-2
Hi,

Adrian Sietsma wrote:
> Does LuaJIT have a (designed) limit on the number of coroutines it supports ?
> OS - Windows XP

No, but read: jitdoc/coco_portability.html

  Note for Windows: Please read the explanation for the default
  Thread Stack Size
    http://msdn.microsoft.com/library/en-us/dllproc/base/thread_stack_size.asp
  in case you want to create large numbers of Fiber-based coroutines.

> I get an out of memory error (from Lua) at about the 5,000 coro mark
> (created, none running).

1MB * 5000 = 5GB (it probably fails earlier)

You need to use a STACKSIZE statement in the .def file.

> collectgarbage reports about 3mb of memory used, before it fails.

The memory for the Fiber stacks is managed by Windows and does
not count towards the Lua memory size.

You should also use the attached interim patch to deallocate
stacks early. This will help reduce memory contention, too.
Yes, a variation of it will be part of the next release.

Bye,
     Mike

fiberdel.patch (432 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: LuaJIT and coroutines

Adrian Sietsma
Mike Pall wrote:

>
> You need to use a STACKSIZE statement in the .def file.
>
> [...]
>
>
> The memory for the Fiber stacks is managed by Windows and does
> not count towards the Lua memory size.
>
> You should also use the attached interim patch to deallocate
> stacks early. This will help reduce memory contention, too.
> Yes, a variation of it will be part of the next release.

Thanks.
Unfortunately, in this case I don't really need CoCo functionality, but I'll
never hit that limit in reality. (I'm stress-testing an i/o library)

ps I also managed to get a most entertaining error when I ran LuaJit (and
obviously my whole pc) out of memory :

"The system dll user.dll was relocated in memory.....
The DLL C:\WINDOWS\system32\ADVAPI32.dll (was to blame....)
The vendor supplying the dll should be contaced..."

Can you guess who that dll is signed/copyrighted by ?

Adrian.
Reply | Threaded
Open this post in threaded view
|

Re: LuaJIT and coroutines

Adrian Sietsma
> Mike Pall wrote:
>
>>
>> You need to use a STACKSIZE statement in the .def file.
>>
>

I see coco setting the stack size in CreateFiber; I assume I could change
COCO_DEFAULT_CSTACKSIZE, if I just wanted to change CoCo ?
And also as a runtime param to coroutine.create() I see.

Adrian (having RTFM).

Reply | Threaded
Open this post in threaded view
|

Re: LuaJIT and coroutines

Mike Pall-4-2
Hi,

Adrian Sietsma wrote:
> >>You need to use a STACKSIZE statement in the .def file.
>
> I see coco setting the stack size in CreateFiber; I assume I could change
> COCO_DEFAULT_CSTACKSIZE, if I just wanted to change CoCo ?
> And also as a runtime param to coroutine.create() I see.

Yes, but this only changes the committed amount on Windows. These
parameters do not allow you to reduce the reserved address space
below STACKSIZE.

> Adrian (having RTFM).

But not the MS docs about 'committed' and 'reserved' memory.
You are usually out of the latter. So you really need to change
the STACKSIZE definition.

Short explanation: Windows reserves a contiguous block of address
space for each Fiber (or Thread) stack. A part of it is committed
and will be allocated from physical RAM or the page file. The
committed amount will auto increase up to the reserved amount
when the non-committed stack pages are accessed.

If the reserved size is large enough (default is 1 MB) and you
create enough Fibers, then you will run out of address space long
before you run out of physical RAM or page file space. Simply
because there is a 4 GB address space limit on a 32 bit CPU.


Note that this is a peculiarity of Windows memory management. On
Linux and most other systems the stack size parameters to
coroutine.create, coroutine.wrap and coroutine.cstacksize
actually determine the number of allocated PTEs ('reserved'
memory). The used PTEs ('committed' memory) are automatically
determined by the kernel and default to one page (4K on x86).

The default stack sizes are much lower for non-Windows systems.
The latest version of Coco/LuaJIT uses 60K (minimum: 36K).

Bye,
     Mike
Reply | Threaded
Open this post in threaded view
|

Re: LuaJIT and coroutines

Adrian Sietsma
Mike Pall wrote:
>
> [...]
>
> The default stack sizes are much lower for non-Windows systems.
> The latest version of Coco/LuaJIT uses 60K (minimum: 36K).
>

I applied your patch, which (obviously) didn't solve the max creation
problem, but both STACKSIZE 64000 in .def and /STACK:64000 in ldflags fix
the problem.
(at least up to 20,000 threads)

It appears that the stack setting must be applied to the .exe, not the .dll
- is that correct ?

Adrian
Reply | Threaded
Open this post in threaded view
|

Re: LuaJIT and coroutines

Mike Pall-4-2
Hi,

Adrian Sietsma wrote:
> I applied your patch, which (obviously) didn't solve the max creation
> problem, but both STACKSIZE 64000 in .def and /STACK:64000 in ldflags fix
> the problem.
> (at least up to 20,000 threads)

Never use an exact multiple of 64K for lots of similar memory
allocations. Use 60K (61440) or 68K (69632) to avoid D-cache
aliasing conflicts.

Also be careful when calling arbitrary C functions. Some
carelessly written DLLs need a lot more stack space (COM services
in particular).

> It appears that the stack setting must be applied to the .exe, not the .dll
> - is that correct ?

Yes.

Bye,
     Mike