_CrtIsValidHeapPointer Problem

classic Classic list List threaded Threaded
17 messages Options
Reply | Threaded
Open this post in threaded view
|

_CrtIsValidHeapPointer Problem

jason zhang-4
Hi,
I am trying to return a array to Lua from the C function as below:

static int Test(lua_State* L)
{
    lua_newtable(L);//create table
    lua_pushnumber(L,1);//push the value
    lua_rawseti(L,-2,1);//set t[1]=v
    lua_pushnumber(L,2);    
    lua_rawseti(L,-2,2);
    lua_pushnumber(L,3);    
    lua_rawseti(L,-2,3);
    return 1;
}

The Lua code is:

data = testext.Test() -- the testext is DLL which include the Test function
for i=1,table.getn(data),1 do
   print(data [i]) 
end

All the three elements are printed correctly but a _CrtIsValidHeapPointer error appears.
I check the FAQ in the www.lua.org, I think there is no stack overflow problem, 
because the stack size is only 2 or 1. 

A very funny thing is that, if I add a element whose index is 0, everything seems ok:

static int Test(lua_State* L)
{     
    lua_newtable(L);//create table
    lua_pushnumber(L,-1);//push any value
    lua_rawseti(L,-2,0);//set t[0]=v
    lua_pushnumber(L,1);//push the value
    lua_rawseti(L,-2,1);//set t[1]=v
    lua_pushnumber(L,2);
    lua_rawseti(L,-2,2);
    lua_pushnumber(L,3);
    lua_rawseti(L,-2,3);
    return 1;
}

The _CrtIsValidHeapPointer error disappear,
but in the Lua code, table.getn(data) return 3, instead of 4. 

With data[0], the element can still be accessed.

Could you please tell me why?
Thank you.

Regards,
Jason


Reply | Threaded
Open this post in threaded view
|

Re: _CrtIsValidHeapPointer Problem

Luiz Henrique de Figueiredo
Your C code seems ok. You mention a DLL. So see
http://lua-users.org/wiki/LuaFaq and look for
_CrtIsValidHeapPointer there.

--lhf

Reply | Threaded
Open this post in threaded view
|

Re: _CrtIsValidHeapPointer Problem

jason zhang-4
Thanks.  I have already read the FAQ about _CrtIsValidHeapPointer.
It describes two possbile reasons:
1. Stack overflow
   No, The stack's size is only 2 or 1 
2. Different Crt library
   I double checked this. The crt of Lua Interpreter, Lua static Library and the extension DLL are 
   the same. In VC6.0, I set all of them to Debug Single-Threaded. Should I use mutli-Threaded Crt?

Thank you.



----- Original Message ----- 
From: "Luiz Henrique de Figueiredo" <[hidden email]>
To: "Lua list" <[hidden email]>
Sent: Thursday, August 17, 2006 10:01 AM
Subject: Re: _CrtIsValidHeapPointer Problem


> Your C code seems ok. You mention a DLL. So see
> http://lua-users.org/wiki/LuaFaq and look for
> _CrtIsValidHeapPointer there.
> 
> --lhf
>


Reply | Threaded
Open this post in threaded view
|

Re: _CrtIsValidHeapPointer Problem

D Burgess-4
You should use multithreaded DLL. (/MD)You cannot have a static
Lua Library and an extension  DLL

David B

jason zhang  wrote:
2. Different Crt library
   I double checked this. The crt of Lua Interpreter, Lua static Library and the extension DLL are
   the same. In VC6.0, I set all of them to Debug Single-Threaded. Should I use mutli-Threaded Crt?

Reply | Threaded
Open this post in threaded view
|

Re: _CrtIsValidHeapPointer Problem

jason zhang-4
I have used the multithreaded DLL for the DLL and multithreaded for the interpreter,
but the error is still there.

ps. The static lua library is linked into the extension DLL, is that incorrect?

what's more, why it works well when I insert a element with index zero?

Thank you.

Regards,
Jason

----- Original Message ----- 
From: "D Burgess" <[hidden email]>
To: "Lua list" <[hidden email]>
Sent: Thursday, August 17, 2006 11:31 AM
Subject: Re: _CrtIsValidHeapPointer Problem


> You should use multithreaded DLL. (/MD)You cannot have a static
> Lua Library and an extension  DLL
> 
> David B
> 
> jason zhang  wrote:
>> 2. Different Crt library
>>    I double checked this. The crt of Lua Interpreter, Lua static Library and the extension DLL are
>>    the same. In VC6.0, I set all of them to Debug Single-Threaded. Should I use mutli-Threaded Crt?
>


Reply | Threaded
Open this post in threaded view
|

Re: _CrtIsValidHeapPointer Problem

Fabian Peña
You must have both binary files compiled of the same way, multithread or
single thread and Debug or Release,
you cannot mix of any way Release and Debug versions.


Reply | Threaded
Open this post in threaded view
|

Re: _CrtIsValidHeapPointer Problem

jason zhang-4
Thank you.
Today I get some progress. I found that the release version of the interpreter works well with the release version of DLL.
But the debug interpreter can't work with the debug DLL. I have checked the crt carefully, they use the same.
Do you know why?


----- Original Message ----- 
From: "Fabian Peña" <[hidden email]>
To: "Lua list" <[hidden email]>
Sent: Thursday, August 17, 2006 10:16 PM
Subject: Re: _CrtIsValidHeapPointer Problem


> You must have both binary files compiled of the same way, multithread or
> single thread and Debug or Release,
> you cannot mix of any way Release and Debug versions.
> 
>


Reply | Threaded
Open this post in threaded view
|

Re: _CrtIsValidHeapPointer Problem

Fabian Peña
Ohh!! certainly the problem still here.I prepared VC++6 proyect files , 
so that some friend can also collaborate.
Download it's with full source code from

http://www.interexport.com.uy:8001/crt_problem.zip

Dirty sugest:
    insert os.exit() at finish :-)      and enjoy.







Reply | Threaded
Open this post in threaded view
|

Re: _CrtIsValidHeapPointer Problem

jason zhang-4
Thank you so much.

I download your project and build them. The problem still occur.
I also test it on my colleague's computer, the same.

I am really mazed....



----- Original Message ----- 
From: "Fabian Peña" <[hidden email]>
To: "Lua list" <[hidden email]>
Sent: Friday, August 18, 2006 11:34 AM
Subject: Re: _CrtIsValidHeapPointer Problem


> Ohh!! certainly the problem still here.I prepared VC++6 proyect files , 
> so that some friend can also collaborate.
> Download it's with full source code from
> 
> http://www.interexport.com.uy:8001/crt_problem.zip
> 
> Dirty sugest:
>    insert os.exit() at finish :-)      and enjoy.
> 
> 
> 
> 
> 
> 
>


Reply | Threaded
Open this post in threaded view
|

Re: _CrtIsValidHeapPointer Problem

Tobias Käs
In reply to this post by jason zhang-4
Probably the checks aren't done in a release build, I guess.

I tried to debug it, but since I don't have a lot of lua-source debugging experience I didn't get far. If I should guess I would say the problem is allocation was called from the DLL and free was called from the main module, this could create problems since the DLL has a heap of its own, and lua tries to free it on the wrong heap. Maybe someone with more experience on lua dlls can comment on this, since it's just a guess for me ;-)

Good luck in solving this bug.


jason zhang wrote:

Thank you.
Today I get some progress. I found that the release version of the interpreter works well with the release version of DLL.
But the debug interpreter can't work with the debug DLL. I have checked the crt carefully, they use the same.
Do you know why?

Reply | Threaded
Open this post in threaded view
|

RE: _CrtIsValidHeapPointer Problem

Jerome Vuarand-2
In reply to this post by jason zhang-4
> I have used the multithreaded DLL for the DLL and multithreaded for the
> interpreter, but the error is still there. 

You should use "Multithreaded DLL" for the DLL, but also "Multithreaded DLL" for the interpreter. The "DLL" word here don't represent the type of project, but the place where the CRT is (either in msvcrt.lib or in msvcrt.dll). All dlls *AND* the exe loading them must use the same version of the msvcrt.

And to explain the difference when you insert at table index 0, it's probably because _CrtIsValidHeapPointer is not a 100% sure detection method and the difference in memory allocations hides the memory corruption problem.

-----Message d'origine-----
De : [hidden email] [[hidden email]] De la part de jason zhang
Envoyé : 16 août 2006 23:44
À : Lua list
Objet : Re: _CrtIsValidHeapPointer Problem

I have used the multithreaded DLL for the DLL and multithreaded for the interpreter, but the error is still there.

ps. The static lua library is linked into the extension DLL, is that incorrect?

what's more, why it works well when I insert a element with index zero?

Thank you.

Regards,
Jason

----- Original Message -----
From: "D Burgess" <[hidden email]>
To: "Lua list" <[hidden email]>
Sent: Thursday, August 17, 2006 11:31 AM
Subject: Re: _CrtIsValidHeapPointer Problem


> You should use multithreaded DLL. (/MD)You cannot have a static
> Lua Library and an extension  DLL
> 
> David B
> 
> jason zhang  wrote:
>> 2. Different Crt library
>>    I double checked this. The crt of Lua Interpreter, Lua static Library and the extension DLL are
>>    the same. In VC6.0, I set all of them to Debug Single-Threaded. Should I use mutli-Threaded Crt?
>


Reply | Threaded
Open this post in threaded view
|

Re: _CrtIsValidHeapPointer Problem

jason zhang-4
I also tried your advice. The result is the same. 
Anyway, Thank you very much;)



----- Original Message ----- 
From: "Jerome Vuarand" <[hidden email]>
To: "Lua list" <[hidden email]>
Sent: Friday, August 18, 2006 10:19 PM
Subject: RE: _CrtIsValidHeapPointer Problem


> I have used the multithreaded DLL for the DLL and multithreaded for the
> interpreter, but the error is still there. 

You should use "Multithreaded DLL" for the DLL, but also "Multithreaded DLL" for the interpreter. The "DLL" word here don't represent the type of project, but the place where the CRT is (either in msvcrt.lib or in msvcrt.dll). All dlls *AND* the exe loading them must use the same version of the msvcrt.

And to explain the difference when you insert at table index 0, it's probably because _CrtIsValidHeapPointer is not a 100% sure detection method and the difference in memory allocations hides the memory corruption problem.

-----Message d'origine-----
De : [hidden email] [[hidden email]] De la part de jason zhang
Envoyé : 16 août 2006 23:44
À : Lua list
Objet : Re: _CrtIsValidHeapPointer Problem

I have used the multithreaded DLL for the DLL and multithreaded for the interpreter, but the error is still there.

ps. The static lua library is linked into the extension DLL, is that incorrect?

what's more, why it works well when I insert a element with index zero?

Thank you.

Regards,
Jason

----- Original Message -----
From: "D Burgess" <[hidden email]>
To: "Lua list" <[hidden email]>
Sent: Thursday, August 17, 2006 11:31 AM
Subject: Re: _CrtIsValidHeapPointer Problem


> You should use multithreaded DLL. (/MD)You cannot have a static
> Lua Library and an extension  DLL
> 
> David B
> 
> jason zhang  wrote:
>> 2. Different Crt library
>>    I double checked this. The crt of Lua Interpreter, Lua static Library and the extension DLL are
>>    the same. In VC6.0, I set all of them to Debug Single-Threaded. Should I use mutli-Threaded Crt?
>


Reply | Threaded
Open this post in threaded view
|

Re: _CrtIsValidHeapPointer Problem

jason zhang-4
In reply to this post by Jerome Vuarand-2
Thanks all of you.
I suspend this problem for about one week and look at it again today.
I compared the DLL case and exe case. The same C code works well
in a exe but cause a heap error in dll. Below is what I get during debuging:

When create a array in a DLL, it will allocate some memory and they are 
stored in the stack of the interpreter. When the script finished, the interpreter
will call lua_close(L) to release the resources. But I found that, it unload the 
dll before clear the stack, so, the memory allocated in the DLL is invalid.

For Lua 5.1

lstate.c, line 210, in the function LUA_API void lua_close (lua_State *L)   
do {  /* repeat until no more errors */
    L->ci = L->base_ci;
    L->base = L->top = L->ci->base;
    L->nCcalls = 0;
  } while (luaD_rawrunprotected(L, callallgcTM, NULL) != 0); // Line 210
Inside the luaD_rawrunprotected function, it will call luaC_callGCTM() function to clear all the user data,
Finally, the routine go to the static int gctm (lua_State *L)  function in loadlib.c, which
unload the dll from the interpreter. So , all the memory allocated in DLL is invalid.

Is my analysis correct?

What I can't understand is, why only the array cause the problem, while normal table works.
And I also don't understand why it works if I insert an element with index zero.

Please help me.

Thank you so much.


----- Original Message ----- 
From: "Jerome Vuarand" <[hidden email]>
To: "Lua list" <[hidden email]>
Sent: Friday, August 18, 2006 10:19 PM
Subject: RE: _CrtIsValidHeapPointer Problem


> I have used the multithreaded DLL for the DLL and multithreaded for the
> interpreter, but the error is still there. 

You should use "Multithreaded DLL" for the DLL, but also "Multithreaded DLL" for the interpreter. The "DLL" word here don't represent the type of project, but the place where the CRT is (either in msvcrt.lib or in msvcrt.dll). All dlls *AND* the exe loading them must use the same version of the msvcrt.

And to explain the difference when you insert at table index 0, it's probably because _CrtIsValidHeapPointer is not a 100% sure detection method and the difference in memory allocations hides the memory corruption problem.

-----Message d'origine-----
De : [hidden email] [[hidden email]] De la part de jason zhang
Envoyé : 16 août 2006 23:44
À : Lua list
Objet : Re: _CrtIsValidHeapPointer Problem

I have used the multithreaded DLL for the DLL and multithreaded for the interpreter, but the error is still there.

ps. The static lua library is linked into the extension DLL, is that incorrect?

what's more, why it works well when I insert a element with index zero?

Thank you.

Regards,
Jason

----- Original Message -----
From: "D Burgess" <[hidden email]>
To: "Lua list" <[hidden email]>
Sent: Thursday, August 17, 2006 11:31 AM
Subject: Re: _CrtIsValidHeapPointer Problem


> You should use multithreaded DLL. (/MD)You cannot have a static
> Lua Library and an extension  DLL
> 
> David B
> 
> jason zhang  wrote:
>> 2. Different Crt library
>>    I double checked this. The crt of Lua Interpreter, Lua static Library and the extension DLL are
>>    the same. In VC6.0, I set all of them to Debug Single-Threaded. Should I use mutli-Threaded Crt?
>


Reply | Threaded
Open this post in threaded view
|

Re[2]: _CrtIsValidHeapPointer Problem

Cloud Wu
Hello jason,

Friday, August 25, 2006, 6:03:32 PM, you wrote:

jz> Thanks all of you.
jz> I suspend this problem for about one week and look at it again today.
jz> I compared the DLL case and exe case. The same C code works well
jz> in a exe but cause a heap error in dll. Below is what I get during debuging:

jz> When create a array in a DLL, it will allocate some memory and they are
jz> stored in the stack of the interpreter. When the script finished, the interpreter
jz> will call lua_close(L) to release the resources. But I found that, it unload the
jz> dll before clear the stack, so, the memory allocated in the DLL is invalid.

jz> For Lua 5.1

jz> lstate.c, line 210, in the function LUA_API void lua_close (lua_State *L)
jz> do {  /* repeat until no more errors */
jz>     L->ci = L->base_ci;
jz>     L->base = L->top = L->ci->base;
jz>     L->nCcalls = 0;
jz>   } while (luaD_rawrunprotected(L, callallgcTM, NULL) != 0); // Line 210
jz> Inside the luaD_rawrunprotected function, it will call
jz> luaC_callGCTM() function to clear all the user data,
jz> Finally, the routine go to the static int gctm (lua_State *L) function in loadlib.c, which
jz> unload the dll from the interpreter. So , all the memory allocated in DLL is invalid.

jz> Is my analysis correct?

jz> What I can't understand is, why only the array cause the
jz> problem, while normal table works.
jz> And I also don't understand why it works if I insert an element with index zero.

jz> Please help me.

jz> Thank you so much.

I guess you linked luacore twice. The lua interpreter static link
luacore lib once, and your dll link it again.

In ltable.c , line 73 :

-------------------------------------------
#define dummynode               (&dummynode_)

static const Node dummynode_ = {
  {{NULL}, LUA_TNIL},  /* value */
  {{{NULL}, LUA_TNIL, NULL}}  /* key */
};
-------------------------------------------

The dummynode is the static stuct .

When you create a table, it may call
------------
  if (size == 0) {  /* no elements to hash part? */
    t->node = cast(Node *, dummynode);  /* use common `dummynode' */
    lsize = 0;
  }
------------

And when the table release, it checks t->node
-----------
void luaH_free (lua_State *L, Table *t) {
  if (t->node != dummynode)
    luaM_freearray(L, t->node, sizenode(t), Node);
  luaM_freearray(L, t->array, t->sizearray, TValue);
  luaM_free(L, t);
}
-----------

If you linked luacore twice, there are 2 copy of dummynode existed. It
may cause your problem.

ps. Lua table will put the index zero to hash part not array part.

-- 
Best regards,
 cloudwu                            [hidden email]
            http://www.codingnow.com

[人生是不公平的]



Reply | Threaded
Open this post in threaded view
|

Re: Re[2]: _CrtIsValidHeapPointer Problem

jason zhang-4
Thank you. 
So what should I do? Dynamic link to the core lua lib?



----- Original Message ----- 
From: "cloudwu" <[hidden email]>
To: "Lua list" <[hidden email]>
Sent: Wednesday, September 13, 2006 5:55 PM
Subject: Re[2]: _CrtIsValidHeapPointer Problem


> Hello jason,
> 
> Friday, August 25, 2006, 6:03:32 PM, you wrote:
> 
> jz> Thanks all of you.
> jz> I suspend this problem for about one week and look at it again today.
> jz> I compared the DLL case and exe case. The same C code works well
> jz> in a exe but cause a heap error in dll. Below is what I get during debuging:
> 
> jz> When create a array in a DLL, it will allocate some memory and they are
> jz> stored in the stack of the interpreter. When the script finished, the interpreter
> jz> will call lua_close(L) to release the resources. But I found that, it unload the
> jz> dll before clear the stack, so, the memory allocated in the DLL is invalid.
> 
> jz> For Lua 5.1
> 
> jz> lstate.c, line 210, in the function LUA_API void lua_close (lua_State *L)
> jz> do {  /* repeat until no more errors */
> jz>     L->ci = L->base_ci;
> jz>     L->base = L->top = L->ci->base;
> jz>     L->nCcalls = 0;
> jz>   } while (luaD_rawrunprotected(L, callallgcTM, NULL) != 0); // Line 210
> jz> Inside the luaD_rawrunprotected function, it will call
> jz> luaC_callGCTM() function to clear all the user data,
> jz> Finally, the routine go to the static int gctm (lua_State *L) function in loadlib.c, which
> jz> unload the dll from the interpreter. So , all the memory allocated in DLL is invalid.
> 
> jz> Is my analysis correct?
> 
> jz> What I can't understand is, why only the array cause the
> jz> problem, while normal table works.
> jz> And I also don't understand why it works if I insert an element with index zero.
> 
> jz> Please help me.
> 
> jz> Thank you so much.
> 
> I guess you linked luacore twice. The lua interpreter static link
> luacore lib once, and your dll link it again.
> 
> In ltable.c , line 73 :
> 
> -------------------------------------------
> #define dummynode               (&dummynode_)
> 
> static const Node dummynode_ = {
>  {{NULL}, LUA_TNIL},  /* value */
>  {{{NULL}, LUA_TNIL, NULL}}  /* key */
> };
> -------------------------------------------
> 
> The dummynode is the static stuct .
> 
> When you create a table, it may call
> ------------
>  if (size == 0) {  /* no elements to hash part? */
>    t->node = cast(Node *, dummynode);  /* use common `dummynode' */
>    lsize = 0;
>  }
> ------------
> 
> And when the table release, it checks t->node
> -----------
> void luaH_free (lua_State *L, Table *t) {
>  if (t->node != dummynode)
>    luaM_freearray(L, t->node, sizenode(t), Node);
>  luaM_freearray(L, t->array, t->sizearray, TValue);
>  luaM_free(L, t);
> }
> -----------
> 
> If you linked luacore twice, there are 2 copy of dummynode existed. It
> may cause your problem.
> 
> ps. Lua table will put the index zero to hash part not array part.
> 
> -- 
> Best regards,
> cloudwu                            [hidden email]
>            http://www.codingnow.com
> 
> [人生是不公平的]
> 
> 
>


Reply | Threaded
Open this post in threaded view
|

Re: Re[2]: _CrtIsValidHeapPointer Problem

Nick Gammon

On 14/09/2006, at 10:42 AM, jason zhang wrote:

So what should I do? Dynamic link to the core lua lib?

When I first added Lua to my application I statically linked it, to save distributing an extra DLL.

However then when I tried to do a loadlib (eg. to the Lua Socket library) I got this exact problem - two different sorts of Lua memory.

My advice is, if one part of the system is going to use DLLs, make sure they all do.

- Nick

Reply | Threaded
Open this post in threaded view
|

Re: Re[2]: _CrtIsValidHeapPointer Problem

jason zhang-4
Thank you so much!

----- Original Message ----- 
From: "Nick Gammon" <[hidden email]>
To: "Lua list" <[hidden email]>
Sent: Thursday, September 14, 2006 8:50 AM
Subject: Re: Re[2]: _CrtIsValidHeapPointer Problem


> 
> On 14/09/2006, at 10:42 AM, jason zhang wrote:
> 
>> So what should I do? Dynamic link to the core lua lib?
> 
> When I first added Lua to my application I statically linked it, to  
> save distributing an extra DLL.
> 
> However then when I tried to do a loadlib (eg. to the Lua Socket  
> library) I got this exact problem - two different sorts of Lua memory.
> 
> My advice is, if one part of the system is going to use DLLs, make  
> sure they all do.
> 
> - Nick
>