How to reload C functions added with "require"?

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

How to reload C functions added with "require"?

Rob Shaw-2
Hi,

We've successfully been using "require" in lua 5.1 under mac osx 10.4.5
to dynamically load external C functions with bindings constructed  
with tolua++.
But what I'd like to do is edit and recompile the C functions, and then
reload them, without exiting and re-entering the lua interpreter.

If you just say "require" again, nothing happens, as there is an  
entry in
package.loaded[name].  I've tried setting package.loaded[name] = nil,
and this will load any new functions added, but previously registered  
functions
stay the same, even if I set func = nil before the second "require".
The old function is living somewhere, and gets reattached to the name  
"func".

How can I load in new versions of the old functions?  It might be  
nice to have
a function "reload" to go along with "require".

Thanks, sorry if I'm missing something obvious.

rob

Reply | Threaded
Open this post in threaded view
|

Re: How to reload C functions added with "require"?

Fabian Peña
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I think that you must add "C" code for unload dynamic library module (
very necesary under windows to unlock the file ) and invoque a require
     or directly call to loadlib function again.




Rob Shaw wrote:

> Hi,
>
> We've successfully been using "require" in lua 5.1 under mac osx 10.4.5
> to dynamically load external C functions with bindings constructed with
> tolua++.
> But what I'd like to do is edit and recompile the C functions, and then
> reload them, without exiting and re-entering the lua interpreter.
>
> If you just say "require" again, nothing happens, as there is an entry in
> package.loaded[name].  I've tried setting package.loaded[name] = nil,
> and this will load any new functions added, but previously registered
> functions
> stay the same, even if I set func = nil before the second "require".
> The old function is living somewhere, and gets reattached to the name
> "func".
>
> How can I load in new versions of the old functions?  It might be nice
> to have
> a function "reload" to go along with "require".
>
> Thanks, sorry if I'm missing something obvious.
>
> rob
>
>

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2.2 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFETNomWmd5q6r1OEoRAqEnAKCRfRpuc8OztpeU+TPP2GKeRvShdgCeII42
NMcwAgzgPUetlOJstGPKafM=
=KaIo
-----END PGP SIGNATURE-----
Reply | Threaded
Open this post in threaded view
|

Re: How to reload C functions added with "require"?

Rob Shaw-2
In reply to this post by Rob Shaw-2
Thanks!  I think you are right, it must be done from the C side.
I looked at loadlib.c, made a small change, and was able to
successfully reload a recompiled dynamic library, but I
still don't completely understand what's going on.

It looks like nearly everything is already there in loadlib.c.
There is a call ll_unloadlib() which for mac osx is a wrapper
over NSUnLinkModule() which should do the job, at least for
PPC architecture.  So I added another entry to pk_funcs[],

  {"unload", ll_unload},

and defined ll_unload as follows:

static int ll_unload (lua_State *L) {
   const char *path = luaL_checkstring(L, 1);
   void **reg = ll_register(L, path);
//  if (*reg) ll_unloadlib(*reg);
   *reg = NULL;  /* mark library as closed */
   return 0;
}

It worked!  In lua:

 > package.loadlib('hello.so','_luaopen_hello' )()
 > =hello()
Hello world!

(now edit and recompile hello.so)

 > package.unload('hello.so')

(if you call hello() now, you will get a satisfying crash)

 > package.loadlib('hello.so','_luaopen_hello')()
 > =hello()
Hello walda!

The interpreter can now live forever, and continuously change its C  
code.
By the way, a friend tells me you can't do this in python, at least not
right now.

But there are mysteries, for example if you comment out the ll_unloadlib
line as above, it still works!  The dynamic loader must be pretty smart,
you just have to let it do its job.  I wouldn't be surprised though if
different loaders behave differently, maybe that's why this
Very Important Feature hasn't been implemented.  Even in macland, the  
docs
discourage the use of the NSModule package, and want you to use
dlopen() / dlclose().  Also, if I understand the docs, the unlink /  
relink
feature only works in PPC, though maybe it isn't even needed.

Also, I don't understand lua internals well enough to get "require"
to work this way.  But I just wrote a little script:

function req(name)
package.unload(name)
package.loadlib(name..'.so','_luaopen_'..name)()
end

so now you can say > req 'hello' and reload the new C module.

Thanks again,

rob

Fabian Pe?a wrote:

> I think that you must add "C" code for unload dynamic library module (
> very necesary under windows to unlock the file ) and invoque a require
>      or directly call to loadlib function again.
>
> Rob Shaw wrote:
>> Hi,
>>
>> We've successfully been using "require" in lua 5.1 under mac osx  
>> 10.4.5
>> to dynamically load external C functions with bindings constructed  
>> with
>> tolua++.
>> But what I'd like to do is edit and recompile the C functions, and  
>> then
>> reload them, without exiting and re-entering the lua interpreter.
>>
>> If you just say "require" again, nothing happens, as there is an  
>> entry in
>> package.loaded[name].  I've tried setting package.loaded[name] = nil,
>> and this will load any new functions added, but previously registered
>> functions
>> stay the same, even if I set func = nil before the second "require".
>> The old function is living somewhere, and gets reattached to the name
>> "func".
>>
>> How can I load in new versions of the old functions?  It might be  
>> nice
>> to have
>> a function "reload" to go along with "require".
>>
>> Thanks, sorry if I'm missing something obvious.
>>
>> rob
>>
>>
>
Reply | Threaded
Open this post in threaded view
|

specified stack level expression evaluation

Dmitriy Iassenev
hello,

I am working with LUA debugging functions for the visual LUA debugger
and there is a question about watches implementation.

Is it possible to evaluate an expression using local variables and
upvalues from the specified stack level?

--
Best regards,
 Dmitriy                            mailto:[hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: specified stack level expression evaluation

Fabio Mascarenhas-2
Hi,

You can put the local variables in a table, and set it as the
envinronment of the function that loadstring returns. If you also want
the current global environment, you can set the global environment as
the __index metamethod of the table where you put the locals.

There is an example of this technique in the RemDebug source code, check
http://luaforge.net/projects/remdebug.

--
Fabio Mascarenhas

On Wed, Apr 26, 2006 at 01:09:46PM +0300, Dmitriy Iassenev wrote:

> hello,
>
> I am working with LUA debugging functions for the visual LUA debugger
> and there is a question about watches implementation.
>
> Is it possible to evaluate an expression using local variables and
> upvalues from the specified stack level?
>
> --
> Best regards,
>  Dmitriy                            mailto:[hidden email]
>

signature.asc (198 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re[2]: specified stack level expression evaluation

Dmitriy Iassenev
Hello Fabio,

Wednesday, April 26, 2006, 6:14:25 PM, you wrote:

FM> You can put the local variables in a table, and set it as the
FM> envinronment of the function that loadstring returns. If you also want
FM> the current global environment, you can set the global environment as
FM> the __index metamethod of the table where you put the locals.

thank you very much!

--
Best regards,
 Dmitriy                            mailto:[hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: How to reload C functions added with "require"?

Rob Shaw-2
In reply to this post by Rob Shaw-2
Sorry, I should correct a mistyping in the previous post, in case anyone
wants to try this.  Also, " require(name) " constructs part of the  
key to the
registry in the form " ./name.so " for a library in the local directory.
So the reload function  should read:

function req(name)
package.unload('./'..name..'.so')
package.loadlib('./'..name..'.so','_luaopen_'..name)()
end

This will now correctly reload a local .so file loaded with "require".

Also, I've been told there's something called ipython which can do
dynamic reloading of C modules.  Actually, ipython looks pretty good,
it has the ability to construct "commands", a logging facility, etc.
But lua is better, especially for someone who enjoys reinventing wheels.

rob

Reply | Threaded
Open this post in threaded view
|

"require" stack bug?

Rob Shaw-2
In reply to this post by Rob Shaw-2
Hi,

I have some code to access a C array from lua, and got it to work
following the example code around pg. 246 of the lua book (1st edition).
If I register the metamethods from the lua side, everything works,
but when I tried to register them in the luaopen_array() function, I got
an 'attempt to index a string value' error.  This occurs if  I say

 > require 'array'

  -- but not if I say

 > package.loadlib('./array.so','_luaopen_array')()

Without going into full detail, luaopen_array() bombs in the
4th line below:

   lua_pushstring(L, "__index");
   lua_pushstring(L, "get");
   lua_gettable(L, 2);           // get array.get
   lua_settable(L, 1);           // array_mt.__index = array.get

The reason for the crash is that the stack is empty when called by
loadlib(), but has one entry ("array") when called by require().
This is easily fixed by indexing from the top of the stack:

   lua_pushstring(L, "__index");
   lua_pushstring(L, "get");
   lua_gettable(L, -3);          // get array.get
   lua_settable(L, -4);          // array_mt.__index = array.get

but I think this counts as a bug.  Do I get a reward?  I only got
my lua book a few weeks ago, only to find it was already obsolete.

Can I get a discount on the 2nd edition?  ;)

rob

Reply | Threaded
Open this post in threaded view
|

Re: "require" stack bug?

Mike Pall-5-2
Hi,

Rob Shaw wrote:
> The reason for the crash is that the stack is empty when called by
> loadlib(), but has one entry ("array") when called by require().
> [...]
> but I think this counts as a bug.  Do I get a reward?

Nope, no cigar. Check the online manual:
  http://www.lua.org/manual/5.1/manual.html#pdf-require

  ... Once a loader is found, require calls the loader with a
  single argument, modname. ...

Bye,
     Mike
Reply | Threaded
Open this post in threaded view
|

Re: How to reload C functions added with "require"?

Chris Marrin
In reply to this post by Rob Shaw-2
Rob Shaw wrote:

> Sorry, I should correct a mistyping in the previous post, in case anyone
> wants to try this.  Also, " require(name) " constructs part of the  key
> to the
> registry in the form " ./name.so " for a library in the local directory.
> So the reload function  should read:
>
> function req(name)
> package.unload('./'..name..'.so')
> package.loadlib('./'..name..'.so','_luaopen_'..name)()
> end
>
> This will now correctly reload a local .so file loaded with "require".

Note that this only works on Linux (and maybe OSX). Does Lua have a
constant or function that can give you the appropriate suffix for a
native dynamic library?


--
chris marrin                ,""$,
[hidden email]          b`    $                             ,,.
                         mP     b'                            , 1$'
         ,.`           ,b`    ,`                              :$$'
      ,|`             mP    ,`                                       ,mm
    ,b"              b"   ,`            ,mm      m$$    ,m         ,`P$$
   m$`             ,b`  .` ,mm        ,'|$P   ,|"1$`  ,b$P       ,`  :$1
  b$`             ,$: :,`` |$$      ,`   $$` ,|` ,$$,,`"$$     .`    :$|
b$|            _m$`,:`    :$1   ,`     ,$Pm|`    `    :$$,..;"'     |$:
P$b,      _;b$$b$1"       |$$ ,`      ,$$"             ``'          $$
  ```"```'"    `"`         `""`        ""`                          ,P`
"As a general rule,don't solve puzzles that open portals to Hell"'