Decoda Lua IDE has become Open Source

classic Classic list List threaded Threaded
23 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Decoda Lua IDE has become Open Source

steve donovan
http://unknownworlds.com/blog/lua-ide-decoda-open-source/

Since looking for a good Lua environment is always a perenial question
hereabouts....

Reply | Threaded
Open this post in threaded view
|

Re: Decoda Lua IDE has become Open Source

sergei karhof
On Thu, Feb 14, 2013 at 10:57 AM, steve donovan
<[hidden email]> wrote:
> http://unknownworlds.com/blog/lua-ide-decoda-open-source/
>
> Since looking for a good Lua environment is always a perenial question
> hereabouts....
>

Great. Is this the best Lua IDE available?

Reply | Threaded
Open this post in threaded view
|

Re: Decoda Lua IDE has become Open Source

Kevin Clancy
I don't think so. It only has a watch window and does not break on errors. AFAIK, the only advantage it has over ZeroBrane and LDT is that it is slightly faster.


On Thu, Feb 14, 2013 at 8:40 AM, sergei karhof <[hidden email]> wrote:
On Thu, Feb 14, 2013 at 10:57 AM, steve donovan
<[hidden email]> wrote:
> http://unknownworlds.com/blog/lua-ide-decoda-open-source/
>
> Since looking for a good Lua environment is always a perenial question
> hereabouts....
>

Great. Is this the best Lua IDE available?


Reply | Threaded
Open this post in threaded view
|

Re: Decoda Lua IDE has become Open Source

Joshua Jensen-2
----- Original Message -----
From: [hidden email]
Date: 2/14/2013 8:29 AM
> I don't think so. It only has a watch window and does not break on
> errors. AFAIK, the only advantage it has over ZeroBrane and LDT is
> that it is slightly faster.
Unless they removed support for it, Decoda can also attach to any
external process that has a Lua core and debug it.

That's a big deal in some cases.

-Josh

Reply | Threaded
Open this post in threaded view
|

Re: Decoda Lua IDE has become Open Source

Hadriel Kaplan

On Feb 14, 2013, at 10:56 AM, Joshua Jensen <[hidden email]>
 wrote:

> Unless they removed support for it, Decoda can also attach to any external process that has a Lua core and debug it.
> That's a big deal in some cases.

So can Koneki LDT and ZeroBrane Studio.  And both of them can also run on non-Windows operating systems, unlike Decoda, which might be a big deal for some folks.  I think both LuaEdit and DforD LuaCoding can do remote debugging too, but they're only for Windows.

But I'm not disparaging Decoda at all - it's great to have so many choices for Lua IDEs.
Naturally it's impossible to name any as the "best", for all people/needs.

-hadriel


Reply | Threaded
Open this post in threaded view
|

Re: Decoda Lua IDE has become Open Source

Peter Cawley
On Thu, Feb 14, 2013 at 4:35 PM, Hadriel Kaplan <[hidden email]> wrote:

On Feb 14, 2013, at 10:56 AM, Joshua Jensen <[hidden email]>
 wrote:

> Unless they removed support for it, Decoda can also attach to any external process that has a Lua core and debug it.
> That's a big deal in some cases.

So can Koneki LDT and ZeroBrane Studio.  And both of them can also run on non-Windows operating systems, unlike Decoda, which might be a big deal for some folks.  I think both LuaEdit and DforD LuaCoding can do remote debugging too, but they're only for Windows.

In ZBT's case, it looks like the program being debugged needs to be modified in order to be debugged by putting `require("mobdebug").start()` somewhere near the start of the Lua code. In Decoda's case, absolutely zero modifications are needed.
Reply | Threaded
Open this post in threaded view
|

Re: Decoda Lua IDE has become Open Source

liam mail
In reply to this post by Hadriel Kaplan
On 14 February 2013 16:35, Hadriel Kaplan <[hidden email]> wrote:
>
> On Feb 14, 2013, at 10:56 AM, Joshua Jensen <[hidden email]>
>  wrote:
>
>> Unless they removed support for it, Decoda can also attach to any external process that has a Lua core and debug it.
>> That's a big deal in some cases.
>
> So can Koneki LDT and ZeroBrane Studio.

They both require that you have access to the Lua code which you want
to debug and can change the code to enable debugging, this is not the
case for Decoda and the Lua code could be embedded into the
executable.

--Liam

Reply | Threaded
Open this post in threaded view
|

Re: Decoda Lua IDE has become Open Source

Benjamin Cabé
>
>They both require that you have access to the Lua code which you want
>to debug and can change the code to enable debugging,

It is true indeed for LDT

>this is not the
>case for Decoda and the Lua code could be embedded into the
>executable.
>

I am curious at how this works behind the scenes? Did anybody have a look
at it?
Thanks
Benjamin


Reply | Threaded
Open this post in threaded view
|

Re: Decoda Lua IDE has become Open Source

Paul K
In reply to this post by Peter Cawley
> In ZBT's case, it looks like the program being debugged needs to be modified
> in order to be debugged by putting `require("mobdebug").start()` somewhere
> near the start of the Lua code. In Decoda's case, absolutely zero
> modifications are needed.

True; although as I understand you'd still need to have debugging
symbols for your executable available for Decoda, which means you may
need to recompile the project anyway.

Paul.

On Thu, Feb 14, 2013 at 8:41 AM, Peter Cawley <[hidden email]> wrote:

> On Thu, Feb 14, 2013 at 4:35 PM, Hadriel Kaplan <[hidden email]>
> wrote:
>>
>>
>> On Feb 14, 2013, at 10:56 AM, Joshua Jensen <[hidden email]>
>>  wrote:
>>
>> > Unless they removed support for it, Decoda can also attach to any
>> > external process that has a Lua core and debug it.
>> > That's a big deal in some cases.
>>
>> So can Koneki LDT and ZeroBrane Studio.  And both of them can also run on
>> non-Windows operating systems, unlike Decoda, which might be a big deal for
>> some folks.  I think both LuaEdit and DforD LuaCoding can do remote
>> debugging too, but they're only for Windows.
>
>
> In ZBT's case, it looks like the program being debugged needs to be modified
> in order to be debugged by putting `require("mobdebug").start()` somewhere
> near the start of the Lua code. In Decoda's case, absolutely zero
> modifications are needed.

Reply | Threaded
Open this post in threaded view
|

Re: Decoda Lua IDE has become Open Source

Peter Cawley
On Thu, Feb 14, 2013 at 5:19 PM, Paul K <[hidden email]> wrote:
> In ZBT's case, it looks like the program being debugged needs to be modified
> in order to be debugged by putting `require("mobdebug").start()` somewhere
> near the start of the Lua code. In Decoda's case, absolutely zero
> modifications are needed.

True; although as I understand you'd still need to have debugging
symbols for your executable available for Decoda, which means you may
need to recompile the project anyway.

Paul.

Either you need debugging symbols (which you'll have by default if you're compiling the code), or you need the Lua API functions to be exported from an image (which you'll have if the Lua core exists in one DLL and is used in a different DLL or EXE).
Reply | Threaded
Open this post in threaded view
|

Re: Decoda Lua IDE has become Open Source

Peter Cawley
In reply to this post by Benjamin Cabé
On Thu, Feb 14, 2013 at 5:15 PM, Benjamin Cabé <[hidden email]> wrote:
>
>They both require that you have access to the Lua code which you want
>to debug and can change the code to enable debugging,

It is true indeed for LDT

>this is not the
>case for Decoda and the Lua code could be embedded into the
>executable.
>

I am curious at how this works behind the scenes? Did anybody have a look
at it?
Thanks
Benjamin

It looks like fairly standard Windows DLL injection and function hooking to me.
Reply | Threaded
Open this post in threaded view
|

Re: Decoda Lua IDE has become Open Source

Joshua Jensen-2
In reply to this post by Paul K
----- Original Message -----
From: Paul K
Date: 2/14/2013 10:19 AM
>> In ZBT's case, it looks like the program being debugged needs to be modified
>> in order to be debugged by putting `require("mobdebug").start()` somewhere
>> near the start of the Lua code. In Decoda's case, absolutely zero
>> modifications are needed.
> True; although as I understand you'd still need to have debugging
> symbols for your executable available for Decoda, which means you may
> need to recompile the project anyway.
One of the best features of this is that it (theoretically... I've never
looked myself) would not cause any additional garbage to be created for
Lua to collect.  When you are working within constrained memory heaps, I
want to debug my Lua code in the same state that it would ship in.  Most
remote debuggers like mobdebug end up loading additional stuff into the
Lua state and causing additional memory usage.

There is another interesting theoretical Decoda feature that hooking the
remote process may offer.  (Again, I haven't looked at the code.)  
Decoda probably did not set up a line hook, as it wouldn't necessarily
have rights to add executable code to your process. Line hooks are
horribly slow; in fact, Adobe posted a patch on the mailing list for Lua
5.1 that added real breakpoints and allowed the Lua code to run at full
speed otherwise.  If they are just monitoring the Lua state from the
Decoda process, then the Lua state can run at 'full' speed.

I look forward to perusing their source code soon.

-Josh

Reply | Threaded
Open this post in threaded view
|

Re: Decoda Lua IDE has become Open Source

liam mail
In reply to this post by Benjamin Cabé
On 14 February 2013 17:15, Benjamin Cabé <[hidden email]> wrote:

>>
>>They both require that you have access to the Lua code which you want
>>to debug and can change the code to enable debugging,
>
> It is true indeed for LDT
>
>>this is not the
>>case for Decoda and the Lua code could be embedded into the
>>executable.
>>
>
> I am curious at how this works behind the scenes? Did anybody have a look
> at it?
> Thanks
> Benjamin
>
>

I have not yet looked at the Decoda code to closely but there are a
couple of methods, you can inject a patched dll/so/dylib, do the work
of the linker yourself fixing up function addresses or if Lua is
statically linked you can patch the executable using trampolines. All
of which are architecture/platform specific (unless using Microsoft's
Detours, which is free for open source usage) hence why I assume
Decoda is only currently able to target Windows x86.

--Liam

Reply | Threaded
Open this post in threaded view
|

Re: Decoda Lua IDE has become Open Source

Hadriel Kaplan
In reply to this post by Joshua Jensen-2

On Feb 14, 2013, at 12:27 PM, Joshua Jensen <[hidden email]> wrote:

> One of the best features of this is that it (theoretically... I've never looked myself) would not cause any additional garbage to be created for Lua to collect.  When you are working within constrained memory heaps, I want to debug my Lua code in the same state that it would ship in.  Most remote debuggers like mobdebug end up loading additional stuff into the Lua state and causing additional memory usage.

Sure, but the reason you may have a "constrained memory heap" is because your Lua-using program is on another system, like in an embedded appliance.  LDT and ZBS let you access that remotely - i.e., true remote debugging, rather than just local other-process debugging.  To get true remote debugging, you have to make some change in that appliance's code somewhere/somehow, for example if just to open a socket and run a remote debugger/API protocol to it (e.g., DGBP).

Although I suppose if your remote appliance is actually running Windows you could use Decoda on it remotely and remote-desktop in or whatever.  But isn't running Windows contradictory with the notion of a constrained memory heap?
;)

-hadriel


Reply | Threaded
Open this post in threaded view
|

Re: Decoda Lua IDE has become Open Source

Joshua Jensen-2
----- Original Message -----
From: Hadriel Kaplan
Date: 2/14/2013 11:29 AM
> On Feb 14, 2013, at 12:27 PM, Joshua Jensen <[hidden email]> wrote:
>
>> One of the best features of this is that it (theoretically... I've never looked myself) would not cause any additional garbage to be created for Lua to collect.  When you are working within constrained memory heaps, I want to debug my Lua code in the same state that it would ship in.  Most remote debuggers like mobdebug end up loading additional stuff into the Lua state and causing additional memory usage.
> Sure, but the reason you may have a "constrained memory heap" is because your Lua-using program is on another system, like in an embedded appliance.  LDT and ZBS let you access that remotely - i.e., true remote debugging, rather than just local other-process debugging.  To get true remote debugging, you have to make some change in that appliance's code somewhere/somehow, for example if just to open a socket and run a remote debugger/API protocol to it (e.g., DGBP).
>
> Although I suppose if your remote appliance is actually running Windows you could use Decoda on it remotely and remote-desktop in or whatever.  But isn't running Windows contradictory with the notion of a constrained memory heap?
> ;)
>
The reason you may have a constrained memory heap is because you are a
video game.  Windows or not, those tend to have limitations on memory
usage, as the same code is shared across platforms.

LuaPlus' old remote debugger code allowed you to remotely debug into the
running Lua state without affecting any of the internal memory usage.  I
no longer use that code, but it is possible to do.  You just have to
remain in C/C++ and do all of your memory manipulations and string and
socket operations there.

Tilde's remote debugger code is written in C++, and it does modify
internal Lua state, but it is nowhere near as bad as full Lua
implementations of remote debugger code.

mobdebug and company are harder to employ in this context.  They
definitely eat more memory and create a lot of garbage.

Also of note with Decoda is that it is the only multi-state Lua debugger
I know of outside of HavokScript.  Hopefully, one of the others will one
day have that.

-Josh

Reply | Threaded
Open this post in threaded view
|

Re: Decoda Lua IDE has become Open Source

Steve Litt
In reply to this post by Hadriel Kaplan
On Thu, 14 Feb 2013 16:35:01 +0000
Hadriel Kaplan <[hidden email]> wrote:

>
> On Feb 14, 2013, at 10:56 AM, Joshua Jensen <[hidden email]>
>  wrote:
>
> > Unless they removed support for it, Decoda can also attach to any
> > external process that has a Lua core and debug it. That's a big
> > deal in some cases.
>
> So can Koneki LDT and ZeroBrane Studio.  And both of them can also
> run on non-Windows operating systems, unlike Decoda, which might be a
> big deal for some folks.  I think both LuaEdit and DforD LuaCoding
> can do remote debugging too, but they're only for Windows.
>
> But I'm not disparaging Decoda at all - it's great to have so many
> choices for Lua IDEs. Naturally it's impossible to name any as the
> "best", for all people/needs.


Do any of these IDEs have database connectivity and screen painters
such that you can quickly develop an application? Something like
Powerbuilder or Delphi or Clarion?

Thanks

SteveT

Reply | Threaded
Open this post in threaded view
|

Re: Decoda Lua IDE has become Open Source

Harley Laue
On 02/14/13 15:03, Steve Litt wrote:

> On Thu, 14 Feb 2013 16:35:01 +0000
> Hadriel Kaplan <[hidden email]> wrote:
>
>> On Feb 14, 2013, at 10:56 AM, Joshua Jensen <[hidden email]>
>>   wrote:
>>
>>> Unless they removed support for it, Decoda can also attach to any
>>> external process that has a Lua core and debug it. That's a big
>>> deal in some cases.
>> So can Koneki LDT and ZeroBrane Studio.  And both of them can also
>> run on non-Windows operating systems, unlike Decoda, which might be a
>> big deal for some folks.  I think both LuaEdit and DforD LuaCoding
>> can do remote debugging too, but they're only for Windows.
>>
>> But I'm not disparaging Decoda at all - it's great to have so many
>> choices for Lua IDEs. Naturally it's impossible to name any as the
>> "best", for all people/needs.
>
> Do any of these IDEs have database connectivity and screen painters
> such that you can quickly develop an application? Something like
> Powerbuilder or Delphi or Clarion?
>
I'm not sure I understand what you're asking for. You're wanting to use
one of the IDE's to manage the database (something like the Kate plugin
that allows you to run SQL queries)? As for the "screen painters" I'm
not familiar with that unless it's some kind of UI specific thing (was
that was it was called in the WinAPI?) The only thing I can think that
you might be getting at with that is perhaps something like ZeroBrane
Studio's live coding
(http://notebook.kulchenko.com/zerobrane/live-coding-with-love.) If
you're asking for something UI toolkit specific for building the
interface, the answer is likely "no."

Reply | Threaded
Open this post in threaded view
|

Re: Decoda Lua IDE has become Open Source

Paul K
In reply to this post by Joshua Jensen-2
Hi Joshua,

> longer use that code, but it is possible to do.  You just have to remain in
> C/C++ and do all of your memory manipulations and string and socket
> operations there.

This would definitely be faster, but I'm not sure how much this would
change in terms of memory interactions (and heap requirements). There
is clearly some processing that is done on every command, but it's not
much and mobdebug should release all the memory it uses. In fact, I'd
argue that most of that allocation is already on the C/C++ side; for
example, for Lua tables that are returned by various debug.getinfo
calls.

This is not to say that there is nothing to improve. In fact, I used
the idea from Stephen Nichols who re-wrote some of the Lua calls that
return tables in C for better performance and integrated it with
mobdebug, but the result is tied to a particular platform (whether
dll, so, or dylib), and even to a particular engine, which has its own
downsides.

On requiring changes to embedded scripts, it may be possible to use
LUA_INIT to include mobdebug.start() call without making any changes
to the app, but it doesn't quite work yet (also, I'm not sure if
embedded engines do LUA_INIT processing).

> mobdebug and company are harder to employ in this context.  They definitely
> eat more memory and create a lot of garbage.

I'm not sure about "a lot of garbage" ;). I've been quite careful
about what variables to use, but clearly there is some overhead simply
because it accounts for many more situations than you may care about.
For example, there is a check for luajit you may not be using, or data
structures that keep track of coroutines you are debugging, which may
not apply if you don't need them and so on. You can easily trim some
of that to customize it for your case, but the idea is that you drop
mobdebug into your environment, call start() and should be able to
debug your Lua app whether embedded or standalone. I'd say with the
addition of Lua 5.2/LuaJIT support and cross-platform file system
mapping, it gives you a lot of coverage in terms of what you can debug
with it.

You can also use off()/on() calls to disable and re-enable debugging,
which turns off debug hook to give you normal Lua performance.

> Also of note with Decoda is that it is the only multi-state Lua debugger I
> know of outside of HavokScript.  Hopefully, one of the others will one day
> have that.

I'm not exactly sure how multi-state debugging interface would look
from the user side, but you should be able to emulate this with two
mobdebug sessions. You can open two ZeroBrane Studio instances (you
need to disable single instance check for the second instance)
listening on two ports and call require("mobdebug").start(nil, port1)
and ...start(nil, port2) from different Lua states to get two
connections. You should then be able to debug them independently. It
may be possible to use one instance of the IDE (in fact, there is an
explicit check to avoid establishing multiple debugging sessions), but
I couldn't come up with a proper UI for it and there has been no
demand for this type of functionality.

Paul.

On Thu, Feb 14, 2013 at 10:37 AM, Joshua Jensen <[hidden email]> wrote:

> ----- Original Message -----
> From: Hadriel Kaplan
> Date: 2/14/2013 11:29 AM
>>
>> On Feb 14, 2013, at 12:27 PM, Joshua Jensen <[hidden email]>
>> wrote:
>>
>>> One of the best features of this is that it (theoretically... I've never
>>> looked myself) would not cause any additional garbage to be created for Lua
>>> to collect.  When you are working within constrained memory heaps, I want to
>>> debug my Lua code in the same state that it would ship in.  Most remote
>>> debuggers like mobdebug end up loading additional stuff into the Lua state
>>> and causing additional memory usage.
>>
>> Sure, but the reason you may have a "constrained memory heap" is because
>> your Lua-using program is on another system, like in an embedded appliance.
>> LDT and ZBS let you access that remotely - i.e., true remote debugging,
>> rather than just local other-process debugging.  To get true remote
>> debugging, you have to make some change in that appliance's code
>> somewhere/somehow, for example if just to open a socket and run a remote
>> debugger/API protocol to it (e.g., DGBP).
>>
>> Although I suppose if your remote appliance is actually running Windows
>> you could use Decoda on it remotely and remote-desktop in or whatever.  But
>> isn't running Windows contradictory with the notion of a constrained memory
>> heap?
>> ;)
>>
> The reason you may have a constrained memory heap is because you are a video
> game.  Windows or not, those tend to have limitations on memory usage, as
> the same code is shared across platforms.
>
> LuaPlus' old remote debugger code allowed you to remotely debug into the
> running Lua state without affecting any of the internal memory usage.  I no
> longer use that code, but it is possible to do.  You just have to remain in
> C/C++ and do all of your memory manipulations and string and socket
> operations there.
>
> Tilde's remote debugger code is written in C++, and it does modify internal
> Lua state, but it is nowhere near as bad as full Lua implementations of
> remote debugger code.
>
> mobdebug and company are harder to employ in this context.  They definitely
> eat more memory and create a lot of garbage.
>
> Also of note with Decoda is that it is the only multi-state Lua debugger I
> know of outside of HavokScript.  Hopefully, one of the others will one day
> have that.
>
> -Josh
>

Reply | Threaded
Open this post in threaded view
|

Re: Decoda Lua IDE has become Open Source

Benoit Germain-2
In reply to this post by Joshua Jensen-2

2013/2/14 Joshua Jensen <[hidden email]>

There is another interesting theoretical Decoda feature that hooking the remote process may offer.  (Again, I haven't looked at the code.)  Decoda probably did not set up a line hook, as it wouldn't necessarily have rights to add executable code to your process. Line hooks are horribly slow; in fact, Adobe posted a patch on the mailing list for Lua 5.1 that added real breakpoints and allowed the Lua code to run at full speed otherwise.  If they are just monitoring the Lua state from the Decoda process, then the Lua state can run at 'full' speed.


I use Decoda, and I can tell you that setting a breakpoint does slow VM execution, so it probably goes through debug hooks. I believe that Decoda patches Lua API functions mainly to catch state creation and destruction. (I haven't looked at the code).


--
Benoit.
Reply | Threaded
Open this post in threaded view
|

Re: Decoda Lua IDE has become Open Source

Rapin Patrick
Let me share a bit of my experience.
I have always considered Decoda as the most powerful Lua debugger for Windows.
This does not mean it is the best IDE (it is clearly not), but it was the only tool that could debug some of my applications like LuaDura.
LuaDura for example does not support LuaSockets which is required by other debuggers, and moreover has a bunch of Lua code included, in a compressed form, inside the executable itself.
Decoda had no difficulty to debug the code in the internal files, because it places hooks on lua_load* API functions. Exactly how, I don't know (I haven't studied the sources yet).
So I think this is a great news that Decoda is now open source. The only bad news is that now I don't need it anymore: Olivetti iJet, my previous employer up to 2012, has stopped all activities!
And I couldn't find up to now a good reason to introduce Lua in my new company :-)

--
-- Patrick Rapin
-- coauthor of "Le guide de Lua et ses applications", D-BookeR
12