flua - Lua VM in Flash - my progress

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

flua - Lua VM in Flash - my progress

Jason Neufeld
Hey all! I guess this is my first time here, but hopefully I'll be
able to make a valuable contribution to the lua community.

I've been working on a game project lately. It's based on the old DOS
games ZZT (by Epic Megagames) and Megazeux, an independent production
by Greg Janson a few years later. These games used simple ANSI topdown
graphics, but what was special about them was that they were full game
creation systems. They had simple (and somewhat crude by today's
standards) scripting languages that let just about anybody who was
willing to put the time in learn basic programming skills and create
fun games to share. I really want to capture some of that simplicity
and fun on the web by creating an online game system that allows
people to create games without worrying about overly complicating
things.

A vital feature for this engine is going to be the ability to script.
I've been looking into alot of competing technologies, and I think
I've finally settled on Lua. The problem is, unless I run things
server side (which is slow and expensive on a large scale,) I'd like
to be able to run the scripts client side. So I decided to start
working on a VM implementation in Actionscript that would run on Flash
Player 9. I've built it so far from the ground up based on Kein-Hong
Man's most excellent No-Frills Introduction to Lua 5.1 VM. So far,
I've implemented enough of the VM to download bytecode from a URL and
execute the test fibonacci program correctly. The main caveat, of
course, is that Actionscript is SLOW. I've done extreme optimizations,
though, and am going to do more. Right now the fibonacci at the
default of 24 executes at approximately 2.6 seconds regular and 0.5
seconds cached. Compared to the C VMs 0.04 and 0.00005 this is
extremely slow, but consider that when I first had it working it would
time out (> 15) seconds, then I got it down to 12 seconds, finally
down to 3.5, and now today down to 2.6, (this all on my 2.4ghz core
duo macbook pro) so there is alot of room to bring things down.

I'm basically asking for any input, and most importantly, just any
interest on the project. I have a couple weeks to work on it until I
go work at a summer camp for kids for 2 months, which won't really
allow me to do any coding, so I want to know if I should finish the
opcodes and get a release version before that, since otherwise I can't
really do any work on it until august. I also need to figure out the
appropriate open source license to use for the code.

Thanks for your time!

Jason Neufeld (Phi. 3:12)

Reply | Threaded
Open this post in threaded view
|

Re: flua - Lua VM in Flash - my progress

Evan DeMond
On Mon, Jun 2, 2008 at 1:05 PM, Jason Neufeld <[hidden email]> wrote:
Hey all! I guess this is my first time here, but hopefully I'll be
able to make a valuable contribution to the lua community.

I've been working on a game project lately. It's based on the old DOS
games ZZT (by Epic Megagames) and Megazeux, an independent production
by Greg Janson a few years later. These games used simple ANSI topdown
graphics, but what was special about them was that they were full game
creation systems. They had simple (and somewhat crude by today's
standards) scripting languages that let just about anybody who was
willing to put the time in learn basic programming skills and create
fun games to share. I really want to capture some of that simplicity
and fun on the web by creating an online game system that allows
people to create games without worrying about overly complicating
things.

A vital feature for this engine is going to be the ability to script.
I've been looking into alot of competing technologies, and I think
I've finally settled on Lua. The problem is, unless I run things
server side (which is slow and expensive on a large scale,) I'd like
to be able to run the scripts client side. So I decided to start
working on a VM implementation in Actionscript that would run on Flash
Player 9. I've built it so far from the ground up based on Kein-Hong
Man's most excellent No-Frills Introduction to Lua 5.1 VM. So far,
I've implemented enough of the VM to download bytecode from a URL and
execute the test fibonacci program correctly. The main caveat, of
course, is that Actionscript is SLOW. I've done extreme optimizations,
though, and am going to do more. Right now the fibonacci at the
default of 24 executes at approximately 2.6 seconds regular and 0.5
seconds cached. Compared to the C VMs 0.04 and 0.00005 this is
extremely slow, but consider that when I first had it working it would
time out (> 15) seconds, then I got it down to 12 seconds, finally
down to 3.5, and now today down to 2.6, (this all on my 2.4ghz core
duo macbook pro) so there is alot of room to bring things down.

I'm basically asking for any input, and most importantly, just any
interest on the project. I have a couple weeks to work on it until I
go work at a summer camp for kids for 2 months, which won't really
allow me to do any coding, so I want to know if I should finish the
opcodes and get a release version before that, since otherwise I can't
really do any work on it until august. I also need to figure out the
appropriate open source license to use for the code.

Thanks for your time!

Hah, nice to see another MZX veteran on the list. One of the MZX community members has been working on integrating Lua as a new scripting language for MegaZeux, and an alternative to Robotic. Not sure how well that will work, but I digress.

Anyways, if you've got any information posted on this project anywhere, I'd like to check up on how it's going. And you can always consult the forums at http://www.digitalmzx.net/ for MZX inspiration and guidance. (unless you're already on there...)

Evan
Reply | Threaded
Open this post in threaded view
|

Re: flua - Lua VM in Flash - my progress

KHMan
In reply to this post by Jason Neufeld
Jason Neufeld wrote:
> [snip snip] The main caveat, of
> course, is that Actionscript is SLOW. I've done extreme optimizations,
> though, and am going to do more. Right now the fibonacci at the
> default of 24 executes at approximately 2.6 seconds regular and 0.5
> seconds cached. Compared to the C VMs 0.04 and 0.00005 this is
> extremely slow, but consider that when I first had it working it would
> time out (> 15) seconds, then I got it down to 12 seconds, finally
> down to 3.5, and now today down to 2.6, (this all on my 2.4ghz core
> duo macbook pro) so there is alot of room to bring things down.
> 
> I'm basically asking for any input, and most importantly, just any
> interest on the project. I have a couple weeks to work on it until I
> go work at a summer camp for kids for 2 months, which won't really
> allow me to do any coding, so I want to know if I should finish the
> opcodes and get a release version before that, since otherwise I can't
> really do any work on it until august. I also need to figure out the
> appropriate open source license to use for the code.

Performance does seem to be a problem... :-) I've always thought
of Javascript as a nice target to write a VM on, but running an
interpreter on an interpreter, even if the latter is heavily
optimized... As an educational tool it would definitely be cool,
but I'm not sure about serious apps like game scripting.

There's also another project that sounds like it's chugging along
fine and there's code released, LuLu : LuaVM on Lua, at:

  http://luaforge.net/projects/lulu/

I wonder if the author is lurking around these parts...

-- 
Cheers,
Kein-Hong Man (esq.)
Kuala Lumpur, Malaysia

Reply | Threaded
Open this post in threaded view
|

Re: flua - Lua VM in Flash - my progress

Matt Campbell-4
In reply to this post by Jason Neufeld
Hey Jason,

This sounds like an interesting project.

Instead of implementing a VM in ActionScript, you should write a compiler that emits AVM2 bytecode, along with a runtime library in ActionScript for Lua operations that don't directly correspond to AVM2 opcodes. For more information about AVM2, including an ECMAScript compiler implemented in ActionScript, see the Mozilla Tamarin project.

As for an open-source license, I would consider these three:

1.  MIT/X license:  Same as Lua 5.

2. Mozilla Public License: Use this if you want to require other developers to share modifications that they make to your code, without preventing them from using your code in proprietary software.

3. GPL: Use this if you want to require other developers to share modifications that they make to your code *and* prevent them from using your code in proprietary software (or you could use a dual licenses, GPL plus a commercial license).

I hope this helps.

Matt

Reply | Threaded
Open this post in threaded view
|

Re: flua - Lua VM in Flash - my progress

Jason Neufeld
Thanks for all the suggestions so far!

As far as the AVM2 suggestion, I think it's a great idea, I actually
had the same suggestion from the author of tinypy when I was
considering using his system. However, the problem in this case is the
domain of the problem. The game system that I'm building is basically
going to allow for custom user scripting that will be run from the
game engine, but in a sense it needs to be heavily sandboxed. For
example, a script could be attached to a particular object and tell it
how to move around and interact with the world around it, but it
should have no privileges beyond whatever built-in functions it has
been given. I think a direct bytecode compilation would be great if
speed were the only concern, but essentially what's going to happen
here is that the Lua code will be compiled serverside and then
executed from the client as part of the customizable aspects of the
game script. Because of the security and the modular nature of the
engine it seems like having some sort of VM running the scripts is the
best solution for my particular problem. I'd love to hear more
suggestions if you think there's a way around the problem that would
allow me to safely use AVM2 bytecode, because I know it would be
faster and much more efficient.

I do have some good news! since I last posted I've been able to get
the fib time down slightly to 2.5 seconds on normal and 0.001 seconds
cached.

Jason Neufeld (Phil. 3:12)

On Mon, Jun 2, 2008 at 1:35 PM, Matt Campbell <[hidden email]> wrote:
> Hey Jason,
>
> This sounds like an interesting project.
>
> Instead of implementing a VM in ActionScript, you should write a compiler
> that emits AVM2 bytecode, along with a runtime library in ActionScript for
> Lua operations that don't directly correspond to AVM2 opcodes.  For more
> information about AVM2, including an ECMAScript compiler implemented in
> ActionScript, see the Mozilla Tamarin project.
>
> As for an open-source license, I would consider these three:
>
> 1.  MIT/X license:  Same as Lua 5.
>
> 2.  Mozilla Public License:  Use this if you want to require other
> developers to share modifications that they make to your code, without
> preventing them from using your code in proprietary software.
>
> 3.  GPL:  Use this if you want to require other developers to share
> modifications that they make to your code *and* prevent them from using your
> code in proprietary software (or you could use a dual licenses, GPL plus a
> commercial license).
>
> I hope this helps.
>
> Matt
>

Reply | Threaded
Open this post in threaded view
|

Re: flua - Lua VM in Flash - my progress

Tony Finch
On Mon, 2 Jun 2008, Jason Neufeld wrote:
>
> The game system that I'm building is basically going to allow for custom
> user scripting that will be run from the game engine, but in a sense it
> needs to be heavily sandboxed. For example, a script could be attached
> to a particular object and tell it how to move around and interact with
> the world around it, but it should have no privileges beyond whatever
> built-in functions it has been given.

You should have a look at Caja which is designed to allow just this, for
Javascript in browsers. Not very on-topic, sorry!
http://code.google.com/p/google-caja/

Tony.
-- 
f.anthony.n.finch  <[hidden email]>  http://dotat.at/
IRISH SEA: SOUTHWESTERLY 3 OR 4, OCCASIONALLY 5. SLIGHT OR MODERATE. RAIN, FOG
PATCHES AT FIRST. MODERATE OR GOOD, OCCASIONALLY VERY POOR AT FIRST.

Reply | Threaded
Open this post in threaded view
|

Re: flua - Lua VM in Flash - my progress

Ben-2
I have a good friend who loves the ZZT games, and happens to be
looking into Lua recently, so he might be interested in a project like
this.  I'll pass this along to him, if you don't mind.

Ben

On Mon, Jun 2, 2008 at 2:36 PM, Tony Finch <[hidden email]> wrote:
> On Mon, 2 Jun 2008, Jason Neufeld wrote:
>>
>> The game system that I'm building is basically going to allow for custom
>> user scripting that will be run from the game engine, but in a sense it
>> needs to be heavily sandboxed. For example, a script could be attached
>> to a particular object and tell it how to move around and interact with
>> the world around it, but it should have no privileges beyond whatever
>> built-in functions it has been given.
>
> You should have a look at Caja which is designed to allow just this, for
> Javascript in browsers. Not very on-topic, sorry!
> http://code.google.com/p/google-caja/
>
> Tony.
> --
> f.anthony.n.finch  <[hidden email]>  http://dotat.at/
> IRISH SEA: SOUTHWESTERLY 3 OR 4, OCCASIONALLY 5. SLIGHT OR MODERATE. RAIN, FOG
> PATCHES AT FIRST. MODERATE OR GOOD, OCCASIONALLY VERY POOR AT FIRST.
>

Reply | Threaded
Open this post in threaded view
|

Re: flua - Lua VM in Flash - my progress

Mark Hamburg-5
In reply to this post by Tony Finch
If you haven't already done so, make sure you are using all of the strong typing features from ActionScript 3. They reportedly make a big difference in ActionScript performance.

Mark


Reply | Threaded
Open this post in threaded view
|

Re: flua - Lua VM in Flash - my progress

Jason Neufeld
Thanks Mark! I actually switched everything to strict typing a few
hours ago, which did bump it up. I'm proud to say the fib 24 test is
running at around 1.3 seconds now, which for this VM is pretty fast!
I'm thinking the speed won't be a major issue because most game
scripts in mind are simple 'move up, move down, shoot, say hi' types
with some interobject communication, and most people shouldn't need to
do complicated recursive functions... we'll see i guess!

To Ben: Go ahead and pass on the info! The site is www.gcre8.com ,
which unfortunately only has a link to the development blog as of now.
That's where I'm keeping all the information about the game engine,
and I'll probably end up putting most of the flua stuff there as well.

Jason Neufeld (Phil. 3:12)


On Mon, Jun 2, 2008 at 8:55 PM, Mark Hamburg <[hidden email]> wrote:
> If you haven't already done so, make sure you are using all of the strong
> typing features from ActionScript 3. They reportedly make a big difference
> in ActionScript performance.
>
> Mark
>
>

Reply | Threaded
Open this post in threaded view
|

Re: flua - Lua VM in Flash - my progress

Stefan Sandberg
as3 really isn't that slow (http://giger.servebeer.com/Gordon/)

I initially used neoswiff for flash stuff, which is a C# compiler for flash, which worked great, except that it didn't have access to the entire api, some things were missing etc, so if you do get this thing off the ground(and I sure hope you do), it's vital that it can actually use all that flash has to offer, or else it wont be used by anyone..
(the C# compiler is gone now, for example)


Jason Neufeld wrote:
Thanks Mark! I actually switched everything to strict typing a few
hours ago, which did bump it up. I'm proud to say the fib 24 test is
running at around 1.3 seconds now, which for this VM is pretty fast!
I'm thinking the speed won't be a major issue because most game
scripts in mind are simple 'move up, move down, shoot, say hi' types
with some interobject communication, and most people shouldn't need to
do complicated recursive functions... we'll see i guess!

To Ben: Go ahead and pass on the info! The site is www.gcre8.com ,
which unfortunately only has a link to the development blog as of now.
That's where I'm keeping all the information about the game engine,
and I'll probably end up putting most of the flua stuff there as well.

Jason Neufeld (Phil. 3:12)


On Mon, Jun 2, 2008 at 8:55 PM, Mark Hamburg <[hidden email]> wrote:
If you haven't already done so, make sure you are using all of the strong
typing features from ActionScript 3. They reportedly make a big difference
in ActionScript performance.

Mark





Reply | Threaded
Open this post in threaded view
|

Re: flua - Lua VM in Flash - my progress

Matt Campbell-4
In reply to this post by Jason Neufeld
Given this most recent increase in performance, I suggest that you implement the rest of the opcodes and release the source as soon as you can, so others can start playing with it and providing feedback.

Also, does this VM support coroutines? I've read that coroutines are quite a powerful feature for game scripting.

Matt

Jason Neufeld wrote:
Thanks Mark! I actually switched everything to strict typing a few
hours ago, which did bump it up. I'm proud to say the fib 24 test is
running at around 1.3 seconds now, which for this VM is pretty fast!
I'm thinking the speed won't be a major issue because most game
scripts in mind are simple 'move up, move down, shoot, say hi' types
with some interobject communication, and most people shouldn't need to
do complicated recursive functions... we'll see i guess!

To Ben: Go ahead and pass on the info! The site is www.gcre8.com ,
which unfortunately only has a link to the development blog as of now.
That's where I'm keeping all the information about the game engine,
and I'll probably end up putting most of the flua stuff there as well.

Jason Neufeld (Phil. 3:12)


On Mon, Jun 2, 2008 at 8:55 PM, Mark Hamburg <[hidden email]> wrote:
If you haven't already done so, make sure you are using all of the strong
typing features from ActionScript 3. They reportedly make a big difference
in ActionScript performance.

Mark




Reply | Threaded
Open this post in threaded view
|

Re: flua - Lua VM in Flash - my progress

Nicolas Cannasse
In reply to this post by Jason Neufeld
Jason Neufeld a écrit :
Thanks for all the suggestions so far!

As far as the AVM2 suggestion, I think it's a great idea, I actually
had the same suggestion from the author of tinypy when I was
considering using his system. However, the problem in this case is the
domain of the problem. The game system that I'm building is basically
going to allow for custom user scripting that will be run from the
game engine, but in a sense it needs to be heavily sandboxed. For
example, a script could be attached to a particular object and tell it
how to move around and interact with the world around it, but it
should have no privileges beyond whatever built-in functions it has
been given. I think a direct bytecode compilation would be great if
speed were the only concern, but essentially what's going to happen
here is that the Lua code will be compiled serverside and then
executed from the client as part of the customizable aspects of the
game script. Because of the security and the modular nature of the
engine it seems like having some sort of VM running the scripts is the
best solution for my particular problem. I'd love to hear more
suggestions if you think there's a way around the problem that would
allow me to safely use AVM2 bytecode, because I know it would be
faster and much more efficient.

You could use hxASM (http://haxe.org/com/libs/hxasm)

It enables you to generate AVM2 bytecode at runtime, either on the client side in Flash, or on the server side / commandline using NekoVM.

This way you could translate your Lua bytecode to AVM2 one on-the-fly by doing appropriate checks for sandbox restrictions.

Also, using haXe to write your VM for Flash9 would surely help since there are many optimizations and it runs faster than AS3.

Hope that helps,
Nicolas