LuaJIT without the JIT?

classic Classic list List threaded Threaded
33 messages Options
12
Reply | Threaded
Open this post in threaded view
|

LuaJIT without the JIT?

Rena
I was just wondering to myself if LuaJIT could ever be hacked to
function as a Lua compiler, that compiles scripts to standalone
executables or libraries. It'd be great if we could take advantage of
its speed and the FFI to write Lua extensions in Lua that would
normally have to be written in C... :)

--
Sent from my toaster.

Reply | Threaded
Open this post in threaded view
|

Re: LuaJIT without the JIT?

Fabio Kaminski
There 's a "common" believe that jitted code can(hypoteticaly) run
even faster than compiled code..

see the famous hp dinamo(1999) for some insights..

http://www.hpl.hp.com/techreports/1999/HPL-1999-78.html

there are optimizations in the output native binary that are only
possible with a dinamic compiler running and getting more
introspection..

so.. i think in the near future.. statically compiled code (like C)..
will eat some dust from jitted code in the benchmarks..as this happens
already in some corner cases..

so i think a real hacker .. would just embed luajit.a into its library
or binary.. and enjoy his code flying at the light speed..

luajit already create code almost fast as c (i think i saw a big
matrix multiplication other day with a lot of inner loops running even
faster than C ) and running a dinamic language..

i think soon we will be running luajit benchmarks against hand-coded
assembly just because no one else left .. :)


On Wed, Jun 22, 2011 at 7:33 PM, HyperHacker <[hidden email]> wrote:

> I was just wondering to myself if LuaJIT could ever be hacked to
> function as a Lua compiler, that compiles scripts to standalone
> executables or libraries. It'd be great if we could take advantage of
> its speed and the FFI to write Lua extensions in Lua that would
> normally have to be written in C... :)
>
> --
> Sent from my toaster.
>
>

Reply | Threaded
Open this post in threaded view
|

Re: LuaJIT without the JIT?

Dimiter 'malkia' Stanev
        LuaJIT is great, but don't buy into JIT would rule over the rest of the
stuff. Think about it as a tool, not as the silver bullet. There are
always going to be faster ways to achieve something, given dedication to
the problem. Use LuaJIT to solves general programming problems. So far
it works very good for me.

        What I find cool about JIT is really that you gain very good speed, and
you don't have to worry about configurations (architecture),
preprocessing (compilation, linking, etc.) + one can communicate using
source code send over the network to get executed, rather than scripts
that need to get compiled.

        For example I'm sitting from on my Mac OS X book, and uploading scripts
to my Efika MX SmartTop (ARM cpu), and they just work. But I had to
spend some time finding proper (for me) "C" and "C++" libraries to use
with it - ZeroMQ, OpenGL, GLES, GLFW, etc, and thanks to the awesome FFI
I don't lose much speed (in fact it might be faster sometimes than
called directly from "C" - not sure).

        But you won't be able to use easily LuaJIT to speed up a DXT
compression for example, while it takes just OpenMP and "C" to do it
with one or two pragmas (split the image by 4x4 blocks, or 4xNNN rows,
NNNx4 columns, and call the DXT encoding routine). Then again LuaJIT
would allow me to easily encode this problem in another language - be it
OpenCL, CUDA, or if needed call gcc and compile "C".

On 6/22/2011 4:48 PM, Fabio Kaminski wrote:

> There 's a "common" believe that jitted code can(hypoteticaly) run
> even faster than compiled code..
>
> see the famous hp dinamo(1999) for some insights..
>
> http://www.hpl.hp.com/techreports/1999/HPL-1999-78.html
>
> there are optimizations in the output native binary that are only
> possible with a dinamic compiler running and getting more
> introspection..
>
> so.. i think in the near future.. statically compiled code (like C)..
> will eat some dust from jitted code in the benchmarks..as this happens
> already in some corner cases..
>
> so i think a real hacker .. would just embed luajit.a into its library
> or binary.. and enjoy his code flying at the light speed..
>
> luajit already create code almost fast as c (i think i saw a big
> matrix multiplication other day with a lot of inner loops running even
> faster than C ) and running a dinamic language..
>
> i think soon we will be running luajit benchmarks against hand-coded
> assembly just because no one else left .. :)
>
>
> On Wed, Jun 22, 2011 at 7:33 PM, HyperHacker<[hidden email]>  wrote:
>> I was just wondering to myself if LuaJIT could ever be hacked to
>> function as a Lua compiler, that compiles scripts to standalone
>> executables or libraries. It'd be great if we could take advantage of
>> its speed and the FFI to write Lua extensions in Lua that would
>> normally have to be written in C... :)
>>
>> --
>> Sent from my toaster.
>>
>>
>
>


Reply | Threaded
Open this post in threaded view
|

Re: LuaJIT without the JIT?

Fabio Kaminski
On Wed, Jun 22, 2011 at 10:17 PM, Dimiter "malkia" Stanev
<[hidden email]> wrote:

>        LuaJIT is great, but don't buy into JIT would rule over the rest of
> the stuff. Think about it as a tool, not as the silver bullet. There are
> always going to be faster ways to achieve something, given dedication to the
> problem. Use LuaJIT to solves general programming problems. So far it works
> very good for me.
>
>        What I find cool about JIT is really that you gain very good speed,
> and you don't have to worry about configurations (architecture),
> preprocessing (compilation, linking, etc.) + one can communicate using
> source code send over the network to get executed, rather than scripts that
> need to get compiled.
>
>        For example I'm sitting from on my Mac OS X book, and uploading
> scripts to my Efika MX SmartTop (ARM cpu), and they just work. But I had to
> spend some time finding proper (for me) "C" and "C++" libraries to use with
> it - ZeroMQ, OpenGL, GLES, GLFW, etc, and thanks to the awesome FFI I don't
> lose much speed (in fact it might be faster sometimes than called directly
> from "C" - not sure).
>
>        But you won't be able to use easily LuaJIT to speed up a DXT
> compression for example, while it takes just OpenMP and "C" to do it with
> one or two pragmas (split the image by 4x4 blocks, or 4xNNN rows, NNNx4
> columns, and call the DXT encoding routine). Then again LuaJIT would allow
> me to easily encode this problem in another language - be it OpenCL, CUDA,
> or if needed call gcc and compile "C".

The voice of experience.. completely agree.. use the right tool for
the right job.

Reply | Threaded
Open this post in threaded view
|

Re: LuaJIT without the JIT?

Josh Haberman
In reply to this post by Rena
HyperHacker <hyperhacker <at> gmail.com> writes:
> I was just wondering to myself if LuaJIT could ever be hacked to
> function as a Lua compiler, that compiles scripts to standalone
> executables or libraries.

By "standalone executable" do you mean:

1. one that embeds the LuaJIT and the Lua program, or
2. one that contains machine code for the original Lua program,
   but not LuaJIT or Lua program itself.

(1) is not very difficult.

(2) is not feasible, AIUI.  LuaJIT can only generate fast code
when it can observe runtime traces.  These traces are not
something that can be generated ahead-of-time, because they
depend on eg. the actual types of variables.  Even if they could
be generated ahead-of-time, the runtime profile of the program
could change as it is running; a tracing JIT can adapt to this,
but AOT compiling could not, which would compromise performance.

Also, since the slow paths of the JIT-ted code fall back to
the interpreter, you'd have to include LuaJIT's interpreter in
the standalone executable.

Josh


Reply | Threaded
Open this post in threaded view
|

Re: LuaJIT without the JIT?

Rena
On Wed, Jun 22, 2011 at 19:51, Josh Haberman <[hidden email]> wrote:

> HyperHacker <hyperhacker <at> gmail.com> writes:
>> I was just wondering to myself if LuaJIT could ever be hacked to
>> function as a Lua compiler, that compiles scripts to standalone
>> executables or libraries.
>
> By "standalone executable" do you mean:
>
> 1. one that embeds the LuaJIT and the Lua program, or
> 2. one that contains machine code for the original Lua program,
>   but not LuaJIT or Lua program itself.
>
> (1) is not very difficult.
>
> (2) is not feasible, AIUI.  LuaJIT can only generate fast code
> when it can observe runtime traces.  These traces are not
> something that can be generated ahead-of-time, because they
> depend on eg. the actual types of variables.  Even if they could
> be generated ahead-of-time, the runtime profile of the program
> could change as it is running; a tracing JIT can adapt to this,
> but AOT compiling could not, which would compromise performance.
>
> Also, since the slow paths of the JIT-ted code fall back to
> the interpreter, you'd have to include LuaJIT's interpreter in
> the standalone executable.
>
> Josh
>
>
>

I hadn't thought of the idea of just statically linking Lua and LuaJIT
into a program, but that's probably a better and more feasible
approach to making standalone executables than trying to compile to
native code. What I was more interested in though was the possibility
of writing a Lua module, which uses the FFI to do things that normally
would require dropping down to C (typically bindings to C libraries),
and being able to compile it into a shared object file that could then
be loaded and used by Lua without LuaJIT.

Doing so by including LuaJIT in the module is still an interesting
idea, but means your module will be rather large, and wastes space if
you have more than one and/or you're using LuaJIT anyway.

By the sounds of it, LuaJIT itself can't feasibly compile to native
code, but I'm still curious about the possibility of writing bindings
to C libraries entirely in Lua, without depending on LuaJIT for the
FFI.

 --
Sent from my toaster.

Reply | Threaded
Open this post in threaded view
|

Re: LuaJIT without the JIT?

Dimiter 'malkia' Stanev
According to Mike, if I'm not wrong,

FFI is way slower with JIT turned off. I guess lots of stack
manipulation has to be done in iterative way (no codegen)

FFI is very fast, if not faster than statically compiled code when JIT
is on (it can generate the optimal calling code, and for DLL's it can
directly put jmp <absolute address>, instead of going through table). I
also remmember that Mike was tracking how the function was behaving
after the first call - whether it was calle save, or caller save - so he
can "memoize" this and reuse it later, and maybe other "magical" stuff
too :)

Nonetheless, the idea of using LuaJIT as FFI embedding device, even with
JIT turned off is not bad. I'm finding myself now relying on this, but
soon or later you would hit situation where a callback needs to be
provided, and that callback would have to be your lua function. That is
still no implemented yet. (for example - Win32 RegisterClass with
WindowProc, or qsort with sorting function). Also one can't reuse the
same LuaJIT context for the callback, has to be new one.



On 6/22/2011 9:07 PM, HyperHacker wrote:

> On Wed, Jun 22, 2011 at 19:51, Josh Haberman<[hidden email]>  wrote:
>> HyperHacker<hyperhacker<at>  gmail.com>  writes:
>>> I was just wondering to myself if LuaJIT could ever be hacked to
>>> function as a Lua compiler, that compiles scripts to standalone
>>> executables or libraries.
>>
>> By "standalone executable" do you mean:
>>
>> 1. one that embeds the LuaJIT and the Lua program, or
>> 2. one that contains machine code for the original Lua program,
>>    but not LuaJIT or Lua program itself.
>>
>> (1) is not very difficult.
>>
>> (2) is not feasible, AIUI.  LuaJIT can only generate fast code
>> when it can observe runtime traces.  These traces are not
>> something that can be generated ahead-of-time, because they
>> depend on eg. the actual types of variables.  Even if they could
>> be generated ahead-of-time, the runtime profile of the program
>> could change as it is running; a tracing JIT can adapt to this,
>> but AOT compiling could not, which would compromise performance.
>>
>> Also, since the slow paths of the JIT-ted code fall back to
>> the interpreter, you'd have to include LuaJIT's interpreter in
>> the standalone executable.
>>
>> Josh
>>
>>
>>
>
> I hadn't thought of the idea of just statically linking Lua and LuaJIT
> into a program, but that's probably a better and more feasible
> approach to making standalone executables than trying to compile to
> native code. What I was more interested in though was the possibility
> of writing a Lua module, which uses the FFI to do things that normally
> would require dropping down to C (typically bindings to C libraries),
> and being able to compile it into a shared object file that could then
> be loaded and used by Lua without LuaJIT.
>
> Doing so by including LuaJIT in the module is still an interesting
> idea, but means your module will be rather large, and wastes space if
> you have more than one and/or you're using LuaJIT anyway.
>
> By the sounds of it, LuaJIT itself can't feasibly compile to native
> code, but I'm still curious about the possibility of writing bindings
> to C libraries entirely in Lua, without depending on LuaJIT for the
> FFI.
>
>   --
> Sent from my toaster.
>
>


Reply | Threaded
Open this post in threaded view
|

Re: LuaJIT without the JIT?

Florian Weimer
* Dimiter Stanev:

> Nonetheless, the idea of using LuaJIT as FFI embedding device, even
> with JIT turned off is not bad. I'm finding myself now relying on
> this, but soon or later you would hit situation where a callback needs
> to be provided, and that callback would have to be your lua
> function. That is still no implemented yet. (for example - Win32
> RegisterClass with WindowProc, or qsort with sorting function). Also
> one can't reuse the same LuaJIT context for the callback, has to be
> new one.

If you're on an embedded target and there's little risk that someone
will run your code with a different LuaJIT version, you can
selectively switch of the JIT compiler for code which calls into the C
code which end up calling back into Lua, using ffi.off.  Right now,
this will make the callback into the same LuaJIT context safe.

Reply | Threaded
Open this post in threaded view
|

Re: LuaJIT without the JIT?

Francesco Abbate
In reply to this post by Rena
2011/6/23 HyperHacker <[hidden email]>:
> I was just wondering to myself if LuaJIT could ever be hacked to
> function as a Lua compiler, that compiles scripts to standalone
> executables or libraries. It'd be great if we could take advantage of
> its speed and the FFI to write Lua extensions in Lua that would
> normally have to be written in C... :)

Hi,

me too I've been thinking about this and I guess it will not happens
because of how the JIT compiler works and because the design of Lua as
a programming language.

The reason is the LuaJIT2 is a trace compiler and, as far as I
understood it works by letting the program run in interpreted mode, it
does analyze in real time the code that is executed and it tries to
generate optimized native code for the running application. This way
of working is very different from a classical optimizing compiler like
for C or C++ where the optimization is based only on static analysis
of the code and the type signature of variables and functions is
strongly used to produce optimal machine code.
LuaJIT2 would never be able to produce optimized code without running
the code because it never perform static analysis of Lua code. I don't
know if you can save in theory the generated optimized code in a file
so that it can be executed later without trace analysis. May be this
is possible but probably this is a very difficult engineering task and
the interest of having that is very low because the JIT can always
produce optimized code on the fly very quickly and for every
architecture where you may run so there is no much interest in
producing an executable.

Another remarkable difference between JIT code and statically
optimized C code is that in C all the code is uniformly optimized in
the sense that every fragment of code is completely optimized
according to the optimization flags. In JIT compiled code this is
different because LJ2 produces highly optimized code for the inner
loops while for the outer loops the optimizations are much less
aggressive or even absent since some code can be left in interpreted
mode if the heuristic algorithm used decide that it is not critical.

The dualism between static analysis (like with C) and trace analysis
is strictly tied to the dualism between programming languages with
static and dynamic typing. This latter kind of languages like Lua or
Python does not admit a static compilation by design because you don't
have any information about what type of value a variable can contain,
the information is only available at run time. Also the mechanism of
table lookup that is extensively used in Lua and Python does not allow
any static analysis because the information are available only at run
time and everything can change during the execution.

Even javascript is of the same family of Lua and Python and it does
also have a JIT compiler. It is interesting to remark how this dualism
is now very clearly defined between C/C++ with a strong typing systems
and optimizing compilers in one side and Lua, Python and JavaScripts
with dynamic typing and associative arrays and very advanced JIT
compiler on the other side. These two approaches are two different
responses to the need of producing optimized code from an high level
programming language. I believe this duality will stay for a long time
now and I'm wondering when a new programming paradigm will emerge to
replace both of them. In my point of view there is a strong demand,
the programming in C or C++ requires to take care manually of a lot of
small details and even a small error can have catastrophic
consequences like your application disappearing suddenly (segmentation
fault) or memory leaks that eats all your memory.

Programming languages like Lua or Python are very attractive because
can let people work on the logic of their applications and it does
allow a much more rapid development of complex applications. The other
side of the coin is that these programming languages are very
difficult to optimize because everything is dynamic and they therefore
defeat any static analysis, only JIT trace compiler are possible.

Another approach in the mid between them is the Java approach that
have both a static strong type system and a VM with JIT code
generation. In my point of view this is the *wrong* solution because
it takes the worst of each world. The typing system and the rigid
class based system with explicit exceptions declaration make the code
very verbose and cumbersome to write taking away a lot of the
programmer's time. From the other side you don't even have a static
optimization but only a JIT code generation that, while effective,
does eat a lot of resources in term of memory and execution time.
The only reason why it was and it is so successful is that managers
likes it because you can have a lot of contracts explicitly declared
to use modular programming techniques an it is really *safe* in the
sense that it cannot crash at any moment just because you forgot to
check a null pointer or something like that (just a small parenthesis
about this latter point, this advantage is often overestimated because
if it true that Java cannot crash it is true that you can have
unhandled exceptions or exceptions that are inappropriately handled
resulting in non-functional behaviours).
So Java is both boring to write with a lot of boiler-plate
declarations and so on and the execution environment is largely
suboptimal with a lot of resources required, the omnipresent GC.

I hope my email was not boring even if it is certainly OT, I've taken
the opportunity of this thread to share some reflections about
optimized code generation and programming languages.

--
Francesco

Reply | Threaded
Open this post in threaded view
|

Re: LuaJIT without the JIT?

steve donovan
On Thu, Jun 23, 2011 at 12:41 PM, Francesco Abbate
<[hidden email]> wrote:

Nice summary.

> So Java is both boring to write with a lot of boiler-plate
> declarations and so on and the execution environment is largely
> suboptimal with a lot of resources required, the omnipresent GC.

Ah, but boring is often good. It is useful to have a compiler to save
us from ourselves, and the straightforward syntax of Java makes the
tool support very good.

Big Lua programs do exist (e.g. Adobe Lightroom) but they need good
engineering and discipline to be manageable;  the industry likes Java
because the discipline is built-in and the coding environment
compensates somewhat for the limitations of the language.[1]

A really good development environment for Lua could approach code
analysis like a JIT compiler; by evaluating the code dynamically, it
learns what the _likely_ argument types are for functions, and _which_
functions were actually called.  A approach similar to Smalltalk could
work, in which you work on a 'live patient': a constantly evolving
process where code is dynamically modified.

This would be a fine project for an intelligent person looking for
something non-trivial to do.

steve d.

[1] ok yes, people still write nonsense. But no environment will stop
that, unless Eclipse achieves sentience and says "I'm sorry Dave, I
can't allow you to do that"

Reply | Threaded
Open this post in threaded view
|

Re: LuaJIT without the JIT?

Alex Queiroz
In reply to this post by Francesco Abbate
Hallo,

On Thu, Jun 23, 2011 at 12:41 PM, Francesco Abbate
<[hidden email]> wrote:

>
> The dualism between static analysis (like with C) and trace analysis
> is strictly tied to the dualism between programming languages with
> static and dynamic typing. This latter kind of languages like Lua or
> Python does not admit a static compilation by design because you don't
> have any information about what type of value a variable can contain,
> the information is only available at run time. Also the mechanism of
> table lookup that is extensively used in Lua and Python does not allow
> any static analysis because the information are available only at run
> time and everything can change during the execution.
>

     This is not true. It is possible, practical and worthy to do
static analysis in dynamic-typed languages implementations. The
literature on this span decades. The "static" here is not about
typing, but instead "analysing without running the code." I myself
have implemented dead-variable elimination, removal of unused
arguments, inlining and removal of known procedures etc. Besides these
simple ones, you can lift computations out of loops (no trace needed
in several cases), do especulative inlining, partial evaluation etc.
Really, you can do *a lot* before running the code.

--
-alex
http://www.artisancoder.com/

Reply | Threaded
Open this post in threaded view
|

Re: LuaJIT without the JIT?

steve donovan
On Thu, Jun 23, 2011 at 2:11 PM, Alex Queiroz <[hidden email]> wrote:
>     This is not true. It is possible, practical and worthy to do
> static analysis in dynamic-typed languages implementations.

You can indeed get pretty far.  E.g. David M's lua-inspect [1]
integrates with SciTE and vim.

It knows about your static variables, and can show you unknown
globals, which usually means 'spelling mistake'

steve d.


[1] https://github.com/davidm/lua-inspect

Reply | Threaded
Open this post in threaded view
|

Re: LuaJIT without the JIT?

Javier Guerra Giraldez
In reply to this post by Alex Queiroz
On Thu, Jun 23, 2011 at 7:11 AM, Alex Queiroz <[hidden email]> wrote:
> The "static" here is not about
> typing, but instead "analysing without running the code."

true, but when the variables don't have a static type, there's very
little that can be known without running the code.

remember that non-JIT compilers can't _guess_, they have to _prove_
they're covering all possibilities.

--
Javier

Reply | Threaded
Open this post in threaded view
|

Re: LuaJIT without the JIT?

Alex Queiroz
Hallo,

On Thu, Jun 23, 2011 at 4:40 PM, Javier Guerra Giraldez
<[hidden email]> wrote:
> On Thu, Jun 23, 2011 at 7:11 AM, Alex Queiroz <[hidden email]> wrote:
>> The "static" here is not about
>> typing, but instead "analysing without running the code."
>
> true, but when the variables don't have a static type, there's very
> little that can be known without running the code.
>

     Not true. A lot can be known without running the code.

http://www.ccs.neu.edu/home/shivers/citations.html#diss

--
-alex
http://www.artisancoder.com/

Reply | Threaded
Open this post in threaded view
|

Re: LuaJIT without the JIT?

Javier Guerra Giraldez
On Thu, Jun 23, 2011 at 9:47 AM, Alex Queiroz <[hidden email]> wrote:
> On Thu, Jun 23, 2011 at 4:40 PM, Javier Guerra Giraldez
> <[hidden email]> wrote:
>>
>> true, but when the variables don't have a static type, there's very
>> little that can be known without running the code.
>>
>
>     Not true. A lot can be known without running the code.

amazing.

so, let's me rephrase:  there's very little that a tracing compiler
(like LuaJIT) can know without running the code.

OTOH, if anybody is secretly developing a static compiler with
control-flow analysis and type inference, more power to them!  it
would be a major accomplishment, likely with as many innovations and
breakthroughs in that field as Mike has done in the JIT space.

--
Javier

Reply | Threaded
Open this post in threaded view
|

Re: LuaJIT without the JIT?

Lorenzo Donati-2
In reply to this post by Francesco Abbate
On 23/06/2011 12.41, Francesco Abbate wrote:
> 2011/6/23 HyperHacker<[hidden email]>:
[snip]

>
> Another approach in the mid between them is the Java approach that
> have both a static strong type system and a VM with JIT code
> generation. In my point of view this is the *wrong* solution because
> it takes the worst of each world. The typing system and the rigid
> class based system with explicit exceptions declaration make the code
> very verbose and cumbersome to write taking away a lot of the
> programmer's time. From the other side you don't even have a static
> optimization but only a JIT code generation that, while effective,
> does eat a lot of resources in term of memory and execution time.
> The only reason why it was and it is so successful is that managers
> likes it because you can have a lot of contracts explicitly declared
> to use modular programming techniques an it is really *safe* in the
> sense that it cannot crash at any moment just because you forgot to
> check a null pointer or something like that

I was a Java programmer, some years ago, and indeed *it is* verbose, but
in a very linear way (sometimes too much!).

If you educate yourself accurately and study the language definition,
you'll see that many of its features are aimed at avoiding shooting
yourself in the foot!

You must be really a sloppy programmer (or a terribly lazy one) to write
non-working code in Java. Of course, the prerequisite is the word
"study": conventions, compiler, APIs, SDK tools, etc, etc., etc.
They are not terribly difficult stuff, but there are *TONS* of them (if
you don't know what I'm talking about, try to unpack the Sun SDK docs
zipball and you'll see!!! - I last programmed in Java some ~7 years ago
and I used Java 6.0 - If the trend is the same, now you could probably
spend 10 years of your life just reading all that stuff and doing
nothing else, but breathing!!!! ;-)

I used dozens of packages and libraries written by other people and the
integration with the systems I worked on were smooth most of the times.

Even the most awful feature (generics) is carefully designed. BTW I say
"awful" because Java programmers waited for 4~5 years (or more? don't
remember) for generics to be introduced into the language, since they
were expecting something like C++ templates (but with Java ease of use)
and the possibility to do duck typing and improve code reuse.
IMO the result it was almost a monstruosity: an extremely complicated
way to overspecify types statically just to avoid some runtime casting.
No duck-typing at all. And the language specification (2nd edition -
which was hard stuff anyway, but somewhat linear and clear) became a
nightmare (3rd ed.) in every part that touched generics!!!

But still, it was (and I suppose it remains), a very robust platform,
especially for web development (even for things different from fancy
webpages. I mean, secure services for home banking, just to name one).
It was *not* just marketing hype. Remember that when Java was hyped -
with some reason - Javascript (which has nothing to do with Java,
besides having a similar C-like syntax) "stole" its name to ride the
wave of its popularity!

Moreover, mastering a good IDE as eclipse skyrockets the productivity of
any programmer. I had a colleague researcher of mine (not primarily a
programmer, but with several years of Java programming experience) who
did not want to switch from her emacs+manual compilation workflow. Java
was a pain for any small refactoring cycle (she was waaaaay less lazy
than myself and I didn't succeed in convincing her in doing the switch! :-)

So yes, Java is not good either for RAD nor for people needing an
elegant language. It needs dedication to learning the tools. It is
terrible when it comes to sheer performance (compared to other
solutions). And it is not friendly with foreign languages (I tried many
times to learn Java Native Interface, the Java's way to interact with C,
and always gave up in despair - I always found a better, although
slower, alternative in pure Java.


I don't like it too much, but it is not crap, as you seem to hint.
Java is ugly as a first time language because for the classic Hello
World example it requires lots of boilerplate. The learning curve is not
steep at all, but it requires lots of time to understand the usefulness
of all that verbosity.

And you really understand it when you see that a single programmer can
produce a fully tested 30kLOC package with tens of classes, organized in
different abstraction layers, which can run with no errors for weeks as
part of a complex web app running on a server.

If you just write small applications, or your need is sheer speed, or
superoptimized code for numerical stuff, Java is either overkill,
bloated or sluggish. But if reliability, robustness, i18n support and
web support is paramount for you, then Java is really good.
Yes, it is frustrating to write thousands of times the same boilerplate
for, say, an anonymous inner class, especially if you know Lua can do
better with closures and upvalues. But most of those problems are solved
with a good IDE.

> (just a small parenthesis
> about this latter point, this advantage is often overestimated because
> if it true that Java cannot crash it is true that you can have
> unhandled exceptions or exceptions that are inappropriately handled
> resulting in non-functional behaviours).

The advantage is not primarily that Java doesn't crash with a segfault,
but that it has a linear and pervasive exception mechanism which is
enforced at various levels. I don't think it is overestimated. Error
handling and management *IS HARD*. The time and resources you want to
spend on this task depends on what kind of safety requirements your
system has. If you are writing a graphical app then a simple message box
notifying the user that, say, the system is low on memory, may be
enough. If you want a system to be fault-tolerant or even
fault-resistant, well, Java is your friend (I heard that Ada is better
in this respect, since it is used for avionics and military stuff, but I
don't have the expertise to comment on that).

As for people not handling an exception: a bug is a bug. It is not
Java's fault that a programmer introduced a bug in error management.
If you catch an exception and then fail to handle it, then it is your
fault. Java doesn't make intrinsically more difficult to handle errors
than other widely known languages.

And if a programmer is not expert enough to design his error management
strategy in advance, well the guilty is him or the boss that has put him
to work on something he was not able to handle. No free beer in error
handling!

Sloppy programmers will always exist. Once a colleague of mine,
debugging and adapting to our needs some ASP pages "borrowed" from
another department of the research institution were we worked, showed me
an .asp file whose first line (IIRC) read:

On error goto 0

That line meant (as she explained to me, since I didn't know anything
about ASP): "if you detect an error everywhere in the page, just ignore
it"!!!!!


> So Java is both boring to write with a lot of boiler-plate
> declarations and so on and the execution environment is largely
> suboptimal with a lot of resources required, the omnipresent GC.
>

Yes, I agree, it is boring and boilerplate litter the code, and it is
not very readable (on the other hand it is fairly linear, it is
difficult not to get the meaning of a snippet if the programmer was
careful in commenting the code and choosing the identifier according to
the best practices). As for resources: it is somewhat avid, but this is
the price for the safety net you play on.



> I hope my email was not boring even if it is certainly OT, I've taken
> the opportunity of this thread to share some reflections about
> optimized code generation and programming languages.
>

Nope. Food for thought :-)

Cheers!
-- Lorenzo


P.S.: as I said above, I haven't touched a line of Java for almost 6
year, so don't flame me if I said something really wrong about what's
Java is today. I described the situation I knew at that time.
Of course a current Java programmer is welcome to correct me! :-)

Reply | Threaded
Open this post in threaded view
|

Re: LuaJIT without the JIT?

Henning Diedrich
On 6/23/11 5:29 PM, Lorenzo Donati wrote:

... welcome to correct me! :-)

Ok then, further OT I guess, both for

* avoidance of shooting in your own foot and
* error handling,

maybe take a look at Erlang for the future.

Since the advent of multi cores, Java has lost the safety edge. Resource deadlocks in truly parallel environments became the new Achilles Heel of systems and Java has no built-in way around it. Like Java prevented C's pointer crashes, by design, Erlang prevents programmers from dead locking.

In a nutshell, it allows no shared memory between processes and that is that. But just like Java still knows undefined pointers, you obviously can produce dead locks and races in Erlang, but not in the usual contentious ways.

Especially because Java Systems can be huge, a race that kills the VM, only once in 24h, is just a nightmare. I knew one $$$ million company that just restarted their Java servers every night because they couldn't locate the leaks from it. That's Java reality today, on multi core machines.

Also Java's way of catching errors, tediously, is not the only way. Erlang has the philosophy to 'just let it crash' and a totally different approach to recovery, which saves the functional code from being overwhelmed and littered by defenses ten times its size. Erlang, in a way, handles the unforeseen as well as the expected problems, which is not what Java's statically typed Exception model would promise.

Erlang is not a random fad for an example, it's a contender in the same realm, with an impressive history. To my knowledge the least academic of the FPLs.

Somewhat back on topic: Erlang has no type system and some extremely bright people failed at giving it one when they tried. But it has an excellent static analyzer that catches type errors of even composite types.

And, Erlang has a VM, and programs are compiled to bytecode but can also be interpreted from source - then some years ago it got a native compiler that speeds things up nicely. That compiler gives speed ups of up to around 10x. To get to that, it translates the VM bytecode into native machine code and bends the call table to the new code.

Henning
Reply | Threaded
Open this post in threaded view
|

Re: LuaJIT without the JIT?

Lorenzo Donati-2
On 23/06/2011 18.41, Henning Diedrich wrote:

> On 6/23/11 5:29 PM, Lorenzo Donati wrote:
>
>> ... welcome to correct me! :-)
>
> Ok then, further OT I guess, both for
>
> * avoidance of shooting in your own foot and
> * error handling,
>
> maybe take a look at Erlang for the future.
>
> Since the advent of multi cores, Java has lost the safety edge. Resource
> deadlocks in truly parallel environments became the new Achilles Heel of
> systems and Java has no built-in way around it. Like Java prevented C's
> pointer crashes, by design, Erlang prevents programmers from dead locking.
>
> In a nutshell, it allows no shared memory between processes and that is
> that. But just like Java still knows undefined pointers, you obviously
> can produce dead locks and races in Erlang, but not in the usual
> contentious ways.
>
> Especially because Java Systems can be huge, a race that kills the VM,
> only once in 24h, is just a nightmare. I knew one $$$ million company
> that just restarted their Java servers every night because they couldn't
> locate the leaks from it. That's Java reality today, on multi core machines.
>
> Also Java's way of catching errors, tediously, is not the only way.
> Erlang has the philosophy to 'just let it crash' and a totally different
> approach to recovery, which saves the functional code from being
> overwhelmed and littered by defenses ten times its size. Erlang, in a
> way, handles the unforeseen as well as the expected problems, which is
> not what Java's statically typed Exception model would promise.
>

Woah! Thanks for the update! I didn't suspect that 6 years could turn
the tide in such a dramatic way for Java!

> Erlang is not a random fad for an example, it's a contender in the same
> realm, with an impressive history. To my knowledge the least academic of
> the FPLs.
>
Interesting!


> Somewhat back on topic: Erlang has no type system and some extremely
> bright people failed at giving it one when they tried. But it has an
> excellent static analyzer that catches type errors of even composite types.
>
> And, Erlang has a VM, and programs are compiled to bytecode but can also
> be interpreted from source - then some years ago it got a native
> compiler that speeds things up nicely. That compiler gives speed ups of
> up to around 10x. To get to that, it translates the VM bytecode into
> native machine code and bends the call table to the new code.
>
> Henning

Thanks for more brain-food! :-)

-- Lorenzo


Reply | Threaded
Open this post in threaded view
|

Re: LuaJIT without the JIT?

Francesco Abbate
In reply to this post by Henning Diedrich
Lorenzo, Henning, all,

thank you very much for the interesting discussion about Java and
Erlang, you are certainly more knowledgeable than me about these
programming languages and I think that I'm going to learn more about
Erlang.

Ultimately it seems that the idea that I had of Java is basically
correct even if I was not aware of a lot of details. I have no
experience of Java programming, my only knowledge came from my work
where we have a lot of Java applications. Sometimes they throw stupid
exceptions and became unusable and they are so terribly slow and
memory hungry so I just hate them.

For the other side my feeling is also that Lua really *lacks* the
possibility of Java to detect errors before execution using a typing
system or something similar. The fact that you have no types at all
and no warnings of any kind even for a small typing error is terrible
when you develop something even moderately complex. In addition you
don't even have a debugger so it is even more painful. And yes I know
that some home-made 'strict' packages and debugger exists but they are
often incomplete and of little aid.

When I work in C or C++ I'm glad that many trivial and not-so-trivial
errors can be detected during compilation and, what a relief, I can
throw the debugger and inspect all the variables step-by-step when I
really don't understand.

This is even more OT but it is strictly related to the Java
discussion. Where Java is very strong Lua is terribly lacking :-) and
please, no flames, I don't want to be polemic, I express just some
reflections about a programming language, Lua, that I ultimately like
and use (beside C, C++ and python).

Francesco

Reply | Threaded
Open this post in threaded view
|

Re: LuaJIT without the JIT?

Dimiter 'malkia' Stanev
        Lua is very much suited for exploratory programming, live coding,
scripting, customizing, tweaking, even creating makefile system (premake
for example) at very fast pace without recompile, relink cycles. My plan
someday is to be able to use it from Autodesk products to write certain
plugins, without the need to recompile, and ability to tweak and fix the
things on the fly. Also any technical artist would prefer Lua much more
than C++.

        Now take that, and in only 300kb executable code you have LuaJIT that
can bring the performance very close to "C" for a lot of things.

        I've seen only few other languages, systems close to that. One is
Common Lisp (and specific implementations), but no Common Lisp system
comes small, and all of them are statically compiled in dynamic
environment - unlike LuaJIT with trace compilation. For that reason the
get fast code in CL you have to specify where needed the types, and
these serve as promises to the compiler, which he would obey and make
your code fast (SBCL, LispWorks, Allegro, ClozureCL and others are doing
pretty good job at it).

On 6/23/2011 2:07 PM, Francesco Abbate wrote:

> Lorenzo, Henning, all,
>
> thank you very much for the interesting discussion about Java and
> Erlang, you are certainly more knowledgeable than me about these
> programming languages and I think that I'm going to learn more about
> Erlang.
>
> Ultimately it seems that the idea that I had of Java is basically
> correct even if I was not aware of a lot of details. I have no
> experience of Java programming, my only knowledge came from my work
> where we have a lot of Java applications. Sometimes they throw stupid
> exceptions and became unusable and they are so terribly slow and
> memory hungry so I just hate them.
>
> For the other side my feeling is also that Lua really *lacks* the
> possibility of Java to detect errors before execution using a typing
> system or something similar. The fact that you have no types at all
> and no warnings of any kind even for a small typing error is terrible
> when you develop something even moderately complex. In addition you
> don't even have a debugger so it is even more painful. And yes I know
> that some home-made 'strict' packages and debugger exists but they are
> often incomplete and of little aid.
>
> When I work in C or C++ I'm glad that many trivial and not-so-trivial
> errors can be detected during compilation and, what a relief, I can
> throw the debugger and inspect all the variables step-by-step when I
> really don't understand.
>
> This is even more OT but it is strictly related to the Java
> discussion. Where Java is very strong Lua is terribly lacking :-) and
> please, no flames, I don't want to be polemic, I express just some
> reflections about a programming language, Lua, that I ultimately like
> and use (beside C, C++ and python).
>
> Francesco
>
>


12