Perl, Python, Ruby, PHP, C, C++, Lua, tcl, javascript and Java comparison

classic Classic list List threaded Threaded
38 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Perl, Python, Ruby, PHP, C, C++, Lua, tcl, javascript and Java comparison

Daniel Bolgheroni
Maybe this should be of interest:

http://raid6.com.au/~onlyjob/posts/arena/

Reply | Threaded
Open this post in threaded view
|

Re: Perl, Python, Ruby, PHP, C, C++, Lua, tcl, javascript and Java comparison

Liao Yu Lei-2
Great, but where is LuaJIT ?


On Tuesday, July 16, 2013 at 12:48 AM, Daniel Bolgheroni wrote:

> Maybe this should be of interest:
>
> http://raid6.com.au/~onlyjob/posts/arena/ 



Reply | Threaded
Open this post in threaded view
|

Re: Perl, Python, Ruby, PHP, C, C++, Lua, tcl, javascript and Java comparison

Roberto Ierusalimschy
In reply to this post by Daniel Bolgheroni
> Maybe this should be of interest:
>
> http://raid6.com.au/~onlyjob/posts/arena/

This test is ridiculous. Whoever wrote the code does not know how
to program in Lua (or in Java or in Javascript).

-- Roberto

Reply | Threaded
Open this post in threaded view
|

Re: Perl, Python, Ruby, PHP, C, C++, Lua, tcl, javascript and Java comparison

Andrew Wilson-4
In reply to this post by Liao Yu Lei-2
These benchmarks assume naive programmers for the languages, making them useless. 


For example lua benchmark, didn't use locals  or table.concat

 
io.stdout:setvbuf "no";             --  io.flush();

-- didn't use local for variables
local str='abcdefgh'..'efghefgh';
local imax=1024/string.len(str)*1024*4;         -- 4mb

local starttime=os.time();
print "exec.tm.sec\tstr.length";

gstr='';
local i=0;

while i < imax+1000 do
        i=i+1;
        gstr= table.concat({gstr}, str) -- was gstr=gstr..str
        gstr=string.gsub(gstr,"efgh","____");
        lngth=string.len(str)*i;
        if(math.mod(lngth,1024*256)==0) then
                print(os.time()-starttime.."sec\t\t"..(lngth/1024).."kb");
        end
end


On Mon, Jul 15, 2013 at 12:54 PM, Liao Yu Lei <[hidden email]> wrote:
Great, but where is LuaJIT ?


On Tuesday, July 16, 2013 at 12:48 AM, Daniel Bolgheroni wrote:

> Maybe this should be of interest:
>
> http://raid6.com.au/~onlyjob/posts/arena/




Reply | Threaded
Open this post in threaded view
|

Re: Perl, Python, Ruby, PHP, C, C++, Lua, tcl, javascript and Java comparison

Ką Mykolas
In reply to this post by Roberto Ierusalimschy
The code is direct translation from one language.
Some guy already mentioned the same at the comments. Also he was asked
for proper code example. Still waiting for that.

On 7/15/13, Roberto Ierusalimschy <[hidden email]> wrote:

>> Maybe this should be of interest:
>>
>> http://raid6.com.au/~onlyjob/posts/arena/
>
> This test is ridiculous. Whoever wrote the code does not know how
> to program in Lua (or in Java or in Javascript).
>
> -- Roberto
>
>

Reply | Threaded
Open this post in threaded view
|

Re: Perl, Python, Ruby, PHP, C, C++, Lua, tcl, javascript and Java comparison

Henning Diedrich
"String manipulation is the core functionality for all languages so this allows to compare languages fairly."

Says it all.

On Jul 15, 2013, at 7:37 PM, kamicc olo <[hidden email]> wrote:

> The code is direct translation from one language.
> Some guy already mentioned the same at the comments. Also he was asked
> for proper code example. Still waiting for that.
>
> On 7/15/13, Roberto Ierusalimschy <[hidden email]> wrote:
>>> Maybe this should be of interest:
>>>
>>> http://raid6.com.au/~onlyjob/posts/arena/
>>
>> This test is ridiculous. Whoever wrote the code does not know how
>> to program in Lua (or in Java or in Javascript).
>>
>> -- Roberto
>>
>>
>


Reply | Threaded
Open this post in threaded view
|

Re: Perl, Python, Ruby, PHP, C, C++, Lua, tcl, javascript and Java comparison

Henrique Gogó
Poor test.
Tendentious to Perl


2013/7/15 Henning Diedrich <[hidden email]>
"String manipulation is the core functionality for all languages so this allows to compare languages fairly."

Says it all.

On Jul 15, 2013, at 7:37 PM, kamicc olo <[hidden email]> wrote:

> The code is direct translation from one language.
> Some guy already mentioned the same at the comments. Also he was asked
> for proper code example. Still waiting for that.
>
> On 7/15/13, Roberto Ierusalimschy <[hidden email]> wrote:
>>> Maybe this should be of interest:
>>>
>>> http://raid6.com.au/~onlyjob/posts/arena/
>>
>> This test is ridiculous. Whoever wrote the code does not know how
>> to program in Lua (or in Java or in Javascript).
>>
>> -- Roberto
>>
>>
>





--
Henrique Gogó
http://gogs.com.br
Reply | Threaded
Open this post in threaded view
|

Re: Perl, Python, Ruby, PHP, C, C++, Lua, tcl, javascript and Java comparison

Roberto Ierusalimschy
In reply to this post by Ką Mykolas
> The code is direct translation from one language.
> Some guy already mentioned the same at the comments. Also he was asked
> for proper code example. Still waiting for that.

The answer to this comment shows all:

  Re: comment 15:
  Your suggestion would be easier to understand if you provide a code
  example. Please do not assume all readers to be profoundly competent
  in Lua to understand what you're saying.

You do not need to be "profoundly" competent in Lua, but you need to be
at least competent to do a benchmark. (A trivial search for "Lua string
concatenation", for instance, would show what is wrong.)

Moreover, his comments about Java at the end of the page shows how
unbiased he is. This "benchmark" does not seem to be made to really
evaluate different languages, but to lambast Java. (As Lua handles
strings similarly to Java, it was caught in the crossfire.)

-- Roberto

Reply | Threaded
Open this post in threaded view
|

Re: Perl, Python, Ruby, PHP, C, C++, Lua, tcl, javascript and Java comparison

Johnson Lin
In fact this benchmark really made my day.

I laughed so hard when I saw the code. And somehow he had the courage to put those codes on the web and say it's for benchmarking.


2013/7/16 Roberto Ierusalimschy <[hidden email]>
> The code is direct translation from one language.
> Some guy already mentioned the same at the comments. Also he was asked
> for proper code example. Still waiting for that.

The answer to this comment shows all:

  Re: comment 15:
  Your suggestion would be easier to understand if you provide a code
  example. Please do not assume all readers to be profoundly competent
  in Lua to understand what you're saying.

You do not need to be "profoundly" competent in Lua, but you need to be
at least competent to do a benchmark. (A trivial search for "Lua string
concatenation", for instance, would show what is wrong.)

Moreover, his comments about Java at the end of the page shows how
unbiased he is. This "benchmark" does not seem to be made to really
evaluate different languages, but to lambast Java. (As Lua handles
strings similarly to Java, it was caught in the crossfire.)

-- Roberto


Reply | Threaded
Open this post in threaded view
|

Re: Perl, Python, Ruby, PHP, C, C++, Lua, tcl, javascript and Java comparison

Javier Guerra Giraldez
In reply to this post by Roberto Ierusalimschy
On Mon, Jul 15, 2013 at 1:10 PM, Roberto Ierusalimschy
<[hidden email]> wrote:
> Moreover, his comments about Java at the end of the page shows how
> unbiased he is. This "benchmark" does not seem to be made to really
> evaluate different languages, but to lambast Java. (As Lua handles
> strings similarly to Java, it was caught in the crossfire.)

right, the comment son Java and Perl are quite telling.

what i found interesting is that Python performed so well, since the
code is similar.  I guess that somewhere in the many megabytes of
special-cases in the language core there's something that manages to
optimize bad code.

--
Javier

Reply | Threaded
Open this post in threaded view
|

Re: Perl, Python, Ruby, PHP, C, C++, Lua, tcl, javascript and Java comparison

Roberto Ierusalimschy
> what i found interesting is that Python performed so well, since the
> code is similar.  I guess that somewhere in the many megabytes of
> special-cases in the language core there's something that manages to
> optimize bad code.

The key to the difference is x += y versus x = x + y (assuming '+' for
the concatenate operator) and the internals implied by that, not any
smart optimization. The same languages that are fast in this particular
code are slow in any code that involves a lot of simple string assignments.

-- Roberto

Reply | Threaded
Open this post in threaded view
|

Re: Perl, Python, Ruby, PHP, C, C++, Lua, tcl, javascript and Java comparison

Daniel Bolgheroni
In reply to this post by Roberto Ierusalimschy
On Mon, Jul 15, 2013 at 02:24:28PM -0300, Roberto Ierusalimschy wrote:
> > Maybe this should be of interest:
> >
> > http://raid6.com.au/~onlyjob/posts/arena/
>
> This test is ridiculous.

Yes.

> Whoever wrote the code does not know how to program in Lua (or in Java
> or in Javascript).

Just for the record, I didn't try to make a point. In fact, I'm always
skeptical about benchmarks. It's hard to get it right (and very hard if
you don't know what you're doing), and almost every one fails in one way
or another.

Sorry for the noise.

Reply | Threaded
Open this post in threaded view
|

Re: Perl, Python, Ruby, PHP, C, C++, Lua, tcl, javascript and Java comparison

Laurent FAILLIE
In reply to this post by Henning Diedrich
Le 15/07/2013 19:41, Henning Diedrich a écrit :
> "String manipulation is the core functionality for all languages so this allows to compare languages fairly."
>
> Says it all.

C code is totally biased :
* it disables output buffering ... so console throughput penalize the result
* I strongly doubt that other languages are all doing such stupid memory
reallocation for every string manipulation : the optimizer should detect
everything we are touching always the same string so it always use the
same quite large temporary buffer. As strings don't exist as such for
the C compiler it can't do such optimization.
Bye

Lolo

Reply | Threaded
Open this post in threaded view
|

Re: Perl, Python, Ruby, PHP, C, C++, Lua, tcl, javascript and Java comparison

Alexey Malyshev
I've tested this bencmark sample, and test says that a bottleneck is
string.gsub.
So, I've replaced string.gsub with my simple realization of  replace
function in C.
Here modified code and results for first 5 steps:


     local str='abcdefgh'..'efghefgh';
     local imax=1024/#str*1024*4;         -- 4mb

     local starttime=os.time();
     print "exec.tm.sec\tstr.length";

     local gstr='';
     local i=0;

     while i < imax+1000 do
             i=i+1;
             gstr=gstr..str;
             gstr=string.swap(gstr,"efgh","____");
             local lngth=#str*i;
             if(math.mod(lngth,1024*256)==0) then
print(os.time()-starttime.."sec\t\t"..(lngth/1024).."kb");
             end
     end


Timings string.swap:
2sec        256kb
11sec        512kb
27sec        768kb
49sec        1024kb
78sec        1280kb

Timings string.gsub:
29sec        256kb
118sec        512kb
275sec        768kb

Tested on Intel Core i5-2300 2.8GHz

Reply | Threaded
Open this post in threaded view
|

Re: Perl, Python, Ruby, PHP, C, C++, Lua, tcl, javascript and Java comparison

oliver-2
In reply to this post by Andrew Wilson-4
On Mon, Jul 15, 2013 at 1:25 PM, Andrew Wilson <[hidden email]> wrote:
These benchmarks assume naive programmers for the languages, making them useless. 
 
They are useless for advanced Lua programmers; they're actually quite representative for novice Lua programmers, and somewhere in between for intermediate :) Granted, use of "local" is so simple, and well documented (wiki, gem) that there is little excuse not to use it. But only advanced Lua users have time/interest to learn such intricacies as the idiom gstr= table.concat({gstr}, str), which is an optimization at the expense of clarity. 

Actually I couldn't understand that line and sure enough, it is not equivalent to gstr=gstr..str. The intended expression was likely table.concat({gstr, str}), showing again how optimization has a price (clarity, which in this case led to a typo which still compiled fine). 

So I ran some tests on stock lua 5.1 running in a virtual machine. It shows that the table.concat({gstr, str}) is in fact slower. This is (surely, but I'm no expert) because that algorithm creates a new table every time through the loop, which is if I'm not mistaken the second optimization discussed by Roberto in his gem article. So I moved the table creation out of the loop: 

...
local tt={gstr,str}
while i < imax+1000 do
        i=i+1;
        tt[1]=gstr; gstr=table.concat(tt) -- was gstr=gstr..str
...

These are the timings that I obtained, I hope the formatting doesn't get clobbered: 

str.length |       exec.tm.sec
           |    A       B        C
-----------|-------------------------
256kb      |  48sec   56sec    47sec 
512kb      | 194sec   225sec   189sec
768kb      | 451sec   473sec   436sec

A: using "gstr=gstr..str"
B: using "table.concat({gstr, str})"
C: using "tt[1]=gstr; table.concat(tt)" (and tt={gstr,str} before the loop)

A good example of how that optimization seemed clever but is probably a bad idea: A) a hurried implementation of the table.concat trick required unclear code, leading to a typo, and even once fixed, was in fact 15% slower than the original technique; B) even once "properly" applied, the trick provided a 2% improvement only, clearly not worth the significant decrease in maintainability.

Oliver


Reply | Threaded
Open this post in threaded view
|

Re: Perl, Python, Ruby, PHP, C, C++, Lua, tcl, javascript and Java comparison

Andrew Starks
On Tue, Jul 16, 2013 at 12:22 AM, oliver <[hidden email]> wrote:

> On Mon, Jul 15, 2013 at 1:25 PM, Andrew Wilson <[hidden email]> wrote:
>>
>> These benchmarks assume naive programmers for the languages, making them
>> useless.
>
>
> They are useless for advanced Lua programmers; they're actually quite
> representative for novice Lua programmers, and somewhere in between for
> intermediate :) Granted, use of "local" is so simple, and well documented
> (wiki, gem) that there is little excuse not to use it. But only advanced Lua
> users have time/interest to learn such intricacies as the idiom gstr=
> table.concat({gstr}, str), which is an optimization at the expense of
> clarity.
>
> Actually I couldn't understand that line and sure enough, it is not
> equivalent to gstr=gstr..str. The intended expression was likely
> table.concat({gstr, str}), showing again how optimization has a price
> (clarity, which in this case led to a typo which still compiled fine).
>
> So I ran some tests on stock lua 5.1 running in a virtual machine. It shows
> that the table.concat({gstr, str}) is in fact slower. This is (surely, but
> I'm no expert) because that algorithm creates a new table every time through
> the loop, which is if I'm not mistaken the second optimization discussed by
> Roberto in his gem article. So I moved the table creation out of the loop:
>
> ...
> local tt={gstr,str}
>
> while i < imax+1000 do
>         i=i+1;
>         tt[1]=gstr; gstr=table.concat(tt) -- was gstr=gstr..str
> ...
>
> These are the timings that I obtained, I hope the formatting doesn't get
> clobbered:
>
> str.length |       exec.tm.sec
>            |    A       B        C
> -----------|-------------------------
> 256kb      |  48sec   56sec    47sec
> 512kb      | 194sec   225sec   189sec
> 768kb      | 451sec   473sec   436sec
>
> A: using "gstr=gstr..str"
> B: using "table.concat({gstr, str})"
> C: using "tt[1]=gstr; table.concat(tt)" (and tt={gstr,str} before the loop)
>
> A good example of how that optimization seemed clever but is probably a bad
> idea: A) a hurried implementation of the table.concat trick required unclear
> code, leading to a typo, and even once fixed, was in fact 15% slower than
> the original technique; B) even once "properly" applied, the trick provided
> a 2% improvement only, clearly not worth the significant decrease in
> maintainability.
>
> Oliver
>
>

My first encounter with table.concat was when a looping string
concatenation was taking wall-clock time. Then it took me a second to
remember that I read that sort of thing was slow and then I've never
had a problem with it.

My point is that you're a novice or intermediate Lua programmer, right
up until the point the wrong way bites you. Then in about 5 seconds,
you're a super-smart-advanced Lua programmer that knows everything
there is to know about the immutability of strings (they are
immutable) and how to do it "the right way."

So, the benchmark is useless and says nothing about how Lua performs
in the wild. IMHO, of course.

-Andrew

Reply | Threaded
Open this post in threaded view
|

Re: Perl, Python, Ruby, PHP, C, C++, Lua, tcl, javascript and Java comparison

g.lister
I personally use this page to make some vague abstract conclusions about what is a good language to use:
http://benchmarksgame.alioth.debian.org/

note that it explicitly says a  benchmarksgame and there are no hard headed opinions to weed through. I do write Java code for a living ... yep ... and I often detest waiting for another workspace rebuild or recompile, and the mountains of abstractions etc. etc. (Which is Ooh more than Java), but I find Java still a competent tool for certain tasks. I find that the more one knows the more all languages seem the same, you just mix the soup (recompile/parse/tockenize/convert to binary etc. Etc.) many times to go back to assembly and massive amounts of caching complex schedulers and special purpose hardware. Speed is a concern, memory too maintainability you bet etc. etc. But things like complexity and maintainability are pretty big opposites once you start pushing both to the max and what your team knows now beats all other arguements ... Just try converting a Java dev to Java script (this benchmark is headed there it seems)

I find that a small sword could be quite a weapon in skillful hands, being an expert in any of these languages is hard - bordering impossible for someone doing the same thing over-and-over (web apps) just check the java-docs and see how many classes one use day-in-day-out. Last but not least a complex tool in incompetend hands is quite dangerous/funny (mood dependend) too watch. There are too many Google Search Copy Paste Certified (GSCPC) brothers and sisters out their and falling the trap is a necesity at times, easy and enticing at others.

Did I miss something or there are really no summaries for all languages.. Reading on a phone :(
Cheers and thanks for the read :)

----- Original message -----

> On Tue, Jul 16, 2013 at 12:22 AM, oliver <[hidden email]>
> wrote:
> > On Mon, Jul 15, 2013 at 1:25 PM, Andrew Wilson <[hidden email]>
> > wrote:
> > >
> > > These benchmarks assume naive programmers for the languages, making
> > > them useless.
> >
> >
> > They are useless for advanced Lua programmers; they're actually quite
> > representative for novice Lua programmers, and somewhere in between for
> > intermediate :) Granted, use of "local" is so simple, and well
> > documented (wiki, gem) that there is little excuse not to use it. But
> > only advanced Lua users have time/interest to learn such intricacies
> > as the idiom gstr= table.concat({gstr}, str), which is an optimization
> > at the expense of clarity.
> >
> > Actually I couldn't understand that line and sure enough, it is not
> > equivalent to gstr=gstr..str. The intended expression was likely
> > table.concat({gstr, str}), showing again how optimization has a price
> > (clarity, which in this case led to a typo which still compiled fine).
> >
> > So I ran some tests on stock lua 5.1 running in a virtual machine. It
> > shows that the table.concat({gstr, str}) is in fact slower. This is
> > (surely, but I'm no expert) because that algorithm creates a new table
> > every time through the loop, which is if I'm not mistaken the second
> > optimization discussed by Roberto in his gem article. So I moved the
> > table creation out of the loop:
> >
> > ...
> > local tt={gstr,str}
> >
> > while i < imax+1000 do
> > i=i+1;
> > tt[1]=gstr; gstr=table.concat(tt) -- was gstr=gstr..str
> > ...
> >
> > These are the timings that I obtained, I hope the formatting doesn't
> > get clobbered:
> >
> > str.length |             exec.tm.sec
> > |       A             B               C
> > -----------|-------------------------
> > 256kb           |   48sec     56sec       47sec
> > 512kb           | 194sec     225sec     189sec
> > 768kb           | 451sec     473sec     436sec
> >
> > A: using "gstr=gstr..str"
> > B: using "table.concat({gstr, str})"
> > C: using "tt[1]=gstr; table.concat(tt)" (and tt={gstr,str} before the
> > loop)
> >
> > A good example of how that optimization seemed clever but is probably
> > a bad idea: A) a hurried implementation of the table.concat trick
> > required unclear code, leading to a typo, and even once fixed, was in
> > fact 15% slower than the original technique; B) even once "properly"
> > applied, the trick provided a 2% improvement only, clearly not worth
> > the significant decrease in maintainability.
> >
> > Oliver
> >
> >
>
> My first encounter with table.concat was when a looping string
> concatenation was taking wall-clock time. Then it took me a second to
> remember that I read that sort of thing was slow and then I've never
> had a problem with it.
>
> My point is that you're a novice or intermediate Lua programmer, right
> up until the point the wrong way bites you. Then in about 5 seconds,
> you're a super-smart-advanced Lua programmer that knows everything
> there is to know about the immutability of strings (they are
> immutable) and how to do it "the right way."
>
> So, the benchmark is useless and says nothing about how Lua performs
> in the wild. IMHO, of course.
>
> -Andrew
>


Reply | Threaded
Open this post in threaded view
|

Re: Perl, Python, Ruby, PHP, C, C++, Lua, tcl, javascript and Java comparison

Tim Mensch
On 7/16/2013 1:51 PM, NU-g.lister wrote:
> I personally use this page to make some vague abstract conclusions about what is a good language to use:
> http://benchmarksgame.alioth.debian.org/
>
> note that it explicitly says a  benchmarksgame and there are no hard headed opinions to weed through.

Look back at the history of the owner of that page in this group, and
the controversy around him removing LuaJIT from the game, and say that
again about "no hard headed opinions" with a straight face.

Tim


Reply | Threaded
Open this post in threaded view
|

Re: Perl, Python, Ruby, PHP, C, C++, Lua, tcl, javascript and Java comparison

g.lister

----- Original message -----

> On 7/16/2013 1:51 PM, NU-g.lister wrote:
> > I personally use this page to make some vague abstract conclusions
> > about what is a good language to use:
> > http://benchmarksgame.alioth.debian.org/
> >
> > note that it explicitly says a   benchmarksgame and there are no hard
> > headed opinions to weed through.
>
> Look back at the history of the owner of that page in this group, and
> the controversy around him removing LuaJIT from the game, and say that
> again about "no hard headed opinions" with a straight face.
>
> Tim
>

:) I didn't know that thanks for pointing it out. i guess the owner can pick but the variety is still interesting and the tone is not preachy to me. How did LuaJIT compare cannot find an archive chart?

I took the link from a Lua book (Wrox) and frankly found plain Lua _pretty interesting_. Plus I did mention that I will not look at anything seriously until it is for something I need to use or implement and i test against my requirements/solution at that time. Language to me are like spoken languages do you want a pizza recepie in Italian or French ... no I really want pizza :).

I saw on the LuaJIT website some performance data so if I am someone looking at getting the most out of Lua code i will consider it ;) without caring whether it is on there or not.

Reply | Threaded
Open this post in threaded view
|

Re: Perl, Python, Ruby, PHP, C, C++, Lua, tcl, javascript and Java comparison

Tim Mensch
On 7/16/2013 10:38 PM, NU-g.lister wrote:
> :) I didn't know that thanks for pointing it out. i guess the owner
> can pick but the variety is still interesting and the tone is not
> preachy to me. How did LuaJIT compare cannot find an archive chart?

LuaJIT benchmarked REALLY well, even against many compiled languages. It
was really the best dynamic language, hands down, and even gave Java a
run for its money. It even beat C in one benchmark, IIRC, though I think
it was the Lua pattern matcher (written in C) that gave it the edge over
the C code written for that test. That combined with (generally) small
code size and (generally) low memory usage meant that it really stood out.

The complaint was that people, for better or worse, use the benchmark
game (which ranks very high in Google searches for many language
benchmarks) to choose between languages, and by removing it didn't show
the full potential of Lua any more. If you've seen the LuaJIT
performance data on luajit.org, then you know that LuaJIT exists and
that it's really fast. But to someone unfamiliar with Lua, it doesn't
stand out as over-the-top awesome any more.

Tim

12