OOLua version 1.2.2 released

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

OOLua version 1.2.2 released

liam mail
OOLua version 1.2.2 is a bug fix, has optimisations over 1.2.1 and now
uses premake 4.2. The use of the latest premake means that xcode is
now supported.

See http://code.google.com/p/oolua/source/browse/trunk/profile/profile.log for current speed comparisons with swig 1.3.40 and Luabind 9.0
Download :http://code.google.com/p/oolua/downloads/list
Please see readme.txt in the download for build instructions or
consult the wiki http://code.google.com/p/oolua/wiki/Building

Change list
*Converted premake scripts to premake4
*optimised checking of a type against a requested type via casting and storing a function taking the parameters to lua_Cfunction which requires less usage of the Lua stack.
*moved Luna licence to a more suitable section of code
*removed dead code from oolua_storage.h
*removed dead code from class_from_stack.h
*added setting a user data's none const name whilst changing it's constant status in set_type_top_to_none_const to prevent potentail errors. As the name and none constant name use a pointer compare when checking for constant status.
*added xcode support, tests required linking gtest and gmock normally i.e. -lgmock instead of gmock specified way gmock config - *
*added xcode test unit bash build script.
*build logs directed to there own directory
*added new test project "tests_may_fail" for issue 7 (http://code.google.com/p/oolua/issues/detail?id=7)
this fails soft on an error i.e. returns 0 from main not 1. moved effected tests (pushing and pulling signed int and signed long)to the new test project.
*changed bash build scripts to run the tests_may_fail swell as unit.tests
*added comment in lua_includes.h for mingw users
*added a readme with details of library as many download locations are now available
* fixed passing a c style string to a member function bug as reported by airbash http://code.google.com/p/oolua/issues/detail?id=8
* fixed a corresponding bug of a member function which returns a c style string.
* added define in lua_includes.h to support Lua 5.2 and 5.1.4 simultaneously
* fixed typo in xcode_premake4.sh
* adjusted premake recursive file inclusion to offset premake4.2 problem
* renamed gnu build script to gnu_tests.sh to reflect what it is doing
* added gnu_build.sh for building a local install
* added xcode_build.sh for building a local install
* added codeblocks_build.sh for building a local install
* renamed codeblocks_premake4.sh to codeblocks_tests.sh to reflect what it is doing
* added dl to link flags for unit tests in linux builds
* added vs2008x86_build.bat for building a local install
* renamed vs2008x86_premake4.bat to vs2008x86_tests.bat to reflect what it is doing
* added codeblocks_build.bat for building a local install
* renamed codeblocks_premake4.bat to codeblocks_tests.bat to reflect what it is doing

Thanks
Liam
Reply | Threaded
Open this post in threaded view
|

Re: OOLua version 1.2.2 released

Daniel Wallin
liam mail wrote:
[...]
> See http://code.google.com/p/oolua/source/browse/trunk/profile/profile.logfor
> current speed comparisons with swig 1.3.40
> and Luabind 9.0
              ^^^ that's luabind 0.9.

Since you are making such a fuss over this I'm going to comment on your
benchmark.

First, it isn't very clear what you are trying to measure. For instance,
the first benchmark - "access" - is simply calling two member functions.
Why is it called "access"? "virtual function" and "pure virtual
function" measure the exact same thing. Why?

But more importantly, *all* of your benchmarks are mixing in the cost of
fetching a member from an object. This is the primary reason your code
appears to perform better. Rewriting:

  for ...
    x:f()
  end

as

  local f = x.f
  for ...
    f(x)
  end

will give a more meaningful benchmark, as it actually measures the cost
of the call, rather than the lookup.

The reason lookups in OOLua is so fast is that __index doesn't call into
C++. Both Swig and luabind does, because they both support "properties".

Further, there is essentially no error checking. For primitive types,
the error check consists of an assert(), so in the release build there
is no checking at all. For both Swig and luabind, calling a function
with the wrong parameter type will give a meaningful error, and at least
for primitive types there is no way of turning this off. For luabind at
least, this is by design.

OOLua also doesn't verify that the userdata it gets from Lua is actually
a OOLua-wrapper. But to be fair, neither does Swig. I guess not doing
this might be OK if you can completely trust the Lua code that you run.
luabind 0.9 has no way of turning this check off, but my working copy
does (it's a one line change).

If you make the lookup change as I outlined above to your benchmarks,
and make the error checking of userdata in luabind equal to that of
OOLua and Swig:

diff --git a/src/object_rep.cpp b/src/object_rep.cpp
index 6977bee..75c32bb 100755
--- a/src/object_rep.cpp
+++ b/src/object_rep.cpp
@@ -244,6 +244,8 @@ namespace luabind { namespace detail
     {
         object_rep* result = static_cast<object_rep*>(lua_touserdata(L,
index));

+        return result;
+
         if (!result || !lua_getmetatable(L, index))
             return 0;

I think you will get very different results. On my machine, this makes
OOLua slower than both luabind and Swig on every one of your benchmarks.

--
Daniel Wallin
BoostPro Computing
http://www.boostpro.com

Reply | Threaded
Open this post in threaded view
|

Re: OOLua version 1.2.2 released

liam mail


2010/1/12 Daniel Wallin <[hidden email]>
Daniel thank for the comments yet some of your claims have flaws.
liam mail wrote:
[...]
> See http://code.google.com/p/oolua/source/browse/trunk/profile/profile.logfor
> current speed comparisons with swig 1.3.40
> and Luabind 9.0
             ^^^ that's luabind 0.9.
 
I am sorry for this typo it was late when I wrote the email, yet I assume there will be no confusion on which version of luabind is actually being used.
 
Since you are making such a fuss over this I'm going to comment on your
benchmark.

Well OOLua was designed with speed as a consideration. I did not realise that a single link was making a fuss yet showing off the it is quicker than other libraries tested.

First, it isn't very clear what you are trying to measure. For instance,
the first benchmark - "access" - is simply calling two member functions.
Why is it called "access"?

The first speed tests where against the hand written code from Game Programming Gems 6, if you look the code used and the test name is surprisingly similar.
local N = 10
local ave = 0

for i = 0, N do
    local h = Hero.Create(i)

    local t0 = os.clock()
    for i=1,1000000 do
        h:SetEnergy(i)
        if h:GetEnergy()~=i then
            error("failed");
        end
    end
    local dt = os.clock()-t0

    if i~=0 then
     ave = ave + dt
     print(h:GetName(),dt)
    end

end

print('access (average elapsed time):',ave/N)

 
"virtual function" and "pure virtual
function" measure the exact same thing. Why?

But more importantly, *all* of your benchmarks are mixing in the cost of
fetching a member from an object. This is the primary reason your code
appears to perform better.

The reason is that normal usage (from my experience) is not to cache the function as it is an object orientated binding and given the fact that OOLua was designed not to have runtime overhead of function lookup (which I comment about in the following link and at other locations http://code.google.com/p/oolua/issues/detail?id=4&can=1)I feel it is correct to test it like so. I could add the test you suggest, yet I am actually waiting for yourself to make the tests you have claimed in the past that luabind 0.8 was on par with swig. To my knowledge you have not made these public yet.


The reason lookups in OOLua is so fast is that __index doesn't call into
C++. Both Swig and luabind does, because they both support "properties".

I have never claimed that OOLua provides the same functionality as other binding libraries and have suggested that when someone is asking for a specifc piece of functionality that Luabind maybe a better alternative. After saying that I am a little unsure what you define as a property? Do you mean C++ side public member variables or Lua side? OOLua does not provide a method to store properties Lua side per instance yet does provide public member variable access via get and set methods, again implemented as function to not incur runtime lookup.

Further, there is essentially no error checking. For primitive types,
the error check consists of an assert(), so in the release build there
is no checking at all. For both Swig and luabind, calling a function
with the wrong parameter type will give a meaningful error, and at least
for primitive types there is no way of turning this off. For luabind at
least, this is by design.

This is true. OOLua assumes that you are passing the correct primitive types to functions which you call and there is only a check to see if the data is of the types which Lua provides using lua_isnumber et al in debug mode. Yet how do you check if an input is a char or int?

OOLua also doesn't verify that the userdata it gets from Lua is actually
a OOLua-wrapper. But to be fair, neither does Swig. I guess not doing
this might be OK if you can completely trust the Lua code that you run.
luabind 0.9 has no way of turning this check off, but my working copy
does (it's a one line change).

Actually it does. It first checks to see if the current type is the type requested and then travels the hierarchy to check if any of the base classes match the requested type.
 
If you make the lookup change as I outlined above to your benchmarks,
and make the error checking of userdata in luabind equal to that of
OOLua and Swig:

As stated I could make these changes yet am waiting for your test cases and it is valid to check the runtime lookup as it is part of the functionality provided.

I think you will get very different results. On my machine, this makes
OOLua slower than both luabind and Swig on every one of your benchmarks.

Interesting I will check this.
--
Daniel Wallin
BoostPro Computing
http://www.boostpro.com

Thanks
Liam
Reply | Threaded
Open this post in threaded view
|

Re: OOLua version 1.2.2 released

liam mail
Sorry looking again at your posted code
@@ -244,6 +244,8 @@ namespace luabind { namespace detail
    {
        object_rep* result = static_cast<object_rep*>(lua_
touserdata(L,
index));

+        return result;
+
        if (!result || !lua_getmetatable(L, index))
            return 0;

OOLua is meant to replace usage of Lua C API calls in client code, yet it still enables you to use the C API. The metatable check you have was present in earlier releases yet profiling showed that it was a costly operation and now it is only preformed if the type is not the requested type, this does rely on the index actually being a Lua_ud. The code you posted only checks that the userdata has a metatable not that it is an actual object_rep from what I see.

Just as you have a method to turn this off locally I may introduce a define to turn it on if a user so requires, in a future release.

Liam

Reply | Threaded
Open this post in threaded view
|

Re: OOLua version 1.2.2 released

Daniel Wallin
In reply to this post by liam mail
liam mail wrote:

> 2010/1/12 Daniel Wallin <[hidden email]>
> Daniel thank for the comments yet some of your claims have flaws.
>
>> liam mail wrote:
>> [...]
>
>> Since you are making such a fuss over this I'm going to comment on your
>> benchmark.
>>
> Well OOLua was designed with speed as a consideration. I did not realise
> that a single link was making a fuss yet showing off the it is quicker than
> other libraries tested.

This is hardly the first time you post links to the benchmark results.
It bugs me because your results makes the other binding libraries come
off as really slow, when in fact the only part that is faster in OOLua
is the trivial table lookup.

It's going to be difficult for someone reading your numbers to
understand what the real differences are.

>> First, it isn't very clear what you are trying to measure. For instance,
>> the first benchmark - "access" - is simply calling two member functions.
>> Why is it called "access"?
>
>
> The first speed tests where against the hand written code from Game
> Programming Gems 6, if you look the code used and the test name is
> surprisingly similar.
[...]

I don't really care where the code came from, I'm saying it's named badly.

>> But more importantly, *all* of your benchmarks are mixing in the cost of
>> fetching a member from an object. This is the primary reason your code
>> appears to perform better.
>
> The reason is that normal usage (from my experience) is not to cache the
> function as it is an object orientated binding and given the fact that OOLua
> was designed not to have runtime overhead of function lookup (which I
> comment about in the following link and at other locations
> http://code.google.com/p/oolua/issues/detail?id=4&can=1)I feel it is correct
> to test it like so.

If you all the same member function repeatedly and it shows up on the
profile, you cache the lookup. This is true in raw Lua, when using OOLua
and when using any other binding library.

My point is that the results are misleading because they don't mention
this. You could very easily have another benchmark that measures the
time it takes to lookup a member.

> I could add the test you suggest, yet I am actually
> waiting for yourself to make the tests you have claimed in the past that
> luabind 0.8 was on par with swig. To my knowledge you have not made these
> public yet.

No, I haven't. I also haven't published any numbers. I mentioned luabind
0.9 in general being close to Swig in speed because someone posted a
benchmark on the luabind mailing list comparing some especially weak
areas of luabind.

I don't have time to make my suite work on anything but my machine. It
tests a few more binding libraries, and it generates the binding code
for some of them in the build step so there's some work to make it
usable outside my environment.

Either way, it doesn't really matter, because your code shows the same
results.

>> The reason lookups in OOLua is so fast is that __index doesn't call into
>> C++. Both Swig and luabind does, because they both support "properties".
>>
> I have never claimed that OOLua provides the same functionality as other
> binding libraries and have suggested that when someone is asking for a
> specifc piece of functionality that Luabind maybe a better alternative.
> After saying that I am a little unsure what you define as a property? Do you
> mean C++ side public member variables or Lua side?

"properties" are functions that are invoked when accessing something
that looks like a data member:

  x = X()
  x.value = 1 -- calls a set-function
  y = x.value -- calls a get-function

Supporting this is the reason both luabind and Swig (and tolua, ..)
roundtrips into C/C++ before making the real table lookup.

>> Further, there is essentially no error checking. For primitive types,
>> the error check consists of an assert(), so in the release build there
>> is no checking at all. For both Swig and luabind, calling a function
>> with the wrong parameter type will give a meaningful error, and at least
>> for primitive types there is no way of turning this off. For luabind at
>> least, this is by design.
>>
> This is true. OOLua assumes that you are passing the correct primitive
> types to functions which you call and there is only a check to see if the
> data is of the types which Lua provides using lua_isnumber et al in debug
> mode. Yet how do you check if an input is a char or int?

You don't, because they are represented by the same type on the Lua
side. Isn't that obvious?

>> OOLua also doesn't verify that the userdata it gets from Lua is actually
>> a OOLua-wrapper. But to be fair, neither does Swig. I guess not doing
>> this might be OK if you can completely trust the Lua code that you run.
>> luabind 0.9 has no way of turning this check off, but my working copy
>> does (it's a one line change).
>>
> Actually it does. It first checks to see if the current type is the type
> requested and then travels the *hierarchy* to check if any of the base
> classes match the requested type.

You missed my point. Consider:

  x = create_arbitrary_userdata()
  instance.f(x)

luabind verifies that the userdata passed in is actually created by
luabind. OOLua and Swig does not.

--
Daniel Wallin
BoostPro Computing
http://www.boostpro.com

Reply | Threaded
Open this post in threaded view
|

Re: OOLua version 1.2.2 released

Daniel Wallin
In reply to this post by liam mail
liam mail wrote:

> Sorry looking again at your posted code
> @@ -244,6 +244,8 @@ namespace luabind { namespace detail
>     {
>         object_rep* result = static_cast<object_rep*>(lua_
> touserdata(L,
> index));
>
> +        return result;
> +
>         if (!result || !lua_getmetatable(L, index))
>             return 0;
>
[...]
> The code you posted only checks
> that the userdata has a metatable not that it is an actual object_rep from
> what I see.

If you actually looked at the code rather than the diff context you
might have noticed that it *does* check that.

> Just as you have a method to turn this off locally I may introduce a define
> to turn it on if a user so requires, in a future release.

Yes, and that will make the benchmark results different.

--
Daniel Wallin
BoostPro Computing
http://www.boostpro.com

Reply | Threaded
Open this post in threaded view
|

Re: OOLua version 1.2.2 released

liam mail
You don't, because they are represented by the same type on the Lua
side. Isn't that obvious?

Yes that was the point.

2010/1/12 Daniel Wallin <[hidden email]>
liam mail wrote:
> Sorry looking again at your posted code
> @@ -244,6 +244,8 @@ namespace luabind { namespace detail
>     {
>         object_rep* result = static_cast<object_rep*>(lua_
> touserdata(L,
> index));
>
> +        return result;
> +
>         if (!result || !lua_getmetatable(L, index))
>             return 0;
>
[...]
> The code you posted only checks
> that the userdata has a metatable not that it is an actual object_rep from
> what I see.

If you actually looked at the code rather than the diff context you
might have noticed that it *does* check that.

> Just as you have a method to turn this off locally I may introduce a define
> to turn it on if a user so requires, in a future release.

Yes, and that will make the benchmark results different.

--
Daniel Wallin
BoostPro Computing
http://www.boostpro.com

Following is a copy of the message I have posted to other places where the release data was posted.
 
I apologise for including a link to the profile results as some people think it is just not fair and you will find the numbers confusing. I would like to add that OOLua may not be what you are looking for, but also Luabind may not!
 
The Luabind author is complaining because OOLua and Swig draw a line between checking what you pass to a function which requires data created by the libraries and a user passing completely incorrect data.

In a future version I will add the ability for a user to define if a check should be made on every instance passed to it that it actually created the data, this will of course impose a penalty for using the library correctly. On the other hand if it is intended to allow third parties to write scripts this could be seen as the correct action to take.
Liam