The real problem with vectors and Lua

classic Classic list List threaded Threaded
30 messages Options
12
Reply | Threaded
Open this post in threaded view
|

The real problem with vectors and Lua

Dylan Cuthbert
Ok, I've worked out what the *real* problem with implementing vectors as
simply tables is.  :-)

No, its not speed or memory! (although these need work on later)

The problem is that tables are passed by reference, meaning I get a lot of
variables all "pointing" to the same value, which I then access by element
(x,y,z, or w) affecting all those variables.  This doesn't happen with ints
or floats, because when you modify the value it *replaces* it with the new
one.

ie.

position = { x=5, y=10, z=20 }
vector = position
position.x = 10        -- vector also "changes"!!   arghhh

and no.. writing vector = vector.copy( position) or some such is *not* an
option. ;-)

What I really need is an int that has semi-table like attributes.  I don't
mind object's referencing the same copy as long as when I do vector.x = 10 I
want the whole of "vector" to be re-written with a copy.

This may not seem very flexible, but neither are ints and floats really are
they?  That's why I suggest a vector should be an atomic(?) type.

I'll start looking at the Lua source today to see how this could be done but
any hints or help would be great.

Regards

---------------------------------
Q-Games, Dylan Cuthbert.
http://www.q-games.com



Reply | Threaded
Open this post in threaded view
|

Re: The real problem with vectors and Lua

RLak-2
> The problem is that tables are passed by reference, meaning I get a lot 
of
> variables all "pointing" to the same value, which I then access by 
element
> (x,y,z, or w) affecting all those variables.  This doesn't happen with 
ints
> or floats, because when you modify the value it *replaces* it with the 
new
> one.

In effect, but that is a little imprecise. Numbers don't have "elements"; 
they are atomic wholes. When you say

  m = n

it replaces m's value with the value of n, whatever that was. Eg:

  m = {x = 5, y = 10, z = 20}

  n = "Hello, world"

  m = n

Now m has been well and truly replaced.

n (and m) are now strings, which in Lua are immutable and atomic. For this 
reason, Lua does not let you say:

  n[6] = ":"

You cannot change a "component" of an atomic object. So if vectors were, 
like Python tuples, immutable sequences, you wouldn't be able to say:

  position.x = 10

either. You would have to say something like:

  position = replace(position, "x", 10)

but I have a hunch that you don't actually want to do that.

Would you settle for this?

  position = position ^ {x = 10}

where ^ is read as "replacing"? That would be easy enough to implement:

function meta:__pow(other)
  local rv = {}
  for k, v in self do
    rv[k] = other[k] or v
  end
  return rv
end

or, less efficient but more general:

function meta:__pow(other)
  local rv = {}
  for k, v in self  do rv[k] = v end
  for k, v in other do rv[k] = v end
  return rv
end

Note: if you replace __pow above with __call, you could write it as:

  position = position{x = 10}

Another possibility would be to define an object method, like:

  position = position.with {x = 10}

> position = { x=5, y=10, z=20 }
> vector = position
> position.x = 10        -- vector also "changes"!!   arghhh

Certainly. You have said that vector and position are the same object. If 
you wanted them to be different objects, you would have had to have said:

  vector = copy(position)

Of course, as we all know, Lua has no copy primitive :)

But clearly the two operations are semantically distinct *for mutable 
objects*.

> and no.. writing vector = vector.copy( position) or some such is *not* 
an
> option. ;-)

Why not? You don't want to say what you mean? :) What if you *meant* for 
them to be the same object?

R.


Reply | Threaded
Open this post in threaded view
|

Re: The real problem with vectors and Lua

Adam D. Moss
In reply to this post by Dylan Cuthbert
Dylan Cuthbert wrote:
The problem is that tables are passed by reference, meaning I get a lot of
variables all "pointing" to the same value, which I then access by element
(x,y,z, or w) affecting all those variables.  This doesn't happen with ints
or floats, because when you modify the value it *replaces* it with the new
one.

ie.

position = { x=5, y=10, z=20 }
vector = position
position.x = 10        -- vector also "changes"!!   arghhh

and no.. writing vector = vector.copy( position) or some such is *not* an
option. ;-)

I feel your pain.  It's not so bad; in my own vector class
(using a home-grown object system in lua) I have to do
vectorb = vectora:copy() quite a lot though... and if I
forget, all hell can /silently/ breaks lose. :)

I often wish that there were a 'copy' metamethod so that
a table on the RHS of an assignment can return a reference
to a copy instead of a reference to itself in these instances.

It just occurred to me that a syntax that might be nice would
be something like vectora = *vectorb -- it's a bit C-like.  But
then I thought that the same thing could (unintuitively) be
achieved right now by overloading the unary-minus metamethod:

vectora = -vectorb    -- take a copy

But of course, this is seven flavours of wrongness-looking,
especially for a vector class!  (-vec already has a meaning.)

If Lua treated the expression '-tab' as __sub(nil,tab)
then unary-minus could disappear, as the user's __sub
metamethod could detect and handle it as a special case (and e.g.
if '*tab' was a valid expression in Lua's syntax, the same
thing could apply for the other binary operations, allowing,
say, the now-pseudo-unary multiply to return a reference to
a copy of the table).

Regards,
--Adam
--
Adam D. Moss   . ,,^^   [hidden email]   http://www.foxbox.org/   co:3
"When I was young, I would break down and cry whenever mom made me
throw an empty toilet paper tube in the trash can. I just imagined
it sitting at the dump, all cardboard and sad." -- T.Farnon


Reply | Threaded
Open this post in threaded view
|

RE: The real problem with vectors and Lua

Nick Trout
In reply to this post by Dylan Cuthbert

> From: Dylan Cuthbert 
> The problem is that tables are passed by reference, meaning I get a
lot of
> variables all "pointing" to the same value, which I then access by
element
> (x,y,z, or w) affecting all those variables.  This doesn't happen with
> ints
> or floats, because when you modify the value it *replaces* it with the
new
> one.
> 
> ie.
> 
> position = { x=5, y=10, z=20 }
> vector = position
> position.x = 10        -- vector also "changes"!!   arghhh
> 
> and no.. writing vector = vector.copy( position) or some such is *not*
an
> option. ;-)


Yes, they're references. I take your point that vectors are pretty much
CPU integral types now but this is an issue about general assignment of
Lua objects - which for everything but numbers are references. Don't
other interpreted languages behave in the same way (e.g. Python, Java,
Perl?), i.e. everything is a reference. It wasn't so long ago you
wouldn't have dreamed of passing vectors around by value.

Isn't this just about semantics?

	position = { x=5, y=10, z=20 }
	vector.copies(position)
	position.x = 10        -- position is new vector

or

	position = { x=5, y=10, z=20 }
	vector = Vector(position)  -- constructed
	position.x = 10        -- position is new vector

What happens when you don't want to make a new vector and you want a
reference?


Regards,
Nick



Reply | Threaded
Open this post in threaded view
|

Re: The real problem with vectors and Lua

Dylan Cuthbert
In reply to this post by RLak-2
<[hidden email]> wrote in message
[hidden email]">news:OF6EBFA4E4.2822513F-ON05256D52.005B0EE5@......
>
> You cannot change a "component" of an atomic object. So if vectors were,
> like Python tuples, immutable sequences, you wouldn't be able to say:
>
>   position.x = 10
>
> either. You would have to say something like:
>
>   position = replace(position, "x", 10)
>
> but I have a hunch that you don't actually want to do that.
>
> Would you settle for this?
>
>   position = position ^ {x = 10}

I don't mind position:set_x( 10 ), but this is getting away from my point,
my problem is the fact that these vectors are classed as *tables* in Lua.  I
need them classed as atomic objects like strings.  There is another mail in
this thread from Adam D Moss and he refers to "all hell breaking loose",
that is a serious problem for a language that is being bandied about as a
scripting environment.

> > and no.. writing vector = vector.copy( position) or some such is *not*
> an
> > option. ;-)
>
> Why not? You don't want to say what you mean? :) What if you *meant* for
> them to be the same object?

There is *never* any reason to want them to be the same object.  Vectors
need to be treated in the same way as "numbers" and strings.  They are
mathematical atomic types and should be treated as such.  (in games at
least)

If I wanted to pass a vector into a function as a reference (for it to be
altered by the function) I would encapsulate it in a table as I would a
number or a string.  Although the best way is to simply return the changed
value.  position = dot_product( position ) for example.

If I could disable the default taking of a reference with an error I would
be happier - ie.

vector = position -- generates some kind of error
vector = Vector( position )  -- takes a copy and is safe

But of course, table copying anything with vectors in would be broken.
(unless I hack together my own table copy that checks each entry :-( )

Regards

---------------------------------
Q-Games, Dylan Cuthbert.
http://www.q-games.com



Reply | Threaded
Open this post in threaded view
|

Re: The real problem with vectors and Lua

Dylan Cuthbert
In reply to this post by Adam D. Moss
> I feel your pain.  It's not so bad; in my own vector class
> (using a home-grown object system in lua) I have to do
> vectorb = vectora:copy() quite a lot though... and if I
> forget, all hell can /silently/ breaks lose. :)

This is exactly what scares the shit out of me quite frankly.

I need a script language to be safe and least likely to fall prey to bugs.
Lua doesn't seem to fulfill these roles with its "leave everything for the
programmer to work out" approach, which is a shame because the fundamental
ideas or basic theories behind Lua are quite smart, its just that somewhere
along the way its been driven down the wrong path. (or maybe it simply
hasn't begun to go down those paths yet)

As I mentioned in my other replies, a step forward might be the ability to
generate an error if someone simply takes a reference rather than taking a
copy as they should.



Reply | Threaded
Open this post in threaded view
|

RE: The real problem with vectors and Lua

Martin Slater
In reply to this post by Dylan Cuthbert
In games yes, but lua was never (AFAIK) designed to work just for games.
You seem to be approaching it as if someone has gone out of their way to
design a scripting language ideally suited for games and are aghast that
they haven't incorporated everything solely to suit us (yep I work in
games too).

Some of this seems to stem from the completely inaccurate view you
posted earlier that virtually all modern applications use 3d in some
major respect. This is obviously just complete rubbish.

If you are having so much trouble in leveraging lua to do what you want
it would seem you are trying to use the wrong tool for the job and
probably need to look at another solution

Some of you other comments just come across as down right rude to the
people who have spent their time working on this for their own domain
and are good enough to support other people using it.

As ever just my 2c

Martin


> 
> There is *never* any reason to want them to be the same object.
Vectors
> need to be treated in the same way as "numbers" and strings.  They are
> mathematical atomic types and should be treated as such.  (in games at
> least)



Reply | Threaded
Open this post in threaded view
|

Re: The real problem with vectors and Lua

Tuomo Valkonen-2
In reply to this post by Dylan Cuthbert
On Mon, Jun 30, 2003 at 11:02:04AM +0900, Dylan Cuthbert wrote:
> I don't mind position:set_x( 10 ), but this is getting away from my point,
> my problem is the fact that these vectors are classed as *tables* in Lua.  I
> need them classed as atomic objects like strings.  

So you just create a vector userdata (lua_newuserdata) that has no 
__newindex metamethod and now your vectors are immutable objects like
strings (unless you provide other methods to modify them). If you
want to change a coordinate, do something like 

foo = change_x(foo, 10).

Of course this copying will generate some garbage, but IIRC you mentioned
that most of the time you will be changing whole vectors anyway.

> There is *never* any reason to want them to be the same object.

Not all vectors are three-dimensional. Should longer vectors be copied
as well when assigned? You could do copy-on-write, but without reference
counting this might be inefficient and as we all know, Lua doesn't use 
reference counting. Do you want Lua to be limited to 3/4D vectors?

-- 
Tuomo

Reply | Threaded
Open this post in threaded view
|

Re: The real problem with vectors and Lua

Dylan Cuthbert
In reply to this post by Martin Slater
Hi there,

I seem to have drawn some fire there, but my intentions are honourable ;-)

If my comments seem rude at all, it isn't intentional, I don't think being
critical (in a positive constructive way) is equal to being rude, and I've
tried to post all my criticisms in a positive way.  On top of that I *am*
trying to find a solution within the framework of the existing Lua design.

The problem isn't really to do with 3d, its a conceptual problem.  What I am
aiming for is a kind of "tuple" implementation, where an atomic type is made
of parts and those parts aren't independent of one another, they work as a
"whole".  It's a missing area between the user-definable tables of Lua and
the non-user-definable atomic types. (number/string etc)

So don't say take it or leave it - I'm willing to put in time and effort and
work with people to push the language further if possible.  Right now I'm
simply rounding up people's ideas on the concept and working out where to go
from now.

Regards

---------------------------------
Q-Games, Dylan Cuthbert.
http://www.q-games.com


"Martin Slater" <[hidden email]> wrote in message
001901c33ec0$c0c477d0$4201000a@CAOZ">news:001901c33ec0$c0c477d0$4201000a@CAOZ...
> In games yes, but lua was never (AFAIK) designed to work just for games.
> You seem to be approaching it as if someone has gone out of their way to
> design a scripting language ideally suited for games and are aghast that
> they haven't incorporated everything solely to suit us (yep I work in
> games too).
>
> Some of this seems to stem from the completely inaccurate view you
> posted earlier that virtually all modern applications use 3d in some
> major respect. This is obviously just complete rubbish.
>
> If you are having so much trouble in leveraging lua to do what you want
> it would seem you are trying to use the wrong tool for the job and
> probably need to look at another solution
>
> Some of you other comments just come across as down right rude to the
> people who have spent their time working on this for their own domain
> and are good enough to support other people using it.
>
> As ever just my 2c
>
> Martin
>
>
> >
> > There is *never* any reason to want them to be the same object.
> Vectors
> > need to be treated in the same way as "numbers" and strings.  They are
> > mathematical atomic types and should be treated as such.  (in games at
> > least)
>
>
>



Reply | Threaded
Open this post in threaded view
|

Re: The real problem with vectors and Lua (Atomic Tuples)

Dylan Cuthbert
In reply to this post by Tuomo Valkonen-2
Hi there, thanks for the input, I'll do some exploring in that direction
(the userdata metatable overriding)


> Not all vectors are three-dimensional. Should longer vectors be copied
> as well when assigned? You could do copy-on-write, but without reference
> counting this might be inefficient and as we all know, Lua doesn't use
> reference counting. Do you want Lua to be limited to 3/4D vectors?

I was thinking along the lines of copy-on-write.  No the problem I'm having,
generally speaking, isn't 3d/4d related at all really, I'd like it to be as
general as possible.  Although huge "tuples" would incur overhead,  I don't
see these "tuples" being expanded upon or modified (like tables are) after
they've been created, so hash-collating is possible for vectors with the
same values. (as it is for strings and numbers)

Something like   (this is totally off the top of my head.. don't rip me to
shreds)

position = << 10, 20, 30 >>    -- create a "tuple"
position.another_value = 10    -- ERROR.. position isn't a table and is a
"const" in effect

position<<x>> = 30        -- creates a new "const" vector with x as 30 and
the rest from position
position<<x,y>> = 50, 70  -- writes to x, y, and takes z from position,
creating a new vector


The main thing to spot, is that apart from the << >> syntax (which I
spuriously made up) position is acting exactly like a regular number or
string containing variable.  The x, y, z are simply "syntactic sugar"; named
indexing for 0, 1, 2 and isn't totally necessary.

Regards

---------------------------------
Q-Games, Dylan Cuthbert.
http://www.q-games.com

"Tuomo Valkonen" <[hidden email]> wrote in message
[hidden email]">news:20030630053405.GA9856@......
> On Mon, Jun 30, 2003 at 11:02:04AM +0900, Dylan Cuthbert wrote:
> > I don't mind position:set_x( 10 ), but this is getting away from my
point,
> > my problem is the fact that these vectors are classed as *tables* in
Lua.  I
> > need them classed as atomic objects like strings.
>
> So you just create a vector userdata (lua_newuserdata) that has no
> __newindex metamethod and now your vectors are immutable objects like
> strings (unless you provide other methods to modify them). If you
> want to change a coordinate, do something like
>
> foo = change_x(foo, 10).
>
> Of course this copying will generate some garbage, but IIRC you mentioned
> that most of the time you will be changing whole vectors anyway.
>
> > There is *never* any reason to want them to be the same object.
>
> Not all vectors are three-dimensional. Should longer vectors be copied
> as well when assigned? You could do copy-on-write, but without reference
> counting this might be inefficient and as we all know, Lua doesn't use
> reference counting. Do you want Lua to be limited to 3/4D vectors?
>
> -- 
> Tuomo
>



Reply | Threaded
Open this post in threaded view
|

Re: The real problem with vectors and Lua

Enno Rehling-2
In reply to this post by Dylan Cuthbert
Dylan Cuthbert wrote:

The problem isn't really to do with 3d, its a conceptual problem.  What I am
aiming for is a kind of "tuple" implementation, where an atomic type is made
of parts and those parts aren't independent of one another, they work as a
"whole".

IIRC, then C99 has a built-in 'complex' datatype. Is that something like what you're looking for, only for something with bigger tuples? I can't say I've used it or even seen how it looks (how do you access the components in the complex number?), is anyone knowledgeable about this? And how it's implemented?

Enno
--
<xterm> The problem with America is stupidity. I'm not saying there should
     be a capital punishment for stupidity, but why don't we just take
     the safety labels off of everything and let the problem solve itself?



Reply | Threaded
Open this post in threaded view
|

Re: The real problem with vectors and Lua

Dylan Cuthbert
Browsing through the python documentation, I found that they have
implemented pretty much exactly what I am talking about:

http://www.python.org/doc/current/tut/node7.html#SECTION007300000000000000000

What's really nice is the "packing" and "unpacking", ie.
frog = 10, 20, 30    -- packs frog with 3 numbers
x, y, z = frog  -- unpacks frog into its 3 separate elements

These tuples are *immutable* sequences, ie. they are constants.

Lua would benefit greatly from this kind of feature.

---------------------------------
Q-Games, Dylan Cuthbert.
http://www.q-games.com

"Enno Rehling" <[hidden email]> wrote in message
bdopj9$eu4$[hidden email]">news:bdopj9$eu4$1@......
> Dylan Cuthbert wrote:
>
> > The problem isn't really to do with 3d, its a conceptual problem.  What
I am
> > aiming for is a kind of "tuple" implementation, where an atomic type is
made
> > of parts and those parts aren't independent of one another, they work as
a
> > "whole".
>
> IIRC, then C99 has a built-in 'complex' datatype. Is that something like
> what you're looking for, only for something with bigger tuples? I can't
say
> I've used it or even seen how it looks (how do you access the components
in
> the complex number?), is anyone knowledgeable about this? And how it's
> implemented?
>
> Enno
> -- 
> <xterm> The problem with America is stupidity. I'm not saying there should
>       be a capital punishment for stupidity, but why don't we just take
>       the safety labels off of everything and let the problem solve
itself?
>
>
>



Reply | Threaded
Open this post in threaded view
|

Re: The real problem with vectors and Lua

Tuomo Valkonen-2
On Mon, Jun 30, 2003 at 06:22:15PM +0900, Dylan Cuthbert wrote:
> What's really nice is the "packing" and "unpacking", ie.
> frog = 10, 20, 30    -- packs frog with 3 numbers
> x, y, z = frog  -- unpacks frog into its 3 separate elements
> 
> Lua would benefit greatly from this kind of feature.

frog={1,3,4}
x, y, z = unpack(frog)

But the elements can be modified. The following wrapper should hie
them:

function pack(...)
    return {unpack=function() return unpack(arg) end}
end

frog=pack(1,2,3)
x, y, z = frog.unpack()

You could also set a metatable with __newindex for the table constructed 
in 'pack' to forbid adding new elements to the table. Or use a userdata?
Or maybe a function?

function pack(...)
    return function(i)
        if not i then 
	    return unpack(arg)
	else
	    return arg[i]
	end
    end
end

frog=pack(1,2,3)
x, y, z=frog()
y=frog(2)

-- 
Tuomo

Reply | Threaded
Open this post in threaded view
|

Re: The real problem with vectors and Lua

Wim Couwenberg-2
In reply to this post by Dylan Cuthbert
Hi Dylan,

> Lua would benefit greatly from this kind of feature.

Well, I think that Tuomo pointed out that you can realise this by
introducing your own userdata without the __newindex metamethod.  You could
give it an unpack method such that

    x, y, z = my_3d_vector:unpack()

does what it suggests.  I think that this would be particularly suited for
your vector type.

On the other hand, I think that Python tuples can be emulated by Lua tables
(arrays) if you use a proxy.  Simple approach:

do
local meta = {
    __index = function(self, index)
            return self.as_table[index]
        end,
    __newindex = function()
            error "tuples are immutable!"
        end,
}

function tuple(...)
    local t = {as_table = arg}
    setmetatable(t, meta)
    return t
end
end

local tup = tuple(10, 20, 30)

print(tup[2])
local x, y, z = unpack(tup.as_table)
tup[2] = 25  -- error!


Note that in this simple approach you can still screw up beacuse the
as_table member is freely available.  A more involved approach would be to
hide the original table completely, leaving the proxy table empty.  Then the
__index metamethod gets just a little more complicated, because it should
also take care of an unpack method etc.

--
Wim



Reply | Threaded
Open this post in threaded view
|

Re: The real problem with vectors and Lua

RLak-2
In reply to this post by Dylan Cuthbert
Dylan Cuthbert escribió (hace tiempo):

> I don't mind position:set_x( 10 ), but this is getting away from my 
point,
> my problem is the fact that these vectors are classed as *tables* in 
Lua.  I
> need them classed as atomic objects like strings.  There is another mail 
in
> this thread from Adam D Moss and he refers to "all hell breaking loose",
> that is a serious problem for a language that is being bandied about as 
a
> scripting environment.

The problem is not that the vectors are classed as tables. As various 
people have pointed out, you can change the behaviour of a table however 
you want to.

The problem is that you want mutable objects and you *also* want to be 
able to mutate them. That's incompatible.

In other words, if you never actually did:

  vector.x = 10

But always did

  newvector = vector:set_x(10)

or some such (I proposed a few alternative syntaxes for your 
consideration), then the problem disappears.

In other words, if vector is an atomic, immutable object, then

  vector.x = 10

is not a meaningful construct. It would be the same as trying to say, for 
example:

  a = 3
  a.sign = '-'

Since Lua character strings are immutable objects, you cannot say this 
either:

  a = "Hello, world"
  a[6] = ":"

as you could in a language with mutable character strings.

If you define the interface for vectors correctly, then you can choose to 
implement them as tables (possibly protected for debugging) in Lua, or you 
can implement them in C. A good interface will let you switch at any 
moment.

<side-issue type="theoretical">

The issue of mutability/immutability is not simply an academic one. It 
affects the notion of object equality (there is an (in)famous paper by 
Henry Baker on the subject, well worth reading: "Equal Rights for 
Functional Objects" -- you can find it here 
<http://home.pipeline.com/~hbaker1/> in html and compressed postscript 
formats). Object equality is a particularly important concept when objects 
can be used as hash table keys. If, for example, strings were mutable, and 
you used a mutable string as a hash table key and then mutated (changed) 
it, "all hell would break loose." But on the other hand, you really want 
to be able to use strings as hash table keys, so Lua wisely implements 
them as immutable objects.

An issue for an implementor of atomic immutable vectors would then be 
whether or not to make it possible for them to be hash table keys. (This 
is an issue for bignums as well.) A naive implementation of an immutable 
object makes hashing difficult -- if you use, for example, the storage 
address as the hash, you will likely not be able to retrieve keys from the 
hash table. The solution Lua uses for strings is to intern the string 
(i.e. ensure that there is only one copy of every string in the system); 
that would be a possibility for tuples as well (or vectors or bignums) but 
it might prove to be inefficient in practice. Another possibility would be 
lazy interning, where the object was only interned when used in equality 
comparison (including hash table lookup). Yet another option is to insist 
that immutable object types come with a hash function which guarantees 
that "equal" objects will have the same hash. Many languages provide the 
last option, but in practice it is quite difficult to come up with correct 
hash functions and incorrect hash functions are generators of very obscure 
bugs. My personal preference, if I were thinking about a language from 
square one, would probably be lazy interning, although its implementation 
is non-trivial as well.

</side-issue>

Rici


Reply | Threaded
Open this post in threaded view
|

Re: The real problem with vectors and Lua

Mark Hamburg-4
In reply to this post by Dylan Cuthbert
This basically has to do with what is sometimes referred to as value
semantics v. object semantics and is an issue I've been worrying about to
some extent myself of late and again for a collection of 2-D geometry types
though I could also see it as relevant to complex numbers.

The distinction could be roughly summarized as being that for value types,
object identity doesn't matter. For objects, it does.

>From a practical standpoint, it means that "variables" hold values and for
value types we change the value held by the variable whereas for object
types we change the object referred to by the value of the variable.

The simple work around for this is to implement new value types as user data
-- though as noted in earlier discussions, doing this well also involves
worrying about how they behave as hash table keys. You then implement all
mutators as returning new objects.

One problem with this approach is that it is less "natural" than the value
semantics version. For example, a lot more people would probably be
comfortable writing:

    position.x = PinToGrid( position.x )

Than writing:

    position = postion:setX( PinToGrid( position.x ) )

Even without the direct assignment, you'd probably get lots of people
writing and then debugging:

    position:setX( PinToGrid( position.x ) )

That generates bugs that are probably easier to detect than the case of
using object-semantics for positions and having someone forget to do a copy,
but being the lesser of two evils doesn't make something good.

A second problem has to do with garbage. The straightforward implementations
of value semantics by having mutators return new objects can generate a fair
amount of garbage for temporary values. Generating garbage at a faster rate
increases the frequency with which one needs to perform garbage collection.
There are two potential fixes in this regard. One is some limited support
for reference counting so that temporaries get recycled quickly. The other
is something like a calculator construct where one could do things like:

    pointCalculator = point:makeCalculator();
    pointCalculator:subtractPoint( center );
    pointCalculator:scaleBy( 10 );
    pointCalculator:rotateBy( 2 );
    pointCalculator:addPoint( center );
    point = pointCalculator:getPoint();

This replaces 3 temporary objects with 1 temporary point calculator. One
could also implement these sort of semantics with a little less safety by
simply changing the metatable for the point:

    point = point:makeEditable();
        -- copies the point and changes the metatable
    point:subtractPoint( center );
    point:scaleBy( 10 );
    point:rotateBy( 2 );
    point:addPoint( center );
    point:freeze(); -- changes the metatable to remove mutation functions

Mark


Reply | Threaded
Open this post in threaded view
|

Re: The real problem with vectors and Lua (Atomic Tuples)

John Belmonte-2
In reply to this post by Dylan Cuthbert
Dylan wrote:
Something like   (this is totally off the top of my head.. don't rip me to
shreds)

position = << 10, 20, 30 >>    -- create a "tuple"
position.another_value = 10    -- ERROR.. position isn't a table and is a
"const" in effect

position<<x>> = 30        -- creates a new "const" vector with x as 30 and
the rest from position
position<<x,y>> = 50, 70  -- writes to x, y, and takes z from position,
creating a new vector

I haven't been following this thread too carefully, but userdata is what you want. It seems like you are on that path anyway.

Don't be afraid to let these vectors exist as either your native aligned format or a Lua array depending on what is convenient. You just need to be able to translate between the two. So I'd rewrite what you have above as:

position = Vector { 10, 20, 30 }        -- Vector_XYZ is assumed
position.another_value = 10             -- ERROR
position:Set(Vector_X { 30 })           -- you might also support Vector_X(30)
                                        --     for efficiency
position:Set(Vector_XY { 50, 70 })

-- other stuff
lua_array = position.ToArray()
lua_array[0] = 25.5                     -- assuming you break Lua convention and
                                        --     start your array at 0 :-(
position_b = Vector(lua_array)
position_c = position * position_b      -- your efficient native operation
point = Vector_XY(position_c)           -- extract XY as vector
z = Vector_Z(position_c):ToNumber()     -- extract Z as Lua number
z = position_c:ToArray()[2]             -- same thing
z = position_c:GetScalarZ()             -- same thing
point_b = Vector(point)			-- copy


If you haven't seen it, "Vector { 1, 2, 3 }" is table constructor syntax, equivalent to Vector({1, 2, 3}).

This isn't too far off from your C++ vector template stuff...

-John



--
http:// if   l .o  /


Reply | Threaded
Open this post in threaded view
|

Re: Vectors/Atomic Tuples/Garbage Collection

Dylan Cuthbert
> Don't be afraid to let these vectors exist as either your native aligned
format
> or a Lua array depending on what is convenient.  You just need to be able
to
> translate between the two.  So I'd rewrite what you have above as:
>
> position = Vector { 10, 20, 30 }        -- Vector_XYZ is assumed
> position.another_value = 10             -- ERROR
> position:Set(Vector_X { 30 })           -- you might also support
Vector_X(30)
>                                          --     for efficiency
> position:Set(Vector_XY { 50, 70 })

Ok, some questions

With heavy user data when I do position = vector, a reference is taken,
right, when I modify it a copy is taken (via those Vector calls).  How and
when is the memory freed?  If this user data is a C++ class do I still have
to use malloc/free with a positional new?
I can imagine there being a whole ton of temporary 12-16 byte vectors (not
including the overheads on the Lua side for methods and metatables) being
malloced, I mean an *absolutely friggin' huge* amount as the game progresses
frame to frame - these are only destroyed when the time-consuming garbage
collector is run, right?  (something you don't want to run whilst the game
is running or you might drop a frame)

So I'm coming to the conclusion that Lua is great for math when it involves
atomic types, but as soon as you start trying to use table-like constructs
in the same way the garbage begins to pile really high.  Or maybe it happens
even with the atomic types? :-/

ie.

frog = 10 + 20 + 30 + 40 + 50 + 60

Are 6 "value" objects created here and not deleted until the garbage
collector runs?

frog = Vector{ 10, 20, 30 } + Vector{ 20, 30, 40 } + Vector{ 30, 40, 50 }

Are 3 Vector userdata (or table) objects created and not deleted until the
garbage collector runs?

Or is there some concept of temporaries - cleaning up of values that haven't
been assigned by the end of the expression stack?

Regards

---------------------------------
Q-Games, Dylan Cuthbert.
http://www.q-games.com

"John Belmonte" <[hidden email]> wrote in message
<a  rel="nofollow" href="news:3F00799E.5080702@prairienet.org">news:3F00799E.5080702@......
> Dylan wrote:
> > Something like   (this is totally off the top of my head.. don't rip me
to
> > shreds)
> >
> > position = << 10, 20, 30 >>    -- create a "tuple"
> > position.another_value = 10    -- ERROR.. position isn't a table and is
a
> > "const" in effect
> >
> > position<<x>> = 30        -- creates a new "const" vector with x as 30
and
> > the rest from position
> > position<<x,y>> = 50, 70  -- writes to x, y, and takes z from position,
> > creating a new vector
>
> I haven't been following this thread too carefully, but userdata is what
you
> want.  It seems like you are on that path anyway.
>
>
> -- other stuff
> lua_array = position.ToArray()
> lua_array[0] = 25.5                     -- assuming you break Lua
convention and
>                                          --     start your array at 0 :-(
> position_b = Vector(lua_array)
> position_c = position * position_b      -- your efficient native operation
> point = Vector_XY(position_c)           -- extract XY as vector
> z = Vector_Z(position_c):ToNumber()     -- extract Z as Lua number
> z = position_c:ToArray()[2]             -- same thing
> z = position_c:GetScalarZ()             -- same thing
> point_b = Vector(point) -- copy
>
>
> If you haven't seen it, "Vector { 1, 2, 3 }" is table constructor syntax,
> equivalent to Vector({1, 2, 3}).
>
> This isn't too far off from your C++ vector template stuff...
>
> -John
>
>
>
> -- 
> http:// if   l .o  /
>
>



Reply | Threaded
Open this post in threaded view
|

Re: Vectors/Atomic Tuples/Garbage Collection

John Belmonte-2
Dylan wrote:
frog = 10 + 20 + 30 + 40 + 50 + 60

Are 6 "value" objects created here and not deleted until the garbage
collector runs?

No

frog = Vector{ 10, 20, 30 } + Vector{ 20, 30, 40 } + Vector{ 30, 40, 50 }

Are 3 Vector userdata (or table) objects created and not deleted until the
garbage collector runs?

Yes-- both userdata and table garbage is being generated there.

Your fear is valid. An incremental GC is the #1 thing Lua needs. It's being worked on.



--
http:// if   l .o  /


Reply | Threaded
Open this post in threaded view
|

Re: Vectors/Atomic Tuples/Garbage Collection

Dylan Cuthbert
"John Belmonte" <[hidden email]> wrote in message
[hidden email]">news:3F012DFF.20903@......
> Dylan wrote:
> > frog = 10 + 20 + 30 + 40 + 50 + 60
> >
> > Are 6 "value" objects created here and not deleted until the garbage
> > collector runs?
>
> No
>
> > frog = Vector{ 10, 20, 30 } + Vector{ 20, 30, 40 } + Vector{ 30, 40,
50 }
> >
> > Are 3 Vector userdata (or table) objects created and not deleted until
the
> > garbage collector runs?
>
> Yes-- both userdata and table garbage is being generated there.
>
> Your fear is valid.  An incremental GC is the #1 thing Lua needs.  It's
being
> worked on.

This is probably the best angle to work from when arguing my case for a way
of expanding atomic types into tuples.

This makes any arithmetic style of operation on anything other than the
atomic types, really *really* expensive. :-(  There's not much point in
having metatables where you can override operators if this kind of overhead
is lurking in the background.


Right now, If I have 400 aliens each doing pos = pos + vec1 + vec2 + vec3, I
get a leak of 400x3x16=20K each frame.. at 60fps that's 115k a second...6.9
meg a minute...4 gigabytes an hour... and that's without any lua table
structures or malloc overheads in my calculation.

:-(

An incremental garbage collector that will run in any loose time at the end
of the frame would definitely go some way towards improving the situation.
:-)

---------------------------------
Q-Games, Dylan Cuthbert.
http://www.q-games.com



12