Indexing strings in Lua?

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Indexing strings in Lua?

Alex Liebowitz
I'm new to Lua.. how would I, for instance, extract the third character of a
string? Or the third-through-fifth characters of it? Or convert a string
into a list of characters and vice-versa so I could do so? ...


Reply | Threaded
Open this post in threaded view
|

Re: Indexing strings in Lua?

Brian Mitchell-2

On Sun, 27 May 2001, Alex Liebowitz wrote:

> I'm new to Lua.. how would I, for instance, extract the third character of a
> string? Or the third-through-fifth characters of it? Or convert a string
> into a list of characters and vice-versa so I could do so? ...
>
>

strsub(s, 3, 5)




Reply | Threaded
Open this post in threaded view
|

Lua's GC in interactive applications (games)... feedback appreciated!

Jonathan  Branam
I've been looking at Lua for a while now, and I think it is an excellent
embedded language. My interest is in using it as a scripting language for 3d
apps, mainly games. I need something that will work as a configuration or
data description language, and also something for scripting events and such.
Lua is excellent at both of these things. [Any information or tips from
others on this would be welcome as well!]

The only real problem that I have had is with Lua's Garbage Collection. It
seems to cause a pretty large hitch when it runs, and this isn't really
acceptable. It is also largely unpredictable as to when it will run
(although I understand that I can control this more).

I am trying to get some feedback on a number of work-arounds that I am
considering.

1) I could run the GC manually, and have it take a set amount of time. This
way I could run it every couple seconds or so, but not let it take too long.
I can use the built-in feedback to find out if I need to run it more often.

2) Avoid operations which cause orphans. The main problem here is that of
userdata. I added new types for Vectors, Rotations, Color, and other types
that are needed. But many operations on them don't alter the existing data,
but rather create new data. This is something that may be related to my use
of Lua, so any suggestions would be greatly appreciated. One thought I had,
was to create += type functions, this will stop some unneccesary orphans
(eg: vec = vec + 1 orphans the old value of vec. But vec:add(1) would not).
However, this will not fix more complicated expressions: vec = p0 + (t *
(p1-p0)).

3) Extend Lua so that my new types are "built-in" such as the Number type.
As far as I can tell, the Number type never impacts GC. It looks like most
of my types can fit into the Value union without adding extra space
overhead. This seems like an attractive option, but I am still not sure how
difficult this change will be. There are only 37 references to LUA_TNUMBER,
so it seems not too difficult. TObject is used 82 times. My concern is that
this won't really fix my problems, just make them much more infrequent
(which is still good).

4) Extend Lua's GC to be use more "active" ref counting. I'm not exactly
sure of the term for this, but basically what I mean is that whenever an
operation takes a reference to something, it increments the refcount, and
when it releases a reference, it decrements the refcount. And every time the
refcount is decremented, if it reaches zero, the object (or value) is
collected immediately. This also does not seem incredibly difficult, but I'm
sure it would take me a while and that I wouldn't catch all the errors for a
while. This seems like it would be generally useful to others as well, so I
might favor it if others are interested.

Currently, #4 (active ref counting) seems the most attractive to me, but the
time required to do it is questionable. I have been experimenting with Lua
for a while, but only recently started looking at the implementation. It is
impressively simple and compact, but I am worried about small changes
impacting the whole system (due to my inexperience).

Well, that's enough for now. Thanks for your time. I am very happy to
provide more information and examples if what I've said isn't clear (or to
determine if I'm off my rocker).

Thanks,
Jonathan


Reply | Threaded
Open this post in threaded view
|

Re: Lua's GC in interactive applications (games)... feedback appreciated!

Jonathan  Branam
John Belmonte has a page about this, as well as a reference paper.
http://lua.swiki.net/9

I've discovered that values on the stack will cause some issues with my
current implementation plans. Most of the time, the stack is popped
simply by changing the pointer to the top. This would have to be changed
for my refcount implementation to work. I believe there will probably
be other places where references to values are removed without
checks, so I'll have to track those down. Also, some C functions may
do this as well.

The reference paper about does a search over memory over time.
This seems good, but has lots of implications. It also seems to require
knowledge of when an object takes a reference to another object, so
that is something that will be needed for my method, and theirs.

The other major problem with a simple refcount system is that
cyclic references are never removed. In my case, this should be
fairly rare. But it brings up a good point. I think if I change anything,
I will attempt to leave the ability to run the current Lua GC as a
debugging aid. This will help to find out if I've missed any data.

Still thinking,
Jonathan

----- Original Message -----
From: "Jonathan Branam" <[hidden email]>
To: "Multiple recipients of list" <[hidden email]>
Sent: Sunday, May 27, 2001 7:55 PM
Subject: Lua's GC in interactive applications (games)... feedback
appreciated!


> I've been looking at Lua for a while now, and I think it is an excellent
> embedded language. My interest is in using it as a scripting language for
3d
> apps, mainly games. I need something that will work as a configuration or
> data description language, and also something for scripting events and
such.
> Lua is excellent at both of these things. [Any information or tips from
> others on this would be welcome as well!]
>
> The only real problem that I have had is with Lua's Garbage Collection. It
> seems to cause a pretty large hitch when it runs, and this isn't really
> acceptable. It is also largely unpredictable as to when it will run
> (although I understand that I can control this more).
>
> I am trying to get some feedback on a number of work-arounds that I am
> considering.
>
> 1) I could run the GC manually, and have it take a set amount of time.
This
> way I could run it every couple seconds or so, but not let it take too
long.
> I can use the built-in feedback to find out if I need to run it more
often.
>
> 2) Avoid operations which cause orphans. The main problem here is that of
> userdata. I added new types for Vectors, Rotations, Color, and other types
> that are needed. But many operations on them don't alter the existing
data,
> but rather create new data. This is something that may be related to my
use
> of Lua, so any suggestions would be greatly appreciated. One thought I
had,
> was to create += type functions, this will stop some unneccesary orphans
> (eg: vec = vec + 1 orphans the old value of vec. But vec:add(1) would
not).
> However, this will not fix more complicated expressions: vec = p0 + (t *
> (p1-p0)).
>
> 3) Extend Lua so that my new types are "built-in" such as the Number type.
> As far as I can tell, the Number type never impacts GC. It looks like most
> of my types can fit into the Value union without adding extra space
> overhead. This seems like an attractive option, but I am still not sure
how
> difficult this change will be. There are only 37 references to
LUA_TNUMBER,
> so it seems not too difficult. TObject is used 82 times. My concern is
that
> this won't really fix my problems, just make them much more infrequent
> (which is still good).
>
> 4) Extend Lua's GC to be use more "active" ref counting. I'm not exactly
> sure of the term for this, but basically what I mean is that whenever an
> operation takes a reference to something, it increments the refcount, and
> when it releases a reference, it decrements the refcount. And every time
the
> refcount is decremented, if it reaches zero, the object (or value) is
> collected immediately. This also does not seem incredibly difficult, but
I'm
> sure it would take me a while and that I wouldn't catch all the errors for
a
> while. This seems like it would be generally useful to others as well, so
I
> might favor it if others are interested.
>
> Currently, #4 (active ref counting) seems the most attractive to me, but
the
> time required to do it is questionable. I have been experimenting with Lua
> for a while, but only recently started looking at the implementation. It
is
> impressively simple and compact, but I am worried about small changes
> impacting the whole system (due to my inexperience).
>
> Well, that's enough for now. Thanks for your time. I am very happy to
> provide more information and examples if what I've said isn't clear (or to
> determine if I'm off my rocker).
>
> Thanks,
> Jonathan
>
>


Reply | Threaded
Open this post in threaded view
|

Re: Lua's GC in interactive applications (games)... feedback appreciated!

John Belmonte-2
In reply to this post by Jonathan Branam
Jonathan Branam wrote:
> I am trying to get some feedback on a number of work-arounds that I am
> considering.
>
> 1) I could run the GC manually, and have it take a set amount of time.
This
> way I could run it every couple seconds or so, but not let it take too
long.
> I can use the built-in feedback to find out if I need to run it more
often.

With Lua's garbage collector, you can't control how much time a gc cycle
takes, other than indirectly by controlling the number of objects in the
system.  In other words, the length of a gc cycle would not be affected by
how often you forced a cycle.  (This is assuming your ratio of
live-to-garbage objects is high.)

> 2) Avoid operations which cause orphans.
> 3) Extend Lua so that my new types are "built-in" such as the Number type.

I don't consider these general solutions.  I wrote a little about this
previously, but the problem is that a gc cycle length is not only related to
how many garbage objects you've produced since the last cycle, but the total
number of all objects (whether they are being used or not).  Don't forget
that functions count as objects.

> 4) Extend Lua's GC to be use more "active" ref counting.

I think Lua's mark-and-sweep collection is fine, it just needs to be made
incremental.  I say "just", but it's probably not a few-day (or few-week)
project.  Switching to reference counting is more of a sideways step than an
improvement, and will take just as much work.

Regards,
-John

p.s.  also see my posting "dream garbage collector"



Reply | Threaded
Open this post in threaded view
|

Re: Lua's GC in interactive applications (games)... feedback appreciated!

Jonathan  Branam
Comments below:
From: "John Belmonte" <[hidden email]>
> Jonathan Branam wrote:
> > I am trying to get some feedback on a number of work-arounds that I am
> > considering.
> >
> > 1) I could run the GC manually, and have it take a set amount of time.
> > This way I could run it every couple seconds or so, but not let it take
> > too long.
> > I can use the built-in feedback to find out if I need to run it more
> > often.
>
> With Lua's garbage collector, you can't control how much time a gc cycle
> takes, other than indirectly by controlling the number of objects in the
> system.  In other words, the length of a gc cycle would not be affected by
> how often you forced a cycle.  (This is assuming your ratio of
> live-to-garbage objects is high.)

Sorry, yes I realize that. I should have said "if you could."

> > 2) Avoid operations which cause orphans.
> > 3) Extend Lua so that my new types are "built-in" such as the Number
type.
>
> a gc cycle length is not only related to
> how many garbage objects you've produced since the last cycle, but the
total
> number of all objects (whether they are being used or not).

This is very true, I was a little misguided on those. However, making new
types
built-in does reduce the number of total objects. This is a big win for me,
because
I will have 8+ vectors and such for every object in my game.

>
> > 4) Extend Lua's GC to be use more "active" ref counting.
>
> I think Lua's mark-and-sweep collection is fine, it just needs to be made
> incremental.  I say "just", but it's probably not a few-day (or few-week)
> project.  Switching to reference counting is more of a sideways step than
an
> improvement, and will take just as much work.

I am definitely beginning to agree here. The refcount was just my first
stab,
it really does not seem like a viable solution in general. I've been looking
at
what is needed to make it incremental, and it seems like a rather large task
as well. It seems like the current stack will change a lot as lua code is
running,
and this is one of the "roots" of the GC. Of course, the global table may
not
change too much. It also requires a write barrier. I am thinking setglobal()
and
settable() will fulfill this requirement somewhat.

> Regards,
> -John
>
> p.s.  also see my posting "dream garbage collector"

I went back and read all the posts I could find, many have similar requests.
In the end, it seems an incremental solution is likely the best. I'm not
sure how
much will change between each incremental call, but it does seem to me that
these algorithms must be called fairly often to get anywhere and they have a
big payoff moment when they are done with everything.

[I am talking about tricolor mark-sweep implementations here, Lua's is not
this way. In these white means currently considered garbage, grey means
the object is not garbage, but all of its links have not yet been traversed,
and
black means it is not garbage and all of its links have been traversed]

I was thinking about the total number of objects problem. It seems like
there
are a few things which could help this. One would be if you could only
reprocess objects which have changed since the last full run of the GC.
After a collection, these tricolor algorithms mark everything as white
again and then start over, but this does not really seem necessary to me.
It seems like, if the write-barrier works properly, that you would never
need to mark everything white and start over.

Another idea that occurs to me is to have some sort of a "static" or "const"
table which never needs to be reprocessed. I will have lots of data such as
this which is purely used for configuration. This could be achieved
automatically by my previous suggestion. It seems that this would do a
great deal to aid the total number of objects problem.

If this were simple, I suppose the answer would already exist!
-Jonathan



Reply | Threaded
Open this post in threaded view
|

Re: Lua's GC in interactive applications (games)... feedback appreciated!

Reuben Thomas-4
> I was thinking about the total number of objects problem. It seems like
> there
> are a few things which could help this. One would be if you could only
> reprocess objects which have changed since the last full run of the GC.

i.e. generational collection.

> If this were simple, I suppose the answer would already exist!

It does, just not implemented in Lua's garbage collector. And of course,
it's not simple. But it does seem that Lua's garbage collector is ripe for
improvement, and for a slight extension of control over it, something like a
call to the garbage collector that says "process at most this many nodes",
so you can roughly control how long it will take. This would need an
incremental garbage collector; using a generational garbage collector would
in addition prevent garbage collection cycles depending on the total number
of objects in the system.

-- 
http://sc3d.org/rrt/ | wit, n.  educated insolence (Aristotle)


Reply | Threaded
Open this post in threaded view
|

Re: Lua's GC in interactive applications (games)... feedback appreciated!

David Jones
In reply to this post by Jonathan Branam
In message <003b01c0e721$b5c27150$7d92c3cf@...>, "Jonathan Branam" writes:
> 
> The only real problem that I have had is with Lua's Garbage Collection. It
> seems to cause a pretty large hitch when it runs, and this isn't really
> acceptable. It is also largely unpredictable as to when it will run
> (although I understand that I can control this more).
> 
> I am trying to get some feedback on a number of work-arounds that I am
> considering.

Another thing worth considering, because it should be easy to do,
is to replace your implementation of malloc and free (the lua
allocator/garbage collector just uses malloc and free "underneath
(well, actually realloc, but hey)).  We replaced malloc and free
in our application and the GC slowed down by about a factor 30, we
then tweaked our allocator and it sped up again.

There are plenty of malloc/free implementations out there ready to use.

Cheers,
 djones