The Lua lexer - what am I missing?

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

The Lua lexer - what am I missing?

Matt Holmes
Greetings,

I am attempting to read and understand the Lua lexer/parser, and I have run
in to a slight snag. For the most part I understand the concept of parsing
for tokens (for instance >= is TK_GE), but what I am missing is the use of
the underscore ( _ ) character as a delimiter to begin searching for a
reserved word or identifier name. For instance, in luaX_lex:

case '_': goto tname;

        tname: {  /* identifier or reserved word */
          TString *ts = luaS_new(LS->L, readname(LS));
          if (ts->marked >= RESERVEDMARK)  /* reserved word? */
            return ts->marked-RESERVEDMARK+FIRST_RESERVED;
          seminfo->ts = ts;
          return TK_NAME;
        }

why would LS->current ever be an underscore? I know I for one do not write
an underscore before every keyword or identifier name. Have I utterly missed
something here? Is this some standard for lexical parsers that I may have
passed over in my research?

I looked to see if some kind of replacement was done on white space,
replacing all spaces with underscores, but I saw no such operation, nor
would I understand the point if I did. I did not see that the zgetc() macro
did any type of conditional return for whitespace either.

Obviously I am missing something fundamental here. Could someone clue me in
please? :)

Matt "Kerion" Holmes



Reply | Threaded
Open this post in threaded view
|

Re: The Lua lexer - what am I missing?

Edgar Toernig
Matt Holmes wrote:
>
> why would LS->current ever be an underscore? I know I for one do not write
> an underscore before every keyword or identifier name. Have I utterly missed
> something here?

The "case '_':" just handles identifiers that _start_ with an underscore.
All other identifiers and keywords are handle by the "default:" case that
falls through to the "tname" label.

Ciao, ET.

Reply | Threaded
Open this post in threaded view
|

Re: The Lua lexer - what am I missing?

Eric Tetz-2
In reply to this post by Matt Holmes
--- Matt Holmes <[hidden email]> wrote:
> I am attempting to read and understand the Lua lexer/parser, and I have run
> in to a slight snag.

Check out the code in Lua4.1. That section of the switch statement has been re-organized and is
much clearer.

Cheers,
Eric

__________________________________________________
Do You Yahoo!?
Make international calls for as low as $.04/minute with Yahoo! Messenger
http://phonecard.yahoo.com/

Reply | Threaded
Open this post in threaded view
|

Re: The Lua lexer - what am I missing?

Matt Holmes
My biggest problem is understanding some of the Lua code at all, lol. Lua is
a wonderful scripting language, but I can tell that the parser/lexer was
written while burning some of that ole midnight oil. Some of the code is
just downright sloppy (no offense to the guys who wrote it, I don't know if
I could write it).

Matt "Kerion" Holmes

----- Original Message -----
From: "Eric Tetz" <[hidden email]>
To: "Multiple recipients of list" <[hidden email]>
Sent: Wednesday, August 15, 2001 5:15 PM
Subject: Re: The Lua lexer - what am I missing?


> --- Matt Holmes <[hidden email]> wrote:
> > I am attempting to read and understand the Lua lexer/parser, and I have
run
> > in to a slight snag.
>
> Check out the code in Lua4.1. That section of the switch statement has
been re-organized and is
> much clearer.
>
> Cheers,
> Eric
>
> __________________________________________________
> Do You Yahoo!?
> Make international calls for as low as $.04/minute with Yahoo! Messenger
> http://phonecard.yahoo.com/
>