tokenize lua string

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

tokenize lua string

neha jonna
Hi All,
 
How do I tokenize a lua string? if someone could point me in the rght direction, it would be very helpful.
 
Thanks,
Neha.
 
 


Luggage? GPS? Comic books?
Check out fitting gifts for grads at Yahoo! Search.
Reply | Threaded
Open this post in threaded view
|

Re: tokenize lua string

Peter Odding
Neha wrote:
How do I tokenize a lua string? if someone could point me in the rght direction, it would be very helpful.

Do you mean a string of Lua source-code? Try ltokens[1], Rici's lexer[2] or my lexer[3]. Or do you mean a "string literal" used in Lua source-code? The manual[4] explains string literals very well, and any of the above lexers should get you started. If neither of those, I don't know.

 - Peter Odding

[1] http://www.tecgraf.puc-rio.br/~lhf/ftp/lua/#ltokens
    based on the actual Lua lexer (IIRC)

[2] http://lua-users.org/wiki/RiciLake
    "code and configuration to lex and colourise Lua"
    written in pure Lua (IIRC), so most portable

[3] http://xolox.ath.cx/lua/lexer.html
    uses LPeg, see http://www.inf.puc-rio.br/~roberto/lpeg.html

[4] http://www.lua.org/manual/5.1/manual.html#2.1
    "Literal strings can be delimited by ..."