Exposing lua lexer

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Exposing lua lexer

Domingo Alvarez Duarte
Hello !

Why not expose the Lua lexer to Lua scripts ?

====

string.lex() -- return a lexer object

io.lex() -- return a lexer object

local lexer = io.open("source.file"):lex()

for k,v in lexer:tokens() do print(k,v) end --pre defined tokens

for token in lexer:next() do -- return the next token or nil

     print(token.line, token.col, token.value)

end

====

The above is only a potential use for it.

It's already there and works very well but Lua scripts need to reinvent
the wheel for several tasks.

Anyone already did it ?

There is this projects that do it but with a custom made lexer:

https://github.com/macton/DDLParser

https://github.com/leiradel/ddlt

Cheers !


Reply | Threaded
Open this post in threaded view
|

Re: Exposing lua lexer

Luiz Henrique de Figueiredo
> Why not expose the Lua lexer to Lua scripts ?

See my ltokenp, a token processor for Lua:
http://webserver2.tecgraf.puc-rio.br/~lhf/ftp/lua/#ltokenp
http://lua-users.org/lists/lua-l/2018-07/msg00683.html