On the subject of Lua's grammar... --[[ ]]

classic Classic list List threaded Threaded
24 messages Options
12
Reply | Threaded
Open this post in threaded view
|

On the subject of Lua's grammar... --[[ ]]

Adam D. Moss
While we're talking about Lua's grammar, and someone had
mentioned Lua's short-comment specifier (which I don't really mind)
there's one thing I'd really rather like to see in a future
version of the language: nestable or otherwise non-accidentally-
terminated long-comments.

I often use --[[ ... ]] for commenting out chunks of code, but
am also often foiled in this because the long-comment terminator,
']]', is something that fairly often occurs naturally in the
code I'm commenting out.  It would be nice if the parser could
either count the [[ and ]] balance and only terminate the
comment when the balance returns to zero, or allowed a different
block-comment format whose terminator sequence wasn't valid
Lua grammar.

Regards,
--Adam
--
Adam D. Moss   . ,,^^   [hidden email]   http://www.foxbox.org/   co:3
Consume Less, Live More

Reply | Threaded
Open this post in threaded view
|

Re: On the subject of Lua's grammar... --[[ ]]

Roberto Ierusalimschy
> It would be nice if the parser could either count the [[ and ]] balance

It does. The problem is that, in regular code, `]]' usually appears
without a corresponding `[[' (e.g., `a[b[i]]').


> or allowed a different block-comment format whose terminator sequence
> wasn't valid Lua grammar.

It will. Lua 5.1 will generalize `[[ ]]' to `[*...*[ ]*...*]', so
that a `[*[' only closes with a `]*]' (same number of stars). That
will allow the inclusion of *any* literal string inside Lua code without
modifications.

The syntax is not very pretty, but to allow any literal string we need
variable delimiters. And the need to insert source code (and other
large chunks of text) inside Lua code is quite common.

-- Roberto

Reply | Threaded
Open this post in threaded view
|

Re: On the subject of Lua's grammar... --[[ ]]

Adam D. Moss
Roberto Ierusalimschy wrote:
It would be nice if the parser could either count the [[ and ]] balance

It does. The problem is that, in regular code, `]]' usually appears
without a corresponding `[[' (e.g., `a[b[i]]').

It occurred to me after sending that this was always the case
for me too...

or allowed a different block-comment format whose terminator sequence
wasn't valid Lua grammar.

It will. Lua 5.1 will generalize `[[ ]]' to `[*...*[ ]*...*]', so

That's great news.

that a `[*[' only closes with a `]*]' (same number of stars). That
will allow the inclusion of *any* literal string inside Lua code without
modifications.

The syntax is not very pretty, but to allow any literal string we need
variable delimiters.

That syntax does look pretty wacky, but in this case I'm
happy with whatever works. :)

Thanks,
--Adam
--
Adam D. Moss   . ,,^^   [hidden email]   http://www.foxbox.org/   co:3
Consume Less, Live More

Reply | Threaded
Open this post in threaded view
|

Re: On the subject of Lua's grammar... --[[ ]]

Roberto Ierusalimschy
> That syntax does look pretty wacky, [...]

Suggestions for improvements are wellcome. The syntax needs to allow the
inclusion of any literal string; it would be nice if the old `[[ ]]'
could fit as a special case of the new format.

-- Roberto

Reply | Threaded
Open this post in threaded view
|

Re: On the subject of Lua's grammar... --[[ ]]

Adam D. Moss
Roberto Ierusalimschy wrote:
That syntax does look pretty wacky, [...]

Suggestions for improvements are wellcome. The syntax needs to allow the
inclusion of any literal string; it would be nice if the old `[[ ]]'
could fit as a special case of the new format.

Yeah, I could see what you were going for.  I can't think of
anything more visually pleasing that keeps [[ compatability, my
main worry was that --[*[ would be inconvenient to type (on many/most
UK/US keyboards needing the use of shift or keypad) and head down
the slippery slope of making Lua code look like line-noise.  But
having spent a few minutes now 'trying it out' it's fine really and
in most cases I expect people can safely stick to [[ ]] anyway.

Regards,
--Adam
--
Adam D. Moss   . ,,^^   [hidden email]   http://www.foxbox.org/   co:3
Consume Less, Live More

Reply | Threaded
Open this post in threaded view
|

Re: On the subject of Lua's grammar... --[[ ]]

Peter Hill-3
In reply to this post by Roberto Ierusalimschy
>> That syntax does look pretty wacky, [...]

Roberto Ierusalimschy:
> Suggestions for improvements are wellcome. The syntax needs to allow the
> inclusion of any literal string; it would be nice if the old `[[ ]]'
> could fit as a special case of the new format.

To wrap an unconstrained text (a long string or comment) there are a few
options:

(1) Specifying its length.

Convenient for computers but not humans, very hard for a person to see the
end of such a string, and prone to easy upset if a person wishes to manually
edit the string.

(2) Have a termination sequence, and use escape padding to prevent that
sequence occuring.

Easy for a computer to generate. Easy for a human to generate from scratch
but a lot of work if one wishes to 'wrap' an exisiting section (unless its
structure is well known). Easy for a human to read unless there are multiple
nested sections (which causes escapes on the escapes on the escapes etc).

(3) Have an arbitrary escape sequence specified at the start.

This is easy for a human to do (especially if the syntax allows nesting,
when one may almost always get away with using the same terminator); a human
can rely on certain sequences never occuring in formatted structures (like
code) and use an editor's "find" facility to double check. Easy for a human
to read, and to modify text (so long as care is taken not to inadvertantly
create the terminator inside the text). Easy for a computer to create IF it
has the whole text available at the start to analyse and make sure the
computer's choice of terminator does not occur in the text. IF the whole
text is not available at the start then neither a human nor a compter can
use this method.

*cheers*
Peter Hill.



Reply | Threaded
Open this post in threaded view
|

Re: On the subject of Lua's grammar... --[[ ]]

Brian Weed-2
In reply to this post by Roberto Ierusalimschy
I can feel the flame already, but...

How about // and /* */

Brian.

----- Original Message ----- 
From: "Roberto Ierusalimschy" <[hidden email]>
To: "Lua list" <[hidden email]>
Sent: Thursday, November 27, 2003 7:33 AM
Subject: Re: On the subject of Lua's grammar... --[[ ]] 


> > That syntax does look pretty wacky, [...]
> 
> Suggestions for improvements are wellcome. The syntax needs to allow the
> inclusion of any literal string; it would be nice if the old `[[ ]]'
> could fit as a special case of the new format.
> 
> -- Roberto
> 

Reply | Threaded
Open this post in threaded view
|

Re: On the subject of Lua's grammar... --[[ ]]

Ron Hudson

Here's a completely different request...

How about Lua sending the text through some preprocessor first?

#define
#include

#ifdef #end (or however that goes)

and all the other fun pre-processor directives.


Reply | Threaded
Open this post in threaded view
|

Re: On the subject of Lua's grammar... --[[ ]]

Mark Hamburg-4
In reply to this post by Roberto Ierusalimschy
--[*label*[    ]*label*]

Where label is drawn from some set of characters (e.g., alphabetic, dashes,
underscores, etc.)

This saves humans the trouble of counting the asterisks when matching things
up. For example:

--[*test-harness*[

< insert Lua code here >

]*test-harness*]--

The trailing "--" is superfluous and exists just for a personal sense of
code symmetry in this case.

Mark

P.S. What's the timetable for Lua 5.1 these days?

on 11/27/03 4:33 AM, Roberto Ierusalimschy at [hidden email] wrote:

>> That syntax does look pretty wacky, [...]
> 
> Suggestions for improvements are wellcome. The syntax needs to allow the
> inclusion of any literal string; it would be nice if the old `[[ ]]'
> could fit as a special case of the new format.
> 
> -- Roberto
> 


Reply | Threaded
Open this post in threaded view
|

Re: On the subject of Lua's grammar... --[[ ]]

Mark Hamburg-4
In reply to this post by Ron Hudson
I think if you check Lua's history this existed at one time. If Lua were to
go this route, I'd rather see something inspired by Lisp macros.

Another option for getting conditional compilation might be to have some
form of compile-time expression notation in the language and have the
compiler be smart about handling "if true" and "if false".

Mark

on 11/27/03 8:04 AM, Ron Hudson at [hidden email] wrote:

> 
> Here's a completely different request...
> 
> How about Lua sending the text through some preprocessor first?
> 
> #define
> #include
> 
> #ifdef #end (or however that goes)
> 
> and all the other fun pre-processor directives.
> 


Reply | Threaded
Open this post in threaded view
|

Re[2]: On the subject of Lua's grammar... --[[ ]]

Gunnar Zötl
In reply to this post by Ron Hudson
RH> Here's a completely different request...

RH> How about Lua sending the text through some preprocessor first?

I think some sort of hook function might be neat, that is called
before the lua code is actually compiled, and may alter the source
code. With such a thing, C style #defines could be implemented, but
also more complex things like a scheme-style hygienic macro
preprocessor. I have no need for that currently, but it would be cool
to have. Might pose some interesting problems, though...

Gunnar
-- 
"They that can give up lib-   Gunnar Zötl           Tel. 06071-612079
 erty to obtain a little      Friedhofstraße 55     Fax. 01212-512317564
 temporary safety deserve     64859 Eppertshausen   E-mail [hidden email]
 neither liberty nor safety" - Benjamin Franklin, 1706-1790



Reply | Threaded
Open this post in threaded view
|

Re: Re[2]: On the subject of Lua's grammar... --[[ ]]

Luiz Henrique de Figueiredo
In reply to this post by Adam D. Moss
>RH> How about Lua sending the text through some preprocessor first?
>
>I think some sort of hook function might be neat, that is called
>before the lua code is actually compiled, and may alter the source
>code. With such a thing, C style #defines could be implemented, but
>also more complex things like a scheme-style hygienic macro
>preprocessor. I have no need for that currently, but it would be cool
>to have. Might pose some interesting problems, though...

A primitive form of hook already exists because lua_load, which is the *only*
way to compile Lua code, calls a user function to get blocks of data. In this
function, you can do as you please. I'm not saying that it's easy, I'm only
saying that the mechanism is already in place.
--lhf

Reply | Threaded
Open this post in threaded view
|

Re[4]: On the subject of Lua's grammar... --[[ ]]

Gunnar Zötl
LHdF> A primitive form of hook already exists because lua_load, which is the *only*
LHdF> way to compile Lua code, calls a user function to get blocks of data. In this
LHdF> function, you can do as you please. I'm not saying that it's easy, I'm only
LHdF> saying that the mechanism is already in place.
LHdF> --lhf

great :-)

Gunnar



Reply | Threaded
Open this post in threaded view
|

Re: On the subject of Lua's grammar... --[[ ]]

Roberto Ierusalimschy
In reply to this post by Mark Hamburg-4
> --[*label*[    ]*label*]

Then `[[' needs special treatment.


> P.S. What's the timetable for Lua 5.1 these days?

Good question. May 2004??  (The turning point is incremental GC, which
we are starting to implement now. That can take one~two months, or
it can go into a never ending spiral of horror and hell.)

-- Roberto

Reply | Threaded
Open this post in threaded view
|

RE: On the subject of Lua's grammar... --[[ ]]

Nick Trout
In reply to this post by Adam D. Moss

> From: Roberto Ierusalimschy [[hidden email]]
> Sent: Thursday, November 27, 2003 4:34 AM
> Subject: Re: On the subject of Lua's grammar... --[[ ]]
> 
> > That syntax does look pretty wacky, [...]
> 
> Suggestions for improvements are wellcome. The syntax needs to allow
the
> inclusion of any literal string; it would be nice if the old `[[ ]]'
> could fit as a special case of the new format.

Woo I didn't think I'd be saying this, but what about the way it works
in Perl?

e.g. <<END;

I can put anything I want here until my own delimiter is met...

END

This could extend to comments and strings, e.g.

-- Single line comment

s = "Single line string"

m = [[Multiline 
string can inhabit
several lines....
]]

--[[Multiline comment can
be spaced over several
lines.]]

m2 = [*[END]*] Multiline strings
can be split over several lines
and [[ optionally contain and characters I like ]]
as long as its not the delimiter I specified.
END

--[*[END]*] Multiline strings
can be split over several lines
and [[ optionally contain and characters I like ]]
as long as its not the delimiter I specified.
END

The following would be the same as --[[ ]]

--[*[]]]*] Multiline strings
can be split over several lines
and [[ optionally contain and characters I like 
as long as its not the delimiter I specified.
]]

[*[...]*] is quite clear, almost like a section heading.

In Python you can enclose anything in """ (except """) so long comments
are strings which are discarded. I quite like this as it slims the
language down slightly. It might be nice to discard --[[ and --[*[ and
just discard [[ and [*[ string which are not assigned to a variable. I
assume this causes problems for the parser (e.g. because a function has
option brackets around a single argument)?

I suppose Robertos suggestion is simpler --[***[ ... ]***] but I thought
I'd suggest this.

--[*[Nick]*]

Regards,
Nick




Reply | Threaded
Open this post in threaded view
|

RE: On the subject of Lua's grammar... --[[ ]]

RLak-2
In reply to this post by Adam D. Moss
Nick Trout didn't think he'd be saying this, but:
> what about the way it works in Perl?

> e.g. <<END;

> I can put anything I want here until my own delimiter is met...

> END

Actually, I think this comes from the Unix shell, even before Perl. It
is also more or less the strategy used by MIME. However, it can also lead
to obscure errors, and it is difficult to implement reliably in a code
generator.

I would actually suggest something much simpler: leave --[[, [[ and ]] as 
they are, except
that they are only recognized if they are the first characters of a line, 
or perhaps
the first non-whitespace characters. 

That does not allow an "arbitrary" string, but it would certainly solve 
all of my
problems with accidental closing by a[b[j]]; I can probably remember not 
to
write that as:

a[b[
]]

:)

This would probably mean changing the semantics of long strings [[ ]] yet 
again,
so that both the first and the last line end are deleted. Oh, well.

Reply | Threaded
Open this post in threaded view
|

Re: On the subject of Lua's grammar... --[[ ]]

Ivaylo Beltchev
From: <[hidden email]>
To: "Lua list" <[hidden email]>
Sent: Thursday, November 27, 2003 1:20 PM
Subject: RE: On the subject of Lua's grammar... --[[ ]]


> Nick Trout didn't think he'd be saying this, but:
> > what about the way it works in Perl?
> 
> > e.g. <<END;
> 
> > I can put anything I want here until my own delimiter is met...
> 
> > END
> 
> Actually, I think this comes from the Unix shell, even before Perl. It
> is also more or less the strategy used by MIME. However, it can also lead
> to obscure errors, and it is difficult to implement reliably in a code
> generator.
> 
> I would actually suggest something much simpler: leave --[[, [[ and ]] as 
> they are, except
> that they are only recognized if they are the first characters of a line, 
> or perhaps
> the first non-whitespace characters. 
> 
> That does not allow an "arbitrary" string, but it would certainly solve 
> all of my
> problems with accidental closing by a[b[j]]; I can probably remember not 
> to
> write that as:
> 
> a[b[
> ]]
> 
> :)
> 
> This would probably mean changing the semantics of long strings [[ ]] yet 
> again,
> so that both the first and the last line end are deleted. Oh, well.

I second that, simpler is better!

I have written a text editor with syntax coloring for Lua. For each line of the text it stores a state at the end of the line to be used in the next line. For example if the current line starts a string and ends on '\' then the string continues on the next line.

The nested comments and strings are a big pain as it is because I have to include the nesting level in my state. Adding variable nested comment delimiters would greatly complicate the end state. (Right now it fits nicely in the range 0..30 and I use it instead of EOL character) I have to include the number of ***s and/or any additional arbitrary identifiers.

I have checked out several other syntax colorers and they use similar approach. Adding variable delimiters will most likely complicate those too.

Regards
Ivo


Reply | Threaded
Open this post in threaded view
|

Re: Re[2]: On the subject of Lua's grammar... --[[ ]]

D Burgess-2
In reply to this post by Adam D. Moss
>A primitive form of hook already exists because lua_load, which is the *only*
>way to compile Lua code, calls a user function to get blocks of data. In this
>function, you can do as you please. I'm not saying that it's easy, I'm only
>saying that the mechanism is already in place.
>--lhf

I think that is enough.

DB
			





Reply | Threaded
Open this post in threaded view
|

Re: On the subject of Lua's grammar... --[[ ]]

Leandro Candido
In reply to this post by RLak-2
Hello all,

    Why not use --]] or ]]-- to end? Like /* and */. Ex:
--[[    ...    ]]-- or
--[[    ...    --]]

    As we are in suggestion time, i will make my one. Can you guys add range
operator? To use like:

    for x in 0..10 do print(x) end or
    for x in 'A'..'Z' do print(x) end

    Of course the .. need be other letters/signs or we need to change the
concat operator, because it is "..".
    Perhaps the best is the word "to":

    for x in 0 to 10 do print(x) end, y = 0 to 10 or y = { 0 to 10 } --
equal y = { 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 }

    This is very useful, really save time.

    Other possible uses:

    z = { "a", "hello", "world", "z" }
    for k in z do
        if k in "a" to "z" then
            print("k (" .. k .. ") in range!");
        else
            print("k (" .. k ..") not in range!");
        end
    end

                                                            The God's Peace,


Leandro.

----- Original Message -----
From: <[hidden email]>
To: "Lua list" <[hidden email]>
Sent: Thursday, November 27, 2003 7:20 PM
Subject: RE: On the subject of Lua's grammar... --[[ ]]


> Nick Trout didn't think he'd be saying this, but:
> > what about the way it works in Perl?
>
> > e.g. <<END;
>
> > I can put anything I want here until my own delimiter is met...
>
> > END
>
> Actually, I think this comes from the Unix shell, even before Perl. It
> is also more or less the strategy used by MIME. However, it can also lead
> to obscure errors, and it is difficult to implement reliably in a code
> generator.
>
> I would actually suggest something much simpler: leave --[[, [[ and ]] as
> they are, except
> that they are only recognized if they are the first characters of a line,
> or perhaps
> the first non-whitespace characters.
>
> That does not allow an "arbitrary" string, but it would certainly solve
> all of my
> problems with accidental closing by a[b[j]]; I can probably remember not
> to
> write that as:
>
> a[b[
> ]]
>
> :)
>
> This would probably mean changing the semantics of long strings [[ ]] yet
> again,
> so that both the first and the last line end are deleted. Oh, well.
>


Reply | Threaded
Open this post in threaded view
|

Re: On the subject of Lua's grammar... --[[ ]]

Philipp Janda
Leandro Candido schrieb:
Hello all,


    As we are in suggestion time, i will make my one. Can you guys add range
operator? To use like:

    for x in 0..10 do print(x) end or
    for x in 'A'..'Z' do print(x) end

    Of course the .. need be other letters/signs or we need to change the
concat operator, because it is "..".
    Perhaps the best is the word "to":

    for x in 0 to 10 do print(x) end, y = 0 to 10 or y = { 0 to 10 } --
equal y = { 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 }

What about this:

local function num_range_iterator( state, var1 )
  -- state is the end of the iteration
  var1 = var1 + 1
  if var1 > state then
    return nil
  end
  return var1
end

local function char_range_iterator( state, var1 )
  -- state is the end of the iteration
  local ret = num_range_iterator( state, string.byte( var1 ) )
  return ret and string.char( ret )
end


function range( start, ending )
  local stype, etype = type( start ), type( ending )
  if stype == "string" and etype == "string" then
    return char_range_iterator, string.byte( ending ),
           string.char( string.byte( start )-1 )
  elseif stype == "number" and etype == "number" then
    return num_range_iterator, ending, start-1
  else
    error( "type error in function `range' (strings or numbers expected)" )
  end
end


function from_to( start, ending )
  local tab = {}
  for x in range( start, ending ) do
    table.insert( tab, x )
  end
  return unpack( tab )
end

--------- test -------------------------------------------------------


for x in range( 1, 10 ) do
  print( x )
end

for x in range( 'A', 'Z' ) do
  print( x )
end

local y = { from_to( 1, 10 ) }
for i,j in ipairs( y ) do
  print( i, j )
end




Philipp



12