Scoped variable *toclose syntax

classic Classic list List threaded Threaded
21 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Scoped variable *toclose syntax

Dibyendu Majumdar
Hi,

I am writing about this again in the hope that it is not too late to
change the syntax of the *toclose attribute. I think there is an
opportunity to allow a general way of introducing annotations in Lua,
which can be exploited by extensions. It would be sad if that
opportunity is lost.

My previous post: http://lua-users.org/lists/lua-l/2018-12/msg00179.html

Regards
Dibyendu

Reply | Threaded
Open this post in threaded view
|

Re: Scoped variable *toclose syntax

Philippe Verdy
I also agree that the "*xxx" or "*(xxx, params...)" syntax is very fuzzy and will not work with the future evolution of annotations

E.g. If you want to add annotations for subexpressions, it will create very bad situation like
  x = 1 + 2 * *nodebug( subexpression ).
  x = 1 + 2 * *(roundmode,tozero)( subexpression ).
It will be much more readable (and less likely to cause syntax ambiguities) like this:
  x = 1 + 2 * @nodebug( subexpression ).
  x = 1 + 2 * @(roundmode,tozero)( subexpression ).
Annotations are very productive features in modern languages, notably for code generation, maintenance, digital signatures and security, interfaces to other languages and platforms (let's remember that Lua is a language intended to be embedded), or design by contract and documented/supported interfaces, and software life cycle. It also helps compilers.


Le jeu. 25 avr. 2019 à 00:22, Dibyendu Majumdar <[hidden email]> a écrit :
Hi,

I am writing about this again in the hope that it is not too late to
change the syntax of the *toclose attribute. I think there is an
opportunity to allow a general way of introducing annotations in Lua,
which can be exploited by extensions. It would be sad if that
opportunity is lost.

My previous post: http://lua-users.org/lists/lua-l/2018-12/msg00179.html

Regards
Dibyendu

Reply | Threaded
Open this post in threaded view
|

Re: Scoped variable *toclose syntax

Dirk Laurie-2
Op Do. 25 Apr. 2019 om 03:33 het Philippe Verdy <[hidden email]> geskryf:

>
> I also agree that the "*xxx" or "*(xxx, params...)" syntax is very fuzzy and will not work with the future evolution of annotations
>
> Le jeu. 25 avr. 2019 à 00:22, Dibyendu Majumdar <[hidden email]> a écrit :
>
>> I am writing about this again in the hope that it is not too late to
>> change the syntax of the *toclose attribute. I think there is an
>> opportunity to allow a general way of introducing annotations in Lua,
>> which can be exploited by extensions. It would be sad if that
>> opportunity is lost.

The ASCII character set consists of 32 control characters, 1 space, 1
delete charater, 10 digits, 52 letters and 32 punctuation characters
(i.e. those that match '%p').

Of the "punctuation" characters, only five have as yet no meaning in Lua.

33    !   exclamation mark
36    $   dollar sign
63    ?   question mark
64    @  at sign
96    `    baclquote

All five carry baggage in the form of a standardized meaning in C
and/or Unix. Among these, the exclamation mark stands out as one in
which the decision not go with that standardized meaning has already
been made. There is no expectation that it will ever mean `not` in
Lua.

Therefore I would suggest that an annotation starts with ! and
continues until but not including the first termination character.
Sensible termination characters might be comma, semicolon and end-of
line, allowing annotations for every item in a parameter/return list,
table constructor etc, but not encumbering the parser with a difficult
decision whether the annotation is done. In the contex of "local", an
empty annotation could default to 'toclose', so that "local a!, b, c!`
is all that one needs to supply "toclose" directives.

Support for annotations could be anything from nothing to customized.
That is to say, an unrecognized annotation should not be an error.

-- Dirk

Reply | Threaded
Open this post in threaded view
|

Re: Scoped variable *toclose syntax

Dibyendu Majumdar
On Thu, 25 Apr 2019 at 06:54, Dirk Laurie <[hidden email]> wrote:

>
> Op Do. 25 Apr. 2019 om 03:33 het Philippe Verdy <[hidden email]> geskryf:
> > I also agree that the "*xxx" or "*(xxx, params...)" syntax is very fuzzy and will not work with the future evolution of annotations
> >
> > Le jeu. 25 avr. 2019 à 00:22, Dibyendu Majumdar <[hidden email]> a écrit :
> >> I am writing about this again in the hope that it is not too late to
> >> change the syntax of the *toclose attribute. I think there is an
> >> opportunity to allow a general way of introducing annotations in Lua,
> >> which can be exploited by extensions. It would be sad if that
> >> opportunity is lost.
>
> Therefore I would suggest that an annotation starts with ! and
> continues until but not including the first termination character.
> Sensible termination characters might be comma, semicolon and end-of
> line, allowing annotations for every item in a parameter/return list,
> table constructor etc, but not encumbering the parser with a difficult
> decision whether the annotation is done. In the contex of "local", an
> empty annotation could default to 'toclose', so that "local a!, b, c!`
> is all that one needs to supply "toclose" directives.
>

I would suggest that the syntax should be:

<special character><optional name><optional table with only literal
keys and values>

So assuming your choice of !, we could have:

local a!
local b !go_to_space { when=true }
local c !array { 20 }

By allowing an optional table after the name would enable a whole host
of possibilities without breaking the lexer / parser I hope.

Thanks and Regards
Dibyendu

Reply | Threaded
Open this post in threaded view
|

Re: Scoped variable *toclose syntax

Roberto Ierusalimschy
> I would suggest that the syntax should be:
>
> <special character><optional name><optional table with only literal
> keys and values>
>
> So assuming your choice of !, we could have:
>
> local a!
> local b !go_to_space { when=true }
> local c !array { 20 }
>
> By allowing an optional table after the name would enable a whole host
> of possibilities without breaking the lexer / parser I hope.

Personal tastes aside, I fail to see how this differ from the current
status. These variations would work there, too:

local *a
local *go_to_space{when=true} b
local *array{20} c
local *go_to_space{when=true}, *array{20} myvar = ...

To make myself clearer, you wrote:

  I think there is an opportunity to allow a general way of introducing
  annotations in Lua, which can be exploited by extensions. It would be
  sad if that opportunity is lost.

What I don't see is how '!' (versus '*') or postfixed (versus prefixed)
are more appropriate for this "general way of introducing annotations in
Lua".

I also don't see why, in the presence of other options, 'toclose' would
be the default. Is it more common/important than a possible '*const' or
'*immutable'? More common than any attribute that is yet to be created?
(Note that we added 'toclose' to the syntax exactly to allow future
extensions.)

-- Roberto

Reply | Threaded
Open this post in threaded view
|

Re: Scoped variable *toclose syntax

Dibyendu Majumdar
> > I would suggest that the syntax should be:
> >
> > <special character><optional name><optional table with only literal
> > keys and values>
> >
> > So assuming your choice of !, we could have:
> >
> > local a!
> > local b !go_to_space { when=true }
> > local c !array { 20 }
> >
> > By allowing an optional table after the name would enable a whole host
> > of possibilities without breaking the lexer / parser I hope.
>
> Personal tastes aside, I fail to see how this differ from the current
> status. These variations would work there, too:
>
> local *a
> local *go_to_space{when=true} b
> local *array{20} c
> local *go_to_space{when=true}, *array{20} myvar = ...
>

Right, sorry I didn't explain fully.
The syntax for annotations should be unambiguous so that they can
appear in multiple places, not just after 'local'.

Taking the example of type annotations, they may appear after function
var names, or as function return value annotation.
They may also appear perhaps in expressions as type cast operators.
Having an unused symbol therefore ensures there are no parsing
conflicts or ambiguities.

I hope this explains why I think * is not a good choice.

I should also clarify that the table I suggested should only allow
literal scalar values, i.e. numbers and strings, as keys and values,
so that the parsing of this is simple.

Regards
Dibyendu

Reply | Threaded
Open this post in threaded view
|

Re: Scoped variable *toclose syntax

Dibyendu Majumdar
In reply to this post by Roberto Ierusalimschy
On Thu, 25 Apr 2019 at 15:42, Roberto Ierusalimschy
<[hidden email]> wrote:
> To make myself clearer, you wrote:
>
>   I think there is an opportunity to allow a general way of introducing
>   annotations in Lua, which can be exploited by extensions. It would be
>   sad if that opportunity is lost.
>
> What I don't see is how '!' (versus '*') or postfixed (versus prefixed)
> are more appropriate for this "general way of introducing annotations in
> Lua".

I think prefixed or postfixed is probably not important.

>
> I also don't see why, in the presence of other options, 'toclose' would
> be the default. Is it more common/important than a possible '*const' or
> '*immutable'? More common than any attribute that is yet to be created?
> (Note that we added 'toclose' to the syntax exactly to allow future
> extensions.)
>

I think using a default meaning was suggested by Dirk, I agree that
that may not be a good idea.

>From personal taste I would prefer 'autoclose' to 'toclose', but if a
generic annotation feature was adopted then I think it woudn't matter
what you decided to use.

Regards
Dibyendu

Reply | Threaded
Open this post in threaded view
|

Re: Scoped variable *toclose syntax

Dibyendu Majumdar
In reply to this post by Roberto Ierusalimschy
> (Note that we added 'toclose' to the syntax exactly to allow future
> extensions.)
>

Just to clarify - I think it is better to introduce a generic
annotation capability with a defined syntactic form and
interpretation, rather than just introducing something specific such
as '*toclose'. The feature can only be called a generic annotation
capability if Lua parser was able to skip over unrecognized
annotations, and if the syntax was expressive enough to be of real
practical use. This means that some definition of what/where is
allowed is required.

Regards
Dibyendu

Reply | Threaded
Open this post in threaded view
|

Re: Scoped variable *toclose syntax

Sean Conner
In reply to this post by Dibyendu Majumdar
It was thus said that the Great Dibyendu Majumdar once stated:

> On Thu, 25 Apr 2019 at 06:54, Dirk Laurie <[hidden email]> wrote:
> >
> > Op Do. 25 Apr. 2019 om 03:33 het Philippe Verdy <[hidden email]> geskryf:
> > > I also agree that the "*xxx" or "*(xxx, params...)" syntax is very fuzzy and will not work with the future evolution of annotations
> > >
> > > Le jeu. 25 avr. 2019 à 00:22, Dibyendu Majumdar <[hidden email]> a écrit :
> > >> I am writing about this again in the hope that it is not too late to
> > >> change the syntax of the *toclose attribute. I think there is an
> > >> opportunity to allow a general way of introducing annotations in Lua,
> > >> which can be exploited by extensions. It would be sad if that
> > >> opportunity is lost.
> >
> > Therefore I would suggest that an annotation starts with ! and
> > continues until but not including the first termination character.
> > Sensible termination characters might be comma, semicolon and end-of
> > line, allowing annotations for every item in a parameter/return list,
> > table constructor etc, but not encumbering the parser with a difficult
> > decision whether the annotation is done. In the contex of "local", an
> > empty annotation could default to 'toclose', so that "local a!, b, c!`
> > is all that one needs to supply "toclose" directives.
> >
>
> I would suggest that the syntax should be:
>
> <special character><optional name><optional table with only literal
> keys and values>

  So I'm looking at Java because they now have annotations.  From a cursory
look, it appears the syntax to be:

        '@' ID

  So things like

        @Override
        @Deprecated
        @SuppressWarnings

  Annotations can be used at development time, compile time or runtime [1].
They can also be targetted towards a particular usage [2].  In Java, custom
annotations are handled with creating an interface.

  Assuming we use something similar to Java, the "to-close" annotation of
Lua 5.4 could probably look somthign like:

        @target(local)
        function @to_close(var)
          -- whatever this does
        end

        do
          local @to_close file = io.open(...)
          lcoal @to_close obj  = custom:blah(...)
          -- code code code
        end
 
> So assuming your choice of !, we could have:
>
> local a!
> local b !go_to_space { when=true }
> local c !array { 20 }

  Well ...

        local @my_annotation a
        local @go_to_space { when = true } b
        local @array(20) c

where '@array' is a special function called with certain parameters
depending on what it's targetting and any parameters given to it.  Too
magical?  Not magical enough?  Too much like Java? I don't know, it's
beginning to look a bit heavy to me.

  -spc

[1] https://stackoverflow.com/questions/1372876/how-and-where-are-annotations-used-in-java

[2] https://beginnersbook.com/2014/09/java-annotations/

Reply | Threaded
Open this post in threaded view
|

Re: Scoped variable *toclose syntax

Tim Hill


On Apr 25, 2019, at 9:28 AM, Sean Conner <[hidden email]> wrote:

It was thus said that the Great Dibyendu Majumdar once stated:
On Thu, 25 Apr 2019 at 06:54, Dirk Laurie <[hidden email]> wrote:
 So I'm looking at Java because they now have annotations.  From a cursory
look, it appears the syntax to be:

'@' ID

 So things like

@Override
@Deprecated
@SuppressWarnings

 Annotations can be used at development time, compile time or runtime [1]. 
They can also be targetted towards a particular usage [2].  In Java, custom
annotations are handled with creating an interface.


But Java annotations are a total mess, and are drifting toward creating an entire meta-language within Java that is (imho) cumbersome and attempting to make Java something it was never intended to be .. a dynamic language.

—Tim


Reply | Threaded
Open this post in threaded view
|

Re: Scoped variable *toclose syntax

Dibyendu Majumdar
In reply to this post by Sean Conner
On Thu, 25 Apr 2019 at 17:29, Sean Conner <[hidden email]> wrote:

>   So I'm looking at Java because they now have annotations.  From a cursory
> look, it appears the syntax to be:
>
>         '@' ID
>
>   So things like
>
>         @Override
>         @Deprecated
>         @SuppressWarnings
>

Hi, above is not exactly correct in that Java allows annotations to
have attributes.

Some examples:

@KafkaListener(id = "fooGroup3", topics = "topic3")
public void listen2(List<String> in) {
}

@RunWith(SpringRunner.class)
@SpringBootTest
@AutoConfigureTestDatabase
@DirtiesContext(classMode = DirtiesContext.ClassMode.BEFORE_CLASS)
@ContextConfiguration(classes = {Application.class, TestConfig.class,
MockKafkaConfig.class})
public class TestMe {
}

Etc.

Since @ symbol is not valid in Java programs, its use in annotations
makes them very visible.

Reply | Threaded
Open this post in threaded view
|

Re: Scoped variable *toclose syntax

Dibyendu Majumdar
In reply to this post by Tim Hill
On Thu, 25 Apr 2019 at 17:44, Tim Hill <[hidden email]> wrote:
> But Java annotations are a total mess, and are drifting toward creating an entire meta-language within Java that is (imho) cumbersome and attempting to make Java something it was never intended to be .. a dynamic language.

I am not sure why you claim they are a mess - annotations have enabled
an absolutely new levels of capability in Java.

Reply | Threaded
Open this post in threaded view
|

Re: Scoped variable *toclose syntax

Diego Nehab-4
I am curious about what happens after the syntax has been settled. I can see how these generalized annotations would be set from the C side, perhaps by a generalized "lua_toclose" function. But how would the C side control the meaning of these annotations? Can the C side even check the value of annotations set from the Lua side? Must all annotations be associated to their own metamethod that receives a table with the additional parameters? Are there other examples of useful things to do with these annotations in Lua, besides block-scope finalization?
Reply | Threaded
Open this post in threaded view
|

Re: Scoped variable *toclose syntax

Dibyendu Majumdar
On Thu, 25 Apr 2019 at 18:53, Diego Nehab <[hidden email]> wrote:
>
> I am curious about what happens after the syntax has been settled. I can see how these generalized annotations would be set from the C side, perhaps by a generalized "lua_toclose" function. But how would the C side control the meaning of these annotations? Can the C side even check the value of annotations set from the Lua side? Must all annotations be associated to their own metamethod that receives a table with the additional parameters? Are there other examples of useful things to do with these annotations in Lua, besides block-scope finalization?

Hi, I think my main concern right now is that a new syntax '*toclose'
is going to be introduced, for a very specific use case. I do not
believe it has yet been fully explored as to how this would be an
extensible syntax in the future. So then it means that Lua will be
stuck with this syntax for good or bad for years to come. As Roberto
says it is easy to add something, hard to take it back.

It is better to take some time and think through if a generic
annotation feature can be introduced of which '*toclose' becomes a
part. This will pay off in the long term.

Alternatively if there isn't time to do that, then drop the '*toclose'
feature ... better not have it than be stuck with a syntax that may
not serve Lua in the long term.

So the first concern is:

a) Is this going to be something generic or very specific?
b) If generic then can it be something that allows other annotations
to be used in a meaningful way? This can vary between saying an
annotation is simply a symbol+name, or it is something more rich -
symbol+name+table as I suggested.
c) The semantics of this needs to be defined; i.e. where can
annotations appear? In my view to be useful they should be allowed at
every name declaration, and possibly also in expressions.
d) What does Lua do when it encounters an annotation it doesn't
recognise? If it doesn't handle such annotations gracefully then it
will not be possible for any annotations to be used other than
predefined ones such as '*toclose'.

I think the secondary question is what happens to the annotations.

Clearly one option is for Lua to just ignore them in general, and for
third-party tools to write pre or post processors to handle these
annotations.
Another more powerful option is to save the annotations and make them
available as metadata at runtime - but I am not sure if we need to go
this far.

As for immediate benefits, I think that several initiatives are trying
to add type annotations to Lua; this will allow Lua to handle code
that contains such type annotations, and for these other
implementations to happily coexist with Lua.

Regards
Dibyendu

Reply | Threaded
Open this post in threaded view
|

Re: Scoped variable *toclose syntax

Egor Skriptunoff-2
In reply to this post by Dibyendu Majumdar
On Thu, Apr 25, 2019 at 9:38 AM Dibyendu Majumdar wrote:
I would suggest that the syntax should be:

<special character><optional name><optional table with only literal
keys and values>

So assuming your choice of !, we could have:

local a!
local b !go_to_space { when=true }
local c !array { 20 }

By allowing an optional table after the name would enable a whole host
of possibilities without breaking the lexer / parser I hope.



A little improvement: arguments for an annotation must have free syntax.
I suggest to allow arbitrary text with the only limitation: if such text contains curly braces, they must be balanced.

local ver_num @type{int16} @const = 0x504

local arr @autoclose = {} @size{10}

local dict = {} @size{hash=1000}

local function add(x, y)
   @type{in = {int32, int32}, out = {int32}}
   @asm{add EAX,EDX / ret}
   return x+y
end

for j = 1, 256 do @parallel{4}
end

But there is an ambiguity with such syntax:
What does
   local x = my_func @inline {1} ()
mean:
   local x = (my_func @inline) {1} ()
or
   local x = my_func (@inline {1}) ()
?
There must be documented way to syntactically distinguish whether each "annotation invocation" has arguments or not.
The compiler must correctly extract and ignore unknown annotations.
Reply | Threaded
Open this post in threaded view
|

Re: Scoped variable *toclose syntax

Roberto Ierusalimschy
In reply to this post by Dibyendu Majumdar
> Just to clarify - I think it is better to introduce a generic
> annotation capability with a defined syntactic form and
> interpretation, rather than just introducing something specific such
> as '*toclose'. The feature can only be called a generic annotation
> capability if Lua parser was able to skip over unrecognized
> annotations, and if the syntax was expressive enough to be of real
> practical use. This means that some definition of what/where is
> allowed is required.

Just to clarify: we have no intention of introducing a generic
annotation capability to Lua now. We only wanted the syntax to
be such that it could be extended in the future. All we want
is to-be-closed variables.

-- Roberto

Reply | Threaded
Open this post in threaded view
|

Re: Scoped variable *toclose syntax

Sean Conner
It was thus said that the Great Roberto Ierusalimschy once stated:

> > Just to clarify - I think it is better to introduce a generic
> > annotation capability with a defined syntactic form and
> > interpretation, rather than just introducing something specific such
> > as '*toclose'. The feature can only be called a generic annotation
> > capability if Lua parser was able to skip over unrecognized
> > annotations, and if the syntax was expressive enough to be of real
> > practical use. This means that some definition of what/where is
> > allowed is required.
>
> Just to clarify: we have no intention of introducing a generic
> annotation capability to Lua now. We only wanted the syntax to
> be such that it could be extended in the future. All we want
> is to-be-closed variables.

  Fair enough.

  Some questions:  Why the '*' and not something unused, like '@'?

        do
          local @toclose f = io.open(filename)
        end

  Also, might the name 'gcscope' be a better name?

        do
          local @gcscope f = io.open(filename)
        end

  Or maybe scopegc?

        do
          local @scopegc f = io.open(filename)
        end

  -spc (I mean, this is the time to bikeshed, right?)


Reply | Threaded
Open this post in threaded view
|

Re: Scoped variable *toclose syntax

Dibyendu Majumdar
In reply to this post by Roberto Ierusalimschy
> Just to clarify: we have no intention of introducing a generic
> annotation capability to Lua now. We only wanted the syntax to
> be such that it could be extended in the future. All we want
> is to-be-closed variables.
>

Fine. Will look forward to the extensions in future.

Regards

Reply | Threaded
Open this post in threaded view
|

Re: Scoped variable *toclose syntax

Philippe Verdy
In reply to this post by Sean Conner
Actually the form "@ID(parameters)" is used in Java, but the host language construct is not embedded inside the parameters of annotations (which are special constructor invocations that will instanciate separate objects, linked to the host language).
They have various use. The oldest form was for hinting the documentation (allowing inserting parsable comments that would be able to build API help files for example). But they evolved to be insertable in many places.
The rule (with which I agree) was to not modify the host language (or modify it a minimum: they are allowed almost anywhere a whitespace is allowed): you can always safely remove the whole annotation and parse the rest of the source code and compile it: all annotations are ignorable if needed and this should not make the program invalid and not working as expected (even if this causes some additional costs in memory or speed when not using them).
As well annotations should be complete bny themselves, not separated in multiple parts, so:
- the concept of using "!" as a prefix, and "{...}" is suffix for parameters is bad, and in fact it just adds various syntaxic ambiguities which are hard to resolve: this would prohibit using "{...}" table constructors in the host language)
- the same is true when using "@ID(..." for the annotation, followed by the host language constructor, and terminated just by a ")" would also be very quirky (dangerous I think)

My opinion is the the non-ambiguous form for annotation is: - "@ID" without parameters, or "@{ID, parameter...}". Then where we can place the whole annotation to attach it to the host language construct if left to specify (but these annotation tags can normally occur only between host tokens where spaces are allowed.
There may however be a need for annotation with long scopes (applying to multiple host-language constructs). Their form could be "@{begin ID, parameter...}"  for the begining of scope, and ""@{end ID}" for the end of scope (the additional reindication of the annotation ID may help diagnose non matching end).

Now we can as well use "{}" or "()" or "[]" or "<>" to bracket the annotation ID and its parameters in the same tag (either the opening tag or close tag). The initial bracket is disambiguated by the leading "@", which should be one of the character unassigned in Lua: "@" is perfect for that fine (I don't like using "!" because sooner or later it will become an interesting operator).

Note that annotations have their own namespace: they create a secondary metadata language. Binding the metadata language to a specific recognizer (with its own resource loaders and so on) can also use scopes. For this reason, it would be convenient to support not just the forms "@ID" or "@{ID, parameters...}" or "@{begin ID, parameters...}, or "@{end ID}", but also "@namespace:ID" or "@{namespace:ID, parameters...}" or "@{begin namespace:ID, parameters...}, or "@{end namespace:ID}".

The identifiers without namespaces should be reserved to Lua specification (or there would be a special namespace declaration to specify which namespace is attached to annotation IDs without namespaces). Other metatdata languages will use their own namespaces. Namespaces may be either "strings" (like "org.lua") or suites of IDs separated by dots or columns. I have no preference about it:

If there are syntaxic ambiguities, the simple form "@namespace:ID" or "@ID" should always be equivalent (as a shortcut notation) for the complete annotation tag using delimiting brackets like "@{ID}" or "@{namespace:ID}" (i.e. the "{}" brackets surrounding the annotation typename and its parameters is also correct without parameters).






Le jeu. 25 avr. 2019 à 18:29, Sean Conner <[hidden email]> a écrit :
It was thus said that the Great Dibyendu Majumdar once stated:
> On Thu, 25 Apr 2019 at 06:54, Dirk Laurie <[hidden email]> wrote:
> >
> > Op Do. 25 Apr. 2019 om 03:33 het Philippe Verdy <[hidden email]> geskryf:
> > > I also agree that the "*xxx" or "*(xxx, params...)" syntax is very fuzzy and will not work with the future evolution of annotations
> > >
> > > Le jeu. 25 avr. 2019 à 00:22, Dibyendu Majumdar <[hidden email]> a écrit :
> > >> I am writing about this again in the hope that it is not too late to
> > >> change the syntax of the *toclose attribute. I think there is an
> > >> opportunity to allow a general way of introducing annotations in Lua,
> > >> which can be exploited by extensions. It would be sad if that
> > >> opportunity is lost.
> >
> > Therefore I would suggest that an annotation starts with ! and
> > continues until but not including the first termination character.
> > Sensible termination characters might be comma, semicolon and end-of
> > line, allowing annotations for every item in a parameter/return list,
> > table constructor etc, but not encumbering the parser with a difficult
> > decision whether the annotation is done. In the contex of "local", an
> > empty annotation could default to 'toclose', so that "local a!, b, c!`
> > is all that one needs to supply "toclose" directives.
> >
>
> I would suggest that the syntax should be:
>
> <special character><optional name><optional table with only literal
> keys and values>

  So I'm looking at Java because they now have annotations.  >From a cursory
look, it appears the syntax to be:

        '@' ID

  So things like

        @Override
        @Deprecated
        @SuppressWarnings

  Annotations can be used at development time, compile time or runtime [1].
They can also be targetted towards a particular usage [2].  In Java, custom
annotations are handled with creating an interface.

  Assuming we use something similar to Java, the "to-close" annotation of
Lua 5.4 could probably look somthign like:

        @target(local)
        function @to_close(var)
          -- whatever this does
        end

        do
          local @to_close file = io.open(...)
          lcoal @to_close obj  = custom:blah(...)
          -- code code code
        end

> So assuming your choice of !, we could have:
>
> local a!
> local b !go_to_space { when=true }
> local c !array { 20 }

  Well ...

        local @my_annotation a
        local @go_to_space { when = true } b
        local @array(20) c

where '@array' is a special function called with certain parameters
depending on what it's targetting and any parameters given to it.  Too
magical?  Not magical enough?  Too much like Java? I don't know, it's
beginning to look a bit heavy to me.

  -spc

[1]     https://stackoverflow.com/questions/1372876/how-and-where-are-annotations-used-in-java

[2]     https://beginnersbook.com/2014/09/java-annotations/

Reply | Threaded
Open this post in threaded view
|

Re: Scoped variable *toclose syntax

Philippe Verdy
Also note that Java-style annotations are ALREADY used in Lua (just like other languages as well), and inserted inside Lua [[long comments]] (the @ is typically used as the first character inside that comment, and possibly several times in it, for example when detailing API parameters or uses, or to add @TODO or @LATER marks.).
They are also used within existing code repositories and with development tools and code generators. Their insertion within Lua comments does not break existing Lua parsers that can safely ignore/discard them if they don't need them.


Le jeu. 25 avr. 2019 à 23:59, Philippe Verdy <[hidden email]> a écrit :
Actually the form "@ID(parameters)" is used in Java, but the host language construct is not embedded inside the parameters of annotations (which are special constructor invocations that will instanciate separate objects, linked to the host language).
They have various use. The oldest form was for hinting the documentation (allowing inserting parsable comments that would be able to build API help files for example). But they evolved to be insertable in many places.
The rule (with which I agree) was to not modify the host language (or modify it a minimum: they are allowed almost anywhere a whitespace is allowed): you can always safely remove the whole annotation and parse the rest of the source code and compile it: all annotations are ignorable if needed and this should not make the program invalid and not working as expected (even if this causes some additional costs in memory or speed when not using them).
As well annotations should be complete bny themselves, not separated in multiple parts, so:
- the concept of using "!" as a prefix, and "{...}" is suffix for parameters is bad, and in fact it just adds various syntaxic ambiguities which are hard to resolve: this would prohibit using "{...}" table constructors in the host language)
- the same is true when using "@ID(..." for the annotation, followed by the host language constructor, and terminated just by a ")" would also be very quirky (dangerous I think)

My opinion is the the non-ambiguous form for annotation is: - "@ID" without parameters, or "@{ID, parameter...}". Then where we can place the whole annotation to attach it to the host language construct if left to specify (but these annotation tags can normally occur only between host tokens where spaces are allowed.
There may however be a need for annotation with long scopes (applying to multiple host-language constructs). Their form could be "@{begin ID, parameter...}"  for the begining of scope, and ""@{end ID}" for the end of scope (the additional reindication of the annotation ID may help diagnose non matching end).

Now we can as well use "{}" or "()" or "[]" or "<>" to bracket the annotation ID and its parameters in the same tag (either the opening tag or close tag). The initial bracket is disambiguated by the leading "@", which should be one of the character unassigned in Lua: "@" is perfect for that fine (I don't like using "!" because sooner or later it will become an interesting operator).

Note that annotations have their own namespace: they create a secondary metadata language. Binding the metadata language to a specific recognizer (with its own resource loaders and so on) can also use scopes. For this reason, it would be convenient to support not just the forms "@ID" or "@{ID, parameters...}" or "@{begin ID, parameters...}, or "@{end ID}", but also "@namespace:ID" or "@{namespace:ID, parameters...}" or "@{begin namespace:ID, parameters...}, or "@{end namespace:ID}".

The identifiers without namespaces should be reserved to Lua specification (or there would be a special namespace declaration to specify which namespace is attached to annotation IDs without namespaces). Other metatdata languages will use their own namespaces. Namespaces may be either "strings" (like "org.lua") or suites of IDs separated by dots or columns. I have no preference about it:

If there are syntaxic ambiguities, the simple form "@namespace:ID" or "@ID" should always be equivalent (as a shortcut notation) for the complete annotation tag using delimiting brackets like "@{ID}" or "@{namespace:ID}" (i.e. the "{}" brackets surrounding the annotation typename and its parameters is also correct without parameters).






Le jeu. 25 avr. 2019 à 18:29, Sean Conner <[hidden email]> a écrit :
It was thus said that the Great Dibyendu Majumdar once stated:
> On Thu, 25 Apr 2019 at 06:54, Dirk Laurie <[hidden email]> wrote:
> >
> > Op Do. 25 Apr. 2019 om 03:33 het Philippe Verdy <[hidden email]> geskryf:
> > > I also agree that the "*xxx" or "*(xxx, params...)" syntax is very fuzzy and will not work with the future evolution of annotations
> > >
> > > Le jeu. 25 avr. 2019 à 00:22, Dibyendu Majumdar <[hidden email]> a écrit :
> > >> I am writing about this again in the hope that it is not too late to
> > >> change the syntax of the *toclose attribute. I think there is an
> > >> opportunity to allow a general way of introducing annotations in Lua,
> > >> which can be exploited by extensions. It would be sad if that
> > >> opportunity is lost.
> >
> > Therefore I would suggest that an annotation starts with ! and
> > continues until but not including the first termination character.
> > Sensible termination characters might be comma, semicolon and end-of
> > line, allowing annotations for every item in a parameter/return list,
> > table constructor etc, but not encumbering the parser with a difficult
> > decision whether the annotation is done. In the contex of "local", an
> > empty annotation could default to 'toclose', so that "local a!, b, c!`
> > is all that one needs to supply "toclose" directives.
> >
>
> I would suggest that the syntax should be:
>
> <special character><optional name><optional table with only literal
> keys and values>

  So I'm looking at Java because they now have annotations.  >From a cursory
look, it appears the syntax to be:

        '@' ID

  So things like

        @Override
        @Deprecated
        @SuppressWarnings

  Annotations can be used at development time, compile time or runtime [1].
They can also be targetted towards a particular usage [2].  In Java, custom
annotations are handled with creating an interface.

  Assuming we use something similar to Java, the "to-close" annotation of
Lua 5.4 could probably look somthign like:

        @target(local)
        function @to_close(var)
          -- whatever this does
        end

        do
          local @to_close file = io.open(...)
          lcoal @to_close obj  = custom:blah(...)
          -- code code code
        end

> So assuming your choice of !, we could have:
>
> local a!
> local b !go_to_space { when=true }
> local c !array { 20 }

  Well ...

        local @my_annotation a
        local @go_to_space { when = true } b
        local @array(20) c

where '@array' is a special function called with certain parameters
depending on what it's targetting and any parameters given to it.  Too
magical?  Not magical enough?  Too much like Java? I don't know, it's
beginning to look a bit heavy to me.

  -spc

[1]     https://stackoverflow.com/questions/1372876/how-and-where-are-annotations-used-in-java

[2]     https://beginnersbook.com/2014/09/java-annotations/

12