lpeg.Carg() weirdness

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

lpeg.Carg() weirdness

Sean Conner

  I have an issue where I thought using lpeg.Carg() would solve the issue.
As a test, I wrote:

        lpeg = require "lpeg"
        pat  = lpeg.P"a" * lpeg.Carg(1)

        x = pat:match(pat)
        print(x)

And I get:

        lua: y.lua:4: reference to absent extra argument #1
        stack traceback:
                [C]: in function 'match'
                y.lua:4: in main chunk
                [C]: ?

Yet if I do:

        x = pat:match(pat,1,nil)
        print(x)

        nil

  So, what, exactly *is* the difference between the two calls?  I mean,
besides the explicit nil being given in the second call.

  -spc (Bummed about this ... )

Reply | Threaded
Open this post in threaded view
|

Re: lpeg.Carg() weirdness

Sean Conner
It was thus said that the Great Sean Conner once stated:
>
>   I have an issue where I thought using lpeg.Carg() would solve the issue.
> As a test, I wrote:
>
> lpeg = require "lpeg"
> pat  = lpeg.P"a" * lpeg.Carg(1)
>
> x = pat:match(pat)
> print(x)

  Oops.  I meant:

        x = pat:match("a")

  -spc (That's what I get for not doing cut-n-paste)


Reply | Threaded
Open this post in threaded view
|

Re: lpeg.Carg() weirdness

Parke
In reply to this post by Sean Conner
On Sat, Jun 24, 2017 at 4:10 PM, Sean Conner <[hidden email]> wrote:
>
>         x = pat:match('a')
>         x = pat:match('a',1,nil)
>
>   So, what, exactly *is* the difference between the two calls?  I mean,
> besides the explicit nil being given in the second call.

According to the docs, lpeg.Carg(n) produces "the value of the nth
extra argument to lpeg.match (matches the empty string)"

http://www.inf.puc-rio.br/~roberto/lpeg/lpeg.html#captures

Therefore:  Carg(n) requires that there be at least n extra arguments.
The difference between the two calls is that the first call provides
no extra arguments, whereas the second call provides the required one
extra argument.

> -spc (Bummed about this ... )

Why?  What are you trying to do?

-Parke

Reply | Threaded
Open this post in threaded view
|

Re: lpeg.Carg() weirdness

Sean Conner
It was thus said that the Great Parke once stated:

> On Sat, Jun 24, 2017 at 4:10 PM, Sean Conner <[hidden email]> wrote:
> >
> >         x = pat:match('a')
> >         x = pat:match('a',1,nil)
> >
> >   So, what, exactly *is* the difference between the two calls?  I mean,
> > besides the explicit nil being given in the second call.
>
> According to the docs, lpeg.Carg(n) produces "the value of the nth
> extra argument to lpeg.match (matches the empty string)"
>
> http://www.inf.puc-rio.br/~roberto/lpeg/lpeg.html#captures
>
> Therefore:  Carg(n) requires that there be at least n extra arguments.
> The difference between the two calls is that the first call provides
> no extra arguments, whereas the second call provides the required one
> extra argument.

  Well ... yes.  I understand all that.  But the third parameter, the
starting position, is optional.  I was just hoping that Carg(n) would return
nil if extra argument didn't exist.

> > -spc (Bummed about this ... )
>
> Why?  What are you trying to do?

  I have LPeg code to decode JSON data [1].  I received email from a person
about the handling of 'null'---that is, I return a Lua nil value, but this
person wanted a custom value.  I'm not completely swayed by the arguments
for a custom JSON null value, but I would like to provide the option for
such.  I thought a good way to handle a situation would be with Carg()---if
you want a custom null value, one could be provided, but it's optional.

  The intent was:

        doc = json:match(data)

would return a Lua nil for a JSON null, but if you wanted a custom value:

        doc = json:match(data,1,customjsonnullvalue)

  I'm just curious about the rational for the current behavior.  Heck, I
would even accept the following:

        x = lpeg.P"a" * lpeg.Carg(1)  -- if Carg(1) exists
          + lpeg.P"a" * lpeg.Cc(blah) -- and if it doesn't

  -spc (Still pondering how to deal with this)

[1] https://github.com/spc476/LPeg-Parsers/blob/master/json.lua


Reply | Threaded
Open this post in threaded view
|

Re: lpeg.Carg() weirdness

Parke
On Sat, Jun 24, 2017 at 9:22 PM, Sean Conner <[hidden email]> wrote:
>> Why?  What are you trying to do?
>
>   I have LPeg code to decode JSON data [1].  I received email from a person
> about the handling of 'null'---that is, I return a Lua nil value, but this
> person wanted a custom value.  I'm not completely swayed by the arguments
> for a custom JSON null value, but I would like to provide the option for
> such.  I thought a good way to handle a situation would be with Carg()---if
> you want a custom null value, one could be provided, but it's optional.

Why not use lpeg.Cc(values)?

Or possibly one of:
patt / number
patt / string
patt / function

http://www.inf.puc-rio.br/~roberto/lpeg/lpeg.html#captures

You would have to rebuild the pattern to change the value that would
represent null.

>   I'm just curious about the rational for the current behavior.

I find the current behavior to be consistent with other parts of Lua
that raise an error when the arguments provided to a function are
obviously wrong.

> -spc (Still pondering how to deal with this)

Use lpeg.Cc(values).  Or, if you don't want to rebuild the pattern,
use patt/function with a closure and with a monkey-patch a wrapper
around the match function to update the value in the closure.

-Parke

Reply | Threaded
Open this post in threaded view
|

Re: lpeg.Carg() weirdness

Sean Conner
It was thus said that the Great Parke once stated:

> On Sat, Jun 24, 2017 at 9:22 PM, Sean Conner <[hidden email]> wrote:
> >> Why?  What are you trying to do?
> >
> >   I have LPeg code to decode JSON data [1].  I received email from a person
> > about the handling of 'null'---that is, I return a Lua nil value, but this
> > person wanted a custom value.  I'm not completely swayed by the arguments
> > for a custom JSON null value, but I would like to provide the option for
> > such.  I thought a good way to handle a situation would be with Carg()---if
> > you want a custom null value, one could be provided, but it's optional.
>
> Why not use lpeg.Cc(values)?

  Given how I've implemented the module, I return an LPeg userdata, not a
table (following the other parsing modules [1]).  I'd like to keep that
format if at all possible (and it's seeming like I can't).  If require()
passed along any additional paramters to the module, then that could be used
to pass along a custom JSON null value, but alas, require() doesn't work
that way.

  So the only way to pass along an alternative null is through additional
arguments to lpeg.match(), which doesn't work the way I would like.

> You would have to rebuild the pattern to change the value that would
> represent null.

  Yes, I'm aware of that.

> >   I'm just curious about the rational for the current behavior.
>
> I find the current behavior to be consistent with other parts of Lua
> that raise an error when the arguments provided to a function are
> obviously wrong.

  A) there is no obvious value for additional arguments---they can be
anything, including nil (as I've shown) and B) it's the value meant to be
used as a capture.  And I can see two interpretations:

        1) pattern * Carg(1) returning the first additional argument, or nil
           if one isn't given.  

        2) pattern * Carg(1) failing (even if pattern does match) because
           the actual capture, the first additional argument, doesn't exist.

  What is happening though, is:

        pattern * Carg(1) crashing [2] because the additional argument
        doesn't exist.

  I can even see that being valid as a method of debugging (an incorrect
number of parameters were given to lpeg.match()) but the documentation
doesn't state that the parameter *has* to exist.  The description for
lpeg.match() doesn't even bring up that additional parameters are even
possible (only documenting the pattern, string and optional (note) initial
position).  And lpeg.Cc(nil) is a valid capture!

> > -spc (Still pondering how to deal with this)
>
> Use lpeg.Cc(values).  Or, if you don't want to rebuild the pattern,
> use patt/function with a closure and with a monkey-patch a wrapper
> around the match function to update the value in the closure.

  I already use the patt/function to return nil, so ...

        local json = require "org.conman.parsers.json"
       
        local customjsonnullvalue = function() end

        do
          local env = debug.getuservalue(json)
          for i,item in ipairs(env) do
            if type(item) == 'function' then
              local info = debug.getinfo(item)
              if info.linedefined == 139 then
                env[i] = function(_,position)
                  return position,customjsonnullvalue
                end
                break
              end
            end
          end
        end
       
        test = '[{"list": [1], "array":{"utf8": "\\u00e9\\u00ea", "next": null}}]'
        data = json:match(test)
        print(data[1].array.next)

        function: 0x9c2fd50

  -spc (So yes, monkeypatching does work, but ... )

[1] https://github.com/spc476/LPeg-Parsers

[2] Well, not a "crash" crash, but a catchable Lua error.

Reply | Threaded
Open this post in threaded view
|

Re: lpeg.Carg() weirdness

Dirk Laurie-2
2017-06-25 7:51 GMT+02:00 Sean Conner <[hidden email]>:

> It was thus said that the Great Parke once stated:
>> On Sat, Jun 24, 2017 at 9:22 PM, Sean Conner <[hidden email]> wrote:
>> >> Why?  What are you trying to do?
>> >
>> >   I have LPeg code to decode JSON data [1].  I received email from a person
>> > about the handling of 'null'---that is, I return a Lua nil value, but this
>> > person wanted a custom value.  I'm not completely swayed by the arguments
>> > for a custom JSON null value, but I would like to provide the option for
>> > such.  I thought a good way to handle a situation would be with Carg()---if
>> > you want a custom null value, one could be provided, but it's optional.
>>
>> Why not use lpeg.Cc(values)?
>
>   Given how I've implemented the module, I return an LPeg userdata, not a
> table (following the other parsing modules [1]).  I'd like to keep that
> format if at all possible (and it's seeming like I can't).  If require()
> passed along any additional paramters to the module, then that could be used
> to pass along a custom JSON null value, but alas, require() doesn't work
> that way.
>
>   So the only way to pass along an alternative null is through additional
> arguments to lpeg.match(), which doesn't work the way I would like.

One very easy method is to use the value of the global variable 'null'.
By default, it is nil, which will give the current behaviour. A user that
needs one can define it himself before invoking the LPeg pattern.

I've made that tiny change to your json.lua, and can report that it works.

-- Dirk

Reply | Threaded
Open this post in threaded view
|

Re: lpeg.Carg() weirdness

Sean Conner
It was thus said that the Great Dirk Laurie once stated:

> 2017-06-25 7:51 GMT+02:00 Sean Conner <[hidden email]>:
> > It was thus said that the Great Parke once stated:
> >> On Sat, Jun 24, 2017 at 9:22 PM, Sean Conner <[hidden email]> wrote:
> >> >> Why?  What are you trying to do?
> >> >
> >> >   I have LPeg code to decode JSON data [1].  I received email from a person
> >> > about the handling of 'null'---that is, I return a Lua nil value, but this
> >> > person wanted a custom value.  I'm not completely swayed by the arguments
> >> > for a custom JSON null value, but I would like to provide the option for
> >> > such.  I thought a good way to handle a situation would be with Carg()---if
> >> > you want a custom null value, one could be provided, but it's optional.
> >>
> >> Why not use lpeg.Cc(values)?
> >
> >   Given how I've implemented the module, I return an LPeg userdata, not a
> > table (following the other parsing modules [1]).  I'd like to keep that
> > format if at all possible (and it's seeming like I can't).  If require()
> > passed along any additional paramters to the module, then that could be used
> > to pass along a custom JSON null value, but alas, require() doesn't work
> > that way.
> >
> >   So the only way to pass along an alternative null is through additional
> > arguments to lpeg.match(), which doesn't work the way I would like.
>
> One very easy method is to use the value of the global variable 'null'.
> By default, it is nil, which will give the current behaviour. A user that
> needs one can define it himself before invoking the LPeg pattern.
>
> I've made that tiny change to your json.lua, and can report that it works.

  Oh.

  -spc (Why didn't I think of that?)


Reply | Threaded
Open this post in threaded view
|

Re: lpeg.Carg() weirdness

Parke
In reply to this post by Sean Conner
> It was thus said that the Great Parke once stated:
>> On Sat, Jun 24, 2017 at 9:22 PM, Sean Conner <[hidden email]> wrote:
>> I find the current behavior to be consistent with other parts of Lua
>> that raise an error when the arguments provided to a function are
>> obviously wrong.

On Sat, Jun 24, 2017 at 10:51 PM, Sean Conner <[hidden email]> wrote:
>   A) there is no obvious value for additional arguments---they can be
> anything, including nil (as I've shown) and B) it's the value meant to be
> used as a capture.

Perhaps I should have said "... that raise an error when the arguments
provided to a function are obviously wrong, including, as in this
case, when the number of arguments provided is obviously wrong."

-Parke