LPeg 0.10

classic Classic list List threaded Threaded
23 messages Options
12
Reply | Threaded
Open this post in threaded view
|

LPeg 0.10

Roberto Ierusalimschy
LPeg version 0.10 is out:

  http://www.inf.puc-rio.br/~roberto/lpeg/


* Changes from version 0.9 to 0.10
  -------------------------------
  + backtrack stack has configurable size
  + better error messages
  + Notation for non-terminals in 're' back to A instead o <A>
  + experimental look-behind pattern
  + support for external extensions
  + works with Lua 5.2
  + consumes less C stack


-- Roberto

Reply | Threaded
Open this post in threaded view
|

Re: LPeg 0.10

Fabio Mascarenhas
I have uploaded LPEG 0.10 rocks to the LuaRocks repository, so you can
install (or upgrade) using LuaRocks:

luarocks install lpeg

--
Fabio Mascarenhas


On Wed, Nov 3, 2010 at 3:50 PM, Roberto Ierusalimschy
<[hidden email]> wrote:

> LPeg version 0.10 is out:
>
>  http://www.inf.puc-rio.br/~roberto/lpeg/
>
>
> * Changes from version 0.9 to 0.10
>  -------------------------------
>  + backtrack stack has configurable size
>  + better error messages
>  + Notation for non-terminals in 're' back to A instead o <A>
>  + experimental look-behind pattern
>  + support for external extensions
>  + works with Lua 5.2
>  + consumes less C stack
>
>
> -- Roberto
>
>

Reply | Threaded
Open this post in threaded view
|

Re: LPeg 0.10

Thomas Harning Jr.
On Wed, Nov 3, 2010 at 2:12 PM, Fabio Mascarenhas <[hidden email]> wrote:
> I have uploaded LPEG 0.10 rocks to the LuaRocks repository, so you can
> install (or upgrade) using LuaRocks:
>
> luarocks install lpeg
Thanks Roberto for the new release and thanks Fabio for the quick
rocks release :)
Was able to track down a bug in my LuaJSON code caused by naive use of
a version comparator, now I use function presence in the LPeg table to
determine what functions to use to manage compatibility with LPeg <
0.9

To those following my LuaJSON project, I plan on releasing new
versions with the fix of my 3 trees (1.0.x, 1.1.x, and 1.2.x) with the
fix already committed to all 3 branches.
--
Thomas Harning Jr.

Reply | Threaded
Open this post in threaded view
|

Re: LPeg 0.10

Tony Finch
In reply to this post by Roberto Ierusalimschy
On Wed, 3 Nov 2010, Roberto Ierusalimschy wrote:

> LPeg version 0.10 is out:
>
>   http://www.inf.puc-rio.br/~roberto/lpeg/
>
> * Changes from version 0.9 to 0.10
>   -------------------------------
>   + backtrack stack has configurable size
>   + better error messages
>   + Notation for non-terminals in 're' back to A instead o <A>
>   + experimental look-behind pattern
>   + support for external extensions
>   + works with Lua 5.2
>   + consumes less C stack

Cool! I'm particularly pleased that most of my patch is now
unnecessary.

Could you please re-generate the tarball to unpack into an lpeg-0.10
subdirectory?

I see that the re module is now using a locale-dependent definition of
space for parsing grammars, which is opposite to the locale-independent
trend of Lua-5.2.

These are the extra operator overloads I like to use with lpeg.

   do
      local mt = getmetatable(lpeg.P(0))
      mt.__mod = lpeg.Cf
      mt.__concat = function (a,b)
         return a * lpeg.P{ b + 1 * lpeg.V(1) }
      end
   end

And this patch adds a similar search operator to the re syntax.

--- a/re.lua
+++ b/re.lua
@@ -159,6 +159,8 @@ local exp = m.P{ "Exp",
         * (#exp_follow + patt_error);
   Prefix = "&" * S * m.V"Prefix" / mt.__len
          + "!" * S * m.V"Prefix" / mt.__unm
+         + ".." * S * m.V"Prefix" /
+                   function (p) return m.P{ p + 1 * m.V(1) } end
          + m.V"Suffix";
   Suffix = m.Cf(m.V"Primary" * S *
           ( ( m.P"+" * m.Cc(1, mt.__pow)

Tony.
--
f.anthony.n.finch  <[hidden email]>  http://dotat.at/
HUMBER THAMES DOVER WIGHT PORTLAND: NORTH BACKING WEST OR NORTHWEST, 5 TO 7,
DECREASING 4 OR 5, OCCASIONALLY 6 LATER IN HUMBER AND THAMES. MODERATE OR
ROUGH. RAIN THEN FAIR. GOOD.

Reply | Threaded
Open this post in threaded view
|

Re: LPeg 0.10

Roberto Ierusalimschy
> Could you please re-generate the tarball to unpack into an lpeg-0.10
> subdirectory?

Done.


> I see that the re module is now using a locale-dependent definition of
> space for parsing grammars, which is opposite to the locale-independent
> trend of Lua-5.2.

This is nothing ideological. (Actually I did not really think about it ;)
I hope locales do not change too much about what they consider spaces...

-- Roberto

Reply | Threaded
Open this post in threaded view
|

Re: LPeg 0.10

Roberto Ierusalimschy
> > I see that the re module is now using a locale-dependent definition of
> > space for parsing grammars, which is opposite to the locale-independent
> > trend of Lua-5.2.
>
> This is nothing ideological. (Actually I did not really think about it ;)
> I hope locales do not change too much about what they consider spaces...

Just in case, I changed it to m.S(" \f\n\r\t\v").

-- Roberto

Reply | Threaded
Open this post in threaded view
|

Re: LPeg 0.10

Tony Finch
In reply to this post by Roberto Ierusalimschy
On 4 Nov 2010, at 19:39, Roberto Ierusalimschy <[hidden email]> wrote:

>> Could you please re-generate the tarball to unpack into an lpeg-0.10
>> subdirectory?
>
> Done.

Magic, thanks!

>
>> I see that the re module is now using a locale-dependent definition of
>> space for parsing grammars, which is opposite to the locale-independent
>> trend of Lua-5.2.
>
> This is nothing ideological. (Actually I did not really think about it ;)
> I hope locales do not change too much about what they consider spaces...

Yes, me too! Thanks for fixing lpeg's re syntax. I have unhappily resorted to explicit coding of what is space in the past because the documentation for the regex implementation I was using was not clear about its definition of \s.

Tony.
--
f.anthony.n.finch  <[hidden email]>  http://dotat.at/
Reply | Threaded
Open this post in threaded view
|

Re: LPeg 0.10

Roberto Ierusalimschy
In reply to this post by Roberto Ierusalimschy
I just updated some little details in the documentation of LPeg
0.10. (It is not very polite to do these changes without changing file
names, but this is a pre-1.0 version...)

In particular, I added a line in HISTORY about a small incompatibility
with 0.9 that I had forgot to mention. In 0.10, "and" predicates
("positive look-aheads") do not keep captures as they did in 0.9.
This restores the equivalence "&e == !!e" of original PEG and solves
a subtle bug in 0.9:

  x = re.compile[[   {~ (&(. ([a-z]* -> '*')) ([a-z]+ -> '+') ' '*)* ~}  ]]
  print(x:match"alo alo")


-- Roberto

Reply | Threaded
Open this post in threaded view
|

Re: LPeg 0.10

Nick Gammon

On 06/11/2010, at 12:07 AM, Roberto Ierusalimschy wrote:

> In particular, I added a line in HISTORY about a small incompatibility
> with 0.9 that I had forgot to mention. In 0.10, "and" predicates
> ("positive look-aheads") do not keep captures as they did in 0.9.
> This restores the equivalence "&e == !!e" of original PEG and solves
> a subtle bug in 0.9:

Thanks for the update Roberto!

Small typo in the amended history:

>   - "and" predicates do not keep catures

That should be "captures".



Reply | Threaded
Open this post in threaded view
|

Re: LPeg 0.10

Roberto Ierusalimschy
> Small typo in the amended history:
>
> >   - "and" predicates do not keep catures
>
> That should be "captures".

Thanks.

-- Roberto

Reply | Threaded
Open this post in threaded view
|

Re: LPeg 0.10

Nick Gammon
In reply to this post by Roberto Ierusalimschy

On 04/11/2010, at 4:50 AM, Roberto Ierusalimschy wrote:

> LPeg version 0.10 is out:
>
>  http://www.inf.puc-rio.br/~roberto/lpeg/
>
>
> * Changes from version 0.9 to 0.10


If I may just point out, doing either a string or numeric comparison, version 0.10 is a lower version than 0.9.


print ("0.10" > "0.9") --> false
print (0.10 > 0.9)  --> false



Reply | Threaded
Open this post in threaded view
|

Re: LPeg 0.10

Miles Bader-2
Nick Gammon <[hidden email]> writes:

>> * Changes from version 0.9 to 0.10
>
> If I may just point out, doing either a string or numeric comparison,
> version 0.10 is a lower version than 0.9.
>
> print ("0.10" > "0.9") --> false
> print (0.10 > 0.9)  --> false

Nonetheless, it's a very standard format for version numbers.

When comparing version strings, you should compare runs of digits in
isolation, as numbers.  Most package-handling programs should do that
automatically (and glibc even contains a variant of strcmp that does it
-- "strverscmp").

-Miles

--
Accord, n. Harmony.

Reply | Threaded
Open this post in threaded view
|

Re: LPeg 0.10

Norbert Kiesel
On Wed, 2010-11-10 at 11:36 +0900, Miles Bader wrote:

> Nick Gammon <[hidden email]> writes:
>
> >> * Changes from version 0.9 to 0.10
> >
> > If I may just point out, doing either a string or numeric comparison,
> > version 0.10 is a lower version than 0.9.
> >
> > print ("0.10" > "0.9") --> false
> > print (0.10 > 0.9)  --> false
>
> Nonetheless, it's a very standard format for version numbers.
>
> When comparing version strings, you should compare runs of digits in
> isolation, as numbers.  Most package-handling programs should do that
> automatically (and glibc even contains a variant of strcmp that does it
> -- "strverscmp").
>
> -Miles
>

Something like this

function strverscmp(a, b)
   if type(a) == 'string' and type(b) == 'string' then
      local aa = a:gmatch('%d+')
      local bb = b:gmatch('%d+')
      while true do
local na = aa()
local nb = bb()
if not na and not nb then return 0 end
if not na then return -1 end
if not nb then return 1 end
local delta = tonumber(na) - tonumber(nb)
if delta < 0 then return -1 end
if delta > 0 then return 1 end
      end
   elseif type(a) == 'number' and type(b) == 'number' then
      return a - b
   end
   return nil
end

function test(a, b)
   print(a, b, strverscmp(a, b))
end

test('0.9', '0.10')
test('0.9', '0.9')
test('0.9.1', '0.10')
test('0.9.1', '0.9.0')
test('0.9.1', '0.9.2')
test(0.9, 0.9)

</nk>



Reply | Threaded
Open this post in threaded view
|

Re: LPeg 0.10

Rena
On Tue, Nov 9, 2010 at 19:57, Norbert Kiesel <[hidden email]> wrote:

> On Wed, 2010-11-10 at 11:36 +0900, Miles Bader wrote:
>> Nick Gammon <[hidden email]> writes:
>>
>> >> * Changes from version 0.9 to 0.10
>> >
>> > If I may just point out, doing either a string or numeric comparison,
>> > version 0.10 is a lower version than 0.9.
>> >
>> > print ("0.10" > "0.9") --> false
>> > print (0.10 > 0.9)  --> false
>>
>> Nonetheless, it's a very standard format for version numbers.
>>
>> When comparing version strings, you should compare runs of digits in
>> isolation, as numbers.  Most package-handling programs should do that
>> automatically (and glibc even contains a variant of strcmp that does it
>> -- "strverscmp").
>>
>> -Miles
>>
>
> Something like this
>
> function strverscmp(a, b)
>   if type(a) == 'string' and type(b) == 'string' then
>      local aa = a:gmatch('%d+')
>      local bb = b:gmatch('%d+')
>      while true do
> local na = aa()
> local nb = bb()
> if not na and not nb then return 0 end
> if not na then return -1 end
> if not nb then return 1 end
> local delta = tonumber(na) - tonumber(nb)
> if delta < 0 then return -1 end
> if delta > 0 then return 1 end
>      end
>   elseif type(a) == 'number' and type(b) == 'number' then
>      return a - b
>   end
>   return nil
> end
>
> function test(a, b)
>   print(a, b, strverscmp(a, b))
> end
>
> test('0.9', '0.10')
> test('0.9', '0.9')
> test('0.9.1', '0.10')
> test('0.9.1', '0.9.0')
> test('0.9.1', '0.9.2')
> test(0.9, 0.9)
>
> </nk>
>
>
>
>

Just for what it's worth, assert() is better for testing:
assert(strverscmp('0.9', '0.10') < 0)
(Also it's for this reason I like to use a datecode for my version
numbers. 20 (or more) bits: yyyyyyymmmmdddddiiii (i=index).)

--
Sent from my toaster.

Reply | Threaded
Open this post in threaded view
|

Re: LPeg 0.10

Nick Gammon
In reply to this post by Miles Bader-2

On 10/11/2010, at 1:36 PM, Miles Bader wrote:

> Nonetheless, it's a very standard format for version numbers.

Yes I understand.

However the documentation for lpeg.version says:

> lpeg.version ()
>
> Returns a string with the running version of LPeg.

This is inconclusive about how to compare versions.

My example:

  print ("0.10" > "0.9") --> false

Norbert Kiesel's comparison:

  test('0.9', '0.10') --> 0.9 0.10 -1

We get different results, and just assuming that you should compare in a certain way is likely to cause grief, as indeed it did recently with the LuaJSON library.


If I may suggest, the documentation could be amended to say something like:

"Versions should be compared, not as pure strings (nor as a decimal number), but using standard "version" comparisons where each group of digits is to be considered individually. More information: http://semver.org/"

By way of comparison, SQLite3 provides both a string version (printable form) and a numeric version along the lines of 3007004 for version 3.7.4.  The numeric version can be directly compared without any mucking around with special comparison functions.


- Nick


Reply | Threaded
Open this post in threaded view
|

Re: LPeg 0.10

Miles Bader-2
Nick Gammon <[hidden email]> writes:
>> Returns a string with the running version of LPeg.
>
> This is inconclusive about how to compare versions.
> If I may suggest, the documentation could be amended to say something like:

I dunno -- once they look at the version string they'd like to compare
against (almost every use of this is likely to be for comparison with a
specific version), it seems pretty clear what type of format is being
used...

-Miles

--
Hers, pron. His.

Reply | Threaded
Open this post in threaded view
|

Re: LPeg 0.10

Wesley Smith
On Wed, Nov 10, 2010 at 4:34 AM, Miles Bader <[hidden email]> wrote:

> Nick Gammon <[hidden email]> writes:
>>> Returns a string with the running version of LPeg.
>>
>> This is inconclusive about how to compare versions.
>> If I may suggest, the documentation could be amended to say something like:
>
> I dunno -- once they look at the version string they'd like to compare
> against (almost every use of this is likely to be for comparison with a
> specific version), it seems pretty clear what type of format is being
> used...


No matter the format, it could be spelled out at least:

Returns a string with the running version of LPeg for the form "major.minor"

Reply | Threaded
Open this post in threaded view
|

Re: LPeg 0.10

Philippe Lhoste
In reply to this post by Nick Gammon
On 09/11/2010 22:40, Nick Gammon wrote:
> If I may just point out, doing either a string or numeric comparison, version 0.10 is a lower version than 0.9.

Why people see these numbers as floating point numbers? I don't believe they have been
designed this way.
What about 2.8.1 (Scala)? And 7.0.5730.13 see on Internet Explorer?

Of course, there are precedents inducing confusion, like Microsoft jumping Windows from
3.1 to 3.11...

--
Philippe Lhoste
--  (near) Paris -- France
--  http://Phi.Lho.free.fr
--  --  --  --  --  --  --  --  --  --  --  --  --  --


Reply | Threaded
Open this post in threaded view
|

Re: LPeg 0.10

Philippe Lhoste
In reply to this post by Wesley Smith
On 10/11/2010 05:56, Wesley Smith wrote:
> On Wed, Nov 10, 2010 at 4:34 AM, Miles Bader<[hidden email]>  wrote:
>> Nick Gammon<[hidden email]>  writes:
>>>> Returns a string with the running version of LPeg.

Indeed, it is a string, in classical format.

>>> This is inconclusive about how to compare versions.

Use common sense? (And tradition.)

> No matter the format, it could be spelled out at least:
>
> Returns a string with the running version of LPeg for the form "major.minor"

It won't hurt to be precise, indeed. There is a strong tradition, but we all know
exceptions to it... :-)

--
Philippe Lhoste
--  (near) Paris -- France
--  http://Phi.Lho.free.fr
--  --  --  --  --  --  --  --  --  --  --  --  --  --


Reply | Threaded
Open this post in threaded view
|

Re: LPeg 0.10

Matthew Wild
On 10 November 2010 13:15, Philippe Lhoste <[hidden email]> wrote:
> On 10/11/2010 05:56, Wesley Smith wrote:
>>
>> On Wed, Nov 10, 2010 at 4:34 AM, Miles Bader<[hidden email]>  wrote:
>>>
>>> Nick Gammon<[hidden email]>  writes:
>>>>>
>>>>> Returns a string with the running version of LPeg.

>>>> This is inconclusive about how to compare versions.
>
> Use common sense? (And tradition.)
>
>> No matter the format, it could be spelled out at least:
>>
>> Returns a string with the running version of LPeg for the form
>> "major.minor"
>
> It won't hurt to be precise, indeed. There is a strong tradition, but we all
> know exceptions to it... :-)
>

Just to note... this is why I always use version numbers of the form
x.y.z in my projects. It clarifies to most people that it is a version
string, and not a decimal number.

Matthew

12