parse hex numerals [PATCH]

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

parse hex numerals [PATCH]

lego-2
Hi,

FYC

the following patch against lua-5.1-rc3 allows the lexer to recognize
hexadecimal numerals in the form 0xa1234

The one thing I realy have problems with lua is the lack of hex numerals.

I've been working into embeding lua to ethereal, not being able to use
hex numerals makes some things very hard to do, try expressing an IP
mask (e.g. 255.255.255.0) as a decimal (4294967040) instead of hex
(0xffffff00) and you will understand what I intend.

I would love not to have to write a preprocessor that does it. So I
decided to try to have this included into lua-5.1 before I get into
writing a preprocessor.

Thanks for your consideration.

Luis

--
This information is top security. When you have read it, destroy yourself.
-- Marshall McLuhan

llex.c.hexnumeral.patch (3K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: parse hex numerals [PATCH]

lego-2
I'm very sorry but I think I've sent in an old version of the patch.

The one I got in was before testing it using float and as a matter of
fact it does not increase the exponential when the 24th bit of the
mantissa is set.

Sorry.


On 2/9/06, LEGO <[hidden email]> wrote:

> Hi,
>
> FYC
>
> the following patch against lua-5.1-rc3 allows the lexer to recognize
> hexadecimal numerals in the form 0xa1234
>
> The one thing I realy have problems with lua is the lack of hex numerals.
>
> I've been working into embeding lua to ethereal, not being able to use
> hex numerals makes some things very hard to do, try expressing an IP
> mask (e.g. 255.255.255.0) as a decimal (4294967040) instead of hex
> (0xffffff00) and you will understand what I intend.
>
> I would love not to have to write a preprocessor that does it. So I
> decided to try to have this included into lua-5.1 before I get into
> writing a preprocessor.
>
> Thanks for your consideration.
>
> Luis
>
> --
> This information is top security. When you have read it, destroy yourself.
> -- Marshall McLuhan
>
>
>

--
This information is top security. When you have read it, destroy yourself.
-- Marshall McLuhan

llex.c.hexnum.patch (2K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: parse hex numerals [PATCH]

Mike Pall-2-2
In reply to this post by lego-2
Hi,

LEGO wrote:
> the following patch against lua-5.1-rc3 allows the lexer to recognize
> hexadecimal numerals in the form 0xa1234

Umm ... have you checked whether plain Lua already supports
this? :-)

$ lua51rc3 -e 'print(0xff)'
255

See:

  http://lua-users.org/lists/lua-l/2006-01/msg00464.html

for an explanation and a (IMHO) better/simpler patch which makes
hex numbers work with non-C99 libraries, too (MSVC for Windows).

PS: Reread the CPP docs, too. '#if foo == bar' is always true.
    CPP has no idea of C types and doesn't do string compares.
    Unknown tokens evaluate to 0.

Bye,
     Mike
Reply | Threaded
Open this post in threaded view
|

Re: parse hex numerals [PATCH]

lego-2
I used  5.0.2 so i noticed there the lack of hex, My instinct told me
to look for hex in the lexer of the new release... as read_numeral()
there would not take an hex I assumed lua didn't so I changed it...

Good thing there's hex already, It was fun anyway going back to the
basics so it wasn't a complete loss of time.

Luis.

On 2/9/06, Mike Pall <[hidden email]> wrote:

> Hi,
>
> LEGO wrote:
> > the following patch against lua-5.1-rc3 allows the lexer to recognize
> > hexadecimal numerals in the form 0xa1234
>
> Umm ... have you checked whether plain Lua already supports
> this? :-)
>
> $ lua51rc3 -e 'print(0xff)'
> 255
>
> See:
>
>   http://lua-users.org/lists/lua-l/2006-01/msg00464.html
>
> for an explanation and a (IMHO) better/simpler patch which makes
> hex numbers work with non-C99 libraries, too (MSVC for Windows).
>
> PS: Reread the CPP docs, too. '#if foo == bar' is always true.
>     CPP has no idea of C types and doesn't do string compares.
>     Unknown tokens evaluate to 0.
>
> Bye,
>      Mike
>


--
This information is top security. When you have read it, destroy yourself.
-- Marshall McLuhan
Reply | Threaded
Open this post in threaded view
|

Re: parse hex numerals [PATCH]

Roberto Ierusalimschy
In reply to this post by Mike Pall-2-2
> for an explanation and a (IMHO) better/simpler patch which makes
> hex numbers work with non-C99 libraries, too (MSVC for Windows).

Do we really need a patch? Can't we just use configuration? Something
like this (untested...):

  #define lua_str2number(s,p)  \
     (s[0]=='0' && s[1] == 'x') ? strtoul(s[2], p, 16) : strtod((s), (p))

-- Roberto
Reply | Threaded
Open this post in threaded view
|

Re: parse hex numerals [PATCH]

lego-2
On 2/9/06, Roberto Ierusalimschy <[hidden email]> wrote:

> > for an explanation and a (IMHO) better/simpler patch which makes
> > hex numbers work with non-C99 libraries, too (MSVC for Windows).
>
> Do we really need a patch? Can't we just use configuration? Something
> like this (untested...):
>
>   #define lua_str2number(s,p)  \
>      (s[0]=='0' && s[1] == 'x') ? strtoul(s[2], p, 16) : strtod((s), (p))
>
> -- Roberto
>

Actually I did not knew but 5.1 does hex numerals already so that
patch is useless.

Not being familiar with lua's codebase, I expected it to be in llex.c
and as it wasn't there I just added it there.

I'm stiil trying to figure out when and where the recognition of the
hex numeral is made.

Luis.

--
This information is top security. When you have read it, destroy yourself.
-- Marshall McLuhan
Reply | Threaded
Open this post in threaded view
|

Re: parse hex numerals [PATCH]

Mike Pall-2-2
In reply to this post by Roberto Ierusalimschy
Hi,

Roberto Ierusalimschy wrote:
> > for an explanation and a (IMHO) better/simpler patch which makes
> > hex numbers work with non-C99 libraries, too (MSVC for Windows).
>
> Do we really need a patch? Can't we just use configuration? Something
> like this (untested...):
>
>   #define lua_str2number(s,p)  \
>      (s[0]=='0' && s[1] == 'x') ? strtoul(s[2], p, 16) : strtod((s), (p))

This doesn't work with leading whitespace or signs. This isn't a
problem for the parser, but consistency with tonumber() is very
desirable.

And ?: with different result types is a problem, too. AFAIK
strtoul() already takes care of the 0x in base 16. Also, strtod()
does a better job with hex numbers on C99 systems, so you want to
try this first.

Ok, I've got another idea (tested):

---- luaconf.h ----
...
@@ lua_xstr2number converts a hex string to a number.
...
#define lua_xstr2number(s,p)    strtoul((s), (p), 16)
...

---- lobject.c ----

int luaO_str2d (const char *s, lua_Number *result) {
  char *endptr;
  *result = lua_str2number(s, &endptr);
  for (;;) {
    if (endptr != s) {  /* at least one char was accepted */
      if (*endptr == '\0') return 1;  /* most common case */
      while (isspace(cast(unsigned char, *endptr))) endptr++;
      if (*endptr == '\0') return 1;  /* ok, only trailing spaces */
    }
    if (s == NULL) break;
    *result = cast_num(lua_xstr2number(s, &endptr));
    s = NULL;
  }
  return 0;  /* conversion failed. */
}

This is really only a five line change, except the logic had to
be reordered. It's a bit ugly but avoids coding the trailer check
twice. Non-C99 compliant systems will parse " +0xff" up to " +0",
so just checking for endptr != s doesn't work.

Another nice thing about it: for integer lua_Number one can use:
  #define lua_str2number(s,p)   strtoul((s), (p), 10)
Avoids the octal number pitfall and hex numbers still work
because of the fallback.

Testcases:

assert(0x10 == 16)
assert(-0x10 == -16)
assert(0xef == 239)
assert(-0xef == -239)

assert(tonumber("0xef") == 239)
assert(tonumber("+0xef") == 239)
assert(tonumber("-0xef") == -239)
assert(tonumber("\t 0xef") == 239)
assert(tonumber("0xef\t ") == 239)
assert(tonumber("\t +0xef\t ") == 239)
assert(tonumber("\t -0xef\t ") == -239)

assert(0x7fffffff == 2147483647)
assert(0xffffffff == 4294967295)               -- Really -1 with int32.
assert(tonumber("-0x7fffffff") == -2147483647)
assert(tonumber("-0xffffffff") == -4294967295) -- Really +1 with int32.

Bye,
     Mike
Reply | Threaded
Open this post in threaded view
|

Re[2]: parse hex numerals [PATCH]

Cloud Wu
In reply to this post by Roberto Ierusalimschy

Hello Roberto,

Thursday, February 9, 2006, 8:18:04 PM, you wrote:


>> for an explanation and a (IMHO) better/simpler patch which makes
>> hex numbers work with non-C99 libraries, too (MSVC for Windows).

RI> Do we really need a patch? Can't we just use configuration? Something
RI> like this (untested...):

RI>   #define lua_str2number(s,p)  \
RI>      (s[0]=='0' && s[1] == 'x') ? strtoul(s[2], p, 16) : strtod((s), (p))

RI> -- Roberto

it works (tested :)

#define lua_str2number(s,p) \
        ((s[0]=='0' && s[1] == 'x') ? strtoul(s+2, p, 16) : strtod((s), (p)))

--
Best regards,
 cloudwu                            mailto:[hidden email]
            http://blog.codingnow.com

[君子有容人之量, 小人存忌妒之心]


Reply | Threaded
Open this post in threaded view
|

Re: parse hex numerals [PATCH]

Roberto Ierusalimschy
In reply to this post by Mike Pall-2-2
What about this?

   char *endptr;
   *result = lua_str2number(s, &endptr);
   if (endptr == s) return 0;  /* conversion failed */
+  if (*endptr == 'x') *result = strtoul(s, &endptr, 16);
   if (*endptr == '\0') return 1;  /* most common case */
   while (isspace(cast(unsigned char, *endptr))) endptr++;
   if (*endptr != '\0') return 0;  /* invalid trailing characters? */

-- Roberto
Reply | Threaded
Open this post in threaded view
|

Re: parse hex numerals [PATCH]

Todor Totev

Is there a chance a official support for hexadecimal numbers to be added  
for lua 5.1 release?
I'll be very happy because now I patch it and it'll be very nice for me to  
not
deviate from official sources.
regards,
Todor

On Thu, 09 Feb 2006 17:11:19 +0200, Roberto Ierusalimschy  
<[hidden email]> wrote:

> What about this?
>
>    char *endptr;
>    *result = lua_str2number(s, &endptr);
>    if (endptr == s) return 0;  /* conversion failed */
> +  if (*endptr == 'x') *result = strtoul(s, &endptr, 16);
>    if (*endptr == '\0') return 1;  /* most common case */
>    while (isspace(cast(unsigned char, *endptr))) endptr++;
>    if (*endptr != '\0') return 0;  /* invalid trailing characters? */
>
> -- Roberto
>


Reply | Threaded
Open this post in threaded view
|

Re: parse hex numerals [PATCH]

Mike Pall-2-2
In reply to this post by Roberto Ierusalimschy
Hi,

Roberto Ierusalimschy wrote:
> What about this?
>
> +  if (*endptr == 'x') *result = strtoul(s, &endptr, 16);

Cute. :-) Improvement:

  if (*endptr == 'x' || *endptr == 'X')
    *result = cast_num(strtoul(s, &endptr, 16));

Tested with original MSVCRT DLLs and works fine. Alas, tonumber()
converts negative hex numbers to large positive doubles. :-/
But the approach in the lexer works. So -0xff == -255, but
tonumber"-0xff" == 4294967041 with lua_Number = double.
Everything's ok for lua_Number = int due to wrap-around.

Let's just pretend signed hex numbers are not too relevant for
tonumber().

Bye,
     Mike