[ANN] mathx for Lua 5.3

classic Classic list List threaded Threaded
19 messages Options
Reply | Threaded
Open this post in threaded view
|

[ANN] mathx for Lua 5.3

Luiz Henrique de Figueiredo
mathx for Lua 5.3 is available at
        http://www.tecgraf.puc-rio.br/~lhf/ftp/lua/5.3/lmathx.tar.gz

This version replaces the provisional one written before Lua 5.3 was released.

mathx is a complete math library for Lua 5.3 with the functions
available in C99. It can replace the standard Lua math library, except
that mathx deals exclusively with floats and mathx does not open itself
inside math. The intent is that mathx is suitable for scientific work
(in as much as C99 is).

mathx includes these functions:

 acos cosh fmax lgamma remainder
 acosh deg fmin log round
 asin erf fmod log10 scalbn
 asinh erfc frexp log1p sin
 atan exp gamma log2 sinh
 atan2 exp2 hypot logb sqrt
 atanh expm1 isfinite modf tan
 cbrt fabs isinf nearbyint tanh
 ceil fdim isnan nextafter trunc
 copysign floor isnormal pow version
 cos fma ldexp rad inf, nan, pi

The names follow the ones in C99 closely, except that I've used gamma
instead of tgamma. Note that it's fabs, fmax, fmin instead of abs, max, min.
You can always rename any functions to names that you like.

As in math, atan can function as atan2 and log accepts an optional base.

mathx does not include these C99 functions and values:

 fpclassify huge random randomseed rint
 signbit

mathx does not include these that are in math because they deal with integers:

 maxinteger mininteger random randomseed tointeger
 type ult

There is no manual: see a C99 reference manual, e.g.
        http://en.wikipedia.org/wiki/C_mathematical_functions       

All feedback welcome. Enjoy.
--lhf

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] mathx for Lua 5.3

Dirk Laurie-2
2015-05-06 2:06 GMT+02:00 Luiz Henrique de Figueiredo <[hidden email]>:
> mathx for Lua 5.3 is available at
>         http://www.tecgraf.puc-rio.br/~lhf/ftp/lua/5.3/lmathx.tar.gz
>
> This version replaces the provisional one written before Lua 5.3 was released.

I followed the suggested directories for globally installed Lua,
typed 'make', was told by the compiler that I should recompile
with -fPIC (this happens with almost all the software I build and
I don't reason why, I just do), typed 'make clean' and 'make'.
The test results appeared with a mystererious 'nil' at the start
and end, which are produced by 'print(math.version)'.

Like previous versions of 'mathx', this places all its functions
directly into 'math', which is the table that "require mathx" returns.
Thus "package.load.math" and "package.load.mathx" are the
same. It is not quite clear from the README that this is what
happens.

Thanks, Luiz! I have immediately aliased "lua" to "lua -l mathx".

The interesting problem of marrying "complex" to "mathx",
complete with children like gamma for complex numbers,
now remains :-)

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] mathx for Lua 5.3

Luiz Henrique de Figueiredo
> typed 'make', was told by the compiler that I should recompile
> with -fPIC (this happens with almost all the software I build and
> I don't reason why, I just do)

Apparently this is needed for 64-bit Linux.

> The test results appeared with a mystererious 'nil' at the start
> and end, which are produced by 'print(math.version)'.

I don't see any nils. Are you sure you're running mathx for Lua 5.3?

Did the local test before installing work?
 
> Like previous versions of 'mathx', this places all its functions
> directly into 'math', which is the table that "require mathx" returns.

The current version does not mess with 'math'.
Are you sure you got the tarbal I uploaded yesterday?

Name                    Last update    Size    MD5 digest
5.3/lmathx.tar.gz       2015-05-05     3540    5b5bd22aec169712173d5e39ce759c34

> The interesting problem of marrying "complex" to "mathx",
> complete with children like gamma for complex numbers,
> now remains :-)

My lcomplex exposes C99 complex numbers and function to Lua but C99 does
not include cgamma. For that, you can try cephes: http://www.netlib.org/cephes/

I was thinking about writing a Lua binding for cephes, prompted by this
question:
        http://stackoverflow.com/questions/29706205/lua-operator-overloading-for-numbers

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] mathx for Lua 5.3

Rob Kendrick-2
On Wed, May 06, 2015 at 06:50:49AM -0300, Luiz Henrique de Figueiredo wrote:
> > typed 'make', was told by the compiler that I should recompile
> > with -fPIC (this happens with almost all the software I build and
> > I don't reason why, I just do)
>
> Apparently this is needed for 64-bit Linux.

Strictly speaking, -fPIC is needed for any shared library.  It just
happens that there's a happy coincidence that means you can often get
away without it on 32 bit x86.

B.

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] mathx for Lua 5.3

Luiz Henrique de Figueiredo
> Strictly speaking, -fPIC is needed for any shared library.  It just
> happens that there's a happy coincidence that means you can often get
> away without it on 32 bit x86.

Mac OS X does not need -fPIC, even in 64-bit machines.

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] mathx for Lua 5.3

Rob Kendrick-2
On Wed, May 06, 2015 at 07:34:18AM -0300, Luiz Henrique de Figueiredo wrote:
> > Strictly speaking, -fPIC is needed for any shared library.  It just
> > happens that there's a happy coincidence that means you can often get
> > away without it on 32 bit x86.
>
> Mac OS X does not need -fPIC, even in 64-bit machines.

My guess here is that this is either that it is enabled by default there
because they do ASLR, or it's some strange fall-out from OS X's bizarre
binary format :)

A shared library needs to be position independent code, that much is
non-negotiable, but some tooling probably handles it for you.  It
should always be safe to add.

B.

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] mathx for Lua 5.3

Eduardo Tongson
PIC is enabled by default on OS X.

From: http://docstore.mik.ua/orelly/unix3/mac/ch05_03.htm
[[ One difference between Darwin and many other Unix systems is that
no position-independent code (PIC) flag is needed, since it is the
default for Darwin. ]]


On Wed, May 6, 2015 at 6:45 PM, Rob Kendrick <[hidden email]> wrote:

> On Wed, May 06, 2015 at 07:34:18AM -0300, Luiz Henrique de Figueiredo wrote:
>> > Strictly speaking, -fPIC is needed for any shared library.  It just
>> > happens that there's a happy coincidence that means you can often get
>> > away without it on 32 bit x86.
>>
>> Mac OS X does not need -fPIC, even in 64-bit machines.
>
> My guess here is that this is either that it is enabled by default there
> because they do ASLR, or it's some strange fall-out from OS X's bizarre
> binary format :)
>
> A shared library needs to be position independent code, that much is
> non-negotiable, but some tooling probably handles it for you.  It
> should always be safe to add.
>
> B.
>

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] mathx for Lua 5.3

Dirk Laurie-2
In reply to this post by Luiz Henrique de Figueiredo
2015-05-06 11:50 GMT+02:00 Luiz Henrique de Figueiredo <[hidden email]>:

>> The test results appeared with a mystererious 'nil' at the start
>> and end, which are produced by 'print(math.version)'.
>
> I don't see any nils. Are you sure you're running mathx for Lua 5.3?
>
> Did the local test before installing work?

That was the local test. However, I note that "test.lua" just says

    local math=require"mathx"

without a previous

   package.cpath = "./?.so"

With the default cpath, the previously installed version is tested.
When I insert that line, the new version is tested, which indeed
does supply "version", and does not mess with "math".

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] mathx for Lua 5.3

Dirk Laurie-2
In reply to this post by Rob Kendrick-2
2015-05-06 12:45 GMT+02:00 Rob Kendrick <[hidden email]>:

> On Wed, May 06, 2015 at 07:34:18AM -0300, Luiz Henrique de Figueiredo wrote:
>> > Strictly speaking, -fPIC is needed for any shared library.  It just
>> > happens that there's a happy coincidence that means you can often get
>> > away without it on 32 bit x86.
>>
>> Mac OS X does not need -fPIC, even in 64-bit machines.
>
> My guess here is that this is either that it is enabled by default there
> because they do ASLR, or it's some strange fall-out from OS X's bizarre
> binary format :)
>
> A shared library needs to be position independent code, that much is
> non-negotiable, but some tooling probably handles it for you.  It
> should always be safe to add.

And the reason that -shared does not imply -fPIC is presumably
that -fpic might be adequate.

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] mathx for Lua 5.3

Luiz Henrique de Figueiredo
In reply to this post by Dirk Laurie-2
> With the default cpath, the previously installed version is tested.
> When I insert that line, the new version is tested, which indeed
> does supply "version", and does not mess with "math".

I'm glad this has been cleared up. Perhaps I should set cpath before running
the test. Thanks for bringing this up.

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] mathx for Lua 5.3

Ignacio Burgueño-2
In reply to this post by Dirk Laurie-2

On Wed, May 6, 2015 at 3:37 AM, Dirk Laurie <[hidden email]> wrote:

I followed the suggested directories for globally installed Lua,
typed 'make', was told by the compiler that I should recompile
with -fPIC (this happens with almost all the software I build and

An updated rockspec for lmathx has been submitted, packing this new release.

Eventually will be published here: https://luarocks.org/modules/hisham/lmathx

so you could do just `luarocks install lmathx`

Regards,
Ignacio

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] mathx for Lua 5.3

Luiz Henrique de Figueiredo
> An updated rockspec for lmathx has been submitted, packing this new release.

Thanks!

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] mathx for Lua 5.3

Alex Queiroz
In reply to this post by Luiz Henrique de Figueiredo
On 6 May 2015 at 12:34, Luiz Henrique de Figueiredo
<[hidden email]> wrote:
>> Strictly speaking, -fPIC is needed for any shared library.  It just
>> happens that there's a happy coincidence that means you can often get
>> away without it on 32 bit x86.
>
> Mac OS X does not need -fPIC, even in 64-bit machines.
>

It's always turned on.

--
-alex
http://unendli.ch/

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] mathx for Lua 5.3

Rena
In reply to this post by Dirk Laurie-2
On Wed, May 6, 2015 at 7:48 AM, Dirk Laurie <[hidden email]> wrote:

> 2015-05-06 12:45 GMT+02:00 Rob Kendrick <[hidden email]>:
>> On Wed, May 06, 2015 at 07:34:18AM -0300, Luiz Henrique de Figueiredo wrote:
>>> > Strictly speaking, -fPIC is needed for any shared library.  It just
>>> > happens that there's a happy coincidence that means you can often get
>>> > away without it on 32 bit x86.
>>>
>>> Mac OS X does not need -fPIC, even in 64-bit machines.
>>
>> My guess here is that this is either that it is enabled by default there
>> because they do ASLR, or it's some strange fall-out from OS X's bizarre
>> binary format :)
>>
>> A shared library needs to be position independent code, that much is
>> non-negotiable, but some tooling probably handles it for you.  It
>> should always be safe to add.
>
> And the reason that -shared does not imply -fPIC is presumably
> that -fpic might be adequate.
>

I recall having to rebuild Lua itself with -fPIC in order to be able
to use it in a shared library. IIRC it was dismissed as a non-issue
for whatever reason, I can't remember exactly...

--
Sent from my Game Boy.

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] mathx for Lua 5.3

Tim Hill
In reply to this post by Rob Kendrick-2

> On May 6, 2015, at 3:45 AM, Rob Kendrick <[hidden email]> wrote:
>
>
> A shared library needs to be position independent code, that much is
> non-negotiable, but some tooling probably handles it for you.  It
> should always be safe to add.
>
> B.
>

Not true at all. Shared libraries have existed for decades and most are NOT pic (Windows NT DLLs for example). Most loadable binary formats have fixup tables of some form that allow the loader to statically relocate the code/data at LOAD time. That’s not pic .. pic involves computing at RUN time addresses that are relative to the current location of the code, which allows it to be relocated on-the-fly. The main reason for using pic is to allow page-sharing across processes when shared libraries are loaded at different locations (typically to prevent various code exploits by malicious code).

—Tim


Reply | Threaded
Open this post in threaded view
|

Re: [ANN] mathx for Lua 5.3

William Ahern
In reply to this post by Rob Kendrick-2
On Wed, May 06, 2015 at 11:45:32AM +0100, Rob Kendrick wrote:

> On Wed, May 06, 2015 at 07:34:18AM -0300, Luiz Henrique de Figueiredo wrote:
> > > Strictly speaking, -fPIC is needed for any shared library.  It just
> > > happens that there's a happy coincidence that means you can often get
> > > away without it on 32 bit x86.
> >
> > Mac OS X does not need -fPIC, even in 64-bit machines.
>
> My guess here is that this is either that it is enabled by default there
> because they do ASLR, or it's some strange fall-out from OS X's bizarre
> binary format :)

Yes, PIC code is the default. OS X doesn't support static linking in as much
there are no static versions of any of the standard libraries--libcrt0,
libSystem.dylib, etc.

> A shared library needs to be position independent code, that much is
> non-negotiable, but some tooling probably handles it for you.  It
> should always be safe to add.

Most people consider Windows DLLs to be shared libraries, but they don't use
position independent code in that there's no GOT table or similar mechanism.
Rather, the linker literally has to hack the binary and the DLL in order to
change function addresses if there's a conflict. Sometimes it can't succeed.
This is one of the reasons for so-called DLL hell. It also means that
Windows often uses more memory than other systems because two processes
sharing the same DLL can't share the virtual memory pages for that DLL if
one of the processes had to rewrite the DLL at load time.

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] mathx for Lua 5.3

Tim Hill

On May 6, 2015, at 12:02 PM, William Ahern <[hidden email]> wrote:

Most people consider Windows DLLs to be shared libraries, but they don't use
position independent code in that there's no GOT table or similar mechanism.
Rather, the linker literally has to hack the binary and the DLL in order to
change function addresses if there's a conflict. Sometimes it can't succeed.
This is one of the reasons for so-called DLL hell. It also means that
Windows often uses more memory than other systems because two processes
sharing the same DLL can't share the virtual memory pages for that DLL if
one of the processes had to rewrite the DLL at load time.

it’s a bit unfair to call fixups a “hack”, and in fact they never fail (DLL hell has another cause entirely). Fixing up object code at load time is no different to the routine relocations linkers routinely do when merging compiled object files (for example, when loading static object code libraries).

To be fair to Microsoft, the standard set of Windows system DLLs always had carefully predefined load addresses so that collisions were rare (and when they did occur they could be statically relocated at load time), which typically meant they COULD be shared across processes via shared copy-on-write pages. It was only when random-address loading was added as an anti-virus measure that pic became desirable/necessary.

And it should be remembered that pic has runtime overhead (slower execution) as the compiler must generate extra code in many cases.

—Tim

Reply | Threaded
Open this post in threaded view
|

Re: [ANN] mathx for Lua 5.3

William Ahern
On Thu, May 07, 2015 at 12:54:23PM -0700, Tim Hill wrote:

>
> > On May 6, 2015, at 12:02 PM, William Ahern <[hidden email]> wrote:
> >
> > Most people consider Windows DLLs to be shared libraries, but they don't use
> > position independent code in that there's no GOT table or similar mechanism.
> > Rather, the linker literally has to hack the binary and the DLL in order to
> > change function addresses if there's a conflict. Sometimes it can't succeed.
> > This is one of the reasons for so-called DLL hell. It also means that
> > Windows often uses more memory than other systems because two processes
> > sharing the same DLL can't share the virtual memory pages for that DLL if
> > one of the processes had to rewrite the DLL at load time.
>
> it???s a bit unfair to call fixups a ???hack???

Oops. I didn't mean it pejoratively. It was just the first word that came to
mind. You're right in that it's not inherently broken.

> and in fact they never fail (DLL hell has another cause entirely).

I was thinking of failures like "Illegal System DLL Relocation", such as

        https://support.microsoft.com/en-us/kb/935448

but those are unique in that the DLLs were intentionally compiled to require
a specific base address.

I also thought the problem with mixing MVSCRT.dll versions[1] was related, but
I was wrong.

[1] See https://msdn.microsoft.com/en-us/library/ms235460.aspx. They're
working to change this. See
http://blogs.msdn.com/b/vcblog/archive/2014/06/10/the-great-crt-refactoring.aspx

> Fixing up object code at load time is no different to the routine
> relocations linkers routinely do when merging compiled object files (for
> example, when loading static object code libraries).
>
> To be fair to Microsoft, the standard set of Windows system DLLs always
> had carefully predefined load addresses so that collisions were rare (and
> when they did occur they could be statically relocated at load time),
> which typically meant they COULD be shared across processes via shared
> copy-on-write pages. It was only when random-address loading was added as
> an anti-virus measure that pic became desirable/necessary.
>
> And it should be remembered that pic has runtime overhead (slower
> execution) as the compiler must generate extra code in many cases.

Processors rely on deep pipelines and prefetching so much that the cost has
shrunk.

Plus, if a library isn't in shared memory, it's less likely to be in L1/2/3
cache after a context switch. So in addition to wasting memory, it could
also be slower even if none of the code is paged out. But I have no idea if
that is significant in real-world software. Probably not. (OTOH, Intel, AMD,
and ARM have gone to considerable lengths to preserve CPU caches across
context switches. And all three recently added tagged TLB tables, so virtual
address remapping is minimized.)


Reply | Threaded
Open this post in threaded view
|

Re: [ANN] mathx for Lua 5.3

Tim Hill

On May 7, 2015, at 7:51 PM, William Ahern <[hidden email]> wrote:

Plus, if a library isn't in shared memory, it's less likely to be in L1/2/3
cache after a context switch. So in addition to wasting memory, it could
also be slower even if none of the code is paged out. But I have no idea if
that is significant in real-world software. Probably not. (OTOH, Intel, AMD,
and ARM have gone to considerable lengths to preserve CPU caches across
context switches. And all three recently added tagged TLB tables, so virtual
address remapping is minimized.)

It’s very significant, at least on x86. Switching threads is very expensive not because of the context switch per-se (which is pretty fast), but the huge overhead with warming the various levels of caches .. it takes a significant amount of time for these to reach a reasonable hit rate after a switch. And yes, TLB tagging and large pages can help, but at the end of the day the culprit is of course our old “friend” .. software bloat. One reason Lua is so fast for a bytecode VM is that the entire thing can fit inside a modern L1/L2 cache, which means you are getting close to a microcoded machine given the aggressive way x86 processors work these days.

—Tim