How math.fmod work

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

How math.fmod work

Albert Chan

>> Lua 5.3.4  Copyright (C) 1994-2017, PUC-Rio
>>> print(1e97 % 97)
>> 3.0
> Lua 5.1/5.2 computed the module (a%b) with (a - floor(a/b) * b),
> which may introduce other rouding errors. Lua 5.3 computes the module
> with 'fmod'; at least in my machine, the only error is the one in the
> representation of 1e97 itself (see my other messages).
> -- Roberto

I wondered how math.fmod work without arbitrary precision math.
Is above example just lucky ?

Just find out ... it is not luck !
If there is no strtod error, math.fmod is correct.

The algorithm is elegant, without multiplication  or division: