How math.fmod work

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

How math.fmod work

Albert Chan

>> Lua 5.3.4  Copyright (C) 1994-2017 Lua.org, PUC-Rio
>>> print(1e97 % 97)
>> 3.0
>
> Lua 5.1/5.2 computed the module (a%b) with (a - floor(a/b) * b),
> which may introduce other rouding errors. Lua 5.3 computes the module
> with 'fmod'; at least in my machine, the only error is the one in the
> representation of 1e97 itself (see my other messages).
>
> -- Roberto

I wondered how math.fmod work without arbitrary precision math.
Is above example just lucky ?

Just find out ... it is not luck !
If there is no strtod error, math.fmod is correct.

The algorithm is elegant, without multiplication  or division:

https://eli.thegreenplace.net/2018/computing-remainders-by-doubling/