Floating point

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Floating point

Andrew Sidwell-2
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

I'm wondering if anyone knows of a way to make sure a value is only to
two decimal points?  I only ask because the obvious solution using
(math.ceil(n * 100) == (n * 100)) doesn't work:

> print(math.floor(3466) == math.floor(34.66 * 100))
false

It would also be quite nice to know why this doesn't work.

Cheers,
Andrew Sidwell
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.3rc2 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFEK5+EnTTdvBpPNwsRAu5SAJ9k2M4GQmFcB+V6hPxiYbZ8DcT3dACg4Ufr
MDFPs4Q9Xf1XFiutl11ys/w=
=n+Yh
-----END PGP SIGNATURE-----
Reply | Threaded
Open this post in threaded view
|

Re: Floating point

Ben Sunshine-Hill
On 3/30/06, Andrew Sidwell <[hidden email]> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Hi,
>
> I'm wondering if anyone knows of a way to make sure a value is only to
> two decimal points?  I only ask because the obvious solution using
> (math.ceil(n * 100) == (n * 100)) doesn't work:
>
> > print(math.floor(3466) == math.floor(34.66 * 100))
> false
>
> It would also be quite nice to know why this doesn't work.

It's because most fractional numbers, including most of those with two
decimal places, cannot be exactly represented in floating point form.
For more information, read "What Every Computer Scientist Should Know
About Floating-Point Arithmetic", located here:

http://docs.sun.com/source/806-3568/ncg_goldberg.html

Ben
Reply | Threaded
Open this post in threaded view
|

Re: Floating point

Doug Rogers-4
In reply to this post by Andrew Sidwell-2
Andrew Sidwell wrote:

> I'm wondering if anyone knows of a way to make sure a value is only to
> two decimal points?  I only ask because the obvious solution using
> (math.ceil(n * 100) == (n * 100)) doesn't work:
>> print(math.floor(3466) == math.floor(34.66 * 100))
> false
> It would also be quite nice to know why this doesn't work.

Ben's right - it's not easy to tell because floating point can only
represent a fixed number of multiples of (possibly negative) powers of
two. So things like 400.5 (801 * 2^-1) and 73.0625 (1169 * 2^-4), can be
represented exactly, but not 51.2 (256 * 5^-1).

You can make the statement above return true by rounding instead of
flooring:

 > math.round = function(x) return math.floor(x + 0.5) end
 > print(math.floor(3466) == math.round(34.66 * 100))
true

You can also determine if a value is within a particular epsilon of a
multiple of 0.01:

 > function ismul(x,base,epsilon)
 >>   epsilon = epsilon or 0.0001
 >>   return math.abs(x/base - math.floor(x/base)) <= epsilon
 >> end
 > print(ismul(45.02, 0.01))
true
 > print(ismul(45.019, 0.01))
false
 > print(ismul(45.020005, 0.01))
false
 > print(ismul(45.0200005, 0.01))
true

Doug

--
--__-__-____------_--_-_-_-___-___-____-_--_-___--____
Doug Rogers - ICI - V:703.893.2007x220 www.innocon.com
-_-_--_------____-_-_-___-_--___-_-___-_-_---_--_-__-_