Rounding errors?

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Rounding errors?

Ian Latham
Hi, I was wondering if someone could enlighten me as to why the
following script does not produce the output I would expect:

x=0
upper_bound = 1
while x<upper_bound do
x=x+.01
print(x)
end

I would expect to see all the values in the interval (0,.5] listed out
at .01 increments to a precision of .01, but instead I get this:

0.01
0.02
0.03
0.04
0.05
0.06000000000000001
0.07000000000000001
0.08
0.09
0.09999999999999999
0.11
0.12
0.13
0.14
0.15
0.16
0.17
0.18
0.19
0.2
0.2100000000000001
0.2200000000000001
0.2300000000000001
0.2400000000000001
0.2500000000000001
0.2600000000000001
0.2700000000000001
0.2800000000000001
0.2900000000000001
0.3000000000000001
0.3100000000000001
0.3200000000000001
0.3300000000000001
0.3400000000000001
0.3500000000000001
0.3600000000000002
0.3700000000000002
0.3800000000000002
0.3900000000000002
0.4000000000000002
0.4100000000000002
0.4200000000000002
0.4300000000000002
0.4400000000000002
0.4500000000000002
0.4600000000000002
0.4700000000000003
0.4800000000000003
0.4900000000000003
0.5000000000000002

Can someone explain the extra decimals? Is this a rounding error? An
error in the internal representation of the float? A bug in the tostring
function? A bug in the addition code? Try changing upper_bound to 50 and
the errors get even more severe.


Thanks,

Ian Latham


Reply | Threaded
Open this post in threaded view
|

RE: Rounding errors?

Ian Latham
Ian Latham wrote:

Hi, I was wondering if someone could enlighten me as to why the
following script does not produce the output I would expect:

x=0
upper_bound = 1
while x<upper_bound do
x=x+.01
print(x)
end

snip/

Oops, upper_bound should have been set to .5 in that sample script, but
you get the idea.

-Ian Latham




Reply | Threaded
Open this post in threaded view
|

Re: Rounding errors?

Sean Middleditch
Correct me if I'm wrong, but floating point values being an
approximation, this is rather normal.  Computers do not accurately store
floating point values (at least on no architecture I've ever used). 
This is why banking software is written with integer values only.  ~,^

On Wed, 2002-09-18 at 13:17, Ian Latham wrote:
> Hi, I was wondering if someone could enlighten me as to why the
> following script does not produce the output I would expect:
> 
> x=0
> upper_bound = 1
> while x<upper_bound do
> x=x+.01
> print(x)
> end
> 
> I would expect to see all the values in the interval (0,.5] listed out
> at .01 increments to a precision of .01, but instead I get this:
> 
> 0.01
> 0.02
> 0.03
> 0.04
> 0.05
> 0.06000000000000001
> 0.07000000000000001
> 0.08
> 0.09
> 0.09999999999999999
> 0.11
> 0.12
> 0.13
> 0.14
> 0.15
> 0.16
> 0.17
> 0.18
> 0.19
> 0.2
> 0.2100000000000001
> 0.2200000000000001
> 0.2300000000000001
> 0.2400000000000001
> 0.2500000000000001
> 0.2600000000000001
> 0.2700000000000001
> 0.2800000000000001
> 0.2900000000000001
> 0.3000000000000001
> 0.3100000000000001
> 0.3200000000000001
> 0.3300000000000001
> 0.3400000000000001
> 0.3500000000000001
> 0.3600000000000002
> 0.3700000000000002
> 0.3800000000000002
> 0.3900000000000002
> 0.4000000000000002
> 0.4100000000000002
> 0.4200000000000002
> 0.4300000000000002
> 0.4400000000000002
> 0.4500000000000002
> 0.4600000000000002
> 0.4700000000000003
> 0.4800000000000003
> 0.4900000000000003
> 0.5000000000000002
> 
> Can someone explain the extra decimals? Is this a rounding error? An
> error in the internal representation of the float? A bug in the tostring
> function? A bug in the addition code? Try changing upper_bound to 50 and
> the errors get even more severe.
> 
> 
> Thanks,
> 
> Ian Latham
> 




Reply | Threaded
Open this post in threaded view
|

RE: Rounding errors?

Ian Latham
I don't know, I might expect some inaccuracy when I'm dealing with
numbers that have 8 or 9 significant digits, but being unable to
accurately represent a float to 2 decimal points seems ridiculously
inaccurate to me. That's why I wonder if it's just a bug in the tostring
function.

Ian Latham

-----Original Message-----
From: Sean Middleditch [[hidden email]] 
Sent: Wednesday, September 18, 2002 10:23 AM
To: Multiple recipients of list
Subject: Re: Rounding errors?

Correct me if I'm wrong, but floating point values being an
approximation, this is rather normal.  Computers do not accurately store
floating point values (at least on no architecture I've ever used). 
This is why banking software is written with integer values only.  ~,^

<snip>

Reply | Threaded
Open this post in threaded view
|

RE: Rounding errors?

RLak
In reply to this post by Sean Middleditch
> I don't know, I might expect some inaccuracy when I'm dealing with
> numbers that have 8 or 9 significant digits, but being unable to
> accurately represent a float to 2 decimal points seems ridiculously
> inaccurate to me. That's why I wonder if it's just a bug in the tostring
> function.

You are probably working on a machine which stores floating point numbers
in binary format, not in decimal format. :) If you tried the same thing
with 1/256 instead of 1/100, it would be accurate; unfortunately, 0.01 does
not have an accurate binary representation, just like 1/3 does not have an
accurate decimal representation.

You might also want to compare your results with the following:

x=0
upper_bound = 100
while x<upper_bound do
  x=x+1
  print(x/100)
end

In general, Fortran-derived DO loops do suffer from floating-point drift on
many architectures (although there are known solutions to this problem, and
it will work correctly on Intel hardware if the control variable is kept in
a floating-point register, because the additional 16 bits correct for the
rounding errors.) In general, it is better to translate such loops into
something like the following:

local repetitions = floor( (upper_bound - lower_bound) / increment )
for i = 0, repetitions do
   local x = lower_bound + i * increment
   print(x)
end

The above code is not perfect (it needs to check for the case where
upper_bound is less than lower_bound, for example) but it is indicative.
There will be objections about the "extra work" done by the computation of
x; in most modern architectures, however, there is a "floating multiply and
add" primitive which does the work quite rapidly. (Lua does not use this
primitive because it is, sadly, not available in standard C.)


Reply | Threaded
Open this post in threaded view
|

Re: Rounding errors?

Christian Vogler-2
In reply to this post by Ian Latham
On Wed, Sep 18, 2002 at 10:33:54AM -0700, Ian Latham wrote:
> I don't know, I might expect some inaccuracy when I'm dealing with
> numbers that have 8 or 9 significant digits, but being unable to
> accurately represent a float to 2 decimal points seems ridiculously
> inaccurate to me.

First of all, it is meaningless to talk about accuracy to decimal
points. Floating point numbers are stored in binary, not decimal. All
and any accuracy that you talk about can only refer to the binary
digits.

All the numbers that you displayed are perfectly accurate to the first
8 or 9 binary digits. That is not what your problem is. The problem is
that these binary numbers cannot store an accurate representation of
0.01. Only sums of powers of two are accurate. In binary, 0.01 becomes

1/2^4 + 1/2^5 + 1/2^8 + 1/2^9 + 1/2^12 + 1/2^13 + ...

which is an infinite fraction with period 4. 

As a result, the least significant digits of the sums will contain
errors. When you convert them to decimal, and if the error propagates
within the number of significant digits in the decimal conversion,
they do show up in the decimal number. Worse, the error will propagate
and eventually overwhelm the computation if you just keep summing long
enough.

This is not a bug in tostring(). Its conversion is accurate to many
digits. The problem is that the underlying binary number is not. If
you want "accuracy" to 2 decimal points, you have to round to the 2nd
digit after the decimal point. 

However, note that doing so only hides the fundamental problem. The
moral of the story is, if you need accurate numbers, either work only
with sums of powers of two, or use exact arithmetic instead of
floating point arithmetic.

Regards
- Christian

Reply | Threaded
Open this post in threaded view
|

Re: Rounding errors?

Thomas Wrensch-2
In reply to this post by Sean Middleditch
Ian,

You're probably used to working in environments where this particular problem is masked by whatever the equivalent to "tostring" is in that system. Many systems   automatically truncate a digit or so and round the result so your table would look correct.

You should be able to acheive exactly the same effect by using format to print fewer digits.

  - Tom



>>> [hidden email] 09/18/02 10:15 AM >>>
Hi, I was wondering if someone could enlighten me as to why the
following script does not produce the output I would expect:

x=0
upper_bound = 1
while x<upper_bound do
x=x+.01
print(x)
end

I would expect to see all the values in the interval (0,.5] listed out
at .01 increments to a precision of .01, but instead I get this:

0.01
0.02
0.03
0.04
0.05
0.06000000000000001
>>...SNIP...
Thanks,

Ian Latham