# Rounding errors?

7 messages
Open this post in threaded view
|

## Rounding errors?

 ```Hi, I was wondering if someone could enlighten me as to why the following script does not produce the output I would expect: x=0 upper_bound = 1 while x
Open this post in threaded view
|

## RE: Rounding errors?

 ```Ian Latham wrote: Hi, I was wondering if someone could enlighten me as to why the following script does not produce the output I would expect: x=0 upper_bound = 1 while x
Open this post in threaded view
|

## Re: Rounding errors?

 ```Correct me if I'm wrong, but floating point values being an approximation, this is rather normal. Computers do not accurately store floating point values (at least on no architecture I've ever used). This is why banking software is written with integer values only. ~,^ On Wed, 2002-09-18 at 13:17, Ian Latham wrote: > Hi, I was wondering if someone could enlighten me as to why the > following script does not produce the output I would expect: > > x=0 > upper_bound = 1 > while x x=x+.01 > print(x) > end > > I would expect to see all the values in the interval (0,.5] listed out > at .01 increments to a precision of .01, but instead I get this: > > 0.01 > 0.02 > 0.03 > 0.04 > 0.05 > 0.06000000000000001 > 0.07000000000000001 > 0.08 > 0.09 > 0.09999999999999999 > 0.11 > 0.12 > 0.13 > 0.14 > 0.15 > 0.16 > 0.17 > 0.18 > 0.19 > 0.2 > 0.2100000000000001 > 0.2200000000000001 > 0.2300000000000001 > 0.2400000000000001 > 0.2500000000000001 > 0.2600000000000001 > 0.2700000000000001 > 0.2800000000000001 > 0.2900000000000001 > 0.3000000000000001 > 0.3100000000000001 > 0.3200000000000001 > 0.3300000000000001 > 0.3400000000000001 > 0.3500000000000001 > 0.3600000000000002 > 0.3700000000000002 > 0.3800000000000002 > 0.3900000000000002 > 0.4000000000000002 > 0.4100000000000002 > 0.4200000000000002 > 0.4300000000000002 > 0.4400000000000002 > 0.4500000000000002 > 0.4600000000000002 > 0.4700000000000003 > 0.4800000000000003 > 0.4900000000000003 > 0.5000000000000002 > > Can someone explain the extra decimals? Is this a rounding error? An > error in the internal representation of the float? A bug in the tostring > function? A bug in the addition code? Try changing upper_bound to 50 and > the errors get even more severe. > > > Thanks, > > Ian Latham > ```
Open this post in threaded view
|

## RE: Rounding errors?

 ```I don't know, I might expect some inaccuracy when I'm dealing with numbers that have 8 or 9 significant digits, but being unable to accurately represent a float to 2 decimal points seems ridiculously inaccurate to me. That's why I wonder if it's just a bug in the tostring function. Ian Latham -----Original Message----- From: Sean Middleditch [[hidden email]] Sent: Wednesday, September 18, 2002 10:23 AM To: Multiple recipients of list Subject: Re: Rounding errors? Correct me if I'm wrong, but floating point values being an approximation, this is rather normal. Computers do not accurately store floating point values (at least on no architecture I've ever used). This is why banking software is written with integer values only. ~,^ ```
Open this post in threaded view
|

## RE: Rounding errors?

 In reply to this post by Sean Middleditch ```> I don't know, I might expect some inaccuracy when I'm dealing with > numbers that have 8 or 9 significant digits, but being unable to > accurately represent a float to 2 decimal points seems ridiculously > inaccurate to me. That's why I wonder if it's just a bug in the tostring > function. You are probably working on a machine which stores floating point numbers in binary format, not in decimal format. :) If you tried the same thing with 1/256 instead of 1/100, it would be accurate; unfortunately, 0.01 does not have an accurate binary representation, just like 1/3 does not have an accurate decimal representation. You might also want to compare your results with the following: x=0 upper_bound = 100 while x
 In reply to this post by Ian Latham ```On Wed, Sep 18, 2002 at 10:33:54AM -0700, Ian Latham wrote: > I don't know, I might expect some inaccuracy when I'm dealing with > numbers that have 8 or 9 significant digits, but being unable to > accurately represent a float to 2 decimal points seems ridiculously > inaccurate to me. First of all, it is meaningless to talk about accuracy to decimal points. Floating point numbers are stored in binary, not decimal. All and any accuracy that you talk about can only refer to the binary digits. All the numbers that you displayed are perfectly accurate to the first 8 or 9 binary digits. That is not what your problem is. The problem is that these binary numbers cannot store an accurate representation of 0.01. Only sums of powers of two are accurate. In binary, 0.01 becomes 1/2^4 + 1/2^5 + 1/2^8 + 1/2^9 + 1/2^12 + 1/2^13 + ... which is an infinite fraction with period 4. As a result, the least significant digits of the sums will contain errors. When you convert them to decimal, and if the error propagates within the number of significant digits in the decimal conversion, they do show up in the decimal number. Worse, the error will propagate and eventually overwhelm the computation if you just keep summing long enough. This is not a bug in tostring(). Its conversion is accurate to many digits. The problem is that the underlying binary number is not. If you want "accuracy" to 2 decimal points, you have to round to the 2nd digit after the decimal point. However, note that doing so only hides the fundamental problem. The moral of the story is, if you need accurate numbers, either work only with sums of powers of two, or use exact arithmetic instead of floating point arithmetic. Regards - Christian ```
 In reply to this post by Sean Middleditch ```Ian, You're probably used to working in environments where this particular problem is masked by whatever the equivalent to "tostring" is in that system. Many systems automatically truncate a digit or so and round the result so your table would look correct. You should be able to acheive exactly the same effect by using format to print fewer digits. - Tom >>> [hidden email] 09/18/02 10:15 AM >>> Hi, I was wondering if someone could enlighten me as to why the following script does not produce the output I would expect: x=0 upper_bound = 1 while x>...SNIP... Thanks, Ian Latham ```