Comment about IEEE floating-point rounding... (was: Setting Float Precision in Lua.c)

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Comment about IEEE floating-point rounding... (was: Setting Float Precision in Lua.c)

sur-behoffski
G'Day,

[Sorry for not replying within the relevant thread, but I read the list
use digest mode.  Apologies if this offends anyone.]

I managed publish an article in the November 1997 edition of Dr. Dobb's
Journal.  In that same edition, was an extended interview with Prof.
William Kahan, entitled "A Conversation with William Kahan":

         http://collaboration.cmc.ec.gc.ca/science/rpn/biblio/ddj/Website/articles/DDJ/1997/9711/9711a/9711a.htm

IMHO, the entire Kahan article is well worth reading.

I'll pick out a couple comments about the difference between abstract/simple
formulas, versus complicated, but numerically stable algorithms, and end
with a longish extract about different rounding modes that the standard
lets the program(mer) select, and why being able to select the rounding
mode at runtime may be valuable:

--

         [...]
         DDJ: What should we programmers be learning?

         WK: What you should be learning are things from numerical-analysis
         classes, such as why the less accurate of two ways of calculating a
         function may be perfectly satisfactory for engineering work.  But I
         also tell programmers that argument [such as on the phone with
         customers] takes time and money, and since we know a way to do it
         which will circumvent the necessity for argument, by all means do
         it that way, if you can. You can dispense with the necessity of
         trying to persuade somebody of something which is, in fact, true --
         that two methods are just as good as one another, but which they
         have every right to disbelieve. There are all sorts of folks who
         tell you, "It's okay, this is just as good," and are wrong.

         Let's calculate accuracy well out of proportion with what anybody
         actually needs. They don't need it, but it doesn't matter. We'll do
         it that way and then there won't be any argument.

         [...]

--

About the difference between "intellectually economical" and simple:

--
         [...]

         DDJ: So what should our industry be doing?

         WK: I think it was Churchill who once said something about the
         essence of leadership being figuring out where people are going to
         go anyway and getting there ahead of them. The industry, in its
         own best interests, has to take a role that leads the customers,
         but not by too much. When it comes to numerical things, part of
         leadership is to provide an intellectually economical numerical
         ambiance.

         "Intellectually economical" doesn't necessarily mean simple. We
         have to understand the various arenas in which people want to
         depend upon computers. We have to understand them in some ways
         better than the customers. The customers don't always understand
         the issues because they are preoccupied with what matters to
         them, not with the design and construction of the systems upon
         which they depend.

         It's like being engaged in sewer repair. The art of sewer repair
         is to do it in such a fashion so as not to oblige others to think
         about it at all. If you have to think about it, the sewer
         repairmen haven't been doing their job.

         DDJ: So we programmers are not doing our job?

         WK: That's correct. It is straightforward to implement numerics
         better. I've written about it in a number of places. Look at my
         article about miscalculating the area and angles of a triangle at:

                 http://http.cs.berkeley.edu/~wkahan/Triangle.ps

         There's an abstract there that points to a number of
         misconceptions or superstitions. It doesn't seem to me to be all
         that difficult for the compiler community to serve these needs.

         If the industry gets far enough down a certain path, it can't
         turn, never mind go back! [...]

--

And, finally, some specific comments on why multiple different rounding
modes were mandated in the IEEE standard:

--

         WK: [...] But if you get rid of a [preprocessor] expression when
         someone has written it in cold blood, you're changing the
         semantics of his program.

         Similar are things like constant folding. You (the optimizer)
         look at an expression and decide, "Oh, this is a constant, so I
         can compute it at compile time." But maybe that expression was
         designed for a different rounding mode at run time than the one
         which prevails at compile time. There are reasons to change the
         rounding mode without mentioning it.

         There's all sorts of stuff in the IEEE Standard designed to help
         ordinary people do things like diagnose what may be screwing
         them in a module they got from someone else. Perhaps there's an
         algorithm that is pretty good for almost all data except yours.
         It doesn't know you personally, you understand! Your data just
         happens to be the kind that that particular algorithm doesn't
         like. How do you find out if this is the case?

         One way is to change the rounding mode, which will create
         different rounding errors. This identifies modules where it
         might be worth your time to investigate. If you didn't have this
         ability to change the rounding mode, you might not have any other
         way to identify which module among many supplied by
         multiple vendors was the likely candidate for further
         investigation.

         In any army, there's a soldier who doesn't get the message.  DEC
         Alpha decided to put the rounding mode not in a control word
         alone, but also in the opcode. That means that when you compile,
         you can compile the rounding modes in such a way that they don't
         respond to a change of control word setting, and therefore, when
         you rerun the code with the changed control word, you're going
         to get the same results as before.

         It isn't that this was a bad idea, to put the rounding mode in
         the opcode as well as the control word. The mistake was that they
         put the default rounding mode into the opcode. The default should
         have been left in the control word.

         [...]

--

I have to sheepishly admit that 99.9% of Numerical Methods are above my
head, but I'm glad that I've come to know about Prof. Kahan's work and
his very careful, but also accessible, writings about computerised
numerical computation.

I hope that this post is of value in the context of the recent/ongoing
discussion(s) about rounding modes, and their interaction with proposals
to set rounding modes for single/double/extended precision in Lua.

cheers,

sur-behoffski (Brenton Hoff)
programmer, Grouse Software

Reply | Threaded
Open this post in threaded view
|

Re: Comment about IEEE floating-point rounding... (was: Setting Float Precision in Lua.c)

Albert Chan
On Jun 10, 2018, at 8:54 AM, sur-behoffski <[hidden email]> wrote:

> I managed publish an article in the November 1997 edition of Dr. Dobb's
> Journal.  In that same edition, was an extended interview with Prof.
> William Kahan, entitled "A Conversation with William Kahan":
>        http://collaboration.cmc.ec.gc.ca/science/rpn/biblio/ddj/Website/articles/DDJ/1997/9711/9711a/9711a.htm
>
> IMHO, the entire Kahan article is well worth reading.

I thoroughly enjoy the read, highly recommended.

Kahan excerpt from above:

"I don't feel that the architects of Java are wicked so much as premature
to think that they can write a universal language for every man to write
everything to run everywhere. That isn't going to happen, it's going to be
a turf war -- Bill Joy against Bill Gates. It's a lousy reason to write a
language, to write it in order to precipitate a turf war to bring things into
a territory more advantageous to Sun than the current situation."

I am reading another Kakan article on Java and IEEE (not yet finished).
It seems there is good reason even for sign of zero. (+/- 0.0)

IEEE people certainly know what they are doing ...

https://people.eecs.berkeley.edu/~wkahan/JAVAhurt.pdf

p.s.
my naming of thread maybe misleading.
I was only proposing a *default* precision setting.


Reply | Threaded
Open this post in threaded view
|

Re: Comment about IEEE floating-point rounding... (was: Setting Float Precision in Lua.c)

Hugo Musso Gualandi
In reply to this post by sur-behoffski
Thanks for the references, I found them very interesting.

By the way, the link to Prof. Kahan's article on miscalculating the
areas and angles of triangles has moved. It can now be found at:

    http://http.cs.berkeley.edu/~wkahan/Triangle.pdf

-- Hugo

Reply | Threaded
Open this post in threaded view
|

Re: Comment about IEEE floating-point rounding... (was: Setting Float Precision in Lua.c)

Hugo Musso Gualandi
In reply to this post by sur-behoffski
Thanks for the references, I found them very interesting.

By the way, the link to Prof. Kahan's article on miscalculating the
areas and angles of triangles has moved. It can now be found at:

    http://http.cs.berkeley.edu/~wkahan/Triangle.pdf

-- Hugo

Reply | Threaded
Open this post in threaded view
|

Re: Comment about IEEE floating-point rounding... (was: Setting Float Precision in Lua.c)

Albert Chan

> By the way, the link to Prof. Kahan's article on miscalculating the
> areas and angles of triangles has moved. It can now be found at:
>
>    http://http.cs.berkeley.edu/~wkahan/Triangle.pdf
>
> -- Hugo

Thanks.  I just think of another way to reduce error,
for calculating the area of the needle-like triangle.

I use another IEEE double property.
If value is integer 53 bits or less, add, subtract, multiply is *exact*
Sorry if it sound obvious to everyone, it is stated in PIL on Numbers

Example, Above article, Table 1 (line 3) sides of triangle:
a, b, c = 1e5, 99999.99979, 0.00029

This is the classic Heron formula for triangle area:
Area = sqrt(s * (s-a) * (s-b) * (s-c)),  where s = (a + b + c) / 2

Instead of plugging in the numbers, just scaled it so all values
are integers.  Scale by 1e5:  a,b,c = 1e10, 9999999979, 29

Area (scaled) = sqrt(10000000004 * 4 * 25 * 9999999975)
= 99999999895

Undo scaling: Area = 99999999895 / 1e10 = 9.9999999895

BTW, the article stated that heron formula return area = 17.6
I cannot reproduce it (even adjusting precision with gmpy2)

Anyone still have the old HP-15C calculator ?


Reply | Threaded
Open this post in threaded view
|

Re: Comment about IEEE floating-point rounding... (was: Setting Float Precision in Lua.c)

Albert Chan
Thanks Hartmut for the HP-15C test. 
It seems HP-15C does *decimal* math, 10 significant digits, half-round-up

s = (a + b + c) / 2
= (1e5 + 99999.99979 + 0.00029) / 2
= (199999.9998 + 0.00029) / 2   -- round up
= 200000.0001 / 2                         -- round up again
= 100000.0001                               -- half-round-up

I have never seen a calculator like this ...
With this, I match your area result of 17.60681685 exactly.

On Jun 12, 2018, at 5:38 PM, Hartmut Henkel <[hidden email]> wrote:

On Tue, 12 Jun 2018, Albert Chan wrote:

BTW, the article stated that heron formula return area = 17.6
I cannot reproduce it (even adjusting precision with gmpy2)

Anyone still have the old HP-15C calculator ?

yes :-) Keys still without bouncing, bright LCD, that's quality.

FWIW, my HP-15C tells 17.60681685 using the first formula in ยง1 on the
3rd example in Table 1, just tried. Else i have not followed the thread.

The 17.6 error comes from s: When it's 100000.000039999998989515
(Lua), Delta is 9.999999809638328685. But the HP-15C shows
 s=<a dir="ltr" href="tel:100000.0001" x-apple-data-detectors="true" x-apple-data-detectors-type="telephone" x-apple-data-detectors-result="2" style="background-color: rgba(255, 255, 255, 0);">100000.0001, then it's 17.6068..., which you can confirm wit Lua.

 Best Regards, Hartmut