Newsgroups: comp.lang.smalltalk
Path: cantaloupe.srv.cs.cmu.edu!europa.chnt.gtegsc.com!news.sprintlink.net!noc.netcom.net!netcomsv!uu3news.netcom.com!netcomsv!uucp3.netcom.com!medicus!billf
From: billf@medicus.com (Bill Foote)
Subject: Re: Rounding Error in Digitalk 3.0.1
Message-ID: <1995Jun20.182314.24571@medicus.com>
Organization: Medicus Systems Corp.
References: <3s6aa2$ssn@raffles.technet.sg>
Date: Tue, 20 Jun 1995 18:23:14 GMT
Lines: 53

In article <3s6aa2$ssn@raffles.technet.sg> UserName <user@technet.sg> writes:
>I have had this experience in Digitalk 3.0.1
>
>0.025 printRounded:2 => 0.03
>1.025 printRounded:2 => 1.02
>
>Looks like a bug to me. 
>Has anyone encountered this shared with me how to fix it ?
>
>Looks like a bug to me. 

No, it's not.

The thing to realize is that 1.025 is an infinitely repeating fraction in
floating point binary, so it cannot be exactly represented in the machine's
"float" type.  (25/1000 asFloat) (i.e. Smalltalk's 0.025) happens to be 
represented as a binary floating point number that is >= the real 0.025, and
(1025/1000 asFloat) happens to be represented as a number that is < 1.025.

You can see this same behavior with C (the following is taken from g++ running
on an Intel machine):

    Script started on Tue Jun 20 10:42:13 1995

    medicus:/u/billf/tmp% cat foo.cc


    #include <stdio.h>

    main()

    {
	double a = 0.025;
	double b = 1.025;

	printf("%4.2lf, %4.2lf\n", a, b);
    }
    medicus:/u/billf/tmp% g++ foo.cc
    medicus:/u/billf/tmp% a.out
    0.03, 1.02

    script done on Tue Jun 20 10:42:33 1995


The moral?  Be very careful when using floats, and never, *ever* use them
to represent money.  You have been warned :-)

--
Bill Foote                | Adde parvum parvo magnus acervus ecrit.
billf@medicus.com         | [Add little to little and there will be a big pile]
Medicus Systems           |    -- Ovid, via Frederick P. Brooks, Jr.
Alameda, CA USA           |

