Browse thread
[Caml-list] float pretty-printing precision, once more.
[
Home
]
[ Index:
by date
|
by threads
]
[ Message by date: previous | next ] [ Message in thread: previous | next ] [ Thread: previous | next ]
[ Message by date: previous | next ] [ Message in thread: previous | next ] [ Thread: previous | next ]
| Date: | -- (:) |
| From: | Xavier Leroy <xavier.leroy@i...> |
| Subject: | Re: [Caml-list] float pretty-printing precision, once more. |
> [string_of_float loses precision on floating-point numbers] > In the current working version (3.06+18), the precision used by > string_of_float has been increased to 17 digits. Yes, this was my feeble attempt to work around the precision loss using only what is available, i.e. sprintf. However, as you say: > However, it has the unfortunate side-effect of revealing this > awful truth about FP numbers: many "interesting" numbers are > impossible to represent in floating-point. For example: > > # 0.1;; > - : float = 0.10000000000000001 > > There is no floating-point number equal to 0.1 and the > best approximation you can get is 0.10000000000000001. If that was really the best approximation, there would be nothing to argue. But both 0.10000000000000001 and 0.1 read back as identical floats, so the latter should be printed instead, but sprintf (on Linux at least) is too stupid to realize this. - Xavier Leroy ------------------- To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/ Beginner's list: http://groups.yahoo.com/group/ocaml_beginners