Browse thread
[Caml-list] float type with better precision than the default
[
Home
]
[ Index:
by date
|
by threads
]
[ Message by date: previous | next ] [ Message in thread: previous | next ] [ Thread: previous | next ]
[ Message by date: previous | next ] [ Message in thread: previous | next ] [ Thread: previous | next ]
| Date: | -- (:) |
| From: | Remi Vanicat <remi.vanicat@l...> |
| Subject: | Re: [Caml-list] float type with better precision than the default |
skaller <skaller@ozemail.com.au> writes: > On Sat, 2003-12-06 at 14:18, Brian Hurt wrote: >> 30 decimal digits is enough accuracy to measure the >> distance between here and the Andromeda Galaxy in microns. Why does he >> need such a ridiculous amount of precision? > > Standard eigen problem sometimes needs ridiculous precision: > the scalar product of two vectors: > > sigma (i=1,n) (v1_i * v2_i) > > sometimes involves cancellation of large values. > To see why there is a problem here, consider: > > 1.000001e20 - 1e20 - > (1.000001e-20 - 1e-20) > [...] This is what is called a numerical unstable algorithm... -- Rémi Vanicat ------------------- To unsubscribe, mail caml-list-request@inria.fr Archives: http://caml.inria.fr Bug reports: http://caml.inria.fr/bin/caml-bugs FAQ: http://caml.inria.fr/FAQ/ Beginner's list: http://groups.yahoo.com/group/ocaml_beginners