Browse thread
bug in floating point implementation ?
-
Shyjan Mahamud
-
Xavier Leroy
- mahamud@c...
- Stefan Monnier
-
Xavier Leroy
[
Home
]
[ Index:
by date
|
by threads
]
[ Message by date: previous | next ] [ Message in thread: previous | next ] [ Thread: previous | next ]
[ Message by date: previous | next ] [ Message in thread: previous | next ] [ Thread: previous | next ]
| Date: | -- (:) |
| From: | mahamud@c... |
| Subject: | Re: bug in floating point implementation ? |
>>>>> "Xavier" == Xavier Leroy <Xavier.Leroy@inria.fr> writes:
Xavier> The Intel x86 has an interesting quirk: it computes
Xavier> internally in extended precision (80 bits), and rounds to
Xavier> double precision (64 bits) only when storing a result in
Xavier> memory. This means that in a piece of code such as
Xavier> let x = a * b + c
Xavier> the virtual machine will compute a * b, round it (by
Xavier> storing it in memory), add c, and round the result. While
Xavier> the native-code compiler will compute a * b, keep the
Xavier> result in extended precision in the top float register,
Xavier> then add c, then round the result.
Xavier> In extreme cases (badly conditioned algorithm, computation
Xavier> of errors), you may thus get different results.
sorry about the late reply. but i was hung up with other stuff.
thanx for the explanation. i've found out that indeed my computation
was ill-conditioned and have fixed it now.
in fact, were it not for this intel quirkiness, i might have
overlooked the bug for some-time. this definitely tops my
list of ways i've debugged code!
before i was able to spot the problem with my code, i tried
downloading the src on a different architecture. however, it
seems that the tar file is broken, since it creates files when it should
be creating dirs. i never faced this problem for my linux box since
i downloaded the binary rpms. anyway it does'nt matter for now since
i know what the problem is. but in future it might be useful to
compile ocaml on a different architecture (i had tried both 2.01 &
2.02).
- shyjan
carnegie mellon university