> [Many questions about float arithmetic and optimizations]

Do yourself a favor and read Goldberg's excellent reference:

"What Every Computer Scientist Should Know About Floating Point Arithmetic"
ACM Computing Surveys 23(1), 5-48, 1991
http://citeseer.ist.psu.edu/goldberg91what.html

It doesn't read like _TV_digest_, but it's well worth the effort.

> Is the compiler allowed to make
> optimizations according to known mathematical laws?

Yes, provided they actually hold.  Goldberg lists a few that hold in
IEEE float arithmetic (section "Optimizers").  But pretty much every
algebraic law that holds over the reals doesn't hold in floating-point
arithmetic.

> Still: With a certain Gcc version and flags combination the OP saw a
> threefold improvement in performance. That in intself is suspicious (I
> don't think that this much optimization potential was left in Gcc ...)
> and I still would check for optimization errors in this case.

That's a good idea.  Note however that there are known optimizations
(loop blocking, loop interchange) that can dramatically improve the
performance of dense matrix multiply by making better use of the caches.
Automatic vectorization (generation of SSE2 instructions that operate
over pairs of double-precision floats) could also have a significant
impact, although not by a factor of 3.  There's only one way to know:
read the assembly code generated by gcc.

- Xavier Leroy