Browse thread
What library to use for arbitrary precision decimals
-
Joshua Smith
-
Brian Hurt
-
Richard Jones
-
Brian Hurt
- Richard Jones
-
Brian Hurt
- Jeremy Shute
-
Richard Jones
-
Brian Hurt
[
Home
]
[ Index:
by date
|
by threads
]
[ Message by date: previous | next ] [ Message in thread: previous | next ] [ Thread: previous | next ]
[ Message by date: previous | next ] [ Message in thread: previous | next ] [ Thread: previous | next ]
Date: | 2006-02-19 (17:01) |
From: | Richard Jones <rich@a...> |
Subject: | Re: [Caml-list] What library to use for arbitrary precision decimals |
On Sun, Feb 19, 2006 at 10:53:47AM -0600, Brian Hurt wrote: > Hmm. There are problems definin that library. What are the semantics? > If I add a NUMERIC(6,4) (2 digits to the left of the point and 4 to the > righ) and a NUMERIC(7,2) (5 digits to the left of the point and 2 to the > right), what precision is the result? NUMERIC(4,2) (2 digits to the left > of the point and 2 to the right)? NUMERIC(9,4) (5 digits to the left of > the point and 4 digits to the right)? How does roundoff work? If I take > the NUMERIC(6,4) number 62.0000 and multiply it by the NUMERIC(6,4) number > 02.0000, what answer do I get? The NUMERIC(6,4) number 24.000? The > NUMERIC(7,4) number 124.000? Or does it throw an exception? Well yes indeed. I'm quite sure there's a SQL specification somewhere which answers this :-) Rich. -- Richard Jones, CTO Merjis Ltd. Merjis - web marketing and technology - http://merjis.com Team Notepad - intranets and extranets for business - http://team-notepad.com