Version française
Home     About     Download     Resources     Contact us    
Browse thread
What library to use for arbitrary precision decimals
[ Home ] [ Index: by date | by threads ]
[ Search: ]

[ Message by date: previous | next ] [ Message in thread: previous | next ] [ Thread: previous | next ]
Date: -- (:)
From: Richard Jones <rich@a...>
Subject: Re: [Caml-list] What library to use for arbitrary precision decimals
On Sun, Feb 19, 2006 at 10:53:47AM -0600, Brian Hurt wrote:
> Hmm.  There are problems definin that library.  What are the semantics? 
> If I add a NUMERIC(6,4) (2 digits to the left of the point and 4 to the 
> righ) and a NUMERIC(7,2) (5 digits to the left of the point and 2 to the 
> right), what precision is the result?  NUMERIC(4,2) (2 digits to the left 
> of the point and 2 to the right)?  NUMERIC(9,4) (5 digits to the left of 
> the point and 4 digits to the right)?  How does roundoff work?  If I take 
> the NUMERIC(6,4) number 62.0000 and multiply it by the NUMERIC(6,4) number 
> 02.0000, what answer do I get?  The NUMERIC(6,4) number 24.000?  The 
> NUMERIC(7,4) number 124.000?  Or does it throw an exception?

Well yes indeed.  I'm quite sure there's a SQL specification somewhere
which answers this :-)

Rich.

-- 
Richard Jones, CTO Merjis Ltd.
Merjis - web marketing and technology - http://merjis.com
Team Notepad - intranets and extranets for business - http://team-notepad.com