Prev: Reading from very large file
Next: Is the UNNEST function standard and/or widespread?
From: Roedy Green on 8 May 2010 17:42
If you use double to compute sales taxes, and round to the nearest
penny, there is a gotcha. You are supposed to round a precise half
penny up. This won't work properly for percentage like 6% (0.06) (of
$0.75) which is not precise in binary.

There a number of ways to avoid problem. Which would you use?
--
Roedy Green Canadian Mind Products
http://mindprod.com

What is the point of a surveillance camera with insufficient resolution to identify culprits?
From: Arne Vajhøj on 8 May 2010 17:44
On 08-05-2010 17:42, Roedy Green wrote:
> If you use double to compute sales taxes, and round to the nearest
> penny, there is a gotcha. You are supposed to round a precise half
> penny up. This won't work properly for percentage like 6% (0.06) (of
> $0.75) which is not precise in binary.
>
> There a number of ways to avoid problem. Which would you use?

BigDecimal should be able to mimic the rules out of the box.

Integers (long or int) could be made to mimic the rules.

And as a general rule there is death penalty for using
floating point for amounts.

Arne

From: Lew on 8 May 2010 19:43
On 05/08/2010 05:42 PM, Roedy Green wrote:
> If you use double to compute sales taxes, and round to the nearest
> penny, there is a gotcha. You are supposed to round a precise half
> penny up. This won't work properly for percentage like 6% (0.06) (of
> $0.75) which is not precise in binary.

In what jurisdictions? That's not the rule in the parts of the U.S. with
which I'm familiar. Sales tax rounds up, not to the nearest.

> There a number of ways to avoid problem. Which would you use?

I'd conform to the law.

--
Lew
From: Arne Vajhøj on 8 May 2010 20:22
On 08-05-2010 19:43, Lew wrote:
> On 05/08/2010 05:42 PM, Roedy Green wrote:
>> If you use double to compute sales taxes, and round to the nearest
>> penny, there is a gotcha. You are supposed to round a precise half
>> penny up. This won't work properly for percentage like 6% (0.06) (of
>> $0.75) which is not precise in binary.
>
> In what jurisdictions? That's not the rule in the parts of the U.S. with
> which I'm familiar. Sales tax rounds up, not to the nearest.

Give that Roedy is Canadian then I would assume Canada.

>> There a number of ways to avoid problem. Which would you use?
>
> I'd conform to the law.

And use something else than double.

Arne
From: Arved Sandstrom on 8 May 2010 21:44
Lew wrote:
> On 05/08/2010 05:42 PM, Roedy Green wrote:
>> If you use double to compute sales taxes, and round to the nearest
>> penny, there is a gotcha. You are supposed to round a precise half
>> penny up. This won't work properly for percentage like 6% (0.06) (of
>> $0.75) which is not precise in binary.
>
> In what jurisdictions? That's not the rule in the parts of the U.S.
> with which I'm familiar. Sales tax rounds up, not to the nearest.
>
>> There a number of ways to avoid problem. Which would you use?
>
> I'd conform to the law.

I'd use software that someone else wrote who had lots more time to
devote to the problem of sales tax arithmetic legalities than I do.

AHS
 |  Next  |  Last
Pages: 1 2 3
Prev: Reading from very large file
Next: Is the UNNEST function standard and/or widespread?