Facing Problem with Sum()

Thursday, 12 March 2015 · 3 messages
AK
ankit kumar12 Mar 2015 15:01 #

Hi,

I am trying to evaluate this expression, it is supposed to give TRUE but, it is giving false.

sum((<a>5923.17</a>,<a>5956.29</a>,<a>5989.28</a>)) = <a>17868.74</a>

But, for this expression it is giving as expected.

sum((<a>5922.78</a>,<a>-5955.78</a>,<a>5988.74</a>)) = <a>5955.74</a>

Thanks

Ankit

FG
Florent Georges12 Mar 2015 15:13 #

Hi,

This looks like a general programming problem to me. It is always dangerous to ask for equality on real numbers. Because of a rounding problem in the calculation (because of the way computers work), the sum is not exactly equal to the number on the right hand-side.

The usual algorithm to compare real numbers for equality, up to say 3 decimal numbers, is the following, comparing integers instead (replace 1000 with e.g. 100 for 2 decimal places):

    fn:round($n1 * 1000) = fn:round($n2 * 1000)

Regards,

Quoted text · ankit kumar · 17 lines
On 12 March 2015 at 15:01, ankit kumar <anky4bugs at gmail.com> wrote: > Hi, > > I am trying to evaluate this expression, it is supposed to give TRUE but, > it is giving false. > > sum((<a>5923.17</a>,<a>5956.29</a>,<a>5989.28</a>)) = <a>17868.74</a> > > But, for this expression it is giving as expected. > > sum((<a>5922.78</a>,<a>-5955.78</a>,<a>5988.74</a>)) = <a>5955.74</a> > > > > Thanks > > Ankit
ML
Marco Letterein reply to ankit kumar, 12 Mar 2015, 15:0112 Mar 2015 15:37 #

Hi Ankit, the problem is with float rounding ...

sum((<a>5923.17</a>,<a>5956.29</a>,<a>5989.28</a>)) = 17868.739999999998

gives true() on my machine. As usual with floats you should probably apply a rounding policy to compare for exact equality. Regards, Marco.

Quoted text · ankit kumar · 17 lines
On 12/03/2015 15:01, ankit kumar wrote: > Hi, > > I am trying to evaluate this expression, it is supposed to give TRUE > but, it is giving false. > > sum((<a>5923.17</a>,<a>5956.29</a>,<a>5989.28</a>)) = <a>17868.74</a> > > But, for this expression it is giving as expected. > > sum((<a>5922.78</a>,<a>-5955.78</a>,<a>5988.74</a>)) = <a>5955.74</a> > > > > Thanks > > Ankit

⚡Generated with XQuery