GLBasic forum

Main forum => GLBasic - en => Topic started by: Qedo on 2020-Apr-24

Title: Precision of the operation with integer and float
Post by: Qedo on 2020-Apr-24
I write this because nobody ever said it (or at least I never read it anywhere).
When performing a number operation having integer variables and float together, pay attention to the order of the variables.
Example with the calculator the operation "911/5/15*2*1.1" returns 26.72266667
in GLbasic:
Aint% = 5, Bint% = 911, Cint% = 15, Dint% = 2, Mflt # = 1.1
1) Nflt# = Bint / Aint / Cint * Dint * Mflt    // returns 26.4    // WRONG
2) Nflt# = Mflt * (Bint / Aint / Cint * Dint)    // returns 26.4    // WRONG
3) Nflt# = Mflt * Bint / Aint / Cint * Dint    // returns 26.72266667    // OK

I think this depends on the order in which operations are performed from left to right. If the first variable is float the precision will remain with the comma and there will be no rounding to the integers.
I hope it is useful
Ciao
Title: Re: Precision of the operation with integer and float
Post by: dreamerman on 2020-Apr-24
Yeah this can cause some unexected behaviour (I overlooked such things so many times :D), and this applies in general to any programming language, it's always advised to use proper type casting in calculations.
In GLBasic it's easy to avoid, as You mentioned type order is important so all what you need to do to get proper ## result is add "1.0 *" before rest of calculation.
In C++ you will get same problems without proper type casting.
Code (glbasic) Select

FUNCTION test_int_float%:
LOCAL ga% = 5, gb% = 911, gc% = 15, gd% = 2, gm# = 1.1
LOCAL result1#, result2#
INLINE
int a = 5, b = 911, c = 15, d = 2;
double m = 1.1, resultf;
result1 = b / a / c * d * m;
result2 = double(b) / double(a) / double(c) * double(d) * m;
ENDINLINE
DEBUG "results, 1: " + result1 + ", 2: " + result2 + ", pure glb: " + (1.0 * gb / ga / gc * gd * gm) + "\n"
ENDFUNCTION

Not sure if that isn't specified in one of tutorials, but it could be.
Title: Re: Precision of the operation with integer and float
Post by: Kitty Hello on 2020-Apr-25
or explicitly cast the left integer that causes the problem with NUMBER(i).
Expressions are evaluated left-to-right. So,
Code (glbasic) Select

LOCAL i=5, k=2, j#=7
i/k*j // = INTEGER(i/k) * j = 2*7.0=14.0
NUMBER(i)/k*j // = 5.0 / 2 * j = 2.5*7.0

If any of the two operands is a floating point, so will be the result. If both are integer, the result is, too.
The problem usually only apears for divisions.
Title: Re: Precision of the operation with integer and float
Post by: Qedo on 2020-Apr-25
I have discovered the existence of NUMBER () but it is not in the help.
What does this command do?
Title: Re: Precision of the operation with integer and float
Post by: dreamerman on 2020-Apr-25
Yeah, quick looks it seems that standard manual / help file doesn't have that info, but it's in online manual.
Quoteflt# = NUMBER(str$)

Explicitly convert a string or any other number into a floating point number.

LOCAL n# = NUMBER("1.23") * 100.0
Title: Re: Precision of the operation with integer and float
Post by: Qedo on 2020-Apr-25
Great  :good: