Precision of the operation with integer and float

Previous topic - Next topic

Qedo

I write this because nobody ever said it (or at least I never read it anywhere).
When performing a number operation having integer variables and float together, pay attention to the order of the variables.
Example with the calculator the operation "911/5/15*2*1.1" returns 26.72266667
in GLbasic:
Aint% = 5, Bint% = 911, Cint% = 15, Dint% = 2, Mflt # = 1.1
1) Nflt# = Bint / Aint / Cint * Dint * Mflt    // returns 26.4    // WRONG
2) Nflt# = Mflt * (Bint / Aint / Cint * Dint)    // returns 26.4    // WRONG
3) Nflt# = Mflt * Bint / Aint / Cint * Dint    // returns 26.72266667    // OK

I think this depends on the order in which operations are performed from left to right. If the first variable is float the precision will remain with the comma and there will be no rounding to the integers.
I hope it is useful
Ciao

dreamerman

Yeah this can cause some unexected behaviour (I overlooked such things so many times :D), and this applies in general to any programming language, it's always advised to use proper type casting in calculations.
In GLBasic it's easy to avoid, as You mentioned type order is important so all what you need to do to get proper ## result is add "1.0 *" before rest of calculation.
In C++ you will get same problems without proper type casting.
Code (glbasic) Select

FUNCTION test_int_float%:
LOCAL ga% = 5, gb% = 911, gc% = 15, gd% = 2, gm# = 1.1
LOCAL result1#, result2#
INLINE
int a = 5, b = 911, c = 15, d = 2;
double m = 1.1, resultf;
result1 = b / a / c * d * m;
result2 = double(b) / double(a) / double(c) * double(d) * m;
ENDINLINE
DEBUG "results, 1: " + result1 + ", 2: " + result2 + ", pure glb: " + (1.0 * gb / ga / gc * gd * gm) + "\n"
ENDFUNCTION

Not sure if that isn't specified in one of tutorials, but it could be.
Check my source code editor for GLBasic - link Update: 20.04.2020

Kitty Hello

or explicitly cast the left integer that causes the problem with NUMBER(i).
Expressions are evaluated left-to-right. So,
Code (glbasic) Select

LOCAL i=5, k=2, j#=7
i/k*j // = INTEGER(i/k) * j = 2*7.0=14.0
NUMBER(i)/k*j // = 5.0 / 2 * j = 2.5*7.0

If any of the two operands is a floating point, so will be the result. If both are integer, the result is, too.
The problem usually only apears for divisions.

Qedo

I have discovered the existence of NUMBER () but it is not in the help.
What does this command do?

dreamerman

Yeah, quick looks it seems that standard manual / help file doesn't have that info, but it's in online manual.
Quoteflt# = NUMBER(str$)

Explicitly convert a string or any other number into a floating point number.

LOCAL n# = NUMBER("1.23") * 100.0
Check my source code editor for GLBasic - link Update: 20.04.2020

Qedo