### Author Topic: Precision of the operation with integer and float  (Read 937 times)

#### Qedo

• Dr. Type
• Posts: 355
• to program what I have do how should programming?
##### Precision of the operation with integer and float
« on: 2020-Apr-24 »
I write this because nobody ever said it (or at least I never read it anywhere).
When performing a number operation having integer variables and float together, pay attention to the order of the variables.
Example with the calculator the operation "911/5/15*2*1.1" returns 26.72266667
in GLbasic:
Aint% = 5, Bint% = 911, Cint% = 15, Dint% = 2, Mflt # = 1.1
1) Nflt# = Bint / Aint / Cint * Dint * Mflt    // returns 26.4    // WRONG
2) Nflt# = Mflt * (Bint / Aint / Cint * Dint)    // returns 26.4    // WRONG
3) Nflt# = Mflt * Bint / Aint / Cint * Dint    // returns 26.72266667    // OK

I think this depends on the order in which operations are performed from left to right. If the first variable is float the precision will remain with the comma and there will be no rounding to the integers.
I hope it is useful
Ciao

#### dreamerman

• Global Moderator
• Dr. Type
• Posts: 407
##### Re: Precision of the operation with integer and float
« Reply #1 on: 2020-Apr-24 »
Yeah this can cause some unexected behaviour (I overlooked such things so many times ), and this applies in general to any programming language, it's always advised to use proper type casting in calculations.
In GLBasic it's easy to avoid, as You mentioned type order is important so all what you need to do to get proper ## result is add "1.0 *" before rest of calculation.
In C++ you will get same problems without proper type casting.
Code: (glbasic) [Select]
`FUNCTION test_int_float%: LOCAL ga% = 5, gb% = 911, gc% = 15, gd% = 2, gm# = 1.1 LOCAL result1#, result2# INLINE int a = 5, b = 911, c = 15, d = 2; double m = 1.1, resultf; result1 = b / a / c * d * m; result2 = double(b) / double(a) / double(c) * double(d) * m; ENDINLINE DEBUG "results, 1: " + result1 + ", 2: " + result2 + ", pure glb: " + (1.0 * gb / ga / gc * gd * gm) + "\n"ENDFUNCTION`Not sure if that isn't specified in one of tutorials, but it could be.
Check my source code editor for GLBasic - link Update: 20.04.2020

#### Kitty Hello

• code monkey
• Prof. Inline
• Posts: 10816
• here on my island the sea says 'hello'
##### Re: Precision of the operation with integer and float
« Reply #2 on: 2020-Apr-25 »
or explicitly cast the left integer that causes the problem with NUMBER(i).
Expressions are evaluated left-to-right. So,
Code: (glbasic) [Select]
`LOCAL i=5, k=2, j#=7i/k*j // = INTEGER(i/k) * j = 2*7.0=14.0NUMBER(i)/k*j // = 5.0 / 2 * j = 2.5*7.0`If any of the two operands is a floating point, so will be the result. If both are integer, the result is, too.
The problem usually only apears for divisions.

#### Qedo

• Dr. Type
• Posts: 355
• to program what I have do how should programming?
##### Re: Precision of the operation with integer and float
« Reply #3 on: 2020-Apr-25 »
I have discovered the existence of NUMBER () but it is not in the help.
What does this command do?

#### dreamerman

• Global Moderator
• Dr. Type
• Posts: 407
##### Re: Precision of the operation with integer and float
« Reply #4 on: 2020-Apr-25 »
Yeah, quick looks it seems that standard manual / help file doesn't have that info, but it's in online manual.
Quote
flt# = NUMBER(str\$)

Explicitly convert a string or any other number into a floating point number.

LOCAL n# = NUMBER("1.23") * 100.0
Check my source code editor for GLBasic - link Update: 20.04.2020

#### Qedo

• Dr. Type
• Posts: 355
• to program what I have do how should programming?
##### Re: Precision of the operation with integer and float
« Reply #5 on: 2020-Apr-25 »
Great