How come I get a rounded number when I multiply doubles?? [modified]
-
Just adding a .0 isn't good practice. Do you know for sure what type it becomes by adding a .0? It could be a decimal, or a float, or a double. Even if you know what type the compiler chooses by default, will other people who have to read or update your code? See my previous post in the thread for how to make sure the number you are dealing with is cast as exactly the type you need.
According to ECMA-334 9.4.4.3 Real literals Paragraph 2: "If no real-type-suffix is specified, the type of the real literal is double."* *: in ECMA-334 9.4.4.3 a grammar form that excludes any number without a real-type-suffix or decimal-digits after the dot to be a real literal. If you are on a non ECMA compilant implementation of C#, move to another compiler/interpreter.
-
ely_bob wrote:
dbRecipeTotal = (double)(((double)(RI.Quantity / dbRecipeTotalQuantity)) * ((double)(InvItem.Price / 100))); this will make sure that you are always in the precision your looking for, indepentant of whether you use a lower precision numeral type.
ely_bob, your solution still doesn't work because (InvItem.Price / 100) will be 24, so ((double)(InvItem.Price / 100)) will be 24.0 It needs to be ((double)InvItem.Price / 100). I didn't cast the 100 to double because when one of the arguments to operator/ is a double, the other one is automatically cast (I think).
Yeah your right.. you need to be more liberal with your
(double)'s.... That just goes to show you.. don't post before coffee.... :laugh:I'd blame it on the Brain farts.. But let's be honest, it really is more like a Methane factory between my ears some days then it is anything else...
-
How come I get a rounded number when I multiply and assign value to a double??
dbRecipeTotal = (RI.Quantity / dbRecipeTotalQuantity) * (InvItem.Price / 100);
double dbRecipeTotal = 0.0;
RI.Quantity is = 28.88887
dbRecipeTotalQuantity is = 28.88887
InvItem.Price is = 2425I get dbRecipeTotal = 24.0 in quickview it should be 24.25 Any help is appreaciated
modified on Wednesday, July 28, 2010 9:44 AM
A floating point number can never be specified exactly in memory, unless the number is an even power of two. If you just ignore the division part of your example. Even if you were to type: Even if you were type: dbRecipeTotal = 24.25; It may display as 24.25 or 24.0 or 24.24999999999999 depending on how print formating is specified. Just remember, you're never going to store 24.25 in a floating point number exactly.
-
A floating point number can never be specified exactly in memory, unless the number is an even power of two. If you just ignore the division part of your example. Even if you were to type: Even if you were type: dbRecipeTotal = 24.25; It may display as 24.25 or 24.0 or 24.24999999999999 depending on how print formating is specified. Just remember, you're never going to store 24.25 in a floating point number exactly.
You are always going to store 24.25 in a floating point number exactly. 24.25 is a power of two. You should never see any of the results you listed (except 24.25) no matter what print formatting is specified.
-
You are always going to store 24.25 in a floating point number exactly. 24.25 is a power of two. You should never see any of the results you listed (except 24.25) no matter what print formatting is specified.
Thanks. I wasn't aware that 24.25 is an even power of two. Learn something new every day.
-
How come I get a rounded number when I multiply and assign value to a double??
dbRecipeTotal = (RI.Quantity / dbRecipeTotalQuantity) * (InvItem.Price / 100);
double dbRecipeTotal = 0.0;
RI.Quantity is = 28.88887
dbRecipeTotalQuantity is = 28.88887
InvItem.Price is = 2425I get dbRecipeTotal = 24.0 in quickview it should be 24.25 Any help is appreaciated
modified on Wednesday, July 28, 2010 9:44 AM
The answer lies in the way implicit data type conversion works. Dividing and integer by and integer will never yeild a float or a double. So, InvItem.Price / 100 becomes 24 and not 24.25. I'm sure you understand the rest. To get correct answer do something like
( InvItem.Price / 100.0 ) * ( RI.Quantity / dbRecipeTotalQuantity )
-
You are always going to store 24.25 in a floating point number exactly. 24.25 is a power of two. You should never see any of the results you listed (except 24.25) no matter what print formatting is specified.
[Stepping on my soapbox] When dealing with floating point numbers, you should never assume that a number will be stored exactly. While some numbers can be stored exactly, most will not be. From the cases I have encountered, too many people assume that a floating point number is good enough for exact numerical needs. But if the value 24.24999999999999 instead of 24.25 would be considered incorrect in your application, you should not be using floating point values. Time and time again I have been called in to analyze why dollar values were coming in off by a penny or two (or more), and it has always* boiled down to the choice of using floating point representation to store exact values. * Since 1992 I have encountered only 2 instances where the problem was an actual bug in the library.
-
How come I get a rounded number when I multiply and assign value to a double??
dbRecipeTotal = (RI.Quantity / dbRecipeTotalQuantity) * (InvItem.Price / 100);
double dbRecipeTotal = 0.0;
RI.Quantity is = 28.88887
dbRecipeTotalQuantity is = 28.88887
InvItem.Price is = 2425I get dbRecipeTotal = 24.0 in quickview it should be 24.25 Any help is appreaciated
modified on Wednesday, July 28, 2010 9:44 AM
I think InvItem.Price is int so 2425/100=24 you can write : InvItem.Price/100.0 to get correct result
-
How come I get a rounded number when I multiply and assign value to a double??
dbRecipeTotal = (RI.Quantity / dbRecipeTotalQuantity) * (InvItem.Price / 100);
double dbRecipeTotal = 0.0;
RI.Quantity is = 28.88887
dbRecipeTotalQuantity is = 28.88887
InvItem.Price is = 2425I get dbRecipeTotal = 24.0 in quickview it should be 24.25 Any help is appreaciated
modified on Wednesday, July 28, 2010 9:44 AM
I think InvItem.Price is int so 2425/100=24 you can write : InvItem.Price/100.0 to get correct result: (this is tested and returns correct result): double Quantity = 28.88887; double dbRecipeTotalQuantity = 28.88887; int Price = 2425; double dbRecipeTotal = (Quantity / dbRecipeTotalQuantity) * (Price / 100.0);
-
ely_bob wrote:
dbRecipeTotal = (double)(((double)(RI.Quantity / dbRecipeTotalQuantity)) * ((double)(InvItem.Price / 100))); this will make sure that you are always in the precision your looking for, indepentant of whether you use a lower precision numeral type.
ely_bob, your solution still doesn't work because (InvItem.Price / 100) will be 24, so ((double)(InvItem.Price / 100)) will be 24.0 It needs to be ((double)InvItem.Price / 100). I didn't cast the 100 to double because when one of the arguments to operator/ is a double, the other one is automatically cast (I think).
double Quantity = 28.88887; double dbRecipeTotalQuantity = 28.88887; int Price = 2425; double dbRecipeTotal = (Quantity / dbRecipeTotalQuantity) * (Price / 100.0);
-
How come I get a rounded number when I multiply and assign value to a double??
dbRecipeTotal = (RI.Quantity / dbRecipeTotalQuantity) * (InvItem.Price / 100);
double dbRecipeTotal = 0.0;
RI.Quantity is = 28.88887
dbRecipeTotalQuantity is = 28.88887
InvItem.Price is = 2425I get dbRecipeTotal = 24.0 in quickview it should be 24.25 Any help is appreaciated
modified on Wednesday, July 28, 2010 9:44 AM