How come I get a rounded number when I multiply doubles?? [modified]

[Luc Pattyn]

Your subject line claims you're multiplying doubles, I guess you are not. Assuming InvItem.Price is an integer, InvItem.Price / 100 is too. if you hadn't used parentheses around it, the result would have been correct. The better approach however is to use either floating-point or decimal for all quantities that aren't integer by their very nature, here Price. :)

Luc Pattyn [Forum Guidelines] [Why QA sucks] [My Articles] Nil Volentibus Arduum

Please use <PRE> tags for code snippets, they preserve indentation, and improve readability.

modified on Tuesday, July 27, 2010 12:18 PM

[Lost User]

Are you sure about the above values, I get 0.04123711340206185567010309278351(oops!)? I mean 24.

It's time for a new signature.

[T M Gray]

Whenever you do math between two number types you should expect your answer will be in precision of the lowest precision operand. You should either declare the 100 as a constant of the appropriate type or use Numeric type suffixes[^].

[William Winner]

Try making 100 100.0 instead. It's taking price and dividing by an integer which forces the result of the operation to an integer. Telling it that 100 isn't an integer by adding the decimal point should give you the right answer.

[T M Gray]

Just adding a .0 isn't good practice. Do you know for sure what type it becomes by adding a .0? It could be a decimal, or a float, or a double. Even if you know what type the compiler chooses by default, will other people who have to read or update your code? See my previous post in the thread for how to make sure the number you are dealing with is cast as exactly the type you need.

[pinskerr]

Your other respondents are right that InvItem.Price appears to be an integer, so you would expect the answer to be 24.0. However you say the answer is 12.0, that is really weird - is that what you meant?

[roman_s]

Sorry there is a mathematical mistake what I needed to know is when dividing an integer with a remainder I was getting double variable(2425) / 100 would give you 24.0 , I was wondering why I wasn't getting 24.25 solution was because I was dividing an integer into another integer, I corrected this simply by multiplying 100.00 instead of 100

[ely_bob]

I've run into this problem Many times... You need to explicitly tell the compiler what type you want from each operation

dbRecipeTotal = (double)(((double)(RI.Quantity / dbRecipeTotalQuantity)) * ((double)(InvItem.Price / 100)));

this will make sure that you are always in the precision your looking for, indepentant of whether you use a lower precision numeral type.

I'd blame it on the Brain farts.. But let's be honest, it really is more like a Methane factory between my ears some days then it is anything else...

[Charvak Karpe]

ely_bob wrote:

dbRecipeTotal = (double)(((double)(RI.Quantity / dbRecipeTotalQuantity)) * ((double)(InvItem.Price / 100))); this will make sure that you are always in the precision your looking for, indepentant of whether you use a lower precision numeral type.

ely_bob, your solution still doesn't work because (InvItem.Price / 100) will be 24, so ((double)(InvItem.Price / 100)) will be 24.0 It needs to be ((double)InvItem.Price / 100). I didn't cast the 100 to double because when one of the arguments to operator/ is a double, the other one is automatically cast (I think).

[Theraot]

According to ECMA-334 9.4.4.3 Real literals Paragraph 2: "If no real-type-suffix is specified, the type of the real literal is double."* *: in ECMA-334 9.4.4.3 a grammar form that excludes any number without a real-type-suffix or decimal-digits after the dot to be a real literal. If you are on a non ECMA compilant implementation of C#, move to another compiler/interpreter.

[ely_bob]

Yeah your right.. you need to be more liberal with your (double)'s.... That just goes to show you.. don't post before coffee.... :laugh:

I'd blame it on the Brain farts.. But let's be honest, it really is more like a Methane factory between my ears some days then it is anything else...

[Kenneth Kasajian]

A floating point number can never be specified exactly in memory, unless the number is an even power of two. If you just ignore the division part of your example. Even if you were to type: Even if you were type: dbRecipeTotal = 24.25; It may display as 24.25 or 24.0 or 24.24999999999999 depending on how print formating is specified. Just remember, you're never going to store 24.25 in a floating point number exactly.

ken@kasajian.com / www.kasajian.com

[PaulLinton]

You are always going to store 24.25 in a floating point number exactly. 24.25 is a power of two. You should never see any of the results you listed (except 24.25) no matter what print formatting is specified.

[Kenneth Kasajian]

Thanks. I wasn't aware that 24.25 is an even power of two. Learn something new every day.

ken@kasajian.com / www.kasajian.com

[sushantpokharel]

The answer lies in the way implicit data type conversion works. Dividing and integer by and integer will never yeild a float or a double. So, InvItem.Price / 100 becomes 24 and not 24.25. I'm sure you understand the rest. To get correct answer do something like

( InvItem.Price / 100.0 ) * ( RI.Quantity / dbRecipeTotalQuantity )

[Mitch Hughes]

[Stepping on my soapbox] When dealing with floating point numbers, you should never assume that a number will be stored exactly. While some numbers can be stored exactly, most will not be. From the cases I have encountered, too many people assume that a floating point number is good enough for exact numerical needs. But if the value 24.24999999999999 instead of 24.25 would be considered incorrect in your application, you should not be using floating point values. Time and time again I have been called in to analyze why dollar values were coming in off by a penny or two (or more), and it has always* boiled down to the choice of using floating point representation to store exact values. * Since 1992 I have encountered only 2 instances where the problem was an actual bug in the library.

[User 3150960]

I think InvItem.Price is int so 2425/100=24 you can write : InvItem.Price/100.0 to get correct result

[User 3150960]

I think InvItem.Price is int so 2425/100=24 you can write : InvItem.Price/100.0 to get correct result: (this is tested and returns correct result): double Quantity = 28.88887; double dbRecipeTotalQuantity = 28.88887; int Price = 2425; double dbRecipeTotal = (Quantity / dbRecipeTotalQuantity) * (Price / 100.0);

[User 3150960]

double Quantity = 28.88887; double dbRecipeTotalQuantity = 28.88887; int Price = 2425; double dbRecipeTotal = (Quantity / dbRecipeTotalQuantity) * (Price / 100.0);

[kxal]

I think the InvItem.Price is an integer,if so,the division will diacard the part after decimal point automatically.