How to round a double correctly... [modified]
-
Yes I was doing that(Math.Rounding) but I wasn't resigning the variable the new value. That works just fine. Thanks!!
You might want to double check the results and do some testing to see if it's going to work for you. Also, doing math on the number you want to round while using a Double type can affect how rounding works. For example, Doing something like:
Math.Round(1.255, 2)results in 1.25, not 1.26. What happens is that 1.255 is multiplied by 100 (the 2 parameter), then the number is rounded to the nearest integer. But, with the Double type, this results in 1.255 * 100 = 125.4999999999, which will round DOWN to the nearest integer of 125. This is then divided by 100, which results in 1.25. If accuracy is mandatory, consider using the Decimal type. It's slower since it's not a native CPU data type, but it is more accurate.A guide to posting questions on CodeProject[^]
Dave Kreskowiak Microsoft MVP Visual Developer - Visual Basic
2006, 2007, 2008 -
I tried.... Dim DisTest As Double = Me.TermDiscPct / 100 Round(DisTest, 2, MidpointRounding.AwayFromZero) and it didn't make any difference
-
You might want to double check the results and do some testing to see if it's going to work for you. Also, doing math on the number you want to round while using a Double type can affect how rounding works. For example, Doing something like:
Math.Round(1.255, 2)results in 1.25, not 1.26. What happens is that 1.255 is multiplied by 100 (the 2 parameter), then the number is rounded to the nearest integer. But, with the Double type, this results in 1.255 * 100 = 125.4999999999, which will round DOWN to the nearest integer of 125. This is then divided by 100, which results in 1.25. If accuracy is mandatory, consider using the Decimal type. It's slower since it's not a native CPU data type, but it is more accurate.A guide to posting questions on CodeProject[^]
Dave Kreskowiak Microsoft MVP Visual Developer - Visual Basic
2006, 2007, 2008Hey Dave, thanks for your reply. Doesn't MidpointRounding.AwayFromZero fix what you are refering to? I was having a similar problem that you descriped and it took care of it. I was doing this.. Dim DisTest As Double = 2.5 / 100 Round(DisTest, 2) And my return was .02 instead of .03 like I would expect. Now I am doing this... Dim DisTest As Double = 2.5 / 100 Round(DisTest, 2,MidpointRounding.AwayFromZero) And now my return is .03 just as I would expect it to be. I am only asking because accuracy is important and it seems to be pretty accurate as of now. But are you saying if I had 1.255 / 100 then it wouldn't be accurate?
-
You might want to double check the results and do some testing to see if it's going to work for you. Also, doing math on the number you want to round while using a Double type can affect how rounding works. For example, Doing something like:
Math.Round(1.255, 2)results in 1.25, not 1.26. What happens is that 1.255 is multiplied by 100 (the 2 parameter), then the number is rounded to the nearest integer. But, with the Double type, this results in 1.255 * 100 = 125.4999999999, which will round DOWN to the nearest integer of 125. This is then divided by 100, which results in 1.25. If accuracy is mandatory, consider using the Decimal type. It's slower since it's not a native CPU data type, but it is more accurate.A guide to posting questions on CodeProject[^]
Dave Kreskowiak Microsoft MVP Visual Developer - Visual Basic
2006, 2007, 2008 -
The Round method doesn't change the value that you pass to it, you have to take care of the return value:
DisTest = Round(DisTest, 2, MidpointRounding.AwayFromZero)Despite everything, the person most likely to be fooling you next is yourself.
-
Hey Dave, thanks for your reply. Doesn't MidpointRounding.AwayFromZero fix what you are refering to? I was having a similar problem that you descriped and it took care of it. I was doing this.. Dim DisTest As Double = 2.5 / 100 Round(DisTest, 2) And my return was .02 instead of .03 like I would expect. Now I am doing this... Dim DisTest As Double = 2.5 / 100 Round(DisTest, 2,MidpointRounding.AwayFromZero) And now my return is .03 just as I would expect it to be. I am only asking because accuracy is important and it seems to be pretty accurate as of now. But are you saying if I had 1.255 / 100 then it wouldn't be accurate?
CCG3 wrote:
Doesn't MidpointRounding.AwayFromZero fix what you are refering to?
No, it doesn't. It just happened to change the behavior of the Round method so THAT particular case worked out properly. The
AwayFromZerooption rounds numbers up, never down.CCG3 wrote:
And now my return is .03 just as I would expect it to be. I am only asking because accuracy is important and it seems to be pretty accurate as of now. But are you saying if I had 1.255 / 100 then it wouldn't be accurate?
Yep. The only reason why it worked is because the math using Double numbers worked out to nice even binary representations. For example, 1.255 / 100 = 0.012549999999999, not 0.01255. In your case, 2.5 / 100 worked out evenly to 0.025. It's those binary approximations that screws up the rounding. It just happened to work out properly on the example you tried. It will NOT work in all cases. You'd probably be better off using the Decimal type instead of Double.
A guide to posting questions on CodeProject[^]
Dave Kreskowiak Microsoft MVP Visual Developer - Visual Basic
2006, 2007, 2008 -
CCG3 wrote:
Doesn't MidpointRounding.AwayFromZero fix what you are refering to?
No, it doesn't. It just happened to change the behavior of the Round method so THAT particular case worked out properly. The
AwayFromZerooption rounds numbers up, never down.CCG3 wrote:
And now my return is .03 just as I would expect it to be. I am only asking because accuracy is important and it seems to be pretty accurate as of now. But are you saying if I had 1.255 / 100 then it wouldn't be accurate?
Yep. The only reason why it worked is because the math using Double numbers worked out to nice even binary representations. For example, 1.255 / 100 = 0.012549999999999, not 0.01255. In your case, 2.5 / 100 worked out evenly to 0.025. It's those binary approximations that screws up the rounding. It just happened to work out properly on the example you tried. It will NOT work in all cases. You'd probably be better off using the Decimal type instead of Double.
A guide to posting questions on CodeProject[^]
Dave Kreskowiak Microsoft MVP Visual Developer - Visual Basic
2006, 2007, 2008Thanks Dave, I think I understand what you are saying... on further testing, I ran these lines just like this... Dim DisTest As Double = 1.255 / 100 MsgBox(DisTest) DisTest = Round(DisTest, 2, MidpointRounding.AwayFromZero) MsgBox(DisTest) Dim DisTest2 As Decimal = 1.255 / 100 MsgBox(DisTest2) DisTest2 = Round(DisTest2, 2) MsgBox(DisTest2) They both gave the same results. (.01) And you are saying that I would be better off to just use Decimal. The preceeding example is all I would have to change right? Instead of using Double change it to Decimal and leave out the MidpointRounding.AwayFromZero. right?
-
Thanks Dave, I think I understand what you are saying... on further testing, I ran these lines just like this... Dim DisTest As Double = 1.255 / 100 MsgBox(DisTest) DisTest = Round(DisTest, 2, MidpointRounding.AwayFromZero) MsgBox(DisTest) Dim DisTest2 As Decimal = 1.255 / 100 MsgBox(DisTest2) DisTest2 = Round(DisTest2, 2) MsgBox(DisTest2) They both gave the same results. (.01) And you are saying that I would be better off to just use Decimal. The preceeding example is all I would have to change right? Instead of using Double change it to Decimal and leave out the MidpointRounding.AwayFromZero. right?
Yep. But, only your testing is going to tell you if it's appropriate or not. Remember, there is a performance hit for using the Deicmal type.
A guide to posting questions on CodeProject[^]
Dave Kreskowiak Microsoft MVP Visual Developer - Visual Basic
2006, 2007, 2008 -
I am working with VB.Net 2005, I have a double that is coming up as .025 and I need to round it correctly. So it should show as .03 once this is done. I currenlty do this and it doesn't seem to work... Dim DisTest As Double = 2.5 / 100 Round(DisTest, 2)
modified on Tuesday, November 11, 2008 11:15 AM
Dim DisTest as Double = 2.5/100
DisTest= (Int((DisTest + 0.005) * 100)) / 100The secret is in the 0.005. 0.001 + 0.005 = 0.006, * 100 = 0.6, int = 0, / 100 = 0.00 0.002 + 0.005 = 0.007, * 100 = 0.7, int = 0, / 100 = 0.00 0.003 + 0.005 = 0.008, * 100 = 0.8, int = 0, / 100 = 0.00 0.004 + 0.005 = 0.009, * 100 = 0.9, int = 0, / 100 = 0.00 0.005 + 0.005 = 0.010, * 100 = 1.0, int = 1, / 100 = 0.01 0.006 + 0.005 = 0.011, * 100 = 1.1, int = 1, / 100 = 0.01 0.007 + 0.005 = 0.012, * 100 = 1.2, int = 1, / 100 = 0.01 0.008 + 0.005 = 0.013, * 100 = 1.3, int = 1, / 100 = 0.01 0.009 + 0.005 = 0.014, * 100 = 1.4, int = 1, / 100 = 0.01 0.010 + 0.005 = 0.015, * 100 = 1.5, int = 1, / 100 = 0.01 etc Also, I don't use "Round" because I can never remember from language to language how it works (round up? round down? round on .5?) If I want to round .005 down, .006 up (standard practise), I use 0.004 instead of 0.005