Problem with double
-
Hello, I have the following simple code and I cant understand the results I am seeing Note. This is a big hack in an old peice of code. It would not be like this if I had my way :( void SomeFn(void) { //these are not hard coded they come from another API in string form CString result = CalcMargin("0.7033", "0.7000"); } CString CalcMargin(LPCTSTR exeRate, LPCTSTR rate) { double dExeRate, dRate, dMargin; dExeRate = atof(exeRate); dRate = atof(rate); dMargin = dExeRate - dRate; result.Format("%f", dMargin); return result; } The returned value is "0.003300" and I wondered why i see the trailing zeros. It turnes out that the result of the subtraction is 0.0033000000000001 Can anyone tell me why? Thanks
-
Hello, I have the following simple code and I cant understand the results I am seeing Note. This is a big hack in an old peice of code. It would not be like this if I had my way :( void SomeFn(void) { //these are not hard coded they come from another API in string form CString result = CalcMargin("0.7033", "0.7000"); } CString CalcMargin(LPCTSTR exeRate, LPCTSTR rate) { double dExeRate, dRate, dMargin; dExeRate = atof(exeRate); dRate = atof(rate); dMargin = dExeRate - dRate; result.Format("%f", dMargin); return result; } The returned value is "0.003300" and I wondered why i see the trailing zeros. It turnes out that the result of the subtraction is 0.0033000000000001 Can anyone tell me why? Thanks
-
Hello, I have the following simple code and I cant understand the results I am seeing Note. This is a big hack in an old peice of code. It would not be like this if I had my way :( void SomeFn(void) { //these are not hard coded they come from another API in string form CString result = CalcMargin("0.7033", "0.7000"); } CString CalcMargin(LPCTSTR exeRate, LPCTSTR rate) { double dExeRate, dRate, dMargin; dExeRate = atof(exeRate); dRate = atof(rate); dMargin = dExeRate - dRate; result.Format("%f", dMargin); return result; } The returned value is "0.003300" and I wondered why i see the trailing zeros. It turnes out that the result of the subtraction is 0.0033000000000001 Can anyone tell me why? Thanks
Rounding error. Floating-point numbers cannot be stored as the exact equivalent of their decimal representation, unless they all happen to be powers of two (2, 4, .125, etc.). So you should never check for strict equality when dealing with
float/doublevalues. --Mike-- Ericahist | CP SearchBar v2.0.2 | Homepage | 1ClickPicGrabber New v2.0! | RightClick-Encrypt "Linux is good. It can do no wrong. It is open source so must be right. It has penguins. I want to eat your brain." -- Paul Watson, Linux Zombie -
Rounding error. Floating-point numbers cannot be stored as the exact equivalent of their decimal representation, unless they all happen to be powers of two (2, 4, .125, etc.). So you should never check for strict equality when dealing with
float/doublevalues. --Mike-- Ericahist | CP SearchBar v2.0.2 | Homepage | 1ClickPicGrabber New v2.0! | RightClick-Encrypt "Linux is good. It can do no wrong. It is open source so must be right. It has penguins. I want to eat your brain." -- Paul Watson, Linux Zombie -
Thanks but im not sure I understand completely. Would you expect "true" or "false" from the following code? It prints true when i run it. double d1 = 5; //not a power of 2 double d2 = 2; if( (d2 * 5.0 / 2.0) == d1) printf("true"); else printf("false");
You are using whole numbers. Try with fractions
double d1 = 1;
double d2 = 3;
double d3 = d1 / d2;
if (d3 == 0.333) // what number to use?
printf ("true");
else
printf ("false);No matter what number you use, you will never get true.
[
](http://www.canucks.com)Sonork 100.11743 Chicken Little "You're obviously a superstar." - Christian Graus about me - 12 Feb '03 Within you lies the power for good - Use it!
-
You are using whole numbers. Try with fractions
double d1 = 1;
double d2 = 3;
double d3 = d1 / d2;
if (d3 == 0.333) // what number to use?
printf ("true");
else
printf ("false);No matter what number you use, you will never get true.
[
](http://www.canucks.com)Sonork 100.11743 Chicken Little "You're obviously a superstar." - Christian Graus about me - 12 Feb '03 Within you lies the power for good - Use it!