When should printf round a floating number up for display?
-
Specifically this statement:
printf("Chi square distribution for %s samples is %1.2f (%1.4f), and randomly\\n", String\_Value, chisq, chisq);I re-implemented this program in MASM to speed it up and I get the following difference (note that I added the (%1.4f)) to display the pre-rounded number to see why I was getting differences:
------------------------------------------------------------------ 17
Chi square distribution for 2048 samples is 6.13 (6.1250), and randomly
-------------------------------------------------------------- 17
Chi square distribution for 2048 samples is 6.12 (6.1250), and randomlyThe question is, which one is correct according to the C spec? Should 6.1250 round up to 6.13 or should it take 6.1250+ to round up? There are other questions such as the following:
printf("of this %s %s file by %d (%2.2f) percent.\\n\\n", String\_Value, samp, (short) (((100 \* ((binary ? 1 : 8) - ent) / (binary ? 1.0 : 8.0)) + 0.5)), ((100 \* ((binary ? 1 : 8) - ent) / (binary ? 1.0 : 8.0))));The cast to a (short) seems to truncate a 5.7 to 5, I had to add the (.... + 0.5) just before the cast in order to get the rounded number. Again, I added the (%2.2f) to display the rounding differences. Which is correct according to the C spec, rounding or truncation? The program being re implemented is John Walker's ENT. Dave.
-
Specifically this statement:
printf("Chi square distribution for %s samples is %1.2f (%1.4f), and randomly\\n", String\_Value, chisq, chisq);I re-implemented this program in MASM to speed it up and I get the following difference (note that I added the (%1.4f)) to display the pre-rounded number to see why I was getting differences:
------------------------------------------------------------------ 17
Chi square distribution for 2048 samples is 6.13 (6.1250), and randomly
-------------------------------------------------------------- 17
Chi square distribution for 2048 samples is 6.12 (6.1250), and randomlyThe question is, which one is correct according to the C spec? Should 6.1250 round up to 6.13 or should it take 6.1250+ to round up? There are other questions such as the following:
printf("of this %s %s file by %d (%2.2f) percent.\\n\\n", String\_Value, samp, (short) (((100 \* ((binary ? 1 : 8) - ent) / (binary ? 1.0 : 8.0)) + 0.5)), ((100 \* ((binary ? 1 : 8) - ent) / (binary ? 1.0 : 8.0))));The cast to a (short) seems to truncate a 5.7 to 5, I had to add the (.... + 0.5) just before the cast in order to get the rounded number. Again, I added the (%2.2f) to display the rounding differences. Which is correct according to the C spec, rounding or truncation? The program being re implemented is John Walker's ENT. Dave.
I think the rules for rounding when a floating point's decimal part ends in 5 is, round up if the previous digit is even, and down if it's odd. So 6.125 rounds up to 6.13 but 6.115 would round down to 6.11; this is easy to check. In the second instance if you cast a float to an integer it discards the decimal part.
One of these days I'm going to think of a really clever signature.
-
I think the rules for rounding when a floating point's decimal part ends in 5 is, round up if the previous digit is even, and down if it's odd. So 6.125 rounds up to 6.13 but 6.115 would round down to 6.11; this is easy to check. In the second instance if you cast a float to an integer it discards the decimal part.
One of these days I'm going to think of a really clever signature.
Richard, Very interesting. I pulled up my 8087 book "THE 8087 PRIMER", John F. Palmer and Steven P. Morse and looked up rounding. The nearest to what you described is the "round to nearest Even" mode, not the round to the nearest Odd as you described. Are you sure about this? Is it in some C spec? If you are programming the FPU and want to cast a float to an integer, you just execute an FIST or FISTP, which will round to the specified rounding mode which is usually "round to nearest Even" unless you specifically set the mode to "round to 0" before you store the value. Dave.
-
Richard, Very interesting. I pulled up my 8087 book "THE 8087 PRIMER", John F. Palmer and Steven P. Morse and looked up rounding. The nearest to what you described is the "round to nearest Even" mode, not the round to the nearest Odd as you described. Are you sure about this? Is it in some C spec? If you are programming the FPU and want to cast a float to an integer, you just execute an FIST or FISTP, which will round to the specified rounding mode which is usually "round to nearest Even" unless you specifically set the mode to "round to 0" before you store the value. Dave.
-
Well I did not have a reference to hand so it's just from what I recall during other development. However, you seem to have more up to date information than me.
One of these days I'm going to think of a really clever signature.
Richard, I'm not trying to argue, just trying to find out what the spec says. "THE 8087 PRIMER" is just a spec, from the Senior Staff Engineers at Intel Corp. explaining why they did what they did. How C uses the hardware should be specified somewhere. Dave.
-
Richard, I'm not trying to argue, just trying to find out what the spec says. "THE 8087 PRIMER" is just a spec, from the Senior Staff Engineers at Intel Corp. explaining why they did what they did. How C uses the hardware should be specified somewhere. Dave.
Member 4194593 wrote:
I'm not trying to argue
I did not mean to imply that you were; I am more than happy to have my assumptions corrected. As to what the C/C++ compiler should do in these cases, you could look at http://www.stroustrup.com/[^], or the Microsoft reference pages[^].
One of these days I'm going to think of a really clever signature.
-
Member 4194593 wrote:
I'm not trying to argue
I did not mean to imply that you were; I am more than happy to have my assumptions corrected. As to what the C/C++ compiler should do in these cases, you could look at http://www.stroustrup.com/[^], or the Microsoft reference pages[^].
One of these days I'm going to think of a really clever signature.
Richard, Thank you for the links. I'll report back with anything I find (if I can understand it). Dave.
-
Member 4194593 wrote:
I'm not trying to argue
I did not mean to imply that you were; I am more than happy to have my assumptions corrected. As to what the C/C++ compiler should do in these cases, you could look at http://www.stroustrup.com/[^], or the Microsoft reference pages[^].
One of these days I'm going to think of a really clever signature.
Richard, I did find out that a float cast as a (short) is truncated, but not yet found out about rounding in printf. Dave.
-
Member 4194593 wrote:
I'm not trying to argue
I did not mean to imply that you were; I am more than happy to have my assumptions corrected. As to what the C/C++ compiler should do in these cases, you could look at http://www.stroustrup.com/[^], or the Microsoft reference pages[^].
One of these days I'm going to think of a really clever signature.
Richard, I did find out only that the number will be rounded to the number of specified precision digits, but there is no mention of which rounding method is used! I did send feedback that I was dissatisfied with their answer. Dave.
-
Specifically this statement:
printf("Chi square distribution for %s samples is %1.2f (%1.4f), and randomly\\n", String\_Value, chisq, chisq);I re-implemented this program in MASM to speed it up and I get the following difference (note that I added the (%1.4f)) to display the pre-rounded number to see why I was getting differences:
------------------------------------------------------------------ 17
Chi square distribution for 2048 samples is 6.13 (6.1250), and randomly
-------------------------------------------------------------- 17
Chi square distribution for 2048 samples is 6.12 (6.1250), and randomlyThe question is, which one is correct according to the C spec? Should 6.1250 round up to 6.13 or should it take 6.1250+ to round up? There are other questions such as the following:
printf("of this %s %s file by %d (%2.2f) percent.\\n\\n", String\_Value, samp, (short) (((100 \* ((binary ? 1 : 8) - ent) / (binary ? 1.0 : 8.0)) + 0.5)), ((100 \* ((binary ? 1 : 8) - ent) / (binary ? 1.0 : 8.0))));The cast to a (short) seems to truncate a 5.7 to 5, I had to add the (.... + 0.5) just before the cast in order to get the rounded number. Again, I added the (%2.2f) to display the rounding differences. Which is correct according to the C spec, rounding or truncation? The program being re implemented is John Walker's ENT. Dave.
Member 4194593 wrote:
The question is, which one is correct according to the C spec?
Depends on what you mean by "C spec". Prior to C99 it was left up to the implementation how rounding occurred. According to the following (but not verified by me) C99 specifies how rounding is done (search for C99) http://linuxgazette.net/144/misc/lg/a_question_of_rounding_in_issue_143.html[^]
-
Member 4194593 wrote:
The question is, which one is correct according to the C spec?
Depends on what you mean by "C spec". Prior to C99 it was left up to the implementation how rounding occurred. According to the following (but not verified by me) C99 specifies how rounding is done (search for C99) http://linuxgazette.net/144/misc/lg/a_question_of_rounding_in_issue_143.html[^]
jschell, Thank you for the excellent link, at least it references a spec, namely IEEE. Personally, I do not think it really makes any difference what the least significant digit is rounded to, but when you are converting a program and want to insure that the conversion is accurate, and the program has many options, the easiest approach is a batch file calling the two programs, supplying the different input files and options to be processed, and then using a file compare routine to compare the outputs for each test instance. In the case of ENT compiled under Visual Studio 2008, I can see truncation, rounding to an even, and common rounding, all in one program. Makes it a bit difficult to "emulate" the original program. I have examined all of the test output (the diffs) for all of the test instances, and all of my results match the ENT results except possibly for that last digit. At this point I am calling this a valid conversion, and a huge learning experience. At least my version is about 5 times as fast for huge files as ENT (16 GB - all DWORD values for a full 2^32 period) 6 min vs 28 min. Dave.