What's y'all's favorite way of dealing with floating point precision?
-
Not sure if this counts as a programming question, since I'm not asking for code but rather preference. I'm in a project that requires complete accuracy on numbers. So, given the following... We all know the famous of examples of stuff like this:
0.1 + 0.2 // 0.30000000000000004
Up until now, I've been content with rounding off any operations after the fact and calling it a day, as close enough was good enough. For applications, say that deal with currency, the age old trick is to just use integers based on a cent value. So, a `$1.23` would be stored as `123` in a variable. Sweet, but, consider this:
// $123.45 / $2.25
12345 / 225 // 54.86666666666667If I move along powers of the base, I never run into issues. But for your typical run of the mill calculations, even with integers, you still have to deal with fractional floating points in the arithmetic. So, I've been using integers _and_ rounding off any calculations to their nearest integer value. Maybe sometimes I'll `floor` or `ceil` depending on context, but that's been my current solution, which is a lot more accurate but not 100% accurate. But, good enough-ish. Soooo.... 1) You guys prefer using a library to handle stuff like this? IMO I don't use one for arithmetic because most libraries for this (at least in JavaScript) are clunky and slow and don't really do a better job anyway. 2) You think integers and rounding is also the way to go? Keeps crap simple and all that, despite needing to remember to always round after division calculations or calculations against fractional types. 3) Never do arithmetic? Tell the user to go home.
Jeremy Falcon
"floating point precision" is an oxymoron. :laugh: Which is why I don't do math in JavaScript unless I don't care about rounding errors.
Latest Articles:
A Lightweight Thread Safe In-Memory Keyed Generic Cache Collection Service A Dynamic Where Implementation for Entity Framework -
"floating point precision" is an oxymoron. :laugh: Which is why I don't do math in JavaScript unless I don't care about rounding errors.
Latest Articles:
A Lightweight Thread Safe In-Memory Keyed Generic Cache Collection Service A Dynamic Where Implementation for Entity FrameworkYeah, but to be fair... this issue is with all languages. Edit: Unless you mean the decimal type in C#? In which case... totally gotta give props to that.
Jeremy Falcon
-
Jeremy Falcon wrote:
complete accuracy on numbers
So, not a government project then? :laugh:
"These people looked deep within my soul and assigned me a number based on the order in which I joined." - Homer
Ha ha ha ha. Nope.
Jeremy Falcon
-
I don't get how noone on this site has mentioned that .Net supports decimal type. [Floating-point numeric types - C# reference - C# | Microsoft Learn](https://learn.microsoft.com/en-us/dotnet/csharp/language-reference/builtin-types/floating-point-numeric-types) As does most proper databases.
Wrong is evil and must be defeated. - Jeff Ello
I was waiting for that. Had to give COBOL some love though.
Jeremy Falcon
-
.toFixed(x) should do the trick. And maybe even
+(0.1 + 0.2).toFixed(2), which displays 0.3 just fine X| A user never wants to see more than three digits anyway. But ultimately, I do all my calculations in C# that has a decent decimal type. I once had a customer who wanted to calculate VAT for each sales order row and then got mad the total didn't add up due to rounding errors :sigh:Best, Sander Azure DevOps Succinctly (free eBook) Azure Serverless Succinctly (free eBook) Migrating Apps to the Cloud with Azure arrgh.js - Bringing LINQ to JavaScript
Sander Rossel wrote:
.toFixed(x) should do the trick
There's actually some rounding errors you'll find with that. I have a means to do the rounding, just curious to know what's peep's favorite way.
Sander Rossel wrote:
But ultimately, I do all my calculations in C# that has a decent decimal type.
Win for C#. This project is in 100% Node, so me no get that. Btw, here's the routine I use to do more accurate rounding, if you want it...
// this part at the top of the module
// according to the ECMAScript specification, the Number type uses double-precision floating points
// which have a 64-bit format (binary64), and consists of a sign bit with 11 exponent bits and 52
// fraction bits (each digit represents 4-bits, hence 64-bit values have a max of 16 decimals)
const MAX_DECIMALS = 16;// ..........
/**
* This will round off a number, but with special considerations for decimal places in a fractional
* floating point number.
* @param {number} number The number to round off.
* @param {number} [decimals=0] The amount of decimal places, if any, to retain when rounding the number.
* @returns {number} Returns the rounded number.
*/export function roundOff(number: number, decimals = 0): number {
const exponent = Math.min(Math.max(decimals, 0), MAX_DECIMALS);
const factor = 10 ** exponent; // we're treating this as base 10// the value of Math.round(x) is the same as the value of Math.floor(x+0.5), except when x is −0 or is less
// than 0 but greater than or equal to -0.5; for these cases Math.round(x) returns −0, but
// Math.floor(x+0.5) returns +0. so, the last OR zero check looks for -0 and converts it to +0
return (Math.round(((number || 0) + Number.EPSILON) * factor) || 0) / factor;
}Sander Rossel wrote:
I once had a customer who wanted to calculate VAT for each sales order row and then got mad the total didn't add up due to rounding errors :sigh:
Good times. Good times. :laugh:
Jeremy Falcon
-
.toFixed(x) should do the trick. And maybe even
+(0.1 + 0.2).toFixed(2), which displays 0.3 just fine X| A user never wants to see more than three digits anyway. But ultimately, I do all my calculations in C# that has a decent decimal type. I once had a customer who wanted to calculate VAT for each sales order row and then got mad the total didn't add up due to rounding errors :sigh:Best, Sander Azure DevOps Succinctly (free eBook) Azure Serverless Succinctly (free eBook) Migrating Apps to the Cloud with Azure arrgh.js - Bringing LINQ to JavaScript
Forgot to mention why the use of Number.EPSILON. Consider this:
// rounding off to two decimal places
1.005.toFixed(2); // outputs 1 instead of 1.01
Math.round(1.005 * 100) / 100; // outputs 1 instead of 1.01Number.EPSILON handles these edge cases in a way that's small enough to not bother non-edge cases. Just be careful with using it though for every day calculations as it's not perfect to always rely on. But for fixing these edge cases it works great.
Jeremy Falcon
-
Sander Rossel wrote:
.toFixed(x) should do the trick
There's actually some rounding errors you'll find with that. I have a means to do the rounding, just curious to know what's peep's favorite way.
Sander Rossel wrote:
But ultimately, I do all my calculations in C# that has a decent decimal type.
Win for C#. This project is in 100% Node, so me no get that. Btw, here's the routine I use to do more accurate rounding, if you want it...
// this part at the top of the module
// according to the ECMAScript specification, the Number type uses double-precision floating points
// which have a 64-bit format (binary64), and consists of a sign bit with 11 exponent bits and 52
// fraction bits (each digit represents 4-bits, hence 64-bit values have a max of 16 decimals)
const MAX_DECIMALS = 16;// ..........
/**
* This will round off a number, but with special considerations for decimal places in a fractional
* floating point number.
* @param {number} number The number to round off.
* @param {number} [decimals=0] The amount of decimal places, if any, to retain when rounding the number.
* @returns {number} Returns the rounded number.
*/export function roundOff(number: number, decimals = 0): number {
const exponent = Math.min(Math.max(decimals, 0), MAX_DECIMALS);
const factor = 10 ** exponent; // we're treating this as base 10// the value of Math.round(x) is the same as the value of Math.floor(x+0.5), except when x is −0 or is less
// than 0 but greater than or equal to -0.5; for these cases Math.round(x) returns −0, but
// Math.floor(x+0.5) returns +0. so, the last OR zero check looks for -0 and converts it to +0
return (Math.round(((number || 0) + Number.EPSILON) * factor) || 0) / factor;
}Sander Rossel wrote:
I once had a customer who wanted to calculate VAT for each sales order row and then got mad the total didn't add up due to rounding errors :sigh:
Good times. Good times. :laugh:
Jeremy Falcon
Somehow, I've never really needed this. I consider myself a lucky man :D I always wonder why JavaScript adds new stuff such as classes, let, const, and what have you, but not a decent rounding method, or an array.remove method. Why does an array have reduce, join and flat, but not remove!? I keep googling "javascript array remove element", oh right, splice (or was it slice?), indexOf, 1 X| JavaScript really seems to be missing the basics, the bare necessities even, but somehow it keeps adding junk most people will never need.
Best, Sander Azure DevOps Succinctly (free eBook) Azure Serverless Succinctly (free eBook) Migrating Apps to the Cloud with Azure arrgh.js - Bringing LINQ to JavaScript
-
Somehow, I've never really needed this. I consider myself a lucky man :D I always wonder why JavaScript adds new stuff such as classes, let, const, and what have you, but not a decent rounding method, or an array.remove method. Why does an array have reduce, join and flat, but not remove!? I keep googling "javascript array remove element", oh right, splice (or was it slice?), indexOf, 1 X| JavaScript really seems to be missing the basics, the bare necessities even, but somehow it keeps adding junk most people will never need.
Best, Sander Azure DevOps Succinctly (free eBook) Azure Serverless Succinctly (free eBook) Migrating Apps to the Cloud with Azure arrgh.js - Bringing LINQ to JavaScript
There is a remove method for the last single element. It's called [Array.prototype.pop](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global\_Objects/Array/pop). If you want to remove chunks of an array at a time, as you mentioned there's [Array.prototype.splice](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global\_Objects/Array/splice). Not sure why the no bueno, just because it's called `splice` rather than `remove`.
Jeremy Falcon
-
Yeah, but to be fair... this issue is with all languages. Edit: Unless you mean the decimal type in C#? In which case... totally gotta give props to that.
Jeremy Falcon
Yes exactly, the decimal type in C#. I was going to mention it but it seemed like the context of your question was JavaScript. ;) And for SQL, there's a really interesting post here[^] about how to use the
moneyanddecimaltypes correctly. Learned something looking that up!Latest Articles:
A Lightweight Thread Safe In-Memory Keyed Generic Cache Collection Service A Dynamic Where Implementation for Entity Framework -
This is why COBOL has a decimal type. Your approach of using integers to represent currency will work, subject to a few caveats: 1. Accounting rules require that calculations (e.g. multiplication, division) be performed with greater than 1 cent accuracy (5 decimal places, IIRC). This allows interest calculations, currency conversions etc. to work properly. 2. Rounding is performed using accounting rules - round to nearest or AWAY. The difference between this and round to nearest or EVEN is when the fraction is exactly 0.5. If this case, one rounds AWAY from 0. For example, 3.145 would round to nearest or EVEN as 3.14, but round to nearest or AWAY as 3.15. There may be other rules for accounting, but these are the major ones.
Freedom is the freedom to say that two plus two make four. If that is granted, all else follows. -- 6079 Smith W.
-
Amarnath S wrote:
we had DOUBLE PRECISION in Fortran and double in C.
Even with binary DOUBLE PRECISION, you would get errors in accounting formulas. This is why COBOL was specified to have a decimal type.
Freedom is the freedom to say that two plus two make four. If that is granted, all else follows. -- 6079 Smith W.
-
This is why COBOL has a decimal type. Your approach of using integers to represent currency will work, subject to a few caveats: 1. Accounting rules require that calculations (e.g. multiplication, division) be performed with greater than 1 cent accuracy (5 decimal places, IIRC). This allows interest calculations, currency conversions etc. to work properly. 2. Rounding is performed using accounting rules - round to nearest or AWAY. The difference between this and round to nearest or EVEN is when the fraction is exactly 0.5. If this case, one rounds AWAY from 0. For example, 3.145 would round to nearest or EVEN as 3.14, but round to nearest or AWAY as 3.15. There may be other rules for accounting, but these are the major ones.
Freedom is the freedom to say that two plus two make four. If that is granted, all else follows. -- 6079 Smith W.
-
Okay, I'm now having flashbacks to my numerical methods class in college. One of the central points in the course was that managing precision was important, but it had to be appropriate to the calculation. You've hit on that with currency. I remember a story where a programmer at a bank embezzled a lot of money based on harvesting roundoff error in the bank's software that he wrote. There are certifications and legal mechanisms that define how calculations are performed now. Scientific computation is similar. It's hard to imagine the analysis that the Large Hadron Collider experiments require, since the data of interest is embedded in unbelievable amounts of noise. Heck, it's even a problem in my world of commercial ink-jet printing systems. Our internal unit of measurement is µinches (micro-inches) in a 64-bit signed integer. We deal with positioning data in 1/3600ths of an inch that can be noisy, and our resolution is 600 dots/inch. Using µinches solves a lot of scaling issues.
Software Zen:
delete this;This is the only correct answer: it has to be appropriate to the calculation. In some cases, perfect precision is not even mathematically possible (e.g. pi = ratio of a circle's diameter to circumference, or anything involving it like sin() or cos()). Rational libraries are great when appropriate, but (a) they can be a lot of computing for no real world benefit, and (b) if you have any intrinsically irrational number like pi in there anywhere, their usefulness instantly becomes limited (or you have to factor out the irrationals and express the answers as ratios of them...) So, yeah, in real life, calculate at higher than needed* precision and round to needed precision. For accounting, use appropriate accounting rounding rules for exact 0.5 values. *what precision? Max total of your errors all added and multiplied together appropriately according to the calculations involved must be less than half of the actual final precision.
-
Not sure if this counts as a programming question, since I'm not asking for code but rather preference. I'm in a project that requires complete accuracy on numbers. So, given the following... We all know the famous of examples of stuff like this:
0.1 + 0.2 // 0.30000000000000004
Up until now, I've been content with rounding off any operations after the fact and calling it a day, as close enough was good enough. For applications, say that deal with currency, the age old trick is to just use integers based on a cent value. So, a `$1.23` would be stored as `123` in a variable. Sweet, but, consider this:
// $123.45 / $2.25
12345 / 225 // 54.86666666666667If I move along powers of the base, I never run into issues. But for your typical run of the mill calculations, even with integers, you still have to deal with fractional floating points in the arithmetic. So, I've been using integers _and_ rounding off any calculations to their nearest integer value. Maybe sometimes I'll `floor` or `ceil` depending on context, but that's been my current solution, which is a lot more accurate but not 100% accurate. But, good enough-ish. Soooo.... 1) You guys prefer using a library to handle stuff like this? IMO I don't use one for arithmetic because most libraries for this (at least in JavaScript) are clunky and slow and don't really do a better job anyway. 2) You think integers and rounding is also the way to go? Keeps crap simple and all that, despite needing to remember to always round after division calculations or calculations against fractional types. 3) Never do arithmetic? Tell the user to go home.
Jeremy Falcon
I work in engineering applications so my answer is "I don't use floating point arithmetic", everything is fixed point - many microcontrollers do not have an FPU at all and what I do is mostly real-time so forget software emulation. Any desktop tool has to provide or receive data from these systems so it's mostly fixed point until it's time to show to the user. On non engineering applications I just use floating points as they are, if I have qualms about precision I just use double or long double and call it a day - I don't work on servers or high throughput / low latency services so I just don't care.
GCS/GE d--(d) s-/+ a C+++ U+++ P-- L+@ E-- W+++ N+ o+ K- w+++ O? M-- V? PS+ PE Y+ PGP t+ 5? X R+++ tv-- b+(+++) DI+++ D++ G e++ h--- r+++ y+++* Weapons extension: ma- k++ F+2 X The shortest horror story: On Error Resume Next
-
I was waiting for that. Had to give COBOL some love though.
Jeremy Falcon
Jeremy Falcon wrote:
Had to give COBOL some love though.
Rightfully so, COBOL is better than its reputation. And still in development, latest version is COBOL-2023.
Wrong is evil and must be defeated. - Jeff Ello
-
Sometimes I do wish JavaScript had better types like that. I can push a precision to about 9 or 10 in JavaScript before the storable value becomes too small to be worth while. Good enough for kiddie stuff at least. But yeah, also what Daniel said. :laugh:
Jeremy Falcon
Javascript's innate number type is but an approximation - everybody will agree. But the serialization of those numbers in JSON makes it worse. For example, what is really meant with 0.67, 0.667, ... , 0.66666666666667 ? Does 0.67 mean exactly 67 cents and is 0.66666666666667 to be understood as an approximation of 2/3? And does 0.667 also stand for 2/3? What does the number of decimals tell about the underlying intentions? JSON should have a standardized notation for lossless serialization of floating point type numbers. Even C has that: the %a printf format for double in hexadecimal notation. And talking about shortcomings in JSON: please also provide a notation for bignums - both integer and rational ones.
-
Not sure if this counts as a programming question, since I'm not asking for code but rather preference. I'm in a project that requires complete accuracy on numbers. So, given the following... We all know the famous of examples of stuff like this:
0.1 + 0.2 // 0.30000000000000004
Up until now, I've been content with rounding off any operations after the fact and calling it a day, as close enough was good enough. For applications, say that deal with currency, the age old trick is to just use integers based on a cent value. So, a `$1.23` would be stored as `123` in a variable. Sweet, but, consider this:
// $123.45 / $2.25
12345 / 225 // 54.86666666666667If I move along powers of the base, I never run into issues. But for your typical run of the mill calculations, even with integers, you still have to deal with fractional floating points in the arithmetic. So, I've been using integers _and_ rounding off any calculations to their nearest integer value. Maybe sometimes I'll `floor` or `ceil` depending on context, but that's been my current solution, which is a lot more accurate but not 100% accurate. But, good enough-ish. Soooo.... 1) You guys prefer using a library to handle stuff like this? IMO I don't use one for arithmetic because most libraries for this (at least in JavaScript) are clunky and slow and don't really do a better job anyway. 2) You think integers and rounding is also the way to go? Keeps crap simple and all that, despite needing to remember to always round after division calculations or calculations against fractional types. 3) Never do arithmetic? Tell the user to go home.
Jeremy Falcon
When seeking consistency, I would wrap all of that in a utility class. This way I know it's consistent and works the same everywhere. If I need something a little different, then I either overload or add a default parameter as the occasion requires. I agree that storing the numerator and denominator would be the best way to prevent most headaches. In C#, I would use a Fraction struct for this (home grown if one doesn't exist already). Only collapse the fraction to a primitive type as necessary. This also has the benefit of letting you use money with a decimal value, so you don't have to do the extra math to get the cents back. Speaking of money, you only have to store 4 decimals with money to be accurate for accounting purposes. I deal with fiduciary escrow accounts for my job, and that's all we've every used. Never had a problem being out of balance by a penny in 24 years. However, we don't do multiplication and division on the money. I don't think that would change much though as long as you kept the division and rounding to the end of the math problem.
Bond Keep all things as simple as possible, but no simpler. -said someone, somewhere
-
Not sure if this counts as a programming question, since I'm not asking for code but rather preference. I'm in a project that requires complete accuracy on numbers. So, given the following... We all know the famous of examples of stuff like this:
0.1 + 0.2 // 0.30000000000000004
Up until now, I've been content with rounding off any operations after the fact and calling it a day, as close enough was good enough. For applications, say that deal with currency, the age old trick is to just use integers based on a cent value. So, a `$1.23` would be stored as `123` in a variable. Sweet, but, consider this:
// $123.45 / $2.25
12345 / 225 // 54.86666666666667If I move along powers of the base, I never run into issues. But for your typical run of the mill calculations, even with integers, you still have to deal with fractional floating points in the arithmetic. So, I've been using integers _and_ rounding off any calculations to their nearest integer value. Maybe sometimes I'll `floor` or `ceil` depending on context, but that's been my current solution, which is a lot more accurate but not 100% accurate. But, good enough-ish. Soooo.... 1) You guys prefer using a library to handle stuff like this? IMO I don't use one for arithmetic because most libraries for this (at least in JavaScript) are clunky and slow and don't really do a better job anyway. 2) You think integers and rounding is also the way to go? Keeps crap simple and all that, despite needing to remember to always round after division calculations or calculations against fractional types. 3) Never do arithmetic? Tell the user to go home.
Jeremy Falcon
One thing to consider, there is a worldwide standard that most operating systems follow, IEEE 64-BIT. To me it wouldn't be unreasonable to follow that. Anyone situation that requires more would be highly specialized.
-
Not sure if this counts as a programming question, since I'm not asking for code but rather preference. I'm in a project that requires complete accuracy on numbers. So, given the following... We all know the famous of examples of stuff like this:
0.1 + 0.2 // 0.30000000000000004
Up until now, I've been content with rounding off any operations after the fact and calling it a day, as close enough was good enough. For applications, say that deal with currency, the age old trick is to just use integers based on a cent value. So, a `$1.23` would be stored as `123` in a variable. Sweet, but, consider this:
// $123.45 / $2.25
12345 / 225 // 54.86666666666667If I move along powers of the base, I never run into issues. But for your typical run of the mill calculations, even with integers, you still have to deal with fractional floating points in the arithmetic. So, I've been using integers _and_ rounding off any calculations to their nearest integer value. Maybe sometimes I'll `floor` or `ceil` depending on context, but that's been my current solution, which is a lot more accurate but not 100% accurate. But, good enough-ish. Soooo.... 1) You guys prefer using a library to handle stuff like this? IMO I don't use one for arithmetic because most libraries for this (at least in JavaScript) are clunky and slow and don't really do a better job anyway. 2) You think integers and rounding is also the way to go? Keeps crap simple and all that, despite needing to remember to always round after division calculations or calculations against fractional types. 3) Never do arithmetic? Tell the user to go home.
Jeremy Falcon
Financial apps as far as I know use four decimal digits during calculations to avoid problems with only two, then round at the end.
-
Not sure if this counts as a programming question, since I'm not asking for code but rather preference. I'm in a project that requires complete accuracy on numbers. So, given the following... We all know the famous of examples of stuff like this:
0.1 + 0.2 // 0.30000000000000004
Up until now, I've been content with rounding off any operations after the fact and calling it a day, as close enough was good enough. For applications, say that deal with currency, the age old trick is to just use integers based on a cent value. So, a `$1.23` would be stored as `123` in a variable. Sweet, but, consider this:
// $123.45 / $2.25
12345 / 225 // 54.86666666666667If I move along powers of the base, I never run into issues. But for your typical run of the mill calculations, even with integers, you still have to deal with fractional floating points in the arithmetic. So, I've been using integers _and_ rounding off any calculations to their nearest integer value. Maybe sometimes I'll `floor` or `ceil` depending on context, but that's been my current solution, which is a lot more accurate but not 100% accurate. But, good enough-ish. Soooo.... 1) You guys prefer using a library to handle stuff like this? IMO I don't use one for arithmetic because most libraries for this (at least in JavaScript) are clunky and slow and don't really do a better job anyway. 2) You think integers and rounding is also the way to go? Keeps crap simple and all that, despite needing to remember to always round after division calculations or calculations against fractional types. 3) Never do arithmetic? Tell the user to go home.
Jeremy Falcon
"project that requires complete accuracy on numbers" A couple of thoughts. Is not the above requirement impossible on a binary system? By the very definition, you are going to lose precision be it float, double, double double.... how far do you want to go? For me, I work a lot in machine HMIs. Some users want metric, others want English. I've always had a requirement to allow the user to switch between units and maintaining what is displayed. For example, 1" is 25.4 mm. If I switch between metric and English, the value must be consistent. As for complete accuracy - this for me has always fallen into fixed point arithmetic to avoid rounding errors. COBOL has been mentioned. I've done COBOL - a very long time ago, but as I recall, it did fixed point arithmetic very well. Or I might be missing something... Please elaborate on what you mean for "complete accuracy"? This sounds like a requirement from someone who really does not understand their request - sort of like a rare steak, but the temp should be 175F....
Charlie Gilley “They who can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety.” BF, 1759 Has never been more appropriate.
