What's y'all's favorite way of dealing with floating point precision?
-
In good old days, we had
DOUBLE PRECISIONin Fortran anddoublein C. At least these had better precision thanREALandfloat. Somehow, it seems that the designers of later languages put more emphasis onstrings, perhaps, because layman-type of users/applications started outnumbering scientific/numerical users/applications.Sometimes I do wish JavaScript had better types like that. I can push a precision to about 9 or 10 in JavaScript before the storable value becomes too small to be worth while. Good enough for kiddie stuff at least. But yeah, also what Daniel said. :laugh:
Jeremy Falcon
-
Daniel Pfeffer wrote:
This is why COBOL has a decimal type.
:laugh: :laugh: :laugh: :laugh: Never thought I'd say this about COBOL, but that's cool.
Daniel Pfeffer wrote:
There may be other rules for accounting, but these are the major ones.
Thanks for this man.
Jeremy Falcon
I don't get how noone on this site has mentioned that .Net supports decimal type. [Floating-point numeric types - C# reference - C# | Microsoft Learn](https://learn.microsoft.com/en-us/dotnet/csharp/language-reference/builtin-types/floating-point-numeric-types) As does most proper databases.
Wrong is evil and must be defeated. - Jeff Ello
-
Not sure if this counts as a programming question, since I'm not asking for code but rather preference. I'm in a project that requires complete accuracy on numbers. So, given the following... We all know the famous of examples of stuff like this:
0.1 + 0.2 // 0.30000000000000004
Up until now, I've been content with rounding off any operations after the fact and calling it a day, as close enough was good enough. For applications, say that deal with currency, the age old trick is to just use integers based on a cent value. So, a `$1.23` would be stored as `123` in a variable. Sweet, but, consider this:
// $123.45 / $2.25
12345 / 225 // 54.86666666666667If I move along powers of the base, I never run into issues. But for your typical run of the mill calculations, even with integers, you still have to deal with fractional floating points in the arithmetic. So, I've been using integers _and_ rounding off any calculations to their nearest integer value. Maybe sometimes I'll `floor` or `ceil` depending on context, but that's been my current solution, which is a lot more accurate but not 100% accurate. But, good enough-ish. Soooo.... 1) You guys prefer using a library to handle stuff like this? IMO I don't use one for arithmetic because most libraries for this (at least in JavaScript) are clunky and slow and don't really do a better job anyway. 2) You think integers and rounding is also the way to go? Keeps crap simple and all that, despite needing to remember to always round after division calculations or calculations against fractional types. 3) Never do arithmetic? Tell the user to go home.
Jeremy Falcon
.toFixed(x) should do the trick. And maybe even
+(0.1 + 0.2).toFixed(2), which displays 0.3 just fine X| A user never wants to see more than three digits anyway. But ultimately, I do all my calculations in C# that has a decent decimal type. I once had a customer who wanted to calculate VAT for each sales order row and then got mad the total didn't add up due to rounding errors :sigh:Best, Sander Azure DevOps Succinctly (free eBook) Azure Serverless Succinctly (free eBook) Migrating Apps to the Cloud with Azure arrgh.js - Bringing LINQ to JavaScript
-
I would argue that “complete accuracy on numbers” doesn’t exist. 1/3 cannot be represented exactly. You can do tricks like working with fractions and postponing the actual division until the very end, but in the end, if you have to print the result, you will have to print an approximation.
Mircea
One effective approach for managing floating-point precision is using relative comparisons or epsilon comparisons. This involves comparing the absolute difference between two floating-point numbers with a small epsilon value, rather than comparing them directly. Additionally, utilizing data types like BigDecimal in Java or Decimal in Python can offer higher precision when dealing with financial or critical calculations to mitigate rounding errors.
-
Not sure if this counts as a programming question, since I'm not asking for code but rather preference. I'm in a project that requires complete accuracy on numbers. So, given the following... We all know the famous of examples of stuff like this:
0.1 + 0.2 // 0.30000000000000004
Up until now, I've been content with rounding off any operations after the fact and calling it a day, as close enough was good enough. For applications, say that deal with currency, the age old trick is to just use integers based on a cent value. So, a `$1.23` would be stored as `123` in a variable. Sweet, but, consider this:
// $123.45 / $2.25
12345 / 225 // 54.86666666666667If I move along powers of the base, I never run into issues. But for your typical run of the mill calculations, even with integers, you still have to deal with fractional floating points in the arithmetic. So, I've been using integers _and_ rounding off any calculations to their nearest integer value. Maybe sometimes I'll `floor` or `ceil` depending on context, but that's been my current solution, which is a lot more accurate but not 100% accurate. But, good enough-ish. Soooo.... 1) You guys prefer using a library to handle stuff like this? IMO I don't use one for arithmetic because most libraries for this (at least in JavaScript) are clunky and slow and don't really do a better job anyway. 2) You think integers and rounding is also the way to go? Keeps crap simple and all that, despite needing to remember to always round after division calculations or calculations against fractional types. 3) Never do arithmetic? Tell the user to go home.
Jeremy Falcon
Jeremy Falcon wrote:
complete accuracy on numbers
So, not a government project then? :laugh:
"These people looked deep within my soul and assigned me a number based on the order in which I joined." - Homer
-
Not sure if this counts as a programming question, since I'm not asking for code but rather preference. I'm in a project that requires complete accuracy on numbers. So, given the following... We all know the famous of examples of stuff like this:
0.1 + 0.2 // 0.30000000000000004
Up until now, I've been content with rounding off any operations after the fact and calling it a day, as close enough was good enough. For applications, say that deal with currency, the age old trick is to just use integers based on a cent value. So, a `$1.23` would be stored as `123` in a variable. Sweet, but, consider this:
// $123.45 / $2.25
12345 / 225 // 54.86666666666667If I move along powers of the base, I never run into issues. But for your typical run of the mill calculations, even with integers, you still have to deal with fractional floating points in the arithmetic. So, I've been using integers _and_ rounding off any calculations to their nearest integer value. Maybe sometimes I'll `floor` or `ceil` depending on context, but that's been my current solution, which is a lot more accurate but not 100% accurate. But, good enough-ish. Soooo.... 1) You guys prefer using a library to handle stuff like this? IMO I don't use one for arithmetic because most libraries for this (at least in JavaScript) are clunky and slow and don't really do a better job anyway. 2) You think integers and rounding is also the way to go? Keeps crap simple and all that, despite needing to remember to always round after division calculations or calculations against fractional types. 3) Never do arithmetic? Tell the user to go home.
Jeremy Falcon
Okay, I'm now having flashbacks to my numerical methods class in college. One of the central points in the course was that managing precision was important, but it had to be appropriate to the calculation. You've hit on that with currency. I remember a story where a programmer at a bank embezzled a lot of money based on harvesting roundoff error in the bank's software that he wrote. There are certifications and legal mechanisms that define how calculations are performed now. Scientific computation is similar. It's hard to imagine the analysis that the Large Hadron Collider experiments require, since the data of interest is embedded in unbelievable amounts of noise. Heck, it's even a problem in my world of commercial ink-jet printing systems. Our internal unit of measurement is µinches (micro-inches) in a 64-bit signed integer. We deal with positioning data in 1/3600ths of an inch that can be noisy, and our resolution is 600 dots/inch. Using µinches solves a lot of scaling issues.
Software Zen:
delete this; -
Not sure if this counts as a programming question, since I'm not asking for code but rather preference. I'm in a project that requires complete accuracy on numbers. So, given the following... We all know the famous of examples of stuff like this:
0.1 + 0.2 // 0.30000000000000004
Up until now, I've been content with rounding off any operations after the fact and calling it a day, as close enough was good enough. For applications, say that deal with currency, the age old trick is to just use integers based on a cent value. So, a `$1.23` would be stored as `123` in a variable. Sweet, but, consider this:
// $123.45 / $2.25
12345 / 225 // 54.86666666666667If I move along powers of the base, I never run into issues. But for your typical run of the mill calculations, even with integers, you still have to deal with fractional floating points in the arithmetic. So, I've been using integers _and_ rounding off any calculations to their nearest integer value. Maybe sometimes I'll `floor` or `ceil` depending on context, but that's been my current solution, which is a lot more accurate but not 100% accurate. But, good enough-ish. Soooo.... 1) You guys prefer using a library to handle stuff like this? IMO I don't use one for arithmetic because most libraries for this (at least in JavaScript) are clunky and slow and don't really do a better job anyway. 2) You think integers and rounding is also the way to go? Keeps crap simple and all that, despite needing to remember to always round after division calculations or calculations against fractional types. 3) Never do arithmetic? Tell the user to go home.
Jeremy Falcon
"floating point precision" is an oxymoron. :laugh: Which is why I don't do math in JavaScript unless I don't care about rounding errors.
Latest Articles:
A Lightweight Thread Safe In-Memory Keyed Generic Cache Collection Service A Dynamic Where Implementation for Entity Framework -
"floating point precision" is an oxymoron. :laugh: Which is why I don't do math in JavaScript unless I don't care about rounding errors.
Latest Articles:
A Lightweight Thread Safe In-Memory Keyed Generic Cache Collection Service A Dynamic Where Implementation for Entity FrameworkYeah, but to be fair... this issue is with all languages. Edit: Unless you mean the decimal type in C#? In which case... totally gotta give props to that.
Jeremy Falcon
-
Jeremy Falcon wrote:
complete accuracy on numbers
So, not a government project then? :laugh:
"These people looked deep within my soul and assigned me a number based on the order in which I joined." - Homer
Ha ha ha ha. Nope.
Jeremy Falcon
-
I don't get how noone on this site has mentioned that .Net supports decimal type. [Floating-point numeric types - C# reference - C# | Microsoft Learn](https://learn.microsoft.com/en-us/dotnet/csharp/language-reference/builtin-types/floating-point-numeric-types) As does most proper databases.
Wrong is evil and must be defeated. - Jeff Ello
I was waiting for that. Had to give COBOL some love though.
Jeremy Falcon
-
.toFixed(x) should do the trick. And maybe even
+(0.1 + 0.2).toFixed(2), which displays 0.3 just fine X| A user never wants to see more than three digits anyway. But ultimately, I do all my calculations in C# that has a decent decimal type. I once had a customer who wanted to calculate VAT for each sales order row and then got mad the total didn't add up due to rounding errors :sigh:Best, Sander Azure DevOps Succinctly (free eBook) Azure Serverless Succinctly (free eBook) Migrating Apps to the Cloud with Azure arrgh.js - Bringing LINQ to JavaScript
Sander Rossel wrote:
.toFixed(x) should do the trick
There's actually some rounding errors you'll find with that. I have a means to do the rounding, just curious to know what's peep's favorite way.
Sander Rossel wrote:
But ultimately, I do all my calculations in C# that has a decent decimal type.
Win for C#. This project is in 100% Node, so me no get that. Btw, here's the routine I use to do more accurate rounding, if you want it...
// this part at the top of the module
// according to the ECMAScript specification, the Number type uses double-precision floating points
// which have a 64-bit format (binary64), and consists of a sign bit with 11 exponent bits and 52
// fraction bits (each digit represents 4-bits, hence 64-bit values have a max of 16 decimals)
const MAX_DECIMALS = 16;// ..........
/**
* This will round off a number, but with special considerations for decimal places in a fractional
* floating point number.
* @param {number} number The number to round off.
* @param {number} [decimals=0] The amount of decimal places, if any, to retain when rounding the number.
* @returns {number} Returns the rounded number.
*/export function roundOff(number: number, decimals = 0): number {
const exponent = Math.min(Math.max(decimals, 0), MAX_DECIMALS);
const factor = 10 ** exponent; // we're treating this as base 10// the value of Math.round(x) is the same as the value of Math.floor(x+0.5), except when x is −0 or is less
// than 0 but greater than or equal to -0.5; for these cases Math.round(x) returns −0, but
// Math.floor(x+0.5) returns +0. so, the last OR zero check looks for -0 and converts it to +0
return (Math.round(((number || 0) + Number.EPSILON) * factor) || 0) / factor;
}Sander Rossel wrote:
I once had a customer who wanted to calculate VAT for each sales order row and then got mad the total didn't add up due to rounding errors :sigh:
Good times. Good times. :laugh:
Jeremy Falcon
-
.toFixed(x) should do the trick. And maybe even
+(0.1 + 0.2).toFixed(2), which displays 0.3 just fine X| A user never wants to see more than three digits anyway. But ultimately, I do all my calculations in C# that has a decent decimal type. I once had a customer who wanted to calculate VAT for each sales order row and then got mad the total didn't add up due to rounding errors :sigh:Best, Sander Azure DevOps Succinctly (free eBook) Azure Serverless Succinctly (free eBook) Migrating Apps to the Cloud with Azure arrgh.js - Bringing LINQ to JavaScript
Forgot to mention why the use of Number.EPSILON. Consider this:
// rounding off to two decimal places
1.005.toFixed(2); // outputs 1 instead of 1.01
Math.round(1.005 * 100) / 100; // outputs 1 instead of 1.01Number.EPSILON handles these edge cases in a way that's small enough to not bother non-edge cases. Just be careful with using it though for every day calculations as it's not perfect to always rely on. But for fixing these edge cases it works great.
Jeremy Falcon
-
Sander Rossel wrote:
.toFixed(x) should do the trick
There's actually some rounding errors you'll find with that. I have a means to do the rounding, just curious to know what's peep's favorite way.
Sander Rossel wrote:
But ultimately, I do all my calculations in C# that has a decent decimal type.
Win for C#. This project is in 100% Node, so me no get that. Btw, here's the routine I use to do more accurate rounding, if you want it...
// this part at the top of the module
// according to the ECMAScript specification, the Number type uses double-precision floating points
// which have a 64-bit format (binary64), and consists of a sign bit with 11 exponent bits and 52
// fraction bits (each digit represents 4-bits, hence 64-bit values have a max of 16 decimals)
const MAX_DECIMALS = 16;// ..........
/**
* This will round off a number, but with special considerations for decimal places in a fractional
* floating point number.
* @param {number} number The number to round off.
* @param {number} [decimals=0] The amount of decimal places, if any, to retain when rounding the number.
* @returns {number} Returns the rounded number.
*/export function roundOff(number: number, decimals = 0): number {
const exponent = Math.min(Math.max(decimals, 0), MAX_DECIMALS);
const factor = 10 ** exponent; // we're treating this as base 10// the value of Math.round(x) is the same as the value of Math.floor(x+0.5), except when x is −0 or is less
// than 0 but greater than or equal to -0.5; for these cases Math.round(x) returns −0, but
// Math.floor(x+0.5) returns +0. so, the last OR zero check looks for -0 and converts it to +0
return (Math.round(((number || 0) + Number.EPSILON) * factor) || 0) / factor;
}Sander Rossel wrote:
I once had a customer who wanted to calculate VAT for each sales order row and then got mad the total didn't add up due to rounding errors :sigh:
Good times. Good times. :laugh:
Jeremy Falcon
Somehow, I've never really needed this. I consider myself a lucky man :D I always wonder why JavaScript adds new stuff such as classes, let, const, and what have you, but not a decent rounding method, or an array.remove method. Why does an array have reduce, join and flat, but not remove!? I keep googling "javascript array remove element", oh right, splice (or was it slice?), indexOf, 1 X| JavaScript really seems to be missing the basics, the bare necessities even, but somehow it keeps adding junk most people will never need.
Best, Sander Azure DevOps Succinctly (free eBook) Azure Serverless Succinctly (free eBook) Migrating Apps to the Cloud with Azure arrgh.js - Bringing LINQ to JavaScript
-
Somehow, I've never really needed this. I consider myself a lucky man :D I always wonder why JavaScript adds new stuff such as classes, let, const, and what have you, but not a decent rounding method, or an array.remove method. Why does an array have reduce, join and flat, but not remove!? I keep googling "javascript array remove element", oh right, splice (or was it slice?), indexOf, 1 X| JavaScript really seems to be missing the basics, the bare necessities even, but somehow it keeps adding junk most people will never need.
Best, Sander Azure DevOps Succinctly (free eBook) Azure Serverless Succinctly (free eBook) Migrating Apps to the Cloud with Azure arrgh.js - Bringing LINQ to JavaScript
There is a remove method for the last single element. It's called [Array.prototype.pop](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global\_Objects/Array/pop). If you want to remove chunks of an array at a time, as you mentioned there's [Array.prototype.splice](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global\_Objects/Array/splice). Not sure why the no bueno, just because it's called `splice` rather than `remove`.
Jeremy Falcon
-
Yeah, but to be fair... this issue is with all languages. Edit: Unless you mean the decimal type in C#? In which case... totally gotta give props to that.
Jeremy Falcon
Yes exactly, the decimal type in C#. I was going to mention it but it seemed like the context of your question was JavaScript. ;) And for SQL, there's a really interesting post here[^] about how to use the
moneyanddecimaltypes correctly. Learned something looking that up!Latest Articles:
A Lightweight Thread Safe In-Memory Keyed Generic Cache Collection Service A Dynamic Where Implementation for Entity Framework -
This is why COBOL has a decimal type. Your approach of using integers to represent currency will work, subject to a few caveats: 1. Accounting rules require that calculations (e.g. multiplication, division) be performed with greater than 1 cent accuracy (5 decimal places, IIRC). This allows interest calculations, currency conversions etc. to work properly. 2. Rounding is performed using accounting rules - round to nearest or AWAY. The difference between this and round to nearest or EVEN is when the fraction is exactly 0.5. If this case, one rounds AWAY from 0. For example, 3.145 would round to nearest or EVEN as 3.14, but round to nearest or AWAY as 3.15. There may be other rules for accounting, but these are the major ones.
Freedom is the freedom to say that two plus two make four. If that is granted, all else follows. -- 6079 Smith W.
-
Amarnath S wrote:
we had DOUBLE PRECISION in Fortran and double in C.
Even with binary DOUBLE PRECISION, you would get errors in accounting formulas. This is why COBOL was specified to have a decimal type.
Freedom is the freedom to say that two plus two make four. If that is granted, all else follows. -- 6079 Smith W.
-
This is why COBOL has a decimal type. Your approach of using integers to represent currency will work, subject to a few caveats: 1. Accounting rules require that calculations (e.g. multiplication, division) be performed with greater than 1 cent accuracy (5 decimal places, IIRC). This allows interest calculations, currency conversions etc. to work properly. 2. Rounding is performed using accounting rules - round to nearest or AWAY. The difference between this and round to nearest or EVEN is when the fraction is exactly 0.5. If this case, one rounds AWAY from 0. For example, 3.145 would round to nearest or EVEN as 3.14, but round to nearest or AWAY as 3.15. There may be other rules for accounting, but these are the major ones.
Freedom is the freedom to say that two plus two make four. If that is granted, all else follows. -- 6079 Smith W.
-
Okay, I'm now having flashbacks to my numerical methods class in college. One of the central points in the course was that managing precision was important, but it had to be appropriate to the calculation. You've hit on that with currency. I remember a story where a programmer at a bank embezzled a lot of money based on harvesting roundoff error in the bank's software that he wrote. There are certifications and legal mechanisms that define how calculations are performed now. Scientific computation is similar. It's hard to imagine the analysis that the Large Hadron Collider experiments require, since the data of interest is embedded in unbelievable amounts of noise. Heck, it's even a problem in my world of commercial ink-jet printing systems. Our internal unit of measurement is µinches (micro-inches) in a 64-bit signed integer. We deal with positioning data in 1/3600ths of an inch that can be noisy, and our resolution is 600 dots/inch. Using µinches solves a lot of scaling issues.
Software Zen:
delete this;This is the only correct answer: it has to be appropriate to the calculation. In some cases, perfect precision is not even mathematically possible (e.g. pi = ratio of a circle's diameter to circumference, or anything involving it like sin() or cos()). Rational libraries are great when appropriate, but (a) they can be a lot of computing for no real world benefit, and (b) if you have any intrinsically irrational number like pi in there anywhere, their usefulness instantly becomes limited (or you have to factor out the irrationals and express the answers as ratios of them...) So, yeah, in real life, calculate at higher than needed* precision and round to needed precision. For accounting, use appropriate accounting rounding rules for exact 0.5 values. *what precision? Max total of your errors all added and multiplied together appropriately according to the calculations involved must be less than half of the actual final precision.
-
Not sure if this counts as a programming question, since I'm not asking for code but rather preference. I'm in a project that requires complete accuracy on numbers. So, given the following... We all know the famous of examples of stuff like this:
0.1 + 0.2 // 0.30000000000000004
Up until now, I've been content with rounding off any operations after the fact and calling it a day, as close enough was good enough. For applications, say that deal with currency, the age old trick is to just use integers based on a cent value. So, a `$1.23` would be stored as `123` in a variable. Sweet, but, consider this:
// $123.45 / $2.25
12345 / 225 // 54.86666666666667If I move along powers of the base, I never run into issues. But for your typical run of the mill calculations, even with integers, you still have to deal with fractional floating points in the arithmetic. So, I've been using integers _and_ rounding off any calculations to their nearest integer value. Maybe sometimes I'll `floor` or `ceil` depending on context, but that's been my current solution, which is a lot more accurate but not 100% accurate. But, good enough-ish. Soooo.... 1) You guys prefer using a library to handle stuff like this? IMO I don't use one for arithmetic because most libraries for this (at least in JavaScript) are clunky and slow and don't really do a better job anyway. 2) You think integers and rounding is also the way to go? Keeps crap simple and all that, despite needing to remember to always round after division calculations or calculations against fractional types. 3) Never do arithmetic? Tell the user to go home.
Jeremy Falcon
I work in engineering applications so my answer is "I don't use floating point arithmetic", everything is fixed point - many microcontrollers do not have an FPU at all and what I do is mostly real-time so forget software emulation. Any desktop tool has to provide or receive data from these systems so it's mostly fixed point until it's time to show to the user. On non engineering applications I just use floating points as they are, if I have qualms about precision I just use double or long double and call it a day - I don't work on servers or high throughput / low latency services so I just don't care.
GCS/GE d--(d) s-/+ a C+++ U+++ P-- L+@ E-- W+++ N+ o+ K- w+++ O? M-- V? PS+ PE Y+ PGP t+ 5? X R+++ tv-- b+(+++) DI+++ D++ G e++ h--- r+++ y+++* Weapons extension: ma- k++ F+2 X The shortest horror story: On Error Resume Next
