sql data types
-
i m working on the stored procedures in SQL that computes the division of two columns having smallint as their datatype,for each row , the results are stored in a column of another table the datatype of the result is taken as float but it is correct upto 1 decimal place How do i increase the precision of float data type that gives the correct result "
-
i m working on the stored procedures in SQL that computes the division of two columns having smallint as their datatype,for each row , the results are stored in a column of another table the datatype of the result is taken as float but it is correct upto 1 decimal place How do i increase the precision of float data type that gives the correct result "
You could cast the two smallints to floating point numbers before doing te calculation. You can also cast the result to any precision you would like as well.
A man said to the universe: "Sir I exist!" "However," replied the Universe, "The fact has not created in me A sense of obligation." -- Stephen Crane
-
You could cast the two smallints to floating point numbers before doing te calculation. You can also cast the result to any precision you would like as well.
A man said to the universe: "Sir I exist!" "However," replied the Universe, "The fact has not created in me A sense of obligation." -- Stephen Crane
7/1612 = 0.004342431761............. 6/1397 = 0.0042949176807................. above examples shows the division of 2 smallints giving a floating pt. result. but in our case I m getting the result column as 0.0 only for all values. y is this happening, y isn't it considering the whole result or atleast it should give 3 to 4 values after decimal.
-
7/1612 = 0.004342431761............. 6/1397 = 0.0042949176807................. above examples shows the division of 2 smallints giving a floating pt. result. but in our case I m getting the result column as 0.0 only for all values. y is this happening, y isn't it considering the whole result or atleast it should give 3 to 4 values after decimal.
Hi saneng In my tests i'm getting the full results which are: 7/1612 =
4.3424317617866E-036/1397 =4.29491768074445E-03I do the same. Values (of type smallint) are stored in one table, and division results (of type float) are stored in another table. Moreover with all my search I didn't find that how to change/set the precision + scale of float type. Its default precision is 53 which cannot be changed.saneng wrote:
7/1612 = 0.004342431761............. 6/1397 = 0.0042949176807................. above examples shows the division of 2 smallints giving a floating pt. result. but in our case I m getting the result column as 0.0 only for all values. y is this happening, y isn't it considering the whole result or atleast it should give 3 to 4 values after decimal.
I guess you are using some other type like numeric or decimal. If this is so then check and set its precision + scale value. Wish you all the best.
_____________________________ Success is not something to wait for, its something to work for.