Gu stav wrote:
I ended up taking the chicken way out, converting my floats to strings
in new columns and adapting my find conditions to look for those
instead. Works like charm. Thanks anyways, I wonder why it didnt work.
As indicated by Frederick that's just the nature of a float data type.
It doesn't matter whether it's Ruby, SQL, Java or C. It is never safe to
compare equality on floating point values due to the inherent nature of
how they are stored.
Example: You may be seeing 11.967 but that value is going to get
converted to an IEEE floating point hexadecimal value. Depending on the
loss of precision it is quite possible to have 11.967 == 11.967 =>
false. Depending on how the two values get translated to and from the
IEEE floating point hexadecimal values. 11.967 might actually be stored
in the hex equivalent of 11.9669999999999.
There are a few techniques for dealing with this problem:
x = 11.967
y = 11.966999999999 <-- loss of precision due to hex<->decimal
1. Compare based on a range (as suggested in a prior post)
11.9670 <= x < 11.968.
2. Compare string representations of the floating point values:
x == y
("%.3f" % x) == ("%.3f" % y)
3. Store values using a fixed point data type (such as DECIMAL in
x -> lng
y -> lng
=> 11.967 <-- rounded based on the rules defined in the database
Making it safe to compare equality on the two values
4. Store fixed decimal values as integers in the database and convert
them when displaying them. I only mention this technique because it is
an option for storing currency values. Rather than store dollars, store
cents instead and convert to dollars with displaying the values on the
$1.60 -> 160 (in the database)