Maybe a Ruby question: Strange math!

rubyonrails-talk
1

irb(main):004:0> (19.9 * 100).to_i => 1989 irb(main):005:0> (19.9 * 100) => 1990.0 irb(main):006:0> 1990.0.to_i => 1990

Can someone explain this? Regards Till Vollmer

2

Floating point number aren’t completely accurate.

http://en.wikipedia.org/wiki/Floating_point#Accuracy_problems

3

crazy stuff indeed.

I understand (vaguely) the issue with FP numbers not being accurate, but I can't wrap my head around this.

I can't seem to find any real discussion of ways to deal with this. Is the only workable solution to this issue to work exclusively with Integers?

Eric

4

etlund wrote:

crazy stuff indeed.

I understand (vaguely) the issue with FP numbers not being accurate, but I can't wrap my head around this.

I can't seem to find any real discussion of ways to deal with this. Is the only workable solution to this issue to work exclusively with Integers?

Eric

Floating point numbers can only approximate some integers. In your case, the final product of 19.9 * 100 is probably something like

1989.99999999999999999999998, which just displays as 1990.0 in most cases.

Now, .to_i truncates off the decimal part pretty ruthlessly.

To be safe, you should probably use a function like .round instead of .to_i unless you are absolutely certain you can do that safely.

.round will round the number to the nearest integer, which is really what you want.

_Kevin

5

Yup. It's standard to deal with integers. If you look at the banking system then everything there is in whole numbers. On the computer level $55 is always represented as 5500. However that method may not work for your needs.

BigDecimal seems to do a bunch. Never used it thought.

001:0> require 'bigdecimal' 002:0> require 'bigdecimal/util' 003:0> (19.9.to_d * 100).to_i => 1990

http://www.ruby-doc.org/stdlib/libdoc/bigdecimal/rdoc/index.html

6

irb(main):004:0> (19.9 * 100).to_i => 1989 irb(main):005:0> (19.9 * 100) => 1990.0 irb(main):006:0> 1990.0.to_i => 1990

Can someone explain this?

to_i truncates, so

irb(main):001:0> 99.9.to_i => 99

You are probably expecting rounding rather than truncation:

irb(main):002:0> 99.9.round => 100

Cheers, Bob