strange behavior of while loop

rubyonrails-talk
1

hai all,

   I am very new to Ruby and when I am doing some small program I found the strange output for my program. Please let me know why this is happening....

num = 10.1

while num < 10.5         puts "num = " + num.to_s         num = num + 0.1 end

output that I am getting is :

num = 10.1 num = 10.2 num = 10.3 num = 10.4 num = 10.5

2

You could also use BigDecimal instead of float:

num = BigDecimal("10.1") while num < BigDecimal("10.5") do   puts "num = #{num.to_s}"   num += BigDecimal("0.1") end

3

Float rounding errors

>> num = 10.1 => 10.1 >> num += 0.1 => 10.2 >> num += 0.1 => 10.3 >> num += 0.1 => 10.4 >> num += 0.1 => 10.5 >> num < 10.5 => true >> num => 10.5

Julian.

4

This is even more fun :wink:

>> (10.1 * 1000).to_i => 10100 >> ((10.1 + 0.1) * 1000).to_i => 10200 >> ((10.1 + 0.1 + 0.1) * 1000).to_i => 10299

Julian.

5

The trouble with rounding floating point numbers • The Register Floating-point arithmetic - Wikipedia

http://docs.sun.com/source/806-3568/ncg_goldberg.html is one of the
classics on this subject.

Fred