rails float type number does not work right

rubyonrails-talk
1

it “should get right settlement percent” do contract = Contract.new contract.settlement_percent = 1.1 / 100.0 contract.settlement_percent.to_f.should == 0.011 contract.settlement_percent.to_s.should == “0.011” end

but test result is :

Failure/Error: contract.settlement_percent.to_f.should == 0.011 expected: 0.011, got: 0.011000000000000001 (using ==) # ./spec/models/contract_spec.rb:16:in `block (3 levels) in <top (required)>’

2

Quoting 蕲春人 <whyruby@gmail.com>:

    it "should get right settlement percent" do       contract = Contract.new       contract.settlement_percent = 1.1 / 100.0       contract.settlement_percent.to_f.should == 0.011       contract.settlement_percent.to_s.should == "0.011"     end

but test result is :

Failure/Error: contract.settlement_percent.to_f.should == 0.011      expected: 0.011,           got: 0.011000000000000001 (using ==)      # ./spec/models/contract_spec.rb:16:in `block (3 levels) in <top (required)>'

Your model of floating point is incorrect. Exact equality with floating point is rarely a good idea. Most modern computers use the IEEE floating point format and arithmetic is done in base 2. There are many base 10 fractions that do not have exact base 2 representations. And once you start doing calculations, they diverge even further.

There are several ways around this. The simplest is:

contract.settlement_percent.round_with_precision(5).should == 0.011 ('%0.3f' % contract.settlement_percent).should == '0.011'

If you have legal requirements (e.g. SEC regulations) on how arithmetic works, you had better write your own class. Depending on floating point on arbitrary architectures to behave in a certain way is inviting trouble.

Jeffrey

3

Jeffrey L. Taylor wrote in post #966404:

Quoting 蕲春人 <whyruby@gmail.com>:

     expected: 0.011,           got: 0.011000000000000001 (using ==)      # ./spec/models/contract_spec.rb:16:in `block (3 levels) in <top (required)>'

Your model of floating point is incorrect. Exact equality with floating point is rarely a good idea. Most modern computers use the IEEE floating point format and arithmetic is done in base 2. There are many base 10 fractions that do not have exact base 2 representations. And once you start doing calculations, they diverge even further.

There are several ways around this. The simplest is:

contract.settlement_percent.round_with_precision(5).should == 0.011 ('%0.3f' % contract.settlement_percent).should == '0.011'

Actually, the simplest thing to do is to just avoid Float for math altogether. Use BigDecimal and Rational instead.

Best,