0
0

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?

More than 3 years have passed since last update.

Float型にて、計算した際に、値に少々ズレが生じる件について

Posted at

環境

Ruby 2.6
Rails 5.1.7

背景

機能実装中、数値を計算する処理が何か合わないなぁと思ったら表題の現象が起きてた。
これは初めて遭遇したので備忘録として。

現象

ruby
irb(main):005:0> 2.2 * 100
=> 220.00000000000003
irb(main):006:0> 2.3 * 100
=> 229.99999999999997
irb(main):007:0> 2.4 * 100
=> 240.0

Float型に100掛けると何故か100倍になっておらず少々の誤差があることに気づいた

対応

https://qiita.com/yusabana/items/fd4a0185c1f120403a74
こちらの記事を参考にBigDecimalを使用してあげると綺麗な数値が出る

ruby
irb(main):017:0> (BigDecimal(2.2.to_s) * 100).to_f
=> 220.0
irb(main):015:0> (BigDecimal(2.3.to_s) * 100).to_f
=> 230.0
irb(main):018:0> (BigDecimal(2.4.to_s) * 100).to_f
=> 240.0

以上

0
0
0

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
0
0

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?