他の日本語で説明されている方法ではGemのuninstallが出来なかったので
英語サイトを参考にやってみたら出来ました。
for i in `gem list --no-versions`; do gem uninstall -aIx $i; done
を実行
Successfully uninstalled actionmailer-3.2.16
Successfully uninstalled actionpack-3.2.16
Successfully uninstalled active_scaffold-3.3.3
Successfully uninstalled active_scaffold-3.3.0
Successfully uninstalled activemodel-3.2.16
Successfully uninstalled activerecord-3.2.16
Successfully uninstalled activeresource-3.2.16
Successfully uninstalled activesupport-3.2.16
Successfully uninstalled addressable-2.3.6
Successfully uninstalled addressable-2.3.5
Successfully uninstalled arel-3.0.3
Successfully uninstalled asset_sync-1.0.0
Successfully uninstalled awesome_print-1.2.0
Removing aws-rb
Successfully uninstalled aws-sdk-1.40.3
Successfully uninstalled bcrypt-3.1.7
Successfully uninstalled better_errors-1.1.0
Successfully uninstalled bigdecimal-1.2.5
Successfully uninstalled binding_of_caller-0.7.2
Successfully uninstalled builder-3.0.4