本記事の目的と結論
railsのconnection pool管理について、
- 何個まで接続を保持できるのか → デフォルト5個、database.yamlから設定値を取得
- 別DBへの繋ぎ直し時のconnection poolの管理 → pool_managerを用いて、DB毎にhashで管理している
version
rails: 6.1.4
ruby: 3.0.2
背景
複数DBを用いたアーキテクチャをrailsで実装する。
DB間の接続切り替えを行った際のconnection poolの管理を調査し、オーバーヘッドがどの程度発生するのか確認したい
connection poolとは
DBの接続状態を確立したコネクションを保持しておき、再利用することでDB接続時間を短縮するもの
1. 何個まで接続を保持できるのか
公式ドキュメントによると、
pool: maximum number of connections the pool may manage (default 5).
ActiveRecord::ConnectionAdapters::ConnectionPoolクラスのpoolオプションで設定されている
poolオプションに値が入っているのは、以下の部分
activerecord/lib/active_record/connection_adapters/abstract/connection_pool.rb
def initialize(pool_config)
super()
@pool_config = pool_config
@db_config = pool_config.db_config
@connection_klass = pool_config.connection_klass
@checkout_timeout = db_config.checkout_timeout
@idle_timeout = db_config.idle_timeout
@size = db_config.pool # <- ここの値がpoolの値
activerecord/lib/active_record/connection_adapters/pool_config.rb
def pool
ActiveSupport::ForkTracker.check!
@pool || synchronize { @pool ||= ConnectionAdapters::ConnectionPool.new(self) } # <- ConnectionPoolが作られている
end
activerecord/lib/active_record/connection_adapters/abstract/connection_handler.rb
def resolve_pool_config(config, owner_name)
db_config = Base.configurations.resolve(config) # <- db_configにpool情報が入っている
raise(AdapterNotSpecified, "database configuration does not specify adapter") unless db_config.adapter
# Require the adapter itself and give useful feedback about
# 1. Missing adapter gems and
# 2. Adapter gems' missing dependencies.
path_to_adapter = "active_record/connection_adapters/#{db_config.adapter}_adapter"
begin
require path_to_adapter
rescue LoadError => e
# We couldn't require the adapter itself. Raise an exception that
# points out config typos and missing gems.
if e.path == path_to_adapter
# We can assume that a non-builtin adapter was specified, so it's
# either misspelled or missing from Gemfile.
raise LoadError, "Could not load the '#{db_config.adapter}' Active Record adapter. Ensure that the adapter is spelled correctly in config/database.yml and that you've added the necessary adapter gem to your Gemfile.", e.backtrace
# Bubbled up from the adapter require. Prefix the exception message
# with some guidance about how to address it and reraise.
else
raise LoadError, "Error loading the '#{db_config.adapter}' Active Record adapter. Missing a gem it depends on? #{e.message}", e.backtrace
end
end
unless ActiveRecord::Base.respond_to?(db_config.adapter_method)
raise AdapterNotFound, "database configuration specifies nonexistent #{db_config.adapter} adapter"
end
ConnectionAdapters::PoolConfig.new(owner_name, db_config) # <- PoolConfigが作られている
end
db_configの情報はdatabase.yamlから取得して入っている
2. 別DBへの繋ぎ直し時のconnection poolの管理
DB間の繋ぎ直しはestablish_connectionを用いる
この時、poolされているconnection情報がどのように参照されているかを見ていく
activerecord/lib/active_record/connection_adapters/abstract/connection_handler.rb
def establish_connection(config, owner_name: Base, role: ActiveRecord::Base.current_role, shard: Base.current_shard)
owner_name = StringConnectionOwner.new(config.to_s) if config.is_a?(Symbol)
pool_config = resolve_pool_config(config, owner_name) # <- pool_configを前述の関数から取得している
db_config = pool_config.db_config
# Protects the connection named `ActiveRecord::Base` from being removed
# if the user calls `establish_connection :primary`.
if owner_to_pool_manager.key?(pool_config.connection_specification_name)
remove_connection_pool(pool_config.connection_specification_name, role: role, shard: shard)
end
message_bus = ActiveSupport::Notifications.instrumenter
payload = {}
if pool_config
payload[:spec_name] = pool_config.connection_specification_name
payload[:shard] = shard
payload[:config] = db_config.configuration_hash
end
if ActiveRecord.legacy_connection_handling
owner_to_pool_manager[pool_config.connection_specification_name] ||= LegacyPoolManager.new
else
# <- owner_to_pool_managerがDB毎のpool managerを管理している
owner_to_pool_manager[pool_config.connection_specification_name] ||= PoolManager.new
end
pool_manager = get_pool_manager(pool_config.connection_specification_name)
pool_manager.set_pool_config(role, shard, pool_config) # <- role, shard, pool_configを受け取る
message_bus.instrument("!connection.active_record", payload) do
pool_config.pool
end
end
activerecord/lib/active_record/connection_adapters/pool_manager.rb
def set_pool_config(role, shard, pool_config)
if pool_config
@name_to_role_mapping[role][shard] = pool_config # <- role * shardでpool_configをhashに詰めている
else
raise ArgumentError, "The `pool_config` for the :#{role} role and :#{shard} shard was `nil`. Please check your configuration. If you want your writing role to be something other than `:writing` set `config.active_record.writing_role` in your application configuration. The same setting should be applied for the `reading_role` if applicable."
end
end
@name_to_role_mappingが最終的にpool_configの情報を管理していることがわかる
message_bus.instrumentで送られたpool情報は以下のようにsubscriberで呼ばれ、接続されていく
activerecord/lib/active_record/test_fixtures.rb
# When connections are established in the future, begin a transaction too
@connection_subscriber = ActiveSupport::Notifications.subscribe("!connection.active_record") do |_, _, _, _, payload|
spec_name = payload[:spec_name] if payload.key?(:spec_name)
shard = payload[:shard] if payload.key?(:shard)
setup_shared_connection_pool
if spec_name
begin
connection = ActiveRecord::Base.connection_handler.retrieve_connection(spec_name, shard: shard)
rescue ConnectionNotEstablished
connection = nil
end
if connection && !@fixture_connections.include?(connection)
connection.begin_transaction joinable: false, _lazy: false
connection.pool.lock_thread = true if lock_threads
@fixture_connections << connection
end
end
end
このように、railsでは複数DBを用いていてもpool_managerが、roleとshardをkeyにしたhashでpool情報を管理していることがわかった
参考URL
https://hackerslab.aktsk.jp/technology/rails4_connection_pooling/
https://qiita.com/katsuyuki/items/42b3c69bcd76c44ad64a