Wordpressのデータベース接続確率エラー

初めに

この手のエラーに関する記事はよくあるかもしれませんが、同じパターンのエラーが起きた時の参考になればと思い投稿します。
ほとんどの場合、以下対処法➀でどうにかなるみたいですが僕の場合はちょっとちがったので・・・

環境

• amazon linux2
• wordpress 5.6.2
• mariaDB 10.2.10

問題点

AWS上にwordpressの環境を構築したのですが、データベース接続エラーにより管理画面にアクセスできなくなりました。

確認すると以下のようにDBが停止してしまっている・・・

 systemctl status mariadb.service
● mariadb.service - MariaDB 10.2 database server
   Loaded: loaded (/usr/lib/systemd/system/mariadb.service; enabled; vendor preset: disabled)
   Active: failed (Result: exit-code) since 土 2021-03-20 16:45:32 JST; 2h 37min ago
  Process: 20717 ExecStart=/usr/libexec/mysqld --basedir=/usr $MYSQLD_OPTS $_WSREP_NEW_CLUSTER 

...省略

対処法➀

log内容の確認

#less /var/log/mariadb/mariadb.log

2021-03-19 17:00:56 140449615568704 [Note] RocksDB: 2 column families found
2021-03-19 17:00:56 140449615568704 [Note] RocksDB: Column Families at start:
2021-03-19 17:00:56 140449615568704 [Note]   cf=default
2021-03-19 17:00:56 140449615568704 [Note]     write_buffer_size=67108864
2021-03-19 17:00:56 140449615568704 [Note]     target_file_size_base=67108864
2021-03-19 17:00:56 140449615568704 [Note]   cf=__system__
2021-03-19 17:00:56 140449615568704 [Note]     write_buffer_size=67108864
2021-03-19 17:00:56 140449615568704 [Note]     target_file_size_base=67108864
2021-03-19 17:00:56 140449615568704 [Note] RocksDB: Table_store: loaded DDL data for 0 tables
2021-03-19 17:00:56 140449615568704 [Note] RocksDB: global statistics using get_sched_indexer_t indexer
2021-03-19 17:00:56 140449615568704 [Note] MyRocks storage engine plugin has been successfully initialized.
Fri Mar 19 17:00:56 2021 PerconaFT recovery starting in env /var/lib/mysql/
Fri Mar 19 17:00:56 2021 PerconaFT recovery scanning backward from 121384
Fri Mar 19 17:00:56 2021 PerconaFT recovery bw_end_checkpoint at 121384 timestamp 1616140842975886 xid 121380 (bw_newer)
Fri Mar 19 17:00:56 2021 PerconaFT recovery bw_begin_checkpoint at 121380 timestamp 1616140842975228 (bw_between)
Fri Mar 19 17:00:56 2021 PerconaFT recovery turning around at begin checkpoint 121380 time 658
Fri Mar 19 17:00:56 2021 PerconaFT recovery starts scanning forward to 121384 from 121380 left 4 (fw_between)
Fri Mar 19 17:00:56 2021 PerconaFT recovery closing 2 dictionaries
Fri Mar 19 17:00:56 2021 PerconaFT recovery making a checkpoint
Fri Mar 19 17:00:56 2021 PerconaFT recovery done
2021-03-19 17:00:56 140449615568704 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2021-03-19 17:00:56 140449615568704 [Note] InnoDB: Uses event mutexes
2021-03-19 17:00:56 140449615568704 [Note] InnoDB: Compressed tables use zlib 1.2.7
2021-03-19 17:00:56 140449615568704 [Note] InnoDB: Using Linux native AIO
2021-03-19 17:00:56 140449615568704 [Note] InnoDB: Number of pools: 1
2021-03-19 17:00:56 140449615568704 [Note] InnoDB: Using SSE2 crc32 instructions
2021-03-19 17:00:56 140449615568704 [Note] InnoDB: Initializing buffer pool, total size = 128M, instances = 1, chunk size = 128M
2021-03-19 17:00:56 140449615568704 [ERROR] InnoDB: mmap(137560064 bytes) failed; errno 12
2021-03-19 17:00:56 140449615568704 [ERROR] InnoDB: Cannot allocate memory for the buffer pool
2021-03-19 17:00:56 140449615568704 [ERROR] InnoDB: Plugin initialization aborted with error Generic error
2021-03-19 17:00:56 140449615568704 [Note] InnoDB: Starting shutdown...

[ERROR] InnoDB: Cannot allocate memory for the buffer pool

メモリが足りてないといわれていたので以下参考にswap領域を作成し有効化しました
https://tech.librastudio.co.jp/entry/index.php/2018/05/22/post-1891/

   #free
           total        used        free      shared  buff/cache   available
Mem:        1006900      424820      400324         360      181756      443208
Swap:       1006896        1792     1005104
コード

そしていざmariadbを再起動

#sudo service mariadb start

Redirecting to /bin/systemctl start mariadb.service
Job for mariadb.service failed because the control process exited with error code. See "systemctl status mariadb.service" and "journalctl -xe" for details.

起動しない・・・
他の原因がないかもう一度log確認

対処法➁

2021-03-19 17:41:56 139824757690176 [Note] RocksDB: 2 column families found
2021-03-19 17:41:56 139824757690176 [Note] RocksDB: Column Families at start:
2021-03-19 17:41:56 139824757690176 [Note]   cf=default
2021-03-19 17:41:56 139824757690176 [Note]     write_buffer_size=67108864
2021-03-19 17:41:56 139824757690176 [Note]     target_file_size_base=67108864
2021-03-19 17:41:56 139824757690176 [Note]   cf=__system__
2021-03-19 17:41:56 139824757690176 [Note]     write_buffer_size=67108864
2021-03-19 17:41:56 139824757690176 [Note]     target_file_size_base=67108864
2021-03-19 17:41:56 139824757690176 [ERROR] RocksDB: Error opening instance, Status Code: 5, Status: IO error: While lock file: ./.rocksdb/LOCK: Resource temporarily unavailable
2021-03-19 17:41:56 139824757690176 [ERROR] Plugin 'ROCKSDB' init function returned error.
2021-03-19 17:41:56 139824757690176 [ERROR] Plugin 'ROCKSDB' registration as a STORAGE ENGINE failed.
Couldn't start tokuft because some other tokuft process is using the same directory [/var/lib/mysql/] for [environment]
2021-03-19 17:41:56 139824757690176 [ERROR] TokuDB unknown error 11
2021-03-19 17:41:56 139824757690176 [ERROR] Plugin 'TokuDB' init function returned error.
2021-03-19 17:41:56 139824757690176 [ERROR] Plugin 'TokuDB' registration as a STORAGE ENGINE failed.
2021-03-19 17:41:56 139824757690176 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2021-03-19 17:41:56 139824757690176 [Note] InnoDB: Uses event mutexes
2021-03-19 17:41:56 139824757690176 [Note] InnoDB: Compressed tables use zlib 1.2.7
2021-03-19 17:41:56 139824757690176 [Note] InnoDB: Using Linux native AIO
2021-03-19 17:41:56 139824757690176 [Note] InnoDB: Number of pools: 1
2021-03-19 17:41:56 139824757690176 [Note] InnoDB: Using SSE2 crc32 instructions
2021-03-19 17:41:56 139824757690176 [Note] InnoDB: Initializing buffer pool, total size = 128M, instances = 1, chunk size = 128M
2021-03-19 17:41:56 139824757690176 [Note] InnoDB: Completed initialization of buffer pool
2021-03-19 17:41:56 139823908087552 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
2021-03-19 17:41:56 139824757690176 [Note] InnoDB: Highest supported file format is Barracuda.
2021-03-19 17:41:56 139824757690176 [Note] InnoDB: 128 out of 128 rollback segments are active.
2021-03-19 17:41:56 139824757690176 [Note] InnoDB: Creating shared tablespace for temporary tables
2021-03-19 17:41:56 139824757690176 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
2021-03-19 17:41:56 139824757690176 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
2021-03-19 17:41:56 139824757690176 [Note] InnoDB: Waiting for purge to start
2021-03-19 17:41:56 139824757690176 [Note] InnoDB: 5.7.20 started; log sequence number 115031297
2021-03-19 17:41:56 139823897597696 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
2021-03-19 17:41:56 139823897597696 [Note] InnoDB: Buffer pool(s) load completed at 210319 17:41:56
2021-03-19 17:41:56 139824757690176 [Note] Plugin 'FEEDBACK' is disabled.
2021-03-19 17:41:56 139824757690176 [Warning] mysqld: GSSAPI plugin : default principal 'mariadb/ip-172-31-28-115.ap-northeast-1.compute.internal@' not found in keytab
2021-03-19 17:41:56 139824757690176 [ERROR] mysqld: Server GSSAPI error (major 851968, minor 2529639093) : gss_acquire_cred failed -Unspecified GSS failure.  Minor code may provide more information. Keytab FILE:/etc/krb5.keytab is nonexistent or empty. 
2021-03-19 17:41:56 139824757690176 [ERROR] Plugin 'gssapi' init function returned error.
2021-03-19 17:41:56 139824757690176 [Note] Recovering after a crash using tc.log
2021-03-19 17:41:56 139824757690176 [ERROR] Recovery failed! You must enable all engines that were enabled at the moment of the crash
2021-03-19 17:41:56 139824757690176 [ERROR] Crash recovery failed. Either correct the problem (if it's, for example, out of memory error) and restart, or delete tc log and start mysqld with --tc-heuristic-recover={commit|rollback}
2021-03-19 17:41:56 139824757690176 [ERROR] Can't init tc log
2021-03-19 17:41:56 139824757690176 [ERROR] Aborting

いくつかエラー出てますがここでは下から二番目に注目

[ERROR] Can't init tc log

上記のようにtc.logが初期化されていないといったエラー
このエラーの内容はtc.logに記載されているログが見つからないといったエラーのようです。

実際に/var/lib/mysql/tc.logのサイズを確認すると空でした。

# rm -f /var/lib/mysql/tc.log

# systemctl status mariadb.service
● mariadb.service - MariaDB 10.2 database server
   Loaded: loaded (/usr/lib/systemd/system/mariadb.service; enabled; vendor preset: disabled)
 　Active: active (running) since 土 2021-03-20 19:23:05 JST; 4s ago
・・・省略

/var/lib/mysql/tc.logを削除することでmysqlへの接続ができるようになりました。

wordpressにブラウザでアクセスするとエラーが解消されログインできるようになっていました。