MongoDBが起動しなくなってしまった．．．

ある日，MongoDBを起動しようとしたら以下のようなエラーが・・・

$ mongo
MongoDB shell version v4.2.10
connecting to: mongodb://127.0.0.1:27017/?compressors=disabled&gssapiServiceName=mongodb
2020-10-23T16:57:38.500+0000 E  QUERY    [js] Error: couldn't connect to server 127.0.0.1:27017, connection attempt failed: SocketException: Error connecting to 127.0.0.1:27017 :: caused by :: Connection refused :
connect@src/mongo/shell/mongo.js:353:17
@(connect):2:6
2020-10-23T16:57:38.501+0000 F  -        [main] exception: connect failed
2020-10-23T16:57:38.501+0000 E  -        [main] exiting with code 1

どうやら，Mongoサーバと接続できません！というエラーらしいです

変な事はしていないので，ひとまず同じ内容で困っている人がいないかGoogle先生にお尋ねしていたところ，同じような人がゾロゾロいることが発覚
ただ，記事のほとんどはbrew servicesコマンドを使っており，Ubuntuを使っている私にとっては苦しい現実でした．

（UbuntuにもLinuxBrewというのがありますが，実はbrew servicesはまだサポートされていないようですorz）

そこで海外サイトに頼ろうとまたGoogle先生に聞いたところ，解決策が載っているサイトを発見．
今回はそこを参考に記事を書いていきたいと思います

参考にさせて頂いたサイト様

ひとまず色々とやってみた(できなかった)

まずmongod --repairというコマンドがあるらしくそれを実行してみる

$ sudo mongod --repair
2020-10-23T17:24:42.552+0000 I  CONTROL  [main] Automatically disabling TLS 1.0, to force-enable TLS 1.0 specify --sslDisabledProtocols 'none'
2020-10-23T17:24:42.553+0000 W  ASIO     [main] No TransportLayer configured during NetworkInterface startup
2020-10-23T17:24:42.554+0000 I  CONTROL  [initandlisten] MongoDB starting : pid=15466 port=27017 dbpath=/data/db 64-bit host=sugimoto-1
2020-10-23T17:24:42.554+0000 I  CONTROL  [initandlisten] db version v4.2.10
2020-10-23T17:24:42.554+0000 I  CONTROL  [initandlisten] git version: 88276238fa97b47c0ef14362b343c5317ecbd739
2020-10-23T17:24:42.554+0000 I  CONTROL  [initandlisten] OpenSSL version: OpenSSL 1.1.1  11 Sep 2018
2020-10-23T17:24:42.554+0000 I  CONTROL  [initandlisten] allocator: tcmalloc
2020-10-23T17:24:42.554+0000 I  CONTROL  [initandlisten] modules: none
2020-10-23T17:24:42.554+0000 I  CONTROL  [initandlisten] build environment:
2020-10-23T17:24:42.554+0000 I  CONTROL  [initandlisten]     distmod: ubuntu1804
2020-10-23T17:24:42.554+0000 I  CONTROL  [initandlisten]     distarch: x86_64
2020-10-23T17:24:42.554+0000 I  CONTROL  [initandlisten]     target_arch: x86_64
2020-10-23T17:24:42.554+0000 I  CONTROL  [initandlisten] options: { repair: true }
2020-10-23T17:24:42.554+0000 I  STORAGE  [initandlisten] exception in initAndListen: NonExistentPath: Data directory /data/db not found. Create the missing directory or specify another path using (1) the --dbpath command line option, or (2) by adding the 'storage.dbPath' option in the configuration file., terminating
2020-10-23T17:24:42.554+0000 I  REPL     [initandlisten] Stepping down the ReplicationCoordinator for shutdown, waitTime: 10000ms
2020-10-23T17:24:42.554+0000 I  SHARDING [initandlisten] Shutting down the WaitForMajorityService
2020-10-23T17:24:42.554+0000 I  NETWORK  [initandlisten] Shutting down the global connection pool
2020-10-23T17:24:42.554+0000 I  STORAGE  [initandlisten] Shutting down the FlowControlTicketholder
2020-10-23T17:24:42.554+0000 I  -        [initandlisten] Stopping further Flow Control ticket acquisitions.
2020-10-23T17:24:42.554+0000 I  INDEX    [initandlisten] Shutting down the IndexBuildsCoordinator
2020-10-23T17:24:42.554+0000 I  NETWORK  [initandlisten] Shutting down the ReplicaSetMonitor
2020-10-23T17:24:42.554+0000 I  CONTROL  [initandlisten] Shutting down free monitoring
2020-10-23T17:24:42.554+0000 I  FTDC     [initandlisten] Shutting down full-time data capture
2020-10-23T17:24:42.554+0000 I  STORAGE  [initandlisten] Shutting down the HealthLog
2020-10-23T17:24:42.554+0000 I  -        [initandlisten] Dropping the scope cache for shutdown
2020-10-23T17:24:42.554+0000 I  CONTROL  [initandlisten] now exiting
2020-10-23T17:24:42.554+0000 I  CONTROL  [initandlisten] shutting down with code:100

特にエラーは発生せず．
いけるかと思い，とりあえず再起動と再度起動を．．．

$ sudo service mongod restart
$ mongo
MongoDB shell version v4.2.10
connecting to: mongodb://127.0.0.1:27017/?compressors=disabled&gssapiServiceName=mongodb
2020-10-23T17:27:53.860+0000 E  QUERY    [js] Error: couldn't connect to server 127.0.0.1:27017, connection attempt failed: SocketException: Error connecting to 127.0.0.1:27017 :: caused by :: Connection refused :
connect@src/mongo/shell/mongo.js:353:17
@(connect):2:6
2020-10-23T17:27:53.861+0000 F  -        [main] exception: connect failed
2020-10-23T17:27:53.861+0000 E  -        [main] exiting with code 1

んー
ログファイル(/var/log/mondodb/mongod.log)を見てみよう・・・

$ sudo service mongod start
$ sudo cat /var/log/mongodb/mongod.log
2020-10-23T17:30:25.047+0000 I  CONTROL  [main] ***** SERVER RESTARTED *****
2020-10-23T17:30:25.047+0000 I  CONTROL  [main] Automatically disabling TLS 1.0, to force-enable TLS 1.0 specify --sslDisabledProtocols 'none'
2020-10-23T17:30:25.053+0000 W  ASIO     [main] No TransportLayer configured during NetworkInterface startup
2020-10-23T17:30:25.053+0000 I  CONTROL  [initandlisten] MongoDB starting : pid=15612 port=27017 dbpath=/var/lib/mongodb 64-bit host=sugimoto-1
2020-10-23T17:30:25.053+0000 I  CONTROL  [initandlisten] db version v4.2.10
2020-10-23T17:30:25.053+0000 I  CONTROL  [initandlisten] git version: 88276238fa97b47c0ef14362b343c5317ecbd739
2020-10-23T17:30:25.053+0000 I  CONTROL  [initandlisten] OpenSSL version: OpenSSL 1.1.1  11 Sep 2018
2020-10-23T17:30:25.053+0000 I  CONTROL  [initandlisten] allocator: tcmalloc
2020-10-23T17:30:25.053+0000 I  CONTROL  [initandlisten] modules: none
2020-10-23T17:30:25.053+0000 I  CONTROL  [initandlisten] build environment:
2020-10-23T17:30:25.053+0000 I  CONTROL  [initandlisten]     distmod: ubuntu1804
2020-10-23T17:30:25.053+0000 I  CONTROL  [initandlisten]     distarch: x86_64
2020-10-23T17:30:25.053+0000 I  CONTROL  [initandlisten]     target_arch: x86_64
2020-10-23T17:30:25.053+0000 I  CONTROL  [initandlisten] options: { config: "/etc/mongod.conf", net: { bindIp: "127.0.0.1", port: 27017 }, processManagement: { timeZoneInfo: "/usr/share/zoneinfo" }, storage: { dbPath: "/var/lib/mongodb", journal: { enabled: true } }, systemLog: { destination: "file", logAppend: true, path: "/var/log/mongodb/mongod.log" } }
2020-10-23T17:30:25.054+0000 I  STORAGE  [initandlisten] exception in initAndListen: IllegalOperation: Attempted to create a lock file on a read-only directory: /var/lib/mongodb, terminating
2020-10-23T17:30:25.054+0000 I  REPL     [initandlisten] Stepping down the ReplicationCoordinator for shutdown, waitTime: 10000ms
2020-10-23T17:30:25.054+0000 I  SHARDING [initandlisten] Shutting down the WaitForMajorityService
2020-10-23T17:30:25.054+0000 I  NETWORK  [initandlisten] shutdown: going to close listening sockets...
2020-10-23T17:30:25.054+0000 I  NETWORK  [initandlisten] Shutting down the global connection pool
2020-10-23T17:30:25.054+0000 I  STORAGE  [initandlisten] Shutting down the FlowControlTicketholder
2020-10-23T17:30:25.054+0000 I  -        [initandlisten] Stopping further Flow Control ticket acquisitions.
2020-10-23T17:30:25.054+0000 I  INDEX    [initandlisten] Shutting down the IndexBuildsCoordinator
2020-10-23T17:30:25.054+0000 I  NETWORK  [initandlisten] Shutting down the ReplicaSetMonitor
2020-10-23T17:30:25.054+0000 I  CONTROL  [initandlisten] Shutting down free monitoring
2020-10-23T17:30:25.054+0000 I  FTDC     [initandlisten] Shutting down full-time data capture
2020-10-23T17:30:25.054+0000 I  STORAGE  [initandlisten] Shutting down the HealthLog
2020-10-23T17:30:25.054+0000 I  -        [initandlisten] Dropping the scope cache for shutdown
2020-10-23T17:30:25.054+0000 I  CONTROL  [initandlisten] now exiting
2020-10-23T17:30:25.054+0000 I  CONTROL  [initandlisten] shutting down with code:100

どうやらさっきのRestartコマンドは効いてるらしい．．．けどstartしておらず．．．

どうしたもんかと思いつつ，他にもいくつかのサイトに載っている方法を試したもののうまくいかず．．．

そこで参考にしたサイトを発見しました．

対処方法(できた)

ではその対処方法について説明します．

まずこのエラーが発生する場合，mongod.lockというファイルが生成されている可能性が高いとのこと．
これはMongoDBを強制終了等をした場合に発生するらしい．

まずこのファイルを削除する

$ sudo rm /var/lib/mongodb/mongod.lock

次に，DBPathを確認します．
確認方法は/etc/mongod.confを参照

$ sudo cat /etc/mongod.conf 
# mongod.conf

# for documentation of all options, see:
#   http://docs.mongodb.org/manual/reference/configuration-options/

# Where and how to store data.
storage:
  dbPath: /var/lib/mongodb
  journal:
    enabled: true
#  engine:
#  mmapv1:
#  wiredTiger:

# where to write logging data.
systemLog:
  destination: file
  logAppend: true
  path: /var/log/mongodb/mongod.log

# network interfaces
net:
  port: 27017
  bindIp: 127.0.0.1


# how the process runs
processManagement:
  timeZoneInfo: /usr/share/zoneinfo

#security:

#operationProfiling:

#replication:

#sharding:

## Enterprise-Only Options:

#auditLog:

#snmp:

私の場合はdbpath = /var/lib/mongodbであることがわかります
(他の場合だと/srv/mongodb、/opt/mongodbの可能性も有り)

そして，dbpathを指定した上で--repairと，--journalを実行する．
(dbpathのデフォルトは/data/dbらしい？)

$ sudo mongod --dbpath /var/lib/mongodb/ --repair
$ sudo mongod --dbpath /var/lib/mongodb/ --journal

上記コマンドを実行すると，

$ sudo mongod --dbpath /var/lib/mongodb/ --journal
2020-10-23T17:50:41.718+0000 I  CONTROL  [main] Automatically disabling TLS 1.0, to force-enable TLS 1.0 specify --sslDisabledProtocols 'none'
2020-10-23T17:50:41.720+0000 W  ASIO     [main] No TransportLayer configured during NetworkInterface startup
2020-10-23T17:50:41.720+0000 I  CONTROL  [initandlisten] MongoDB starting : pid=15886 port=27017 dbpath=/var/lib/mongodb/ 64-bit host=sugimoto-1
2020-10-23T17:50:41.720+0000 I  CONTROL  [initandlisten] db version v4.2.10
2020-10-23T17:50:41.720+0000 I  CONTROL  [initandlisten] git version: 88276238fa97b47c0ef14362b343c5317ecbd739
2020-10-23T17:50:41.720+0000 I  CONTROL  [initandlisten] OpenSSL version: OpenSSL 1.1.1  11 Sep 2018
2020-10-23T17:50:41.720+0000 I  CONTROL  [initandlisten] allocator: tcmalloc
2020-10-23T17:50:41.720+0000 I  CONTROL  [initandlisten] modules: none
2020-10-23T17:50:41.720+0000 I  CONTROL  [initandlisten] build environment:
2020-10-23T17:50:41.720+0000 I  CONTROL  [initandlisten]     distmod: ubuntu1804
2020-10-23T17:50:41.720+0000 I  CONTROL  [initandlisten]     distarch: x86_64
2020-10-23T17:50:41.720+0000 I  CONTROL  [initandlisten]     target_arch: x86_64
2020-10-23T17:50:41.720+0000 I  CONTROL  [initandlisten] options: { storage: { dbPath: "/var/lib/mongodb/", journal: { enabled: true } } }
2020-10-23T17:50:41.720+0000 I  STORAGE  [initandlisten] Detected data files in /var/lib/mongodb/ created by the 'wiredTiger' storage engine, so setting the active storage engine to 'wiredTiger'.
2020-10-23T17:50:41.720+0000 I  STORAGE  [initandlisten] 
2020-10-23T17:50:41.720+0000 I  STORAGE  [initandlisten] ** WARNING: Using the XFS filesystem is strongly recommended with the WiredTiger storage engine
2020-10-23T17:50:41.720+0000 I  STORAGE  [initandlisten] **          See http://dochub.mongodb.org/core/prodnotes-filesystem
2020-10-23T17:50:41.720+0000 I  STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=3476M,cache_overflow=(file_max=0M),session_max=33000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000,close_scan_interval=10,close_handle_minimum=250),statistics_log=(wait=0),verbose=[recovery_progress,checkpoint_progress],
2020-10-23T17:50:42.329+0000 I  STORAGE  [initandlisten] WiredTiger message [1603475442:329070][15886:0x7f91cd5dfb00], txn-recover: Recovering log 5 through 6
2020-10-23T17:50:42.366+0000 I  STORAGE  [initandlisten] WiredTiger message [1603475442:366919][15886:0x7f91cd5dfb00], txn-recover: Recovering log 6 through 6
2020-10-23T17:50:42.415+0000 I  STORAGE  [initandlisten] WiredTiger message [1603475442:415007][15886:0x7f91cd5dfb00], txn-recover: Main recovery loop: starting at 5/28160 to 6/256
2020-10-23T17:50:42.487+0000 I  STORAGE  [initandlisten] WiredTiger message [1603475442:487244][15886:0x7f91cd5dfb00], txn-recover: Recovering log 5 through 6
2020-10-23T17:50:42.528+0000 I  STORAGE  [initandlisten] WiredTiger message [1603475442:528110][15886:0x7f91cd5dfb00], txn-recover: Recovering log 6 through 6
2020-10-23T17:50:42.563+0000 I  STORAGE  [initandlisten] WiredTiger message [1603475442:563757][15886:0x7f91cd5dfb00], txn-recover: Set global recovery timestamp: (0, 0)
2020-10-23T17:50:42.599+0000 I  RECOVERY [initandlisten] WiredTiger recoveryTimestamp. Ts: Timestamp(0, 0)
2020-10-23T17:50:42.604+0000 I  STORAGE  [initandlisten] Timestamp monitor starting
2020-10-23T17:50:42.606+0000 I  CONTROL  [initandlisten] 
2020-10-23T17:50:42.606+0000 I  CONTROL  [initandlisten] ** WARNING: Access control is not enabled for the database.
2020-10-23T17:50:42.606+0000 I  CONTROL  [initandlisten] **          Read and write access to data and configuration is unrestricted.
2020-10-23T17:50:42.606+0000 I  CONTROL  [initandlisten] ** WARNING: You are running this process as the root user, which is not recommended.
2020-10-23T17:50:42.607+0000 I  CONTROL  [initandlisten] 
2020-10-23T17:50:42.607+0000 I  CONTROL  [initandlisten] ** WARNING: This server is bound to localhost.
2020-10-23T17:50:42.607+0000 I  CONTROL  [initandlisten] **          Remote systems will be unable to connect to this server. 
2020-10-23T17:50:42.607+0000 I  CONTROL  [initandlisten] **          Start the server with --bind_ip <address> to specify which IP 
2020-10-23T17:50:42.607+0000 I  CONTROL  [initandlisten] **          addresses it should serve responses from, or with --bind_ip_all to
2020-10-23T17:50:42.607+0000 I  CONTROL  [initandlisten] **          bind to all interfaces. If this behavior is desired, start the
2020-10-23T17:50:42.607+0000 I  CONTROL  [initandlisten] **          server with --bind_ip 127.0.0.1 to disable this warning.
2020-10-23T17:50:42.607+0000 I  CONTROL  [initandlisten] 
2020-10-23T17:50:42.609+0000 I  SHARDING [initandlisten] Marking collection local.system.replset as collection version: <unsharded>
2020-10-23T17:50:42.610+0000 I  STORAGE  [initandlisten] Flow Control is enabled on this deployment.
2020-10-23T17:50:42.610+0000 I  SHARDING [initandlisten] Marking collection admin.system.roles as collection version: <unsharded>
2020-10-23T17:50:42.610+0000 I  SHARDING [initandlisten] Marking collection admin.system.version as collection version: <unsharded>
2020-10-23T17:50:42.611+0000 I  SHARDING [initandlisten] Marking collection local.startup_log as collection version: <unsharded>
2020-10-23T17:50:42.611+0000 I  FTDC     [initandlisten] Initializing full-time diagnostic data capture with directory '/var/lib/mongodb/diagnostic.data'
2020-10-23T17:50:42.612+0000 I  SHARDING [LogicalSessionCacheRefresh] Marking collection config.system.sessions as collection version: <unsharded>
2020-10-23T17:50:42.612+0000 I  NETWORK  [listener] Listening on /tmp/mongodb-27017.sock
2020-10-23T17:50:42.612+0000 I  SHARDING [LogicalSessionCacheReap] Marking collection config.transactions as collection version: <unsharded>
2020-10-23T17:50:42.612+0000 I  NETWORK  [listener] Listening on 127.0.0.1
2020-10-23T17:50:42.612+0000 I  NETWORK  [listener] waiting for connections on port 27017
2020-10-23T17:50:43.000+0000 I  SHARDING [ftdc] Marking collection local.oplog.rs as collection version: <unsharded>

NETWORK [listener] waiting for connections on port 27017となっていればひとまずOK

そして，この実行したターミナルはそのままにしておいて，別ターミナルウィンドウでmongodbを起動してみてください．

$ mongo
MongoDB shell version v4.2.10
connecting to: mongodb://127.0.0.1:27017/?compressors=disabled&gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("51271f1b-ddc0-4774-9aed-c014ed2224ec") }
MongoDB server version: 4.2.10
Server has startup warnings: 
2020-10-23T17:50:41.720+0000 I  STORAGE  [initandlisten] 
2020-10-23T17:50:41.720+0000 I  STORAGE  [initandlisten] ** WARNING: Using the XFS filesystem is strongly recommended with the WiredTiger storage engine
2020-10-23T17:50:41.720+0000 I  STORAGE  [initandlisten] **          See http://dochub.mongodb.org/core/prodnotes-filesystem
2020-10-23T17:50:42.606+0000 I  CONTROL  [initandlisten] 
2020-10-23T17:50:42.606+0000 I  CONTROL  [initandlisten] ** WARNING: Access control is not enabled for the database.
2020-10-23T17:50:42.606+0000 I  CONTROL  [initandlisten] **          Read and write access to data and configuration is unrestricted.
2020-10-23T17:50:42.606+0000 I  CONTROL  [initandlisten] ** WARNING: You are running this process as the root user, which is not recommended.
2020-10-23T17:50:42.607+0000 I  CONTROL  [initandlisten] 
2020-10-23T17:50:42.607+0000 I  CONTROL  [initandlisten] ** WARNING: This server is bound to localhost.
2020-10-23T17:50:42.607+0000 I  CONTROL  [initandlisten] **          Remote systems will be unable to connect to this server. 
2020-10-23T17:50:42.607+0000 I  CONTROL  [initandlisten] **          Start the server with --bind_ip <address> to specify which IP 
2020-10-23T17:50:42.607+0000 I  CONTROL  [initandlisten] **          addresses it should serve responses from, or with --bind_ip_all to
2020-10-23T17:50:42.607+0000 I  CONTROL  [initandlisten] **          bind to all interfaces. If this behavior is desired, start the
2020-10-23T17:50:42.607+0000 I  CONTROL  [initandlisten] **          server with --bind_ip 127.0.0.1 to disable this warning.
2020-10-23T17:50:42.607+0000 I  CONTROL  [initandlisten] 
---
Enable MongoDB's free cloud-based monitoring service, which will then receive and display
metrics about your deployment (disk utilization, CPU, operation statistics, etc).

The monitoring data will be available on a MongoDB website with a unique URL accessible to you
and anyone you share the URL with. MongoDB may use this information to make product
improvements and to suggest MongoDB products and deployment options to you.

To enable free monitoring, run the following command: db.enableFreeMonitoring()
To permanently disable this reminder, run the following command: db.disableFreeMonitoring()
---

>

プロンプトが開けたので大丈夫そうです．
ひとまず，現時点で私ができる最大限の対処法となっています．

ただ，一度コマンドを実行したあと別ウィンドウで起動するというのがかなりめんどくさいので他のやり方がわかればまた記事を書きたいと思います．

最後に一応公式ドキュメントのリンクは載せておきます