Go to Qiita Advent Calendar 2024 Top
LoginSignup
1
1

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?

More than 3 years have passed since last update.

@yokomotod(Daisuke Yokomoto)

Elasticsearchコードリーディング 番外編: ローカルES起動Tips

Last updated at Posted at 2020-12-23

コードリーディングの強い味方、デバッグ実行のために見出した小技集

公式ドキュメント: TESTING.asciidoc

起動

./gradlew run

なおデフォルトでセキュリティが有効なのでアクセスには要認証 username: 'elastic-admin', password: 'elastic-password' (gradle/run.grade に記述)

デバッグ実行

IntelliJ IDEAでデバッガを起動しておいてから

./gradlew run --debug-jvm

xpack.security 無効化

デバッグ実行時にsecurityチェック系のステップが邪魔なときに

gradle/run.gradle
 testClusters {
   runTask {
     testDistribution = System.getProperty('run.distribution', 'default')
     if (System.getProperty('run.distribution', 'default') == 'default') {
       String licenseType = System.getProperty("run.license_type", "basic")
       if (licenseType == 'trial') {
         setting 'xpack.ml.enabled', 'true'
         setting 'xpack.graph.enabled', 'true'
         setting 'xpack.watcher.enabled', 'true'
         setting 'xpack.license.self_generated.type', 'trial'
       } else if (licenseType != 'basic') {
         throw new IllegalArgumentException("Unsupported self-generated license type: [" + licenseType + "[basic] or [trial].")
       }
-      setting 'xpack.security.enabled', 'true'
+      setting 'xpack.security.enabled', 'false'
       keystore 'bootstrap.password', 'password'
       user username: 'elastic-admin', password: 'elastic-password', role: 'superuser'
     }
   }
 }

起動時にクラスタのデータをリセットさせない

./gradlew run --data-dir=/tmp/foo

なおデフォルトだと build/testclusters/runTask-0/data

ビルドのたびに消える

複数ノード起動

2020/12/23現在、まだ公式にはサポートされていなさそう https://github.com/elastic/elasticsearch/issues/51837

ひと手間かけると出来た

gradle/run.gradle
 testClusters {
   runTask {
+    numberOfNodes = 2
     testDistribution = System.getProperty('run.distribution', 'default')
     if (System.getProperty('run.distribution', 'default') == 'default') {
       String licenseType = System.getProperty("run.license_type", "basic")
       if (licenseType == 'trial') {
         setting 'xpack.ml.enabled', 'true'
         setting 'xpack.graph.enabled', 'true'
         setting 'xpack.watcher.enabled', 'true'
         setting 'xpack.license.self_generated.type', 'trial'
       } else if (licenseType != 'basic') {
         throw new IllegalArgumentException("Unsupported self-generated license type: [" + licenseType + "[basic] or [trial].")
       }
       setting 'xpack.security.enabled', 'true'
       keystore 'bootstrap.password', 'password'
       user username: 'elastic-admin', password: 'elastic-password', role: 'superuser'
     }
   }
 }

[WARN ][o.e.d.FileBasedSeedHostsProvider] [runTask-1] expected, but did not find, a dynamic hosts list at [echo "127.0.0.1:9300\n127.0.0.1:53398\n" >]
[WARN ][o.e.d.FileBasedSeedHostsProvider] [runTask-0] expected, but did not find, a dynamic hosts list at [/Users/xxxxxx/build/testclusters/runTask-0/config/unicast_hosts.txt]
[WARN ][o.e.c.c.ClusterFormationFailureHelper] [runTask-0] master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eligible nodes [runTask-0, runTask-1] to bootstrap a cluster: have discovered [{runTask-0}{jh5avgLQRySNsv8hlbrYVw}{2OKFKwzfQlmfCZAWvmjNZw}{127.0.0.1}{127.0.0.1:9300}{cdhilmrstw}{ml.machine_memory=17179869184, xpack.installed=true, transform.node=true, testattr=test, ml.max_open_jobs=20}]; discovery will continue using [] from hosts providers and [{runTask-0}{jh5avgLQRySNsv8hlbrYVw}{2OKFKwzfQlmfCZAWvmjNZw}{127.0.0.1}{127.0.0.1:9300}{cdhilmrstw}{ml.machine_memory=17179869184, xpack.installed=true, transform.node=true, testattr=test, ml.max_open_jobs=20}] from last-known cluster state; node term 0, last-accepted version 0 in term 0
[WARN ][o.e.c.c.ClusterFormationFailureHelper] [runTask-1] master not discovered yet, this node has not previously joined a bootstrapped (v7+) cluster, and this node must discover master-eligible nodes [runTask-0, runTask-1] to bootstrap a cluster: have discovered [{runTask-1}{wC1-z8tpRTG60jV7r2bxrQ}{ISsboB0QQaux9nzO8m2ogg}{127.0.0.1}{127.0.0.1:53398}{cdhilmrstw}{ml.machine_memory=17179869184, xpack.installed=true, transform.node=true, testattr=test, ml.max_open_jobs=20}]; discovery will continue using [] from hosts providers and [{runTask-1}{wC1-z8tpRTG60jV7r2bxrQ}{ISsboB0QQaux9nzO8m2ogg}{127.0.0.1}{127.0.0.1:53398}{cdhilmrstw}{ml.machine_memory=17179869184, xpack.installed=true, transform.node=true, testattr=test, ml.max_open_jobs=20}] from last-known cluster state; node term 0, last-accepted version 0 in term 0

という状態になるので、

echo "127.0.0.1:9300\n127.0.0.1:53398\n" > build/testclusters/runTask-0/config/unicast_hosts.txt
echo "127.0.0.1:9300\n127.0.0.1:53398\n" > build/testclusters/runTask-1/config/unicast_hosts.txt

curl "localhost:9200/_cat/nodes?pretty"

127.0.0.1 39 99 20 2.27   cdhilmrstw - runTask-1
127.0.0.1 38 99 20 2.27   cdhilmrstw * runTask-0

curl "localhost:53400/_cat/nodes?pretty"

127.0.0.1 39 98 20 2.42   cdhilmrstw - runTask-1
127.0.0.1 37 98 20 2.42   cdhilmrstw * runTask-0

:tada:

複数ノード起動でデバッグ実行

デバッガを複数プロセス起動できるなら不要なはず

IntelliJ IDEAで↑のやり方がわからず、１プロセスだけだとデバッグサーバー接続失敗でエラーになるので、強引にデバッグモードは1プロセスだけに

buildSrc/src/main/java/org/elasticsearch/gradle/testclusters/RunTask.java
     @Override
     public void beforeStart() {
         int debugPort = 5005;
         int httpPort = 9200;
         int transportPort = 9300;
         Map<String, String> additionalSettings = System.getProperties()
             .entrySet()
             .stream()
             .filter(entry -> entry.getKey().toString().startsWith(CUSTOM_SETTINGS_PREFIX))
             .collect(
                 Collectors.toMap(
                     entry -> entry.getKey().toString().substring(CUSTOM_SETTINGS_PREFIX.length()),
                     entry -> entry.getValue().toString()
                 )
             );
         boolean singleNode = getClusters().stream().flatMap(c -> c.getNodes().stream()).count() == 1;
         final Function<ElasticsearchNode, Path> getDataPath;
         if (singleNode) {
             getDataPath = n -> dataDir;
         } else {
             getDataPath = n -> dataDir.resolve(n.getName());
         }

         for (ElasticsearchCluster cluster : getClusters()) {
             cluster.getFirstNode().setHttpPort(String.valueOf(httpPort));
             httpPort++;
             cluster.getFirstNode().setTransportPort(String.valueOf(transportPort));
             transportPort++;
             for (ElasticsearchNode node : cluster.getNodes()) {
                 additionalSettings.forEach(node::setting);
                 if (dataDir != null) {
                     node.setDataPath(getDataPath.apply(node));
                 }
-                if (debug) {
+                if (debug && debugPort == 5005) {
                     logger.lifecycle("Running elasticsearch in debug mode, {} expecting running debug server on port {}", node, debugPort);
                     node.jvmArgs("-agentlib:jdwp=transport=dt_socket,server=n,suspend=y,address=" + debugPort);
                     debugPort += 1;
                 }
                 if (keystorePassword.length() > 0) {
                     node.keystorePassword(keystorePassword);
                 }
             }
         }
     }

ログレベル変更

curl -X PUT -H 'content-type: application/json' localhost:9200/_cluster/settings -d '
{
  "transient": {
    "logger.org.elasticsearch.cluster.service.MasterService": "TRACE"
  }
}
'
1
1
0

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
Sign upLogin
1
1

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?