storm-mqttを起動させる その１

Stormの新バージョンで色々つながるものが増える、とのことで、早速storm-mqttを試そう、というもの。

storm-mqttって？

これです。こいつのexamplesをまずは動かそうじゃないか、という試み。

想定している流れ

1. githubからのstormのclone https://github.com/apache/storm
2. cloneしてきたstormのビルド
3. examplesに載っている手順に則って実行

今回はひとまず2番まで書こうと思います。残りはまた次のタイミングで。

Stormがビルドできない

そもそもapache/stormにアクセスすると、README部分にこんな表示が。。。

なるほど。個別にビルドしていかないと無理かな。ということでひとまずターゲットとなるexternals/storm-mqttのみに絞ってビルドを開始。

storm−mqttのビルドが通らない

早速cloneしてきたstormのビルドに入ります。

で、こんな感じで、storm-mqttのビルドは通りませんでした。まぁ色々と依存関係あるから、そりゃそうだよね。一つずつ関連する部分をビルドして解消していきましょう。

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 14.614 s
[INFO] Finished at: 2016-03-05T15:28:10+09:00
[INFO] Final Memory: 16M/305M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project storm-mqtt-examples: Could not resolve dependencies for project org.apache.storm:storm-mqtt-examples:jar:2.0.0-SNAPSHOT: The following artifacts could not be resolved: org.apache.storm:storm-mqtt:jar:2.0.0-SNAPSHOT, org.apache.storm:storm-core:jar:2.0.0-SNAPSHOT, org.apache.storm:flux-core:jar:2.0.0-SNAPSHOT: Could not find artifact org.apache.storm:storm-mqtt:jar:2.0.0-SNAPSHOT in clojars (https://clojars.org/repo/) -> [Help 1]

全体でもビルドは通らない

なるほど、ではstorm-coreをビルドしに行こう、と思ったけど、一度頭からビルドしてみることに。意外と通ったりして。

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] Storm .............................................. SUCCESS [  1.319 s]
[INFO] multilang-javascript ............................... SUCCESS [  0.868 s]
[INFO] multilang-python ................................... SUCCESS [  0.115 s]
[INFO] multilang-ruby ..................................... SUCCESS [  0.104 s]
[INFO] maven-shade-clojure-transformer .................... SUCCESS [  1.463 s]
[INFO] storm-maven-plugins ................................ SUCCESS [  3.161 s]
[INFO] Storm Core ......................................... SUCCESS [01:11 min]
[INFO] storm-rename-hack .................................. SUCCESS [  4.624 s]
[INFO] storm-kafka ........................................ SUCCESS [  0.978 s]
[INFO] storm-hdfs ......................................... FAILURE [  3.219 s]
[INFO] storm-hbase ........................................ SKIPPED
[INFO] storm-hive ......................................... SKIPPED
[INFO] storm-jdbc ......................................... SKIPPED
[INFO] storm-redis ........................................ SKIPPED
[INFO] storm-eventhubs .................................... SKIPPED
[INFO] flux ............................................... SKIPPED
[INFO] flux-wrappers ...................................... SKIPPED
[INFO] flux-core .......................................... SKIPPED
[INFO] flux-examples ...................................... SKIPPED
[INFO] storm-sql-runtime .................................. SKIPPED
[INFO] storm-sql-core ..................................... SKIPPED
[INFO] storm-sql-kafka .................................... SKIPPED
[INFO] sql ................................................ SKIPPED
[INFO] storm-elasticsearch ................................ SKIPPED
[INFO] storm-solr ......................................... SKIPPED
[INFO] storm-metrics ...................................... SKIPPED
[INFO] storm-cassandra .................................... SKIPPED
[INFO] storm-mqtt-parent .................................. SKIPPED
[INFO] storm-mqtt ......................................... SKIPPED
[INFO] storm-mqtt-examples ................................ SKIPPED
[INFO] storm-clojure ...................................... SKIPPED
[INFO] storm-starter ...................................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:28 min
[INFO] Finished at: 2016-03-05T16:06:18+09:00
[INFO] Final Memory: 110M/1587M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project storm-hdfs: Could not resolve dependencies for project org.apache.storm:storm-hdfs:jar:2.0.0-SNAPSHOT: Could not find artifact org.apache.storm:storm-core:jar:tests:2.0.0-SNAPSHOT in confluent (http://packages.confluent.io/maven) -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :storm-hdfs

storm-hdfsが悪者っぽい

...storm-hdfsを個別にビルドしてみますか。

$mvn install package -Dmaven.test.skip=true
[INFO] Scanning for projects...
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building storm-hdfs 2.0.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
Downloading: http://packages.confluent.io/maven/org/apache/storm/storm-core/2.0.0-SNAPSHOT/maven-metadata.xml
Downloading: http://packages.confluent.io/maven/org/apache/storm/storm/2.0.0-SNAPSHOT/maven-metadata.xml
Downloading: http://packages.confluent.io/maven/org/apache/storm/storm-core/2.0.0-SNAPSHOT/storm-core-2.0.0-SNAPSHOT-tests.jar
Downloading: https://clojars.org/repo/org/apache/storm/storm-core/2.0.0-SNAPSHOT/storm-core-2.0.0-SNAPSHOT-tests.jar
Downloading: http://repository.apache.org/snapshots/org/apache/storm/storm-core/2.0.0-SNAPSHOT/storm-core-2.0.0-SNAPSHOT-tests.jar
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 8.308 s
[INFO] Finished at: 2016-03-06T14:05:48+09:00
[INFO] Final Memory: 18M/225M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project storm-hdfs: Could not resolve dependencies for project org.apache.storm:storm-hdfs:jar:2.0.0-SNAPSHOT: Could not find artifact org.apache.storm:storm-core:jar:tests:2.0.0-SNAPSHOT in confluent (http://packages.confluent.io/maven) -> [Help 1]

oh...

ちなみに面倒だと思ってstorm-hdfsをビルド対象から除外しましたが、同じエラーがstorm-hiveで発生しました。どうやらこいつを解消しないとビルドは通らないようです。

結論：test-jarが必要。

storm-coreでおそらくtest-jarをすれば良い、と見えるので、その通り実施。

mvn test jar:test-jar install

この後、頭からのビルドも綺麗に通りました。

さっそく面倒なことになりましたが、なんとかビルドまでは完了。ここから今回の目標としたstorm-mqttのexamplesを動かそうと思います。

最初に頭からビルドして通った人っているんですかね？？？