まとめ
- embulk本体はJava7で動作するように作られているようです
- embulkの各プラグインは、プラグイン開発者の環境に依存しJava7で動かないこともあるようです。
- Java7で動かない場合でも自分でコンパイルをすれば動くことがあるかもしれません。
- build.gradleにオプションを追加することで、Java8上でJava7互換のバイナリを作ることができるようです。
Embulkの開発環境
@h_michael_z 今のところ7でコンパイルしてますね
— FURUHASHI Sadayuki (@frsyuki) 2015, 7月 24
JDK8で作ったプラグインをJava7上で動かした場合
私が作った、embulk-parser-apache-logはJava8上で作っていました。
これを、Java7(OpenJDK1.7)で動してみます。
OpenJDK Runtime Environment (rhel-2.5.5.1.el7_1-x86_64 u79-b14)
OpenJDK 64-Bit Server VM (build 24.79-b02, mixed mode)
次のようなエラーがでます。
2015-07-24 17:51:36.258 +0900: Embulk v0.6.19
LoadError: load error: embulk/parser/apache-log -- java.lang.UnsupportedClassVersionError: org/embulk/parser/ApacheLogParserPlugin : Unsupported major.minor version 52.0
version 52.0はJava8のことを表しているようです。
参考 Unsupported major.minor version 51.0 = Java7 以上使え
自分でビルド
一般的なビルド手順は次のとおりです。
(※私が作ったプラグインはJava8から導入された構文をつかっておりコンパイルができないので他のプラグインで例示します。)
git clone https://github.com/civitaspo/embulk-output-hdfs.git
cd embulk-output-hdfs
./gradlew gem
cd pkg
embulk gem install -l embulk-output-hdfs-0.1.0.gem
Java8でJava7のプラグインを作るには
build.gradleに次の行を追加すれば良いようです。(どこに追加すれば良いのか不明だったのでversionのところに追加
version = "0.1.0"
sourceCompatibility = 1.7
targetCompatibility = 1.7
確認
Compatibilityをつけている場合(major versionが51になりJava7用のバイナリになっている)
javap -cp build/libs/embulk-output-hdfs-0.1.0.jar -verbose org/embulk/output/HdfsOutputPlugin | grep version
minor version: 0
major version: 51
Compatibilityなしの場合
javap -cp build/libs/embulk-output-hdfs-0.1.0.jar -verbose org/embulk/output/HdfsOutputPlugin | grep version
minor version: 0
major version: 52
注意点
上記の対策は不十分な場合もあるようです。
最後この warning 出るけど、これは java7 入れたほうがてっとり速そうだな ... > http://t.co/Uq6dVf6CXs
— そのっつ (SEO Naotoshi) (@sonots) 2015, 7月 24
参考
ログ
% embulk run test.yml
2015-07-24 17:51:36.258 +0900: Embulk v0.6.19
LoadError: load error: embulk/parser/apache-log -- java.lang.UnsupportedClassVersionError: org/embulk/parser/ApacheLogParserPlugin : Unsupported major.minor version 52.0
require at org/jruby/RubyKernel.java:1071
require at /path/to/embulk/bin/embulk!/META-INF/jruby.home/lib/ruby/shared/rubygems/core_ext/kernel_require.rb:135
require at /path/to/embulk/bin/embulk!/META-INF/jruby.home/lib/ruby/shared/rubygems/core_ext/kernel_require.rb:133
require at /path/to/embulk/bin/embulk!/META-INF/jruby.home/lib/ruby/shared/rubygems/core_ext/kernel_require.rb:124
require_and_show at file:/path/to/embulk/bin/embulk!/embulk/plugin_registry.rb:79
search at file:/path/to/embulk/bin/embulk!/embulk/plugin_registry.rb:39
lookup at file:/path/to/embulk/bin/embulk!/embulk/plugin_registry.rb:26
lookup at file:/path/to/embulk/bin/embulk!/embulk/plugin.rb:196
new_java_parser at file:/path/to/embulk/bin/embulk!/embulk/plugin.rb:166
new_java_parser at /path/to/embulk/bin/embulk!/META-INF/jruby.home/lib/ruby/1.9/forwardable.rb:201
run at file:/path/to/embulk/bin/embulk!/embulk/command/embulk_run.rb:351
(root) at classpath:embulk/command/embulk.rb:47
org.embulk.config.ConfigException: ParserPlugin 'apache-log' is not found.
(LoadError) load error: embulk/parser/apache-log -- java.lang.UnsupportedClassVersionError: org/embulk/parser/ApacheLogParserPlugin : Unsupported major.minor version 52.0
at org.embulk.plugin.PluginManager.buildPluginNotFoundException(org/embulk/plugin/PluginManager.java:59)
at org.embulk.plugin.PluginManager.newPlugin(org/embulk/plugin/PluginManager.java:44)
at org.embulk.spi.ExecSession.newPlugin(org/embulk/spi/ExecSession.java:166)
at org.embulk.spi.Exec.newPlugin(org/embulk/spi/Exec.java:74)
at org.embulk.spi.FileInputRunner.newParserPlugin(org/embulk/spi/FileInputRunner.java:56)
at org.embulk.spi.FileInputRunner$RunnerControl.<init>(org/embulk/spi/FileInputRunner.java:100)
at org.embulk.spi.FileInputRunner.transaction(org/embulk/spi/FileInputRunner.java:63)
at org.embulk.exec.BulkLoader.doRun(org/embulk/exec/BulkLoader.java:480)
at org.embulk.exec.BulkLoader.access$100(org/embulk/exec/BulkLoader.java:36)
at org.embulk.exec.BulkLoader$1.run(org/embulk/exec/BulkLoader.java:342)
at org.embulk.exec.BulkLoader$1.run(org/embulk/exec/BulkLoader.java:338)
at org.embulk.spi.Exec.doWith(org/embulk/spi/Exec.java:25)
at org.embulk.exec.BulkLoader.run(org/embulk/exec/BulkLoader.java:338)
at org.embulk.command.Runner.run(org/embulk/command/Runner.java:149)
at org.embulk.command.Runner.main(org/embulk/command/Runner.java:101)
at java.lang.reflect.Method.invoke(java/lang/reflect/Method.java:606)
at RUBY.run(file:/path/to/embulk/bin/embulk!/embulk/command/embulk_run.rb:351)
at classpath_3a_embulk.command.embulk.(root)(classpath:embulk/command/embulk.rb:47)
at classpath_3a_embulk.command.embulk.(root)(classpath_3a_embulk/command/classpath:embulk/command/embulk.rb:47)
at org.embulk.cli.Main.main(org/embulk/cli/Main.java:11)
Suppressed: org.embulk.plugin.PluginSourceNotMatchException
at org.embulk.plugin.InjectedPluginSource.newPlugin(InjectedPluginSource.java:54)
at org.embulk.plugin.PluginManager.newPlugin(PluginManager.java:38)
at org.embulk.spi.ExecSession.newPlugin(ExecSession.java:166)
at org.embulk.spi.Exec.newPlugin(Exec.java:74)
at org.embulk.spi.FileInputRunner.newParserPlugin(FileInputRunner.java:56)
at org.embulk.spi.FileInputRunner$RunnerControl.<init>(FileInputRunner.java:100)
at org.embulk.spi.FileInputRunner.transaction(FileInputRunner.java:63)
at org.embulk.exec.BulkLoader.doRun(BulkLoader.java:480)
at org.embulk.exec.BulkLoader.access$100(BulkLoader.java:36)
at org.embulk.exec.BulkLoader$1.run(BulkLoader.java:342)
at org.embulk.exec.BulkLoader$1.run(BulkLoader.java:338)
at org.embulk.spi.Exec.doWith(Exec.java:25)
at org.embulk.exec.BulkLoader.run(BulkLoader.java:338)
at org.embulk.command.Runner.run(Runner.java:149)
at org.embulk.command.Runner.main(Runner.java:101)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(JavaMethod.java:470)
at org.jruby.javasupport.JavaMethod.invokeDirect(JavaMethod.java:328)
at org.jruby.java.invokers.InstanceMethodInvoker.call(InstanceMethodInvoker.java:71)
at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:346)
at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:204)
at org.jruby.ast.CallTwoArgNode.interpret(CallTwoArgNode.java:59)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)
at org.jruby.ast.RescueNode.executeBody(RescueNode.java:221)
at org.jruby.ast.RescueNode.interpret(RescueNode.java:116)
at org.jruby.ast.BeginNode.interpret(BeginNode.java:83)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)
at org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
at org.jruby.ast.CaseNode.interpret(CaseNode.java:138)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)
at org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
at org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74)
at org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:182)
at org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:203)
at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:326)
at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:170)
at classpath_3a_embulk.command.embulk.__file__(classpath:embulk/command/embulk.rb:47)
at classpath_3a_embulk.command.embulk.load(classpath:embulk/command/embulk.rb)
at org.jruby.Ruby.runScript(Ruby.java:866)
at org.jruby.Ruby.runScript(Ruby.java:859)
at org.jruby.Ruby.runNormally(Ruby.java:728)
at org.jruby.Ruby.runFromMain(Ruby.java:577)
at org.jruby.Main.doRunFromMain(Main.java:395)
at org.jruby.Main.internalRun(Main.java:290)
at org.jruby.Main.run(Main.java:217)
at org.jruby.Main.main(Main.java:197)
at org.embulk.cli.Main.main(Main.java:11)
Suppressed: org.embulk.plugin.PluginSourceNotMatchException: org.jruby.exceptions.RaiseException: (LoadError) load error: embulk/parser/apache-log -- java.lang.UnsupportedClassVersionError: org/embulk/parser/ApacheLogParserPlugin : Unsupported major.minor version 52.0
at org.embulk.jruby.JRubyPluginSource.newPlugin(JRubyPluginSource.java:69)
at org.embulk.plugin.PluginManager.newPlugin(PluginManager.java:38)
at org.embulk.spi.ExecSession.newPlugin(ExecSession.java:166)
at org.embulk.spi.Exec.newPlugin(Exec.java:74)
at org.embulk.spi.FileInputRunner.newParserPlugin(FileInputRunner.java:56)
at org.embulk.spi.FileInputRunner$RunnerControl.<init>(FileInputRunner.java:100)
at org.embulk.spi.FileInputRunner.transaction(FileInputRunner.java:63)
at org.embulk.exec.BulkLoader.doRun(BulkLoader.java:480)
at org.embulk.exec.BulkLoader.access$100(BulkLoader.java:36)
at org.embulk.exec.BulkLoader$1.run(BulkLoader.java:342)
at org.embulk.exec.BulkLoader$1.run(BulkLoader.java:338)
at org.embulk.spi.Exec.doWith(Exec.java:25)
at org.embulk.exec.BulkLoader.run(BulkLoader.java:338)
at org.embulk.command.Runner.run(Runner.java:149)
at org.embulk.command.Runner.main(Runner.java:101)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(JavaMethod.java:470)
at org.jruby.javasupport.JavaMethod.invokeDirect(JavaMethod.java:328)
at org.jruby.java.invokers.InstanceMethodInvoker.call(InstanceMethodInvoker.java:71)
at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:346)
at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:204)
at org.jruby.ast.CallTwoArgNode.interpret(CallTwoArgNode.java:59)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)
at org.jruby.ast.RescueNode.executeBody(RescueNode.java:221)
at org.jruby.ast.RescueNode.interpret(RescueNode.java:116)
at org.jruby.ast.BeginNode.interpret(BeginNode.java:83)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)
at org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
at org.jruby.ast.CaseNode.interpret(CaseNode.java:138)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)
at org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
at org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74)
at org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:182)
at org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:203)
at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:326)
at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:170)
at classpath_3a_embulk.command.embulk.__file__(classpath:embulk/command/embulk.rb:47)
at classpath_3a_embulk.command.embulk.load(classpath:embulk/command/embulk.rb)
at org.jruby.Ruby.runScript(Ruby.java:866)
at org.jruby.Ruby.runScript(Ruby.java:859)
at org.jruby.Ruby.runNormally(Ruby.java:728)
at org.jruby.Ruby.runFromMain(Ruby.java:577)
at org.jruby.Main.doRunFromMain(Main.java:395)
at org.jruby.Main.internalRun(Main.java:290)
at org.jruby.Main.run(Main.java:217)
at org.jruby.Main.main(Main.java:197)
at org.embulk.cli.Main.main(Main.java:11)
Caused by: org.jruby.exceptions.RaiseException: (LoadError) load error: embulk/parser/apache-log -- java.lang.UnsupportedClassVersionError: org/embulk/parser/ApacheLogParserPlugin : Unsupported major.minor version 52.0
at org.jruby.RubyKernel.require(org/jruby/RubyKernel.java:1071)
at Kernel.require(/path/to/embulk/bin/embulk!/META-INF/jruby.home/lib/ruby/shared/rubygems/core_ext/kernel_require.rb:135)
at Kernel.require(/path/to/embulk/bin/embulk!/META-INF/jruby.home/lib/ruby/shared/rubygems/core_ext/kernel_require.rb:133)
at Kernel.require(/path/to/embulk/bin/embulk!/META-INF/jruby.home/lib/ruby/shared/rubygems/core_ext/kernel_require.rb:124)
at RUBY.require_and_show(file:/path/to/embulk/bin/embulk!/embulk/plugin_registry.rb:79)
at RUBY.search(file:/path/to/embulk/bin/embulk!/embulk/plugin_registry.rb:39)
at RUBY.lookup(file:/path/to/embulk/bin/embulk!/embulk/plugin_registry.rb:26)
at RUBY.lookup(file:/path/to/embulk/bin/embulk!/embulk/plugin.rb:196)
at RUBY.new_java_parser(file:/path/to/embulk/bin/embulk!/embulk/plugin.rb:166)
at RUBY.new_java_parser(/path/to/embulk/bin/embulk!/META-INF/jruby.home/lib/ruby/1.9/forwardable.rb:201)
at RUBY.run(file:/path/to/embulk/bin/embulk!/embulk/command/embulk_run.rb:351)
at classpath_3a_embulk.command.embulk.(root)(classpath:embulk/command/embulk.rb:47)
Error: ParserPlugin 'apache-log' is not found.
(LoadError) load error: embulk/parser/apache-log -- java.lang.UnsupportedClassVersionError: org/embulk/parser/ApacheLogParserPlugin : Unsupported major.minor version 52.0