Finagle から Solr の javabin フォーマットを使う

Finagle から Solr 6 へアクセスしてみる で Finagle から Solr へリクエストした際に指定したレスポンスフォーマットは wt=json だったが、 これを Solrj と同じ wt=javabin を指定するようにして試してみる。

javabin フォーマットのパースには Solrj の org.apache.solr.common.util.JavaBinCodec クラスを使った。

build.sbt は以下のような内容。

build.sbt

name := "finagle-solr-ex3"

version := "1.0"

scalaVersion := "2.11.8"

resolvers += "twitter-repo" at "https://maven.twttr.com"

libraryDependencies ++= Seq(
  "com.twitter" %% "finagle-http" % "6.35.+",
  "com.twitter" %% "twitter-server" % "1.20.+",
  "org.apache.solr" % "solr-solrj" % "6.0.0"
)

コードは以下。

HTTPServer.scala

import java.io.ByteArrayInputStream

import com.twitter.finagle.http.{Request, RequestBuilder, Response}
import com.twitter.finagle.{Http, Service}
import com.twitter.logging.Logger
import com.twitter.server.TwitterServer
import com.twitter.util.{Await, Future, FuturePool}
import org.apache.commons.io.IOUtils
import org.apache.solr.common.util.JavaBinCodec

class SolrService extends Service[Request, Response] {

  private[this] val log = Logger.get(getClass)
  private[this] val solrServer = "http://192.168.33.98:8983/solr/mycore"
  private[this] val client: Service[Request, Response] = Http.client.newService("192.168.33.98:8983")
  private[this] val solrRequest = RequestBuilder.safeBuildGet(RequestBuilder.create().url(s"${solrServer}/select?q=*:*&wt=javabin"))

  override def apply(request: Request): Future[Response] =  {
    log.info("request received.")
    FuturePool.unboundedPool {
      val f = client(solrRequest) flatMap { res =>
        val bs = new ByteArrayInputStream(IOUtils.toByteArray(res.getInputStream()))
        val m = new JavaBinCodec().unmarshal(bs)
        bs.close()
        res.setContentString(m.toString)
        Future.value(res)
      }
      Await.result(f)
    }
  }
}

object HTTPServer extends TwitterServer {

  def main:Unit = {
    val service = new SolrService

    val server = Http.serve(":8080", service)
    onExit {
      log.info("close http server")
      service.close()
      server.close()
    }
    Await.ready(server)
  }
}

前回同様 sbt run して ab で適当にリクエストを発行。

 % ab -n 10000 -c 10 http://localhost:8080/
This is ApacheBench, Version 2.3 <$Revision: 1528965 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 1000 requests
Completed 2000 requests
Completed 3000 requests
Completed 4000 requests
Completed 5000 requests
Completed 6000 requests
Completed 7000 requests
Completed 8000 requests
Completed 9000 requests
Completed 10000 requests
Finished 10000 requests


Server Software:        
Server Hostname:        localhost
Server Port:            8080

Document Path:          /
Document Length:        3283 bytes

Concurrency Level:      10
Time taken for tests:   19.285 seconds
Complete requests:      10000
Failed requests:        34
   (Connect: 0, Receive: 0, Length: 34, Exceptions: 0)
Total transferred:      33640034 bytes
HTML transferred:       32830034 bytes
Requests per second:    518.54 [#/sec] (mean)
Time per request:       19.285 [ms] (mean)
Time per request:       1.928 [ms] (mean, across all concurrent requests)
Transfer rate:          1703.48 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.1      0       4
Processing:     3   19  28.9     16     836
Waiting:        3   19  28.8     15     836
Total:          3   19  28.9     16     837

Percentage of the requests served within a certain time (ms)
  50%     16
  66%     20
  75%     24
  80%     26
  90%     33
  95%     41
  98%     51
  99%     62
 100%    837 (longest request)

何度か試してみたが、なんか前より遅くなったっぽい。
javabin より json の方がパースが軽いのか？

まぁその分なんとなく Solr サーバ側の負荷は下がりそうな気がしなくもないが、取り敢えずそっちはまだ計測してない。

参考

javabin - Solr Wiki
https://wiki.apache.org/solr/javabin