ScalaにおけるHTTP Client サンプルコード集+α

背景

ScalaではHTTP Clientを実装する際に多くの選択肢が存在するが、それぞれの使い方やざっくりした特性が載っている資料がかなり少ない。

概要

　本投稿では比較的メジャーなOSSライブラリに含まれているHTTP Clientを対象にサクッとCRUDを実現するためのサンプルコードとそれぞれの良し悪しについてざっくりまとめる。サンプルコードは動作確認済みで、Githubにも載せてあります。本サンプルではHTTPによる同期通信を対象とします。
　より詳細なライブラリの使い方を知りたい方は参考資料の各種ドキュメントページをご参照ください。

対象読者

• Scalaで使いたいHTTP Clientを選定する際の足掛かりにしたい人
• 対象ライブラリの使い方を知りたい人

対象ライブラリ

2019/03/04現在
- scalaj-http(★854)
- skinny-framework(★693)
- sttp(★647)
- Akka(★9585)
- Dispatch(★419)
- Apache(★765)

※これにhttp4sも含める予定でしたが、http4sで作成したAPIに対して依存度の高い設計になっていた点と学習コストの高さにより、今回は除外しました。
※各種ライブラリはHTTP Clientに特化したものではないので、スター数はあんまり当てにならないですがご参考まで。

各種ドキュメント

• scalaj-http(scaladoc)
• skinny-framework(doc)
• sttp(doc)
• Akka(doc)
• Dispatch(doc)
• Apache(doc)

各種ライブラリ特性早見表

	scalaj-http	skinny	sttp	Akka	Dispatch	Apache
ドキュメントの見やすさ	×	△	○	○	△	△
シンプルさ	◎	◎	○	×	△	△
機能豊富さ	○	△	△	◎	×	◎
ステータスコード定数型の有無	×	×	○	○	×	○
非同期処理	×	○	○	◎	○	○

用途によりけりですが、単にHTTPリクエストを出したいだけであれば、シンプルなscalaj-http,skinny-frameworkが使い勝手が良いと思います。Future型を使ってスマートにレスポンスを取得したい方はAkka,Dispatchあたりが使えると思います。

HTTP Clientサンプルコード

サンプルコードではリクエストを送信し、取得したレスポンス結果をコンバートして受け取るところまでを想定しています。
※サーバー側は含まれておりません。

各種ライブラリVer

build.sbt

name := "scala-http-client-sample"

version := "0.1"

scalaVersion := "2.12.8"

libraryDependencies ++= Seq(
  //akka
  "com.typesafe.akka" %% "akka-http" % "10.1.7",
  "com.typesafe.akka" %% "akka-http-testkit" % "10.1.7" % Test,
  "com.typesafe.akka" %% "akka-actor" % "2.5.21",
  "com.typesafe.akka" %% "akka-stream" % "2.5.21",
  "com.typesafe.akka" %% "akka-stream-testkit" % "2.5.21" % Test,

  //apache
  "org.apache.httpcomponents" % "httpclient" % "4.5.7",

  //skinny
  "org.skinny-framework" %% "skinny-http-client" % "3.0.1",
  "org.slf4j" % "slf4j-log4j12" % "1.7.26" % Test,
  "log4j" % "log4j" % "1.2.17",

  //sttp
  "com.softwaremill.sttp" %% "core" % "1.5.11",

  //dispatch
  "org.dispatchhttp" %% "dispatch-core" % "1.0.0",

  //scalaj
  "org.scalaj" %% "scalaj-http" % "2.4.1"

  //test
  "org.scalactic" %% "scalactic" % "3.0.5",
  "org.scalatest" %% "scalatest" % "3.0.5" % "test",
  "junit" % "junit" % "4.12",
  "com.novocode" % "junit-interface" % "0.11" % "test",
  "net.liftweb" %% "lift-json" % "3.3.0",
)

各種ライブラリはMVNRepositoryで最新の安定版を使用しています。

INPUT

• URL
• ヘッダ(複数)
• postData(POST,PUTのみ)

OUTPUT

• ステータスコード
• レスポンスボディ

共通親クラス

CommonHtttpClientSample.scala

package http.client


trait WebMethod {
  def Get(url: String, headers: Map[String, String]): CustomResponse

  def Post(url: String, headers: Map[String, String], requestBody: String): CustomResponse

  def Put(url: String, headers: Map[String, String], requestBody: String): CustomResponse

  def Delete(url: String, headers: Map[String, String]): CustomResponse
}

abstract class CommonHttpClientSample extends WebMethod {

  val readTimeoutMillis = 5000
  var connectTimeoutMillis = 5000
}

case class CustomResponse(statusCode: Int, status: String, body: String)

各種ライブラリで使用するHTTP通信の外部インタフェースをGET,POST,PUT,DELETEで統一してあります。
read timeout,connect timeoutは5sに設定してあります。

scalaj-http sample

ScalajHttpClientSample.scala

package http.client

import scalaj.http._

class ScalajHttpClientSample extends CommonHttpClientSample {

  def Get(url: String, headers: Map[String, String]): CustomResponse = httpRequest("GET", url, headers)

  def Post(url: String, headers: Map[String, String], requestBody: String): CustomResponse = httpRequest("POST", url, headers, requestBody)

  def Put(url: String, headers: Map[String, String], requestBody: String): CustomResponse = httpRequest("PUT", url, headers, requestBody)

  def Delete(url: String, headers: Map[String, String]): CustomResponse = httpRequest("DELETE", url, headers)

  def httpRequest(webMethod: String, url: String, headers: Map[String, String], requestBody: String = ""): CustomResponse = {

    var request = Http(url)
      .headers(headers)
      .timeout(connTimeoutMs = connectTimeoutMillis, readTimeoutMs = readTimeoutMillis)
    if (requestBody != "") {
      request = request.postData(requestBody)
    }
    request = request.method(webMethod) // when request is put methods, Next to web method of postData method.
    val response: HttpResponse[String] = request.execute()

    return extractResponse(response)
  }

  def extractResponse(res: HttpResponse[String]): CustomResponse = {

    val statusCode = res.code
    val httpStatus = statusCode match {
      case 200 => "OK"
      case 201 => "CREATED"
      case 204 => "NO_CONTENT"
      case 401 => "UNAUTHORIZED"
      case 400 => "BAD_REQUEST"
      case 404 => "NOT_FOUND"
      case 500 => "INTERNAL_SERVER_ERROR"
      case _ => "OTHER_STATUS_CODE"
    }
    val body = res.body
    return CustomResponse(statusCode, httpStatus, body)
  }

}

メリット

• headersがTurple,Map,Seqに対応している
• シンプルでわかりやすい

デメリット

• Webメソッド用の型がない
• ステータスコードの定数型がない

skinny-framework sample

skinny-framework.scala

package http.client

import skinny.http._

class SkinnyHttpClientSample extends CommonHttpClientSample {

  def Get(url: String, headers: Map[String, String]): CustomResponse = httpRequest(Method.GET, url, headers)

  def Post(url: String, headers: Map[String, String], requestBody: String): CustomResponse = httpRequest(Method.POST, url, headers, requestBody)

  def Put(url: String, headers: Map[String, String], requestBody: String): CustomResponse = httpRequest(Method.PUT, url, headers, requestBody)

  def Delete(url: String, headers: Map[String, String]): CustomResponse = httpRequest(Method.DELETE, url, headers)

  def httpRequest(webMethod: Method, url: String, headers: Map[String, String], requestBody: String = ""): CustomResponse = {

    val request = new Request(url)
    request.connectTimeoutMillis(this.readTimeoutMillis)
    request.readTimeoutMillis(this.connectTimeoutMillis)
    if (requestBody != "") {
      request.body(requestBody.getBytes)
    }
    headers.foreach {
      case (k, v) => {
        request.header(k, v)
      }
    }
    val response = HTTP.request(webMethod, request)

    return extractResponse(response)
  }

  def extractResponse(res: Response): CustomResponse = {

    val statusCode = res.status
    val httpStatus = statusCode match {
      case 200 => "OK"
      case 201 => "CREATED"
      case 204 => "NO_CONTENT"
      case 401 => "UNAUTHORIZED"
      case 400 => "BAD_REQUEST"
      case 404 => "NOT_FOUND"
      case 500 => "INTERNAL_SERVER_ERROR"
      case _ => "OTHER_STATUS_CODE"
    }
    val body = res.textBody
    return CustomResponse(statusCode, httpStatus, body)
  }
}

メリット

• シンプルでわかりやすい
• 依存モジュールが少ない

デメリット

• ステータスコードの定数型がない

sttp sample

SttpHttpClientSample.scala

package http.client

import com.softwaremill.sttp._

import scala.concurrent.duration._

class SttpHttpClientSample extends CommonHttpClientSample {

  implicit val backend = HttpURLConnectionBackend()
  val readTimeout: Duration = 5.seconds

  def Get(url: String, headers: Map[String, String]): CustomResponse = httpRequest(Method.GET, url, headers)

  def Post(url: String, headers: Map[String, String], requestBody: String): CustomResponse = httpRequest(Method.POST, url, headers, requestBody)

  def Put(url: String, headers: Map[String, String], requestBody: String): CustomResponse = httpRequest(Method.PUT, url, headers, requestBody)

  def Delete(url: String, headers: Map[String, String]): CustomResponse = httpRequest(Method.DELETE, url, headers)

  def httpRequest(webMethod: Method, url: String, headers: Map[String, String], requestBody: String = ""): CustomResponse = {

    var request = sttp
      .method(webMethod, uri"$url")
      .headers(headers)
      .readTimeout(DefaultReadTimeout)
    if (requestBody != "") {
      request = request.body(requestBody)
    }
    val response = request.send()
    extractResponse(response)
  }

  def extractResponse(response: Response[String]): CustomResponse = {

    val statusCode = response.code
    val httpStatus = statusCode match {
      case StatusCodes.Ok => "OK"
      case StatusCodes.Created => "CREATED"
      case StatusCodes.Unauthorized => "UNAUTHORIZED"
      case StatusCodes.BadRequest => "BAD_REQUEST"
      case StatusCodes.NotFound => "NOT_FOUND"
      case StatusCodes.NoContent => "NO_CONTENT"
      case StatusCodes.InternalServerError => "INTERNAL_SERVER_ERROR"
      case _ => "OTHER_STATUS_CODE"
    }
    val body = response.body.right.get
    return new CustomResponse(statusCode, httpStatus, body)
  }
}

メリット

• ステータスコード用の型がある
• Either型を使ったレスポンス結果の取得

デメリット

• implictの定義が必要

Akka sample

AkkaHttpClientSample.scala

package http.client

import akka.actor.ActorSystem
import akka.http.scaladsl.Http
import akka.http.scaladsl.model.HttpMethods._
import akka.http.scaladsl.model._
import akka.http.scaladsl.model.headers.RawHeader
import akka.http.scaladsl.unmarshalling.Unmarshal
import akka.stream.ActorMaterializer
import akka.util.Timeout

import scala.concurrent.Await
import scala.concurrent.duration._

class AkkaHttpClientSample() extends CommonHttpClientSample{

  implicit val timeout = Timeout(5 seconds)
  implicit val system = ActorSystem()
  implicit val materializer = ActorMaterializer()

  def Get(uri: String, headers: Map[String, String]): CustomResponse = httpRequest(GET, uri, headers)

  def Post(uri: String, headers: Map[String, String], body: String): CustomResponse = httpRequest(POST, uri, headers, body)

  def Put(uri: String, headers: Map[String, String], body: String): CustomResponse = httpRequest(PUT, uri, headers, body: String)

  def Delete(uri: String, headers: Map[String, String]): CustomResponse = httpRequest(DELETE, uri, headers)

  def httpRequest(webMethod: HttpMethod, uri: String, headers: Map[String, String], requestBody: String = ""): CustomResponse = {

    val rawHeaders = headers.map {
      case (k, v) => {
        RawHeader(k, v)
      }
    }.toList

    val httpEntity = requestBody match {
      case "" => HttpEntity.Empty
      case _ => HttpEntity(requestBody)
    }
    val req = HttpRequest(webMethod, uri = Uri(uri), entity = httpEntity)
      .withHeaders(
        rawHeaders: _*,
      )
    val responseFuture = Http().singleRequest(req)
    return extractResponse(Await.result(responseFuture, timeout.duration)) //同期取得
  }

  def extractResponse(res: HttpResponse): CustomResponse = {

    val statusCode = res.status.intValue()
    val httpStatus = statusCode match {
      case StatusCodes.OK.intValue => "OK"
      case StatusCodes.Created.intValue => "CREATED"
      case StatusCodes.Unauthorized.intValue => "UNAUTHORIZED"
      case StatusCodes.BadRequest.intValue => "BAD_REQUEST"
      case StatusCodes.NotFound.intValue => "NOT_FOUND"
      case StatusCodes.NoContent.intValue => "NO_CONTENT"
      case StatusCodes.InternalServerError.intValue => "INTERNAL_SERVER_ERROR"
      case _ => "OTHER_STATUS_CODE"
    }
    val body = Unmarshal(res.entity).to[String]
    return CustomResponse(statusCode, httpStatus, Await.result(body, timeout.duration))
  }
}

メリット

• ステータスコード用の型がある
• 柔軟なエラー制御

デメリット

• ソースが複雑
• implictの定義が必要

Dispatch sample

DispatchHttpClientSample.scala

package http.client

import dispatch._
import org.asynchttpclient.util.HttpConstants.Methods
import org.asynchttpclient.{RequestBuilder, Response}

class DispatcherHttpClientSample extends CommonHttpClientSample {

  def Get(url: String, headers: Map[String, String]): CustomResponse = httpRequest(Methods.GET, url, headers)

  def Post(url: String, headers: Map[String, String], requestBody: String): CustomResponse = httpRequest(Methods.POST, url, headers, requestBody)

  def Put(url: String, headers: Map[String, String], requestBody: String): CustomResponse = httpRequest(Methods.PUT, url, headers, requestBody)

  def Delete(url: String, headers: Map[String, String]): CustomResponse = httpRequest(Methods.DELETE, url, headers)

  def httpRequest(webMethod: String, uri: String, headers: Map[String, String], requestBody: String = ""): CustomResponse = {

    var request = new RequestBuilder(webMethod)
      .setUrl(uri)
      .setReadTimeout(readTimeoutMillis)
      .setRequestTimeout(connectTimeoutMillis)

    for (header <- headers) {
      request = request.setHeader(header._1, header._2)
    }
    if (requestBody != "") {
      request = request.setBody(requestBody)
    }
    val futureResponse = Http.default.client.prepareRequest(request).execute()
    return extractResponse(futureResponse.get())
  }

  def extractResponse(res: Response): CustomResponse = {

    val statusCode = res.getStatusCode
    val httpStatus = statusCode match {
      case 200 => "OK"
      case 201 => "CREATED"
      case 204 => "NO_CONTENT"
      case 401 => "UNAUTHORIZED"
      case 400 => "BAD_REQUEST"
      case 404 => "NOT_FOUND"
      case 500 => "INTERNAL_SERVER_ERROR"
      case _ => "OTHER_STATUS_CODE"
    }
    val body = res.getResponseBody
    return CustomResponse(statusCode, httpStatus, body)
  }


}

メリット

• Future型を使ったレスポンスの取得

デメリット

• ステータスコード用の型がない
• asynchttpclientに依存する

Apache sample

ApacheHttpClientSample.scala

package http.client

import org.apache.http.HttpStatus
import org.apache.http.client.config.RequestConfig
import org.apache.http.client.methods._
import org.apache.http.entity.StringEntity
import org.apache.http.impl.client.{CloseableHttpClient, HttpClients}
import org.apache.http.util.EntityUtils


class ApacheHttpClientSample()extends CommonHttpClientSample {

  def Get(url: String, headers: Map[String, String]): CustomResponse = httpRequestNoBody(headers, new HttpGet(url))

  def Post(url: String, headers: Map[String, String], requestBody: String): CustomResponse = httpRequest(headers, new HttpPost(url), requestBody)

  def Put(url: String, headers: Map[String, String], requestBody: String): CustomResponse = httpRequest(headers, new HttpPut(url), requestBody)

  def Delete(url: String, headers: Map[String, String]): CustomResponse = httpRequestNoBody(headers, new HttpDelete(url))

  //timeout setting
  private val requestConfig: RequestConfig = RequestConfig.custom()
    .setConnectTimeout(this.connectTimeoutMillis)
    .setConnectionRequestTimeout(this.readTimeoutMillis)
    .build()

  private val defaultHttpClient: CloseableHttpClient = HttpClients.createDefault()

  def extractResponse(response: CloseableHttpResponse): CustomResponse = {

    val statusCode = response.getStatusLine.getStatusCode()
    val httpStatus = statusCode match {
      case HttpStatus.SC_OK => "OK"
      case HttpStatus.SC_CREATED => "CREATED"
      case HttpStatus.SC_UNAUTHORIZED => "UNAUTHORIZED"
      case HttpStatus.SC_BAD_REQUEST => "BAD_REQUEST"
      case HttpStatus.SC_NOT_FOUND => "NOT_FOUND"
      case HttpStatus.SC_NO_CONTENT => "NO_CONTENT"
      case HttpStatus.SC_INTERNAL_SERVER_ERROR => "INTERNAL_SERVER_ERROR"
      case _ => "OTHER_STATUS_CODE"
    }
    val body = statusCode match {
      case HttpStatus.SC_OK | HttpStatus.SC_CREATED => EntityUtils.toString(response.getEntity)
      case _ => ""
    }
    return new CustomResponse(statusCode, httpStatus, body)
  }

  def httpRequest(headers: Map[String, String], webMethod: HttpEntityEnclosingRequestBase, body: String): CustomResponse = {
    webMethod.setConfig(requestConfig)
    headers.foreach {
      case (k, v) => {
        webMethod.addHeader(k, v)
      }
    }
    webMethod.setEntity(new StringEntity(body))
    val response = defaultHttpClient.execute(webMethod)
    return extractResponse(response)
  }

  def httpRequestNoBody(headers: Map[String, String], webMethod: HttpRequestBase): CustomResponse = {
    webMethod.setConfig(requestConfig)
    headers.foreach {
      case (k, v) => {
        webMethod.addHeader(k, v)
      }
    }
    val response = defaultHttpClient.execute(webMethod)
    return extractResponse(response)
  }
}

メリット

• ステータスコード用の定数型がある
• Javaとの互換性

デメリット

• GET系とPOST系でWebメソッドの型が異なる。
• importする型が多い

まとめ

HTTP Clientが使用可能ないくつかのライブラリについてCRUDを実現する簡単なリクエストのサンプルを紹介しました。
　全体的にHTTPリクエストする際の基本的な設定機能は備わっているので、request,response周りに用意されている変数を参照しつつ用途に応じて選ぶのが良いかと思います。