LoginSignup
1
0

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?

@nekopanchi0423

jqコマンドを使ってJSONをいい感じに整える

Posted at

はじめに

APIのレスポンスデータをJSON形式で扱う際に、そのままでは見づらいという問題がありました。何かいい方法はないかと探したところ、jqというコマンドラインツールを見つけたので紹介します。

jqコマンド

jqは、JSONデータを整形、フィルタリング、変換するための強力なツールで、データの操作を簡単にしてくれます。今回は試しにUSGS（アメリカ地質調査所）の地震情報APIを使って、レスポンスデータをいい感じに整形していきます。
エンドポイント：https://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/4.5_day.geojson

jqコマンドなしの場合

以下は単純にcurlコマンドのみでAPIを叩いた場合です。

curl -s "https://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/4.5_day.geojson"

ぱっと見自分が欲しい値を見つけるのが大変ですよね。

{"type":"FeatureCollection","metadata":{"generated":1725522255000,"url":"https://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/4.5_day.geojson","title":"USGS Magnitude 4.5+ Earthquakes, Past Day","status":200,"api":"1.10.3","count":8},"features":[{"type":"Feature","properties":{"mag":4.9,"place":"south of Panama","time":1725514018720,"updated":1725517438040,"tz":null,"url":"https://earthquake.usgs.gov/earthquakes/eventpage/us6000npsw","detail":"https://earthquake.usgs.gov/earthquakes/feed/v1.0/detail/us6000npsw.geojson","felt":null,"cdi":null,"mmi":null,"alert":null,"status":"reviewed","tsunami":0,"sig":369,"net":"us","code":"6000npsw","ids":",us6000npsw,","sources":",us,","types":",origin,phase-data,","nst":52,"dmin":3.731,"rms":1.08,"gap":141,"magType":"mww","type":"earthquake","title":"M 4.9 - south of Panama"},"geometry":{"type":"Point","coordinates":[-82.756,4.6982,10]},"id":"us6000npsw"},
{"type":"Feature","properties":{"mag":5.1,"place":"71 km SSE of Modisi, Indonesia","time":1725510691387,"updated":1725511749040,"tz":null,"url":"https://earthquake.usgs.gov/earthquakes/eventpage/us6000npsr","detail":"https://earthquake.usgs.gov/earthquakes/feed/v1.0/detail/us6000npsr.geojson","felt":null,"cdi":null,"mmi":null,"alert":null,"status":"reviewed","tsunami":0,"sig":400,"net":"us","code":"6000npsr","ids":",us6000npsr,","sources":",us,","types":",origin,phase-data,","nst":72,"dmin":2.959,"rms":1.13,"gap":56,"magType":"mb","type":"earthquake","title":"M 5.1 - 71 km SSE of Modisi, Indonesia"},"geometry":{"type":"Point","coordinates":[124.565,-0.1858,64.088]},"id":"us6000npsr"},
{"type":"Feature","properties":{"mag":4.5,"place":"270 km SSW of Severo-Kuril’sk, Russia","time":1725505023932,"updated":1725507179040,"tz":null,"url":"https://earthquake.usgs.gov/earthquakes/eventpage/us6000npsh","detail":"https://earthquake.usgs.gov/earthquakes/feed/v1.0/detail/us6000npsh.geojson","felt":null,"cdi":null,"mmi":null,"alert":null,"status":"reviewed","tsunami":0,"sig":312,"net":"us","code":"6000npsh","ids":",us6000npsh,","sources":",us,","types":",origin,phase-data,","nst":59,"dmin":5.047,"rms":0.57,"gap":128,"magType":"mb","type":"earthquake","title":"M 4.5 - 270 km SSW of Severo-Kuril’sk, Russia"},"geometry":{"type":"Point","coordinates":[154.4504,48.5055,65.355]},"id":"us6000npsh"},
⋮

jqのインストール

brew install jq

jqの基本的な使い方

jqをパイプで繋ぎます(jq .はjqでも可)。

curl -s "https://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/4.5_day.geojson" | jq .

レスポンスは以下の通り。

{
  "type": "FeatureCollection",
  "metadata": {
    "generated": 1725522795000,
    "url": "https://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/4.5_day.geojson",
    "title": "USGS Magnitude 4.5+ Earthquakes, Past Day",
    "status": 200,
    "api": "1.10.3",
    "count": 8
  },
  "features": [
    {
      "type": "Feature",
      "properties": {
        "mag": 4.9,
        "place": "south of Panama",
        "time": 1725514018720,
        "updated": 1725517438040,
        "tz": null,
        "url": "https://earthquake.usgs.gov/earthquakes/eventpage/us6000npsw",
        "detail": "https://earthquake.usgs.gov/earthquakes/feed/v1.0/detail/us6000npsw.geojson",
        "felt": null,
        "cdi": null,
        "mmi": null,
        "alert": null,
        "status": "reviewed",
        "tsunami": 0,
        "sig": 369,
        "net": "us",
        "code": "6000npsw",
        "ids": ",us6000npsw,",
        "sources": ",us,",
        "types": ",origin,phase-data,",
        "nst": 52,
        "dmin": 3.731,
        "rms": 1.08,
        "gap": 141,
        "magType": "mww",
        "type": "earthquake",
        "title": "M 4.9 - south of Panama"
      },
      "geometry": {
        "type": "Point",
        "coordinates": [
          -82.756,
          4.6982,
          10
        ]
      },
      "id": "us6000npsw"
    },
    {
      "type": "Feature",
      "properties": {
        "mag": 5.1,
        "place": "71 km SSE of Modisi, Indonesia",
        "time": 1725510691387,
        "updated": 1725511749040,
        "tz": null,
        "url": "https://earthquake.usgs.gov/earthquakes/eventpage/us6000npsr",
        "detail": "https://earthquake.usgs.gov/earthquakes/feed/v1.0/detail/us6000npsr.geojson",
        "felt": null,
        "cdi": null,
        "mmi": null,
        "alert": null,
        "status": "reviewed",
        "tsunami": 0,
        "sig": 400,
        "net": "us",
        "code": "6000npsr",
        "ids": ",us6000npsr,",
        "sources": ",us,",
        "types": ",origin,phase-data,",
        "nst": 72,
        "dmin": 2.959,
        "rms": 1.13,
        "gap": 56,
        "magType": "mb",
        "type": "earthquake",
        "title": "M 5.1 - 71 km SSE of Modisi, Indonesia"
      },
      "geometry": {
        "type": "Point",
        "coordinates": [
          124.565,
          -0.1858,
          64.088
        ]
      },
      "id": "us6000npsr"
    },
    ⋮

JSONデータのフィルタリング

jqを使って特定のフィールドを取り出すこともできます。以下の例ではマグニチュード(mag)と場所(place)を取り出します。

curl -s "https://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/4.5_day.geojson" | jq '.features[].properties | {mag: .mag, place: .place}'

features配列の各要素に対して、magとplaceフィールドを抽出します。

{
  "mag": 4.9,
  "place": "south of Panama"
}
{
  "mag": 5.1,
  "place": "71 km SSE of Modisi, Indonesia"
}
{
  "mag": 4.5,
  "place": "270 km SSW of Severo-Kuril’sk, Russia"
}
⋮

高度なフィルタリング

jqには、条件付きフィルタリングや集計、再帰的なデータ操作なども可能です。

条件付きフィルタリング

以下の例では、マグニチュードが5以上の地震情報だけを抽出します。

curl -s "https://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/4.5_day.geojson" | jq '.features[].properties | select(.mag >= 5) | {mag: .mag, place: .place}'

{
  "mag": 5.1,
  "place": "71 km SSE of Modisi, Indonesia"
}
{
  "mag": 6.2,
  "place": "66 km NNE of Angoram, Papua New Guinea"
}
{
  "mag": 5.3,
  "place": "11 km ESE of Lata, Solomon Islands"
}
⋮

データの集計

以下の例では、地震のマグニチュードの平均値を計算することもできます。

curl -s "https://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/4.5_day.geojson" | jq '[.features[].properties.mag] | add / length'

5.075

再帰的なデータ操作

再帰的にデータを探索も可能です。事前に条件を指定することで、それに一致するデータを抽出することもできます。例えば、ネストされたすべてのオブジェクトからmag（マグニチュード）フィールドを持つものを抽出します。

curl -s "https://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/4.5_day.geojson" | jq '.. | objects | select(has("mag"))'

このコマンドは、以下のように動作します：

  1. 再帰的にすべての要素を探索
  2. objectsで、JSONオブジェクトだけを選択
  3. select(has("mag"))で、magフィールドを持つオブジェクトだけを抽出
{
  "mag": 5,
  "place": "south of the Kermadec Islands",
  "time": 1725524717650,
  "updated": 1725526251040,
  "tz": null,
  "url": "https://earthquake.usgs.gov/earthquakes/eventpage/us6000nptr",
  "detail": "https://earthquake.usgs.gov/earthquakes/feed/v1.0/detail/us6000nptr.geojson",
  "felt": null,
  "cdi": null,
  "mmi": null,
  "alert": null,
  "status": "reviewed",
  "tsunami": 0,
  "sig": 385,
  "net": "us",
  "code": "6000nptr",
  "ids": ",us6000nptr,",
  "sources": ",us,",
  "types": ",origin,phase-data,",
  "nst": 36,
  "dmin": 4.165,
  "rms": 0.94,
  "gap": 124,
  "magType": "mb",
  "type": "earthquake",
  "title": "M 5.0 - south of the Kermadec Islands"
}
{
  "mag": 4.9,
  "place": "south of Panama",
  "time": 1725514018720,
  "updated": 1725517438040,
  "tz": null,
  "url": "https://earthquake.usgs.gov/earthquakes/eventpage/us6000npsw",
  "detail": "https://earthquake.usgs.gov/earthquakes/feed/v1.0/detail/us6000npsw.geojson",
  "felt": null,
  "cdi": null,
  "mmi": null,
  "alert": null,
  "status": "reviewed",
  "tsunami": 0,
  "sig": 369,
  "net": "us",
  "code": "6000npsw",
  "ids": ",us6000npsw,",
  "sources": ",us,",
  "types": ",origin,phase-data,",
  "nst": 52,
  "dmin": 3.731,
  "rms": 1.08,
  "gap": 141,
  "magType": "mww",
  "type": "earthquake",
  "title": "M 4.9 - south of Panama"
}
{
  "mag": 5.1,
  "place": "71 km SSE of Modisi, Indonesia",
  "time": 1725510691387,
  "updated": 1725511749040,
  "tz": null,
  "url": "https://earthquake.usgs.gov/earthquakes/eventpage/us6000npsr",
  "detail": "https://earthquake.usgs.gov/earthquakes/feed/v1.0/detail/us6000npsr.geojson",
  "felt": null,
  "cdi": null,
  "mmi": null,
  "alert": null,
  "status": "reviewed",
  "tsunami": 0,
  "sig": 400,
  "net": "us",
  "code": "6000npsr",
  "ids": ",us6000npsr,",
  "sources": ",us,",
  "types": ",origin,phase-data,",
  "nst": 72,
  "dmin": 2.959,
  "rms": 1.13,
  "gap": 56,
  "magType": "mb",
  "type": "earthquake",
  "title": "M 5.1 - 71 km SSE of Modisi, Indonesia"
}
⋮

JSONデータからCSVへの変換

jqを使えばJSONデータをCSV形式に変換することも可能です。マグニチュードと場所を抽出してCSVに変換してみます。

curl -s "https://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/4.5_day.geojson" | jq -r '.features[].properties | [.mag, .place] | @csv'

-rオプションは、jqの出力を生の文字列として表示するためのものです。

4.9,"south of Panama"
5.1,"71 km SSE of Modisi, Indonesia"
4.5,"270 km SSW of Severo-Kuril’sk, Russia"
6.2,"66 km NNE of Angoram, Papua New Guinea"
⋮

最後に

jqは、JSONデータを扱う上で非常に便利なツールです。整形、フィルタリング、変換など、さまざまな操作を簡単に行うことができます。特に、APIのレスポンスを解析したり、大量のJSONデータを処理する際には、その威力を発揮します。ぜひ、jqを使ってJSONデータの操作を効率化してみてください。

1
0
0

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
Sign upLogin
1
0

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?