awscliとjqを組み合わせた JSON Parse方法解説

はじめに

awscli で出力する結果から、一部の値を抜き出すときのjqの組み立て方について解説します

Sample Output

aws コマンドでPolicy一覧を出力した結果を基に、jqでparseする考えかたを解説していきます

以下がSampleの出力結果です

[root@sugi-awscli policies]# aws --output json --profile managepolicy iam list-policies
{
    "Policies": [
        {
            "PolicyName": "AmazonEKSManagementPolicy", 
            "PermissionsBoundaryUsageCount": 0, 
            "CreateDate": "2018-08-18T15:17:34Z", 
            "AttachmentCount": 0, 
            "IsAttachable": true, 
            "PolicyId": "ANPAJD3NVUDDKGF53CTZ4", 
            "DefaultVersionId": "v1", 
            "Path": "/", 
            "Arn": "arn:aws:iam::999999999999:policy/AmazonEKSManagementPolicy", 
            "UpdateDate": "2018-08-18T15:17:34Z"
        }, 
        {
            "PolicyName": "CloudWatchEventsFullAccess", 
            "PermissionsBoundaryUsageCount": 0, 
            "CreateDate": "2016-01-14T18:37:08Z", 
            "AttachmentCount": 0, 
            "IsAttachable": true, 
            "PolicyId": "ANPAJZLOYLNHESMYOJAFU", 
            "DefaultVersionId": "v1", 
            "Path": "/", 
            "Arn": "arn:aws:iam::aws:policy/CloudWatchEventsFullAccess", 
            "UpdateDate": "2016-01-14T18:37:08Z"
        }, 
        {
            "PolicyName": "AWSSupportServiceRolePolicy", 
            "PermissionsBoundaryUsageCount": 0, 
            "CreateDate": "2018-04-19T18:04:44Z", 
            "AttachmentCount": 1, 
            "IsAttachable": false, 
            "PolicyId": "ANPAJ7W6266ELXF5MISDS", 
            "DefaultVersionId": "v2", 
            "Path": "/aws-service-role/", 
            "Arn": "arn:aws:iam::aws:policy/aws-service-role/AWSSupportServiceRolePolicy", 
            "UpdateDate": "2018-08-06T20:58:25Z"
        }
    ]
}

最上位 Policies の値を取得

jsonの出力結果の最上位に "Policies" が存在しています。
"Policies" 配下の値を全部取得するjqのクエリは以下の指定となります

aws --output json --profile managepolicy iam list-policies | jq '.Policies'

outputされるJSON。Policies配下に絞り込みされています

[root@sugi-awscli policies]# aws --output json --profile managepolicy iam list-policies | jq '.Policies'
[
  {
    "PolicyName": "AmazonEKSManagementPolicy",
    "PermissionsBoundaryUsageCount": 0,
    "CreateDate": "2018-08-18T15:17:34Z",
    "AttachmentCount": 0,
    "IsAttachable": true,
    "PolicyId": "ANPAJD3NVUDDKGF53CTZ4",
    "DefaultVersionId": "v1",
    "Path": "/",
    "Arn": "arn:aws:iam::999999999999:policy/AmazonEKSManagementPolicy",
    "UpdateDate": "2018-08-18T15:17:34Z"
  },
  {
    "PolicyName": "CloudWatchEventsFullAccess",
    "PermissionsBoundaryUsageCount": 0,
    "CreateDate": "2016-01-14T18:37:08Z",
    "AttachmentCount": 0,
    "IsAttachable": true,
    "PolicyId": "ANPAJZLOYLNHESMYOJAFU",
    "DefaultVersionId": "v1",
    "Path": "/",
    "Arn": "arn:aws:iam::aws:policy/CloudWatchEventsFullAccess",
    "UpdateDate": "2016-01-14T18:37:08Z"
  },
  {
    "PolicyName": "AWSSupportServiceRolePolicy",
    "PermissionsBoundaryUsageCount": 0,
    "CreateDate": "2018-04-19T18:04:44Z",
    "AttachmentCount": 1,
    "IsAttachable": false,
    "PolicyId": "ANPAJ7W6266ELXF5MISDS",
    "DefaultVersionId": "v2",
    "Path": "/aws-service-role/",
    "Arn": "arn:aws:iam::aws:policy/aws-service-role/AWSSupportServiceRolePolicy",
    "UpdateDate": "2018-08-06T20:58:25Z"
  }
]

Policies配下で特定の値が含まれている要素を取得

map(select()) の指定で、特定の値が含まれている要素のみ取得します

aws --output json --profile managepolicy iam list-policies | jq -r '.Policies | map(select(.PolicyName == "AmazonEKSManagementPolicy"))'

json output

[root@sugi-awscli policies]# aws --output json --profile managepolicy iam list-policies | jq -r '.Policies | map(select(.PolicyName == "AmazonEKSManagementPolicy"))'
[
  {
    "PolicyName": "AmazonEKSManagementPolicy",
    "PermissionsBoundaryUsageCount": 0,
    "CreateDate": "2018-08-18T15:17:34Z",
    "AttachmentCount": 0,
    "IsAttachable": true,
    "PolicyId": "ANPAJD3NVUDDKGF53CTZ4",
    "DefaultVersionId": "v1",
    "Path": "/",
    "Arn": "arn:aws:iam::999999999999:policy/AmazonEKSManagementPolicy",
    "UpdateDate": "2018-08-18T15:17:34Z"
  }
]

上記の結果は配列となっているため、配列を除外するために、map(select()) の後ろに [] を付与します

aws --output json --profile managepolicy iam list-policies | jq -r '.Policies | map(select(.PolicyName == "AmazonEKSManagementPolicy"))[]'

出力例

[root@sugi-awscli policies]# aws --output json --profile managepolicy iam list-policies | jq -r '.Policies | map(select(.PolicyName == "AmazonEKSManagementPolicy"))[]'
{
  "PolicyName": "AmazonEKSManagementPolicy",
  "PermissionsBoundaryUsageCount": 0,
  "CreateDate": "2018-08-18T15:17:34Z",
  "AttachmentCount": 0,
  "IsAttachable": true,
  "PolicyId": "ANPAJD3NVUDDKGF53CTZ4",
  "DefaultVersionId": "v1",
  "Path": "/",
  "Arn": "arn:aws:iam::999999999999:policy/AmazonEKSManagementPolicy",
  "UpdateDate": "2018-08-18T15:17:34Z"
}

さらに上記から、Arn(AWS Resource Name) の値を取得します。なお出力結果にダブルクォーテーションを付与したくない場合は、 option に r を付けます

aws --output json --profile managepolicy iam list-policies | jq -r '.Policies | map(select(.PolicyName == "AmazonEKSManagementPolicy"))[].Arn'

実行例

[root@sugi-awscli policies]# aws --output json --profile managepolicy iam list-policies | jq -r '.Policies | map(select(.PolicyName == "AmazonEKSManagementPolicy"))[].Arn'
arn:aws:iam::999999999999:policy/AmazonEKSManagementPolicy

これで特定の要素の特定の値を取得することが出来ました。bash scriptなどで自動化する際にうまく活用できそうです。

memo1

階層が深いJSONからSelectする例

aws --output json ec2 describe-subnets | jq -r '.Subnets | map(select(.Tags[0].Value == "eks_service_subnet2"))[].SubnetId'

memo2

ハイフンが含まれている場合は、["aaaa-bbbb"] みたいにカッコで囲む必要がある
以下 Oracle Cloud 用のコマンドを利用した例

oci network vcn list | jq -r '.data | map(select(.["display-name"] == "TutorialVCN"))'
oci network security-list list --vcn-id $vcn_ocid | jq '.data[0]."egress-security-rules"

memo3

filterした結果、複数の値を取得。

oci lb load-balancer list | jq -r '.data | map(select(."ip-addresses"[0]."ip-address" == "129.213.14.248"))[] | ."id", ."display-name", ."time-created"'

参考　https://hacknote.jp/archives/35873/

参考URL

awscliでqueryを使用した絞り込み
https://qiita.com/draco/items/fa09ae0c2f51de9de449