はじめに

aws opensearch describe-reserved-instance-offerings コマンドは､AWS OpenSearch Serviceで利用可能なリザーブドインスタンスの情報を取得するためのコマンドです｡購入したいリザーブドインスタンスのReservedInstanceOfferingIdを取得し､それを aws opensearch purchase-reserved-instance-offering コマンドの --reserved-instance-offering-id オプションに渡すことで､リザーブドインスタンスを購入することができます｡

しかし､一度に取得可能な件数は最大100件､--max-resultsオプションで指定可能な件数も最大100件です｡それ以降はページ分割されているため､全件を取得したい､または2ページ目以降にある結果を取得するためには､出力内容に含まれるNextTokenを使用しページ遷移しながら情報を取得する必要があります｡

加えて､aws opensearch describe-reserved-instance-offeringsコマンドには､ aws ec2 describe-reserved-instances-offeringsやaws rds describe-reserved-db-instances-offeringsと異なり､ --filterオプションや抽出条件を指定するオプションがありません｡条件を指定して結果を取得するためには出力結果に対して--queryで絞り込む必要があります｡

describe-reserved-instance-offerings のドキュメントにあるように､条件を指定するオプションが何もありません｡

describe-reserved-instance-offerings
[--reserved-instance-offering-id <value>]
[--max-results <value>]
[--next-token <value>]
[--cli-input-json | --cli-input-yaml]
[--generate-cli-skeleton <value>]

解決したいこと

"何度もコマンドを実行することなく､必要な情報を取得したい｡"

そこで､marjamis/aws_cli_paginator.shを参考に(というか､ほぼそのまま使って)､スクリプトを作成します｡

参考情報

準備するもの

AWS CLI version 2
jqコマンド

ゴール

一回のコマンド実行でReservedInstanceOfferingIdを全件取得する｡
条件を指定してReservedInstanceOfferingIdを取得する｡

コマンドライン引数で条件を指定できるようにすると､他のスクリプトとの連携などが容易になりそうです｡

全件取得

#!/bin/bash -xe

# Below command can be replaced to the required CLI, including with custom JSON output, assuming the NextToken is in the same location.
AWS_CLI_COMMAND="aws opensearch describe-reserved-instance-offerings --max-results 100 --query={NextToken:NextToken,ReservedInstanceOfferings:ReservedInstanceOfferings[*]}"
OUTPUT_FILE="./output-$(date +%s)"

function CLI_call() {
  if [ -z $NEXT_TOKEN ]; then
    cli_output=$($AWS_CLI_COMMAND)
  else
    cli_output=$($AWS_CLI_COMMAND --next-token $NEXT_TOKEN)
  fi

  echo $cli_output >> $OUTPUT_FILE
  echo $cli_output | jq -r ".NextToken"
}

while [ "$NEXT_TOKEN" != "null" ]; do
  NEXT_TOKEN=$(CLI_call $NEXT_TOKEN)
done

# Not 100% necessary but can be used to cleanup the output from the multiple API calls into a neater JSON formatted output.
cat $OUTPUT_FILE | jq . > $OUTPUT_FILE-master.json
rm $OUTPUT_FILE

出力結果

384件あったため､100件ごとに区切られたJSONが合計4つ結合されていました｡

{
  "NextToken": "AAIAAWcI8bC00cCMg7h7BkKSN53AIfFNQqRHXn2twpHLhhOA_QEYltOqMGmZz7qrueh3IONiuwGLoww=",
  "ReservedInstanceOfferings": [
    {
      "ReservedInstanceOfferingId": "006325ce-5966-42b1-b5fd-1ced436bf90c",
      "InstanceType": "r5.xlarge.search",
      "Duration": 31536000,
      "FixedPrice": 0,
      "UsagePrice": 0,
      "CurrencyCode": "USD",
      "PaymentOption": "NO_UPFRONT",
      "RecurringCharges": [
        {
          "RecurringChargeAmount": 0.309,
          "RecurringChargeFrequency": "Hourly"
        }
      ]
    },
    {
      "ReservedInstanceOfferingId": "01022ef5-8986-435c-8598-11c55c3ad7ea",
      "InstanceType": "r5.large.search",
      "Duration": 94608000,
      "FixedPrice": 1472,
      "UsagePrice": 0,
      "CurrencyCode": "USD",
      "PaymentOption": "PARTIAL_UPFRONT",
      "RecurringCharges": [
        {
          "RecurringChargeAmount": 0.056,
          "RecurringChargeFrequency": "Hourly"
        }
      ]
    },
    {
      "ReservedInstanceOfferingId": "01849fb3-0d36-4539-8688-02c894f5c055",
      "InstanceType": "c6g.8xlarge.search",
      "Duration": 94608000,
      "FixedPrice": 14940.18,
      "UsagePrice": 0,
      "CurrencyCode": "USD",
      "PaymentOption": "PARTIAL_UPFRONT",
      "RecurringCharges": [
        {
          "RecurringChargeAmount": 0.569,
          "RecurringChargeFrequency": "Hourly"
        }
      ]
    },
(以下略)

条件を1つ指定

条件に､InstanceType==r5.xlarge.search を指定します｡

#!/bin/bash -xe

# Below command can be replaced to the required CLI, including with custom JSON output, assuming the NextToken is in the same location.
AWS_CLI_COMMAND="aws opensearch describe-reserved-instance-offerings --max-results 100 --query={NextToken:NextToken,ReservedInstanceOfferings:ReservedInstanceOfferings[?InstanceType==\`r5.xlarge.search\`]}"
OUTPUT_FILE="./output-$(date +%s)"

function CLI_call() {
  if [ -z $NEXT_TOKEN ]; then
    cli_output=$($AWS_CLI_COMMAND)
  else
    cli_output=$($AWS_CLI_COMMAND --next-token $NEXT_TOKEN)
  fi

  echo $cli_output >> $OUTPUT_FILE
  echo $cli_output | jq -r ".NextToken"
}

while [ "$NEXT_TOKEN" != "null" ]; do
  NEXT_TOKEN=$(CLI_call $NEXT_TOKEN)
done

# Not 100% necessary but can be used to cleanup the output from the multiple API calls into a neater JSON formatted output.
cat $OUTPUT_FILE | jq . > $OUTPUT_FILE-master.json
rm $OUTPUT_FILE

出力結果

2ページ目には結果がなかったため､ "ReservedInstanceOfferings": []のように空になっていました｡

{
  "NextToken": "AAIAAek8tAQdrbKJF2I-kotFAREr6THhOIxirDdAouqCiCfERYHxEu1AyjQj2q6yxsiAJqCCdHUGJqc=",
  "ReservedInstanceOfferings": [
    {
      "ReservedInstanceOfferingId": "006325ce-5966-42b1-b5fd-1ced436bf90c",
      "InstanceType": "r5.xlarge.search",
      "Duration": 31536000,
      "FixedPrice": 0,
      "UsagePrice": 0,
      "CurrencyCode": "USD",
      "PaymentOption": "NO_UPFRONT",
      "RecurringCharges": [
        {
          "RecurringChargeAmount": 0.309,
          "RecurringChargeFrequency": "Hourly"
        }
      ]
    },
    {
      "ReservedInstanceOfferingId": "16ab40af-9838-43fd-92c9-b5f9d623f16c",
      "InstanceType": "r5.xlarge.search",
      "Duration": 31536000,
      "FixedPrice": 2551,
      "UsagePrice": 0,
      "CurrencyCode": "USD",
      "PaymentOption": "ALL_UPFRONT",
      "RecurringCharges": [
        {
          "RecurringChargeAmount": 0,
          "RecurringChargeFrequency": "Hourly"
        }
      ]
    }
  ]
}
{
  "NextToken": "AAIAATBuTCZwN9gEO8iALwBxYYmmld9-ohdFNRtebGJCwyO5u8RWQI5kcCHQcF8v-IHr2UygjCHlrX8=",
  "ReservedInstanceOfferings": []
}
{
  "NextToken": "AAIAAR1WT4qYAwKZVWwNmhwKiKWNCCc5x_Qkj6yN5KVpkIxqBZZFMc1er1zPWKKGkzxQ1tptqXbPZCw=",
  "ReservedInstanceOfferings": [
    {
      "ReservedInstanceOfferingId": "8f519035-781a-430a-9623-5290ed341ff1",
      "InstanceType": "r5.xlarge.search",
      "Duration": 94608000,
      "FixedPrice": 5651,
      "UsagePrice": 0,
      "CurrencyCode": "USD",
      "PaymentOption": "ALL_UPFRONT",
      "RecurringCharges": [
        {
          "RecurringChargeAmount": 0,
          "RecurringChargeFrequency": "Hourly"
        }
      ]
    },
    {
      "ReservedInstanceOfferingId": "b8d201c5-a1e9-45f1-9597-e7e764497a69",
      "InstanceType": "r5.xlarge.search",
      "Duration": 94608000,
      "FixedPrice": 2943,
      "UsagePrice": 0,
      "CurrencyCode": "USD",
      "PaymentOption": "PARTIAL_UPFRONT",
      "RecurringCharges": [
        {
          "RecurringChargeAmount": 0.112,
          "RecurringChargeFrequency": "Hourly"
        }
      ]
    }
  ]
}
{
  "NextToken": null,
  "ReservedInstanceOfferings": [
    {
      "ReservedInstanceOfferingId": "d2fd55c4-ddb2-48a3-8aa9-b95b5f3c5065",
      "InstanceType": "r5.xlarge.search",
      "Duration": 31536000,
      "FixedPrice": 1315,
      "UsagePrice": 0,
      "CurrencyCode": "USD",
      "PaymentOption": "PARTIAL_UPFRONT",
      "RecurringCharges": [
        {
          "RecurringChargeAmount": 0.15,
          "RecurringChargeFrequency": "Hourly"
        }
      ]
    },
    {
      "ReservedInstanceOfferingId": "e39674cf-4804-498e-8bd8-b673e1753213",
      "InstanceType": "r5.xlarge.search",
      "Duration": 94608000,
      "FixedPrice": 0,
      "UsagePrice": 0,
      "CurrencyCode": "USD",
      "PaymentOption": "NO_UPFRONT",
      "RecurringCharges": [
        {
          "RecurringChargeAmount": 0.233,
          "RecurringChargeFrequency": "Hourly"
        }
      ]
    }
  ]
}

条件を複数個指定

条件に､InstanceType==r5.xlarge.search&&Duration==31536000 を指定します｡ && の前後はスペースを空けないように｡スペースがあるとエラーとなります｡

#!/bin/bash -xe

# Below command can be replaced to the required CLI, including with custom JSON output, assuming the NextToken is in the same location.
AWS_CLI_COMMAND="aws opensearch describe-reserved-instance-offerings --max-results 100 --query={NextToken:NextToken,ReservedInstanceOfferings:ReservedInstanceOfferings[?InstanceType==\`r5.xlarge.search\`&&Duration==\`31536000\`]}"
OUTPUT_FILE="./output-$(date +%s)"

function CLI_call() {
  if [ -z $NEXT_TOKEN ]; then
    cli_output=$($AWS_CLI_COMMAND)
  else
    cli_output=$($AWS_CLI_COMMAND --next-token $NEXT_TOKEN)
  fi

  echo $cli_output >> $OUTPUT_FILE
  echo $cli_output | jq -r ".NextToken"
}

while [ "$NEXT_TOKEN" != "null" ]; do
  NEXT_TOKEN=$(CLI_call $NEXT_TOKEN)
done

# Not 100% necessary but can be used to cleanup the output from the multiple API calls into a neater JSON formatted output.
cat $OUTPUT_FILE | jq . > $OUTPUT_FILE-master.json
rm $OUTPUT_FILE

出力結果

PaymentOptionごとに3件出力されました｡

{
  "NextToken": "AAIAAUXaqWoOUNFsfGUDQD_-TNdm-kafqGiIKyCcMR83v2eufDvbgGTBMmqNHwNdY68soC7L5cYfMHI=",
  "ReservedInstanceOfferings": [
    {
      "ReservedInstanceOfferingId": "006325ce-5966-42b1-b5fd-1ced436bf90c",
      "InstanceType": "r5.xlarge.search",
      "Duration": 31536000,
      "FixedPrice": 0,
      "UsagePrice": 0,
      "CurrencyCode": "USD",
      "PaymentOption": "NO_UPFRONT",
      "RecurringCharges": [
        {
          "RecurringChargeAmount": 0.309,
          "RecurringChargeFrequency": "Hourly"
        }
      ]
    },
    {
      "ReservedInstanceOfferingId": "16ab40af-9838-43fd-92c9-b5f9d623f16c",
      "InstanceType": "r5.xlarge.search",
      "Duration": 31536000,
      "FixedPrice": 2551,
      "UsagePrice": 0,
      "CurrencyCode": "USD",
      "PaymentOption": "ALL_UPFRONT",
      "RecurringCharges": [
        {
          "RecurringChargeAmount": 0,
          "RecurringChargeFrequency": "Hourly"
        }
      ]
    }
  ]
}
{
  "NextToken": "AAIAAWVyCelMgGN8pXv2zT7IRi7_ekuM7tSXk5BsLn8kjsgQ7F_jPe1aXJ5_VlD-uwec8ZdEgH5erdA=",
  "ReservedInstanceOfferings": []
}
{
  "NextToken": "AAIAAaVn4wXAkfVFpIl3rl6UbIVJWcpNoasEN3OwOJDwVV68j5pMg3ORmxxUHbWrfQt9Otcuk8smFOw=",
  "ReservedInstanceOfferings": []
}
{
  "NextToken": null,
  "ReservedInstanceOfferings": [
    {
      "ReservedInstanceOfferingId": "d2fd55c4-ddb2-48a3-8aa9-b95b5f3c5065",
      "InstanceType": "r5.xlarge.search",
      "Duration": 31536000,
      "FixedPrice": 1315,
      "UsagePrice": 0,
      "CurrencyCode": "USD",
      "PaymentOption": "PARTIAL_UPFRONT",
      "RecurringCharges": [
        {
          "RecurringChargeAmount": 0.15,
          "RecurringChargeFrequency": "Hourly"
        }
      ]
    }
  ]
}

条件を複数個指定し､テキスト形式で出力

条件に､InstanceType==r5.xlarge.search&&Duration==31536000を指定し､ReservedInstanceOfferingId のみをテキストで出力します｡

#!/bin/bash -xe

# Below command can be replaced to the required CLI, including with custom JSON output, assuming the NextToken is in the same location.
AWS_CLI_COMMAND="aws opensearch describe-reserved-instance-offerings --max-results 100 --query={NextToken:NextToken,ReservedInstanceOfferings:ReservedInstanceOfferings[?InstanceType==\`r5.xlarge.search\`&&Duration==\`31536000\`].ReservedInstanceOfferingId}"
OUTPUT_FILE="./output-$(date +%s)"

function CLI_call() {
  if [ -z $NEXT_TOKEN ]; then
    cli_output=$($AWS_CLI_COMMAND)
  else
    cli_output=$($AWS_CLI_COMMAND --next-token $NEXT_TOKEN)
  fi

  echo $cli_output >> $OUTPUT_FILE
  echo $cli_output | jq -r ".NextToken"
}

while [ "$NEXT_TOKEN" != "null" ]; do
  NEXT_TOKEN=$(CLI_call $NEXT_TOKEN)
done

# Not 100% necessary but can be used to cleanup the output from the multiple API calls into a neater JSON formatted output.
cat $OUTPUT_FILE | jq -r .ReservedInstanceOfferings[] > $OUTPUT_FILE-master.txt
rm $OUTPUT_FILE

出力結果

006325ce-5966-42b1-b5fd-1ced436bf90c
16ab40af-9838-43fd-92c9-b5f9d623f16c
d2fd55c4-ddb2-48a3-8aa9-b95b5f3c5065

メモ

最終出力は以下のようにページ毎のJSONをひとつのファイルに結合した形式となっているため､このままではJSON構文として正しくありません｡jqコマンドで必要な形式に成形し出力すると使い勝手が良くなると思います｡

{
  "NextToken": "AAI...",
  "ReservedInstanceOfferings": [
    {
      "ReservedInstanceOfferingId": "006325ce-5966-42b1-b5fd-1ced436bf90c",
      "InstanceType": "r5.xlarge.search",
      "Duration": 31536000,
    ...
    }
   ]
}
{
  "NextToken": "AAI...",
  "ReservedInstanceOfferings": [
    ...
    }
  ]
}
{
  "NextToken": "AAI...",
  "ReservedInstanceOfferings": [
    ...
    }
  ]
}