Sysdig Monitoring API (python)

IBM Cloud Monitoring with Sysdigサービスは、管理者、DevOpsチーム、開発者向けの、フル管理のエンタープライズ・グレードのモニタリング・サービスです。コンテナの深い可視性、サービス指向のビュー、包括的なメトリックが得られます。

Sysdig UI の Dashboard や Panel を利用して Metric 情報を確認することが一般的ですが、Monitoring API を使用することで、JSON Format で Metric Data を取得することが可能です。
Panel の Export 機能でも Metric Data を取得することができますが、Monitoring API は、より詳細な設定が可能です。

Monitoring API は、curl または python で利用することができます。

Extracting metrics from a Monitoring instance by using the Monitoring API
https://cloud.ibm.com/docs/monitoring?topic=monitoring-metrics_api
Extracting metrics from an instance by using the Monitoring Python client
https://cloud.ibm.com/docs/monitoring?topic=monitoring-metrics_python

ここでは、python による Sysdig Monitoring API の利用方法をご紹介します。
curl については、以下をご覧下さい。
Sysdig Monitoring API (curl)
https://qiita.com/qiita2021user/items/4568cd9f8210e62ac3f1

Install required modules

以下の手順に従って、必要な Module を導入します。
Using the Python client
https://cloud.ibm.com/docs/monitoring?topic=monitoring-python-client

Code Basic

基本的な python code は以下の通りです。

# Reference the Python client
from sdcclient import IbmAuthHelper, SdMonitorClient

# Add the monitoring instance information that is required for authentication
URL = <MONITORING-ENDPOINT>
APIKEY = <IAM_APIKEY>
GUID = <GUID>
ibm_headers = IbmAuthHelper.get_headers(URL, APIKEY, GUID)

# Instantiate the Python client
sdclient = SdMonitorClient(sdc_url=URL, custom_headers=ibm_headers)

# Specify the ID for keys, and ID with aggregation for values
metrics = [
    {"id": "cpu.used.percent", "aggregations": {"time": "timeAvg", "group": "avg"}}
]

# Add a data filter or set to None if you want to see "everything"
filter = None

# Time window:
#   - for "from A to B": start is equal to A, end is equal to B (expressed in seconds)
#   - for "last X seconds": start is equal to -X, end is equal to 0
start = -600
end = 0

# Sampling time:
#   - for time series: sampling is equal to the "width" of each data point (expressed in seconds)
#   - for aggregated data (similar to bar charts, pie charts, tables, etc.): sampling is equal to 0
sampling = 60

# Load data
ok, res = sdclient.get_data(metrics, start, end, sampling, filter=filter)
if ok:
    print(res)

IbmAuthHelper, SdMonitorClient

IbmAuthHelper, SdMonitorClient の Help は以下の通りです。

>>> from sdcclient import IbmAuthHelper, SdMonitorClient

>>> help(IbmAuthHelper)
Help on class IbmAuthHelper in module sdcclient.ibm_auth_helper:

class IbmAuthHelper(builtins.object)
 |  Authenticate with IBM Cloud IAM.
 |
 |  **Arguments**
 |      **url**: Sysdig endpoint URL that should point to IBM Cloud
 |      **apikey**: IBM Cloud IAM apikey that will be used to retrieve an access token
 |      **guid**: GUID of an IBM Cloud Monitoring with Sysdig instance
 |
 |  **Returns**
 |      A dictionary that will authenticate you with the IBM Cloud IAM API.
 |
 |  Static methods defined here:
 |
 |  get_headers(url, apikey, guid)
 |
 |  ----------------------------------------------------------------------
 |  Data descriptors defined here:
 |
 |  __dict__
 |      dictionary for instance variables (if defined)
 |
 |  __weakref__
 |      list of weak references to the object (if defined)

>>> help(SdMonitorClient)
Help on class SdMonitorClient in module sdcclient._monitor:

class SdMonitorClient(sdcclient.monitor._dashboards_v3.DashboardsClientV3, sdcclient.monitor._events_v2.EventsClientV2, sdcclient._common._SdcCommon)
 |  SdMonitorClient(token='', sdc_url='https://app.sysdigcloud.com', ssl_verify=True, custom_headers=None)
 |
 |  Method resolution order:
 |      SdMonitorClient
 |      sdcclient.monitor._dashboards_v3.DashboardsClientV3
 |      sdcclient.monitor._events_v2.EventsClientV2
 |      sdcclient._common._SdcCommon
 |      builtins.object
 |
～省略～

URL

Sysdig の Endpoint を指定します。Endpoint は以下から確認できます。
Endpoints
https://cloud.ibm.com/docs/monitoring?topic=monitoring-endpoints#endpoints_monitoring

ここでは、以下の Endpoint を使用します。

URL = "https://jp-tok.monitoring.cloud.ibm.com"

APIKEY

Sysdig Instance へのアクセスに必要な API Key を指定します。
Authenticate your user or service ID by using IAM
https://cloud.ibm.com/docs/monitoring?topic=monitoring-python-client#python-client-iam-auth

APIKEY = "<API Key>"

参考

以下のようなエラーが発生する場合は、API Key が正しくない可能性があります。

Traceback (most recent call last):
  File "./sample.py", line 14, in <module>
    ibm_headers = IbmAuthHelper.get_headers(URL, APIKEY, GUID)
  File "/usr/local/lib/python3.9/site-packages/sdcclient/ibm_auth_helper.py", line 18, in get_headers
    iam_token = IbmAuthHelper.__get_iam_token(url, apikey)
  File "/usr/local/lib/python3.9/site-packages/sdcclient/ibm_auth_helper.py", line 51, in __get_iam_token
    response.raise_for_status()
  File "/root/.local/lib/python3.9/site-packages/requests/models.py", line 1021, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: https://iam.cloud.ibm.com/identity/token

Note: If you get the error 400 Client Error: Bad Request for url: https://iam.cloud.ibm.com/identity/token, check the API key. The value that you are passing is not valid.

GUID

Sysdig Instance の GUID を指定します。GUID は以下から確認できます。

$ ibmcloud monitoring service-instances --all-resource-groups
Listing service instances for jp-tok region and for all resource groups...
OK
Name           Region   State    Service Name     Platform Metrics
my-sysdig-01   jp-tok   active   sysdig-monitor   Enabled

$ ibmcloud resource service-instance my-sysdig-01 --output json | jq -r '.[].guid'
****-****-****-****-****

GUID = "****-****-****-****-****"

Sample Code

上記の情報に加えて、いくつかの修正を行った Sample Code です。

$ cat sample.py
#!/usr/bin/env python3.9

from sdcclient import IbmAuthHelper, SdMonitorClient

# Add the monitoring instance information that is required for authentication
# URL = <MONITORING-ENDPOINT>
URL = "https://jp-tok.monitoring.cloud.ibm.com"

# APIKEY = <IAM_APIKEY>
APIKEY = "****-****-****-****"

# GUID = <GUID>
GUID = "****-****-****-****-****"
ibm_headers = IbmAuthHelper.get_headers(URL, APIKEY, GUID)

# Instantiate the Python client
sdclient = SdMonitorClient(sdc_url=URL, custom_headers=ibm_headers)

# Specify the ID for keys, and ID with aggregation for values
metrics = [
    {"id": "cpu.used.percent", "aggregations": {"time": "timeAvg", "group": "avg"}}
]

# Add a data filter or set to None if you want to see "everything"
filter = None

# Time window:
#   - for "from A to B": start is equal to A, end is equal to B (expressed in seconds)
#   - for "last X seconds": start is equal to -X, end is equal to 0
start = -600
end = 0

# Sampling time:
#   - for time series: sampling is equal to the "width" of each data point (expressed in seconds)
#   - for aggregated data (similar to bar charts, pie charts, tables, etc.): sampling is equal to 0
sampling = 60

# Load data
ok, res = sdclient.get_data(metrics, start, end, sampling, filter=filter)
if ok:
    import json
    print(json.dumps(res))

実行すると、以下のように Metric が取得できます。

$ python3.9 ./sample.py | jq -r .
{
  "data": [
    {
      "t": 1693278720,
      "d": [
        41.230233
      ]
    },
    {
      "t": 1693278780,
      "d": [
        41.604561
      ]
    },
    {
      "t": 1693278840,
      "d": [
        40.772554
      ]
    },
    {
      "t": 1693278900,
      "d": [
        41.400421
      ]
    },
    {
      "t": 1693278960,
      "d": [
        41.202891
      ]
    },
    {
      "t": 1693279020,
      "d": [
        40.846263
      ]
    },
    {
      "t": 1693279080,
      "d": [
        40.325589
      ]
    },
    {
      "t": 1693279140,
      "d": [
        42.116796
      ]
    },
    {
      "t": 1693279200,
      "d": [
        43.555129
      ]
    },
    {
      "t": 1693279260,
      "d": [
        42.547032
      ]
    }
  ],
  "start": 1693278660,
  "end": 1693279260
}