Cloudflare GraphQL データセットの保管期間

以前はこのような保管期間がドキュメントで公開されていましたが、現時点ではドキュメントで確認できないため、自身で該当データを取得し、確認する必要があります。

前提条件

HTTPie を Cloudflare API / GraphQL 用に設定する - Qiita

ゾーン

変数をセットします。

export GQL_ENDPOINT='https://api.cloudflare.com/client/v4/graphql'
export ZONE_ID='YOUR_ZONE_ID' # 今回は Enterprise Plan のゾーン情報を用います

ゾーンデータセット一覧を変数にセットします。

export GQL_ENDPOINT='https://api.cloudflare.com/client/v4/graphql'

export ZONE_DATASET=$(http -A creds $GQL_ENDPOINT query='query
{
  __type(name: "zone") {
    name
    fields {
      name
      description,
      isDeprecated,
      deprecationReason
    }
  }
}
' | jq -r '.data.__type.fields[]|select(.isDeprecated == false)|.name')

echo $ZONE_DATASET

データセットを読み込みながら、それぞれの保管期間を取得します。

echo '"dataset","enabled","maxDuration","notOlderThan"' > zone_dataset_settings.csv

echo $ZONE_DATASET | while read row; do \
case ${row} in "settings" ) sleep 1 ;; "zoneTag" ) sleep 1 ;; \
* ) http -A creds --ignore-stdin $GQL_ENDPOINT query='query
{
  viewer {
    zones(filter: { zoneTag: "'${ZONE_ID}'" }) {
      settings {
        '${row}' {
          enabled
          maxDuration
          notOlderThan
        }
      }
    }
  }
}
' | jq -r '.data.viewer.zones[].settings.'${row}' |select(.enabled == true)| ["'${row}'",.enabled,(.maxDuration/60/60/24|floor|tostring)+" day "+(.maxDuration/60/60%24|tostring)+" hour",(.notOlderThan/60/60/24|floor|tostring)+" day "+(.notOlderThan/60/60%24|tostring)+" hour"]|@csv' >> zone_dataset_settings.csv ;; esac done

csv2md zone_dataset_settings.csv > zone_dataset_settings.md
cat zone_dataset_settings.md | pbcopy

CSV から Markdown に変換したものが以下の内容です。

最大で notOlderThan の時間を遡ることができ、一度に maxDuration の範囲でデータ取得できます。

dataset	enabled	maxDuration	notOlderThan
apiRequestSequencesGroups	true	1 day 0 hour	31 day 0 hour
cacheReserveOperationsAdaptiveGroups	true	30 day 0 hour	90 day 0 hour
cacheReserveStorageAdaptiveGroups	true	30 day 0 hour	90 day 0 hour
emailRoutingAdaptive	true	1 day 0 hour	31 day 0 hour
emailRoutingAdaptiveGroups	true	31 day 0 hour	31 day 0 hour
firewallEventsAdaptive	true	31 day 0 hour	31 day 0 hour
firewallEventsAdaptiveByTimeGroups	true	31 day 0 hour	31 day 0 hour
firewallEventsAdaptiveGroups	true	31 day 0 hour	31 day 0 hour
healthCheckEventsAdaptive	true	7 day 0 hour	90 day 1 hour
healthCheckEventsAdaptiveGroups	true	7 day 0 hour	90 day 1 hour
httpRequests1dGroups	true	365 day 1 hour	365 day 1 hour
httpRequests1hGroups	true	3 day 0 hour	90 day 1 hour
httpRequests1mGroups	true	0 day 15 hour	7 day 1 hour
httpRequestsAdaptive	true	31 day 0 hour	90 day 0 hour
httpRequestsAdaptiveGroups	true	31 day 0 hour	90 day 0 hour
httpRequestsOverviewAdaptiveGroups	true	31 day 0 hour	90 day 0 hour
imageResizingRequests1mGroups	true	30 day 1 hour	365 day 0 hour
loadBalancingRequestsAdaptive	true	7 day 0 hour	30 day 1 hour
loadBalancingRequestsAdaptiveGroups	true	7 day 0 hour	30 day 1 hour
logpushHealthAdaptiveGroups	true	30 day 0 hour	30 day 1 hour
nelReportsAdaptiveGroups	true	93 day 0 hour	93 day 0 hour
synAvgPps1mGroups	true	7 day 0 hour	7 day 1 hour
waitingRoomAnalyticsAdaptive	true	30 day 0 hour	90 day 0 hour
waitingRoomAnalyticsAdaptiveGroups	true	30 day 0 hour	90 day 0 hour
workersZoneInvocationsAdaptiveGroups	true	33 day 8 hour	33 day 8 hour
workersZoneSubrequestsAdaptiveGroups	true	33 day 8 hour	33 day 8 hour
zarazActionsAdaptiveGroups	true	31 day 0 hour	31 day 0 hour
zarazTrackAdaptiveGroups	true	31 day 0 hour	31 day 0 hour
zarazTriggersAdaptiveGroups	true	31 day 0 hour	31 day 0 hour

アカウント

変数をセットします。

export GQL_ENDPOINT='https://api.cloudflare.com/client/v4/graphql'
export ACCOUNT_ID='YOUR_ZONE_ID' # 今回は Enterprise Account のアカウント情報を用います

アカウントデータセット一覧を変数にセットします。

export GQL_ENDPOINT='https://api.cloudflare.com/client/v4/graphql'

export ACCOUNT_DATASET=$(http -A creds $GQL_ENDPOINT query='query
{
  __type(name: "account") {
    name
    fields {
      name
      description,
      isDeprecated,
      deprecationReason
    }
  }
}
' | jq -r '.data.__type.fields[]|select(.isDeprecated == false)|.name')

echo $ACCOUNT_DATASET

データセットを読み込みながら、それぞれの保管期間を取得します。

echo '"dataset","enabled","maxDuration","notOlderThan"' > account_dataset_settings.csv

echo $ACCOUNT_DATASET | while read row; do \
case ${row} in "settings" ) sleep 1 ;; "accountTag" ) sleep 1 ;; "zones" ) sleep 1 ;; \
* ) http -A creds --ignore-stdin $GQL_ENDPOINT query='query
{
  viewer {
    accounts(filter: { accountTag: "'${ACCOUNT_ID}'" }) {
      settings {
        '${row}' {
          enabled
          maxDuration
          notOlderThan
        }
      }
    }
  }
}
' | jq -r '.data.viewer.accounts[].settings.'${row}' |select(.enabled == true)| ["'${row}'",.enabled,(.maxDuration/60/60/24|floor|tostring)+" day "+(.maxDuration/60/60%24|tostring)+" hour",(.notOlderThan/60/60/24|floor|tostring)+" day "+(.notOlderThan/60/60%24|tostring)+" hour"]|@csv' >> account_dataset_settings.csv ;; esac done

csv2md account_dataset_settings.csv > account_dataset_settings.md
cat account_dataset_settings.md | pbcopy

CSV から Markdown に変換したものが以下の内容です。

最大で notOlderThan の時間を遡ることができ、一度に maxDuration の範囲でデータ取得できます。

dataset	enabled	maxDuration	notOlderThan
accessLoginRequestsAdaptiveGroups	true	1 day 0 hour	31 day 0 hour
cdnNetworkAnalyticsAdaptiveGroups	true	30 day 0 hour	112 day 0 hour
dosdAttackAnalyticsGroups	true	30 day 0 hour	112 day 0 hour
dosdNetworkAnalyticsAdaptiveGroups	true	30 day 0 hour	112 day 0 hour
durableObjectsInvocationsAdaptiveGroups	true	30 day 0 hour	90 day 0 hour
durableObjectsPeriodicGroups	true	30 day 0 hour	90 day 0 hour
durableObjectsStorageGroups	true	30 day 0 hour	90 day 0 hour
durableObjectsSubrequestsAdaptiveGroups	true	30 day 0 hour	90 day 0 hour
fbmAttackAnalyticsGroups	true	30 day 0 hour	112 day 0 hour
firewallEventsAdaptive	true	31 day 0 hour	31 day 0 hour
firewallEventsAdaptiveGroups	true	31 day 0 hour	31 day 0 hour
flowtrackdNetworkAnalyticsAdaptiveGroups	true	30 day 0 hour	112 day 0 hour
gatewayL4DownstreamSessionsAdaptiveGroups	true	7 day 0 hour	365 day 0 hour
gatewayL4SessionsAdaptiveGroups	true	30 day 0 hour	30 day 0 hour
gatewayL4UpstreamSessionsAdaptiveGroups	true	7 day 0 hour	365 day 0 hour
gatewayL7RequestsAdaptiveGroups	true	30 day 0 hour	30 day 0 hour
gatewayResolverByCategoryAdaptiveGroups	true	30 day 0 hour	30 day 0 hour
gatewayResolverQueriesAdaptiveGroups	true	30 day 0 hour	30 day 0 hour
httpRequests1dGroups	true	365 day 1 hour	365 day 1 hour
httpRequests1hGroups	true	3 day 0 hour	3 day 1 hour
httpRequestsAdaptive	true	31 day 0 hour	91 day 0 hour
httpRequestsAdaptiveGroups	true	31 day 0 hour	91 day 0 hour
httpRequestsOverviewAdaptiveGroups	true	31 day 0 hour	91 day 0 hour
imagesRequestsAdaptiveGroups	true	31 day 0 hour	31 day 0 hour
ipFlows1dGroups	true	180 day 0 hour	366 day 0 hour
ipFlows1hGroups	true	3 day 1 hour	180 day 0 hour
ipFlows1mAttacksGroups	true	180 day 0 hour	366 day 0 hour
ipFlows1mGroups	true	1 day 1 hour	30 day 0 hour
logpushHealthAdaptiveGroups	true	30 day 0 hour	30 day 1 hour
magicFirewallNetworkAnalyticsAdaptiveGroups	true	30 day 0 hour	112 day 0 hour
magicFirewallSamplesAdaptiveGroups	true	30 day 0 hour	112 day 0 hour
magicTransitNetworkAnalyticsAdaptiveGroups	true	30 day 0 hour	112 day 0 hour
magicTransitTunnelHealthChecksAdaptiveGroups	true	31 day 0 hour	92 day 0 hour
magicTransitTunnelTrafficAdaptiveGroups	true	30 day 0 hour	112 day 0 hour
mnmFlowDataAdaptiveGroups	true	30 day 0 hour	90 day 0 hour
nelReportsAdaptiveGroups	true	93 day 0 hour	93 day 0 hour
pagesFunctionsInvocationsAdaptiveGroups	true	30 day 0 hour	90 day 0 hour
queueBacklogAdaptiveGroups	true	30 day 0 hour	90 day 0 hour
queueMessageOperationsAdaptiveGroups	true	30 day 0 hour	90 day 0 hour
r2OperationsAdaptiveGroups	true	30 day 0 hour	90 day 0 hour
r2StorageAdaptiveGroups	true	30 day 0 hour	90 day 0 hour
rumPageloadEventsAdaptiveGroups	true	93 day 0 hour	184 day 0 hour
rumPerformanceEventsAdaptiveGroups	true	93 day 0 hour	184 day 0 hour
rumWebVitalsEventsAdaptive	true	93 day 0 hour	184 day 0 hour
rumWebVitalsEventsAdaptiveGroups	true	93 day 0 hour	184 day 0 hour
spectrumNetworkAnalyticsAdaptiveGroups	true	30 day 0 hour	112 day 0 hour
streamMinutesViewedAdaptiveGroups	true	31 day 0 hour	92 day 0 hour
turnstileAdaptiveGroups	true	7 day 1 hour	30 day 0 hour
videoBufferEventsAdaptiveGroups	true	31 day 0 hour	92 day 0 hour
videoPlaybackEventsAdaptiveGroups	true	31 day 0 hour	92 day 0 hour
videoQualityEventsAdaptiveGroups	true	31 day 0 hour	92 day 0 hour
workersAnalyticsEngineAdaptiveGroups	true	31 day 0 hour	31 day 0 hour
workersInvocationsAdaptive	true	30 day 0 hour	90 day 0 hour
workersInvocationsScheduled	true	7 day 0 hour	90 day 0 hour
workersSubrequestsAdaptiveGroups	true	30 day 0 hour	90 day 0 hour
zeroTrustPrivateNetworkDiscoveryGroups	true	30 day 0 hour	30 day 0 hour

ダッシュボード Analytics 画面のマッピング

ゾーン

dataset	maxDuration	notOlderThan	dashboard
apiRequestSequencesGroups	1 day 0 hour	31 day 0 hour	N/A
cacheReserveOperationsAdaptiveGroups	30 day 0 hour	90 day 0 hour	https://dash.cloudflare.com/?to=/:account/:zone/caching/cache-reserve
cacheReserveStorageAdaptiveGroups	30 day 0 hour	90 day 0 hour	https://dash.cloudflare.com/?to=/:account/:zone/caching/cache-reserve
emailRoutingAdaptive	1 day 0 hour	31 day 0 hour	https://dash.cloudflare.com/?to=/:account/:zone/email/routing/overview
emailRoutingAdaptiveGroups	31 day 0 hour	31 day 0 hour	https://dash.cloudflare.com/?to=/:account/:zone/email/routing/overview
firewallEventsAdaptive	31 day 0 hour	31 day 0 hour	https://dash.cloudflare.com/?to=/:account/:zone/security/events
firewallEventsAdaptiveByTimeGroups	31 day 0 hour	31 day 0 hour	https://dash.cloudflare.com/?to=/:account/:zone/security/events
firewallEventsAdaptiveGroups	31 day 0 hour	31 day 0 hour	https://dash.cloudflare.com/?to=/:account/:zone/security/events
healthCheckEventsAdaptive	7 day 0 hour	90 day 1 hour	https://dash.cloudflare.com/?to=/:account/:zone/traffic/health-check-analytics
healthCheckEventsAdaptiveGroups	7 day 0 hour	90 day 1 hour	https://dash.cloudflare.com/?to=/:account/:zone/traffic/health-check-analytics
httpRequests1dGroups	365 day 1 hour	365 day 1 hour	https://dash.cloudflare.com/?to=/:account/:zone/
httpRequests1hGroups	3 day 0 hour	90 day 1 hour	https://dash.cloudflare.com/?to=/:account/:zone/
httpRequests1mGroups	0 day 15 hour	7 day 1 hour	https://dash.cloudflare.com/?to=/:account/:zone/
httpRequestsAdaptive	31 day 0 hour	90 day 0 hour	https://dash.cloudflare.com/?to=/:account/:zone/analytics/traffic
httpRequestsAdaptiveGroups	31 day 0 hour	90 day 0 hour	https://dash.cloudflare.com/?to=/:account/:zone/analytics/traffic
httpRequestsOverviewAdaptiveGroups	31 day 0 hour	90 day 0 hour	https://dash.cloudflare.com/?to=/:account/:zone/analytics/traffic
imageResizingRequests1mGroups	30 day 1 hour	365 day 0 hour	https://dash.cloudflare.com/?to=/:account/:zone/speed/optimization
loadBalancingRequestsAdaptive	7 day 0 hour	30 day 1 hour	https://dash.cloudflare.com/?to=/:account/:zone/traffic/load-balancing-analytics
loadBalancingRequestsAdaptiveGroups	7 day 0 hour	30 day 1 hour	https://dash.cloudflare.com/?to=/:account/:zone/traffic/load-balancing-analytics
logpushHealthAdaptiveGroups	30 day 0 hour	30 day 1 hour	N/A
nelReportsAdaptiveGroups	93 day 0 hour	93 day 0 hour	https://dash.cloudflare.com/?to=/:account/:zone/analytics/edgeReachability
synAvgPps1mGroups	7 day 0 hour	7 day 1 hour	N/A
waitingRoomAnalyticsAdaptive	30 day 0 hour	90 day 0 hour	https://dash.cloudflare.com/?to=/:account/:zone/traffic/waiting-rooms
waitingRoomAnalyticsAdaptiveGroups	30 day 0 hour	90 day 0 hour	https://dash.cloudflare.com/?to=/:account/:zone/traffic/waiting-rooms
workersZoneInvocationsAdaptiveGroups	33 day 8 hour	33 day 8 hour	https://dash.cloudflare.com/?to=/:account/:zone/analytics/workers
workersZoneSubrequestsAdaptiveGroups	33 day 8 hour	33 day 8 hour	https://dash.cloudflare.com/?to=/:account/:zone/analytics/workers
zarazActionsAdaptiveGroups	31 day 0 hour	31 day 0 hour	N/A
zarazTrackAdaptiveGroups	31 day 0 hour	31 day 0 hour	N/A
zarazTriggersAdaptiveGroups	31 day 0 hour	31 day 0 hour	N/A

アカウント

dataset	maxDuration	notOlderThan	dashboard
accessLoginRequestsAdaptiveGroups	1 day 0 hour	31 day 0 hour	N/A
cdnNetworkAnalyticsAdaptiveGroups	30 day 0 hour	112 day 0 hour	N/A
dosdAttackAnalyticsGroups	30 day 0 hour	112 day 0 hour	N/A
dosdNetworkAnalyticsAdaptiveGroups	30 day 0 hour	112 day 0 hour	https://dash.cloudflare.com/?to=/:account/network-analytics/ddos-managed-rules
durableObjectsInvocationsAdaptiveGroups	30 day 0 hour	90 day 0 hour	https://dash.cloudflare.com/?to=/:account/workers
durableObjectsPeriodicGroups	30 day 0 hour	90 day 0 hour	https://dash.cloudflare.com/?to=/:account/workers
durableObjectsStorageGroups	30 day 0 hour	90 day 0 hour	https://dash.cloudflare.com/?to=/:account/workers
durableObjectsSubrequestsAdaptiveGroups	30 day 0 hour	90 day 0 hour	https://dash.cloudflare.com/?to=/:account/workers
fbmAttackAnalyticsGroups	30 day 0 hour	112 day 0 hour	N/A
firewallEventsAdaptive	31 day 0 hour	31 day 0 hour	N/A
firewallEventsAdaptiveGroups	31 day 0 hour	31 day 0 hour	N/A
flowtrackdNetworkAnalyticsAdaptiveGroups	30 day 0 hour	112 day 0 hour	https://dash.cloudflare.com/?to=/:account/network-analytics/advanced-tcp-protection
gatewayL4DownstreamSessionsAdaptiveGroups	7 day 0 hour	365 day 0 hour	N/A
gatewayL4SessionsAdaptiveGroups	30 day 0 hour	30 day 0 hour	N/A
gatewayL4UpstreamSessionsAdaptiveGroups	7 day 0 hour	365 day 0 hour	N/A
gatewayL7RequestsAdaptiveGroups	30 day 0 hour	30 day 0 hour	N/A
gatewayResolverByCategoryAdaptiveGroups	30 day 0 hour	30 day 0 hour	N/A
gatewayResolverQueriesAdaptiveGroups	30 day 0 hour	30 day 0 hour	N/A
httpRequests1dGroups	365 day 1 hour	365 day 1 hour	https://dash.cloudflare.com/?to=/:account/analytics
httpRequests1hGroups	3 day 0 hour	3 day 1 hour	https://dash.cloudflare.com/?to=/:account/analytics
httpRequestsAdaptive	31 day 0 hour	91 day 0 hour	https://dash.cloudflare.com/?to=/:account/analytics
httpRequestsAdaptiveGroups	31 day 0 hour	91 day 0 hour	https://dash.cloudflare.com/?to=/:account/analytics
httpRequestsOverviewAdaptiveGroups	31 day 0 hour	91 day 0 hour	https://dash.cloudflare.com/?to=/:account/analytics
imagesRequestsAdaptiveGroups	31 day 0 hour	31 day 0 hour	https://dash.cloudflare.com/?to=/:account//images
ipFlows1dGroups	180 day 0 hour	366 day 0 hour	N/A
ipFlows1hGroups	3 day 1 hour	180 day 0 hour	N/A
ipFlows1mAttacksGroups	180 day 0 hour	366 day 0 hour	N/A
ipFlows1mGroups	1 day 1 hour	30 day 0 hour	N/A
logpushHealthAdaptiveGroups	30 day 0 hour	30 day 1 hour	N/A
magicFirewallNetworkAnalyticsAdaptiveGroups	30 day 0 hour	112 day 0 hour	https://dash.cloudflare.com/?to=/:account/network-analytics/magic-firewall
magicFirewallSamplesAdaptiveGroups	30 day 0 hour	112 day 0 hour	https://dash.cloudflare.com/?to=/:account/network-security/magic_firewall
magicTransitNetworkAnalyticsAdaptiveGroups	30 day 0 hour	112 day 0 hour	https://dash.cloudflare.com/?to=/:account/network-analytics/all-traffic
magicTransitTunnelHealthChecksAdaptiveGroups	31 day 0 hour	92 day 0 hour	https://dash.cloudflare.com/?to=/:account/magic-transit/tunnel-healthchecks
magicTransitTunnelTrafficAdaptiveGroups	30 day 0 hour	112 day 0 hour	https://dash.cloudflare.com/?to=/:account/magic-transit/tunnel-healthchecks
mnmFlowDataAdaptiveGroups	30 day 0 hour	90 day 0 hour	https://dash.cloudflare.com/?to=/:account/network-monitoring
nelReportsAdaptiveGroups	93 day 0 hour	93 day 0 hour	N/A
pagesFunctionsInvocationsAdaptiveGroups	30 day 0 hour	90 day 0 hour	https://dash.cloudflare.com/?to=/:account/pages
queueBacklogAdaptiveGroups	30 day 0 hour	90 day 0 hour	https://dash.cloudflare.com/?to=/:account/workers/queues
queueMessageOperationsAdaptiveGroups	30 day 0 hour	90 day 0 hour	https://dash.cloudflare.com/?to=/:account/workers/queues
r2OperationsAdaptiveGroups	30 day 0 hour	90 day 0 hour	https://dash.cloudflare.com/?to=/:account/r2
r2StorageAdaptiveGroups	30 day 0 hour	90 day 0 hour	https://dash.cloudflare.com/?to=/:account/r2
rumPageloadEventsAdaptiveGroups	93 day 0 hour	184 day 0 hour	https://dash.cloudflare.com/?to=/:account/web-analytics/sites
rumPerformanceEventsAdaptiveGroups	93 day 0 hour	184 day 0 hour	https://dash.cloudflare.com/?to=/:account/web-analytics/sites
rumWebVitalsEventsAdaptive	93 day 0 hour	184 day 0 hour	https://dash.cloudflare.com/?to=/:account/web-analytics/sites
rumWebVitalsEventsAdaptiveGroups	93 day 0 hour	184 day 0 hour	https://dash.cloudflare.com/?to=/:account/web-analytics/sites
spectrumNetworkAnalyticsAdaptiveGroups	30 day 0 hour	112 day 0 hour	https://dash.cloudflare.com/?to=/:account/network-analytics/all-traffic
streamMinutesViewedAdaptiveGroups	31 day 0 hour	92 day 0 hour	https://dash.cloudflare.com/?to=/:account/stream/analytics
turnstileAdaptiveGroups	7 day 1 hour	30 day 0 hour	https://dash.cloudflare.com/?to=/:account/turnstile
videoBufferEventsAdaptiveGroups	31 day 0 hour	92 day 0 hour	N/A
videoPlaybackEventsAdaptiveGroups	31 day 0 hour	92 day 0 hour	N/A
videoQualityEventsAdaptiveGroups	31 day 0 hour	92 day 0 hour	N/A
workersAnalyticsEngineAdaptiveGroups	31 day 0 hour	31 day 0 hour	https://dash.cloudflare.com/?to=/:account/workers
workersInvocationsAdaptive	30 day 0 hour	90 day 0 hour	https://dash.cloudflare.com/?to=/:account/workers
workersInvocationsScheduled	7 day 0 hour	90 day 0 hour	https://dash.cloudflare.com/?to=/:account/workers
workersSubrequestsAdaptiveGroups	30 day 0 hour	90 day 0 hour	https://dash.cloudflare.com/?to=/:account/workers
zeroTrustPrivateNetworkDiscoveryGroups	30 day 0 hour	30 day 0 hour	https://dash.cloudflare.com/?to=/:account/analytics/access

参考