Go to Qiita Advent Calendar 2024 Top
LoginSignup
1
0

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?

More than 1 year has passed since last update.

CloudflareAdvent Calendar 2022

Day 2
@khayama(Kyouhei Hayama)inCloudflare Japan 株式会社

Cloudflare GraphQL データセットの保管期間を確認する

Last updated at Posted at 2023-02-06

Cloudflare GraphQL データセットの保管期間

以前はこのような保管期間がドキュメントで公開されていましたが、現時点ではドキュメントで確認できないため、自身で該当データを取得し、確認する必要があります。

image-20221014013347556

前提条件

ゾーン

変数をセットします。

export GQL_ENDPOINT='https://api.cloudflare.com/client/v4/graphql'
export ZONE_ID='YOUR_ZONE_ID' # 今回は Enterprise Plan のゾーン情報を用います

ゾーンデータセット一覧を変数にセットします。

export GQL_ENDPOINT='https://api.cloudflare.com/client/v4/graphql'

export ZONE_DATASET=$(http -A creds $GQL_ENDPOINT query='query
{
  __type(name: "zone") {
    name
    fields {
      name
      description,
      isDeprecated,
      deprecationReason
    }
  }
}
' | jq -r '.data.__type.fields[]|select(.isDeprecated == false)|.name')

echo $ZONE_DATASET

データセットを読み込みながら、それぞれの保管期間を取得します。

echo '"dataset","enabled","maxDuration","notOlderThan"' > zone_dataset_settings.csv

echo $ZONE_DATASET | while read row; do \
case ${row} in "settings" ) sleep 1 ;; "zoneTag" ) sleep 1 ;; \
* ) http -A creds --ignore-stdin $GQL_ENDPOINT query='query
{
  viewer {
    zones(filter: { zoneTag: "'${ZONE_ID}'" }) {
      settings {
        '${row}' {
          enabled
          maxDuration
          notOlderThan
        }
      }
    }
  }
}
' | jq -r '.data.viewer.zones[].settings.'${row}' |select(.enabled == true)| ["'${row}'",.enabled,(.maxDuration/60/60/24|floor|tostring)+" day "+(.maxDuration/60/60%24|tostring)+" hour",(.notOlderThan/60/60/24|floor|tostring)+" day "+(.notOlderThan/60/60%24|tostring)+" hour"]|@csv' >> zone_dataset_settings.csv ;; esac done

csv2md zone_dataset_settings.csv > zone_dataset_settings.md
cat zone_dataset_settings.md | pbcopy

CSV から Markdown に変換したものが以下の内容です。

最大で notOlderThan の時間を遡ることができ、一度に maxDuration の範囲でデータ取得できます。

dataset enabled maxDuration notOlderThan
apiRequestSequencesGroups true 1 day 0 hour 31 day 0 hour
cacheReserveOperationsAdaptiveGroups true 30 day 0 hour 90 day 0 hour
cacheReserveStorageAdaptiveGroups true 30 day 0 hour 90 day 0 hour
emailRoutingAdaptive true 1 day 0 hour 31 day 0 hour
emailRoutingAdaptiveGroups true 31 day 0 hour 31 day 0 hour
firewallEventsAdaptive true 31 day 0 hour 31 day 0 hour
firewallEventsAdaptiveByTimeGroups true 31 day 0 hour 31 day 0 hour
firewallEventsAdaptiveGroups true 31 day 0 hour 31 day 0 hour
healthCheckEventsAdaptive true 7 day 0 hour 90 day 1 hour
healthCheckEventsAdaptiveGroups true 7 day 0 hour 90 day 1 hour
httpRequests1dGroups true 365 day 1 hour 365 day 1 hour
httpRequests1hGroups true 3 day 0 hour 90 day 1 hour
httpRequests1mGroups true 0 day 15 hour 7 day 1 hour
httpRequestsAdaptive true 31 day 0 hour 90 day 0 hour
httpRequestsAdaptiveGroups true 31 day 0 hour 90 day 0 hour
httpRequestsOverviewAdaptiveGroups true 31 day 0 hour 90 day 0 hour
imageResizingRequests1mGroups true 30 day 1 hour 365 day 0 hour
loadBalancingRequestsAdaptive true 7 day 0 hour 30 day 1 hour
loadBalancingRequestsAdaptiveGroups true 7 day 0 hour 30 day 1 hour
logpushHealthAdaptiveGroups true 30 day 0 hour 30 day 1 hour
nelReportsAdaptiveGroups true 93 day 0 hour 93 day 0 hour
synAvgPps1mGroups true 7 day 0 hour 7 day 1 hour
waitingRoomAnalyticsAdaptive true 30 day 0 hour 90 day 0 hour
waitingRoomAnalyticsAdaptiveGroups true 30 day 0 hour 90 day 0 hour
workersZoneInvocationsAdaptiveGroups true 33 day 8 hour 33 day 8 hour
workersZoneSubrequestsAdaptiveGroups true 33 day 8 hour 33 day 8 hour
zarazActionsAdaptiveGroups true 31 day 0 hour 31 day 0 hour
zarazTrackAdaptiveGroups true 31 day 0 hour 31 day 0 hour
zarazTriggersAdaptiveGroups true 31 day 0 hour 31 day 0 hour

アカウント

変数をセットします。

export GQL_ENDPOINT='https://api.cloudflare.com/client/v4/graphql'
export ACCOUNT_ID='YOUR_ZONE_ID' # 今回は Enterprise Account のアカウント情報を用います

アカウントデータセット一覧を変数にセットします。

export GQL_ENDPOINT='https://api.cloudflare.com/client/v4/graphql'

export ACCOUNT_DATASET=$(http -A creds $GQL_ENDPOINT query='query
{
  __type(name: "account") {
    name
    fields {
      name
      description,
      isDeprecated,
      deprecationReason
    }
  }
}
' | jq -r '.data.__type.fields[]|select(.isDeprecated == false)|.name')

echo $ACCOUNT_DATASET

データセットを読み込みながら、それぞれの保管期間を取得します。

echo '"dataset","enabled","maxDuration","notOlderThan"' > account_dataset_settings.csv

echo $ACCOUNT_DATASET | while read row; do \
case ${row} in "settings" ) sleep 1 ;; "accountTag" ) sleep 1 ;; "zones" ) sleep 1 ;; \
* ) http -A creds --ignore-stdin $GQL_ENDPOINT query='query
{
  viewer {
    accounts(filter: { accountTag: "'${ACCOUNT_ID}'" }) {
      settings {
        '${row}' {
          enabled
          maxDuration
          notOlderThan
        }
      }
    }
  }
}
' | jq -r '.data.viewer.accounts[].settings.'${row}' |select(.enabled == true)| ["'${row}'",.enabled,(.maxDuration/60/60/24|floor|tostring)+" day "+(.maxDuration/60/60%24|tostring)+" hour",(.notOlderThan/60/60/24|floor|tostring)+" day "+(.notOlderThan/60/60%24|tostring)+" hour"]|@csv' >> account_dataset_settings.csv ;; esac done

csv2md account_dataset_settings.csv > account_dataset_settings.md
cat account_dataset_settings.md | pbcopy

CSV から Markdown に変換したものが以下の内容です。

最大で notOlderThan の時間を遡ることができ、一度に maxDuration の範囲でデータ取得できます。

dataset enabled maxDuration notOlderThan
accessLoginRequestsAdaptiveGroups true 1 day 0 hour 31 day 0 hour
cdnNetworkAnalyticsAdaptiveGroups true 30 day 0 hour 112 day 0 hour
dosdAttackAnalyticsGroups true 30 day 0 hour 112 day 0 hour
dosdNetworkAnalyticsAdaptiveGroups true 30 day 0 hour 112 day 0 hour
durableObjectsInvocationsAdaptiveGroups true 30 day 0 hour 90 day 0 hour
durableObjectsPeriodicGroups true 30 day 0 hour 90 day 0 hour
durableObjectsStorageGroups true 30 day 0 hour 90 day 0 hour
durableObjectsSubrequestsAdaptiveGroups true 30 day 0 hour 90 day 0 hour
fbmAttackAnalyticsGroups true 30 day 0 hour 112 day 0 hour
firewallEventsAdaptive true 31 day 0 hour 31 day 0 hour
firewallEventsAdaptiveGroups true 31 day 0 hour 31 day 0 hour
flowtrackdNetworkAnalyticsAdaptiveGroups true 30 day 0 hour 112 day 0 hour
gatewayL4DownstreamSessionsAdaptiveGroups true 7 day 0 hour 365 day 0 hour
gatewayL4SessionsAdaptiveGroups true 30 day 0 hour 30 day 0 hour
gatewayL4UpstreamSessionsAdaptiveGroups true 7 day 0 hour 365 day 0 hour
gatewayL7RequestsAdaptiveGroups true 30 day 0 hour 30 day 0 hour
gatewayResolverByCategoryAdaptiveGroups true 30 day 0 hour 30 day 0 hour
gatewayResolverQueriesAdaptiveGroups true 30 day 0 hour 30 day 0 hour
httpRequests1dGroups true 365 day 1 hour 365 day 1 hour
httpRequests1hGroups true 3 day 0 hour 3 day 1 hour
httpRequestsAdaptive true 31 day 0 hour 91 day 0 hour
httpRequestsAdaptiveGroups true 31 day 0 hour 91 day 0 hour
httpRequestsOverviewAdaptiveGroups true 31 day 0 hour 91 day 0 hour
imagesRequestsAdaptiveGroups true 31 day 0 hour 31 day 0 hour
ipFlows1dGroups true 180 day 0 hour 366 day 0 hour
ipFlows1hGroups true 3 day 1 hour 180 day 0 hour
ipFlows1mAttacksGroups true 180 day 0 hour 366 day 0 hour
ipFlows1mGroups true 1 day 1 hour 30 day 0 hour
logpushHealthAdaptiveGroups true 30 day 0 hour 30 day 1 hour
magicFirewallNetworkAnalyticsAdaptiveGroups true 30 day 0 hour 112 day 0 hour
magicFirewallSamplesAdaptiveGroups true 30 day 0 hour 112 day 0 hour
magicTransitNetworkAnalyticsAdaptiveGroups true 30 day 0 hour 112 day 0 hour
magicTransitTunnelHealthChecksAdaptiveGroups true 31 day 0 hour 92 day 0 hour
magicTransitTunnelTrafficAdaptiveGroups true 30 day 0 hour 112 day 0 hour
mnmFlowDataAdaptiveGroups true 30 day 0 hour 90 day 0 hour
nelReportsAdaptiveGroups true 93 day 0 hour 93 day 0 hour
pagesFunctionsInvocationsAdaptiveGroups true 30 day 0 hour 90 day 0 hour
queueBacklogAdaptiveGroups true 30 day 0 hour 90 day 0 hour
queueMessageOperationsAdaptiveGroups true 30 day 0 hour 90 day 0 hour
r2OperationsAdaptiveGroups true 30 day 0 hour 90 day 0 hour
r2StorageAdaptiveGroups true 30 day 0 hour 90 day 0 hour
rumPageloadEventsAdaptiveGroups true 93 day 0 hour 184 day 0 hour
rumPerformanceEventsAdaptiveGroups true 93 day 0 hour 184 day 0 hour
rumWebVitalsEventsAdaptive true 93 day 0 hour 184 day 0 hour
rumWebVitalsEventsAdaptiveGroups true 93 day 0 hour 184 day 0 hour
spectrumNetworkAnalyticsAdaptiveGroups true 30 day 0 hour 112 day 0 hour
streamMinutesViewedAdaptiveGroups true 31 day 0 hour 92 day 0 hour
turnstileAdaptiveGroups true 7 day 1 hour 30 day 0 hour
videoBufferEventsAdaptiveGroups true 31 day 0 hour 92 day 0 hour
videoPlaybackEventsAdaptiveGroups true 31 day 0 hour 92 day 0 hour
videoQualityEventsAdaptiveGroups true 31 day 0 hour 92 day 0 hour
workersAnalyticsEngineAdaptiveGroups true 31 day 0 hour 31 day 0 hour
workersInvocationsAdaptive true 30 day 0 hour 90 day 0 hour
workersInvocationsScheduled true 7 day 0 hour 90 day 0 hour
workersSubrequestsAdaptiveGroups true 30 day 0 hour 90 day 0 hour
zeroTrustPrivateNetworkDiscoveryGroups true 30 day 0 hour 30 day 0 hour

ダッシュボード Analytics 画面のマッピング

ゾーン

dataset maxDuration notOlderThan dashboard
apiRequestSequencesGroups 1 day 0 hour 31 day 0 hour N/A
cacheReserveOperationsAdaptiveGroups 30 day 0 hour 90 day 0 hour https://dash.cloudflare.com/?to=/:account/:zone/caching/cache-reserve
cacheReserveStorageAdaptiveGroups 30 day 0 hour 90 day 0 hour https://dash.cloudflare.com/?to=/:account/:zone/caching/cache-reserve
emailRoutingAdaptive 1 day 0 hour 31 day 0 hour https://dash.cloudflare.com/?to=/:account/:zone/email/routing/overview
emailRoutingAdaptiveGroups 31 day 0 hour 31 day 0 hour https://dash.cloudflare.com/?to=/:account/:zone/email/routing/overview
firewallEventsAdaptive 31 day 0 hour 31 day 0 hour https://dash.cloudflare.com/?to=/:account/:zone/security/events
firewallEventsAdaptiveByTimeGroups 31 day 0 hour 31 day 0 hour https://dash.cloudflare.com/?to=/:account/:zone/security/events
firewallEventsAdaptiveGroups 31 day 0 hour 31 day 0 hour https://dash.cloudflare.com/?to=/:account/:zone/security/events
healthCheckEventsAdaptive 7 day 0 hour 90 day 1 hour https://dash.cloudflare.com/?to=/:account/:zone/traffic/health-check-analytics
healthCheckEventsAdaptiveGroups 7 day 0 hour 90 day 1 hour https://dash.cloudflare.com/?to=/:account/:zone/traffic/health-check-analytics
httpRequests1dGroups 365 day 1 hour 365 day 1 hour https://dash.cloudflare.com/?to=/:account/:zone/
httpRequests1hGroups 3 day 0 hour 90 day 1 hour https://dash.cloudflare.com/?to=/:account/:zone/
httpRequests1mGroups 0 day 15 hour 7 day 1 hour https://dash.cloudflare.com/?to=/:account/:zone/
httpRequestsAdaptive 31 day 0 hour 90 day 0 hour https://dash.cloudflare.com/?to=/:account/:zone/analytics/traffic
httpRequestsAdaptiveGroups 31 day 0 hour 90 day 0 hour https://dash.cloudflare.com/?to=/:account/:zone/analytics/traffic
httpRequestsOverviewAdaptiveGroups 31 day 0 hour 90 day 0 hour https://dash.cloudflare.com/?to=/:account/:zone/analytics/traffic
imageResizingRequests1mGroups 30 day 1 hour 365 day 0 hour https://dash.cloudflare.com/?to=/:account/:zone/speed/optimization
loadBalancingRequestsAdaptive 7 day 0 hour 30 day 1 hour https://dash.cloudflare.com/?to=/:account/:zone/traffic/load-balancing-analytics
loadBalancingRequestsAdaptiveGroups 7 day 0 hour 30 day 1 hour https://dash.cloudflare.com/?to=/:account/:zone/traffic/load-balancing-analytics
logpushHealthAdaptiveGroups 30 day 0 hour 30 day 1 hour N/A
nelReportsAdaptiveGroups 93 day 0 hour 93 day 0 hour https://dash.cloudflare.com/?to=/:account/:zone/analytics/edgeReachability
synAvgPps1mGroups 7 day 0 hour 7 day 1 hour N/A
waitingRoomAnalyticsAdaptive 30 day 0 hour 90 day 0 hour https://dash.cloudflare.com/?to=/:account/:zone/traffic/waiting-rooms
waitingRoomAnalyticsAdaptiveGroups 30 day 0 hour 90 day 0 hour https://dash.cloudflare.com/?to=/:account/:zone/traffic/waiting-rooms
workersZoneInvocationsAdaptiveGroups 33 day 8 hour 33 day 8 hour https://dash.cloudflare.com/?to=/:account/:zone/analytics/workers
workersZoneSubrequestsAdaptiveGroups 33 day 8 hour 33 day 8 hour https://dash.cloudflare.com/?to=/:account/:zone/analytics/workers
zarazActionsAdaptiveGroups 31 day 0 hour 31 day 0 hour N/A
zarazTrackAdaptiveGroups 31 day 0 hour 31 day 0 hour N/A
zarazTriggersAdaptiveGroups 31 day 0 hour 31 day 0 hour N/A

アカウント

dataset maxDuration notOlderThan dashboard
accessLoginRequestsAdaptiveGroups 1 day 0 hour 31 day 0 hour N/A
cdnNetworkAnalyticsAdaptiveGroups 30 day 0 hour 112 day 0 hour N/A
dosdAttackAnalyticsGroups 30 day 0 hour 112 day 0 hour N/A
dosdNetworkAnalyticsAdaptiveGroups 30 day 0 hour 112 day 0 hour https://dash.cloudflare.com/?to=/:account/network-analytics/ddos-managed-rules
durableObjectsInvocationsAdaptiveGroups 30 day 0 hour 90 day 0 hour https://dash.cloudflare.com/?to=/:account/workers
durableObjectsPeriodicGroups 30 day 0 hour 90 day 0 hour https://dash.cloudflare.com/?to=/:account/workers
durableObjectsStorageGroups 30 day 0 hour 90 day 0 hour https://dash.cloudflare.com/?to=/:account/workers
durableObjectsSubrequestsAdaptiveGroups 30 day 0 hour 90 day 0 hour https://dash.cloudflare.com/?to=/:account/workers
fbmAttackAnalyticsGroups 30 day 0 hour 112 day 0 hour N/A
firewallEventsAdaptive 31 day 0 hour 31 day 0 hour N/A
firewallEventsAdaptiveGroups 31 day 0 hour 31 day 0 hour N/A
flowtrackdNetworkAnalyticsAdaptiveGroups 30 day 0 hour 112 day 0 hour https://dash.cloudflare.com/?to=/:account/network-analytics/advanced-tcp-protection
gatewayL4DownstreamSessionsAdaptiveGroups 7 day 0 hour 365 day 0 hour N/A
gatewayL4SessionsAdaptiveGroups 30 day 0 hour 30 day 0 hour N/A
gatewayL4UpstreamSessionsAdaptiveGroups 7 day 0 hour 365 day 0 hour N/A
gatewayL7RequestsAdaptiveGroups 30 day 0 hour 30 day 0 hour N/A
gatewayResolverByCategoryAdaptiveGroups 30 day 0 hour 30 day 0 hour N/A
gatewayResolverQueriesAdaptiveGroups 30 day 0 hour 30 day 0 hour N/A
httpRequests1dGroups 365 day 1 hour 365 day 1 hour https://dash.cloudflare.com/?to=/:account/analytics
httpRequests1hGroups 3 day 0 hour 3 day 1 hour https://dash.cloudflare.com/?to=/:account/analytics
httpRequestsAdaptive 31 day 0 hour 91 day 0 hour https://dash.cloudflare.com/?to=/:account/analytics
httpRequestsAdaptiveGroups 31 day 0 hour 91 day 0 hour https://dash.cloudflare.com/?to=/:account/analytics
httpRequestsOverviewAdaptiveGroups 31 day 0 hour 91 day 0 hour https://dash.cloudflare.com/?to=/:account/analytics
imagesRequestsAdaptiveGroups 31 day 0 hour 31 day 0 hour https://dash.cloudflare.com/?to=/:account//images
ipFlows1dGroups 180 day 0 hour 366 day 0 hour N/A
ipFlows1hGroups 3 day 1 hour 180 day 0 hour N/A
ipFlows1mAttacksGroups 180 day 0 hour 366 day 0 hour N/A
ipFlows1mGroups 1 day 1 hour 30 day 0 hour N/A
logpushHealthAdaptiveGroups 30 day 0 hour 30 day 1 hour N/A
magicFirewallNetworkAnalyticsAdaptiveGroups 30 day 0 hour 112 day 0 hour https://dash.cloudflare.com/?to=/:account/network-analytics/magic-firewall
magicFirewallSamplesAdaptiveGroups 30 day 0 hour 112 day 0 hour https://dash.cloudflare.com/?to=/:account/network-security/magic_firewall
magicTransitNetworkAnalyticsAdaptiveGroups 30 day 0 hour 112 day 0 hour https://dash.cloudflare.com/?to=/:account/network-analytics/all-traffic
magicTransitTunnelHealthChecksAdaptiveGroups 31 day 0 hour 92 day 0 hour https://dash.cloudflare.com/?to=/:account/magic-transit/tunnel-healthchecks
magicTransitTunnelTrafficAdaptiveGroups 30 day 0 hour 112 day 0 hour https://dash.cloudflare.com/?to=/:account/magic-transit/tunnel-healthchecks
mnmFlowDataAdaptiveGroups 30 day 0 hour 90 day 0 hour https://dash.cloudflare.com/?to=/:account/network-monitoring
nelReportsAdaptiveGroups 93 day 0 hour 93 day 0 hour N/A
pagesFunctionsInvocationsAdaptiveGroups 30 day 0 hour 90 day 0 hour https://dash.cloudflare.com/?to=/:account/pages
queueBacklogAdaptiveGroups 30 day 0 hour 90 day 0 hour https://dash.cloudflare.com/?to=/:account/workers/queues
queueMessageOperationsAdaptiveGroups 30 day 0 hour 90 day 0 hour https://dash.cloudflare.com/?to=/:account/workers/queues
r2OperationsAdaptiveGroups 30 day 0 hour 90 day 0 hour https://dash.cloudflare.com/?to=/:account/r2
r2StorageAdaptiveGroups 30 day 0 hour 90 day 0 hour https://dash.cloudflare.com/?to=/:account/r2
rumPageloadEventsAdaptiveGroups 93 day 0 hour 184 day 0 hour https://dash.cloudflare.com/?to=/:account/web-analytics/sites
rumPerformanceEventsAdaptiveGroups 93 day 0 hour 184 day 0 hour https://dash.cloudflare.com/?to=/:account/web-analytics/sites
rumWebVitalsEventsAdaptive 93 day 0 hour 184 day 0 hour https://dash.cloudflare.com/?to=/:account/web-analytics/sites
rumWebVitalsEventsAdaptiveGroups 93 day 0 hour 184 day 0 hour https://dash.cloudflare.com/?to=/:account/web-analytics/sites
spectrumNetworkAnalyticsAdaptiveGroups 30 day 0 hour 112 day 0 hour https://dash.cloudflare.com/?to=/:account/network-analytics/all-traffic
streamMinutesViewedAdaptiveGroups 31 day 0 hour 92 day 0 hour https://dash.cloudflare.com/?to=/:account/stream/analytics
turnstileAdaptiveGroups 7 day 1 hour 30 day 0 hour https://dash.cloudflare.com/?to=/:account/turnstile
videoBufferEventsAdaptiveGroups 31 day 0 hour 92 day 0 hour N/A
videoPlaybackEventsAdaptiveGroups 31 day 0 hour 92 day 0 hour N/A
videoQualityEventsAdaptiveGroups 31 day 0 hour 92 day 0 hour N/A
workersAnalyticsEngineAdaptiveGroups 31 day 0 hour 31 day 0 hour https://dash.cloudflare.com/?to=/:account/workers
workersInvocationsAdaptive 30 day 0 hour 90 day 0 hour https://dash.cloudflare.com/?to=/:account/workers
workersInvocationsScheduled 7 day 0 hour 90 day 0 hour https://dash.cloudflare.com/?to=/:account/workers
workersSubrequestsAdaptiveGroups 30 day 0 hour 90 day 0 hour https://dash.cloudflare.com/?to=/:account/workers
zeroTrustPrivateNetworkDiscoveryGroups 30 day 0 hour 30 day 0 hour https://dash.cloudflare.com/?to=/:account/analytics/access

参考

1
0
0

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
Sign upLogin
1
0

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?