Audit Log Retention

デフォルトで 18 ヶ月保持されますが、それ以上遡って保管したい場合は、Logpush で随時外部にエクスポートすることができます。

Understanding Cloudflare Audit Logs – Cloudflare Help Center

To maintain Audit Logs query performance, the public Audit Logs API was modified on Jun 30 2019 to return records with a maximum age of 18 months.

こちらのドキュメントを参考に進めます。

Manage Logpush with cURL · Cloudflare Logs docs

アカウントを特定する変数をセット

API 経由で Logpush Job を設定するために、変数をセットします。

Cloudflare API v4 Documentation

export EMAIL='YOUR_EMAIL'
export APIKEY='YOUR_APIKEY'
export ACCOUNT_ID='YOUR_ACCOUNT_ID'
export DATASET='audit_logs' # can be gateway_dns, gateway_http
export BUCKET='kyouhei'
export FOLDER='audit_log'
export JOB_NAME='audit-log' # cannot be changed after job creation. don't use "_"

Ownership challenge を取得

Google Cloud Storage を Logpush のターゲットにするための Ownership challenge を行います。

事前に Google Cloud Storage でバケット kyouhei とディレクトリ audit_log を作成します。
{DATE} を指定することでディレクトリ内でログを日付ごとのフォルダに分けて保管できます。

export CHALLENGE_PATH=$(curl -s -X POST "https://api.cloudflare.com/client/v4/accounts/$ACCOUNT_ID/logpush/ownership" \
-H "X-Auth-Email: $EMAIL" \
-H "X-Auth-Key: $APIKEY" \
-H "Content-Type: application/json" \
--data '{"destination_conf":"gs://'$BUCKET'/'$FOLDER'/{DATE}"}' | jq -r .result.filename)
# Ownership challenge を取得
export CHALLENGE=$(gsutil cat gs://$BUCKET/$CHALLENGE_PATH)
echo $CHALLENGE

result.txt

eyJhbGciOiJkaXIiLCJlbmMiOxxxxxxxxxx

Field を変数にセット

以下のコマンドでそれぞれセットします。

# 全フィールドを指定
export FIELDS=$(curl -s \
-H "X-Auth-Email: $EMAIL" \
-H "X-Auth-Key: $APIKEY" \
"https://api.cloudflare.com/client/v4/accounts/$ACCOUNT_ID/logpush/datasets/$DATASET/fields" | jq -r '.result | keys | join(",")')
echo $FIELDS

Logpush Job を作成

dataset を audit_logs として、account レベルの Logpush Job を作成します。

curl -s -X POST \
https://api.cloudflare.com/client/v4/accounts/$ACCOUNT_ID/logpush/jobs \
-H "X-Auth-Email: $EMAIL" \
-H "X-Auth-Key: $APIKEY" \
-H "Content-Type: application/json" \
-d'{"name": "'$JOB_NAME'", "destination_conf":"gs://'$BUCKET'/'$FOLDER'/{DATE}", "dataset": "'$DATASET'", "logpull_options":"fields='$FIELDS'&timestamps=rfc3339", "ownership_challenge":"'$CHALLENGE'"}' | jq .

result.json

{
  "errors": [],
  "messages": [],
  "result": {
    "id": 42883,
    "dataset": "audit_logs",
    "frequency": "high",
    "kind": "",
    "enabled": false,
    "name": audit-log,
    "logpull_options": "fields=ActionResult,ActionType,ActorEmail,ActorID,ActorIP,ActorType,ID,Interface,Metadata,NewValue,OldValue,OwnerID,ResourceID,ResourceType,When&timestamps=rfc3339",
    "destination_conf": "gs://kyouhei/audit_log/{DATE}",
    "last_complete": null,
    "last_error": null,
    "error_message": null
  },
  "success": true
}

作成した Logpush Job を有効化

JobID を指定して、enabled が false になっているのを true にセットして有効化します。

curl -s -X PUT \
-d'{"enabled":true}' \
-H "X-Auth-Email: $EMAIL" \
-H "X-Auth-Key: $APIKEY" \
-H "Content-Type: application/json" \
https://api.cloudflare.com/client/v4/accounts/$ACCOUNT_ID/logpush/jobs/42883 | jq .

result.json

{
  "errors": [],
  "messages": [],
  "result": {
    "id": 42883,
    "dataset": "audit_logs",
    "frequency": "high",
    "kind": "",
    "enabled": true,
    "name": null,
    "logpull_options": "fields=ActionResult,ActionType,ActorEmail,ActorID,ActorIP,ActorType,ID,Interface,Metadata,NewValue,OldValue,OwnerID,ResourceID,ResourceType,When&timestamps=rfc3339",
    "destination_conf": "gs://kyouhei/audit_log/{DATE}",
    "last_complete": null,
    "last_error": null,
    "error_message": null
  },
  "success": true
}

Logpush Job のリスト（アカウントレベル）

アカウントレベルでの Logpush Job リストを確認できます。

last_complete が最後にエクスポートが実行された時間です。

curl -X GET "https://api.cloudflare.com/client/v4/accounts/$ACCOUNT_ID/logpush/jobs" \
-H "X-Auth-Email: $EMAIL" \
-H "X-Auth-Key: $APIKEY" \
-H "Content-Type: application/json" | jq .

result.json

{
  "errors": [],
  "messages": [],
  "result": [
    {
      "id": 42883,
      "dataset": "audit_logs",
      "frequency": "high",
      "kind": "",
      "enabled": true,
      "name": null,
      "logpull_options": "fields=ActionResult,ActionType,ActorEmail,ActorID,ActorIP,ActorType,ID,Interface,Metadata,NewValue,OldValue,OwnerID,ResourceID,ResourceType,When&timestamps=rfc3339",
      "destination_conf": "gs://kyouhei/audit_log/{DATE}",
      "last_complete": "2021-10-28T02:24:15Z",
      "last_error": null,
      "error_message": null
    }
  ],
  "success": true
}

Logpush の確認

UI

Account Home > Audit Log から確認できます。

Google Cloud Storage

該当するログファイルを確認できました。

gsutil cat gs://kyouhei/audit_log/20211027/20211027T051004Z_20211027T051034Z_32068b24.log.gz | gunzip | jq .

result.json

{
  "ActionResult": true,
  "ActionType": "waiting_rooms_edit",
  "ActorEmail": "xxx",
  "ActorID": "xx",
  "ActorIP": "240d:xxx",
  "ActorType": "user",
  "ID": "0a2fc5f6-4900-46ed-9364-c31a7cd9a110",
  "Interface": "UI",
  "Metadata": {
    "actor_email": "xxx",
    "zone_name": "xxx",
    "zone_tag": "xxx"
  },
  "NewValue": {
    "id": "ea05cb62a47a8e66e387002eae48ab1d",
    "name": "waiting_room",
    "description": "",
    "host": "wait.xxx",
    "path": "/",
    "suspended": false,
    "queue_all": false,
    "new_users_per_minute": 200,
    "total_active_users": 200,
    "session_duration": 5,
    "custom_page_html": "",
    "disable_session_renewal": false,
    "json_response_enabled": false,
    "queueing_method": "fifo",
    "cookie_attributes": {
      "samesite": "auto",
      "secure": "auto"
    },
    "created_on": "2021-10-20T05:14:59.449217Z",
    "modified_on": "2021-10-27T05:08:09.273506Z"
  },
  "OldValue": {
    "id": "ea05cb62a47a8e66e387002eae48ab1d",
    "name": "wait",
    "description": "",
    "host": "wait.xxx",
    "path": "/",
    "suspended": false,
    "queue_all": false,
    "new_users_per_minute": 200,
    "total_active_users": 200,
    "session_duration": 5,
    "custom_page_html": "",
    "disable_session_renewal": false,
    "json_response_enabled": false,
    "queueing_method": "fifo",
    "cookie_attributes": {
      "samesite": "auto",
      "secure": "auto"
    },
    "created_on": "2021-10-20T05:14:59.449217Z",
    "modified_on": "2021-10-22T05:45:19.625349Z"
  },
  "OwnerID": "xxx",
  "ResourceID": "xxx",
  "ResourceType": "Waiting Room",
  "When": "2021-10-27T05:08:09Z"
}

参考：Logpush Job を削除

JobID を指定して Logpush Job を削除できます。

curl -s -X DELETE \
-H "X-Auth-Email: $EMAIL" \
-H "X-Auth-Key: $APIKEY" \
-H "Content-Type: application/json" \
https://api.cloudflare.com/client/v4/accounts/$ACCOUNT_ID/logpush/jobs/42080 | jq .

result.json

{
  "errors": [],
  "messages": [],
  "result": null,
  "success": true
}

参考：Google Cloud Storage にある最新10個のログファイルの中身を表示

export BUCKET='kyouhei'
export FOLDER='audit_log'

gsutil list gs://$BUCKET/$FOLDER/$(date +"%Y%m%d") | grep .log.gz | tail -n 10 | while read line
do
  gsutil cat $line | gunzip | jq .
done

参考；ZONE ベースのスクリプト

export ZONE_ID='YOUR_ZONE_ID'
---
# Ownership challenge を取得
export CHALLENGE_PATH=$(curl -s -X POST "https://api.cloudflare.com/client/v4/zones/$ZONE_ID/logpush/ownership" \
-H "X-Auth-Email: $EMAIL" \
-H "X-Auth-Key: $APIKEY" \
-H "Content-Type: application/json" \
--data '{"destination_conf":"gs://'$BUCKET'/'$FOLDER'/{DATE}"}' | jq -r .result.filename)

export CHALLENGE=$(gsutil cat gs://$BUCKET/$CHALLENGE_PATH)
echo $CHALLENGE
---
# 全フィールドを指定
export FIELDS=$(curl -s \
-H "X-Auth-Email: $EMAIL" \
-H "X-Auth-Key: $APIKEY" \
"https://api.cloudflare.com/client/v4/zones/$ZONE_ID/logpush/datasets/$DATASET/fields" | jq -r '.result | keys | join(",")')
echo $FIELDS
---
# ジョブ作成
curl -s -X POST \
https://api.cloudflare.com/client/v4/zones/$ZONE_ID/logpush/jobs \
-H "X-Auth-Email: $EMAIL" \
-H "X-Auth-Key: $APIKEY" \
-H "Content-Type: application/json" \
-d'{"name": "'$JOB_NAME'", "destination_conf":"gs://'$BUCKET'/'$FOLDER'/{DATE}", "dataset": "'$DATASET'", "logpull_options":"fields='$FIELDS'&timestamps=rfc3339", "ownership_challenge":"'$CHALLENGE'"}' | jq .
---
# 有効化
curl -s -X PUT \
-d'{"enabled":true}' \
-H "X-Auth-Email: $EMAIL" \
-H "X-Auth-Key: $APIKEY" \
-H "Content-Type: application/json" \
https://api.cloudflare.com/client/v4/zones/$ZONE_ID/logpush/jobs/42883 | jq .
---
# ジョブのリスト
curl -X GET "https://api.cloudflare.com/client/v4/zones/$ZONE_ID/logpush/jobs" \
-H "X-Auth-Email: $EMAIL" \
-H "X-Auth-Key: $APIKEY" \
-H "Content-Type: application/json" | jq .