GCP Cloud Run (Python) 備忘録

1. 超基礎Deploy (Build + Deploy)

最初はこれが楽

gcloud run deploy .

2. BuildとDeployを分ける

Cloud RunのConfigだけを変更したい場合などがでてくるとDocker imageのBuildとDeployは分けたくなる

gcloud builds submit . --image xxx

gcloud run services deploy --image xxx

3. Cloud Run用のイメージをCloud BuildでBuildする際にentrypointを指定する

script_cloud_run.py

app = Flask(__init__)

Options

1. env=GOOGLE_ENTRYPOINT='gunicorn --bind :8080 script_cloud_run:app'を指定

   gcloud builds submit . \
        --pack "image=${{ steps.set-conf.outputs.ARTIFACTS_REGSTRY_REPO }}/${{ steps.set-conf.outputs.IMAGE_NAME }},env=GOOGLE_ENTRYPOINT='gunicorn --bind :8080 script_cloud_run:app'" \
        --project "${PROJECT_ID}"
   

2. Procfileを用意

   web: gunicorn --bind :8080 script_cloud_run:app
   

   gcloud builds submit . \
        --pack "image=${{ steps.set-conf.outputs.ARTIFACTS_REGSTRY_REPO }}/${{ steps.set-conf.outputs.IMAGE_NAME }}" \
        --project "${PROJECT_ID}"
   

Cloud Run and Cloud Functions environment variables

4. Cloud Buildの設定を書く

Cloud BuildのConfigを毎回CommandLineに書くのがめんどくさくなるのでcloudbuild.yamlを導入

gcloud builds submit --config cloudbuild.yaml

5. Cloud Run のYamlを書く

env varやmemoryなど設定項目が増えてくるのでservice.yamlを書く

gcloud run services -- service.yaml

6. tips

invoke cloudrun

URL=$(gcloud run services describe $CLOUD_RUN_SERVICE --project "$PROJECT" --region "$REGION" --format json | jq -r .status.url)
curl $URL \
  --fail-with-body \
  -H "Authorization: Bearer $(gcloud auth print-identity-token)" \
  -X POST \
  -H "Content-Type: application/json" \
  -H "ce-id: 111111111111111" \
  -H "ce-specversion: 1.0" \
  -H "ce-time: 2020-01-02T12:34:56.789Z" \
  -H "ce-type: google.cloud.audit.log.v1.written" \
  -H "ce-source: //cloudaudit.googleapis.com/projects/PROJECT_ID/logs/data_access" \
  -d "$body"

Streamlit

gcloud builds submit . --pack env="GOOGLE_ENTRYPOINT='streamlit run app.py'",image="$REGION-docker.pkg.dev/${PROJECT}/cloud-run-source-deploy/streamlitapp" --project ${PROJECT}

gcloud run deploy gpt-poc \
    --image=$REGION-docker.pkg.dev/$PROJECT/cloud-run-source-deploy/streamlitapp \
    --platform managed \
    --region asia-northeast1 \
    --allow-unauthenticated \
    --service-account streamlitapp@${PROJECT}.iam.gserviceaccount.com \
    --set-secrets OPENAI_ORGANIZATION=openai-organization:latest \
    --set-secrets OPENAI_API_KEY=openai-api-key:latest \
    --set-env-vars PYTHONPATH=/workspace \
    --set-env-vars STREAMLIT_SERVER_MAX_UPLOAD_SIZE=32 \
    --project ${PROJECT}

Cloud Builds

buildpacksでbuildするとpipは以下にinstallされる/layers/google.python.pip/pip

Examples

GCS -> PubSub -> EventarcからTriggerする

Route Cloud Storage events across projects

import base64
import json
import os
from logging import INFO, getLogger

import google.cloud.logging
from flask import Flask, request
from google.cloud.logging_v2.handlers import StructuredLogHandler, setup_logging

# Instantiates a client
if os.getenv("ENV") in ["prod", "dev"]:  # on Cloud Run
    client = google.cloud.logging.Client()
    client.setup_logging()
# Manual configuration
handler = StructuredLogHandler()
setup_logging(handler, log_level=INFO)

logger = getLogger(__name__)  # Creates a logger bound to the current client.

app = Flask(__name__)


# [START eventarc_pubsub_handler]
@app.route("/", methods=["POST"])
def index():
    logger.info("Received Pub/Sub message")
    data = request.get_json()
    if not data:
        msg = "no Pub/Sub message received"
        logger.error(f"error: {msg}")
        return f"Bad Request: {msg}", 400

    if not isinstance(data, dict) or "message" not in data:
        msg = "invalid Pub/Sub message format"
        logger.error(f"error: {msg}")
        return f"Bad Request: {msg}", 400

    pubsub_message = data["message"]
    logger.info(f"Received Pub/Sub message: {pubsub_message}, ID: {request.headers.get('ce-id')}")

    if isinstance(pubsub_message, dict) and "data" in pubsub_message:
        data = json.loads(base64.b64decode(pubsub_message["data"]).decode("utf-8").strip())
    logger.info("Successfully parsed message data: %s", data)
    logger.info(f"bucket: {data['bucket']}, name: {data['name']}")

    return "OK", 200


# [END eventarc_pubsub_handler]


# [START eventarc_pubsub_server]
if __name__ == "__main__":
    app.run(debug=True, host="0.0.0.0", port=int(os.environ.get("PORT", 8080)))  # noqa: S201,S104
# [END eventarc_pubsub_server]

Ref

1. GCP Cloud RunのSLO monitoringをterraformで作成
2. Cloud Runを開発するGitHubレポを爆速設定するテンプレートを作成した!