学習用モノレポ: 動画ストリーミング + メタデータ マイクロサービス(Go + gRPC, Clean Architecture, Kubernetes, Next.js)
技術選定の要点
- Go + gRPC: 高速・型安全・スキーマ駆動(Protocol Buffers)。スケールしやすく、観測性やエラーハンドリングの標準化が容易。
-
Clean Architecture:
domain
/usecase
/interface
/infrastructure
を分離し、変更に強くテスト容易。ビジネスルールを外部依存から切り離す。 - Kubernetes/GKE: マイクロサービスの独立デプロイ、オートスケール(HPA)、ローリングアップデート。
- AlloyDB (PostgreSQL互換): 動画メタデータなどリレーショナルな正規化情報の管理に最適。ローカルは PostgreSQL 代替。
- Memorystore (Redis): キャッシュ、ランキング、レート制限、セッションなどの低遅延用途。
- Pub/Sub: アップロード完了/エンコード完了/視聴イベントの非同期連携。
- Bigtable: 時系列メトリクスや大量読み取りに最適。ローカルは Bigtable Emulator を利用可能(任意)。
- Cloud Functions / Cloud Run: 軽量イベントハンドラやバッチ。gRPC 本体は GKE、補助イベントは Functions/Run。
- Grafana (+Prometheus): ログ/メトリクス/トレースの可視化。
- Next.js + TypeScript: App Router と API ルートで BFF 的に活用。開発体験が良い。
- tailwindCSS + shadcn/ui: UI 実装速度と一貫性を向上。
- Zod: スキーマとランタイムバリデーションの同期。
- Axios: API クライアントの共通化(リトライ、トークン更新等)。
- Jotai: 軽量な状態管理。
- pnpm: 高速・ディスク効率の良いパッケージ管理。
- Docker/docker-compose: ローカルのインフラ再現。
- Terraform: GCP インフラを IaC で構築。
環境構築(ローカル)
前提条件(ローカルにインストール)
- macOS + Docker Desktop
- Homebrew
- Go 1.22+ / Node.js 20+ / pnpm 9+ / protoc 3.21+ / buf / kubectl / gcloud / terraform / grpcurl
brew install go node pnpm protobuf bufbuild/buf/buf kubectl google-cloud-sdk terraform grpcurl
# 確認
go version
node -v
pnpm -v
protoc --version
buf --version
kubectl version --client
gcloud version
terraform -version
grpcurl -version
google-cloud-sdkのインストールの際にPythonのインストールが必要な場合があります。
brew install python@3.12
プロジェクト配置
- 本 README は
backend/
,frontend/
,infra/
を前提とします。ディレクトリがない場合は作成してください。
Docker Compose(PostgreSQL/Redis/GCS 互換エミュレータ/Prometheus/Grafana)
infra/docker-compose.yml
と infra/prometheus.yml
を作成後、起動します。
# infra/docker-compose.yml(完全)
version: "3.9"
services:
# PostgreSQL: アプリケーションのリレーショナルDB(ユーザー/パスワード/DB 名は学習用の固定値)
postgres:
image: postgres:16
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: app
ports: ["5432:5432"]
healthcheck:
test: ["CMD-SHELL", "pg_isready -U postgres"]
interval: 5s
timeout: 5s
retries: 10
volumes: ["pgdata:/var/lib/postgresql/data"]
# Redis: セッション・キャッシュ・レート制限などで使うインメモリKVS(永続化なし設定)
redis:
image: redis:7
ports: ["6379:6379"]
command: ["redis-server", "--save", "", "--appendonly", "no"]
# GCS 互換エミュレータ: GCS の JSON/Download API を模擬(学習用)。API は http://localhost:4443
fake-gcs:
image: fsouza/fake-gcs-server:latest
command: ["-backend", "memory", "-scheme", "http"]
ports: ["4443:4443"]
# Prometheus: メトリクス収集。自身(:9090)および任意のアプリをスクレイプ
prometheus:
image: prom/prometheus:latest
ports: ["9090:9090"]
volumes: ["./prometheus.yml:/etc/prometheus/prometheus.yml:ro"]
# Grafana: ダッシュボード可視化。初期ログインは admin / admin(環境変数で上書き可)
grafana:
image: grafana/grafana:latest
ports: ["3000:3000"]
environment:
- GF_SECURITY_ADMIN_PASSWORD=admin
volumes: ["grafana:/var/lib/grafana"]
volumes:
pgdata:
grafana:
# infra/prometheus.yml
global:
scrape_interval: 15s
scrape_configs:
- job_name: "self"
static_configs:
- targets: ["localhost:9090"]
起動:
docker compose -f infra/docker-compose.yml up -d
実装
モノレポ構成(提案)
.
├── backend/
│ ├── go.work # マルチモジュール(任意)
│ ├── proto/ # .proto(サービス間共通)
│ │ ├── video/streaming/v1/streaming.proto
│ │ └── video/metadata/v1/metadata.proto
│ ├── services/
│ │ ├── video-streaming/
│ │ │ ├── cmd/server/main.go
│ │ │ ├── interface/handler/grpc.go
│ │ │ ├── usecase/
│ │ │ ├── domain/
│ │ │ │ ├── entity/
│ │ │ │ ├── repository/
│ │ │ │ └── service/
│ │ │ └── infrastructure/
│ │ │ ├── storage/ (GCS/GCS互換)
│ │ │ ├── cache/ (Redis)
│ │ │ └── pubsub/
│ │ └── video-metadata/
│ │ ├── cmd/server/main.go
│ │ ├── interface/handler/grpc.go
│ │ ├── usecase/
│ │ ├── domain/
│ │ │ ├── entity/
│ │ │ ├── repository/
│ │ │ └── service/
│ │ └── infrastructure/
│ │ ├── postgres/
│ │ ├── redis/
│ │ └── pubsub/
│ ├── gateway/ # gRPC-Gateway (REST for frontend)
│ ├── build/ # Dockerfile, Makefile, Skaffold 等
│ └── deploy/ # k8s manifests (kustomize)
├── frontend/
│ ├── apps/web/ # Next.js (App Router)
│ ├── packages/ui/ # shadcn/ui ラップ
│ ├── packages/config/ # eslint, tsconfig 等共有
│ └── pnpm-workspace.yaml
├── infra/
│ ├── docker-compose.yml
│ └── terraform/ # GCP (GKE, AlloyDB, Memorystore, Pub/Sub, Bigtable)
└── README.md
全体アーキテクチャ(PlantUML)
バックエンド: gRPC スキーマ(Protocol Buffers)
// backend/proto/video/metadata/v1/metadata.proto
syntax = "proto3";
package video.metadata.v1;
option go_package = "backend/protogo/video/metadata/v1;metadatav1";
import "google/api/annotations.proto";
message Video {
string id = 1;
string title = 2;
string description = 3;
uint64 duration_sec = 4;
string hls_path = 5; // ex: gs://bucket/hls/{id}/index.m3u8
}
message ListVideosRequest { uint32 page = 1; uint32 page_size = 2; }
message ListVideosResponse { repeated Video videos = 1; }
message GetVideoRequest { string id = 1; }
message GetVideoResponse { Video video = 1; }
service MetadataService {
rpc ListVideos(ListVideosRequest) returns (ListVideosResponse) {
option (google.api.http) = { get: "/v1/videos" };
}
rpc GetVideo(GetVideoRequest) returns (GetVideoResponse) {
option (google.api.http) = { get: "/v1/videos/{id}" };
}
}
// backend/proto/video/streaming/v1/streaming.proto
syntax = "proto3";
package video.streaming.v1;
option go_package = "backend/protogo/video/streaming/v1;streamingv1";
import "google/api/annotations.proto";
message GetPlaybackURLRequest { string video_id = 1; }
message GetPlaybackURLResponse { string playback_url = 1; }
service StreamingService {
rpc GetPlaybackURL(GetPlaybackURLRequest) returns (GetPlaybackURLResponse) {
option (google.api.http) = { get: "/v1/videos/{video_id}/play" };
}
}
# backend/proto/buf.gen.yaml(例)
version: v2
plugins:
- plugin: buf.build/protocolbuffers/go
out: ../protogo
opt: paths=source_relative
- plugin: buf.build/grpc/go
out: ../protogo
opt: paths=source_relative
- plugin: buf.build/grpc-ecosystem/gateway:v2.20.0
out: ../gateway
opt:
- paths=source_relative
- generate_unbound_methods=true
バックエンド: Clean Architecture スケルトン(動画メタデータ)
// backend/services/video-metadata/domain/entity/video.go
package entity
type Video struct {
ID string
Title string
Description string
DurationSec uint64
HLSPath string
}
// backend/services/video-metadata/domain/repository/video_repository.go
package repository
import "backend/services/video-metadata/domain/entity"
type VideoRepository interface {
List(page, pageSize uint32) ([]entity.Video, error)
GetByID(id string) (*entity.Video, error)
}
// backend/services/video-metadata/usecase/video_usecase.go
package usecase
import (
"backend/services/video-metadata/domain/entity"
"backend/services/video-metadata/domain/repository"
)
type VideoUsecase struct {
repo repository.VideoRepository
}
func NewVideoUsecase(repo repository.VideoRepository) *VideoUsecase {
return &VideoUsecase{repo: repo}
}
func (u *VideoUsecase) List(page, pageSize uint32) ([]entity.Video, error) {
return u.repo.List(page, pageSize)
}
func (u *VideoUsecase) Get(id string) (*entity.Video, error) {
return u.repo.GetByID(id)
}
// backend/services/video-metadata/infrastructure/postgres/video_repository.go
package postgres
import (
"context"
"database/sql"
"backend/services/video-metadata/domain/entity"
"backend/services/video-metadata/domain/repository"
)
type VideoRepositoryPG struct { db *sql.DB }
func NewVideoRepositoryPG(db *sql.DB) repository.VideoRepository { return &VideoRepositoryPG{db: db} }
func (r *VideoRepositoryPG) List(page, pageSize uint32) ([]entity.Video, error) {
offset := int((page - 1) * pageSize)
rows, err := r.db.QueryContext(context.Background(),
`SELECT id, title, description, duration_sec, hls_path FROM videos ORDER BY id LIMIT $1 OFFSET $2`,
pageSize, offset,
)
if err != nil { return nil, err }
defer rows.Close()
var vids []entity.Video
for rows.Next() {
var v entity.Video
if err := rows.Scan(&v.ID, &v.Title, &v.Description, &v.DurationSec, &v.HLSPath); err != nil { return nil, err }
vids = append(vids, v)
}
return vids, rows.Err()
}
func (r *VideoRepositoryPG) GetByID(id string) (*entity.Video, error) {
var v entity.Video
err := r.db.QueryRowContext(context.Background(),
`SELECT id, title, description, duration_sec, hls_path FROM videos WHERE id = $1`, id,
).Scan(&v.ID, &v.Title, &v.Description, &v.DurationSec, &v.HLSPath)
if err != nil { return nil, err }
return &v, nil
}
// backend/services/video-metadata/interface/handler/grpc.go
package handler
import (
"context"
metadatav1 "backend/protogo/video/metadata/v1"
"backend/services/video-metadata/usecase"
)
type MetadataHandler struct {
metadatav1.UnimplementedMetadataServiceServer
uc *usecase.VideoUsecase
}
func NewMetadataHandler(uc *usecase.VideoUsecase) *MetadataHandler { return &MetadataHandler{uc: uc} }
func (h *MetadataHandler) ListVideos(ctx context.Context, req *metadatav1.ListVideosRequest) (*metadatav1.ListVideosResponse, error) {
vids, err := h.uc.List(req.GetPage(), req.GetPageSize())
if err != nil { return nil, err }
out := make([]*metadatav1.Video, 0, len(vids))
for _, v := range vids {
out = append(out, &metadatav1.Video{Id: v.ID, Title: v.Title, Description: v.Description, DurationSec: v.DurationSec, HlsPath: v.HLSPath})
}
return &metadatav1.ListVideosResponse{Videos: out}, nil
}
func (h *MetadataHandler) GetVideo(ctx context.Context, req *metadatav1.GetVideoRequest) (*metadatav1.GetVideoResponse, error) {
v, err := h.uc.Get(req.GetId())
if err != nil { return nil, err }
return &metadatav1.GetVideoResponse{Video: &metadatav1.Video{Id: v.ID, Title: v.Title, Description: v.Description, DurationSec: v.DurationSec, HlsPath: v.HLSPath}}, nil
}
// backend/services/video-metadata/cmd/server/main.go
package main
import (
"database/sql"
"log"
"net"
_ "github.com/jackc/pgx/v5/stdlib"
"google.golang.org/grpc"
"google.golang.org/grpc/reflection"
metadatav1 "backend/protogo/video/metadata/v1"
"backend/services/video-metadata/infrastructure/postgres"
"backend/services/video-metadata/interface/handler"
"backend/services/video-metadata/usecase"
)
func main() {
db, err := sql.Open("pgx", "postgres://postgres:postgres@localhost:5432/app?sslmode=disable")
if err != nil { log.Fatal(err) }
defer db.Close()
repo := postgres.NewVideoRepositoryPG(db)
uc := usecase.NewVideoUsecase(repo)
h := handler.NewMetadataHandler(uc)
lis, err := net.Listen("tcp", ":50051")
if err != nil { log.Fatal(err) }
s := grpc.NewServer()
metadatav1.RegisterMetadataServiceServer(s, h)
reflection.Register(s)
log.Println("metadata gRPC listening :50051")
if err := s.Serve(lis); err != nil { log.Fatal(err) }
}
バックエンド: Clean Architecture スケルトン(動画ストリーミング)
// backend/services/video-streaming/usecase/playback_usecase.go
package usecase
type PlaybackSigner interface { SignHLSURL(hlsPath string) (string, error) }
type PlaybackUsecase struct { signer PlaybackSigner }
func NewPlaybackUsecase(signer PlaybackSigner) *PlaybackUsecase { return &PlaybackUsecase{signer: signer} }
func (u *PlaybackUsecase) GetPlaybackURL(hlsPath string) (string, error) { return u.signer.SignHLSURL(hlsPath) }
// backend/services/video-streaming/infrastructure/storage/local_signer.go
package storage
type LocalSigner struct{}
func NewLocalSigner() *LocalSigner { return &LocalSigner{} }
func (s *LocalSigner) SignHLSURL(hlsPath string) (string, error) {
// 例: GCS 互換エミュレータ用の固定URLに置き換えてください
return "http://localhost:4443/download/storage/v1/b/media/o/" + url.PathEscape(hlsPath) + "?alt=media", nil
}
// backend/services/video-streaming/interface/handler/grpc.go
package handler
import (
"context"
streamingv1 "backend/protogo/video/streaming/v1"
"backend/services/video-streaming/usecase"
)
type StreamingHandler struct {
streamingv1.UnimplementedStreamingServiceServer
uc *usecase.PlaybackUsecase
}
func NewStreamingHandler(uc *usecase.PlaybackUsecase) *StreamingHandler { return &StreamingHandler{uc: uc} }
func (h *StreamingHandler) GetPlaybackURL(ctx context.Context, req *streamingv1.GetPlaybackURLRequest) (*streamingv1.GetPlaybackURLResponse, error) {
// 実際には metadata 参照して hlsPath を取得
hlsPath := "hls/" + req.GetVideoId() + "/index.m3u8"
url, err := h.uc.GetPlaybackURL(hlsPath)
if err != nil { return nil, err }
return &streamingv1.GetPlaybackURLResponse{PlaybackUrl: url}, nil
}
// backend/services/video-streaming/cmd/server/main.go
package main
import (
"log"
"net"
"google.golang.org/grpc"
"google.golang.org/grpc/reflection"
streamingv1 "backend/protogo/video/streaming/v1"
"backend/services/video-streaming/infrastructure/storage"
"backend/services/video-streaming/interface/handler"
"backend/services/video-streaming/usecase"
)
func main() {
signer := storage.NewLocalSigner()
uc := usecase.NewPlaybackUsecase(signer)
h := handler.NewStreamingHandler(uc)
lis, err := net.Listen("tcp", ":50052")
if err != nil { log.Fatal(err) }
s := grpc.NewServer()
streamingv1.RegisterStreamingServiceServer(s, h)
reflection.Register(s)
log.Println("streaming gRPC listening :50052")
if err := s.Serve(lis); err != nil { log.Fatal(err) }
}
gRPC-Gateway(REST公開)
// backend/gateway/main.go(簡易・単一プロセス例)
package main
import (
"context"
"log"
"net/http"
gwMeta "backend/gateway/video/metadata/v1"
gwStream "backend/gateway/video/streaming/v1"
)
func main() {
mux := http.NewServeMux()
ctx := context.Background()
if err := gwMeta.RegisterMetadataServiceHandlerFromEndpoint(ctx, mux, "localhost:50051", nil); err != nil { log.Fatal(err) }
if err := gwStream.RegisterStreamingServiceHandlerFromEndpoint(ctx, mux, "localhost:50052", nil); err != nil { log.Fatal(err) }
log.Println("gateway listening :8080")
log.Fatal(http.ListenAndServe(":8080", mux))
}
フロントエンド(Next.js + Tailwind + shadcn/ui + Axios + Zod + Jotai)
// frontend/apps/web/package.json(抜粋)
{
"name": "web",
"private": true,
"scripts": {
"dev": "next dev",
"build": "next build",
"start": "next start"
},
"dependencies": {
"next": "14.2.4",
"react": "18.2.0",
"react-dom": "18.2.0",
"axios": "^1.6.8",
"zod": "^3.23.8",
"jotai": "^2.6.0",
"tailwindcss": "^3.4.7"
}
}
# frontend/pnpm-workspace.yaml
packages:
- "apps/*"
- "packages/*"
// frontend/apps/web/src/lib/api.ts
import axios from "axios";
export const api = axios.create({ baseURL: process.env.NEXT_PUBLIC_API_BASE || "http://localhost:8080" });
api.interceptors.response.use((res) => res, async (err) => {
return Promise.reject(err);
});
// frontend/apps/web/src/lib/schema.ts
import { z } from "zod";
export const Video = z.object({
id: z.string(),
title: z.string(),
description: z.string(),
duration_sec: z.number().int().nonnegative(),
hls_path: z.string()
});
export const ListVideosResponse = z.object({ videos: z.array(Video) });
// frontend/apps/web/src/app/page.tsx
import { api } from "../lib/api";
import { ListVideosResponse } from "../lib/schema";
export default async function Home() {
const res = await api.get("/v1/videos");
const parsed = ListVideosResponse.parse(res.data);
return (
<main className="p-6 space-y-4">
<h1 className="text-2xl font-bold">Videos</h1>
<ul className="space-y-2">
{parsed.videos.map((v) => (
<li key={v.id} className="border p-3 rounded">
<div className="font-semibold">{v.title}</div>
<div className="text-sm opacity-70">{v.description}</div>
<a className="text-blue-600 underline" href={`/watch/${v.id}`}>Play</a>
</li>
))}
</ul>
</main>
);
}
// frontend/apps/web/src/app/watch/[id]/page.tsx
import { api } from "../../../lib/api";
type Props = { params: { id: string } };
export default async function Watch({ params }: Props) {
const res = await api.get(`/v1/videos/${params.id}/play`);
const url = res.data.playback_url as string;
return (
<main className="p-6 space-y-4">
<h1 className="text-2xl font-bold">Watch</h1>
<video controls src={url} className="w-full max-w-3xl" />
</main>
);
}
補足
- 実配信は CDN と署名 URL(Cloud CDN Signed URL/Cloud Storage Signed URL)を用いるのが一般的。
- 観測性は OpenTelemetry + Prometheus + Grafana で収集・可視化を推奨。
- 本 README のコードは学習用の最小スケルトン。実運用では認可(JWT/OIDC)、レート制限、リトライ/サーキットブレーカ、マイグレーション等を追加する。
ここから先は省略なしの実行手順(ローカル・GCP)
前提条件(ローカルにインストール)
- macOS + Docker Desktop
- Homebrew
- Go 1.22+ / Node.js 20+ / pnpm 9+ / protoc 3.21+ / buf / kubectl / gcloud / terraform / grpcurl
brew install go node pnpm protobuf bufbuild/buf/buf kubectl google-cloud-sdk terraform grpcurl
# 確認
go version
node -v
pnpm -v
protoc --version
buf --version
kubectl version --client
gcloud version
terraform -version
grpcurl -version
echo "grpcurl $(grpcurl -version || true)"
プロジェクト配置(このREADMEに沿って進めるだけでOK)
- 本 README は
backend/
,frontend/
,infra/
を前提にしています(このリポジトリ直下に作成)。 - 以降のコードブロックはファイルパスとともに記載します。ディレクトリがない場合は作成してください。
1. ローカル環境構築(省略なし)
1-1. Docker Compose(PostgreSQL/Redis/GCS 互換エミュレータ/Prometheus/Grafana)
以下の完全なファイルを infra/docker-compose.yml
として作成します。
version: "3.9"
services:
# PostgreSQL: リレーショナルDB(学習用のユーザー/パスワード)
postgres:
image: postgres:16
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: app
ports: ["5432:5432"]
healthcheck:
test: ["CMD-SHELL", "pg_isready -U postgres"]
interval: 5s
timeout: 5s
retries: 10
volumes: ["pgdata:/var/lib/postgresql/data"]
# Redis: インメモリKVS。セッション・キャッシュ用途。
redis:
image: redis:7
ports: ["6379:6379"]
command: ["redis-server", "--save", "", "--appendonly", "no"]
# GCS 互換エミュレータ: GCS JSON/Download API を模擬
fake-gcs:
image: fsouza/fake-gcs-server:latest
command: ["-backend", "memory", "-scheme", "http"]
ports: ["4443:4443"]
# Prometheus: メトリクス収集
prometheus:
image: prom/prometheus:latest
ports: ["9090:9090"]
volumes: ["./prometheus.yml:/etc/prometheus/prometheus.yml:ro"]
# Grafana: 可視化(admin パスワードは環境変数で指定)
grafana:
image: grafana/grafana:latest
ports: ["3000:3000"]
environment:
- GF_SECURITY_ADMIN_PASSWORD=admin
volumes: ["grafana:/var/lib/grafana"]
volumes:
pgdata:
grafana:
Prometheus 設定を infra/prometheus.yml
に作成します(最小)。
global:
scrape_interval: 15s
scrape_configs:
- job_name: "self"
static_configs:
- targets: ["localhost:9090"]
起動:
docker compose -f infra/docker-compose.yml up -d
1-2. GCS 互換エミュレータでのバケット作成とサンプル HLS 配置
# サンプル HLS ディレクトリ作成
mkdir -p /tmp/hls/vid001
cat > /tmp/hls/vid001/index.m3u8 <<'M3U8'
#EXTM3U
#EXT-X-VERSION:3
#EXT-X-TARGETDURATION:10
#EXT-X-MEDIA-SEQUENCE:0
#EXTINF:10.0,
seg0.ts
#EXTINF:10.0,
seg1.ts
#EXT-X-ENDLIST
M3U8
# ダミーTSセグメント
dd if=/dev/zero of=/tmp/hls/vid001/seg0.ts bs=1k count=100
dd if=/dev/zero of=/tmp/hls/vid001/seg1.ts bs=1k count=100
# バケット作成(media)
curl -s -X POST "http://localhost:4443/storage/v1/b?project=fake" \
-H "Content-Type: application/json" \
-d '{"name":"media"}' | jq .
# アップロード(バケット配下に hls/vid001)
curl -s -X POST --data-binary @/tmp/hls/vid001/index.m3u8 \
-H "Content-Type: application/octet-stream" \
"http://localhost:4443/upload/storage/v1/b/media/o?uploadType=media&name=hls/vid001/index.m3u8" | jq .
curl -s -X POST --data-binary @/tmp/hls/vid001/seg0.ts \
-H "Content-Type: application/octet-stream" \
"http://localhost:4443/upload/storage/v1/b/media/o?uploadType=media&name=hls/vid001/seg0.ts" | jq .
curl -s -X POST --data-binary @/tmp/hls/vid001/seg1.ts \
-H "Content-Type: application/octet-stream" \
"http://localhost:4443/upload/storage/v1/b/media/o?uploadType=media&name=hls/vid001/seg1.ts" | jq .
再生 URL(ダウンロード API の例): http://localhost:4443/download/storage/v1/b/media/o/hls%2Fvid001%2Findex.m3u8?alt=media
1-3. DB スキーマ準備
docker compose -f infra/docker-compose.yml exec -T postgres psql -U postgres -d app <<'SQL'
CREATE TABLE IF NOT EXISTS videos (
id text primary key,
title text not null,
description text not null,
duration_sec bigint not null,
hls_path text not null
);
INSERT INTO videos (id,title,description,duration_sec,hls_path) VALUES
('vid001','Sample 1','Demo',120,'hls/vid001/index.m3u8')
ON CONFLICT (id) DO NOTHING;
SQL
単発での書き込み例:
docker compose -f infra/docker-compose.yml exec -T postgres psql -U postgres -d app -c \
"INSERT INTO videos(id,title,description,duration_sec,hls_path) VALUES ('vid002','Sample 2','Demo 2',180,'hls/vid002/index.m3u8') ON CONFLICT (id) DO NOTHING;"
1-4. Protocol Buffers 生成
backend/proto/buf.yaml
を作成:
version: v2
modules:
- path: .
deps:
- buf.build/googleapis/googleapis
breaking:
use:
- FILE
lint:
use:
- DEFAULT
backend/proto/buf.gen.yaml
(完全版):
version: v2
plugins:
- plugin: buf.build/protocolbuffers/go
out: ../protogo
opt: paths=source_relative
- plugin: buf.build/grpc/go
out: ../protogo
opt: paths=source_relative
- plugin: buf.build/grpc-ecosystem/gateway
out: ../gateway
opt:
- paths=source_relative
- generate_unbound_methods=true
生成コマンド:
cd backend/proto
buf mod update
buf generate
cd -
1-5. Go でローカル起動
ターミナル1(メタデータ)
go run ./backend/services/video-metadata/cmd/server
ターミナル2(ストリーミング)
go run ./backend/services/video-streaming/cmd/server
ターミナル3(gRPC-Gateway)
go run ./backend/gateway
確認(REST):
curl -s http://localhost:8080/v1/videos | jq .
curl -s http://localhost:8080/v1/videos/vid001 | jq .
curl -s http://localhost:8080/v1/videos/vid001/play | jq .
確認(gRPC):
grpcurl -plaintext localhost:50051 list
grpcurl -plaintext localhost:50052 list
1-6. フロントエンド起動
cd frontend/apps/web
pnpm i
export NEXT_PUBLIC_API_BASE=http://localhost:8080
pnpm dev
http://localhost:3000
をブラウザで開く。
デプロイ準備(Docker イメージ)
2-1. Dockerfile(完全)
メタデータサービス backend/services/video-metadata/Dockerfile
:
FROM golang:1.22 AS build
WORKDIR /app
COPY . /app
RUN go mod download
RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -o /out/server ./services/video-metadata/cmd/server
FROM gcr.io/distroless/base-debian12
COPY --from=build /out/server /server
USER nonroot:nonroot
ENTRYPOINT ["/server"]
ストリーミングサービス backend/services/video-streaming/Dockerfile
:
FROM golang:1.22 AS build
WORKDIR /app
COPY . /app
RUN go mod download
RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -o /out/server ./services/video-streaming/cmd/server
FROM gcr.io/distroless/base-debian12
COPY --from=build /out/server /server
USER nonroot:nonroot
ENTRYPOINT ["/server"]
ゲートウェイ backend/gateway/Dockerfile
:
FROM golang:1.22 AS build
WORKDIR /app
COPY . /app
RUN go mod download
RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -o /out/gateway ./gateway
FROM gcr.io/distroless/base-debian12
COPY --from=build /out/gateway /gateway
USER nonroot:nonroot
ENTRYPOINT ["/gateway"]
ビルド(ローカル):
docker build -t video-metadata:local ./backend
docker build -f ./backend/services/video-streaming/Dockerfile -t video-streaming:local ./backend
docker build -f ./backend/gateway/Dockerfile -t api-gateway:local ./backend
デプロイ(Kubernetes マニフェスト)
backend/deploy/base/namespace.yaml
:
apiVersion: v1
kind: Namespace
metadata:
name: video
ConfigMap/Secrets(学習用に平文。実運用は Secret Manager 等)
backend/deploy/base/configmap.yaml
:
apiVersion: v1
kind: ConfigMap
metadata:
name: app-config
namespace: video
data:
MEDIA_BASE_URL: "http://media-cdn.example" # 学習用。実際は Cloud CDN 署名URL等
REDIS_ADDR: "redis:6379"
backend/deploy/base/secret.yaml
:
apiVersion: v1
kind: Secret
metadata:
name: metadata-secrets
namespace: video
type: Opaque
stringData:
DATABASE_URL: "postgres://postgres:postgres@postgres.video.svc.cluster.local:5432/app?sslmode=disable"
メタデータ Deployment/Service(完全)
backend/deploy/base/video-metadata.yaml
:
apiVersion: apps/v1
kind: Deployment
metadata:
name: video-metadata
namespace: video
spec:
replicas: 2
selector:
matchLabels: { app: video-metadata }
template:
metadata:
labels: { app: video-metadata }
spec:
containers:
- name: server
image: gcr.io/PROJECT_ID/video-metadata:TAG
imagePullPolicy: IfNotPresent
ports: [{ containerPort: 50051, name: grpc }]
env:
- name: DATABASE_URL
valueFrom:
secretKeyRef: { name: metadata-secrets, key: DATABASE_URL }
readinessProbe:
tcpSocket: { port: 50051 }
initialDelaySeconds: 5
periodSeconds: 10
livenessProbe:
tcpSocket: { port: 50051 }
initialDelaySeconds: 10
periodSeconds: 20
---
apiVersion: v1
kind: Service
metadata:
name: video-metadata
namespace: video
spec:
selector: { app: video-metadata }
ports:
- name: grpc
port: 50051
targetPort: 50051
type: ClusterIP
ストリーミング Deployment/Service(完全)
backend/deploy/base/video-streaming.yaml
:
apiVersion: apps/v1
kind: Deployment
metadata:
name: video-streaming
namespace: video
spec:
replicas: 2
selector:
matchLabels: { app: video-streaming }
template:
metadata:
labels: { app: video-streaming }
spec:
containers:
- name: server
image: gcr.io/PROJECT_ID/video-streaming:TAG
imagePullPolicy: IfNotPresent
ports: [{ containerPort: 50052, name: grpc }]
env:
- name: MEDIA_BASE_URL
valueFrom:
configMapKeyRef: { name: app-config, key: MEDIA_BASE_URL }
readinessProbe:
tcpSocket: { port: 50052 }
initialDelaySeconds: 5
periodSeconds: 10
livenessProbe:
tcpSocket: { port: 50052 }
initialDelaySeconds: 10
periodSeconds: 20
---
apiVersion: v1
kind: Service
metadata:
name: video-streaming
namespace: video
spec:
selector: { app: video-streaming }
ports:
- name: grpc
port: 50052
targetPort: 50052
type: ClusterIP
Gateway Deployment/Service/Ingress(完全)
backend/deploy/base/api-gateway.yaml
:
apiVersion: apps/v1
kind: Deployment
metadata:
name: api-gateway
namespace: video
spec:
replicas: 2
selector:
matchLabels: { app: api-gateway }
template:
metadata:
labels: { app: api-gateway }
spec:
containers:
- name: gateway
image: gcr.io/PROJECT_ID/api-gateway:TAG
imagePullPolicy: IfNotPresent
ports: [{ containerPort: 8080 }]
env:
- name: GRPC_METADATA_ADDR
value: "video-metadata.video.svc.cluster.local:50051"
- name: GRPC_STREAMING_ADDR
value: "video-streaming.video.svc.cluster.local:50052"
readinessProbe:
httpGet: { path: /healthz, port: 8080 }
initialDelaySeconds: 5
periodSeconds: 10
livenessProbe:
httpGet: { path: /healthz, port: 8080 }
initialDelaySeconds: 10
periodSeconds: 20
---
apiVersion: v1
kind: Service
metadata:
name: api-gateway
namespace: video
spec:
type: ClusterIP
selector: { app: api-gateway }
ports:
- port: 80
targetPort: 8080
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: api
namespace: video
annotations:
kubernetes.io/ingress.class: "gce"
spec:
rules:
- http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: api-gateway
port: { number: 80 }
HPA(例)
backend/deploy/base/hpa.yaml
:
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata: { name: video-metadata, namespace: video }
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: video-metadata
minReplicas: 2
maxReplicas: 10
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 70
---
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata: { name: api-gateway, namespace: video }
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: api-gateway
minReplicas: 2
maxReplicas: 10
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 70
適用方法:
kubectl apply -f backend/deploy/base/namespace.yaml
kubectl apply -f backend/deploy/base/configmap.yaml
kubectl apply -f backend/deploy/base/secret.yaml
kubectl apply -f backend/deploy/base/video-metadata.yaml
kubectl apply -f backend/deploy/base/video-streaming.yaml
kubectl apply -f backend/deploy/base/api-gateway.yaml
kubectl apply -f backend/deploy/base/hpa.yaml
インフラ(Terraform / GCP 構築)
以下を infra/terraform
配下に作成します。
infra/terraform/variables.tf
:
variable "project" { type = string }
variable "region" { type = string default = "asia-northeast1" }
variable "zone" { type = string default = "asia-northeast1-a" }
variable "gke_num_nodes" { type = number default = 2 }
infra/terraform/providers.tf
:
terraform {
required_version = ">= 1.6.0"
required_providers {
google = { source = "hashicorp/google", version = ">= 5.0" }
}
}
provider "google" {
project = var.project
region = var.region
zone = var.zone
}
infra/terraform/main.tf
:
resource "google_project_service" "services" {
for_each = toset([
"container.googleapis.com",
"artifactregistry.googleapis.com",
"compute.googleapis.com",
"pubsub.googleapis.com",
"redis.googleapis.com",
"alloydb.googleapis.com",
"bigtableadmin.googleapis.com",
"iam.googleapis.com",
"servicenetworking.googleapis.com",
"cloudresourcemanager.googleapis.com",
"cloudkms.googleapis.com",
"monitoring.googleapis.com",
"logging.googleapis.com"
])
service = each.key
}
# VPC とサブネット
resource "google_compute_network" "vpc" {
name = "video-vpc"
auto_create_subnetworks = false
}
resource "google_compute_subnetwork" "subnet" {
name = "video-subnet"
ip_cidr_range = "10.10.0.0/16"
region = var.region
network = google_compute_network.vpc.id
secondary_ip_range {
range_name = "pods"
ip_cidr_range = "10.20.0.0/16"
}
secondary_ip_range {
range_name = "services"
ip_cidr_range = "10.30.0.0/20"
}
}
# Artifact Registry
resource "google_artifact_registry_repository" "docker" {
location = var.region
repository_id = "video-docker"
format = "DOCKER"
}
# GKE クラスタ
resource "google_container_cluster" "primary" {
name = "video-cluster"
location = var.region
network = google_compute_network.vpc.self_link
subnetwork = google_compute_subnetwork.subnet.self_link
remove_default_node_pool = true
initial_node_count = 1
ip_allocation_policy {
cluster_secondary_range_name = "pods"
services_secondary_range_name = "services"
}
}
resource "google_container_node_pool" "nodes" {
name = "default-pool"
location = var.region
cluster = google_container_cluster.primary.name
node_count = var.gke_num_nodes
node_config {
machine_type = "e2-standard-2"
oauth_scopes = [
"https://www.googleapis.com/auth/cloud-platform"
]
}
}
# Memorystore (Redis)
resource "google_redis_instance" "cache" {
name = "metadata-cache"
tier = "BASIC"
memory_size_gb = 1
region = var.region
authorized_network = google_compute_network.vpc.id
}
# Pub/Sub
resource "google_pubsub_topic" "video_events" { name = "video-events" }
# Bigtable(最小)
resource "google_bigtable_instance" "metrics" {
name = "video-metrics"
cluster {
cluster_id = "video-metrics-c1"
zone = var.zone
num_nodes = 1
storage_type = "HDD"
}
}
resource "google_bigtable_table" "view_counts" {
name = "view_counts"
instance_name = google_bigtable_instance.metrics.name
column_family { family = "cf1" }
}
# AlloyDB(最小構成。課金に注意)
resource "google_alloydb_cluster" "cluster" {
cluster_id = "video-alloydb"
location = var.region
network = google_compute_network.vpc.id
}
resource "google_alloydb_instance" "primary" {
cluster = google_alloydb_cluster.cluster.name
instance_id = "primary"
instance_type = "PRIMARY"
machine_config { cpu_count = 2 }
}
output "gke_cluster_name" { value = google_container_cluster.primary.name }
output "artifact_registry_repo" { value = google_artifact_registry_repository.docker.repository_id }
infra/terraform/outputs.tf
は上記 main.tf に含めたため不要です。
infra/terraform/terraform.tfvars
例:
project = "YOUR_GCP_PROJECT_ID"
region = "asia-northeast1"
zone = "asia-northeast1-a"
初期化と適用:
cd infra/terraform
gcloud auth application-default login
terraform init
terraform plan -out tfplan
terraform apply tfplan
cd -
GCP へデプロイ
5-1. 認証とコンテナレジストリ設定
gcloud config set project YOUR_GCP_PROJECT_ID
gcloud auth login
gcloud auth configure-docker ${REGION}-docker.pkg.dev
REGION=asia-northeast1
REPO=video-docker
PROJECT=YOUR_GCP_PROJECT_ID
# Artifact Registry(TF で作成済み)に push するタグ
META_IMG=${REGION}-docker.pkg.dev/${PROJECT}/${REPO}/video-metadata:v1
STREAM_IMG=${REGION}-docker.pkg.dev/${PROJECT}/${REPO}/video-streaming:v1
GATE_IMG=${REGION}-docker.pkg.dev/${PROJECT}/${REPO}/api-gateway:v1
# ビルド & Push(リポジトリルートで)
docker build -f backend/services/video-metadata/Dockerfile -t ${META_IMG} ./backend
docker push ${META_IMG}
docker build -f backend/services/video-streaming/Dockerfile -t ${STREAM_IMG} ./backend
docker push ${STREAM_IMG}
docker build -f backend/gateway/Dockerfile -t ${GATE_IMG} ./backend
docker push ${GATE_IMG}
5-2. GKE 認証取得と Namespace 作成
gcloud container clusters get-credentials video-cluster --region ${REGION} --project ${PROJECT}
kubectl apply -f backend/deploy/base/namespace.yaml
5-3. k8s マニフェストの置換と適用
backend/deploy/base/*
の PROJECT_ID
と TAG
を環境変数で置換して適用します。
export PROJECT=${PROJECT}
export REGION=${REGION}
export META_IMG=${REGION}-docker.pkg.dev/${PROJECT}/${REPO}/video-metadata:v1
export STREAM_IMG=${REGION}-docker.pkg.dev/${PROJECT}/${REPO}/video-streaming:v1
export GATE_IMG=${REGION}-docker.pkg.dev/${PROJECT}/${REPO}/api-gateway:v1
# 置換適用(envsubst)
cat backend/deploy/base/video-metadata.yaml | \
sed "s#gcr.io/PROJECT_ID/video-metadata:TAG#${META_IMG}#" | kubectl apply -f -
cat backend/deploy/base/video-streaming.yaml | \
sed "s#gcr.io/PROJECT_ID/video-streaming:TAG#${STREAM_IMG}#" | kubectl apply -f -
cat backend/deploy/base/api-gateway.yaml | \
sed "s#gcr.io/PROJECT_ID/api-gateway:TAG#${GATE_IMG}#" | kubectl apply -f -
kubectl apply -f backend/deploy/base/configmap.yaml
kubectl apply -f backend/deploy/base/secret.yaml
kubectl apply -f backend/deploy/base/hpa.yaml
Ingress が作成されるまで待機し、外部 IP を確認:
kubectl -n video get ingress api -w | cat
5-4. 動作確認
EXTERNAL_IP=$(kubectl -n video get ingress api -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
curl -s http://$EXTERNAL_IP/v1/videos | jq .
6. よくあるトラブルと対処
- Ingress が
address
を持たない: GKE Ingress のコントローラ有効化に時間がかかります。数分待機。リージョン/クォータ制限に注意。 - Pod が CrashLoopBackOff:
kubectl -n video logs deploy/<name>
でログ確認。環境変数(DATABASE_URL 等)や Service 名を見直し。 - AlloyDB への接続: 本 README の k8s Secret は PostgreSQL ローカル用。GCP では AlloyDB のプライベート IP に向けた接続文字列へ置換してください(VPC 内接続)。
- Artifact Registry 認証:
gcloud auth configure-docker ${REGION}-docker.pkg.dev
を再実行。