AWS re:Invent 2019 に参加したまとめ。

感想

設計について blast radius というキーワードが目についたので個人的に頭に入れる必要がある。

キーとして train your builders というものがあり、学習型カンファレンスである re:invent 自体もそうだし、 builders' library によるベストプラクティスの提供、DeepRacer のような機械学習に対する開発者の学習サービスの提供など、aws がその機会を提供しているのも各所で感じられた。

動向としては新サービスとしては ai/ml 系、on-prem との障壁を無くすために高性能な処理と低遅延のネットワーク系の追加が主であるように見えた。

イベントについてメモ

las vegas のホテル＆カジノ群で行われる大規模なイベントという感じで、隣の会場まで歩いて 20 min くらい掛かるので歩き疲れた（Bally's という交通の便が最適ではないホテルに滞在したので顕著だった）。可能なら会場となるホテルのどれかに滞在した方がシャトルでの移動で楽になりそう。

食事は朝昼で buffet か grab & go、食べるところは大抵広かったので初日の昼だけ grab & go にしたけど後は buffet で食べた。料理はアジア料理、メキシコ料理、etc. と日によって変わっていたようで、特に食事に支障はなかった。

セッション

動画リスト

Keynotes 気になったスライド

compute
- 高性能な instances への対応 ¹
- fargate for eks、eks インフラの managed
- networking、言及なし

cassandra (mcs) の追加、 wide column database

ai/ml シリーズ

breaking through barriers ¹
- outposts (on-prem workload)
- local zones (low latency to local end-users)
- wavelength (5G)

Keynotes 気になったサービス

Amazon CodeGuru
- Java のみっぽいので色々な言語の対応が待ち遠しい
Amazon Kendra
- 企業内ドキュメントの検索は皆困っていて需要があるんだろう
Amazon Builders' Library
- 読む必要がある

以下、雑多な参加したセッションの個人メモ

Midnight Madness

Queue up at 8 pm とあったが、開始まで二時間待つのも待機列で一時間ほど掛かるのもどちらも辛い。
ラスベガスにたどり着くまでに体力を消耗しすぎて最後まで参加できず。

ARC411-R - [REPEAT] Reducing blast radius with cell-based architectures

state machine
serverless (aws takes care av zone)
zone based
- regional control plane -> zone (zone control plane, data plane)
well architectured framework
tenets
- minimize blast radius (spike, etc.)
- serverless (lambda, kinesis - no zone based)
- infra as code
analytics cell router -> analytics cells -> KINESIS -> s3 for datastore -> (engineer, asap, ml, etc.)
s3 -> asap -> detections cell router -> detection cells -> findings repos
- router
  - s3 event log -> sns -> (analytics cell router)[sqs -> (lambda -> dynamodb rules)] -> analytics cells
  - rules
    - load balancing, data segregation (source recognized source a to cell a)
    - data replication (prod cells copy to test cells), blue/green deployment
    - never look s3? spof (よくわからんかった）
  - cell
    - apache flink checkpoint and save
key takeways
- small vs. large
- design for failure
- thinnest possible layer

CON421-R - [REPEAT] Amazon EKS under the hood

under the hood
- eks managed vpc | customer managed vpc
  - zone 間はどうふりわけるのか？ -> 53 っぽい
- eks celluar arch (1 cell = 1 aws account = frontend, cluster events, control plane management, etc.)
- pipeline -> prow cluster
enhancement
- managed node groups (single command, up to date, ha)
  - simplify worker node management, self managed, k8s ecosystem tooling (autoscaler)
- vision (globally available, easy to use, production ready, cost-effective, high performance)
snap service mesh on eks
- eks, envoy, switchboard, spinnaker

ARC335-R - [REPEAT] Designing for failure: Architecting resilient systems on AWS

RPO, RTO -> mission
tier 1, 2, ...
well architected framework: shared responsibility model
resilient aws infra
- fault isolation zones: cell-based, multi-az
- microservice
- distributed best practices: throttling, retry, circuit breaker
strategy: backup & restore (multi-region) high level rto, rpo
strategy: pilot light
strategy: warm standby
strategy: active-active
- (one ex. read locally, write globally)
s3 - cross-region replication
- replication time control (new! two weeks ago)
ebs - snapshot copy
dynamodb - multi-master, multi-region (no complexity write globally)
rds:- cross-region read replicas
inter-region vpc peering -> white paper
snapchat
- 99.99% (tier0), 99.95% (tier1), 99% (tier2), 95% (tier3)
- legacy
  - monolithic, single region -> multi-region active-active
  - repl: dynamodb streams -> stream service -> other regions, etc.
continuous resilience
- disaster recovery -> chaos engineering -> continuous resilience

KYN201 - Monday Night Live

目新しい情報はなかったらしい？
席を立つ人が目立った。

KYN202 - Keynote by Andy Jassy

65000+ attendees
3000+ sessions

! transformation

senior leadership team conviction and alignment
top down aggressive goals
train your builders
don't let paralysis stop you before you start

97% on premises

user: goldman saches
comupute (nitro, chip)
m6g, r6g, c6g instances (gravition chip)
inf1 (infrantia chip)
containers
fargate for eks https://aws.amazon.com/jp/blogs/aws/amazon-eks-on-aws-fargate-now-generally-available/
- serverless
data silos -> data lake
- s3 access point https://aws.amazon.com/jp/blogs/aws/easily-manage-shared-data-sets-with-amazon-s3-access-points/
- redshift ra3 instances with managed storage https://aws.amazon.com/jp/blogs/aws/amazon-redshift-update-next-generation-compute-instances-and-managed-analytics-optimized-storage/
- elasticsearch service (ultrawarm) https://aws.amazon.com/jp/blogs/aws/announcing-ultrawarm-preview-for-amazon-elasticsearch-service/
database
- managed cassandra service (wide column) https://aws.amazon.com/jp/blogs/aws/new-amazon-managed-apache-cassandra-service-mcs/
ml
- usecase: health care
- sage maker studio, notebooks, experiments, debugger, model monitor, autopilot
- fraud detector
- codeguru (一番盛り上がった）
contact lens for amazon connect
kendra
break on-prem barriers
outposts GA
- native aws or vmware cloud on aws
local zones
5G
- 8 capabilities
- wavelength

SVS310-R1 - [REPEAT 1] Securing enterprise-grade serverless apps

speed + security
security: where to start?
- identity
  - diy identity store - vulunerable
  - hashed - attask, salt hashed - ...
  - (ok) ssecure remote password protocol (SRP)
  - Amazon Cognito (SRP, etc.)
x multiple identities o centralize identity management and privilege management
- delegation: OIDC + OAUTH, federation: SAML, no long-term credentials, rbac
- delegation
  - aws security token service -> temporary iam creds
- jwt: identity token, access token, refresh token
- least privilege, iam condition
access control to api gateway
- with cognito
- allowed -> jwt + context.identity
- with lambda authorizer -> policy
- others
  - basic request vailidations on api gateway: parameters, payload with JSON schema
  - CORS
access control to dynamodb (iam, cognito condition), s3 (iam, bucket policies)
- s3 access point (multi access control)
lambda
- shared responsibility model
- iam invoke, actions, assume
common vulnerabilities: ddos
- aws shield
common: OWASP top risks to web apps - XSS
- aws waf filtering rules
common: SQL injection
- use prepared
apply security at all layers
secure coding practices
- accessing db creds: x hard coded, ! env or config file, o aws secrets manager, o iam authentication for amazon rds
- data api for amazon aurora serverless https://docs.aws.amazon.com/ja_jp/AmazonRDS/latest/AuroraUserGuide/data-api.html
- monolitic functions -> microservies + event-driven architectures (small functions)
- owasp secure coding practices

STP203 - The modern startup: A look at today’s most successful architectures

find and eliminate blockers and wait states
- remove human interaction, self service tools, abstraction layers allow for modular evolution
enable your developers
- preparing hackathon
everything continuous, all the time
- how long x takes?
keep your systems simple
- gall's law
democratize your data
develop a culture of openness
legacy: ecs containers
- goal: 100x clients, faster, more data
shift
- docker -> serverless, tdd, devops, devs, and QA are all one

CON323-R1 - [REPEAT 1] AWS App Mesh under the hood

working backward
- customer want: routing, observability, security, transparency
- arch
  - proxy
  - virtual node, virtual service, virtual router
design
- frontend service, envoy management service, transformer
- frontend service
  - authentication
  - authorization
  - validation
  - persistence
    - picking
      - customer configuration/metadata, relationships between resources, serializable cross-key/table -> JournalDB
- transformer
  - mesh id -> buckets (split brain -> allow extra)
  - event-driven processing
    - reactive, efficient, creates backlog, requires reconciliation
    - level-driven processing: bounded workload (bimodal behavior), self-healing, less efficient
- envoy
  - envoy data store
    - synthesized envoy configuration, relationships node to manifest, eventually consistency, single key -> dynamodb
  - discovery - listeners - routes - secrets - clusters - endpoints
  - management
    - actor system(connection manager, manifest manager, aws cloud map, aws certificate manager)
operations (canary)
- deployment: first (container images and cloudformation templates), second, remaining
- monitoring: rate, errors, duration (+ cloudwatch log insight)

AIM207-R5 - [REPEAT 5] Get started with AWS DeepRacer

CON325-R - [NEW LAUNCH!] [REPEAT] Enabling application-first thinking with Amazon ECS capacity providers

orchestraction ecs/eks, compute ec2/fargate
containers: 150%+ growth, 80% share (cloud)
ecs internal customers (sagemaker, etc) - dogfooding
infrastructure first
- terminology: ecs cluster (namespace), ecs task (like pods), service (many tasks)
- start ec2 instances first, and then run ecs tasks (infra first)
- task placement: available instances -> resources (like resource limit) -> placement constraints (like affinity) -> strategy
- infra first: available instances not found, spread placement strategies, scaling (cloudwatch metrics -> alarm auto scaling groups: metrics are based only on existing tasks and resources)
application first
- tenets: applications own their requirements, infrastructure responds to application requirements
ECS Capacity Providers
- ecs cluster ( ecs cp ( ec2 asg (ec2 instances)))
- ecs cp
  - abstracts capacity
  - 0-10 /cluster
  - used by tasks/services
- run tasks under cp without instances -> instances start -> tasks placed
- ec2 spot / fargate spot
  - price up to 90%/80%
  - reclaimed by ec2/ reclaimed
  - instances pools/automatic

ARC349-R1 - [REPEAT 1] Beyond five 9s: Lessons from our highest available data planes

api caller -> | (api) control plane - config -> data plane (server) | <- client
10 patterns
- insist on the highest standards
- raise the bar for testing
  - 1000s unit tests, 100s integration tests, pre-prod env, roll-forward/back testing
- be technically fearless
  - mitigate fear with professionalism, testing, and open-@minded scrutiny
  - aws lindbergh award
- modeling
- focus on blast radius
  - shuffle sharding https://aws.amazon.com/jp/blogs/architecture/shuffle-sharding-massive-and-magical-fault-isolation/
- modular separation
- double down on simplicity
  - saying no to many features,
- unreasonable redundancy
  - striping, shuffle sharding
  - teeny cache
- static stability
- degrade gracefully

KYN204 - Keynote by Dr. Werner Vogels

DOP210-L - Leadership Session: Developer Tools on AWS

external/internal, builder.tools
dev/test
- delivery: dev/test -> review -> pre-production -> production
- dev/test
  - use cloud desktops at amazon
  - use cloud9 internally
  - support aws toolkit for third party IDE
    - debugging timeline 30min -> 10 min
    - demo
review
- codecommit supports approval rules
- codeguru review/profiler
ci/cd
- 2001: monolith -> 2002: 2-pizza teams (devops, full ownership, full accountability, focused innovation)
  - delivery pipelines (dev tool team)
- delivery: source -> build -> alpha (pre-production, automated tests) -> beta (pre-production, automated integration tests, load/perf tests, browser tests) -> gamma (pre-production, automated integration tests, synthetics tests, api smoke tests) -> one az/fractional (production, synthetic monitoring) ... -> one region...
- delivery is pessimistic
- pipeline blockers: time windows, pipeline policies, ocverage, review, security scans, dependency updates, etc.
modern applications
- monolithic -> n-tier -> micoservices & serverless
- aws cdk
a look ahead
- security

SEC209-R1 - [REPEAT 1] Getting started with AWS identity

before the cloud (firewall, etc.) -> in the cloud (iam)
iam roles: recommendation - have at least admin and readonly
authentication and authorization
- iam role -> resource
  - short-term creds for iam role
  - if any policy denies -> access denied
  - if some policy allows -> allow
  - otherwise denied
cross account: recommendation - keep it simple
- resource based policy
- role trust policy

on-prem が 97% 残ってるのでそれを取っていく感じの進化？ ↩ ↩²