Help us understand the problem. What is going on with this article?

AWS re:Invent 2019 ノート

AWS re:Invent 2019 に参加したまとめ。

感想

設計について blast radius というキーワードが目についたので個人的に頭に入れる必要がある。

キーとして train your builders というものがあり、学習型カンファレンスである re:invent 自体もそうだし、 builders' library によるベストプラクティスの提供、DeepRacer のような機械学習に対する開発者の学習サービスの提供など、aws がその機会を提供しているのも各所で感じられた。

動向としては新サービスとしては ai/ml 系、on-prem との障壁を無くすために高性能な処理と低遅延のネットワーク系の追加が主であるように見えた。

イベントについてメモ

las vegas のホテル&カジノ群で行われる大規模なイベントという感じで、隣の会場まで歩いて 20 min くらい掛かるので歩き疲れた(Bally's という交通の便が最適ではないホテルに滞在したので顕著だった)。 可能なら会場となるホテルのどれかに滞在した方がシャトルでの移動で楽になりそう。

食事は朝昼で buffet か grab & go、食べるところは大抵広かったので初日の昼だけ grab & go にしたけど後は buffet で食べた。料理はアジア料理、メキシコ料理、etc. と日によって変わっていたようで、特に食事に支障はなかった。

セッション

動画リスト

Keynotes 気になったスライド

https://youtu.be/7-31KgImGgU?list=PLhr1KZpdzukcAXSVwQ3L9cWD4QgKPCQ5S&t=1745

  • compute
    • 高性能な instances への対応 1
    • fargate for eks、eks インフラの managed
    • networking、言及なし

https://youtu.be/7-31KgImGgU?list=PLhr1KZpdzukcAXSVwQ3L9cWD4QgKPCQ5S&t=4676

  • cassandra (mcs) の追加、 wide column database

https://youtu.be/7-31KgImGgU?list=PLhr1KZpdzukcAXSVwQ3L9cWD4QgKPCQ5S&t=8142

  • ai/ml シリーズ

https://youtu.be/7-31KgImGgU?list=PLhr1KZpdzukcAXSVwQ3L9cWD4QgKPCQ5S&t=9371

Keynotes 気になったサービス

以下、雑多な参加したセッションの個人メモ

Midnight Madness

Queue up at 8 pm とあったが、開始まで二時間待つのも待機列で一時間ほど掛かるのもどちらも辛い。
ラスベガスにたどり着くまでに体力を消耗しすぎて最後まで参加できず。

ARC411-R - [REPEAT] Reducing blast radius with cell-based architectures
  • state machine
  • serverless (aws takes care av zone)
  • zone based
    • regional control plane -> zone (zone control plane, data plane)
  • well architectured framework
  • tenets
    • minimize blast radius (spike, etc.)
    • serverless (lambda, kinesis - no zone based)
    • infra as code
  • analytics cell router -> analytics cells -> KINESIS -> s3 for datastore -> (engineer, asap, ml, etc.)
  • s3 -> asap -> detections cell router -> detection cells -> findings repos
    • router
    • s3 event log -> sns -> (analytics cell router)[sqs -> (lambda -> dynamodb rules)] -> analytics cells
    • rules
      • load balancing, data segregation (source recognized source a to cell a)
      • data replication (prod cells copy to test cells), blue/green deployment
      • never look s3? spof (よくわからんかった)
    • cell
      • apache flink checkpoint and save
  • key takeways
    • small vs. large
    • design for failure
    • thinnest possible layer
CON421-R - [REPEAT] Amazon EKS under the hood

https://www.youtube.com/watch?v=7vxDWDD2YnM

  • under the hood
    • eks managed vpc | customer managed vpc
      • zone 間はどうふりわけるのか? -> 53 っぽい
    • eks celluar arch (1 cell = 1 aws account = frontend, cluster events, control plane management, etc.)
    • pipeline -> prow cluster
  • enhancement
    • managed node groups (single command, up to date, ha)
      • simplify worker node management, self managed, k8s ecosystem tooling (autoscaler)
    • vision (globally available, easy to use, production ready, cost-effective, high performance)
  • snap service mesh on eks
    • eks, envoy, switchboard, spinnaker
ARC335-R - [REPEAT] Designing for failure: Architecting resilient systems on AWS

https://www.youtube.com/watch?v=BJVzwaTiOdk

  • RPO, RTO -> mission
  • tier 1, 2, ...
  • well architected framework: shared responsibility model
  • resilient aws infra
    • fault isolation zones: cell-based, multi-az
    • microservice
    • distributed best practices: throttling, retry, circuit breaker
  • strategy: backup & restore (multi-region) high level rto, rpo
  • strategy: pilot light
  • strategy: warm standby
  • strategy: active-active
    • (one ex. read locally, write globally)
  • s3 - cross-region replication
    • replication time control (new! two weeks ago)
  • ebs - snapshot copy
  • dynamodb - multi-master, multi-region (no complexity write globally)
  • rds:- cross-region read replicas
  • inter-region vpc peering -> white paper
  • snapchat
    • 99.99% (tier0), 99.95% (tier1), 99% (tier2), 95% (tier3)
    • legacy
      • monolithic, single region -> multi-region active-active
      • repl: dynamodb streams -> stream service -> other regions, etc.
  • continuous resilience
    • disaster recovery -> chaos engineering -> continuous resilience
KYN201 - Monday Night Live

https://www.youtube.com/watch?v=GPUWATKe15E

目新しい情報はなかったらしい?
席を立つ人が目立った。

KYN202 - Keynote by Andy Jassy

https://www.youtube.com/watch?v=7-31KgImGgU

65000+ attendees
3000+ sessions

! transformation

  1. senior leadership team conviction and alignment
  2. top down aggressive goals
  3. train your builders
  4. don't let paralysis stop you before you start

97% on premises

SVS310-R1 - [REPEAT 1] Securing enterprise-grade serverless apps

https://www.youtube.com/watch?v=D2JyI7QV8c4

  • speed + security
  • security: where to start?
    • identity
      • diy identity store - vulunerable
      • hashed - attask, salt hashed - ...
      • (ok) ssecure remote password protocol (SRP)
      • Amazon Cognito (SRP, etc.)
  • x multiple identities o centralize identity management and privilege management
    • delegation: OIDC + OAUTH, federation: SAML, no long-term credentials, rbac
    • delegation
      • aws security token service -> temporary iam creds
    • jwt: identity token, access token, refresh token
    • least privilege, iam condition
  • access control to api gateway
    • with cognito
    • allowed -> jwt + context.identity
    • with lambda authorizer -> policy
    • others
      • basic request vailidations on api gateway: parameters, payload with JSON schema
      • CORS
  • access control to dynamodb (iam, cognito condition), s3 (iam, bucket policies)
    • s3 access point (multi access control)
  • lambda
    • shared responsibility model
    • iam invoke, actions, assume
  • common vulnerabilities: ddos
    • aws shield
  • common: OWASP top risks to web apps - XSS
    • aws waf filtering rules
  • common: SQL injection
    • use prepared
  • apply security at all layers
  • secure coding practices
STP203 - The modern startup: A look at today’s most successful architectures

https://www.youtube.com/watch?v=pHxuFKGALK0

  • find and eliminate blockers and wait states
    • remove human interaction, self service tools, abstraction layers allow for modular evolution
  • enable your developers
    • preparing hackathon
  • everything continuous, all the time
    • how long x takes?
  • keep your systems simple
    • gall's law
  • democratize your data
  • develop a culture of openness

  • legacy: ecs containers

    • goal: 100x clients, faster, more data
  • shift

    • docker -> serverless, tdd, devops, devs, and QA are all one
CON323-R1 - [REPEAT 1] AWS App Mesh under the hood

https://www.youtube.com/watch?v=h3syq1vbplE

  • working backward
    • customer want: routing, observability, security, transparency
    • arch
      • proxy
        • virtual node, virtual service, virtual router
  • design
    • frontend service, envoy management service, transformer
    • frontend service
      • authentication
      • authorization
      • validation
      • persistence
        • picking
          • customer configuration/metadata, relationships between resources, serializable cross-key/table -> JournalDB
    • transformer
      • mesh id -> buckets (split brain -> allow extra)
      • event-driven processing
        • reactive, efficient, creates backlog, requires reconciliation
        • level-driven processing: bounded workload (bimodal behavior), self-healing, less efficient
    • envoy
      • envoy data store
        • synthesized envoy configuration, relationships node to manifest, eventually consistency, single key -> dynamodb
      • discovery - listeners - routes - secrets - clusters - endpoints
      • management
        • actor system(connection manager, manifest manager, aws cloud map, aws certificate manager)
  • operations (canary)
    • deployment: first (container images and cloudformation templates), second, remaining
    • monitoring: rate, errors, duration (+ cloudwatch log insight)
AIM207-R5 - [REPEAT 5] Get started with AWS DeepRacer

https://github.com/aws-samples/aws-deepracer-workshops

CON325-R - [NEW LAUNCH!] [REPEAT] Enabling application-first thinking with Amazon ECS capacity providers

https://www.youtube.com/watch?v=v9xuKAdShFw

  • orchestraction ecs/eks, compute ec2/fargate
  • containers: 150%+ growth, 80% share (cloud)
  • ecs internal customers (sagemaker, etc) - dogfooding
  • infrastructure first
    • terminology: ecs cluster (namespace), ecs task (like pods), service (many tasks)
    • start ec2 instances first, and then run ecs tasks (infra first)
    • task placement: available instances -> resources (like resource limit) -> placement constraints (like affinity) -> strategy
    • infra first: available instances not found, spread placement strategies, scaling (cloudwatch metrics -> alarm auto scaling groups: metrics are based only on existing tasks and resources)
  • application first
    • tenets: applications own their requirements, infrastructure responds to application requirements
  • ECS Capacity Providers
    • ecs cluster ( ecs cp ( ec2 asg (ec2 instances)))
    • ecs cp
      • abstracts capacity
      • 0-10 /cluster
      • used by tasks/services
    • run tasks under cp without instances -> instances start -> tasks placed
    • ec2 spot / fargate spot
      • price up to 90%/80%
      • reclaimed by ec2/ reclaimed
      • instances pools/automatic
ARC349-R1 - [REPEAT 1] Beyond five 9s: Lessons from our highest available data planes

https://www.youtube.com/watch?v=2L1S0zfnIzo

  • api caller -> | (api) control plane - config -> data plane (server) | <- client
  • 10 patterns
    • insist on the highest standards
    • raise the bar for testing
      • 1000s unit tests, 100s integration tests, pre-prod env, roll-forward/back testing
    • be technically fearless
      • mitigate fear with professionalism, testing, and open-@minded scrutiny
      • aws lindbergh award
    • modeling
    • focus on blast radius
    • modular separation
    • double down on simplicity
      • saying no to many features,
    • unreasonable redundancy
      • striping, shuffle sharding
      • teeny cache
    • static stability
    • degrade gracefully
KYN204 - Keynote by Dr. Werner Vogels

https://www.youtube.com/watch?v=OdzaTbaQwTg

DOP210-L - Leadership Session: Developer Tools on AWS

https://www.youtube.com/watch?v=p9IybVJp5QM

  • external/internal, builder.tools
  • dev/test
    • delivery: dev/test -> review -> pre-production -> production
    • dev/test
      • use cloud desktops at amazon
      • use cloud9 internally
      • support aws toolkit for third party IDE
        • debugging timeline 30min -> 10 min
        • demo
  • review
    • codecommit supports approval rules
    • codeguru review/profiler
  • ci/cd
    • 2001: monolith -> 2002: 2-pizza teams (devops, full ownership, full accountability, focused innovation)
      • delivery pipelines (dev tool team)
    • delivery: source -> build -> alpha (pre-production, automated tests) -> beta (pre-production, automated integration tests, load/perf tests, browser tests) -> gamma (pre-production, automated integration tests, synthetics tests, api smoke tests) -> one az/fractional (production, synthetic monitoring) ... -> one region...
    • delivery is pessimistic
    • pipeline blockers: time windows, pipeline policies, ocverage, review, security scans, dependency updates, etc.
  • modern applications
    • monolithic -> n-tier -> micoservices & serverless
    • aws cdk
  • a look ahead
    • security
SEC209-R1 - [REPEAT 1] Getting started with AWS identity

https://www.youtube.com/watch?v=Zvz-qYYhvMk

  • before the cloud (firewall, etc.) -> in the cloud (iam)
  • iam roles: recommendation - have at least admin and readonly
  • authentication and authorization
    • iam role -> resource
      • short-term creds for iam role
      • if any policy denies -> access denied
      • if some policy allows -> allow
      • otherwise denied
  • cross account: recommendation - keep it simple
    • resource based policy
    • role trust policy

  1. on-prem が 97% 残ってるのでそれを取っていく感じの進化? 

Why not register and get more from Qiita?
  1. We will deliver articles that match you
    By following users and tags, you can catch up information on technical fields that you are interested in as a whole
  2. you can read useful information later efficiently
    By "stocking" the articles you like, you can search right away
Comments
No comments
Sign up for free and join this conversation.
If you already have a Qiita account
Why do not you register as a user and use Qiita more conveniently?
You need to log in to use this function. Qiita can be used more conveniently after logging in.
You seem to be reading articles frequently this month. Qiita can be used more conveniently after logging in.
  1. We will deliver articles that match you
    By following users and tags, you can catch up information on technical fields that you are interested in as a whole
  2. you can read useful information later efficiently
    By "stocking" the articles you like, you can search right away
ユーザーは見つかりませんでした