【GCP】Associate Cloud Engineer - 学習記録

Documents

Google Cloud Platform Services 基礎編
【GCP】Associate Cloud Engineer - Commands

Compute Services

• The startup script is executed every time an instance is started. By stopping an instance and launching it again will start it on a host resulting in execution of startup script.
• Marketplace provides you with pre-built images which can be launched with just a few clicks without doing any configuration
• API Explorer lets you make API calss to the service without actually integrating it within your application
  • https://developers.google.com/apis-explorer/#p/
• while querying metadata of an instance you must provide header "Metadata-Flavor: Google". This header indicates that the request was send with the intention of retrieving metadata values, rather than unintentionally from an insecure source, and allows the metadata server to return the data you requested. if you do not provide this header. the metadata server denies your request.
• default server account does not allow write permission on bucket
• Startup Script: Stop the instance, add a metadata with startup-script as key, and script content as value, and start the instance

Google App Engine (GAE)

• Scaling types
  • Automatic
    • free daily usage quota of 28 instance hours
  • Basic
    • free daily usage quota of 8 instance hours
  • Manual
    • free daily usage quota of 8 instance hours
• GAE Flexible environment lets you deploy docker based application easily without managing the underlying infrastructure.
• Maximum version per app is 15 for free app and 210 for paid app
• Increasing the cool down period will make scaling policy to wait to slightly longer period before taking next action of either scale up or down.
• Using --no-promote flag while deploying new version of the app will not automatically send traffic to new version
  • https://cloud.google.com/sdk/gcloud/reference/app/deploy
• Using traffic splitting allows for easily redirecting a small amount of traffic to the new version and can also be quickly reverted without application downtime
• Flexible environment is not support traffic migration

Managed Instance Groups (MIG)

• AutoScaling upto 1000 instances can be specified in a single request
• If you increase the size, the managed instance group(MIG) uses the current instance template to add new instances.
• The group deletes instances with a currentAction of DELETING, CREATING, and RECREATING before it deletes instances that are running with no scheduled actions.
• If the group is part of a backend service that has enabled connection draining, it can take up to 60 seconds after the connection draining duration has elapsed before the VM instance is removed or deleted.

Preemptible VMs

• up to 80% cheaper than regular instances
• highly affordable, short-lived compute instances suitable for batch jobs and fault-tolerant workloads.
• offer the same machine types and options as regular compute instances and last for up to 24 hours

Google Kubenetes Engine (GKE)

• max limit is 300,000 containers per GEK cluster
• DaemonSet is responsible for making sure that one pod is always running on each node and scales automatically depending on the count of nodes.
• https://cloud.google.com/kubernetes-engine/docs/concepts/daemonset
• https://cloud.google.com/kubernetes-engine/docs/concepts/pod
• https://cloud.google.com/kubernetes-engine/docs/concepts/statefulset
• https://kubernetes.io/docs/concepts/workloads/controllers/replicaset/
• You cannot change the machine type for an individual node pool after creation. You must creaet a new node pool in the same cluster and migrate the workload to the new pool.

Google Cloud Functions

• (Python): dependencies could be resolved by using the pip package manager's requirements.txt file or packaging local dependencies alongside your function.

Network Services

Virtual Private Cloud (VPC)

• Flow Logs are used to track network related findings.
• Flow logs capture each and every packet flowing within your network. it will record details like source IP, destination IP, source port, destination port, timestamp, etc.
• The system generated default route is 0.0.0.0/0 as it is the broadest possible range.
• VPC spans across region
• while creating a custom automatic VPC, two rules are created by default and cannot be unchecked during creation time. These are "deny all ingress and allow all egress". All other rules like allow SSH, RDP, ICMP and internal traffic are by default unchecked and are not created by default.
• Auto mode VPC can be converted to manual but the vice versa is not possible.

Subnet

• Google Cloud always reserves 4 IP addresses for every subnet you create.
  • First IP is a network address
  • Second is reserved for the default gateway
  • Third one is reserved for future use
  • Fourth one is the broadcast address

Cloud Load Balancing

• Network Load Balancing
  • To load balance UPD, TCP and SSL traffic on ports that are not supported by the TCP proxy and SSL proxy load balancers
• HTTPS load balancer
  • uses a target HTTPS proxy instead of a target HTTP proxy
• SSL Proxy Load Balancing
  • SSL connections are terminated at the load balancing layer the proxied to the closest available instance group.
• TCP Proxy Load Balancing
  • intended for non-HTTP traffic.

Firewall

• A firewall rule applicable to specific instances by using target tags and source tags.
• A route applicable to specific instances by using a tag.
• You can use the combination of IP range and tags or IP range and service account for traffic filter
• Using tag or service account will keep your firewall rule tidy as easy to understand because if you have to apply the same rule to hundreds of instances imagine going through each and every instance and adding their IP in the source. Instead, you can just use common tag or service account among VMs.
• The firewall rule priority is an integer from 0 to 65535, inclusive. Lower integers indicate higher priorities. If you do not specify a priority when creating a rule, it is assigned a priority of 1000.
• Service account and tags cannot be used together at the same time

IP Address

• Static external IP addresses
  • Assigned to a project long term until they are explicitly released from that assignment, and remain attached to a resource until they are explicitly detached.
    For VM instances, static external IP addresses remain attached to stopped instances until they are removed.
• Ephemeral external IP addresses
  • Available to VM instances and forwarding rules.
  • Remain attached to a VM instance only until the VM is stopped and restarted or the instance is terminated.
  • If an instance is stopped, any ephemeral external IP addresses that are assigned to the instance are released back into the general Compute Engine pool
    and become available for use by other projects. When a stopped instance is started again, a new ephemeral external IP address is assigned to the instance.

Choose load balancer Flow chart

Google IAM Services and Billings

App Engine Admin

Update the app configuration, enable and disable the App Engine application.
Access to update default cookie expiration but no access to update cron schedules

Roles

• Primitive Role
  • Owner
  • Editor
  • Viewer
• billing.account.update 〇 billing.account.upgrade ×
• change a billing account
  • roles/owner - Project Owner
  • roles/billing.admin - Billing Account Administrator
• hierarchy from Organization -> Folder -> Project -> Resources
• Role types
  • Primitive roles
    • include the Owner, Editor, and Viewer roles that existed prior to the introduction of Cloud IAM
  • Predefined roles
    • provide granular access for a specific service and are managed by Google Cloud
  • Custom roles
    • provide granular access according to a user-specified list of permissions
• Cloud Storage
  • Admin vs Creator vs Viewer
• Compute Engine
  • Admin vs Instance Admin
• Cloud Spanner
  • Viewer vs Database User
• BigQuery
  • User vs JobUser
• allAuthenticatedUsers will allow any Gmail or Cloud Identity supported authenticated user to access the file
• roles/bigquery.dataOwner role provides permission to read, update, and delete the dataset, can create table but not new dataset.
• storage.object.update
  • update object metadata, excluding ACLs
• storage.object.setIamPolicy
  • update object ACLs
• storage.object.create
  • add new objects to the bucket
• App Engine Admin
  • enable / disable the App Engine Application
  • update app configuration
  • ssh to VM instance

Billings

• Billing alerts are not send to Project Owner
• Cost table
  • gives a detailed tabular view of monthly costs for a given invoice.
• Report
  • Lets you view you costs at a glance to discover and analyze trends
• Cost Breakdown
  • Gives you a waterfall view of your costs
• Billing Exports
  • Used for moniting, analyzing and optimizing costs.
• Billing Cycle
  • Monthly
    • A regular monthly cycle
  • Threshold
    • When your account has accrued a certain amount of charges
• when billing costs exceed a percentage of your budget. based on the rules you set, alert notifications are sent to billing administrators and billing account users.

Storage Services

Google Cloud Storage - Bucket locations

• region
  • a specific geographic place, such as London.
• dual-region
  • a specific pair of regions, such as Finland and the Netherlands.
• multi-region
  • a large geographic area, such as the United States, that contains two or more geographic places.

Google Cloud Storage

• By default, the object will have null version as versioning was enabled after uploading the files.
• Once the bucket is created. you have no option to change the location of the bucket. The only option is to delete the bucket and create a new bucket as per requirement.
• Reliable object storage with global edge-caching service

Persistent Disk

• Persistent disks are used to share read-only data across multiple instance with high performance and no edge caching.
• In order to mount a persistent disk, you need to create a PersistentVolumeChaim
  after creating a PersistentVolume and then attach the PersistentVolumeChaim to the pod.
• Standard Persistent Disk
  • supports upto 3000 IOPS
• SSD Persistent Disk
  • supports upto 15000-60000 IOPS per instance
• Local SSD(SCSI)
  • supports upto 400000 IOPS
• Local SSD(NVME)
  • supports upto 680000 IOPS
• disks can only be upgraded in terms of size.
• the snapshot schedules can only be added to disk. Hence you need to add snapshot schedule name 'daily-backup' to the disk that is being used by app compute instance.
• Used to share read-only data across multiple instance with high performance and no edge caching

Database Services

CloudSQL

• only stores last 7 automated backups and this setting cannot be changed
• provides database migration service as well as MySQL managed DB service.

Stackdriver

• Log sinks can be exported to Cloud Storage, Pub/Sub and BigQuery only.

Stackdriver Logging

• supports BigQuery and GCS as export option for long term storage
• a native Google Cloud Solution for storing logs. The only drawback is that it stores data only for 30 days.

Stackdriver Monitoring

• helps you monitor GCP resources and create alerts for metrics such as CPU Utilization, Disk Usage

Stackdriver Error Repoting

• counts, analyzes and aggregates the crashes in your running cloud services. A centralized error management interface displays the results with sorting and filtering capabilities.

Stackdriver Trace

• a distributed tracing system that collects latency data from your applications and displays it in the Google Cloud Console. You can track how request propagate through your application and receive detailed near real-time performance insights.
• advance filter lets you filter logs with parameters like severity, resource type, request status.