FinOps with IaC: Infracost

Infracost¹ について。

breakdown

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┓
┃ Project                                                         ┃ Monthly Cost ┃
┣━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╋━━━━━━━━━━━━━━┫
┃ danny-yamamoto/terraform-example-gke/env/dev/firewall           ┃ $0.00        ┃
┃ danny-yamamoto/terraform-example-gke/env/dev/gke                ┃ $408         ┃
┃ danny-yamamoto/terraform-example-gke/env/dev/instance_templates ┃ $53          ┃
┃ danny-yamamoto/terraform-example-gke/env/dev/network            ┃ $0.00        ┃
┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┻━━━━━━━━━━━━━━┛

現職において cloud の利用は定着してきた。

一方で cost の管理が問題だと感じている。

2019年の cloud 利用開始時から cost は pricing calculator（calculator）² で概算を計算している。

個人的には calculator に入力するのは効率的ではないと思う。

IaC が進んでいるのに cost 計算や管理はスプレッドシートなのか？と。

tl;dr

• Infracost は cost を HCL に基づいて算出する。
• Infracost は Terraform の official partner³ となっているため HashiCorp のライセンス変更⁴の影響は無い？
• 構成変更時の差分を確認するには breakdown 時に JSON --out-file を作成する。
• auto scaling を使う場合や常時起動しない resource は calculator が必要となる。実行時間で計算する。

Star History

競合という競合が存在しないためか GitHub star は伸び続けている。
そして Infracost は HashiCorp の official partner³。

Infracost is an official HashiCorp partner. We work together to ensure that Infracost can be used alongside Terraform Cloud/Enterprise and integrated into your workflow.

InfracostはHashiCorpの公式パートナーです。Infracost を Terraform Cloud/Enterprise と共に使用し、お客様のワークフローに統合できるように協力しています。

Get started

Install Infracost

local に CLI を install する。

vscode ➜ /workspaces/terraform-example-gke (main) $ curl -fsSL https://raw.githubusercontent.com/infracost/infracost/master/scripts/install.sh | sh
Downloading version latest of infracost-linux-arm64...

Validating checksum for infracost-linux-arm64...

Moving /tmp/infracost-linux-arm64 to /usr/local/bin/infracost (you might be asked for your password due to sudo)

Completed installing Infracost v0.10.31
vscode ➜ /workspaces/terraform-example-gke (main) $ infracost --version
Infracost v0.10.31

Get API key

事前に Infracost のサイトでアカウントを作成しておく。Free を使用した。

vscode ➜ /workspaces/terraform-example-gke (main) $ infracost auth login
We're redirecting you to our log in page, please complete that,
and return here to continue using Infracost.

If the redirect doesn't work, either:
- Use this URL:
    https://dashboard.infracost.io/login?cli_port=aaaaa&cli_state=a-b-c-d-e&cli_version=v0.10.31&os=linux&utm_source=cli

- Or log in/sign up at https://dashboard.infracost.io, copy your API key
    from Org Settings and run `infracost configure set api_key MY_KEY`

Waiting...

The API key was saved to /home/vscode/.config/infracost/credentials.yml

Your account has been authenticated. Run Infracost on your Terraform project by running:

  infracost breakdown --path=.

Show Cost estimate breakdown

main.tf の directory で command を実行する。

vscode ➜ /workspaces/terraform-example-gke (main) $ cd env/dev/
vscode ➜ /workspaces/terraform-example-gke/env/dev (main) $ infracost breakdown --path .
2024-01-05T10:18:59Z INF Enabled policies V2
2024-01-05T10:18:59Z INF Enabled tag policies
Evaluating Terraform directory at .
Detected Terraform project at firewall
Detected Terraform project at gke
Detected Terraform project at network
Detected Terraform project at instance_templates
2024-01-05T10:18:59Z INF Starting: Downloading Terraform modules
2024-01-05T10:18:59Z INF Starting: Downloading Terraform modules
2024-01-05T10:18:59Z INF Starting: Downloading Terraform modules
2024-01-05T10:18:59Z INF Starting: Downloading Terraform modules
2024-01-05T10:18:59Z INF Starting: Evaluating Terraform directory
2024-01-05T10:18:59Z INF Starting: Evaluating Terraform directory
2024-01-05T10:18:59Z INF Starting: Evaluating Terraform directory
2024-01-05T10:18:59Z INF Starting: Evaluating Terraform directory
2024-01-05T10:18:59Z WRN Input values were not provided for following Terraform variables: "variable.GOOGLE_CREDENTIALS", "variable.PROJECT_ID". Use --terraform-var-file or --terraform-var to specify them.
2024-01-05T10:19:00Z WRN Input values were not provided for following Terraform variables: "variable.GOOGLE_CREDENTIALS", "variable.PROJECT_ID". Use --terraform-var-file or --terraform-var to specify them.
2024-01-05T10:19:00Z WRN Input values were not provided for following Terraform variables: "variable.GOOGLE_CREDENTIALS", "variable.PROJECT_ID". Use --terraform-var-file or --terraform-var to specify them.
2024-01-05T10:19:01Z WRN Input values were not provided for following Terraform variables: "variable.GOOGLE_CREDENTIALS", "variable.PROJECT_ID". Use --terraform-var-file or --terraform-var to specify them.
2024-01-05T10:19:01Z INF Starting: Retrieving cloud prices to calculate costs

Project: danny-yamamoto/terraform-example-gke/env/dev/firewall
Module path: firewall

 Name           Monthly Qty  Unit  Monthly Cost 
                                                
 Project total                            $0.00 

──────────────────────────────────
Project: danny-yamamoto/terraform-example-gke/env/dev/gke
Module path: gke

 Name                                                      Monthly Qty  Unit   Monthly Cost 
                                                                                            
 module.gke.google_container_cluster.k8s                                                    
 ├─ Cluster management fee                                         730  hours        $73.00 
 └─ default_pool                                                                            
    ├─ Instance usage (Linux/UNIX, on-demand, e2-medium)           730  hours        $31.38 
    └─ Standard provisioned storage (pd-standard)                  100  GB            $5.20 
                                                                                            
 module.gke.google_container_node_pool.pool-hoge                                            
 ├─ Instance usage (Linux/UNIX, on-demand, e2-standard-4)          730  hours       $125.51 
 └─ Standard provisioned storage (pd-standard)                     100  GB            $5.20 
                                                                                            
 module.gke.google_container_node_pool.prod-default-pool                                    
 ├─ Instance usage (Linux/UNIX, on-demand, e2-medium)              730  hours        $31.38 
 └─ Standard provisioned storage (pd-standard)                     100  GB            $5.20 
                                                                                            
 module.gke.google_container_node_pool.prod-pool-hoge                                       
 ├─ Instance usage (Linux/UNIX, on-demand, e2-standard-4)          730  hours       $125.51 
 └─ Standard provisioned storage (pd-standard)                     100  GB            $5.20 
                                                                                            
 Project total                                                                      $407.57 

──────────────────────────────────
Project: danny-yamamoto/terraform-example-gke/env/dev/instance_templates
Module path: instance_templates

 Name                                              Monthly Qty  Unit  Monthly Cost 
                                                                                   
 module.instance_templates.google_compute_disk.pv                                  
 └─ Standard provisioned storage (pd-standard)           1,024  GB          $53.25 
                                                                                   
 Project total                                                              $53.25 

──────────────────────────────────
Project: danny-yamamoto/terraform-example-gke/env/dev/network
Module path: network

 Name           Monthly Qty  Unit  Monthly Cost 
                                                
 Project total                            $0.00 

 OVERALL TOTAL                          $460.82 
──────────────────────────────────
12 cloud resources were detected:
∙ 5 were estimated, 4 of which include usage-based costs, see https://infracost.io/usage-file
∙ 7 were free, rerun with --show-skipped to see details

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┓
┃ Project                                                         ┃ Monthly Cost ┃
┣━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╋━━━━━━━━━━━━━━┫
┃ danny-yamamoto/terraform-example-gke/env/dev/firewall           ┃ $0.00        ┃
┃ danny-yamamoto/terraform-example-gke/env/dev/gke                ┃ $408         ┃
┃ danny-yamamoto/terraform-example-gke/env/dev/instance_templates ┃ $53          ┃
┃ danny-yamamoto/terraform-example-gke/env/dev/network            ┃ $0.00        ┃
┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┻━━━━━━━━━━━━━━┛
vscode ➜ /workspaces/terraform-example-gke/env/dev (main) $ 

What if the configuration changes?

diff option で構成変更前後の差額を確認できる。以下のとおり official の calculator の金額差分と同じ。

• calculator

Cost Estimate Summary As of Jan 7, 2024 • 4:55 PM

	Total estimated cost
e2-standard-4	$126.55
e2-standard-8	$252.06
	$125.51

• Infracost
  • e2-standard-4 → e2-standard-8 に変更。予測は $126 の金額アップ。

diff

vscode ➜ /workspaces/terraform-example-gke/env/dev (main) $ infracost diff --path . --compare-to infracost-base.json
2024-01-07T07:54:07Z INF Enabled policies V2
2024-01-07T07:54:07Z INF Enabled tag policies
Evaluating Terraform directory at .
Detected Terraform project at firewall
Detected Terraform project at gke
Detected Terraform project at instance_templates
Detected Terraform project at network
2024-01-07T07:54:07Z INF Starting: Downloading Terraform modules
2024-01-07T07:54:07Z INF Starting: Downloading Terraform modules
2024-01-07T07:54:07Z INF Starting: Downloading Terraform modules
2024-01-07T07:54:07Z INF Starting: Downloading Terraform modules
2024-01-07T07:54:07Z INF Starting: Evaluating Terraform directory
2024-01-07T07:54:07Z INF Starting: Evaluating Terraform directory
2024-01-07T07:54:07Z INF Starting: Evaluating Terraform directory
2024-01-07T07:54:07Z INF Starting: Evaluating Terraform directory
2024-01-07T07:54:07Z WRN Input values were not provided for following Terraform variables: "variable.GOOGLE_CREDENTIALS", "variable.PROJECT_ID". Use --terraform-var-file or --terraform-var to specify them.
2024-01-07T07:54:08Z WRN Input values were not provided for following Terraform variables: "variable.GOOGLE_CREDENTIALS", "variable.PROJECT_ID". Use --terraform-var-file or --terraform-var to specify them.
2024-01-07T07:54:09Z WRN Input values were not provided for following Terraform variables: "variable.GOOGLE_CREDENTIALS", "variable.PROJECT_ID". Use --terraform-var-file or --terraform-var to specify them.
2024-01-07T07:54:09Z WRN Input values were not provided for following Terraform variables: "variable.GOOGLE_CREDENTIALS", "variable.PROJECT_ID". Use --terraform-var-file or --terraform-var to specify them.
2024-01-07T07:54:09Z INF Starting: Retrieving cloud prices to calculate costs

──────────────────────────────────
Project: danny-yamamoto/terraform-example-gke/env/dev/gke
Module path: gke

~ module.gke.google_container_node_pool.prod-pool-hoge
  +$126 ($131 → $256)

    ~ Instance usage (Linux/UNIX, on-demand, e2-standard-4 → e2-standard-8)
      +$126 ($126 → $251)

Monthly Cost change for danny-yamamoto/terraform-example-gke/env/dev/gke (Module path: gke)
Amount:  +$126 ($408 → $533)
Percent: +31%

──────────────────────────────────
Key: ~ changed, + added, - removed

12 cloud resources were detected:
∙ 5 were estimated, 4 of which include usage-based costs, see https://infracost.io/usage-file
∙ 7 were free, rerun with --show-skipped to see details

Infracost estimate: Monthly Cost will increase by $126 ↑
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┓
┃ Project                                            ┃ Cost change  ┃ New monthly Cost ┃
┣━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╋━━━━━━━━━━━━━━╋━━━━━━━━━━━━━━━━━━┫
┃ danny-yamamoto/terraform-example-gke/env/dev/gke   ┃ +$126 (+31%) ┃ $533             ┃
┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┻━━━━━━━━━━━━━━┻━━━━━━━━━━━━━━━━━━┛
vscode ➜ /workspaces/terraform-example-gke/env/dev (main) $

今回のコードはこちら。

Infracost に関する調査は以上。

1. https://www.infracost.io/ ↩

2. https://cloud.google.com/products/calculator?hl=ja ↩

3. https://www.infracost.io/docs/integrations/terraform_cloud_enterprise/ ↩ ↩²

4. https://www.hashicorp.com/blog/hashicorp-adopts-business-source-license ↩