LoginSignup
1
0

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?

More than 1 year has passed since last update.

@mnrst

Azure Functions で HttpClient のリトライ処理を試してみた

Posted at

HTTP のリターンコード 429 TooManyRequests が発生する場合、メソッドを再起的に呼び出し自分でリトライをコントロールしていました。本職はプログラマではないので、日常的にプログラミングコードを書いておらず、コードを書く必要に迫られたら生成 AI のお世話になっております。その生成 AI に TooManyRequests のリトライ処理を聞いてみたところ、メソッドを再起的に呼び出すのではなく、もっと簡単な方法を教えてくれました。そこでサンプルコードを書きながら、Azure Functions で HttpClient のリトライ処理を試してみました。

検証用 Azure Functions を作成

bash
appname=mnrazcost

func init $appname --dotnet

cd $appname

func new --name http --template 'Http Trigger'

func start

Azure Functions から Azure REST API を使用するためのサービスプリンシパルを作成

bash
az ad sp create-for-rbac \
  --display-name $appname \
  --role Contributor \
  --scopes /subscriptions/$(az account show --query id --output tsv) \
  --years 100

{
  "appId": "xxxxxxxx-0b44-413e-bf51-ea8c4657e66d",
  "displayName": "mnrazcost",
  "password": "xxxxxxxxq_7KE1wAYX.lHYa8d8-ZHlR9nZ1iibhI",
  "tenant": "xxxxxxxx-7c45-4904-b075-9bf13f4dceba"
}

環境変数にサービスプリンシパル情報をセット

bash
export MNR_AZCOST_ID=xxxxxxxx-0b44-413e-bf51-ea8c4657e66d
export MNR_AZCOST_PW=xxxxxxxxq_7KE1wAYX.lHYa8d8-ZHlR9nZ1iibhI
export MNR_AZCOST_TN=xxxxxxxx-7c45-4904-b075-9bf13f4dceba
export MNR_AZCOST_SB=$(az account show --query id --output tsv)

http.cs にコードを追加し TooManyRequests が発生する状況を作る

http.cs
using System;
using System.IO;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Extensions.Http;
using Microsoft.AspNetCore.Http;
using Microsoft.Extensions.Logging;
using Newtonsoft.Json;
using System.Net;
using System.Net.Http;
using System.Net.Http.Headers;
using Newtonsoft.Json.Linq;
using System.Threading;
using System.Text;
using System.Collections.Generic;

namespace mnrazcost
{
    public class http
    {
        [FunctionName("http")]
        public async Task<IActionResult> Run(
            [HttpTrigger(AuthorizationLevel.Function, "get", "post", Route = null)] HttpRequest req,
            ILogger log)
        {
            log.LogInformation("C# HTTP trigger function processed a request.");

            await GetAzureToken(log);

            return new OkObjectResult("Done");
        }

        private async Task GetAzureToken(ILogger log)
        {
            string MNR_AZCOST_ID = Environment.GetEnvironmentVariable("MNR_AZCOST_ID");
            string MNR_AZCOST_PW = Environment.GetEnvironmentVariable("MNR_AZCOST_PW");
            string MNR_AZCOST_TN = Environment.GetEnvironmentVariable("MNR_AZCOST_TN");
            string MNR_AZCOST_SB = Environment.GetEnvironmentVariable("MNR_AZCOST_SB");

            HttpClient httpClient = new HttpClient();
            string url = $"https://login.microsoftonline.com/{MNR_AZCOST_TN}/oauth2/v2.0/token";
            string postData = $"grant_type=client_credentials"
                + $"&scope=https://management.azure.com/.default"
                + $"&client_id={MNR_AZCOST_ID}"
                + $"&client_secret={MNR_AZCOST_PW}";
            StringContent content = new StringContent(postData, Encoding.UTF8,
                "application/x-www-form-urlencoded");
            var response = await httpClient.PostAsync(url, content);
            if (response.StatusCode == HttpStatusCode.OK)
            {
                JObject token = JObject.Parse(await response.Content.ReadAsStringAsync());
                httpClient.DefaultRequestHeaders.Add("Authorization", "Bearer " + token.GetValue("access_token"));
                List<Task> tasks = new List<Task>();
                for (int i = 0; i < 20; i++)
                {
                    tasks.Add(GetAzureCost(httpClient, log, MNR_AZCOST_SB));
                }
                await Task.WhenAll(tasks);
            }
            else
            {
                log.LogInformation($"response.StatusCode: {response.StatusCode}");
            }
        }

        private async Task GetAzureCost(HttpClient httpClient, ILogger log, string subscriptionId)
        {
            string url = $"https://management.azure.com/subscriptions/{subscriptionId}/providers/Microsoft.CostManagement/query?api-version=2023-03-01";
            string postData = @"{
                ""type"": ""Usage"",
                ""timeframe"": ""MonthToDate"",
                ""dataset"": {
                    ""granularity"": ""None"",
                    ""aggregation"": {
                        ""totalCost"": {
                            ""name"": ""PreTaxCost"",
                            ""function"": ""Sum""
                        }
                    }
                }
            }";
            StringContent content = new StringContent(postData, Encoding.UTF8,
                "application/json");
            var response = await httpClient.PostAsync(url, content);
            if (response.StatusCode == HttpStatusCode.OK)
            {
                JObject data = JObject.Parse(await response.Content.ReadAsStringAsync());
                var PreTaxCost = data["properties"]["rows"][0][0];
                var Currency = data["properties"]["rows"][0][1];
                log.LogInformation($"{Currency} {PreTaxCost}");
            }
            else
            {
                log.LogInformation($"response.StatusCode: {response.StatusCode}");
            }
        }
    }
}

TooManyRequests が発生した状況

bash
[2023-09-22T23:59:48.419Z] C# HTTP trigger function processed a request.
[2023-09-22T23:59:49.612Z] response.StatusCode: TooManyRequests
[2023-09-22T23:59:49.734Z] response.StatusCode: TooManyRequests
[2023-09-22T23:59:49.751Z] response.StatusCode: TooManyRequests
[2023-09-22T23:59:49.755Z] response.StatusCode: TooManyRequests
[2023-09-22T23:59:49.782Z] response.StatusCode: TooManyRequests
[2023-09-22T23:59:49.800Z] response.StatusCode: TooManyRequests
[2023-09-22T23:59:49.867Z] response.StatusCode: TooManyRequests
[2023-09-22T23:59:49.872Z] response.StatusCode: TooManyRequests
[2023-09-22T23:59:49.875Z] response.StatusCode: TooManyRequests
[2023-09-22T23:59:49.877Z] response.StatusCode: TooManyRequests
[2023-09-22T23:59:49.927Z] response.StatusCode: TooManyRequests
[2023-09-22T23:59:50.225Z] response.StatusCode: TooManyRequests
[2023-09-22T23:59:50.225Z] response.StatusCode: TooManyRequests
[2023-09-22T23:59:50.225Z] response.StatusCode: TooManyRequests
[2023-09-22T23:59:50.225Z] response.StatusCode: TooManyRequests
[2023-09-22T23:59:50.225Z] response.StatusCode: TooManyRequests
[2023-09-22T23:59:50.226Z] response.StatusCode: TooManyRequests
[2023-09-22T23:59:50.234Z] response.StatusCode: TooManyRequests
[2023-09-22T23:59:50.283Z] response.StatusCode: TooManyRequests
[2023-09-22T23:59:50.414Z] response.StatusCode: TooManyRequests
[2023-09-22T23:59:50.451Z] Executed 'http' (Succeeded, Id=06b69b53-b4ca-48dd-8bf7-671b4f5281dc, Duration=2023ms)

Polly ライブラリを導入

bash
dotnet add package Polly
dotnet add package Polly.Extensions.Http

HttpClient のリトライ処理を追加した http.cs

数行程度のコード追加でリトライ処理が実現できるようです。

http.cs
using System;
using System.IO;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Extensions.Http;
using Microsoft.AspNetCore.Http;
using Microsoft.Extensions.Logging;
using Newtonsoft.Json;
using System.Net;
using System.Net.Http;
using System.Net.Http.Headers;
using Newtonsoft.Json.Linq;
using System.Threading;
using System.Text;
using System.Collections.Generic;
using Polly;
using Polly.Extensions.Http;

namespace mnrazcost
{
    public class http
    {
        [FunctionName("http")]
        public async Task<IActionResult> Run(
            [HttpTrigger(AuthorizationLevel.Function, "get", "post", Route = null)] HttpRequest req,
            ILogger log)
        {
            log.LogInformation("C# HTTP trigger function processed a request.");

            await GetAzureToken(log);

            return new OkObjectResult("Done");
        }

        private async Task GetAzureToken(ILogger log)
        {
            string MNR_AZCOST_ID = Environment.GetEnvironmentVariable("MNR_AZCOST_ID");
            string MNR_AZCOST_PW = Environment.GetEnvironmentVariable("MNR_AZCOST_PW");
            string MNR_AZCOST_TN = Environment.GetEnvironmentVariable("MNR_AZCOST_TN");
            string MNR_AZCOST_SB = Environment.GetEnvironmentVariable("MNR_AZCOST_SB");

            HttpClient httpClient = new HttpClient();
            string url = $"https://login.microsoftonline.com/{MNR_AZCOST_TN}/oauth2/v2.0/token";
            string postData = $"grant_type=client_credentials"
                + $"&scope=https://management.azure.com/.default"
                + $"&client_id={MNR_AZCOST_ID}"
                + $"&client_secret={MNR_AZCOST_PW}";
            StringContent content = new StringContent(postData, Encoding.UTF8,
                "application/x-www-form-urlencoded");
            var response = await httpClient.PostAsync(url, content);
            if (response.StatusCode == HttpStatusCode.OK)
            {
                JObject token = JObject.Parse(await response.Content.ReadAsStringAsync());
                httpClient.DefaultRequestHeaders.Add("Authorization", "Bearer " + token.GetValue("access_token"));
                List<Task> tasks = new List<Task>();
                for (int i = 0; i < 20; i++)
                {
                    tasks.Add(GetAzureCost(httpClient, log, MNR_AZCOST_SB));
                }
                await Task.WhenAll(tasks);
            }
            else
            {
                log.LogInformation($"response.StatusCode: {response.StatusCode}");
            }
        }

        private async Task GetAzureCost(HttpClient httpClient, ILogger log, string subscriptionId)
        {
            var retryPolicy = Policy.HandleResult<HttpResponseMessage>(response =>
            {
                return response.StatusCode == HttpStatusCode.TooManyRequests;
            }).WaitAndRetryAsync(3, retryAttempt => TimeSpan.FromSeconds(Math.Pow(2, retryAttempt)));

            string url = $"https://management.azure.com/subscriptions/{subscriptionId}/providers/Microsoft.CostManagement/query?api-version=2023-03-01";
            string postData = @"{
                ""type"": ""Usage"",
                ""timeframe"": ""MonthToDate"",
                ""dataset"": {
                    ""granularity"": ""None"",
                    ""aggregation"": {
                        ""totalCost"": {
                            ""name"": ""PreTaxCost"",
                            ""function"": ""Sum""
                        }
                    }
                }
            }";
            StringContent content = new StringContent(postData, Encoding.UTF8,
                "application/json");
            // var response = await httpClient.PostAsync(url, content);
            var response = await retryPolicy.ExecuteAsync(() =>
            {
                return httpClient.PostAsync(url, content);
            });
            if (response.StatusCode == HttpStatusCode.OK)
            {
                JObject data = JObject.Parse(await response.Content.ReadAsStringAsync());
                var PreTaxCost = data["properties"]["rows"][0][0];
                var Currency = data["properties"]["rows"][0][1];
                log.LogInformation($"{Currency} {PreTaxCost}");
            }
            else
            {
                log.LogInformation($"response.StatusCode: {response.StatusCode}");
            }
        }
    }
}

リトライ処理の結果

Azure REST API の Microsoft.CostManagement/query が頻繁に TooManyRequests を出すのでサンプルで使用しましたが、多少の効果はありそうです。また、リトライ回数を増やすと処理全体の時間がかかるので、確実にリトライはしているものと思われます。もっとほど良い TooManyRequests の再現環境があると良さそうです。

bash
[2023-09-23T00:04:12.850Z] C# HTTP trigger function processed a request.
[2023-09-23T00:04:15.646Z] JPY 5430.4168160589425
[2023-09-23T00:04:15.794Z] JPY 5430.4168160589425
[2023-09-23T00:04:15.805Z] JPY 5430.4168160589425
[2023-09-23T00:04:15.875Z] JPY 5430.4168160589425
[2023-09-23T00:04:29.859Z] response.StatusCode: TooManyRequests
[2023-09-23T00:04:29.943Z] response.StatusCode: TooManyRequests
[2023-09-23T00:04:29.956Z] response.StatusCode: TooManyRequests
[2023-09-23T00:04:30.287Z] response.StatusCode: TooManyRequests
[2023-09-23T00:04:30.287Z] response.StatusCode: TooManyRequests
[2023-09-23T00:04:30.637Z] response.StatusCode: TooManyRequests
[2023-09-23T00:04:30.682Z] response.StatusCode: TooManyRequests
[2023-09-23T00:04:30.701Z] response.StatusCode: TooManyRequests
[2023-09-23T00:04:31.047Z] response.StatusCode: TooManyRequests
[2023-09-23T00:04:31.134Z] response.StatusCode: TooManyRequests
[2023-09-23T00:04:31.160Z] response.StatusCode: TooManyRequests
[2023-09-23T00:04:31.250Z] response.StatusCode: TooManyRequests
[2023-09-23T00:04:31.409Z] response.StatusCode: TooManyRequests
[2023-09-23T00:04:31.532Z] response.StatusCode: TooManyRequests
[2023-09-23T00:04:31.926Z] response.StatusCode: TooManyRequests
[2023-09-23T00:04:31.926Z] response.StatusCode: TooManyRequests
[2023-09-23T00:04:31.960Z] Executed 'http' (Succeeded, Id=1d24fa94-b7d7-4322-ad32-a0e5dab6c86a, Duration=19102ms)

参考

1
0
0

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
Sign upLogin
1
0

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?