処理

0. 前提

記事「Azure AI Foundry から gpt-4o-miniをファインチューニング」の「0. 前提」および「1. 訓練/検証ファイル準備」まで完了していることが前提です。

1. 環境変数設定

あとで確認

後に作成するPythonから読み込むために.envファイルを作成して、以下の２つの環境変数を設定します。

.env

AZURE_OPENAI_API_KEY=<API Key>
AZURE_OPENAI_ENDPOINT=<End Point>

2. Pythonプログラム作成

Jupyterで実行しています。

環境

種類	バージョン	備考
Python(Pyenv)	3.10.2	古いのに特に意味はないです。たまたますぐ使えた環境
Ubuntu(WSL2)	22.04.5 LTS
Poetry	1.7.1

Python 追加パッケージ

種類	バージョン	備考
openai	1.63.2
requests	2.32.3
jupyterlab	4.3.5

Pythonプログラム

1. パッケージインポート

パッケージインポート

import json

from openai import AzureOpenAI

2. JSONLファイル読込

ファイルは「Azure AI Foundry から gpt-4o-miniをファインチューニング」で使ったものと同じです。

JSONLファイル読込

DIR = './sample_data/'
TRAIN_FILE = DIR + 'train.jsonl'
VALIDATION_FILE = DIR + 'validation.jsonl'

def read_jsonl(file_name: str) -> list:
    with open(file_name, 'r', encoding='utf-8') as f:
        dataset = [json.loads(line) for line in f]

    # Dataset stats
    print(f"Number of examples in {file_name}: {len(dataset)}")
    print(f"First example in {file_name}:")
    for message in dataset[0]["messages"]:
        print(message)
    return dataset

train_dataset = read_jsonl(TRAIN_FILE)
print("----------")
validation_dataset = read_jsonl(VALIDATION_FILE)

Number of examples in ./sample_data/train.jsonl: 80
First example in ./sample_data/train.jsonl:
{'role': 'system', 'content': '情報を見つけるのに役立つ AI アシスタントです。'}
{'role': 'user', 'content': '福原洋平さんの職業は何ですか？'}
{'role': 'assistant', 'content': '福原洋平さんは都市計画コンサルタント兼環境デザイナーとして活躍しています。  '}

----------
Number of examples in ./sample_data/validation.jsonl: 20
First example in ./sample_data/validation.jsonl:
{'role': 'system', 'content': '情報を見つけるのに役立つ AI アシスタントです。'}
{'role': 'user', 'content': '福原洋平さんがプロジェクトで最初に行う作業は？'}
{'role': 'assistant', 'content': '現地調査と住民の意見収集を最初に行います。  '}

3. ファイルをアップロード

ファイルをアップロードします。面倒だったので、End Point と API Keyは環境変数から読み込まず固定値渡しています。
以下を参考にしてください。

ファイルアップロード

client = AzureOpenAI(
  azure_endpoint = "<End Point>",
  api_key = "<API Key>",
  api_version = "2024-08-01-preview"  # This API version or later is required to access seed/events/checkpoint features
)

# Upload the training and validation dataset files to Azure OpenAI with the SDK.
training_response = client.files.create(
    file = open(TRAIN_FILE, "rb"), purpose="fine-tune"
)

validation_response = client.files.create(
    file = open(VALIDATION_FILE, "rb"), purpose="fine-tune"
)

print("Training file ID:", training_response.id)
print("Validation file ID:", validation_response.id)

Training file ID: file-79600873356049e084973a13ea571a40
Validation file ID: file-637528dab95e418f97956a625a424960

Azure AI Foundryからもアップロードしたファイルを確認可能(実行後のresults.csvを画面上見えているが、この時点では未作成)。

4. ファインチューニング実行

まずはジョブ実行をします。

ファインチューニング実行

# Submit fine-tuning training job
response = client.fine_tuning.jobs.create(
    training_file = training_response.id,
    validation_file = validation_response.id,
    model = "gpt-4o-mini", # Enter base model name. Note that in Azure OpenAI the model name contains dashes and cannot contain dot/period characters.
    seed = 105 # seed parameter controls reproducibility of the fine-tuning job. If no seed is specified one will be generated automatically.
)

# responseはこの後他の変数を入れるので退避
job_id = response.id

# You can use the job ID to monitor the status of the fine-tuning job.
# The fine-tuning job will take some time to start and complete.
print("Job ID:", job_id)
print("Status:", response.status)
print(response.model_dump_json(indent=2))

Job ID: ftjob-d78af615cda84674b541c026e764bcb7
Status: pending
{
  "id": "ftjob-d78af615cda84674b541c026e764bcb7",
  "created_at": 1740444480,
  "error": null,
  "fine_tuned_model": null,
  "finished_at": null,
  "hyperparameters": {
    "batch_size": -1,
    "learning_rate_multiplier": 1.0,
    "n_epochs": -1
  },
  "model": "gpt-4o-mini-2024-07-18",
  "object": "fine_tuning.job",
  "organization_id": null,
  "result_files": null,
  "seed": 105,
  "status": "pending",
  "trained_tokens": null,
  "training_file": "file-79600873356049e084973a13ea571a40",
  "validation_file": "file-637528dab95e418f97956a625a424960",
  "estimated_finish": 1740445382,
  "integrations": null,
  "method": null
}

5. ジョブ実行ステータス確認

ジョブ実行をJupyter上で確認します。

# Track training status
from IPython.display import clear_output
import time

start_time = time.time()

# Get the status of our fine-tuning job.
response = client.fine_tuning.jobs.retrieve(job_id)

status = response.status

# If the job isn't done yet, poll it every 10 seconds.
while status not in ["succeeded", "failed"]:
    time.sleep(10)

    response = client.fine_tuning.jobs.retrieve(job_id)
    print(response.model_dump_json(indent=2))
    print("Elapsed time: {} minutes {} seconds".format(int((time.time() - start_time) // 60), int((time.time() - start_time) % 60)))
    status = response.status
    print(f'Status: {status}')
    clear_output(wait=True)

print(f'Fine-tuning job {job_id} finished with status: {status}')

# List all fine-tuning jobs for this resource.
print('Checking other fine-tune jobs for this resource.')
response = client.fine_tuning.jobs.list()
print(f'Found {len(response.data)} fine-tune jobs.')

ジョブが終わると以下のような出力がされます。

Fine-tuning job ftjob-d78af615cda84674b541c026e764bcb7 finished with status: succeeded
Checking other fine-tune jobs for this resource.
Found 1 fine-tune jobs.

6. 実行結果確認

実行結果を確認します。
グラフィカルに確認したい場合には、当然Azure AI Foundryからの方が見やすいです

# イベント確認
response = client.fine_tuning.jobs.list_events(fine_tuning_job_id=job_id, limit=10)
print(response.model_dump_json(indent=2))

# チェックポイント確認
response = client.fine_tuning.jobs.checkpoints.list(job_id)
print(response.model_dump_json(indent=2))

# モデル情報確認
response = client.fine_tuning.jobs.retrieve(job_id)
print(response.model_dump_json(indent=2))
fine_tuned_model = response.fine_tuned_model

イベント

{
  "data": [
    {
      "id": "ftevent-f7b41a245cf24b8a8c0fa5f189ca54c9",
      "created_at": 1740447221,
      "level": "info",
      "message": "Training tokens billed: 18000",
      "object": "fine_tuning.job.event",
      "data": null,
      "type": "message"
    },
    {
      "id": "ftevent-28ff0c1de65f47a8b900f20073aaeccc",
      "created_at": 1740447221,
      "level": "info",
      "message": "Model Evaluation Passed.",
      "object": "fine_tuning.job.event",
      "data": null,
      "type": "message"
    },
    {
      "id": "ftevent-89a23c27377e4066b96e684d64408bf8",
      "created_at": 1740447221,
      "level": "info",
      "message": "Completed results file: file-3bf8c4ac95124fcf92947d63183685b4",
      "object": "fine_tuning.job.event",
      "data": null,
      "type": "message"
    },
    {
      "id": "ftevent-f3fb4f3718fb4e908a8c4868003ad603",
      "created_at": 1740447217,
      "level": "info",
      "message": "Postprocessing started.",
      "object": "fine_tuning.job.event",
      "data": null,
      "type": "message"
    },
    {
      "id": "ftevent-93570f3440b941968d81d717815b08c3",
      "created_at": 1740447172,
      "level": "info",
      "message": "Job succeeded.",
      "object": "fine_tuning.job.event",
      "data": null,
      "type": "message"
    },
    {
      "id": "ftevent-008dd553b129a1e008dd553b129a1e00",
      "created_at": 1740446636,
      "level": "info",
      "message": "Step 240: training loss=0.11896172910928726",
      "object": "fine_tuning.job.event",
      "data": {
        "step": 240,
        "train_loss": 0.11896172910928726,
        "train_mean_token_accuracy": 0.9583333134651184,
        "valid_loss": 0.7514890943254743,
        "valid_mean_token_accuracy": 0.75,
        "full_valid_loss": 0.7561612964140302,
        "full_valid_mean_token_accuracy": 0.7859922178988327
      },
      "type": "metrics"
    },
    {
      "id": "ftevent-008dd553b0ca43d008dd553b0ca43d00",
      "created_at": 1740446626,
      "level": "info",
      "message": "Step 230: training loss=0.20184214413166046",
      "object": "fine_tuning.job.event",
      "data": {
        "step": 230,
        "train_loss": 0.20184214413166046,
        "train_mean_token_accuracy": 0.9166666865348816,
        "valid_loss": 1.520263459947374,
        "valid_mean_token_accuracy": 0.5185185185185185
      },
      "type": "metrics"
    },
    {
      "id": "ftevent-008dd553b06ae5c008dd553b06ae5c00",
      "created_at": 1740446616,
      "level": "info",
      "message": "Step 220: training loss=0.4186785817146301",
      "object": "fine_tuning.job.event",
      "data": {
        "step": 220,
        "train_loss": 0.4186785817146301,
        "train_mean_token_accuracy": 0.875,
        "valid_loss": 0.7736611366271973,
        "valid_mean_token_accuracy": 0.7857142857142857
      },
      "type": "metrics"
    },
    {
      "id": "ftevent-008dd553b00b87b008dd553b00b87b00",
      "created_at": 1740446606,
      "level": "info",
      "message": "Step 210: training loss=0.29435476660728455",
      "object": "fine_tuning.job.event",
      "data": {
        "step": 210,
        "train_loss": 0.29435476660728455,
        "train_mean_token_accuracy": 0.875,
        "valid_loss": 1.528468096697772,
        "valid_mean_token_accuracy": 0.5555555555555556
      },
      "type": "metrics"
    },
    {
      "id": "ftevent-008dd553afac29a008dd553afac29a00",
      "created_at": 1740446596,
      "level": "info",
      "message": "Step 200: training loss=0.1939312368631363",
      "object": "fine_tuning.job.event",
      "data": {
        "step": 200,
        "train_loss": 0.1939312368631363,
        "train_mean_token_accuracy": 0.9545454382896423,
        "valid_loss": 0.8527921949114118,
        "valid_mean_token_accuracy": 0.8214285714285714
      },
      "type": "metrics"
    }
  ],
  "has_more": true,
  "object": "list"
}

チェックポイント

{
  "data": [
    {
      "id": "ftchkpt-2c40f488cf7345609c73a8cadcabeb25",
      "created_at": 1740446850,
      "fine_tuned_model_checkpoint": "gpt-4o-mini-2024-07-18.ft-d78af615cda84674b541c026e764bcb7",
      "fine_tuning_job_id": "ftjob-d78af615cda84674b541c026e764bcb7",
      "metrics": {
        "full_valid_loss": 0.7561612964140302,
        "full_valid_mean_token_accuracy": 0.7859922178988327,
        "step": 240.0,
        "train_loss": 0.11896172910928726,
        "train_mean_token_accuracy": 0.9583333134651184,
        "valid_loss": 0.7514890943254743,
        "valid_mean_token_accuracy": 0.75
      },
      "object": "fine_tuning.job.checkpoint",
      "step_number": 240
    },
    {
      "id": "ftchkpt-18c4b5df9a69445aa6060921028f5fc0",
      "created_at": 1740446647,
      "fine_tuned_model_checkpoint": "gpt-4o-mini-2024-07-18.ft-d78af615cda84674b541c026e764bcb7:ckpt-step-160",
      "fine_tuning_job_id": "ftjob-d78af615cda84674b541c026e764bcb7",
      "metrics": {
        "full_valid_loss": 0.681790496588681,
        "full_valid_mean_token_accuracy": 0.7957198443579766,
        "step": 160.0,
        "train_loss": 0.25228747725486755,
        "train_mean_token_accuracy": 0.9629629850387573,
        "valid_loss": 0.807875428880964,
        "valid_mean_token_accuracy": 0.75
      },
      "object": "fine_tuning.job.checkpoint",
      "step_number": 160
    },
    {
      "id": "ftchkpt-6e5a718fc6db47958e2b7a195b318ae3",
      "created_at": 1740446522,
      "fine_tuned_model_checkpoint": "gpt-4o-mini-2024-07-18.ft-d78af615cda84674b541c026e764bcb7:ckpt-step-80",
      "fine_tuning_job_id": "ftjob-d78af615cda84674b541c026e764bcb7",
      "metrics": {
        "full_valid_loss": 0.6571730231496611,
        "full_valid_mean_token_accuracy": 0.7957198443579766,
        "step": 80.0,
        "train_loss": 0.6486645936965942,
        "train_mean_token_accuracy": 0.8888888955116272,
        "valid_loss": 0.8158819334847587,
        "valid_mean_token_accuracy": 0.75
      },
      "object": "fine_tuning.job.checkpoint",
      "step_number": 80
    }
  ],
  "has_more": false,
  "object": "list"
}

モデル

{
  "id": "ftjob-d78af615cda84674b541c026e764bcb7",
  "created_at": 1740444480,
  "error": null,
  "fine_tuned_model": "gpt-4o-mini-2024-07-18.ft-d78af615cda84674b541c026e764bcb7",
  "finished_at": 1740447221,
  "hyperparameters": {
    "batch_size": 1,
    "learning_rate_multiplier": 1.0,
    "n_epochs": 3
  },
  "model": "gpt-4o-mini-2024-07-18",
  "object": "fine_tuning.job",
  "organization_id": null,
  "result_files": [
    "file-3bf8c4ac95124fcf92947d63183685b4"
  ],
  "seed": 105,
  "status": "succeeded",
  "trained_tokens": 12009,
  "training_file": "file-79600873356049e084973a13ea571a40",
  "validation_file": "file-637528dab95e418f97956a625a424960",
  "estimated_finish": 1740446395,
  "integrations": null,
  "method": null
}

7. ファインチューニング済モデルデプロイ

az account get-access-tokenコマンドでTokenを取得し、コード内に埋め込みます。
他にもリソースグループ、Azure OpenAI リソースの名前などをコード内に埋め込んでください。

詳しくは以下を参照。

import requests

# Deploy fine-tuned model

token = <token>
subscription = "<Subscription ID>"
resource_group = "<Resource Group Name>"
resource_name = "<Azure Open AI Resource name>"
model_deployment_name = "gpt-4o-mini-2024-07-18-ft" # Custom deployment name you chose for your fine-tuning model

deploy_params = {'api-version': "2023-05-01"}
deploy_headers = {'Authorization': 'Bearer {}'.format(token), 'Content-Type': 'application/json'}

deploy_data = {
    "sku": {"name": "standard", "capacity": 1},
    "properties": {
        "model": {
            "format": "OpenAI",
            "name": fine_tuned_model, #retrieve this value from the previous call, it will look like gpt-4o-mini-2024-07-18.ft-0e208cf33a6a466994aff31a08aba678
            "version": "1"
        }
    }
}
deploy_data = json.dumps(deploy_data)

request_url = f'https://management.azure.com/subscriptions/{subscription}/resourceGroups/{resource_group}/providers/Microsoft.CognitiveServices/accounts/{resource_name}/deployments/{model_deployment_name}'

print('Creating a new deployment...')

r = requests.put(request_url, params=deploy_params, headers=deploy_headers, data=deploy_data)

print(r)
print(r.reason)
print(r.json())

8. 結果確認

Chatを投げて結果を確認します。

response = client.chat.completions.create(
    model = "gpt-4o-mini-2024-07-18-ft", # model = "Custom deployment name you chose for your fine-tuning model"
    messages = [
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "福原 洋平の趣味をいくつかあげて"},
    ]
)

print(response.choices[0].message.content)

ファインチューニングしたデータが反映されています。

福原 洋平の趣味には、登山、写真撮影、料理が含まれます。