steps functionでtraining jobを実行する

背景

• AWS Step FunctionsでAWS APIを利用してAWS Serviceを操作できる
• APIのパラメーターは、SDKと違いある

SageMaker CreateTrainingJob

APIパラメーター

{
  "AlgorithmSpecification": {
    "TrainingImage": "{AWS::AccountId}.dkr.ecr.ap-northeast-1.amazonaws.com/{repository_name}:latest",
    "TrainingInputMode": "File"
  },
  "InputDataConfig": [
    {
      "ChannelName": "train",
      "ContentType": "text/csv",
      "DataSource": {
        "S3DataSource": {
          "S3DataDistributionType": "FullyReplicated",
          "S3DataType": "S3Prefix",
          "S3Uri.$": "s3://{bucket_name}/path/"
        }
      }
    }
  ],
  "OutputDataConfig": {
    "S3OutputPath.$": "s3://{bucket_name}/path/"
  },
  "ResourceConfig": {
    "InstanceCount": 1,
    "InstanceType": "ml.m5.4xlarge",
    "VolumeSizeInGB": 5
  },
  "StoppingCondition": {
    "MaxRuntimeInSeconds": 3000
  },
  "RoleArn": "${TrainingRole}",
  "TrainingJobName.$": "xxxxxx",
  "VpcConfig": {
    "SecurityGroupIds.$": "$.security_group",
    "Subnets.$": "$.subnets"
  }
}

注意点

  "InputDataConfig": [
    {
      "ChannelName": "train",

• チャンネルネームが上記のように、「train」を決めると、SM_CHANNEL_TRAININGのパスは'/opt/ml/input/data/train'になる
• 参照：https://github.com/aws/sagemaker-containers#list-of-provided-environment-variables-by-sagemaker-containers