LoginSignup
13
11

More than 5 years have passed since last update.

@terapyon(Manabu TERADA)

AWSのストリーミング 自動化する為の事前調査

Last updated at Posted at 2014-08-01

この記事は、AWS S3 + CloudFront で動画ストリームの続きです。

HLSへの対応、サムネイルの取得などを Elastic Transcoder を使って行う。

事前準備

  • AWSアカウントを準備
  • 一時置き用のS3バケットを作る この記事参照
    • 1日に後にファイルを削除するように設定
  • 配信用プライベートS3バケットを作る この記事参照

  • CloudFrontを準備 (上記URLを参照)
    2つ準備、共に、signed urlに対応させる

    • RTMP用
    • HLS用(Web)

自動化の流れ

一時置き用のS3バケットに動画(mp4)ファイルを置く

以下のサンプルで簡単にできます。

from boto.s3.connection import S3Connection
from boto.s3.key import Key

ACCESS_KEY_ID = "xxxxxxxxxxxxxxxxx"
SECRET_ACCESS_KEY = "xxxxxxxxxxxxxxxxxxxxxxxxxx"
BUCKET_NAME = "c2-video-test-in"

conn = S3Connection(ACCESS_KEY_ID, SECRET_ACCESS_KEY)
bucket = conn.get_bucket(BUCKET_NAME)
k = Key(bucket)

local_filename = "/Users/xxxxxxxxxxxxxxxxx/MVI_0119.MP4"
sub_folder = "MVI_0119"
movie_name = "video.mp4"

k.key = '/'.join(f for f in (sub_folder, movie_name) if f)
k.set_contents_from_filename(local_filename)

配信用プライベートS3バケットにファイルをコピー

以下のメソッドで可能

copy_key(new_key_name, src_bucket_name, src_key_name, metadata=None, src_version_id=None, storage_class='STANDARD', preserve_acl=False, encrypt_key=False, headers=None, query_args=None)

具体的なコードは以下のとおり

from boto.s3.connection import S3Connection

ACCESS_KEY_ID = "xxxxxxxxxxxxxxxxx"
SECRET_ACCESS_KEY = "xxxxxxxxxxxxxxxxxxxxxxxxxx"
BUCKET_NAME = "c2-video-test-private"
SRC_BUCKET_NAME = "c2-video-test-in"

conn = S3Connection(ACCESS_KEY_ID, SECRET_ACCESS_KEY)
bucket = conn.get_bucket(BUCKET_NAME)

sub_folder = "MVI_0119"
movie_name = "video.mp4"

src_key = '/'.join(f for f in (sub_folder, movie_name) if f)
new_key = '/'.join(f for f in ("video", src_key) if f)
bucket.copy_key(new_key, SRC_BUCKET_NAME, src_key)

ジョブを作り一時置きバケットに置いたファイルからiOS用にHLSファイルを作り配信用バケットに置く

(サムネイルも出来上がる)

まずは、Pipelineを作る。今回は以前作成済みのPipelineで試してみた。
Pipelineの作成は、 このエントリーを参照

次に、Jobを実行

create_job(pipeline_id=None, input_name=None, output=None, outputs=None, output_key_prefix=None, playlists=None)
  • 公式サイトのパラメータの説明
    • pipeline_id (string) – The Id of the pipeline that you want Elastic Transcoder to use for transcoding. The pipeline determines several settings, including the Amazon S3 bucket from which Elastic Transcoder gets the files to transcode and the bucket into which Elastic Transcoder puts the transcoded files.
    • input_name (dict) – A section of the request body that provides information about the file that is being transcoded.
    • output (dict) – The CreateJobOutput structure.
    • outputs (list) – A section of the request body that provides information about the transcoded (target) files. We recommend that you use the Outputs syntax instead of the Output syntax.
    • output_key_prefix (string) – The value, if any, that you want Elastic Transcoder to prepend to the names of all files that this job creates, including output files, thumbnails, and playlists.
    • playlists (list) – If you specify a preset in PresetId for which the value of Container is ts (MPEG-TS), Playlists contains information about the master playlists that you want Elastic Transcoder to create.
from boto.elastictranscoder.layer1 import ElasticTranscoderConnection
from boto.regioninfo import RegionInfo


ACCESS_KEY_ID = "xxxxxxxxxxxxxxxxx"
SECRET_ACCESS_KEY = "xxxxxxxxxxxxxxxxxxxxxxxxxx"
REGION_NAME = "ap-northeast-1"
REGION_ENDPOINT = "elastictranscoder.ap-northeast-1.amazonaws.com"

region = RegionInfo(endpoint =REGION_ENDPOINT)
et_conn = ElasticTranscoderConnection(aws_access_key_id=ACCESS_KEY_ID, aws_secret_access_key=SECRET_ACCESS_KEY, region=region)


pipeline_id = "1405740268554-2sag8p"
input_file = "MVI_0119/video.mp4"

input_name = {
    "Key":input_file,
    "FrameRate":"auto",
    "Resolution":"auto",
    "AspectRatio":"auto",
    "Interlaced":"auto",
    "Container":"auto",
    }
outputs = [
    {'Key': 'video',
     "PresetId": "1351620000001-200050", # HLS (Apple HTTP Live Streaming), 400 kilobits/second
     "SegmentDuration":"10",
     'Rotate': 'auto',
     'ThumbnailPattern': "HLS{count}",
    },
    ]
output_key_prefix = "video/MVI_0119/video"

et_conn.create_job(pipeline_id=pipeline_id, input_name=input_name, outputs=outputs, output_key_prefix=output_key_prefix)

実行中Jobの確認

et_conn.create_job() を行うと以下の様なJSONが戻ってくる

{u'Job': {u'Arn': u'arn:aws:elastictranscoder:ap-northeast-1:526022276386:job/1407128189338-6cqeib',
          u'Id': u'1407128189338-6cqeib',
          u'Input': {u'AspectRatio': u'auto',
                     u'Container': u'auto',
                     u'FrameRate': u'auto',
                     u'Interlaced': u'auto',
                     u'Key': u'MVI_0119/video.mp4',
                     u'Resolution': u'auto'},
          u'Output': {u'AlbumArt': None,
                      u'Captions': None,
                      u'Composition': None,
                      u'Duration': None,
                      u'Height': None,
                      u'Id': u'1',
                      u'Key': u'video',
                      u'PresetId': u'1351620000001-200050',
                      u'Rotate': u'auto',
                      u'SegmentDuration': u'10.0',
                      u'Status': u'Submitted',
                      u'StatusDetail': None,
                      u'ThumbnailPattern': u'HLS{count}',
                      u'Watermarks': [],
                      u'Width': None},
          u'OutputKeyPrefix': u'video/MVI_0119/video',
          u'Outputs': [{u'AlbumArt': None,
                        u'Captions': None,
                        u'Composition': None,
                        u'Duration': None,
                        u'Height': None,
                        u'Id': u'1',
                        u'Key': u'video',
                        u'PresetId': u'1351620000001-200050',
                        u'Rotate': u'auto',
                        u'SegmentDuration': u'10.0',
                        u'Status': u'Submitted',
                        u'StatusDetail': None,
                        u'ThumbnailPattern': u'HLS{count}',
                        u'Watermarks': [],
                        u'Width': None}],
          u'PipelineId': u'1405740268554-2sag8p',
          u'Playlists': [],
          u'Status': u'Submitted'}}

情報を取ってみる

>>> result = _ # 上記のJSONをreaultに保存
>>> result[u"Job"][u"Id"]
u'1407128189338-6cqeib'
>>> result[u"Job"][u"Status"]
u'Submitted'

Idを取得すると、状態を確認することが可能
以下のように u'Complete'
となっていれば成功

>>> job_id = result[u"Job"][u"Id"]
>>> result2 = et_conn.read_job(job_id)
>>> result2[u"Job"][u"Status"]
u'Progressing'

>>> result3 = et_conn.read_job(job_id)
>>> result3[u"Job"][u"Status"]
u'Complete'

u'Error'の場合は、原因が u"StatusDetail"に入ってくるようだ。

>>> result4 = et_conn.read_job(job_id)
>>> result4[u"Job"][u"Status"]
u'Error'
>>> result4[u"Job"][u"Output"][u"StatusDetail"]
u'ERROR MESSAGE'

参考情報

PresetIdの一覧

Pipelineの作成又は取得

以前に作った、「c2-video-test-transcoder」があるかどうか?

>>> pi = et_conn.list_pipelines()
>>> pprint(pi)
{u'NextPageToken': None,
 u'Pipelines': [{u'Arn': u'arn:aws:elastictranscoder:ap-northeast-1:526022276386:pipeline/1405740268554-2sag8p',
                 u'ContentConfig': {u'Bucket': u'c2-video-test',
                                    u'Permissions': [{u'Access': [u'Read'],
                                                      u'Grantee': u'AllUsers',
                                                      u'GranteeType': u'Group'}],
                                    u'StorageClass': u'Standard'},
                 u'Id': u'1405740268554-2sag8p',
                 u'InputBucket': u'c2-video-test-in',
                 u'Name': u'c2-video-test-transcoder',
                 u'Notifications': {u'Completed': u'',
                                    u'Error': u'',
                                    u'Progressing': u'',
                                    u'Warning': u''},
                 u'OutputBucket': u'c2-video-test',
                 u'Role': u'arn:aws:iam::526022276386:role/Elastic_Transcoder_Default_Role',
                 u'Status': u'Active',
                 u'ThumbnailConfig': {u'Bucket': u'c2-video-test',
                                      u'Permissions': [{u'Access': [u'Read'],
                                                        u'Grantee': u'AllUsers',
                                                        u'GranteeType': u'Group'}],
                                      u'StorageClass': u'Standard'}}]}

>>> [p[u"Id"] for p in pi[u"Pipelines"] if p[u"Name"] == u'c2-video-test-transcoder']
[u'1405740268554-2sag8p']

「c2-video-test-transcoder-2014080401」という名前で作ってみよう

>>> pipeline_name = "c2-video-test-transcoder-2014080401"
>>> input_bucket = "c2-video-test-in"
>>> output_bucket = "c2-video-test-private"
>>> role = u'arn:aws:iam::526022276386:role/Elastic_Transcoder_Default_Role'
>>> notifications = {"Progressing":"", "Completed":"", "Warning":"", "Error":""}
>>> et_conn.create_pipeline(name=pipeline_name, input_bucket=input_bucket, output_bucket=output_bucket, role=role, notifications= notifications)
pprint(_)
{u'Pipeline': {u'Arn': u'arn:aws:elastictranscoder:ap-northeast-1:526022276386:pipeline/1407136782865-dnc1ts',
               u'ContentConfig': {u'Bucket': u'c2-video-test-private',
                                  u'Permissions': [],
                                  u'StorageClass': None},
               u'Id': u'1407136782865-dnc1ts',
               u'InputBucket': u'c2-video-test-in',
               u'Name': u'c2-video-test-transcoder-2014080401',
               u'Notifications': {u'Completed': u'',
                                  u'Error': u'',
                                  u'Progressing': u'',
                                  u'Warning': u''},
               u'OutputBucket': u'c2-video-test-private',
               u'Role': u'arn:aws:iam::526022276386:role/Elastic_Transcoder_Default_Role',
               u'Status': u'Active',
               u'ThumbnailConfig': {u'Bucket': u'c2-video-test-private',
                                    u'Permissions': [],
                                    u'StorageClass': None}}}

u'Id': u'1407136782865-dnc1ts'
u'Name': u'c2-video-test-transcoder-2014080401'
で出来上がった。

実際にpipelineのリストを取得して見る

>>> pi2 = et_conn.list_pipelines()
>>> pprint(pi2)
{u'NextPageToken': None,
 u'Pipelines': [{u'Arn': u'arn:aws:elastictranscoder:ap-northeast-1:526022276386:pipeline/1405740268554-2sag8p',
                 u'ContentConfig': {u'Bucket': u'c2-video-test',
                                    u'Permissions': [{u'Access': [u'Read'],
                                                      u'Grantee': u'AllUsers',
                                                      u'GranteeType': u'Group'}],
                                    u'StorageClass': u'Standard'},
                 u'Id': u'1405740268554-2sag8p',
                 u'InputBucket': u'c2-video-test-in',
                 u'Name': u'c2-video-test-transcoder',
                 u'Notifications': {u'Completed': u'',
                                    u'Error': u'',
                                    u'Progressing': u'',
                                    u'Warning': u''},
                 u'OutputBucket': u'c2-video-test',
                 u'Role': u'arn:aws:iam::526022276386:role/Elastic_Transcoder_Default_Role',
                 u'Status': u'Active',
                 u'ThumbnailConfig': {u'Bucket': u'c2-video-test',
                                      u'Permissions': [{u'Access': [u'Read'],
                                                        u'Grantee': u'AllUsers',
                                                        u'GranteeType': u'Group'}],
                                      u'StorageClass': u'Standard'}},
                {u'Arn': u'arn:aws:elastictranscoder:ap-northeast-1:526022276386:pipeline/1407136752887-fpaajv',
                 u'ContentConfig': {u'Bucket': u'c2-video-test-private',
                                    u'Permissions': [],
                                    u'StorageClass': None},
                 u'Id': u'1407136752887-fpaajv',
                 u'InputBucket': u'c2-video-test-in',
                 u'Name': u'c2-video-test-transcoder-2014080401',
                 u'Notifications': {u'Completed': u'',
                                    u'Error': u'',
                                    u'Progressing': u'',
                                    u'Warning': u''},
                 u'OutputBucket': u'c2-video-test-private',
                 u'Role': u'arn:aws:iam::526022276386:role/Elastic_Transcoder_Default_Role',
                 u'Status': u'Active',
                 u'ThumbnailConfig': {u'Bucket': u'c2-video-test-private',
                                      u'Permissions': [],
                                      u'StorageClass': None}},
                {u'Arn': u'arn:aws:elastictranscoder:ap-northeast-1:526022276386:pipeline/1407136782865-dnc1ts',
                 u'ContentConfig': {u'Bucket': u'c2-video-test-private',
                                    u'Permissions': [],
                                    u'StorageClass': None},
                 u'Id': u'1407136782865-dnc1ts',
                 u'InputBucket': u'c2-video-test-in',
                 u'Name': u'c2-video-test-transcoder-2014080401',
                 u'Notifications': {u'Completed': u'',
                                    u'Error': u'',
                                    u'Progressing': u'',
                                    u'Warning': u''},
                 u'OutputBucket': u'c2-video-test-private',
                 u'Role': u'arn:aws:iam::526022276386:role/Elastic_Transcoder_Default_Role',
                 u'Status': u'Active',
                 u'ThumbnailConfig': {u'Bucket': u'c2-video-test-private',
                                      u'Permissions': [],
                                      u'StorageClass': None}}]}

このpipelineで実際に変換してみる

pipeline_id = "1407136782865-dnc1ts"
input_file = "MVI_0119/video.mp4"

input_name = {
    "Key":input_file,
    "FrameRate":"auto",
    "Resolution":"auto",
    "AspectRatio":"auto",
    "Interlaced":"auto",
    "Container":"auto",
    }
outputs = [
    {'Key': 'video',
     "PresetId": "1351620000001-200050", # HLS (Apple HTTP Live Streaming), 400 kilobits/second
     "SegmentDuration":"10",
     'Rotate': 'auto',
     'ThumbnailPattern': "HLS{count}",
    },
    ]
output_key_prefix = "video/MVI_0119/video"

et_conn.create_job(pipeline_id=pipeline_id, input_name=input_name, outputs=outputs, output_key_prefix=output_key_prefix)

成功した。

roleの取得または作成

上記だと、roleを直接指定しているので、自動スクリプトとしては不十分

>>> from boto.iam.connection import IAMConnection
>>> iam_conn = IAMConnection(aws_access_key_id=ACCESS_KEY_ID, aws_secret_access_key=SECRET_ACCESS_KEY)
>>> et_roles = iam_conn.get_role("Elastic_Transcoder_Default_Role") # 無いときは boto.exception.BotoServerError: BotoServerError: 404 Not Found が返ってくる
>>> et_roles[u"get_role_response"][u"get_role_result"][u"role"][u"arn"]
u'arn:aws:iam::526022276386:role/Elastic_Transcoder_Default_Role'

上記でroleが取得できなければ、新規にroleを作る必要がありそう。

create_role(role_name, assume_role_policy_document=None, path=None)
を使って作れそう。

>>> iam_conn.create_role("Elastic_Transcoder_Default_Role")
13
11
0

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
Sign upLogin
13
11