PyAVとは
「PyAVで動画の任意のフレームを取得する」を参照してください．
動画ファイルの読み込みと情報の取得
動画ファイルに使われるコンテナ・コーデックは様々です．それらの情報を確認したいと思ったのですが，あまり記事がないので作成してみました．
- ファイルサイズ
- ビットレート
- 動画長（秒）
- フレーム数
- フレームレート（fps）
- 縦・横
- キーフレム判定
またフレームをndarrayに変換する方法がちょっとトリッキーだったので工夫してみました．
import av
def load_from_file(
filename,
any_frame=False,
backward=True
):
container = av.open(filename)
stream = container.streams.video[0] # 1つ目のvideo stream．普通は1つしかないからこれでOK
print("filename:", container.name)
# print("filesize [bytes]:", container.size)
print("filesize [kB]:", container.size // 1024)
if container.size > 1024 * 1024:
print("filesize [MB]:", container.size // 1024 // 1024)
# print("bit_rate [b/s]:", container.bit_rate)
print("bit_rate [kb/s]:", float(container.bit_rate) / 1024)
print("container duration [sec]:",
float(container.duration) / av.time_base)
print("stream duration:", stream.duration) # 謎
print("frames:", stream.frames)
print("guessed frames:",
int(float(container.duration) / av.time_base * stream.base_rate))
print("container format name:", container.format.name)
print("container format long name:", container.format.long_name)
print("codec name:", stream.codec_context.codec.name)
print("codec long name:", stream.codec_context.codec.long_name)
print("codec tag:", stream.codec_context.codec_tag)
for md_str in container.metadata.keys():
print(f"container metadata {md_str}:", container.metadata[md_str])
print("base_rate [fps]:", stream.base_rate)
print("rate [fps]:", stream.codec_context.rate)
print("width [pix]:", stream.codec_context.width)
print("height [pix]:", stream.codec_context.height)
sec = 2 # 2秒時点へシーク
container.seek(
offset=sec // stream.time_base,
any_frame=any_frame,
backward=backward,
stream=stream)
for i, frame in enumerate(container.decode(video=0)):
print(f"i:{i:3d}, time:{frame.time:.3f}, pts:{frame.pts}, "
f"{frame.width}x{frame.height}, {frame.format.name} ",
end="")
if frame.key_frame:
print("keyframe", end="")
print()
# ffmpegの機能でリサイズ．おそらく高速．
frame = frame.reformat(width=244, height=244)
# 以下ndarrayへの変換
img = frame.to_ndarray() # このままだと1チャンネルndarray．多分Yチャンネルのgray scale（未確認．コーデックによってはエラー発生）
img = frame.to_rgb().to_ndarray() # 3チャンネルndarray（RGB）
img = frame.to_ndarray(format="rgb24") # 同上
img = frame.to_rgb().to_ndarray(width=244, height=244) # リサイズしながらRGBへ変換
img = frame.to_ndarray(format="rgb24", width=244, height=244) # 同上
# 以下PILへの変換
img = frame.to_image() # PIL image
img = frame.to_image(width=244, height=244) # リサイズしながらPILへ変換
img.save(
"frames.{:04d}.jpg".format(i),
quality=80,
) # PIL imageの保存
if i > 5:
break
if __name__ == '__main__':
filenames = [
# long (untrimmed) videos
"zlVkeKC6Ha8.mp4", # AVA, mp4, h264/avc
"v_nHE7u40plD0.mkv", # ActivityNet, mkv, vp9
"001YG.mp4", # Charades, mp4, h264/avc
"-4wsuPCjDBc_5_15.avi", # MSVD, avi, h264
"s07-d72-cam-002.avi", # MPII Cooking 2, avi, msmpeg4v2/MP42
# short (trimmed) videos
"-3B32lodo2M_000059_000069.mp4", # Kinetics, mp4, h264/avc
"v_ApplyEyeMakeup_g01_c01.avi", # UCF101, avi, mpeg4/XVID
"April_09_brush_hair_u_nm_np1_ba_goo_1.avi", # HMDB, avi, mpeg4/DX50
"200000.webm", # SSv2, webm, vp9
]
for filename in filenames:
print("==================")
try:
load_from_file(filename, any_frame=False, backward=True)
except BaseException:
continue
以下実行結果．
AVA．mp4のh264/acvなので素直．
filename: zlVkeKC6Ha8.mp4
filesize [kB]: 224274
filesize [MB]: 219
bit_rate [kb/s]: 637.2333984375
container duration [sec]: 2815.605
stream duration: 253392000
frames: 84464
guessed frames: 84468
container format name: mov,mp4,m4a,3gp,3g2,mj2
container format long name: QuickTime / MOV
codec name: h264
codec long name: H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10
codec tag: avc1
container metadata major_brand: isom
container metadata minor_version: 512
container metadata compatible_brands: isomiso2avc1mp41
container metadata encoder: Lavf56.40.101
base_rate [fps]: 30
rate [fps]: 30
width [pix]: 640
height [pix]: 480
i: 0, time:0.000, pts:0, 640x480, yuv420p keyframe
i: 1, time:0.033, pts:3000, 640x480, yuv420p
i: 2, time:0.067, pts:6000, 640x480, yuv420p
i: 3, time:0.100, pts:9000, 640x480, yuv420p
i: 4, time:0.133, pts:12000, 640x480, yuv420p
i: 5, time:0.167, pts:15000, 640x480, yuv420p
i: 6, time:0.200, pts:18000, 640x480, yuv420p
ActivityNet．ほとんどはmp4のh264/acvだがたまにmkvでvp9が混在．
filename: v_nHE7u40plD0.mkv
filesize [kB]: 49262
filesize [MB]: 48
bit_rate [kb/s]: 2708.220703125
container duration [sec]: 145.519
stream duration: None
frames: 0
guessed frames: 7275
container format name: matroska,webm
container format long name: Matroska / WebM
codec name: vp9
codec long name: Google VP9
codec tag:
container metadata ENCODER: Lavf56.25.101
base_rate [fps]: 50
rate [fps]: 50
width [pix]: 1280
height [pix]: 720
i: 0, time:0.000, pts:0, 1280x720, yuv420p keyframe
i: 1, time:0.020, pts:20, 1280x720, yuv420p
i: 2, time:0.040, pts:40, 1280x720, yuv420p
i: 3, time:0.060, pts:60, 1280x720, yuv420p
i: 4, time:0.080, pts:80, 1280x720, yuv420p
i: 5, time:0.100, pts:100, 1280x720, yuv420p
i: 6, time:0.120, pts:120, 1280x720, yuv420p
Charadesはmp4でh264/avc
filename: 001YG.mp4
filesize [kB]: 1484
filesize [MB]: 1
bit_rate [kb/s]: 386.2177734375
container duration [sec]: 30.744
stream duration: 919919
frames: 919
guessed frames: 921
container format name: mov,mp4,m4a,3gp,3g2,mj2
container format long name: QuickTime / MOV
codec name: h264
codec long name: H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10
codec tag: avc1
container metadata major_brand: isom
container metadata minor_version: 512
container metadata compatible_brands: isomiso2avc1mp41
container metadata encoder: Lavf56.40.101
base_rate [fps]: 30000/1001
rate [fps]: 30000/1001
width [pix]: 480
height [pix]: 270
i: 0, time:0.000, pts:0, 480x270, yuv420p keyframe
i: 1, time:0.033, pts:1001, 480x270, yuv420p
i: 2, time:0.067, pts:2002, 480x270, yuv420p
i: 3, time:0.100, pts:3003, 480x270, yuv420p
i: 4, time:0.133, pts:4004, 480x270, yuv420p
i: 5, time:0.167, pts:5005, 480x270, yuv420p
i: 6, time:0.200, pts:6006, 480x270, yuv420p
MSVDはaviでh264/avc
filename: -4wsuPCjDBc_5_15.avi
filesize [kB]: 679
bit_rate [kb/s]: 541.75
container duration [sec]: 10.027743
stream duration: 300
frames: 300
guessed frames: 299
container format name: avi
container format long name: AVI (Audio Video Interleaved)
codec name: h264
codec long name: H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10
codec tag: H264
container metadata software: MEncoder SVN-r33477-4.2.1
base_rate [fps]: 359/12
rate [fps]: 29917/1000
width [pix]: 480
height [pix]: 360
i: 0, time:1.136, pts:34, 480x360, yuv420p keyframe
i: 1, time:1.170, pts:35, 480x360, yuv420p
i: 2, time:1.203, pts:36, 480x360, yuv420p
i: 3, time:1.237, pts:37, 480x360, yuv420p
i: 4, time:1.270, pts:38, 480x360, yuv420p
i: 5, time:1.304, pts:39, 480x360, yuv420p
i: 6, time:1.337, pts:40, 480x360, yuv420p
MPII Cooking 2はaviでMPEG4（h264ではない）
filename: s07-d72-cam-002.avi
filesize [kB]: 19374
filesize [MB]: 18
bit_rate [kb/s]: 2460.560546875
container duration [sec]: 62.993197
stream duration: 1852
frames: 1852
guessed frames: 1851
container format name: avi
container format long name: AVI (Audio Video Interleaved)
codec name: msmpeg4v2
codec long name: MPEG-4 part 2 Microsoft variant version 2
codec tag: MP42
container metadata software: MEncoder dev-SVN-r26940
base_rate [fps]: 147/5
rate [fps]: 147/5
width [pix]: 1624
height [pix]: 1224
i: 0, time:0.000, pts:0, 1624x1224, yuv420p keyframe
i: 1, time:0.034, pts:1, 1624x1224, yuv420p
i: 2, time:0.068, pts:2, 1624x1224, yuv420p
i: 3, time:0.102, pts:3, 1624x1224, yuv420p
i: 4, time:0.136, pts:4, 1624x1224, yuv420p
i: 5, time:0.170, pts:5, 1624x1224, yuv420p
i: 6, time:0.204, pts:6, 1624x1224, yuv420p
Kineticsはyoutube-dlだからmp4でh264/acv
filename: -3B32lodo2M_000059_000069.mp4
filesize [kB]: 1330
filesize [MB]: 1
bit_rate [kb/s]: 1063.935546875
container duration [sec]: 10.008
stream duration: 128000
frames: 250
guessed frames: 250
container format name: mov,mp4,m4a,3gp,3g2,mj2
container format long name: QuickTime / MOV
codec name: h264
codec long name: H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10
codec tag: avc1
container metadata major_brand: isom
container metadata minor_version: 512
container metadata compatible_brands: isomiso2avc1mp41
container metadata encoder: Lavf57.41.100
base_rate [fps]: 25
rate [fps]: 25
width [pix]: 640
height [pix]: 360
i: 0, time:0.000, pts:0, 640x360, yuv420p keyframe
i: 1, time:0.040, pts:512, 640x360, yuv420p
i: 2, time:0.080, pts:1024, 640x360, yuv420p
i: 3, time:0.120, pts:1536, 640x360, yuv420p
i: 4, time:0.160, pts:2048, 640x360, yuv420p
i: 5, time:0.200, pts:2560, 640x360, yuv420p
i: 6, time:0.240, pts:3072, 640x360, yuv420p
UC101はAVIでMPEG4（XviD使用）
filename: v_ApplyEyeMakeup_g01_c01.avi
filesize [kB]: 287
bit_rate [kb/s]: 350.806640625
container duration [sec]: 6.56
stream duration: 164
frames: 164
guessed frames: 164
container format name: avi
container format long name: AVI (Audio Video Interleaved)
codec name: mpeg4
codec long name: MPEG-4 part 2
codec tag: XVID
container metadata software: MEncoder r33883
base_rate [fps]: 25
rate [fps]: 25
width [pix]: 320
height [pix]: 240
i: 0, time:0.040, pts:1, 320x240, yuv420p keyframe
i: 1, time:0.080, pts:2, 320x240, yuv420p
i: 2, time:0.160, pts:4, 320x240, yuv420p
i: 3, time:0.120, pts:3, 320x240, yuv420p
i: 4, time:0.200, pts:5, 320x240, yuv420p
i: 5, time:0.280, pts:7, 320x240, yuv420p
i: 6, time:0.240, pts:6, 320x240, yuv420p
HMDB51はaviでMPEG4（DivX使用）
filename: April_09_brush_hair_u_nm_np1_ba_goo_1.avi
filesize [kB]: 754
bit_rate [kb/s]: 458.4296875
container duration [sec]: 13.166667
stream duration: 395
frames: 395
guessed frames: 395
container format name: avi
container format long name: AVI (Audio Video Interleaved)
codec name: mpeg4
codec long name: MPEG-4 part 2
codec tag: DX50
base_rate [fps]: 30
rate [fps]: 30
width [pix]: 320
height [pix]: 240
i: 0, time:0.033, pts:1, 320x240, yuv420p keyframe
i: 1, time:0.100, pts:3, 320x240, yuv420p
i: 2, time:0.167, pts:5, 320x240, yuv420p
i: 3, time:0.133, pts:4, 320x240, yuv420p
i: 4, time:0.200, pts:6, 320x240, yuv420p
i: 5, time:0.267, pts:8, 320x240, yuv420p
i: 6, time:0.233, pts:7, 320x240, yuv420p
SSv2はwebm/vp9．（SSv1はjpeg frameだったが）
filename: 200000.webm
filesize [kB]: 146
bit_rate [kb/s]: 335.42578125
container duration [sec]: 3.5
stream duration: None
frames: 0
guessed frames: 42
container format name: matroska,webm
container format long name: Matroska / WebM
codec name: vp9
codec long name: Google VP9
codec tag:
container metadata title: (C) 2017 Twenty Billion Neurons GmbH, 20BN-Someting-Something-Dataset V2
container metadata encoder: Lavf56.40.101
base_rate [fps]: 12
rate [fps]: 12
width [pix]: 427
height [pix]: 240
i: 0, time:0.000, pts:0, 427x240, yuv420p keyframe
i: 1, time:0.083, pts:83, 427x240, yuv420p
i: 2, time:0.167, pts:167, 427x240, yuv420p
i: 3, time:0.250, pts:250, 427x240, yuv420p
i: 4, time:0.333, pts:333, 427x240, yuv420p
i: 5, time:0.417, pts:417, 427x240, yuv420p
i: 6, time:0.500, pts:500, 427x240, yuv420p