今回やったこと

Pytorch-openposeを利用

当初、次のリポジトリの資源をgit cloneしようとしたが、__tesorflow=1.4.1が入れられなかった__ため、利用を諦めた。

https://github.com/jiajunhua/ildoonet-tf-pose-estimation

変わりに、PyTorch実装の資源を次から入れることにした。

https://github.com/Hzzone/pytorch-openpose

Terminal

electron@diynoMacBook-Pro Desktop % git clone https://github.com/Hzzone/pytorch-openpose.git
Cloning into 'pytorch-openpose'...
remote: Enumerating objects: 154, done.
remote: Counting objects: 100% (3/3), done.
remote: Compressing objects: 100% (3/3), done.
remote: Total 154 (delta 0), reused 0 (delta 0), pack-reused 151
Receiving objects: 100% (154/154), 20.18 MiB | 13.12 MiB/s, done.
Resolving deltas: 100% (68/68), done.
electron@diynoMacBook-Pro Desktop %

Terminal

electron@diynoMacBook-Pro Desktop % cd pytorch-openpose 
electron@diynoMacBook-Pro pytorch-openpose % ls
README.md		demo_camera.py		images			notebooks		src
demo.py			demo_video.py		model			requirements.txt
electron@diynoMacBook-Pro pytorch-openpose %

以下から、body_pose_model.pth hand_pose_model.pthをダウンロード

https://www.dropbox.com/sh/7xbup2qsn7vvjxo/AABWFksdlgOMXR_r5v3RwKRYa?dl=0

両ファイルを、./model直下に移動

Terminal

electron@diynoMacBook-Pro pytorch-openpose % ls model
electron@diynoMacBook-Pro pytorch-openpose % ls *.pth
body_pose_model.pth	hand_pose_model.pth
electron@diynoMacBook-Pro pytorch-openpose % 
electron@diynoMacBook-Pro pytorch-openpose % mv *.pth ./model/
electron@diynoMacBook-Pro pytorch-openpose % ls *.pth         
zsh: no matches found: *.pth
electron@diynoMacBook-Pro pytorch-openpose %

Terminal

electron@diynoMacBook-Pro pytorch-openpose % ls model         
body_pose_model.pth	hand_pose_model.pth
electron@diynoMacBook-Pro pytorch-openpose %

Terminal

electron@diynoMacBook-Pro pytorch-openpose % python3 demo_camera.py                                                           
Traceback (most recent call last):
  File "/Users/electron/Desktop/pytorch-openpose/demo_camera.py", line 5, in <module>
    import torch
  File "/usr/local/lib/python3.9/site-packages/torch/__init__.py", line 196, in <module>
    from torch._C import *
RuntimeError: module compiled against API version 0xe but this version of numpy is 0xd
electron@diynoMacBook-Pro pytorch-openpose %

以下のエラーメッセージが出た。

RuntimeError: module compiled against API version 0xe but this version of numpy is 0xd

Googleで調べると、numpyのバージョンに問題があるらしい。
requirement.txtに列挙されているnumpuのバージョン番号を見てみる。

Terminal

electron@diynoMacBook-Pro pytorch-openpose % cat requirements.txt 
numpy
matplotlib
opencv-python
scipy
scikit-image
tqdm%   
electron@diynoMacBook-Pro pytorch-openpose %

バージョンは特に指定されていない。

以下のサイトから、test.pyを写経。

https://self-development.info/openpose%E3%81%A7%E5%A7%BF%E5%8B%A2%E6%8E%A8%E5%AE%9A%E3%82%92%E8%A1%8C%E3%81%86%E3%80%90pytorch-openpose%E3%80%91/

test.py

import os
import cv2
import copy
from src import util
from src.body import Body
 
INPUT_FILE_NAME = "test.jpg"
 
if __name__ == "__main__":
 
    body_estimation = Body('model/body_pose_model.pth')
 
    target_image_path = 'images/' + INPUT_FILE_NAME
    oriImg = cv2.imread(target_image_path)  # B,G,R order
    candidate, subset = body_estimation(oriImg)
    canvas = copy.deepcopy(oriImg)
    canvas = util.draw_bodypose(canvas, candidate, subset)
 
    basename_name = os.path.splitext(os.path.basename(target_image_path))[0]
 
    result_image_path = "result/pose_" + basename_name + ".jpg"
    cv2.imwrite(result_image_path, canvas)

imageファイルをimages直下に格納。
ファイル名は、pose_test.jpg

Terminal

electron@diynoMacBook-Pro pytorch-openpose % ls images | grep test 
test.jpg
electron@diynoMacBook-Pro pytorch-openpose %

やはり同じエラーが出る。

Terminal

electron@diynoMacBook-Pro pytorch-openpose % python3 test.py
Traceback (most recent call last):
  File "/Users/electron/Desktop/pytorch-openpose/test.py", line 5, in <module>
    from src.body import Body
  File "/Users/electron/Desktop/pytorch-openpose/src/body.py", line 8, in <module>
    import torch
  File "/usr/local/lib/python3.9/site-packages/torch/__init__.py", line 196, in <module>
    from torch._C import *
RuntimeError: module compiled against API version 0xe but this version of numpy is 0xd
electron@diynoMacBook-Pro pytorch-openpose %

Python 3.7.0に切り替える

Terminal

electron@diynoMacBook-Pro pytorch-openpose % pyenv local 3.7.0
electron@diynoMacBook-Pro pytorch-openpose % pyenv versions   
  system
  2.7.16
  3.6.0
  3.6.1
  3.6.3
  3.6.3/envs/gpt2_ja
  3.6.6
* 3.7.0 (set by /Users/electron/Desktop/pytorch-openpose/.python-version)
  3.7.4
  3.9.0
  gpt2_ja
electron@diynoMacBook-Pro pytorch-openpose %

改めて、依存ライブラリをinstall

Terminal

electron@diynoMacBook-Pro pytorch-openpose % pip3 install -r requirements.txt
Requirement already satisfied: numpy in /usr/local/lib/python3.9/site-packages (from -r requirements.txt (line 1)) (1.19.5)
Requirement already satisfied: matplotlib in /usr/local/lib/python3.9/site-packages (from -r requirements.txt (line 2)) (3.4.2)
Requirement already satisfied: opencv-python in /usr/local/lib/python3.9/site-packages (from -r requirements.txt (line 3)) (4.5.3.56)
Requirement already satisfied: scipy in /usr/local/lib/python3.9/site-packages (from -r requirements.txt (line 4)) (1.7.0)
Requirement already satisfied: scikit-image in /usr/local/lib/python3.9/site-packages (from -r requirements.txt (line 5)) (0.18.2)
Requirement already satisfied: tqdm in /usr/local/lib/python3.9/site-packages (from -r requirements.txt (line 6)) (4.61.2)
Requirement already satisfied: pillow>=6.2.0 in /usr/local/lib/python3.9/site-packages (from matplotlib->-r requirements.txt (line 2)) (8.3.1)
Requirement already satisfied: pyparsing>=2.2.1 in /usr/local/lib/python3.9/site-packages (from matplotlib->-r requirements.txt (line 2)) (2.4.7)
Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.9/site-packages (from matplotlib->-r requirements.txt (line 2)) (0.10.0)
Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.9/site-packages (from matplotlib->-r requirements.txt (line 2)) (1.3.1)
Requirement already satisfied: python-dateutil>=2.7 in /usr/local/lib/python3.9/site-packages (from matplotlib->-r requirements.txt (line 2)) (2.8.1)
Requirement already satisfied: six in /usr/local/lib/python3.9/site-packages (from cycler>=0.10->matplotlib->-r requirements.txt (line 2)) (1.15.0)
Requirement already satisfied: networkx>=2.0 in /usr/local/lib/python3.9/site-packages (from scikit-image->-r requirements.txt (line 5)) (2.5.1)
Requirement already satisfied: imageio>=2.3.0 in /usr/local/lib/python3.9/site-packages (from scikit-image->-r requirements.txt (line 5)) (2.9.0)
Requirement already satisfied: tifffile>=2019.7.26 in /usr/local/lib/python3.9/site-packages (from scikit-image->-r requirements.txt (line 5)) (2021.7.2)
Requirement already satisfied: PyWavelets>=1.1.1 in /usr/local/lib/python3.9/site-packages (from scikit-image->-r requirements.txt (line 5)) (1.1.1)
Requirement already satisfied: decorator<5,>=4.3 in /usr/local/lib/python3.9/site-packages (from networkx>=2.0->scikit-image->-r requirements.txt (line 5)) (4.4.2)
electron@diynoMacBook-Pro pytorch-openpose %

またダメ

Terminal

electron@diynoMacBook-Pro pytorch-openpose % python3 demo_camera.py
Traceback (most recent call last):
  File "/Users/electron/Desktop/pytorch-openpose/demo_camera.py", line 5, in <module>
    import torch
  File "/usr/local/lib/python3.9/site-packages/torch/__init__.py", line 196, in <module>
    from torch._C import *
RuntimeError: module compiled against API version 0xe but this version of numpy is 0xd
electron@diynoMacBook-Pro pytorch-openpose %

以下の次のアドバイスを採用

https://github.com/freqtrade/freqtrade/issues/4281

pip install numpy --upgrade works, something has to be updated in the files, when you figure out what, could you tell me please, thanks

Terminal

electron@diynoMacBook-Pro pytorch-openpose % pip3 install numpy --upgrade

（・・・省略・・・）

tensorflow 2.5.0 requires numpy~=1.19.2, but you have numpy 1.21.1 which is incompatible.
Successfully installed numpy-1.21.1
electron@diynoMacBook-Pro pytorch-openpose %

今度は、cudaがないエラー

Terminal

electron@diynoMacBook-Pro pytorch-openpose % python3 demo_camera.py
Traceback (most recent call last):
  File "/Users/electron/Desktop/pytorch-openpose/demo_camera.py", line 15, in <module>
    print(f"Torch device: {torch.cuda.get_device_name()}")
  File "/usr/local/lib/python3.9/site-packages/torch/cuda/__init__.py", line 276, in get_device_name
    return get_device_properties(device).name
  File "/usr/local/lib/python3.9/site-packages/torch/cuda/__init__.py", line 306, in get_device_properties
    _lazy_init()  # will define _get_device_properties
  File "/usr/local/lib/python3.9/site-packages/torch/cuda/__init__.py", line 164, in _lazy_init
    raise AssertionError("Torch not compiled with CUDA enabled")
AssertionError: Torch not compiled with CUDA enabled
electron@diynoMacBook-Pro pytorch-openpose %

以下のサイトの次のアドバイスを採用

https://qiita.com/val_geass/items/0035b2d0d8dd5ea32db6

AssertionError: Torch not compiled with CUDA enabled
はい、cudaね。test_video.pyの110行目と134行目に.cuda()があるので、.cuda()だけを削除し、再び実行。

以下をコメントアウト

print(f"Torch device: {torch.cuda.get_device_name()}")

Terminal

electron@diynoMacBook-Pro pytorch-openpose % cp demo_camera.py demo_camera_cpu.py

今度はちゃんと動いた

Webカメラの画像が動画で表示され、自分の顔に骨格線が重畳された画像が表示された。

体を動かすと、ややタイムラグを伴って、骨格線も追従してついてきた。

CTRL-Zで強制終了

Terminal

electron@diynoMacBook-Pro pytorch-openpose % python3 demo_camera_cpu.py
^Z
zsh: suspended  python3 demo_camera_cpu.py
electron@diynoMacBook-Pro pytorch-openpose %

今度は、test.pyもうまく動いた

Terminal

electron@diynoMacBook-Pro pytorch-openpose % python3 test.py

__result__ディレクトリ直下に、解析後の画像ファイルが出力されている。

ファイル名：pose + 元のファイル名 * .jpg*

Terminal

electron@diynoMacBook-Pro pytorch-openpose % ls result
pose_test.jpg

electron@diynoMacBook-Pro pytorch-openpose %

demo.pyもうまく動いた

Terminal

electron@diynoMacBook-Pro pytorch-openpose % python3 demo.py
electron@diynoMacBook-Pro pytorch-openpose %

数秒後にウインドウが立ち上がり、次の画像が表示された。

動画は実行に成功しなかった。

Terminal

electron@diynoMacBook-Pro pytorch-openpose % pip3 install ffmpeg

Terminal

electron@diynoMacBook-Pro pytorch-openpose % python3 demo_video.py dance.mp4                                
ffmpeg version 4.4 Copyright (c) 2000-2021 the FFmpeg developers
  built with Apple clang version 12.0.0 (clang-1200.0.32.29)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/4.4_2 --enable-shared --enable-pthreads --enable-version3 --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libbluray --enable-libdav1d --enable-libmp3lame --enable-libopus --enable-librav1e --enable-librubberband --enable-libsnappy --enable-libsrt --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libspeex --enable-libsoxr --enable-libzmq --enable-libzimg --disable-libjack --disable-indev=jack --enable-avresample --enable-videotoolbox
  libavutil      56. 70.100 / 56. 70.100
  libavcodec     58.134.100 / 58.134.100
  libavformat    58. 76.100 / 58. 76.100
  libavdevice    58. 13.100 / 58. 13.100
  libavfilter     7.110.100 /  7.110.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  9.100 /  5.  9.100
  libswresample   3.  9.100 /  3.  9.100
  libpostproc    55.  9.100 / 55.  9.100
Input #0, rawvideo, from 'pipe:':
  Duration: N/A, start: 0.000000, bitrate: 294796 kb/s
  Stream #0:0: Video: rawvideo (BGR[24] / 0x18524742), bgr24, 853x480, 294796 kb/s, 30 tbr, 30 tbn, 30 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (libx264))
[libx264 @ 0x7f8c14809a00] width not divisible by 2 (853x480)
Error initializing output stream 0:0 -- Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height
Conversion failed!
Traceback (most recent call last):
  File "/Users/electron/Desktop/pytorch-openpose/demo_video.py", line 132, in <module>
    writer(posed_frame)
  File "/Users/electron/Desktop/pytorch-openpose/demo_video.py", line 108, in __call__
    self.ff_proc.stdin.write(frame.tobytes())
BrokenPipeError: [Errno 32] Broken pipe
electron@diynoMacBook-Pro pytorch-openpose %

ウインドウが立ち上がり、次の１フレームで止まった状態で画像が表示された後、落ちた。

test.pyを書き換えて、もう２つ、画像ファイルを加工してみる

取り扱う画像ファイルのパスを、コマンドライン引数で受け取るように書き換える。

test2.py

import os
import cv2
import copy
from src import util
from src.body import Body
import argparse    

# 動画ファイル名をコマンドライン引数から受け取る
parser = argparse.ArgumentParser(description='')    #
parser.add_argument('file_name')  
args = parser.parse_args()  

file = args.file_name

 
if __name__ == "__main__":
 
    body_estimation = Body('model/body_pose_model.pth')
 
    target_image_path = file
    oriImg = cv2.imread(target_image_path)  # B,G,R order
    candidate, subset = body_estimation(oriImg)
    canvas = copy.deepcopy(oriImg)
    canvas = util.draw_bodypose(canvas, candidate, subset)
 
    basename_name = os.path.splitext(os.path.basename(target_image_path))[0]
 
    result_image_path = "result/pose_" + basename_name + ".jpg"
    cv2.imwrite(result_image_path, canvas)

Terminal

electron@diynoMacBook-Pro pytorch-openpose % ls images/21stEmon.jpg                                            
images/21stEmon.jpg
electron@diynoMacBook-Pro pytorch-openpose %

Terminal

electron@diynoMacBook-Pro pytorch-openpose %  python3 test2.py ./images/21stEmon.jpg                         
electron@diynoMacBook-Pro pytorch-openpose %  ls result/            
pose_21stEmon.jpg	pose_test.jpg
electron@diynoMacBook-Pro pytorch-openpose %

骨格は推定できなかった

この名画面の骨格推定は成功した

Terminal

electron@diynoMacBook-Pro pytorch-openpose % ls images/moon_walker.jpg             
images/moon_walker.jpg
electron@diynoMacBook-Pro pytorch-openpose %

Terminal

electron@diynoMacBook-Pro pytorch-openpose % python3 test2.py ./images/moon_walker.jpg 
electron@diynoMacBook-Pro pytorch-openpose % ls result/pose_moon_walker.jpg 
result/pose_moon_walker.jpg
electron@diynoMacBook-Pro pytorch-openpose %