[メモ](らずぱい)Pan-Tilt HATとカメラで、顔追跡(Face Tracking) #Python

概要

PimoroniのPan-Tilt HAT Full Kit w/ Micro Servosを購入した
- https://shop.pimoroni.com/products/pan-tilt-hat
- https://www.adafruit.com/products/3353 - Pimoroni Pan-Tilt HAT for Raspberry Pi
- https://www.adafruit.com/products/1967 - Mini Pan-Tilt Kit - Assembled with Micro Servos
らずぱいカメラモジュール V2.1 をくっつけて、サンプルのフェイストラッキングのデモを動かす。
https://github.com/pimoroni/PanTiltFacetracker
(Neo pixel stickは買ってない..)

Pythonでラクに制御できる

実行例）pan()とtilt()を使う。-90～90度で指定

pi@raspberrypi:~ $ python
Python 2.7.9 (default, Sep 17 2016, 20:26:04) 
[GCC 4.9.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from pantilthat import *
>>> angle_pan  = 15   # 角度:-90 - +90
>>> angle_tilt = -30  # 角度:-90 - +90
>>> pan(angle_pan)
>>> tilt(angle_tilt)
>>> quit()
pi@raspberrypi:~ $

環境

Raspberry Pi 2 / 3
Camera Module V2.1 (V1.3でも動作した)
Raspbian: 2017-01-11-raspbian-jessie.img (OpenCVで、ウィンドウにカメラ画像を表示させるので、Liteじゃないよ)

手順

Pan-Tiltキットの組み立て Assembling Pan-Tilt HAT
https://learn.pimoroni.com/tutorial/sandyj/assembling-pan-tilt-hat
写真を見ながらやればできるかと。
- Pan-TiltモジュールをHATの基板に固定
  - 黒いナイロンねじは、長めなので、きってしまうのもいいね(It's a good idea to trim the protruding end of the nylon bolt with a pair of scissors or tin snips,～)らしい
- サーボモータの線をうらのコネクタに挿す (GND=茶, 5V=赤, PWM-黄)
  - Pan(クビを横に振る)をチャンネル１ (SERVO1)
  - Tilt(クビを縦に振る)をチャネル２ (SERVO2)
- カメラ固定台組み立て
  - V1.3用とV2.1用で、表に出る側の切り込みが違うんだと思う。
  - 白いねじは長めにできてるので、切ってもいいね(Again, it's a good idea to trim off the excess nylon bolt with a pair of scissors or tin snips.)、らしい
- カメラ固定台をPan-Tiltモジュールにつける
  - カメラのケーブルは上にでるよ
- Neopixel stick ... 購入してないので、スキップ。
- カメラのケーブルをらずぱいに挿す。
  - ラズパイの差込口の上にある部品(茶色?とか黒とか?)をちょいと持ち上げる1mmくらい。力を入れすぎると、壊れるよ
  - カメラケーブルを差し込んで
  - 持ち上げたのを、押し込む、と固定される。
- らずぱい40ピンにHATを挿す。
起動システムの準備(いつもの)
- Raspbianをやいて
- sudo apt-get update && sudo apt-get upgrade -y #パッケージ更新
Pan-Tilt 関係のパッケージをインストール
curl -sS https://get.pimoroni.com/pantilthat | bash
で何か聞かれたら、y [Enter]
カメラモジュールの有効化
sudo raspi-config nonint do_camera 0 # 0:Enable/1:Disable
sudo reboot # 再起動
PanTiltモジュールの動作確認
- ターミナルを開いて、
- python[Enter] で、python起動
- 以下、コピペ
```
from pantilthat import *
angle_pan  = 15   # 角度:-90 - +90
angle_tilt = -30  # 角度:-90 - +90
pan(angle_pan)
tilt(angle_tilt)
quit()
```
- 動く？数字を変えると、向きがかわる angle_pan=0 / angle_tilt = 0で正面向く感じ？

フェイストラッキングのデモプログラム

必要なパッケージ導入
sudo apt-get install python-smbus python-opencv opencv-data

git clone

git clone https://github.com/pimoroni/PanTiltFacetracker
cd PanTiltFacetracker

実行
./facetracker_lbp.py
- ウィンドウが開いて、カメラの画像が映る
- 顔を検出すると、それがセンターにくるように、Pan-Tiltモジュールが動く.

その他

ログ

pi@raspberrypi:~ $ curl -sS https://get.pimoroni.com/pantilthat | bash

This script will install everything needed to use
Pan-Tilt HAT

Always be careful when running scripts and commands
copied from the internet. Ensure they are from a
trusted source.

If you want to see what this script does before
running it, you should run:
\curl -sS https://get.pimoroni.com/pantilthat

Note: Pan-Tilt HAT requires I2C communication

Do you wish to continue? [y/N] y

Checking environment...
Updating apt indexes...
.........
Reading package lists...
.........

Checking hardware requirements...

I2C must be enabled for Pan-Tilt HAT to work
I2C is now enabled

Checking packages required by I2C interface...

Pan-Tilt HAT comes with examples and documentation that you may wish to install.
Performing a full install will ensure those resources are installed,
along with all required dependencies. It may however take a while!

Do you wish to perform a full install? [y/N] y
Checking install requirements...

Checking for dependencies...

Installing python-pantilthat...
install ok installed

Installing python3-pantilthat...
install ok installed

Checking for additional software...

python-picamera is already installed
python3-picamera is already installed

Downloading examples and documentation...
Resources for your Pan-Tilt HAT were copied to
/home/pi/Pimoroni/pantilthat

All done!

Enjoy your Pan-Tilt HAT!

pi@raspberrypi:~ $ sudo reboot

なんか、色がおかしい？
赤いのを映すと青になる、青いのが赤になる..RGB並びの設定がおかしい？
OpenCVから、v4l2の設定ができない? CV_CAP_PROP_FORMAT ??
直し方が分からないので、PythonのPiCameraを使用するように変更。

sudo apt-get install python-picamera

picameraを使用するようにごにょごにょしたコード。

#!/usr/bin/env python
## http://qiita.com/mt08/items/97d89a09e5129e10f88c

from picamera.array import PiRGBArray
from picamera import PiCamera
import cv2, sys, time, os
from pantilthat import *

# Frame Size. Smaller is faster, but less accurate.
# Wide and short is better, since moving your head
# vertically is kinda hard!
FRAME_W = 192
FRAME_H = 112

# Default Pan/Tilt for the camera in degrees.
# Camera range is from -90 to 90
cam_pan = 90
cam_tilt = 60

# Set up the CascadeClassifier for face tracking
#cascPath = 'haarcascade_frontalface_default.xml' # sys.argv[1]
cascPath = '/usr/share/opencv/lbpcascades/lbpcascade_frontalface.xml'
faceCascade = cv2.CascadeClassifier(cascPath)

# Set up the capture with our frame size
camera = PiCamera()
camera.resolution = (FRAME_W, FRAME_H)
camera.framerate = 32
rawCapture = PiRGBArray(camera, size=(FRAME_W, FRAME_H))
time.sleep(0.1)

# Turn the camera to the default position
pan(cam_pan-90)
tilt(cam_tilt-90)
light_mode(WS2812)

def lights(r,g,b,w):
    for x in range(18):
        set_pixel_rgbw(x,r if x in [3,4] else 0,g if x in [3,4] else 0,b,w if x in [0,1,6,7] else 0)
    show()

lights(0,0,0,50)

for image in camera.capture_continuous(rawCapture, format="bgr", use_video_port=True):

    frame = image.array

    # This line lets you mount the camera the "right" way up, with neopixels above
    frame = cv2.flip(frame, -1)

    # Convert to greyscale for detection
    gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
    gray = cv2.equalizeHist( gray )

    # Do face detection
    faces = faceCascade.detectMultiScale(gray, 1.1, 3, 0, (10, 10))

    # Slower method 
    '''faces = faceCascade.detectMultiScale(
        gray,
        scaleFactor=1.1,
        minNeighbors=4,
        minSize=(20, 20),
        flags=cv2.cv.CV_HAAR_SCALE_IMAGE | cv2.cv.CV_HAAR_FIND_BIGGEST_OBJECT | cv2.cv.CV_HAAR_DO_ROUGH_SEARCH
    )'''

    lights(50 if len(faces) == 0 else 0, 50 if len(faces) > 0 else 0,0,50)

    for (x, y, w, h) in faces:
        # Draw a green rectangle around the face
        cv2.rectangle(frame, (x, y), (x+w, y+h), (0, 255, 0), 2)

        # Track first face

        # Get the center of the face
        x = x + (w/2)
        y = y + (h/2)

        # Correct relative to center of image
        turn_x  = float(x - (FRAME_W/2))
        turn_y  = float(y - (FRAME_H/2))

        # Convert to percentage offset
        turn_x  /= float(FRAME_W/2)
        turn_y  /= float(FRAME_H/2)

        # Scale offset to degrees
        turn_x   *= 2.5 # VFOV
        turn_y   *= 2.5 # HFOV
        cam_pan  += -turn_x
        cam_tilt += turn_y

        print(cam_pan-90, cam_tilt-90)

        # Clamp Pan/Tilt to 0 to 180 degrees
        cam_pan = max(0,min(180,cam_pan))
        cam_tilt = max(0,min(180,cam_tilt))

        # Update the servos
        pan(int(cam_pan-90))
        tilt(int(cam_tilt-90))

        break

    frame = cv2.resize(frame, (540,300))
    frame = cv2.flip(frame, 1)

    # Display the image, with rectangle
    # on the Pi desktop 
    cv2.imshow('Video', frame)
    key = cv2.waitKey(1) & 0xFF

    # clear the stream in preparation for the next frame
    rawCapture.truncate(0)

    # if the `q` key was pressed, break from the loop
    if key == ord("q"):
        break

# When everything is done, release the capture
cv2.destroyAllWindows()

TODO

スクショとか、動作動画とか、載せる?
笑い男マーク対応?
複数顔検出して、一番大きいやつを追跡とか?