dlibで顔器官検出してOpenCVでドロネー図を描く

概要

dlibで顔器官を認識した後に、それぞれの顔パーツに分解するために三角形で分割します。
複数の点を三角形で表す際にドロネー図というものを利用します。
詳しくは下記記事を参照。

• Delaunay Triangulation and Voronoi Diagram using OpenCV ( C++ / Python )
• OpenCV C++/IFでボロノイ図・ドロネー図を描く

結果

画像が荒いですがこんな感じになります。

ソースコード

dlibのサンプルコードのwebcam_face_pose_ex.cppを改変しています。

        cv::VideoCapture cap(0);
        if (!cap.isOpened())
        {
            cerr << "Unable to connect to camera" << endl;
            return 1;
        }

        image_window win;

        // Load face detection and pose estimation models.
        frontal_face_detector detector = get_frontal_face_detector();
        shape_predictor pose_model;
        deserialize("shape_predictor_68_face_landmarks.dat") >> pose_model;

        // Grab and process frames until the main window is closed by the user.
        while(!win.is_closed())
        {
            // Grab a frame
            cv::Mat temp;
            cap >> temp;
            // Turn OpenCV's Mat into something dlib can deal with.  Note that this just
            // wraps the Mat object, it doesn't copy anything.  So cimg is only valid as
            // long as temp is valid.  Also don't do anything to temp that would cause it
            // to reallocate the memory which stores the image as that will make cimg
            // contain dangling pointers.  This basically means you shouldn't modify temp
            // while using cimg.
            cv_image<bgr_pixel> cimg(temp);

            // Detect faces 
            std::vector<rectangle> faces = detector(cimg);
            
            // Find the pose of each face.
            std::vector<full_object_detection> shapes;
            for (unsigned long i = 0; i < faces.size(); ++i)
                shapes.push_back(pose_model(cimg, faces[i]));

            // Display it all on the screen
            win.clear_overlay();
            win.set_image(cimg);
            win.add_overlay(render_face_detections(shapes));
            
            // Subdiv2D初期化
            cv::Subdiv2D subdiv;
            subdiv.initDelaunay(cv::Rect(0, 0, temp.cols, temp.rows));
            
            // 点を追加
            for (unsigned long i = 0; i < shapes.size(); ++i)
            {
                full_object_detection& d = shapes[i];
                for (unsigned long j = 1; j <= 67; ++j) {
                    if(d.part(j).y() < 0 || d.part(j).y() >= temp.rows || d.part(j).x() >= temp.cols || d.part(j).x() < 0 ) {
                        break;
                    }
                    subdiv.insert(cv::Point2f(d.part(j).x(), d.part(j).y()));
                }
            }
            
            // ドロネー図を取得
            std::vector<cv::Vec4f> edgeList;
            subdiv.getEdgeList(edgeList);
            
            // ドロネー図を描画
            for(auto edge = edgeList.begin(); edge != edgeList.end(); edge++)
            {
                std::vector<image_window::overlay_line> triangleLine;
                triangleLine.push_back(image_window::overlay_line(dlib::vector<long, 2>((long)edge->val[0], (long)edge->val[1]), dlib::vector<long, 2>((long)edge->val[2], (long)edge->val[3]), rgb_pixel(200,200,0)));
                win.add_overlay(triangleLine);
            }
        }