3D Face Detection with RealSense D400 series and ROS

#TL;DR

• Intel discontinued developing a RealSense SDK with face detection and so on[1].
• However, we can use some face detector via wrappers in Intel RealSense SDK 2.0 (v2.8.1).
• Then, I explain how to use a face tracker of ROS packages via a ROS wrapper developed by Intel.

Prerequisites

• PC with x86 CPU and a USB 3.0 port
• Ubuntu 16.04
• RealSense D415 or D435

Install

1. Install ROS kinetic from the website. I recommend to install ros-kinectic-desktop-full.

   • http://wiki.ros.org/kinetic/Installation/Ubuntu

2. Download the source code of Intel RealSense SDK 2.0 (build 2.8.1).

1. Install the Intel RealSense SDK

   • https://github.com/IntelRealSense/librealsense/blob/master/doc/installation.md

2. Download the source code of ROS Wrapper 2.0 for Intel RealSense Devices (build 2.0.1).

   • https://github.com/intel-ros/realsense/releases/tag/2.0.1

3. Install the ROS Wrapper.

   • https://github.com/intel-ros/realsense/tree/2.0.1

4. Download the person package (zip) including the face detector developed by Willow Garage.

   • https://github.com/wg-perception/people/releases/tag/1.0.11

5. Install the person package.

cd ~/Downloads
unzip people-1.0.11
cp -r people-1.0.11 ~/catkin_ws/src/
cd ~/catkin_ws/

#Navigation package was required by people package.
sudo apt-get install ros-kinetic-navigation

catkin_make

Usage

• Open a terminal and type the following command to launch your RealSense camera.

roslaunch realsense_ros_camera rs_camera.launch

• Open another terminal and type the following command to launch the face detector.

roslaunch face_detector face_detector.rgbd.launch \
 rgb_ns:=color image_topic:=image_raw \
 depth_ns:=depth depth_topic:=image_raw

• Open another terminal and type the following command to launch a visualizer　(rviz).

rosrun rviz rviz

• Then, you'll see the following rviz window.
  

• Click "Fixed Frame" tab, and select "camera_link" to fix the world coordinate in the rviz.
  

• Push "Add" button, and select "/face_detector/faces_cloud/PointCloud" to show your face position.
  

• [Optional] To easy to see your face position, Click "PointCloud". And change "Flat Squares" to "Spheres" in "Style" tab. Finnaly, change "0.01" to "0.2" in "Size(m)" tab.
  

Result

You can find your face position as a red point on the rviz window like the following one.

And you can find the messages in the face_detector terminal.

[ INFO] [s.s]: Detection duration = 0.0598671sec
[ INFO] [s.s]: Found face to match with id 0
[ INFO] [s.s]: Number of faces found: 1, number with good depth and size: 1

Conclusion

Now, you can face detection via ROS.
You can also try another ROS pacakge.

Enjoy ROS life.

Thank you for your reading.

See also

[1] https://software.intel.com/en-us/realsense-sdk-windows-eol
[2] http://wiki.ros.org/face_detector