Go to Qiita Advent Calendar 2024 Top
LoginSignup
20
31

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?

More than 3 years have passed since last update.

3D SensorAdvent Calendar 2019

Day 3
@SatoshiGachiFujimoto(藤本 賢志)

PythonでRealSenseD400シリーズ入門（Windows10、Python3.6）

Posted at

はじめに

3D Sensor Advent Calendar 2019の3日目の記事です。
今日は、PythonでRealSenseD400シリーズに入門しましょう。（というか、備忘録）

センサーについてはこちらがよくまとまっていますのでご参考まで。
【記事】RealSense D415とD435の比較と選び方
【記事】RealSense D415とD435 および D435iとT265の比較と選び方

開発環境

・Windows 10
・Anaconda 2019.10
・Python 3.6.9
・opencv-python 4.1.1.26
・opencv-contrib-python 4.1.1.26
・pyrealsense2 2.30.0.1184
・open3d-python 0.7.0.0
・RealSenseD435（D415／D435i）

RealSense SDK のインストール

Intel® RealSense™ SDK 2.0 (build 2.29.0) の Intel.RealSense.SDK.exe を実行し、インストールしてください。

Anaconda のインストール

1．Anaconda3-2019.10-Windows-x86_64.exe を実行し、インストールする
2．Anaconda Promptを開き、仮想環境を作成する

(base) C:\Users\xxx> conda create -n py36 python=3.6 anaconda

3．仮想環境を立ち上げる

(base) C:\Users\xxx> conda activate py36
(py36) C:\Users\xxx>

各ライブラリのインストール

(py36) C:\Users\xxx> pip install open3d-python
(py36) C:\Users\xxx> pip install pyrealsense2
(py36) C:\Users\xxx> pip install opencv-python
(py36) C:\Users\xxx> pip install opencv-contrib-python

※アップデートする場合、pip installの後に***-U***オプションをつける

サンプルコードのダウンロード

・OpenCV4.1.1 のサンプルはこちら
・OpenCV Contrib 4.1.1 のサンプルはこちら
・RealSense のサンプルはこちら
・Open3Dのサンプルはこちら

RealSenseのサンプルコードをやってみる

ダウンロードした librealsense-2.29.0\wrappers\python\examples へ移動

(py36) C:\Users\xxx> cd librealsense-2.29.0\wrappers\python\examples

align-depth2color.py

RGBD画像の取得、表示するプログラム

(py36) C:\Users\xxx> python align-depth2color.py
Depth Scale is:  0.0010000000474974513

qキーで閉じる

export_ply_example.py

1フレーム読み取り、3D点群を.plyとして保存するプログラム

(py36) C:\Users\xxx>python export_ply_example.py
Saving to 1.ply...
Done

examplesフォルダ以下に1.plyが生成されているので、MeshLab2016.12等で開いて確認してみましょう。

frame_queue_example.py

slower processing functionがどうとか

(py36) C:\Users\xxx> python frame_queue_example.py
Slow callback
1
2
3
・・・
Slower callback + queue
1
2
3
・・・
138
139
140
Slower callback + keeping queue
__init__(): incompatible constructor arguments. The following argument types are supported:
    1. pyrealsense2.pyrealsense2.frame_queue(arg0: int)
    2. pyrealsense2.pyrealsense2.frame_queue()

Invoked with: 50; kwargs: keep_frames=True

opencv_pointcloud_viewer.py

OpenCVで3D点群を表示するプログラム

(py36) C:\Users\xxx> python opencv_pointcloud_viewer.py

qキーで閉じる

opencv_viewer_example.py

OpenCVでDepth画像、Color画像を表示するプログラム

(py36) C:\Users\xxx> python opencv_viewer_example.py

コマンドプロンプトからCtrl+Cで閉じる

cv2.waitKey(1)のところを

if cv2.waitKey(1) & 0xFF == ord('q'):
    break

に書き換えると、qキーで閉じれる

pybackend_example_1_general.py

(py36) C:\Users\xxx> python pybackend_example_1_general.py
Traceback (most recent call last):
  File "pybackend_example_1_general.py", line 9, in <module>
    import pybackend2 as rs
ModuleNotFoundError: No module named 'pybackend2'

pip install できなそうだからやめとく
ModuleNotFoundError: No module named 'pybackend2' #3204

pyglet_pointcloud_viewer.py

pygletを用いて3D点群を表示するプログラム

(py36) C:\Users\xxx> pip install pyglet
(py36) C:\Users\xxx> pyglet_pointcloud_viewer.py
Traceback (most recent call last):
  File "pyglet_pointcloud_viewer.py", line 151, in <module>
    fps_display = pyglet.clock.ClockDisplay()
AttributeError: module 'pyglet.clock' has no attribute 'ClockDisplay'

Python2っぽい

python-rs400-advanced-mode-example.py

D400シリーズのパラメータを取得するプログラム

(py36) C:\Users\xxx> python python-rs400-advanced-mode-example.py
Found device that supports advanced mode: Intel RealSense D435
Advanced mode is enabled
Depth Control:
 minusDecrement: 10, deepSeaMedianThreshold: 500, scoreThreshA: 1, scoreThreshB: 2047, textureDifferenceThreshold: 0, textureCountThreshold: 0, deepSeaSecondPeakThreshold: 325, deepSeaNeighborThreshold: 7, lrAgreeThreshold: 24
RSM:
 rsmBypass: 0, diffThresh: 4, sloRauDiffThresh: 1, removeThresh: 63
RAU Support Vector Control:
 minWest: 1, minEast: 1, minWEsum: 3, minNorth: 1, minSouth: 1, minNSsum: 3, uShrink: 3, vShrink: 1
Color Control:
 disableSADColor: 0, disableRAUColor: 0, disableSLORightColor: 0, disableSLOLeftColor: 0, disableSADNormalize: 0
RAU Thresholds Control:
 rauDiffThresholdRed: 51, rauDiffThresholdGreen: 51, rauDiffThresholdBlue: 51
SLO Color Thresholds Control:
 diffThresholdRed: 72, diffThresholdGreen: 72, diffThresholdBlue: 72
SLO Penalty Control:
 sloK1Penalty: 60, sloK2Penalty: 342, sloK1PenaltyMod1: 105, sloK2PenaltyMod1: 190, sloK1PenaltyMod2: 70, sloK2PenaltyMod2: 130
HDAD:
 lambdaCensus: 26, lambdaAD: 800, ignoreSAD: 0
Color Correction:
 colorCorrection1: 0.461914, colorCorrection2: 0.540039, colorCorrection3: 0.540039, colorCorrection4: 0.208008, colorCorrection5: -0.332031, colorCorrection6: -0.212891, colorCorrection7: -0.212891, colorCorrection8: 0.68457, colorCorrection9: 0.930664, colorCorrection10: -0.553711, colorCorrection11: -0.553711, colorCorrection12: 0.0458984
Depth Table:
 depthUnits: 1000, depthClampMin: 0, depthClampMax: 65536, disparityMode: 0, disparityShift: 0
Auto Exposure Control:
 Mean Intensity Set Point: 1536
Census:
 uDiameter: 9, vDiameter: 9
Depth Control Min Values:
  minusDecrement: 0, deepSeaMedianThreshold: 0, scoreThreshA: 0, scoreThreshB: 0, textureDifferenceThreshold: 0, textureCountThreshold: 0, deepSeaSecondPeakThreshold: 0, deepSeaNeighborThreshold: 0, lrAgreeThreshold: 0
Depth Control Max Values:
  minusDecrement: 255, deepSeaMedianThreshold: 1023, scoreThreshA: 1023, scoreThreshB: 4095, textureDifferenceThreshold: 4095, textureCountThreshold: 1023, deepSeaSecondPeakThreshold: 1023, deepSeaNeighborThreshold: 1023, lrAgreeThreshold: 2047
After Setting new value, Depth Control:
 minusDecrement: 10, deepSeaMedianThreshold: 500, scoreThreshA: 511, scoreThreshB: 2047, textureDifferenceThreshold: 0, textureCountThreshold: 0, deepSeaSecondPeakThreshold: 325, deepSeaNeighborThreshold: 7, lrAgreeThreshold: 24
Controls as JSON:
 {
    "aux-param-autoexposure-setpoint": "1536",
    "aux-param-colorcorrection1": "0.461914",
    "aux-param-colorcorrection10": "-0.553711",
    "aux-param-colorcorrection11": "-0.553711",
    "aux-param-colorcorrection12": "0.0458984",
    "aux-param-colorcorrection2": "0.540039",
    "aux-param-colorcorrection3": "0.540039",
    "aux-param-colorcorrection4": "0.208008",
    "aux-param-colorcorrection5": "-0.332031",
    "aux-param-colorcorrection6": "-0.212891",
    "aux-param-colorcorrection7": "-0.212891",
    "aux-param-colorcorrection8": "0.68457",
    "aux-param-colorcorrection9": "0.930664",
    "aux-param-depthclampmax": "65536",
    "aux-param-depthclampmin": "0",
    "aux-param-disparityshift": "0",
    "controls-autoexposure-auto": "True",
    "controls-autoexposure-manual": "8500",
    "controls-color-autoexposure-auto": "True",
    "controls-color-autoexposure-manual": "156",
    "controls-color-backlight-compensation": "0",
    "controls-color-brightness": "0",
    "controls-color-contrast": "50",
    "controls-color-gain": "64",
    "controls-color-gamma": "300",
    "controls-color-hue": "0",
    "controls-color-power-line-frequency": "3",
    "controls-color-saturation": "64",
    "controls-color-sharpness": "50",
    "controls-color-white-balance-auto": "True",
    "controls-color-white-balance-manual": "4600",
    "controls-depth-gain": "16",
    "controls-laserpower": "150",
    "controls-laserstate": "on",
    "ignoreSAD": "0",
    "param-amplitude-factor": "0",
    "param-autoexposure-setpoint": "1536",
    "param-censusenablereg-udiameter": "9",
    "param-censusenablereg-vdiameter": "9",
    "param-censususize": "9",
    "param-censusvsize": "9",
    "param-depthclampmax": "65536",
    "param-depthclampmin": "0",
    "param-depthunits": "1000",
    "param-disableraucolor": "0",
    "param-disablesadcolor": "0",
    "param-disablesadnormalize": "0",
    "param-disablesloleftcolor": "0",
    "param-disableslorightcolor": "0",
    "param-disparitymode": "0",
    "param-disparityshift": "0",
    "param-lambdaad": "800",
    "param-lambdacensus": "26",
    "param-leftrightthreshold": "24",
    "param-maxscorethreshb": "2047",
    "param-medianthreshold": "500",
    "param-minscorethresha": "511",
    "param-neighborthresh": "7",
    "param-raumine": "1",
    "param-rauminn": "1",
    "param-rauminnssum": "3",
    "param-raumins": "1",
    "param-rauminw": "1",
    "param-rauminwesum": "3",
    "param-regioncolorthresholdb": "0.0499022",
    "param-regioncolorthresholdg": "0.0499022",
    "param-regioncolorthresholdr": "0.0499022",
    "param-regionshrinku": "3",
    "param-regionshrinkv": "1",
    "param-robbinsmonrodecrement": "10",
    "param-robbinsmonroincrement": "10",
    "param-rsmdiffthreshold": "4",
    "param-rsmrauslodiffthreshold": "1",
    "param-rsmremovethreshold": "0.375",
    "param-scanlineedgetaub": "72",
    "param-scanlineedgetaug": "72",
    "param-scanlineedgetaur": "72",
    "param-scanlinep1": "60",
    "param-scanlinep1onediscon": "105",
    "param-scanlinep1twodiscon": "70",
    "param-scanlinep2": "342",
    "param-scanlinep2onediscon": "190",
    "param-scanlinep2twodiscon": "130",
    "param-secondpeakdelta": "325",
    "param-texturecountthresh": "0",
    "param-texturedifferencethresh": "0",
    "param-usersm": "1",
    "param-zunits": "1000"
}

パラメータがいっぱい

python-tutorial-1-depth.py

Depthを取得して文字で表現するプログラム

(py36) C:\Users\xxx> python python-tutorial-1-depth.py

read_bag_example.py

ROSでよく使う.bagファイルを読み込み、表示するプログラム

１．Interl RealSense Viewerを用いて、Recordボタンを押し録画。
２．.bagファイルができているので、それを読み込む。

(py36) C:\Users\xxx> python read_bag_example.py -i 20191203_143057.bag

ESCキーで閉じる

FPS-解像度の対応表

解像度に対する最大FPS

Resolution Depth RGB
320, 180 　 60
320, 240 　 60
424, 240 90 60
480, 270 90 　
640, 360 90 60
640, 480 90 60
848, 480 90 60
960, 540 　 60
1280, 720 30 30
1920, 1080 30

FPSに対する最大解像度

FPS Depth RGB
6 1280, 720 1920, 1080
15 1280, 720 1920, 1080
30 1280, 720 1920, 1080
60 848, 480 960, 540
90 848, 480

まとめ

RealSenseのサンプルをひととおり動かしてみました。open3dやopencvは今後紹介したいと思います。
明日は @UnaNancyOwen さんによる「みんな大好きOpenNI2でAzure Kinectを動かしてみる」です。お楽しみに！

Intel RealSense Viewer のフィルター項目一覧

Stereo Module

  • Emitter Enabled
  • Enable Auto Exposure Set ROI
  • Controls
    • Exposure
    • Gain
    • Laser Power
    • Asic Temperature
    • Error Polling Enabled
    • Projector Temperature
    • Output Trigger Enabled
    • Depth Units
    • Stereo Baseline
    • Inter Cam Sync Mode
    • Emitter On Off
    • Global Time Enabled
  • Advanced Controls
    • Depth Control
      • DS Second Peak Threshold
      • DS Neighbor Threshold
      • DS Median Threshold
      • Estimate Median Increment
      • Estimate Median Decrement
      • Score Minimum Threshold
      • Score Maximum Threshold
      • DS LR Threshold
      • Texture Count Threshold
      • Texture Difference Threshold
    • Rsm
      • RSM Bypass
      • Disparity Difference Threshold
      • SLO RAU Difference Threshold
      • Remove Threshold
    • Rau Support Vector Control
      • Min West
      • Min East
      • Min WE Sum
      • Min North
      • Min South
      • Min NS Sum
      • U Shrink
      • V Shrink
    • Color Control
      • Disable SAD Color
      • Disable RAU Color
      • Disable SLO Right Color
      • Disable SLO Left Color
      • Disable SAD Normalize
    • Rau Color Thresholds Control
      • Diff Threshold Red
      • Diff Threshold Green
      • Diff Threshold Blue
    • SLO Color Thresholds Control
      • Diff Threshold Red
      • Diff Threshold Green
      • Diff Threshold Blue
    • SLO Penalty Control
      • K1 Penalty
      • K2 Penalty
      • K1 Penalty Mod1
      • K1 Penalty Mod2
      • K2 Penalty Mod1
      • K2 Penalty Mod2
    • HDAD
      • Ignore SAD
      • AD Lambda
      • Census Lambda
    • Color Correction
      • Color Correction 1
      • Color Correction 2
      • Color Correction 3
      • Color Correction 4
      • Color Correction 5
      • Color Correction 6
      • Color Correction 7
      • Color Correction 8
      • Color Correction 9
      • Color Correction 10
      • Color Correction 11
      • Color Correction 12
    • Depth Table
      • Depth Units
      • Depth Clamp Min
      • Depth Clamp Max
      • Disparity Mode
      • Disparity Shift
    • AE Control
      • Mean Intensity Set Point
    • Census Enable Reg
      • u-Diameter
      • v-Diameter
    • Disparity Modulation
      • A Factor
  • Depth Visualization
    • Visual Preset
      • Dynamic, Fixed, Near, Far
    • Color Scheme
      • Jet, Classic, White to Black, Black to White, Bio, Cold, Warm
    • Histogram Equalization Enabled
    • Min Distance
    • Max Distance
  • Post-Processing
    • Decimation Filter
      • Filter Magnitude
    • Threshold Filter
      • Min Distance
      • Max Distance
    • Depth to Disparity
    • Spatial Filter
      • Filter Magnitude
      • Filter Smooth Alpha
      • Filter Delta
      • Holes filling mode
        • Disabled, 2-pixel radius, 4-pixel radius, 8-pixel radius, 16-pixel radius, Unlimited
    • Temporal Filter
      • Filter Smooth Alpha
      • Filter Smooth Delta
      • Persistency mode
        • Disabled, Valid in 8/8, Valid in 2/, Valid in 2/8, Valid in 1/la, Valid in 1/la, Valid in 2/
    • Hole Filling Filter
      • Hole Filling mode
        • Fill from Left, Farest from a, Nearest from
    • Disparity to Depth

RGB Camera

  • Color
    • YUYV, BGR8, RGBA8, BGRA8, Y16, RGB8, RAW16
  • Enable Auto Exposure
  • Controls
    • Backlight Compensation
    • Brightness
    • Contrast
    • Exposure
    • Gain
    • Gamma
    • Hue
    • Saturation
    • Sharpness
    • White Balance
    • Enable Auto White Balance
    • Power Line Frequency
    • Auto Exposure Priority
    • Global Time Enabled
  • Post-Processing
    • Decimation Filter
      • Filter Magnitude
20
31
1

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
Sign upLogin
20
31

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?