360度カメラでDeepStreamSDKのnvdewarperの最新版を試してみる

必要なソフトウェア

• DeepStreamSDK 5.1
• https://github.com/NVIDIA-AI-IOT/Deepstream-Dewarper-App のlibnvdgst_dewarper.soとnvds_dewarper_meta.h

必要なハードウェア

画像	型番
	OCAM-VRU01BK
	USB TypeCメス->USB TypeAオス変換ケーブル

DeepStreamSDK用のDockerを展開

xhost +

sudo docker run -it --ipc=host --net=host --runtime nvidia  --device /dev/video0:/dev/video0:mwr -e DISPLAY=$DISPLAY -w /opt/nvidia/deepstream/deepstream-5.1  -v /tmp/argus_socket:/tmp/argus_socket -v /tmp/.X11-unix/:/tmp/.X11-unix nvcr.io/nvidia/deepstream-l4t:5.1-21.02-samples

必要なパッケージをインストール

apt update
apt install v4l-utils vim -y

動作テスト

gst-launch-1.0 v4l2src device="/dev/video0" ! \
'image/jpeg, width=1472, height=736, framerate=30/1' ! \
nvv4l2decoder mjpeg=1 ! \
nvvideoconvert ! \
nvegltransform ! \
nveglglessink sync=0

nvdewarperの最新ライブラリに更新

cd /opt/nvidia/deepstream/deepstream-5.1/lib/gst-plugins/
mv libnvdsgst_dewarper.so libnvdsgst_dewarper_backup.so
wget https://github.com/NVIDIA-AI-IOT/Deepstream-Dewarper-App/raw/main/plugin_libraries/jetson_bin/libnvdsgst_dewarper.so
chmod 755 libnvdsgst_dewarper.so

cd /opt/nvidia/deepstream/deepstream-5.1/sources/includes/
mv nvds_dewarper_meta.h nvds_dewarper_meta_backup.h
wget https://raw.githubusercontent.com/NVIDIA-AI-IOT/Deepstream-Dewarper-App/main/nvds_dewarper_meta.h

nvdewarperで歪み補正

dewarp_config.txt

[property]

[surface0]
# 1=PushBroom, 2=VertRadCyl 3= Perspective_Perspective FISH_PERSPECTIVE=4, FISH_FISH=5, FISH_CYL=6, FISH_EQUIRECT=7,
# FISH_PANINI=8, PERSPECTIVE_EQUIRECT=9, PERSPECTIVE_PANINI=10, EQUIRECT_CYLINDER=11, EQUIRECT_EQUIRECT=12 EQUIRECT_FISHEYE=13,
# EQUIRECT_PANINI=14, EQUIRECT_PERSPECTIVE=15, EQUIRECT_PUSHBROOM=16, EQUIRECT_STEREOGRAPHIC=17, EQUIRECT_VERTCYLINDER=18
projection-type=15
surface-index=0
#dewarped surface parameters
width=600
height=480
top-angle=15
bottom-angle=-15
pitch=0
yaw=0
roll=0
src-x0=380
src-y0=368
focal-length=850

[surface1]
# 1=PushBroom, 2=VertRadCyl 3= Perspective_Perspective FISH_PERSPECTIVE=4, FISH_FISH=5, FISH_CYL=6, FISH_EQUIRECT=7,
# FISH_PANINI=8, PERSPECTIVE_EQUIRECT=9, PERSPECTIVE_PANINI=10, EQUIRECT_CYLINDER=11, EQUIRECT_EQUIRECT=12 EQUIRECT_FISHEYE=13,
# EQUIRECT_PANINI=14, EQUIRECT_PERSPECTIVE=15, EQUIRECT_PUSHBROOM=16, EQUIRECT_STEREOGRAPHIC=17, EQUIRECT_VERTCYLINDER=18
projection-type=15
surface-index=1
#dewarped surface parameters
width=600
height=480
top-angle=15
bottom-angle=-15
pitch=0
yaw=0
roll=0
src-x0=1080
src-y0=368
focal-length=850

gst-launch-1.0 v4l2src device="/dev/video0" ! 'image/jpeg, width=1472, height=736, framerate=30/1' ! nvv4l2decoder mjpeg=1 ! nvvideoconvert ! nvdewarper config-file=dewarp_config.txt  ! m.sink_0 nvstreammux name=m width=600 height=480 batch-size=2 num-surfaces-per-frame=2 ! nvmultistreamtiler width=600 height=480 ! nvegltransform ! nveglglessink sync=0

参考