GTC2021 11月の講演で使ったDeepStream/GStreamer pipeline集

JetPack4.6でのDeepStreamSDK6.0の動作状況

GTC2021の講演時、JetPack4.6でDeepStreamSDK6.0がTensorRTバージョン不一致で動かない旨、発表してしまいましたが、DeepStreamSDK6.0 Betaの検証結果で、DeepStreamSDK6.0の正式版リリースではJetPack4.6上でも問題なく動作する事を確認いたしました。発表内容を訂正いたします。

使用機材

USBカメラ	URL
C920	Web
OCAM-VRU01BK	Web

nvvideoconvert(JetPack4.5.1)

gst-launch-1.0 \
v4l2src device="/dev/video0" io-mode=2 ! \
'image/jpeg, width=1280, height=720, framerate=30/1' ! \
jpegdec ! \
tee name=t ! \
queue ! \
nvvideoconvert ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
nvegltransform ! \
nveglglessink \
window-x=50 window-y=0 \
window-width=355 window-height=200 \
t. ! \
queue ! \
nvvideoconvert src-crop=”20:40:640:360” ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
nvegltransform ! \
nveglglessink \
window-x=405 window-y=0 \
window-width=355 window-height=200 \
t. ! \
queue ! \
nvvideoconvert dest-crop=”200:200:640:320” ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
nvegltransform ! \
nveglglessink \
window-x=760 window-y=0 \
window-width=355 window-height=200

nvvidconv(JetPack4.5.1)

gst-launch-1.0 \
v4l2src device="/dev/video0" io-mode=2 ! \
'image/jpeg, width=1280, height=720, framerate=30/1' ! \
jpegdec ! \
tee name=t ! \
queue ! \
nvvidconv ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
nvegltransform ! \
nveglglessink \
window-x=50 window-y=0 \
window-width=355 window-height=200 \
t. ! \
queue ! \
nvvidconv flip-method=1 ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
nvegltransform ! \
nveglglessink \
window-x=405 window-y=0 \
window-width=355 window-height=200 \
t. ! \
queue ! \
nvvidconv flip-method=2 ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
nvegltransform ! \
nveglglessink \
window-x=760 window-y=0 \
window-width=355 window-height=200

360度カメラ x 2個(JetPack4.5.1)

gst-launch-1.0 \
nvcompositor name=m \
sink_0::xpos=0 sink_0::ypos=0 \
sink_1::xpos=0 sink_1::ypos=736 ! \
nvegltransform ! \
nveglglessink sync=no \
v4l2src device=/dev/video0 io-mode=2 ! \
'image/jpeg, width=1472, height=736, framerate=30/1' ! \
jpegdec ! \
nvvidconv ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
m.sink_0 \
v4l2src device=/dev/video1 io-mode=2 ! \
'image/jpeg, width=1472, height=736, framerate=30/1' ! \
jpegdec ! \
nvvidconv ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
m.sink_1

nvdewrper(JetPack4.5.1)

gst-launch-1.0 v4l2src device="/dev/video0" ! \
'image/jpeg, width=1472, height=736, framerate=30/1' ! \
jpegdec ! \
nvvideoconvert ! \
'video/x-raw(memory:NVMM)' ! \
nvdewarper config-file=cam0_dewarp_config.txt  ! \
m.sink_0 nvstreammux name=m width=600 height=480 \
batch-size=2 \
num-surfaces-per-frame=2 ! \
nvmultistreamtiler width=600 height=480 ! \
nvegltransform ! \
nveglglessink sync=0 v4l2src device="/dev/video1" ! \
'image/jpeg, width=1472, height=736, framerate=30/1' ! \
jpegdec ! \
nvvideoconvert ! \
'video/x-raw(memory:NVMM)' ! \
nvdewarper config-file=cam1_dewarp_config.txt ! \
m.sink_1

dewarp_config.txt

gst-launch-1.0 v4l2src device="/dev/video0" ! \
'image/jpeg, width=1472, height=736, framerate=30/1' ! \
jpegdec ! \
nvvideoconvert ! \
'video/x-raw(memory:NVMM)' ! \
nvdewarper config-file=cam0_dewarp_config.txt  ! \
m.sink_0 nvstreammux name=m width=600 height=480 \
batch-size=2 \
num-surfaces-per-frame=2 ! \
nvmultistreamtiler width=600 height=480 ! \
nvegltransform ! \
nveglglessink sync=0 v4l2src device="/dev/video1" ! \
'image/jpeg, width=1472, height=736, framerate=30/1' ! \
jpegdec ! \
nvvideoconvert ! \
'video/x-raw(memory:NVMM)' ! \
nvdewarper config-file=cam1_dewarp_config.txt ! \
m.sink_1

4つのUSBカメラ(nvv4l2decoder未使用)(JetPack4.5.1)

gst-launch-1.0 \
nvcompositor name=m \
sink_0::xpos=0 sink_0::ypos=0 \
sink_1::xpos=960 sink_1::ypos=0  \
sink_2::xpos=0 sink_2::ypos=720 \
sink_3::xpos=960 sink_3::ypos=720 ! \
nvegltransform ! \
nveglglessink sync=no \
v4l2src device=/dev/video0 io-mode=2 ! \
'image/jpeg, width=960, height=720, framerate=30/1' ! \
jpegdec ! \
nvvidconv ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
m.sink_0 \
v4l2src device=/dev/video1 io-mode=2 ! \
'image/jpeg, width=960, height=720, framerate=30/1' ! \
jpegdec ! \
nvvidconv ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
m.sink_1 \
v4l2src device=/dev/video2 io-mode=2 ! \
'image/jpeg, width=960, height=720, framerate=30/1' ! \
jpegdec ! \
nvvidconv ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
m.sink_2 \
v4l2src device=/dev/video3 io-mode=2 ! \
'image/jpeg, width=960, height=720, framerate=30/1' ! \
jpegdec ! \
nvvidconv ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
m.sink_3

4つのUSBカメラ(nvv4l2decoder使用) (JetPack4.6)

gst-launch-1.0 \
nvcompositor name=m \
sink_0::xpos=0 sink_0::ypos=0 \
sink_1::xpos=1280 sink_1::ypos=0  \
sink_2::xpos=0 sink_2::ypos=720 \
sink_3::xpos=1280 sink_3::ypos=720 ! \
nvegltransform ! \
nveglglessink sync=no \
v4l2src device=/dev/video2 io-mode=2 ! \
'image/jpeg, width=1280, height=720, framerate=30/1' ! \
jpegdec ! \
nvvidconv ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
m.sink_0 \
v4l2src device=/dev/video3 io-mode=2 ! \
'image/jpeg, width=1280, height=720, framerate=30/1' ! \
jpegdec ! \
nvvidconv ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
m.sink_1 \
v4l2src device=/dev/video4 io-mode=2 ! \
'image/jpeg, width=1280, height=720, framerate=30/1' ! \
jpegdec ! \
nvvidconv ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
m.sink_2 \
v4l2src device=/dev/video5 io-mode=2 ! \
'image/jpeg, width=1280, height=720, framerate=30/1' ! \
jpegdec ! \
nvvidconv ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
m.sink_3

4カメラでObject Detection(JetPack4.5.1)

gst-launch-1.0 \
nvstreammux name=m batch-size=4 width=960 height=720 ! \
nvinfer config-file-path=/opt/nvidia/deepstream/deepstream-5.1/samples/configs/deepstream-app/config_infer_primary.txt ! \
nvmultistreamtiler rows=2 columns=2 width=960 height=720 ! \
nvvideoconvert ! \
nvdsosd ! \
nvegltransform ! \
nveglglessink sync=0 \
v4l2src device="/dev/video0" ! \
'image/jpeg, width=960, height=720, framerate=30/1' ! \
jpegdec ! \
nvvideoconvert ! \
'video/x-raw(memory:NVMM)' ! \
m.sink_0 \
v4l2src device="/dev/video1" ! \
'image/jpeg, width=960, height=720, framerate=30/1' ! \
jpegdec ! \
nvvideoconvert ! \
'video/x-raw(memory:NVMM)' ! \
m.sink_1 \
v4l2src device="/dev/video2" ! \
'image/jpeg, width=960, height=720, framerate=30/1' ! \
jpegdec ! \
nvvideoconvert ! \
'video/x-raw(memory:NVMM)' ! \
m.sink_2 \
v4l2src device="/dev/video3" ! \
'image/jpeg, width=960, height=720, framerate=30/1' ! \
jpegdec ! \
nvvideoconvert ! \
'video/x-raw(memory:NVMM)' ! \
m.sink_3

4カメラでOptical flowとObject Detection(JetPack4.5.1)

gst-launch-1.0 \
nvstreammux name=m batch-size=4 width=960 height=720 ! \
nvinfer config-file-path=dsanomaly_pgie_config.txt ! \
tee name=t ! \
nvof ! \
nvofvisual ! \
queue ! \
nvmultistreamtiler rows=2 columns=2 width=480 height=360 ! \
nvvideoconvert ! \
nvegltransform ! \
nveglglessink sync=0  sync=0 t. ! \
queue ! \
nvmultistreamtiler rows=2 columns=2 width=480 height=360 ! \
nvvideoconvert ! \
nvdsosd ! \
nvegltransform ! \
nveglglessink sync=0 \
v4l2src device="/dev/video0" ! \
'image/jpeg, width=960, height=720, framerate=30/1' ! \
jpegdec ! \
nvvideoconvert ! \
'video/x-raw(memory:NVMM)' ! \
m.sink_0 \
v4l2src device="/dev/video1" ! \
'image/jpeg, width=960, height=720, framerate=30/1' ! \
jpegdec ! \
nvvideoconvert ! \
'video/x-raw(memory:NVMM)' ! \
m.sink_1 \
v4l2src device="/dev/video2" ! \
'image/jpeg, width=960, height=720, framerate=30/1' ! \
jpegdec ! \
nvvideoconvert ! \
'video/x-raw(memory:NVMM)' ! \
m.sink_2 \
v4l2src device="/dev/video3" ! \
'image/jpeg, width=960, height=720, framerate=30/1' ! \
jpegdec ! \
nvvideoconvert ! \
'video/x-raw(memory:NVMM)' ! \
m.sink_3

GST_DEBUG=2 gst-launch-1.0 -v \
videomixer name=m \
sink_0::xpos=0 sink_0::ypos=0  \
sink_1::xpos=960 sink_1::ypos=0  \
sink_2::xpos=0 sink_2::ypos=720  \
sink_3::xpos=960 sink_3::ypos=720 ! \
nvvideoconvert ! \
'video/x-raw(memory:NVMM)' ! \
mux.sink_0 \
nvstreammux name=mux batch-size=1 width=1920 height=1440 ! \
nvinfer config-file-path=dsanomaly_pgie_config.txt ! \
nvmultistreamtiler rows=1 columns=1 width=960 height=720 ! \
nvvideoconvert ! \
nvegltransform ! \
nveglglessink sync=0 \
v4l2src device=/dev/video0 io-mode=2 ! \
'image/jpeg, width=960, height=720, framerate=30/1' ! \
jpegdec ! \
m.sink_0 \
v4l2src device=/dev/video1 io-mode=2 ! \
'image/jpeg, width=960, height=720, framerate=30/1' ! \
jpegdec ! \
m.sink_1 \
v4l2src device=/dev/video2 io-mode=2 ! \
'image/jpeg, width=960, height=720, framerate=30/1' ! \
jpegdec ! \
m.sink_2 \
v4l2src device=/dev/video3 io-mode=2 ! \
'image/jpeg, width=960, height=720, framerate=30/1' ! \
jpegdec ! \
m.sink_3

転移学習 TAO(JetPack4.5.1)

gst-launch-1.0 \
nvstreammux name=m batch-size=4 width=960 height=720 ! \
nvinfer config-file-path=/home/jetson/deepstream_tao_apps/configs/ssd_tao/pgie_ssd_tao_config.txt ! \
nvmultistreamtiler rows=2 columns=2 width=1920 height=1440 ! \
nvvideoconvert ! \
nvdsosd ! \
nvegltransform ! \
nveglglessink sync=0 \
v4l2src device="/dev/video0" ! \
'image/jpeg, width=960, height=720, framerate=30/1' ! \
jpegdec ! \
nvvideoconvert ! \
'video/x-raw(memory:NVMM)' ! \
m.sink_0 \
v4l2src device="/dev/video1" ! \
'image/jpeg, width=960, height=720, framerate=30/1' ! \
jpegdec ! \
nvvideoconvert ! \
'video/x-raw(memory:NVMM)' ! \
m.sink_1 \
v4l2src device="/dev/video2" ! \
'image/jpeg, width=960, height=720, framerate=30/1' ! \
jpegdec ! \
nvvideoconvert ! \
'video/x-raw(memory:NVMM)' ! \
m.sink_2 \
v4l2src device="/dev/video3" ! \
'image/jpeg, width=960, height=720, framerate=30/1' ! \
jpegdec ! \
nvvideoconvert ! \
'video/x-raw(memory:NVMM)' ! \
m.sink_3