配線
| デバイス名 | ポート | 種類 |
|---|---|---|
| /dev/video0 | cam0 | IMX219 CSI Camera |
| /dev/video1 | cam1 | IMX219 CSI Camera |
| /dev/video2 | USB | Logicool 720p |
| /dev/video3 | USB | Logicool 720p |
| /dev/video4 | USB | Insta360 |
| /dev/video5 | USB | Insta360 |
Jetapckのバージョン確認
cat /etc/nv_tegra_release
# R32 (release), REVISION: 5.1, GCID: 26202423, BOARD: t186ref, EABI: aarch64, DATE: Fri Feb 19 16:50:29 UTC 2021
取り込み
| ライブラリ | 処理 |
|---|---|
| nvarguscamerasrc | ARGUS APIを使用したカメラプラグイン |
| nvv4l2camerasrc | V4L2 APIを使用したカメラプラグイン |
CSIカメラ取り込み(nvarguscamerasrc)
ヘルプの表示
nvgstcapture-1.0 --help
nvarguscamerasrchは、L4T Multimedia APIのLibargus Camera APIがベースになっており、CSIカメラのRawデータをGPUメモリ空間(memory:NVMM)に高速に取り込む事が可能。
cam0を取り込む
gst-launch-1.0 \
nvarguscamerasrc sensor-id=0 ! \
nv3dsink
cam1を取り込む
gst-launch-1.0 \
nvarguscamerasrc sensor-id=1 ! \
nv3dsink
サイズ指定で取り込む
gst-launch-1.0 \
nvarguscamerasrc sensor-id=0 ! \
'video/x-raw(memory:NVMM), width=1280, height=720, framerate=60/1' ! \
nv3dsink
gst-launch-1.0 \
nvarguscamerasrc sensor-id=0 ! \
'video/x-raw(memory:NVMM), width=3264, height=2464, framerate=21/1' ! \
nv3dsink
gst-launch-1.0 \
nvarguscamerasrc sensor-id=0 ! \
nvinfer config-file-path=/opt/nvidia/deepstream/deepstream-6.0/samples/configs/deepstream-app/config_infer_primary.txt ! \
nvmultistreamtiler rows=1 columns=1 width=960 height=720 ! \
nvvideoconvert ! \
nvdsosd ! \
nvegltransform ! \
nveglglessink
USBカメラ
nvv4l2camerasrcを用いて取り込む。nvv4l2camerasrcで取り込んだ映像は、nvv4l2decoderやnvvideoconvert等で、GPUメモリ空間(memory:NVMM)に展開する。
USBカメラのMotionJpegを取り込む
gst-launch-1.0 \
v4l2src device="/dev/video2" ! \
'image/jpeg, width=1280, height=720, framerate=30/1' ! \
nvv4l2decoder mjpeg=1 ! \
nvvidconv ! \
nvegltransform ! \
nveglglessink sync=no
USBカメラのYUYVを取り込む
gst-launch-1.0 \
v4l2src device="/dev/video2" ! \
'video/x-raw, width=640, height=480, framerate=30/1' ! \
nvvideoconvert ! \
nv3dsink
USBカメラのH264を取り込む
gst-launch-1.0 \
v4l2src device="/dev/video4" ! \
'video/x-h264, width=1920, height=960, framerate=30/1' ! \
nvv4l2decoder ! \
nv3dsink sync=no
変換系API
| ライブラリ | 処理 |
|---|---|
| nvvidconv | フォーマット変換、スケーリング処理 |
| nvvideoconvert | データのメモリ空間移動(CPUメモリ空間, GPUメモリ空間) NVMMからNVMMへの移動, RAWからNVMMへの移動, NVMMからRAWへの移動, RAWからRAW bufferへの移動 |
VideoSkin API
| ライブラリ | 処理 |
|---|---|
| nvoverlaysink | (deprecated) OpenMAX ILを使用したVideoSkin |
| nveglglessink | X11やWaylandで使えるEGL/GLESを使用したVideoSkin |
| nv3dsink | EGL/GLESを使用したVideoSkin |
| nvdrmvideosink | DRM（デジタル著作権管理）対応のVideoSkin |
CSI Cameraテスト
gst-launch-1.0 \
nvarguscamerasrc sensor-id=0 ! \
nvoverlaysink
gst-launch-1.0 \
nvarguscamerasrc sensor-id=0 ! \
nvegltransform ! \
nveglglessink
gst-launch-1.0 \
nvarguscamerasrc sensor-id=0 ! \
nv3dsink
USBカメラ(MJPEG)テスト
gst-launch-1.0 \
v4l2src device="/dev/video2" ! \
'image/jpeg, width=1280, height=720, framerate=30/1' ! \
nvv4l2decoder mjpeg=1 ! \
nvvidconv ! \
nvegltransform ! \
nveglglessink sync=no
USBカメラ(YUYV)テスト
gst-launch-1.0 \
v4l2src device="/dev/video2" ! \
'video/x-raw, width=640, height=480, framerate=30/1' ! \
nvvideoconvert ! \
nveglglessink
gst-launch-1.0 \
v4l2src device="/dev/video2" ! \
'video/x-raw, width=640, height=480, framerate=30/1' ! \
nvvideoconvert ! \
nv3dsink
USBカメラ(H264)テスト
gst-launch-1.0 \
v4l2src device="/dev/video4" ! \
'video/x-h264, width=1920, height=960, framerate=30/1' ! \
nvv4l2decoder ! \
nvoverlaysink sync=no
gst-launch-1.0 \
v4l2src device="/dev/video4" ! \
'video/x-h264, width=1920, height=960, framerate=30/1' ! \
nvv4l2decoder ! \
nvvidconv ! \
nvegltransform ! \
nveglglessink sync=no
gst-launch-1.0 \
v4l2src device="/dev/video4" ! \
'video/x-h264, width=1920, height=960, framerate=30/1' ! \
nvv4l2decoder ! \
nv3dsink sync=no
Docoder
| ライブラリ | 処理 |
|---|---|
| omxh265dec | OpenMAX IL H.265 Video decoder |
| omxh264dec | OpenMAX IL H.264 Video decoder |
| omxmpeg4videodec | OpenMAX IL MPEG4 Video decoder |
| omxmpeg2videodec | OpenMAX IL MPEG2 Video decoder |
| omxvp8dec | OpenMAX IL VP8 Video decoder |
| omxvp9dec | OpenMAX IL VP9 video decoder |
| nvv4l2decoder | V4L2 H.265, H.264, VP8, VP9, MPEG4, MPEG2 Video decoder |
nvv4l2decoderでほぼ解決できる。
nvv4l2decoderで、MotionJpegを取り込みたい場合は、mjpeg=1オプションをつける
nvv4l2decoder mjpeg=1
Encoder
| ライブラリ | 処理 |
|---|---|
| omxh265dec | OpenMAX IL H.265 Video decoder |
| omxh264dec | OpenMAX IL H.264 Video decoder |
| omxmpeg4videodec | OpenMAX IL MPEG4 Video decoder |
| omxmpeg2videodec | OpenMAX IL MPEG2 Video decoder |
| omxvp8dec | OpenMAX IL VP8 Video decoder |
| omxvp9dec | OpenMAX IL VP9 video decoder |
| nvv4l2decoder | V4L2 H.265, H.264, VP8, VP9, MPEG4, MPEG2 Video decoder |
| nvjpegdec | JPEG decoder element |
nvcompositor
nvcompositorで複数カメラから取り込んだ画像の結合ができる
2個のUSBカメラ(JetPack4.6)
gst-launch-1.0 \
nvcompositor name=m \
sink_0::xpos=0 sink_0::ypos=0 \
sink_1::xpos=1280 sink_1::ypos=0 ! \
nvegltransform ! \
nveglglessink sync=no \
v4l2src device=/dev/video2 io-mode=2 ! \
'image/jpeg, width=1280, height=720, framerate=30/1' ! \
nvv4l2decoder mjpeg=1 ! \
queue ! \
nvvidconv ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
m.sink_0 \
v4l2src device=/dev/video3 io-mode=2 ! \
'image/jpeg, width=1280, height=720, framerate=30/1' ! \
nvv4l2decoder mjpeg=1 ! \
queue ! \
nvvidconv ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
m.sink_1
2個USBカメラ(JetPack4.5.1)
gst-launch-1.0 \
nvcompositor name=m \
sink_0::xpos=0 sink_0::ypos=0 \
sink_1::xpos=1280 sink_1::ypos=0 ! \
nvegltransform ! \
nveglglessink sync=no \
v4l2src device=/dev/video2 io-mode=2 ! \
'image/jpeg, width=1280, height=720, framerate=30/1' ! \
jpegdec ! \
nvvidconv ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
m.sink_0 \
v4l2src device=/dev/video3 io-mode=2 ! \
'image/jpeg, width=1280, height=720, framerate=30/1' ! \
jpegdec ! \
nvvidconv ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
m.sink_1
2個のCSIカメラ
gst-launch-1.0 \
nvcompositor name=m \
sink_0::xpos=0 sink_0::ypos=0 \
sink_1::xpos=1280 sink_1::ypos=0 ! \
nvegltransform ! \
nveglglessink sync=no \
nvarguscamerasrc sensor-id=0 ! \
'video/x-raw(memory:NVMM), width=1280, height=720, framerate=30/1' ! \
queue ! \
nvvidconv ! \
m.sink_0 \
nvarguscamerasrc sensor-id=1 ! \
'video/x-raw(memory:NVMM), width=1280, height=720, framerate=30/1' ! \
queue ! \
nvvidconv ! \
m.sink_1
1個CSIと1個USBカメラ
gst-launch-1.0 \
nvcompositor name=m \
sink_0::xpos=0 sink_0::ypos=0 \
sink_1::xpos=1280 sink_1::ypos=0 ! \
nvegltransform ! \
nveglglessink sync=no \
nvarguscamerasrc sensor-id=0 ! \
'video/x-raw(memory:NVMM), width=1280, height=720, framerate=30/1' ! \
queue ! \
nvvidconv ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
m.sink_0 \
nvarguscamerasrc sensor-id=1 ! \
'video/x-raw(memory:NVMM), width=1280, height=720, framerate=30/1' ! \
queue ! \
nvvidconv ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
m.sink_1
2個のCSIカメラと, 2個のUSBカメラ(JetPack4.6)
gst-launch-1.0 \
nvcompositor name=m \
sink_0::xpos=0 sink_0::ypos=0 \
sink_1::xpos=1280 sink_1::ypos=0 \
sink_2::xpos=0 sink_2::ypos=720 \
sink_3::xpos=1280 sink_3::ypos=720 ! \
nvegltransform ! \
nveglglessink sync=no \
nvarguscamerasrc sensor-id=0 ! \
'video/x-raw(memory:NVMM), width=1280, height=720, framerate=30/1' ! \
queue ! \
nvvidconv ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
m.sink_0 \
nvarguscamerasrc sensor-id=1 ! \
'video/x-raw(memory:NVMM), width=1280, height=720, framerate=30/1' ! \
queue ! \
nvvidconv ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
m.sink_1 \
v4l2src device=/dev/video2 io-mode=2 ! \
'image/jpeg, width=1280, height=720, framerate=30/1' ! \
nvv4l2decoder mjpeg=1 ! \
queue ! \
nvvidconv ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
m.sink_2 \
v4l2src device=/dev/video3 io-mode=2 ! \
'image/jpeg, width=1280, height=720, framerate=30/1' ! \
nvv4l2decoder mjpeg=1 ! \
queue ! \
nvvidconv ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
m.sink_3
2個のCSIカメラと, 2個のUSBカメラ(JetPack4.5.1)
gst-launch-1.0 \
nvcompositor name=m \
sink_0::xpos=0 sink_0::ypos=0 \
sink_1::xpos=1280 sink_1::ypos=0 \
sink_2::xpos=0 sink_2::ypos=720 \
sink_3::xpos=1280 sink_3::ypos=720 ! \
nvegltransform ! \
nveglglessink sync=no \
nvarguscamerasrc sensor-id=0 ! \
'video/x-raw(memory:NVMM), width=1280, height=720, framerate=30/1' ! \
queue ! \
nvvidconv ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
m.sink_0 \
nvarguscamerasrc sensor-id=1 ! \
'video/x-raw(memory:NVMM), width=1280, height=720, framerate=30/1' ! \
queue ! \
nvvidconv ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
m.sink_1 \
v4l2src device=/dev/video2 io-mode=2 ! \
'image/jpeg, width=1280, height=720, framerate=30/1' ! \
jpegdec ! \
nvvidconv ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
m.sink_2 \
v4l2src device=/dev/video3 io-mode=2 ! \
'image/jpeg, width=1280, height=720, framerate=30/1' ! \
jpegdec ! \
nvvidconv ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
m.sink_3
2個のCSIカメラと, 2個のUSBカメラ(MJPEG), 2個のUSBカメラ(H264) (JetPack4.6)
gst-launch-1.0 \
nvcompositor name=m \
sink_0::xpos=0 sink_0::ypos=0 \
sink_1::xpos=1280 sink_1::ypos=0 \
sink_2::xpos=0 sink_2::ypos=720 \
sink_3::xpos=1280 sink_3::ypos=720 \
sink_4::xpos=0 sink_4::ypos=1440 \
sink_5::xpos=1920 sink_5::ypos=1440 ! \
nvegltransform ! \
nveglglessink sync=no \
nvarguscamerasrc sensor-id=0 ! \
'video/x-raw(memory:NVMM), width=1280, height=720, framerate=30/1' ! \
queue ! \
nvvidconv ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
m.sink_0 \
nvarguscamerasrc sensor-id=1 ! \
'video/x-raw(memory:NVMM), width=1280, height=720, framerate=30/1' ! \
queue ! \
nvvidconv ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
m.sink_1 \
v4l2src device=/dev/video2 io-mode=2 ! \
'image/jpeg, width=1280, height=720, framerate=30/1' ! \
nvv4l2decoder mjpeg=1 ! \
queue ! \
nvvidconv ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
m.sink_2 \
v4l2src device=/dev/video3 io-mode=2 ! \
'image/jpeg, width=1280, height=720, framerate=30/1' ! \
nvv4l2decoder mjpeg=1 ! \
queue ! \
nvvidconv ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
m.sink_3 \
v4l2src device=/dev/video4 io-mode=2 ! \
'video/x-h264, width=1920, height=960, framerate=30/1' ! \
nvv4l2decoder ! \
queue ! \
nvvidconv ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
m.sink_4 \
v4l2src device=/dev/video5 io-mode=2 ! \
'video/x-h264, width=1920, height=960, framerate=30/1' ! \
nvv4l2decoder ! \
queue ! \
nvvidconv ! \
"video/x-raw(memory:NVMM),format=RGBA" ! \
m.sink_5