Is it possible to stream video using gstreamer with hardware encoding in vim4?

Which system do you use? Android, Ubuntu, OOWOW or others?

Ubuntu

Which version of system do you use? Please provide the version of the system here:

Ubuntu 22.04

Please describe your issue below:

With the help of the khadas team, I’ve been able to get some great deep learning models up and running.

I want to stream the visualization of the model’s results as a video.

gst-launch-1.0 filesrc location=/home/khadas/bbb_sunflower_1080p_60fps_normal.mp4 ! qtdemux name=d d.video_0 ! h264parse ! v4l2h264dec ! video/x-raw,format=NV12 ! amlvenc ! h264parse ! filesink location=/home/khadas/h264dec.h264

I was sure that encoding worked fine with that command.

But I don’t know how to do streaming with this.
I’m curious about how to continue streaming images visualized with opencv.

I want to try some light streaming with hardware acceleration from “amlvenc”.

How to get started?


    std::string pipeline =
        "appsrc ! videoconvert ! "
        "x264enc tune=zerolatency speed-preset=ultrafast bitrate=2500 key-int-max=30 ! "
        "rtspclientsink location=rtsp://localhost:8554/mystream protocols=tcp";

I was able to successfully stream using this pipeline and mediamtx, but the CPU load was around 250-300%.

But when I use amlvenc, I get the following error.

Set log level to 1
[selectHW:105] selectEncoder
[selectHW:109] amvenc_multi 264 present
[amvenc_multi_load_Module:40] amvenc_ports multi loadModule!, load_count:0

Set log level to 4
[initEncParams:266] enc_feature_opts is 0x1 , GopPresetis 0x0 

[vp5xx_vcpu_status:789] PC=0x112c

[vp5xx_vcpu_status:789] PC=0x112c

[vp5xx_vcpu_status:789] PC=0x112c

[vp5xx_vcpu_status:789] PC=0x112c

[vp5xx_vcpu_status:789] PC=0x112c

[vp5xx_vcpu_status:789] PC=0x112c

[vp5xx_vcpu_status:789] PC=0x112c

[vp5xx_vcpu_status:789] PC=0x112c

[vp5xx_vcpu_status:789] PC=0x112c

[vp5xx_vcpu_status:789] PC=0x112c

[vp5xx_vcpu_status:789] PC=0x112c

[vp5xx_vcpu_status:789] PC=0x112c

[vp5xx_vcpu_status:789] PC=0x112c

[vp5xx_vcpu_status:789] PC=0x112c

[vp5xx_vcpu_status:789] PC=0x112c

[vp5xx_vcpu_status:789] PC=0x112c

[vp5xx_vcpu_status:789] PC=0x112c

[vp5xx_vcpu_status:789] PC=0x112c

[vp5xx_vcpu_status:789] PC=0x112c

[vp5xx_vcpu_status:789] PC=0x112c

[AML_MultiEncInitialize:1382] VPU_EncOpen failed Error code is 0x13 

Set log level to 1
[selectHW:105] selectEncoder
[selectHW:109] amvenc_multi 264 present
[amvenc_multi_load_Module:40] amvenc_ports multi loadModule!, load_count:1

Set log level to 4
[initEncParams:266] enc_feature_opts is 0x1 , GopPresetis 0x0 

[vdi_init:193] [VDI] opend already.

[vp5xx_vcpu_status:789] PC=0x112c

[vp5xx_vcpu_status:789] PC=0x112c

[vp5xx_vcpu_status:789] PC=0x112c

[vp5xx_vcpu_status:789] PC=0x112c

[vp5xx_vcpu_status:789] PC=0x112c

[vp5xx_vcpu_status:789] PC=0x112c

[vp5xx_vcpu_status:789] PC=0x112c

[vp5xx_vcpu_status:789] PC=0x112c

[vp5xx_vcpu_status:789] PC=0x112c

[vp5xx_vcpu_status:789] PC=0x112c

[vp5xx_vcpu_status:789] PC=0x112c

[vp5xx_vcpu_status:789] PC=0x112c

[vp5xx_vcpu_status:789] PC=0x112c

[vp5xx_vcpu_status:789] PC=0x112c

[vp5xx_vcpu_status:789] PC=0x112c

[vp5xx_vcpu_status:789] PC=0x112c

[vp5xx_vcpu_status:789] PC=0x112c

[vp5xx_vcpu_status:789] PC=0x112c

[vp5xx_vcpu_status:789] PC=0x112c

[vp5xx_vcpu_status:789] PC=0x112c

[AML_MultiEncInitialize:1382] VPU_EncOpen failed Error code is 0x13 

Set log level to 1
[selectHW:105] selectEncoder
[selectHW:109] amvenc_multi 264 present
[amvenc_multi_load_Module:40] amvenc_ports multi loadModule!, load_count:2

Set log level to 4
[initEncParams:266] enc_feature_opts is 0x1 , GopPresetis 0x0 

[vdi_init:193] [VDI] opend already.

[ WARN:0] global ./modules/videoio/src/cap_gstreamer.cpp (2075) handleMessage OpenCV | GStreamer warning: Embedded video playback halted; module amlvenc0 reported: Can not initialize v encoder.

How can I fix it?

Hello @GHdevlog

You can try mediamtx + gstreamer:

gst-launch-1.0 -v v4l2src device=/dev/media0 ! queue ! amlvenc ! rtspclientsink location=rtsp://localhost:8554/test

I added a queue ! in between and was able to use amlvenc smoothly.

Conversely, can I also see a pipeline that fetches video from an RTSP stream into opencv?