Hardware H.264 decode and encode

VIM4
1

Which system do you use? Android, Ubuntu, OOWOW or others?

Ubuntu

Which version of system do you use? Khadas official images, self built images, or others?

Khadas Ubuntu 22.04 desktop

Please describe your issue below:

What options are there to perform hardware H.264 decode and encode?
Are there libraries available?
This would be integrated into an application so command line utilities are not an option.

2

Decode: VIM4 HDMI In to ffmpeg/gstreamer h.265 - #18 by numbqq

Encode: https://docs.khadas.com/linux/vim4/hardware_encoding.html

3

I tried using gstreamer for decoding and found it uses avdec_h264 (software) rather than v4l2h264dec (hardware) when using playbin. I switched to clutterautovideosink rather than waylandsink for the output stage

Trying to explicitly specify the elements fails with the following:

Setting pipeline to PAUSED …
Pipeline is PREROLLING …
/GstPipeline:pipeline0/GstH264Parse:h264parse0.GstPad:sink: caps = video/x-h264, stream-format=(string)avc, alignment=(string)au, level=(string)3.1, profile=(string)constrained-baseline, codec_data=(buffer)0142c01fffe1001a6742c01fda03c045fb9a810080a0000003002000000641e30
65401000468ce04f2, width=(int)960, height=(int)540, framerate=(fraction)25/1, pixel-aspect-ratio=(fraction)1/1
/GstPipeline:pipeline0/GstH264Parse:h264parse0.GstPad:src: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, level=(string)3.1, profile=(string)baseline, width=(int)960, height=(int)540, framerate=(fraction)25/1, pixel-aspect-ratio=(fraction)1
/1, chroma-format=(string)4:2:0, bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, colorimetry=(string)2:3:5:0, parsed=(boolean)true
ERROR: from element /GstPipeline:pipeline0/GstQTDemux:qtdemux0: Internal data stream error.
Additional debug info:
…/gst/isomp4/qtdemux.c(6747): gst_qtdemux_loop (): /GstPipeline:pipeline0/GstQTDemux:qtdemux0:
streaming stopped, reason not-negotiated (-4)
ERROR: pipeline doesn’t want to preroll.
Setting pipeline to NULL …
Freeing pipeline …

4

Encoding
aml_enc_test is not a solution for integration into an application; need a library/API solution.

Decoding
Must the settings in /sys/module/amvdec_ports/parameters always be set? Is there documentation available?

GStreamer decoding fails when using
gst-launch-1.0 filesrc location=.mp4 ! qtdemux name=d d.video_0 ! h264parse ! v4l2h264dec capture-io-mode=4 ! video/x-raw,format=NV12 ! clutterautovideosink

Using GStreamer playbin command line selects software decoder from libav (avdec_h264)

I use clutterautovideosink to save having to switch to wayland session for testing

5

Yes, we will release the encoder libraries source code in Q3, and we will also add gstreamer encoder plugin in the future.

Yes, you need to set such parameters.

Gstreamer video display have many limitations, at this moment suggest you to use waylandsink on weston.

6

I get the same failure in the weston session using waylandsink.

There seems to be a caps negotiation failure:

ERROR: from element /GstPipeline:pipeline0/GstQTDemux:qtdemux0: Internal data stream error.
Additional debug info:
…/gst/isomp4/qtdemux.c(6747): gst_qtdemux_loop (): /GstPipeline:pipeline0/GstQTDemux:qtdemux0:
streaming stopped, reason not-negotiated (-4)

7

How about this one ?

$ gst-launch-1.0 filesrc location=/home/khadas/bbb_sunflower_1080p_60fps_normal.mp4 ! qtdemux name=d d.video_0 ! h264parse ! v4l2h264dec capture-io-mode=4 ! video/x-raw,format=NV12 ! waylandsink
8

That stream is playing fine using clutterautovideosink.
Of course I have to find and download the file first since I didn’t realise it was Big Buck Bunny.

Not sure why the file I was using failed. It plays fine using various other hardware decoders.

10

What’s the status on the mentioned encoder libraries source code?