Mali-G610 HW decode/encode

Edge2 Ubuntu
1

Hello,
Since Edge2 has only been released a couple of weeks ago, there isn’t much information about the hardware accelerated video decode/encode routines, so I thought it would be a good starting point to create this forum thread.

As stated in official description of the board, device boasts a Mali-G610 GPU which supports 8K@60fps decoding and 8K@30fps (H.265/H.264) encoding.
Encoding one or multiple video streams is extremely convenient if not vital for any tipe of automation and having easy support for such hardware feature would greatly increase the value of the device.

For most high-end applications GStreamer would most likely be used, but as this device is often used for prototyping and testing FFmpeg support would allow for easier setup and testing.

If anyone have any experience in HW API’s, GPU drivers or would like to see HW accelerated video decode/encode on this device, feel free to chime in!

2

Hardware decoding with GStreamer already supported.

1 Like
3

Cool! Could we get an example gst-launch line for testing?
I assume the required module comes precompiled with the official Ubuntu installation, right?

4

Hello @Frank

Please provide the gstreamer pipe.

5

Hi!
I’ve tried some more gst hardware decoders, but none of them seem to work.
Can someone point me to the correct decoder and/or encoder plugin for h264/h265?

Thanks

6

@Jezaus.Tevas

gst-launch-1.0 filesrc location=./bbb2.h264 ! h264parse ! mppvideodec ! video/x-raw,format=NV12 ! filesink location=bbb3.264
7

Thanks, @Frank!
mppvideodec decoder works great. I have also tested mpph264enc and mpph265enc, and have some issues with them.

  • mpph264enc seems to always output 30fps stream, regardless of the input framerate or videorate filter,
  • cpu usage keeps creeping up during encoding process (starts at around 2-3%, reaches 5-6% after around 5 minutes of encoding)
  • sometimes pipeline will hang before or on first keyframe (1-2 seconds after start of transcoding)
  • it also has a weird bug where sometimes it will encode some random garbage blocks that seem to contain data of old frames onto new ones

Have you ever encountered similar issues and how can we combat this?

Thank You

8

Regarding cpu usage, mppvideodec arm-afbc=true seems to fix the issue

9

While helping with cpu usage, this parameter causes some of the input sources to output only green frames.

10

Need to check, how about the h265 encoding?

Both for h264 and h265?

11

After some testing today I can confirm h265 encoder does also only output 30fps stream, also has the same issue where some sources will not roll over 2 seconds mark, but it does not seem to have the blocky glitch behaviour where old frame blocks are being shown on random parts of the video.

12

I also experimented a bit with decoder, as I thought that the stream may be stopping there, but after swaping mppvideodec to decodebin to avdec_h264 the issue persists, pipeline will preroll normally, but will stop before reaching 2 seconds of runtime and will just hang there indefinitely

13

Hello, any news about the encoder?

14

About 30fps issue you need to use qtmux or other plugin to mux, the raw h264 and h265 content lack pts so can only be 30fps.

e.g.

gst-launch-1.0 filesrc location=./bbb2.h264 ! h264parse ! mppvideodec ! video/x-raw,format=NV12 ! qtmux ! filesink location=bbb3.mp4
15

Yes, I am using the mpegtsmux muxer. I have also tested with software encoders in the same pipe, producing wide variety of different framerates, yet mpph264enc/mpph265enc only produce 30fps content.

16

Hello @Jezaus.Tevas

Please double check on your side or provide your test method to reproduce the fps issue.

Just checked on my side with a H264 60fps video decoding & encoding to H264 & H265 format then mux to mp4 format, then fps is correct.

H264 encoding:

gst-launch-1.0 filesrc location=./bbb_sunflower_1080p_60fps_normal.mp4  num-buffers=3000 ! qtdemux name=d d.video_0 ! h264parse ! mppvideodec ! video/x-raw,format=NV12 ! mpph264enc ! h264parse ! qtmux ! filesink location=./h264dec-h264.mp4

H265 encoding:

gst-launch-1.0 filesrc location=./bbb_sunflower_1080p_60fps_normal.mp4  num-buffers=3000 ! qtdemux name=d d.video_0 ! h264parse ! mppvideodec ! video/x-raw,format=NV12 ! mpph265enc ! h265parse ! qtmux ! filesink location=./h264dec-h265.mp4

Check encoding file fps:

H264:

$ ffprobe /tmp/h264dec-h264.mp4
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/tmp/h264dec-h264.mp4':
  Metadata:
    major_brand     : qt  
    minor_version   : 537199360
    compatible_brands: qt  
    creation_time   : 2022-12-12T01:48:12.000000Z
    artist          : Blender Foundation 2008, Janus Bager Kristensen 2013
    comment         : Creative Commons Attribution 3.0 - http://bbb3d.renderfarming.net
    composer        : Sacha Goedegebure
    genre           : Animation
    title           : Big Buck Bunny, Sunflower version
  Duration: 00:00:22.53, start: 0.000000, bitrate: 13608 kb/s
  Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(bt709, progressive), 1920x1080, 13605 kb/s, SAR 1:1 DAR 16:9, 59.96 fps, 60 tbr, 6k tbn, 60 tbc (default)
    Metadata:
      creation_time   : 2022-12-12T01:48:12.000000Z
      handler_name    : VideoHandler
      vendor_id       : [0][0][0][0]

H265:

$ ffprobe /tmp/h264dec-h265.mp4
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/tmp/h264dec-h265.mp4':
  Metadata:
    major_brand     : qt  
    minor_version   : 537199360
    compatible_brands: qt  
    creation_time   : 2022-12-12T01:48:38.000000Z
    artist          : Blender Foundation 2008, Janus Bager Kristensen 2013
    comment         : Creative Commons Attribution 3.0 - http://bbb3d.renderfarming.net
    composer        : Sacha Goedegebure
    genre           : Animation
    title           : Big Buck Bunny, Sunflower version
  Duration: 00:00:22.53, start: 0.000000, bitrate: 13369 kb/s
  Stream #0:0(und): Video: hevc (Main) (hvc1 / 0x31637668), yuv420p(tv, progressive), 1920x1080, 13366 kb/s, SAR 1:1 DAR 16:9, 59.96 fps, 60 tbr, 6k tbn, 30 tbc (default)
    Metadata:
      creation_time   : 2022-12-12T01:48:38.000000Z
      handler_name    : VideoHandler
      vendor_id       : [0][0][0][0]

The fps is 60.