VIM3L Video decoding not working

VIM3
1

Hello,

I’m trying to get working video decoding on my VIM3L, with my frankenstein Armbian (I use Sid repos …)
3D acceleration seems to be ok with Mesa 22.3.5 + Panfrost + Wayland
My kernel is : 6.1.12

According to https://linux-meson.com/ and few patches on Google, the video decoders has been mainlined, and should be working at least for the H264 and the VP9.
But I can’t get it working :frowning:

I do my test with https://test-videos.co.uk/bigbuckbunny/webm-vp9

I tested with MPV 0.35.1 and VLC 3.0.18
I tries a few options:

mpv video.webm
mpv -vo=gpu --gpuu-contex=wayland video.webm

=> Slow decoding and 100% cpu usage

mpv --hwdec=v4l2m2m-copy video.webm
=> Seg fault

/dev/video0 is present

Same problem with Firefox/Chromium on Youtube

Do you have any ideas ?

Thanks

2

Hello @Ticc0

Hardware decoding not work for 6.x kernel.

3

Do you know why 6.x breaks it ?

4

The current (staging) driver was developed against older GXBB/GXL/GXM hardware and is unfinished and missing things needed with newer G12A/G12B/SM1 hardware (VIM3L is SM1). It works well-enough with H264, and 8-bit HEVC and VP9 content (HEVC isn’t perfect but usable) but MPEG1/2/4 is not working (became broken at some point) and VC1 was never started. The current driver code doesn’t really handle newer hardware differences (and requirements) and there are also some major bugs, e.g. on G12A and up, 10-bit media (most 4K HEVC and VP9 media) will wedge the board requiring reboot.

If your media needs are simple the drivers are usable; with the right ffmpeg sources. It’s also possible to simply software decode everything up to 1080p resolutions as the boards have enough CPU grunt. This LE image represents “as good as it gets” if you want to experiment and see what works:

https://chewitt.libreelec.tv/testing/LibreELEC-AMLGX.arm-10.95.2-khadas-vim3l.img.gz

If you want more than that, you will be forced to use the downstream BSP codebase.

5

I did some tests but still not working :frowning:

I recompiled ffmpeg 4.4.1 with V4L2-requests patches from LibreElec, and MPV.

But I have nothing more than :

Input #0, matroska,webm, from 'Big_Buck_Bunny_1080_10s_30MB.webm':
  Metadata:
    title           : Big Buck Bunny, Sunflower version
    GENRE           : Animation
    MAJOR_BRAND     : isom
    MINOR_VERSION   : 1
    COMPATIBLE_BRANDS: isomavc1
    COMPOSER        : Sacha Goedegebure
    ARTIST          : Blender Foundation 2008, Janus Bager Kristensen 2013
    COMMENT         : Creative Commons Attribution 3.0 - http://bbb3d.renderfarming.net
    ENCODER         : Lavf58.14.100
  Duration: 00:00:10.00, start: 0.000000, bitrate: 25206 kb/s
  Stream #0:0: Video: vp9 (Profile 0), yuv420p(tv), 1920x1080, SAR 1:1 DAR 16:9, 30 fps, 30 tbr, 1k tbn, 1k tbc (default)
    Metadata:
      HANDLER_NAME    : GPAC ISO Video Handler
      ENCODER         : Lavc58.19.102 libvpx-vp9
      DURATION        : 00:00:10.000000000
[vp9_v4l2m2m @ 0xaaaacf34af50] Using device /dev/video0
[vp9_v4l2m2m @ 0xaaaacf34af50] driver 'meson-vdec' on card 'Amlogic Video Decoder' in mplane mode
[vp9_v4l2m2m @ 0xaaaacf34af50] requesting formats: output=VP90 capture=NM12
Stream mapping:
  Stream #0:0 -> #0:0 (vp9 (vp9_v4l2m2m) -> wrapped_avframe (native))
Press [q] to stop, [?] for help
[vp9_v4l2m2m @ 0xaaaacf34af50] output POLLERR
    Last message repeated 4663 times

I thought the problem was my ffmpeg versions or flags, but it comes from the kernel I guess:

v4l2-compliance 1.22.1, 64 bits, 64-bit time_t

Compliance test for meson-vdec device /dev/video0:

Driver Info:
	Driver name      : meson-vdec
	Card type        : Amlogic Video Decoder
	Bus info         : platform:meson-vdec
	Driver version   : 6.1.12
	Capabilities     : 0x84204000
		Video Memory-to-Memory Multiplanar
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x04204000
		Video Memory-to-Memory Multiplanar
		Streaming
		Extended Pix Format
	Detected Stateful Decoder

Required ioctls:
	test VIDIOC_QUERYCAP: OK
	test invalid ioctls: OK

Allow for multiple opens:
	test second /dev/video0 open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
	test VIDIOC_QUERYCTRL: OK
	test VIDIOC_G/S_CTRL: OK
	test VIDIOC_G/S/TRY_EXT_CTRLS: OK
	test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
	test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
	Standard Controls: 2 Private Controls: 0

Format ioctls:
		fail: v4l2-test-formats.cpp(263): fmtdesc.description mismatch: was 'Y/UV 4:2:0 (N-C)', expected 'Y/CbCr 4:2:0 (N-C)'
	test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: FAIL
	test VIDIOC_G/S_PARM: OK (Not Supported)
	test VIDIOC_G_FBUF: OK (Not Supported)
		fail: v4l2-test-formats.cpp(478): pixelformat 32314d4e (NM12) for buftype 9 not reported by ENUM_FMT
	test VIDIOC_G_FMT: FAIL
		fail: v4l2-test-formats.cpp(478): pixelformat 32314d4e (NM12) for buftype 9 not reported by ENUM_FMT
	test VIDIOC_TRY_FMT: FAIL
		fail: v4l2-test-formats.cpp(478): pixelformat 32314d4e (NM12) for buftype 9 not reported by ENUM_FMT
	test VIDIOC_S_FMT: FAIL
	test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
	test Cropping: OK (Not Supported)
	test Composing: OK (Not Supported)
	test Scaling: OK (Not Supported)

Codec ioctls:
	test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
	test VIDIOC_G_ENC_INDEX: OK (Not Supported)
		fail: v4l2-test-codecs.cpp(104): node->function != MEDIA_ENT_F_PROC_VIDEO_DECODER
	test VIDIOC_(TRY_)DECODER_CMD: FAIL

Buffer ioctls:
	test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
	test VIDIOC_EXPBUF: OK
	test Requests: OK (Not Supported)

Total for meson-vdec device /dev/video0: 45, Succeeded: 

40, Failed: 5, Warnings: 0

I found a another solution maybe, with libva-v4l2-request) but it needs some work to port it on newer kernel, and I don’t want to loose more times if the problem comes from the kernel.

It’s sad to see that 3-4 years ago the development was active, and now, everything is unusable.

I will try with the 6.2 kernel, but I have low expectations :roll_eyes:

6

Well, I found the solution :sweat_smile:
I checked dmesg and vdec firmware was missing in /lib/firmware/meson/vdec. I download them from meson/vdec - pub/scm/linux/kernel/git/firmware/linux-firmware - Git at Google

And now it works, but not very great.

  • VP9 still have dropped frames, 2-3/sec
  • Some h265 video (1080p, no HDR …) makes the board freeze/crash
  • Any videos > 1080p are not working at all

The v4l2-compliance test still have some errors

7

https://github.com/LibreELEC/meson-firmware has newer firmware than upstream linux-firmware but that probably doesn’t influence the outcome in a meaningful way. The root problems remain: the current decoders were largely authored against GXL hardware (giving a level of compat to GXBB and GXM) but G12A and up (and SM1 didn’t exist at the time) have differences, and the drivers are unfinished (hence v4l2-complianace shows issues). In the last year I was able to fixup the HEVC codec (which is not merged upstream) and combined with some ffmpeg changes from RPi developers it works quite well on the older boards (but not newer ones). In January I’ve submitted that code to staging in the hope putting it somewhere more public might inspire someone to take an interest in it … but I’m not optimistic as it’s already 3-years since Maxime stepped back and nobody showed interest so far.

btw, Amlogic uses v4l2_m2m (stateful) decoding not v4l2_requests (stateless); there is a combination of both in the branch since RPi4 uses both methods (H264 is stateful, HEVC is stateless).

1 Like