Faulty h264 encoder on one (nonNPU) VIM4

VIM4
1

Which system do you use? Android, Ubuntu, OOWOW or others?

Ubuntu from OOWOW

Which version of system do you use? Please provide the version of the system here:

24 and 22 tested

Please describe your issue below:

Gstreamer can’t allocate an amlvenc instance for h264 encoding.

gst-launch-1.0 \
        v4l2src device=/dev/video50 io-mode=mmap \
        ! video/x-raw,width=1920,height=1080,framerate=30/1,format=NV12 \
        ! amlvenc gop=0 bitrate=5000 ! video/x-h264,profile=high \
        ! h264parse ! rtph264pay config-interval=1 mtu=1208 \
        ! "application/x-rtp, payload=(int)96, ssrc=(uint)555555" \
        !  udpsink host=127.0.0.1 port=${1}

fails.

This works on _every other VIM4 (non-npu) I have tried, but on this single VIM4 it fails.
The camera works if I use a software encoder. (openh264)

Is there an encoder license/fuse missing, or a firmware blob I need install?
Or is the board just faulty?

Post a console log of your issue below:


No log available yet.
2

@steely-glint does it work without this particular segment of the pipeline ?

If possible please share the outputs from gstreamer in the terminal when running, and dmesg logs after executing the command.

3

@Electr1

Happy New (western) Year!

I updated the test so I do not need a camera and removed the profile setting…

   exec gst-launch-1.0 \
   souphttpsrc location=https://rdmedia.bbc.co.uk/testcard/lowlatency/manifests/ll-avc-single-128kbps-1920x1080p50-en.mpd ! \
   dashdemux presentation-delay=5s !\
   qtdemux name=d d.video_0 !\
   h264parse ! openh264dec ! \
   queue leaky=downstream max-size-buffers=3 !\
   videoconvert !\
   amlvenc gop=0 bitrate=2000 !\
   h264parse !\
   rtph264pay config-interval=15 mtu=1208 !\
"application/x-rtp, payload=(int)96, ssrc=(uint)555555" !\
udpsink host=127.0.0.1 port=${1}

Here is the output.

khadas@LuigiFagioli:~/pipe-kvim$ ./videorelay.hwenctest 11223
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
Got context from element 'souphttpsrc0': gst.soup.session=context, session=(GstSoupSession)NULL;
Got context from element 'souphttpsrc2': gst.soup.session=context, session=(GstSoupSession)NULL;
Got context from element 'souphttpsrc3': gst.soup.session=context, session=(GstSoupSession)NULL;
Got context from element 'souphttpsrc2': gst.soup.session=context, session=(GstSoupSession)NULL;
Set log level to 1
[selectHW:105] selectEncoder
[selectHW:123] can not find available h.264 driver!!!,please check!
ERROR: from element /GstPipeline:pipeline0/GstAmlVEnc:amlvenc0: Can not initialize v encoder.
Additional debug info:
gstamlvenc.c(562): gst_amlvenc_init_encoder (): /GstPipeline:pipeline0/GstAmlVEnc:amlvenc0
ERROR: pipeline doesn't want to preroll.
Setting pipeline to NULL ...
Set log level to 1
[selectHW:105] selectEncoder
[selectHW:123] can not find available h.264 driver!!!,please check!
ERROR: from element /GstPipeline:pipeline0/GstAmlVEnc:amlvenc0: Can not initialize v encoder.
Additional debug info:
gstamlvenc.c(562): gst_amlvenc_init_encoder (): /GstPipeline:pipeline0/GstAmlVEnc:amlvenc0
ERROR: pipeline doesn't want to preroll.
Freeing pipeline ...
khadas@LuigiFagioli:~/pipe-kvim$

dmesg has nothing new after 13 seconds - nothing added after the gstreamer command is run.

Here’s the cpu id

khadas@LuigiFagioli:~/pipe-kvim$ cat /proc/cpu_chipid 
Serial:		 360b0103000000000f0f610911605690
Hardware:	 Khadas VIM4

if that helps.

(We now have 2 of these boards - and 5 that work !)

Please let me know how to enable the hardware encoder…

Thanks

Tim.

4

Hello @steely-glint

Could you check with this pipeline ?

https://docs.khadas.com/products/sbc/vim4/applications/hardware-encoding

5 
khadas@LuigiFagioli:~/pipe-kvim$ aml_enc_test 1080p.nv12 dump.h264 1920 1080 30 30 6000000 100 1 0 2 4
src_url is	: 1080p.nv12 ;
out_url is	: dump.h264 ;
width   is	: 1920 ;
height  is	: 1080 ;
gop     is	: 30 ;
frmrate is	: 30 ;
bitrate is	: 6000000 ;
frm_num is	: 100 ;
fmt     is	: 1 ;
buf_type is	: 0 ;
num_planes is	: 2 ;
codec is	: 4 ;
codec is H264
Set log level to 4
[initEncParams:266] enc_feature_opts is 0x0 , GopPresetis 0x0 

[vdi_init:209] [VDI] Init open vpu driver fail retrying 

[vdi_init:209] [VDI] Init open vpu driver fail retrying 

[vdi_init:209] [VDI] Init open vpu driver fail retrying 

[vdi_init:209] [VDI] Init open vpu driver fail retrying 

[vdi_init:209] [VDI] Init open vpu driver fail retrying 

[vdi_init:209] [VDI] Init open vpu driver fail retrying 

[vdi_init:209] [VDI] Init open vpu driver fail retrying 

[vdi_init:209] [VDI] Init open vpu driver fail retrying 

[vdi_init:209] [VDI] Init open vpu driver fail retrying 

[vdi_init:209] [VDI] Init open vpu driver fail retrying

and

Setting pipeline to PAUSED ...
devname : /dev/video26
driver : aml-vcodec-dec
device : platform:amlogic
bus_info : platform:amlogic
version : 331657
error tvin-port use -1 
Pipeline is PREROLLING ...
Set log level to 1
[selectHW:105] selectEncoder
[selectHW:123] can not find available h.264 driver!!!,please check!
ERROR: from element /GstPipeline:pipeline0/GstAmlVEnc:amlvenc0: Can not initialize v encoder.
Additional debug info:
gstamlvenc.c(562): gst_amlvenc_init_encoder (): /GstPipeline:pipeline0/GstAmlVEnc:amlvenc0
ERROR: pipeline doesn't want to preroll.
Setting pipeline to NULL ...
Set log level to 1
[selectHW:105] selectEncoder
[selectHW:123] can not find available h.264 driver!!!,please check!
ERROR: from element /GstPipeline:pipeline0/GstAmlVEnc:amlvenc0: Can not initialize v encoder.
Additional debug info:
gstamlvenc.c(562): gst_amlvenc_init_encoder (): /GstPipeline:pipeline0/GstAmlVEnc:amlvenc0
ERROR: pipeline doesn't want to preroll.
Freeing pipeline ...
khadas@LuigiFagioli:~/pipe-kvim$

I am using GitHub - pipe/encoder_libs_aml_vim3n4: A fork that adds support for bitrate and full frame requests via shared memory which is forked from your lib - it works on 6 other VIM4s.

Tim.

6

I tried building GitHub - numbqq/encoder_libs_aml at VIM4 and re-installing libvpcodec.so but I got the same result.

7

I confirm I’m haveing the same issues on a brand new VIM4 board purchased directly from Khadas Shop.

8

Could you try the orignal OS and do not change any thing to check?

  1. Install vim4-ubuntu-24.04-gnome-linux-5.15-fenix-1.7.3-241129.img.xz
  2. Check with command: aml_enc_test 1080p.nv12 dump.h264 1920 1080 30 30 6000000 100 1 0 2 4
  3. Provide all the aml_enc_test command execute log to us, if success, it should be like this:
khadas@Khadas:~$ aml_enc_test 1080p.nv12 dump.h264 1920 1080 30 30 6000000 100 1 0 2 4
src_url is      : 1080p.nv12 ;
out_url is      : dump.h264 ;
width   is      : 1920 ;
height  is      : 1080 ;
gop     is      : 30 ;
frmrate is      : 30 ;
bitrate is      : 6000000 ;
frm_num is      : 100 ;
fmt     is      : 1 ;
buf_type is     : 0 ;
num_planes is   : 2 ;
codec is        : 4 ;
codec is H264
Set log level to 4
[initEncParams:266] enc_feature_opts is 0x0 , GopPresetis 0x0 

[vdi_sys_sync_inst_param:673] [VDI] fail to deliver sync instance param inst_idx=0

[AML_MultiEncInitialize:1388] VPU instance param sync with open param failed

[SetSequenceInfo:989] Required buffer fb_num=3, src_num=1, actual src=3 1920x1080

[vl_multi_encoder_adjust_h264_header:626] sps_start=0

[vl_multi_encoder_adjust_h264_sps:558] crop top:0,left:0,right:0,bottom:4,enable:1
read input file error!
Encode End!width:1920

And provide the below information to us:

$ cat /proc/cpu_chipid 
$ cat /etc/fenix-release 
$ uname -a
$ lsb_release -a
$ systemctl --failed
$ dmesg > kernel.txt

And also provide the kernel.txt file to us.

9

@numbqq We will try that test today.
Just to check, that is a desktop OS - not a server os. We normally run server os.

Thanks.

Tim.

10

It is ok, you can use latest server image to check: ubuntu-24.04/vim4-ubuntu-24.04-server-linux-5.15-fenix-1.7.3-241129.img.xz

11

Would you please prove dhe 1080p.nv12 sample file as not included in the distro ?

12 
khadas@Khadas:~$ cat /proc/cpu_chipid
Serial:          360b0103000000000f0f610911605690
Hardware:        Khadas VIM4
khadas@Khadas:~$  cat /etc/fenix-release
# PLEASE DO NOT EDIT THIS FILE
BOARD=VIM4
VENDOR=Amlogic
VERSION=1.7.3
ARCH=arm64
INITRD_ARCH=arm64
IMAGE_VERSION=1.7.3-241129
################ GIT VERSION ################
UBOOT_GIT_VERSION=khadas-vims-u-boot-2019.01-v1.7.3-release
LINUX_GIT_VERSION=khadas-vims-linux-5.15-v1.7.3-release
FENIX_GIT_VERSION=v1.7.3
#############################################
khadas@Khadas:~$ uname -a
Linux Khadas 5.15.137 #1.7.3 SMP PREEMPT Fri Nov 29 10:53:24 UTC 2024 aarch64 aarch64 aarch64 GNU/Linux

khadas@Khadas:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 24.04.1 LTS
Release:        24.04
Codename:       noble
khadas@Khadas:~$ systemctl --failed
  UNIT LOAD ACTIVE SUB DESCRIPTION

0 loaded units listed.

dmseg dump link

13

meanwhile we get the right test file i made a conversion with ffmpeg of an mp4 i had handy

ffmpeg -i SampleVideo_1280x720_20mb.mp4 out.yuv
ffmpeg version 6.1.1-3ubuntu5 Copyright (c) 2000-2023 the FFmpeg developers
  built with gcc 13 (Ubuntu 13.2.0-23ubuntu3)
  configuration: --prefix=/usr --extra-version=3ubuntu5 --toolchain=hardened --libdir=/usr/lib/aarch64-linux-gnu --incdir=/usr/include/aarch64-linux-gnu --arch=arm64 --enable-gpl --disable-stripping --disable-omx --enable-gnutls --enable-libaom --enable-libass --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libglslang --enable-libgme --enable-libgsm --enable-libharfbuzz --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-openal --enable-opencl --enable-opengl --disable-sndio --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-ladspa --enable-libbluray --enable-libjack --enable-libpulse --enable-librabbitmq --enable-librist --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libx264 --enable-libzmq --enable-libzvbi --enable-lv2 --enable-sdl2 --enable-libplacebo --enable-librav1e --enable-pocketsphinx --enable-librsvg --enable-libjxl --enable-shared
  libavutil      58. 29.100 / 58. 29.100
  libavcodec     60. 31.102 / 60. 31.102
  libavformat    60. 16.100 / 60. 16.100
  libavdevice    60.  3.100 / 60.  3.100
  libavfilter     9. 12.100 /  9. 12.100
  libswscale      7.  5.100 /  7.  5.100
  libswresample   4. 12.100 /  4. 12.100
  libpostproc    57.  3.100 / 57.  3.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'SampleVideo_1280x720_20mb.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    creation_time   : 1970-01-01T00:00:00.000000Z
    encoder         : Lavf53.24.2
  Duration: 00:01:57.31, start: 0.000000, bitrate: 1436 kb/s
  Stream #0:0[0x1](und): Video: h264 (Main) (avc1 / 0x31637661), yuv420p(progressive), 1280x720 [SAR 1:1 DAR 16:9], 1048 kb/s, 25 fps, 25 tbr, 12800 tbn (default)
    Metadata:
      creation_time   : 1970-01-01T00:00:00.000000Z
      handler_name    : VideoHandler
      vendor_id       : [0][0][0][0]
  Stream #0:1[0x2](und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, 5.1, fltp, 383 kb/s (default)
    Metadata:
      creation_time   : 1970-01-01T00:00:00.000000Z
      handler_name    : SoundHandler
      vendor_id       : [0][0][0][0]
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> rawvideo (native))
Press [q] to stop, [?] for help
Output #0, rawvideo, to 'out.yuv':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf60.16.100
  Stream #0:0(und): Video: rawvideo (I420 / 0x30323449), yuv420p(progressive), 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 276480 kb/s, 25 fps, 25 tbn (default)
    Metadata:
      creation_time   : 1970-01-01T00:00:00.000000Z
      handler_name    : VideoHandler
      vendor_id       : [0][0][0][0]
      encoder         : Lavc60.31.102 rawvideo
[out#0/rawvideo @ 0x558845bae0] video:3958200kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.000000%
frame= 2932 fps=113 q=-0.0 Lsize= 3958200kB time=00:01:57.24 bitrate=276574.3kbits/s speed=4.52x

And used that dump to feed aml_enc_test

khadas@Khadas:~$ aml_enc_test out.yuv dump.h264 1280 720 30 30 6000000 100 1 0 2 4
src_url is      : out.yuv ;
out_url is      : dump.h264 ;
width   is      : 1280 ;
height  is      : 720 ;
gop     is      : 30 ;
frmrate is      : 30 ;
bitrate is      : 6000000 ;
frm_num is      : 100 ;
fmt     is      : 1 ;
buf_type is     : 0 ;
num_planes is   : 2 ;
codec is        : 4 ;
codec is H264
Set log level to 4
[initEncParams:266] enc_feature_opts is 0x0 , GopPresetis 0x0

[vdi_sys_sync_inst_param:673] [VDI] fail to deliver sync instance param inst_idx=0

[AML_MultiEncInitialize:1388] VPU instance param sync with open param failed

[SetSequenceInfo:989] Required buffer fb_num=3, src_num=1, actual src=3 1280x720

[vl_multi_encoder_adjust_h264_header:626] sps_start=0

[Total time: 536515 us], [IO time: 207499 us]
[Encode Total time: 329016 us], [avgTime: 3290 us], [avgfps: 303]
Encode End!width:1280

dump and source files here

14

Hello @Dario_Armellin

I checked your encoded file dump.h264, it can play well, but the color is not correct, because the yuv file you used is yuv420, you need to use nv12 if you use this command: aml_enc_test out.yuv dump.h264 1280 720 30 30 6000000 100 1 0 2 4

But anyway, the result indicate that the H264 encoding works.

You can follow the steps below to double check again:

Get nv12 test file:

ffmpeg -i SampleVideo_1280x720_20mb.mp4  -vcodec rawvideo -s 1280x720 -pix_fmt nv12 out_nv12.yuv

Ecnoding:

aml_enc_test out_nv12.yuv dump.h264 1280 720 30 30 6000000 100 1 0 2 4

Then you can use VLC to play the encoded file dump.h264.

15

Thank, then we have to dig in the issue with our sw.
many thanks