VIM4 Encoding HDMI Input

datkins · April 16, 2024, 6:39am

re-installed from owowo the 5.15 image with desktop to test HDMI input encoding using gstreamer.

khadas@Khadas:~$ uname -a
Linux Khadas 5.15.119 #1.6.4 SMP PREEMPT Sat Feb 3 10:31:33 CST 2024 aarch64 aarch64 aarch64 GNU/Linux

followed updates to ensure up to date

khadas@Khadas:~$ sudo apt install gstreamer-aml
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
gstreamer-aml is already the newest version (0.3-202311).
0 upgraded, 0 newly installed, 0 to remove and 11 not upgraded.

checking the modules are present

khadas@Khadas:~$ gst-inspect-1.0 | grep aml
amljpegenc:  amljpegenc: Amlogic JPEG image encoder
amlv4l2:  amlv4l2av1dec: AML V4L2 AV1 Decoder
amlv4l2:  amlv4l2avs2dec: AML V4L2 AVS2 Decoder
amlv4l2:  amlv4l2avs3dec: AML V4L2 AVS3 Decoder
amlv4l2:  amlv4l2avsdec: AML V4L2 AVS Decoder
amlv4l2:  amlv4l2h264dec: AML V4L2 H264 Decoder
amlv4l2:  amlv4l2h265dec: AML V4L2 H265 Decoder
amlv4l2:  amlv4l2jpegdec: AML V4L2 JPEG Decoder
amlv4l2:  amlv4l2mpeg4dec: AML V4L2 MPEG4 Decoder
amlv4l2:  amlv4l2vc1dec: AML V4L2 VC1 Decoder
amlv4l2:  amlv4l2vp9dec: AML V4L2 VP9 Decoder
amlvenc:  amlvenc: Amlogic h264/h265 Multi-Encoder

run the command given here

https://docs.khadas.com/products/sbc/vim4/applications/hardware-encoding

khadas@Khadas:~$ gst-launch-1.0 v4l2src device=/dev/video71 io-mode=dmabuf num-buffers=300 ! video/x-raw,format=NV12,width=1920,height=1080,framerate=30/1,interlace-mode=interleaved ! amlvenc ! h264parse ! qtmux ! filesink location=/tmp/test71.mp4
Setting pipeline to PAUSED ...
[API:aml_v4l2src_connect:271]Enter, devname : /dev/video71
driver : vdin_video
device : mesont7
bus_info : vdin1 v4l2
version : 539099424
strcmp cap.driver: vdin_video
func_name: aml_src_get_hdmi_method
initialize func addr: 0x7f88920cbc
finalize func addr: 0x7f88920ff0
start func addr: 0x7f88921198
stop func addr: 0x7f889212d8
change_divide_framerate, L69, dev /dev/video71, change_frame 1
change_divide_framerate L91 Set framerate div 2 success
[2024-04-16 07:30:11]  DEBUG  [amlv4l2src hdmisrc.c:102:hdmi_src_initialize]enter
khadas     16880  0.0  0.0   2304   832 pts/3    S+   07:30   0:00 sh -c ps -aux | grep hdmictrl
khadas     16882  0.0  0.0   6020  1988 pts/3    S+   07:30   0:00 grep hdmictrl
[2024-04-16 07:30:11]  DEBUG  [amlv4l2src hdmisrc.c:68:hdmi_src_obtain_devname]enter
[2024-04-16 07:30:11]  DEBUG  [amlv4l2src hdmisrc.c:79:hdmi_src_obtain_devname]hdmi_src_obtain_devname, pid=16883
[2024-04-16 07:30:11]  DEBUG  [amlv4l2src hdmisrc.c:87:hdmi_src_obtain_devname]hdmi_src_obtain_devname end, pid=16883
[2024-04-16 07:30:11]  DEBUG  [amlv4l2src hdmisrc.c:79:hdmi_src_obtain_devname]hdmi_src_obtain_devname, pid=0
[2024-04-16 07:30:11]  DEBUG  [amlv4l2src hdmisrc.c:83:hdmi_src_obtain_devname]ready to execl /usr/bin/hdmictrl, pid=0
[2024-04-16 07:30:11]  DEBUG  [amlv4l2src hdmictrl.c:304:main]enter, prepare enter hdmi_rx_svctx_init
[2024-04-16 07:30:11]  DEBUG  [amlv4l2src hdmictrl.c:256:hdmi_rx_svctx_init]enter
[2024-04-16 07:30:11]  DEBUG  [amlv4l2src hdmictrl.c:264:hdmi_rx_svctx_init]prepare get the GetInstance
1970-01-04 01:27:25:923 16883 16883 D TV [TvClientWrapper]: GetInstance: start GetInstance.
1970-01-04 01:27:25:924 16883 16883 D TV [TvClient]: TvClient.
1970-01-04 01:27:25:925 16883 16883 D TV [TvClient]: Connected to tvservice.
1970-01-04 01:27:25:925 16883 16883 D TV [TvClient]: setTvClientObserver
1970-01-04 01:27:25:925 16883 16883 D TV [TvClient]: setTvClientObserver: observer is 0x55b9263ff0.
[2024-04-16 07:30:11]  DEBUG  [amlv4l2src hdmictrl.c:271:hdmi_rx_svctx_init]go hdmi_rx_svctx_perform_connect
[2024-04-16 07:30:11]  DEBUG  [amlv4l2src hdmictrl.c:202:hdmi_rx_svctx_perform_connect]enter
[2024-04-16 07:30:11]  DEBUG  [amlv4l2src hdmictrl.c:229:hdmi_rx_svctx_perform_connect][81241076]UNIX domain socket (/tmp/hdmi-rx0) bound
[2024-04-16 07:30:11]  DEBUG  [amlv4l2src common/common.c:70:udp_sock_create][81277742][/tmp/hdmi-rx0] start connect
[2024-04-16 07:30:11]  DEBUG  [amlv4l2src hdmictrl.c:238:hdmi_rx_svctx_perform_connect][81321407]Accepting connections,get the client_unix_len: 110
[2024-04-16 07:30:11]  DEBUG  [amlv4l2src hdmisrc.c:97:hdmi_src_obtain_devname]exit
[2024-04-16 07:30:11]  DEBUG  [amlv4l2src hdmisrc.c:109:hdmi_src_initialize]send_buffer:fd:9 buf:connect
[2024-04-16 07:30:11]  DEBUG  [amlv4l2src hdmisrc.c:112:hdmi_src_initialize]exit
[API:aml_v4l2src_connect:324]Exit, HDMI RX
devname : /dev/video71
driver : vdin_video
device : mesont7
bus_info : vdin1 v4l2
version : 539099424
hdmirx tvin-port use 0
[API:aml_v4l2src_streamon:373]Enter
[2024-04-16 07:30:11]  DEBUG  [amlv4l2src hdmisrc.c:138:hdmi_src_start]enter
unix  2      [ ACC ]     STREAM     LISTENING     124392   /tmp/hdmi-rx0
unix  3      [ ]         STREAM     CONNECTED     124393   /tmp/hdmi-rx0
[2024-04-16 07:30:11]  DEBUG  [amlv4l2src hdmictrl.c:248:hdmi_rx_svctx_perform_connect]g_t_svctx prepare get the connect_socker_fd 10
[2024-04-16 07:30:11]  DEBUG  [amlv4l2src hdmictrl.c:251:hdmi_rx_svctx_perform_connect]exit
[2024-04-16 07:30:11]  DEBUG  [amlv4l2src hdmictrl.c:278:hdmi_rx_svctx_init]exit
[2024-04-16 07:30:11]  DEBUG  [amlv4l2src hdmictrl.c:311:main]prepare enter setTvEventCallback
[2024-04-16 07:30:11]  DEBUG  [amlv4l2src hdmictrl.c:314:main]prepare enter process_socket_thread
[2024-04-16 07:30:11]  DEBUG  [amlv4l2src hdmictrl.c:139:process_socket_thread]enter
[2024-04-16 07:30:11]  DEBUG  [amlv4l2src hdmictrl.c:144:process_socket_thread]hdmirxsrc is hdmi2
[2024-04-16 07:30:11]  DEBUG  [amlv4l2src hdmictrl.c:156:process_socket_thread]HDMISRC is hdmi2
[2024-04-16 07:30:11]  DEBUG  [amlv4l2src hdmictrl.c:165:process_socket_thread]e_currentsource = 6
[2024-04-16 07:30:11]  DEBUG  [amlv4l2src hdmictrl.c:169:process_socket_thread]loop wait recv cmd
[2024-04-16 07:30:11]  DEBUG  [amlv4l2src hdmictrl.c:171:process_socket_thread]recv_buffer = connect
[2024-04-16 07:30:11]  DEBUG  [amlv4l2src hdmictrl.c:172:process_socket_thread]r = 32 errno = 0
[2024-04-16 07:30:11]  DEBUG  [amlv4l2src hdmictrl.c:183:process_socket_thread]Enter connect_hdmi
1970-01-04 01:27:25:941 16883 16890 D TV [TvClient]: StartTv
1970-01-04 01:27:25:941 16883 16890 D TV [TvClient]: SendMethodCall.
[2024-04-16 07:30:11]  DEBUG  [amlv4l2src hdmictrl.c:186:process_socket_thread]Exit connect_hdmi
[2024-04-16 07:30:11]  DEBUG  [amlv4l2src hdmictrl.c:189:process_socket_thread]startv is ok
[2024-04-16 07:30:11]  DEBUG  [amlv4l2src hdmictrl.c:169:process_socket_thread]loop wait recv cmd
1970-01-04 01:27:34:112 16883 16884 D TV [TvClient]: TvClient get tanscode: 5
1970-01-04 01:27:34:112 16883 16884 D TV [TvClient]: SendTvClientEvent
1970-01-04 01:27:34:112 16883 16884 D TV [TvClient]: SendTvClientEvent: now has 1 tvclient.
1970-01-04 01:27:34:112 16883 16884 D TV [TvClientWrapper]: onTvClientEvent: eventType: 10.
1970-01-04 01:27:34:112 16883 16884 D TV [TvClientWrapper]: onTvClientEvent: connect event.
1970-01-04 01:27:34:113 16883 16884 D TV [TvClientWrapper]: 

no apparent activity, unplug the HDMI cable from camera gives this

HandleTvClientEvent: source: 6, connectStatus: 0
[2024-04-16 07:30:19]  DEBUG  [amlv4l2src hdmictrl.c:130:TvEventCallback]source: 6, connectStatus: 0
1970-01-04 01:27:34:113 16883 16884 D TV [TvClient]: send event for 1 count TvClientObserver!
1970-01-04 01:27:36:780 16883 16884 D TV [TvClient]: TvClient get tanscode: 5
1970-01-04 01:27:36:781 16883 16884 D TV [TvClient]: SendTvClientEvent
1970-01-04 01:27:36:781 16883 16884 D TV [TvClient]: SendTvClientEvent: now has 1 tvclient.
1970-01-04 01:27:36:781 16883 16884 D TV [TvClientWrapper]: onTvClientEvent: eventType: 10.
1970-01-04 01:27:36:781 16883 16884 D TV [TvClientWrapper]: onTvClientEvent: connect event.
1970-01-04 01:27:36:781 16883 16884 D TV [TvClientWrapper]: HandleTvClientEvent: source: 6, connectStatus: 1

plug HDMI back into camera

[2024-04-16 07:30:21]  DEBUG  [amlv4l2src hdmictrl.c:130:TvEventCallback]source: 6, connectStatus: 1
1970-01-04 01:27:36:781 16883 16884 D TV [TvClient]: send event for 1 count TvClientObserver!
^C[2024-04-16 07:30:28]  DEBUG  [amlv4l2src hdmictrl.c:171:process_socket_thread]recv_buffer = connect
[2024-04-16 07:30:28]  DEBUG  [amlv4l2src hdmictrl.c:172:process_socket_thread]r = 0 errno = 0
1970-01-04 01:27:43:603 16883 16890 D TV [TvClient]: StopTv
1970-01-04 01:27:43:603 16883 16890 D TV [TvClient]: SendMethodCall.

khadas@Khadas:~$ [2024-04-16 07:30:28]  DEBUG  [amlv4l2src hdmictrl.c:99:_hdmi_rc_svctx_release]stoptv is ok
1970-01-04 01:27:43:634 16883 16890 D TV [TvClientWrapper]: ~TvClientWrapper.
1970-01-04 01:27:43:634 16883 16890 D TV [TvClient]: ~TvClient.
[2024-04-16 07:30:28]  DEBUG  [amlv4l2src hdmictrl.c:101:_hdmi_rc_svctx_release]ReleaseInstance is ok
[2024-04-16 07:30:28]  DEBUG  [amlv4l2src hdmictrl.c:110:_hdmi_rc_svctx_release]exit _hdmi_rc_svctx_release
[2024-04-16 07:30:28]  DEBUG  [amlv4l2src hdmictrl.c:196:process_socket_thread]exit
[2024-04-16 07:30:28]  DEBUG  [amlv4l2src hdmictrl.c:323:main]exit process_socket_thread
[2024-04-16 07:30:28]  DEBUG  [amlv4l2src hdmictrl.c:328:main]exit

in the end CTRL + C to terminate

whats wrong here that causes no encoding to take please at all?

seems its ready to go but does not start.

david

numbqq · April 17, 2024, 12:56am

Hello @datkins

What’s the HDMI IN source? And what’s the resolution?

@ivan.li please help to check.

datkins · April 17, 2024, 4:30am

Works fine with hdmi input test app, its a panasonic hcv 160, hd 1920x1080 25 fps interlaced camera

datkins · April 17, 2024, 6:49am

just tried this command line as well

gst-launch-1.0 v4l2src device=/dev/video71 io-mode=dmabuf num-buffers=30 ! video/x-raw,format=NV12,width=1920,height=1080,framerate=25/1,interlace-mode=interleaved ! amlvenc ! h264parse ! qtmux ! filesink location=/tmp/test71.mp4

as I have a 25hz camera

is there an example of encoding from HDMI that’s not gstreamer?

numbqq · April 17, 2024, 8:00am

You can check test app source code here: GitHub - numbqq/aml_tvserver at yocto-202305

BTW, have you tried other HDMI IN source, e.g. PC HDMI output.

datkins · April 17, 2024, 12:06pm

does that do encoding? i think i already ahve that working, needs to run without GUI if that’s correct?

numbqq · April 17, 2024, 1:20pm

If you just mean encoding, please check here:

https://docs.khadas.com/products/sbc/vim4/applications/hardware-encoding#h264-hardware-encoding

datkins · April 18, 2024, 5:38am

that’s hardware encoding from file, i need hardware encode from hdmi

numbqq · April 18, 2024, 6:06am

For now only works for gstreamer.

datkins · April 18, 2024, 6:26am

so how do i make it work for gstreamer!

can we get ore useful logs, is there an update I’m missing

has it been tested with 1080i50 video?

numbqq · April 18, 2024, 8:55am

It works on my side whit a PC HDMI output, as I don’t have camera on my hand now so I can’t check.

BTW, have you attach the HDMI out cable to the screen? For some logic design, you need also attach HDMI out to screen if you want to use the HDMI IN function.

datkins · April 18, 2024, 11:01am

Yes, output connected, good to know but an issue in a headless encoder scaniro, is that a software fixable thing or hardware?

I have 4k screen connected was planning to connect a 1080 screen to see if was an issue.

I’m sure I had the hdmiin test working but its not working at the moment, it does see me plug and unplug the hdmi input though o the signal is getting through

David

datkins · April 19, 2024, 5:26am

i have installed ubuntu server 5.4 from owowo ad the HDMI input works fine in that.

I’ll switch to 5.15 server and see if that works the same for HDMI input and then retry gstreamer

datkins · April 19, 2024, 5:57am

and the same with server 5.15, will test gstreamer next

datkins · April 19, 2024, 6:07am

so the h264 encode works , i have a 25 fps camera but despite using the following

gst-launch-1.0 v4l2src device=/dev/video71 io-mode=dmabuf num-buffers=500 ! video/x-raw,format=NV12,width=1920,height=1080,framerate=25/1,interlace-mode=interleaved ! amlvenc ! h264parse ! qtmux ! filesink location=./test71.mp4

what parameters exist for the amlvenc plugin? how do we control it?

datkins · April 19, 2024, 6:08am

and to follow up on the issues I had before switching to server from GUI it looks like this all works better on server than UI mode kernels

datkins · April 20, 2024, 6:14am

trying to work out how to build the amlvenc gstreamer plugin so I can understand what parameters exist, do you have any more information?

datkins · April 20, 2024, 6:29am

is there support for hardware-assisted scaling on the video input?
can we encode the same input at two different resolutions easily?

Electr1 · April 20, 2024, 8:21am

For hardware assisted scaling, you can scale image frames using the onboard GE2D graphics engine, but it’s not part of gstreamer, you need to create your own C program that reads the v4l2 video input through gstreamer, scales the image buffer with the graphics engine and creates an appsrc to read with the pipeline.

you can refer to how to use GE2D for image resizing and documentation for appsrc

This example may point you to how a appsrc program is made, note it hasn’t been tested.

datkins · April 20, 2024, 8:59am

thanks, i’l take a look, i have worked out how to get the available params for amlvenc using

gst-inspect-1.0 amlvenc

I can change the bitrate, I set it using

! amlvenc bitrate=6000 \

but get about 8mbit/s

also how do I change the codec from h264 to h265?