Issues with recording custom video + audio when using `v4l2-ctl` and `gst-launch-1.0`

omarc · June 12, 2024, 6:39pm

Which system do you use? Android, Ubuntu, OOWOW or others?

Ubuntu 22.04, (5.15.119 kernel)

Which version of system do you use? Please provide the version of the system here:

VIM4 with NPU
+
Khadas OS08A10 camera

Please describe your issue below:

Issues with recording video + audio using v4l2-ctl and gst-launch-1.0.

I am trying to record a video with gst-launch-1.0 as described in the documentation:

gst-launch-1.0 -e v4l2src device=/dev/media0 io-mode=mmap num-buffers=300 ! video/x-raw,format=NV21,width=1920,height=1080,framerate=30/1 ! amlvenc bitrate=8000 ! h265parse ! qtmux ! filesink location=test-h265.mp4

(New VIM4 MIPI camera usage [Khadas Docs])

However, I noticed that format changes like the framerate are ignored when running the command. Even the example in the documentation (shown before) does not record the video at 30 framerate; it records the video at 25.

I changed the frame rate to a lower value (10/1) to see significant changes but the video was still recorded at 25.

My final goal is to record a video + audio using the OS08A10 + the integrated microphone in the VIM4. I want to change the exposure of the camera using v4l2-ctl with v4l2-ctl -d /dev/v4l-subdev4 --set-ctrl exposure=32768 but even this command seems to not take effect when using gst-launch-1.0 afterward. Are parameters hardcoded?

The only parameter that seems to change is the video size.

What am I missing from the documentation? Do I need to change some configuration values at startup to fix the parameters?

Can you provide an example of a gst-launch-1.0 pipeline that records video and audio for the specific video + mic devices in the vim4?

As an additional note, when I record a video using gst-launch-1.0, I see a lot of debug logging messages coming from amlv4l2src module, is this normal? For example,
[2024-06-12 18:19:13] DEBUG [amlv4l2src camctrl.cc:758:isp_alg_process_one]process one frame successfully

Post a console log of your issue below:

numbqq · June 13, 2024, 6:19am

Hello @omarc

@ivan.li will help you then.

omarc · June 13, 2024, 7:36am

Hello @numbqq ,

Great, I look forward to @ivan.li help.

Omar

omarc · June 13, 2024, 12:14pm

Hi @ivan.li ,

In case it is needed, this is the result of v4l2-ctl -d /dev/v4l-subdev4 -l

User Controls
                       exposure 0x00980911 (int)    : min=0 max=65535 step=1 default=0 value=451
                           gain 0x00980913 (int)    : min=0 max=240 step=1 default=0 value=0
                     sensor_fps 0x00981900 (int)    : min=1 max=30 step=1 default=30 value=30 flags=volatile, execute-on-write
                       wdr_mode 0x00981904 (int)    : min=0 max=2 step=1 default=0 value=0
                   sensor_lanes 0x00981907 (int)    : min=1 max=4 step=1 default=4 value=4 flags=read-only, volatile

Image Processing Controls

                 link_frequency 0x009f0901 (intmenu): min=0 max=1 default=0 value=0 (1440000000 0x55d4a800) flags=read-only
                     pixel_rate 0x009f0902 (int64)  : min=1 max=2147483647 step=1 default=1152000000 value=1152000000 flags=read-only

I also noticed that each time I record a video via gst-launch-1.0, the exposure parameter changes. I thought it was related to the wdr_mode parameter, so I changed it to see if it lets me keep the parameters I set before, but it does not. It seems there is an auto mode that sets those values and ignores the ones set by the v4l2-ctl --set-control

omarc · June 13, 2024, 6:08pm

Hi @ivan.li ,

I have managed to record a video with audio using the OS08A10 and the onboard microphone using this command:

gst-launch-1.0 -e v4l2src device=/dev/media0 io-mode=mmap ! video/x-raw,format=NV21,width=1920,height=1080,framerate=30/1 ! queue ! amlvenc bitrate=4000 ! h264parse ! mux. alsasrc device="hw:AMLAUGESOUND,4" ! queue ! audio/x-raw,format=S16LE,channels=2,rate=48000 ! volume volume=2.0 ! audioconvert ! audioresample ! voaacenc ! aacparse ! qtmux name=mux ! filesink location=/home/khadas/src/web/app/static/preview.mp4 sync=true

The microphone sounds OK, but there is a static noise on the back all the time. I thought it was because of the volume module I added, but even at value 1.0, the static is still quite present. (here is a link to a preview of the recording SWITCHfilesender)

Nevertheless, I still have the same issues I mentioned before. The controls I set via v4l2-ctl are still ignored by gst-launch-1.0.

I looked at the v4l2_test_raw repository mentioned in the khadas docs about the new mipi camera documentation:

(New VIM4 MIPI camera usage [Khadas Docs])

The source code of your tool seems to consider the camera source as a dual camera and uses a WDR mode = 4,

if defined (DUAL_CAMERA)
        .fmt_code   = MEDIA_BUS_FMT_SRGGB12_1X12,
        .wdr_mode   = ISP_SDR_DCAM_MODE,

Maybe this code can help to see why the control settings seem to be ignored.

omarc · June 13, 2024, 10:48pm

@numbqq, I was using the Edge2 and the OS08A10 camera to record in night mode (without the IR filter), and you helped me use the camera and check if the IR filter was enabled.

However, now that I am doing tests with the VIM4 and the same OS08A10 camera, it seems that the camera has IR Filter enabled. Is that correct? Or maybe this behavior has to do with the WDR mode, too.

You mentioned
```In dts file, the IR-CUT disabled in the GPIO by default. So if one day you need the function you can open the node manully. We only set the reset-gpio.``

Where can I find this dts file in the VIM4 to disable the IR CUT of the OS08A10?

Thanks in advance,

ivan.li · June 14, 2024, 3:04am

@omarc

1.Get source code and compile reference documentation：
http://docs.khadas.com/products/sbc/vim4/development/start

2.You can annotate ircat gpio in DTS or modify the driver program.

vim /common_drivers/arch/arm64/boot/dts/amlogic/kvim4n.dts

     amlsens_0: sensor0@36 {
         compatible = "amlogic, sensor";
         status = "okay";
         index = <0>;
         reg = <0x36>;
         reg-addr-type = <2>;
         reg-data-type = <1>;
 
         clocks = <&clkc CLKID_MCLK_0>,
                 <&clkc CLKID_MCLK_0_SEL>,
                 <&clkc CLKID_MCLK_0_PRE>,
                 <&clkc CLKID_MCLK_PLL>,
                 <&xtal>;
         clock-names = "mclk","mclk_sel","mclk_pre","mclk_p","mclk_x";
         reset-gpios = <&gpio GPIOM_5 GPIO_ACTIVE_HIGH>;
 //      ircut-gpios = <&gpio GPIOM_11 GPIO_ACTIVE_HIGH>;                                       
         port {
             amlsens_0_ep: endpoint {
                 data-lanes = <0 1 2 3>;
                 link-frequencies = /bits/ 64 <1440000000>;
                 remote-endpoint = <&csiphy0_ep_2>;
             };
         };
     }

Or

vim common_drivers/drivers/amlisp/amlsens/aml_sens.c

112     // IRCUT                                                                                    
113     ircut = devm_gpiod_get_optional(sensor->dev, "ircut", GPIOD_OUT_LOW);
114     if (IS_ERR(ircut)) {
115         rtn = PTR_ERR(ircut);
116         dev_err(sensor->dev, "Cannot get ircut gpio: %d\n", rtn);
117         goto err_return;
118     }
119     gpiod_set_value(ircut, 1);   //1:open IRCUT , 0:close IRCUT
120     pr_info("ircut init\n");

omarc · June 14, 2024, 5:31am

Hello @ivan.li ,

Thanks for the information; I will download the source code for the common drivers only (is it right?), change and compile it right away.

Do you have some updates regarding why the parameters set via v4l2-ctl are being ignored?

ivan.li · June 14, 2024, 7:28am

@omarc

http://docs.khadas.com/products/sbc/vim4/development/linux/build-ubuntu
No, you need to use fenix, please refer to this document. repositories (U-Boot, Linux,common drivers) will be cloned automatically from the Khadas GitHub. then modify and compile on this basis.

Sorry, I am also analyzing this issue. It seems that these parameters can only be set by modifying the driver source code.

common_drivers/drivers/amlisp

omarc · June 14, 2024, 7:52am

@ivan.li, thanks for the prompt reply.

I used fenix in docker to do the compilation of the kernel, boot, and common_drivers (I used the steps in the documentation).

Then, I modified the automatically downloaded common_drivers repository with the changes you suggested and compiled it again. This time, it took less time (I disabled git_update so that the tool would not override my changes).

I saw that the compilation tool generated deb files in an OTA folder and in a vim4 folder. Can I copy the deb files to my vim4 and update the packages via apt update path_file?

linux-board-package-jammy-vim4_1.6.8_arm64
linux-dtb-amlogic-5.15_1.6.8_arm64
linux-headers-amlogic-5.15_1.6.8_arm64
linux-image-amlogic-5.15_1.6.8_arm64
linux-u-boot-vim4-vendor_1.6.8-2019.01_arm64

Or do I need to create the USB drive and reinstall the vim4 with the generated image? I don’t want to reinstall the system again, as I have other tools installed already.

Thanks for looking into the v4l2-ctl issue. If you find out that the solution (for the moment) is to recompile the driver to fix the desired controls, I can do it.

ivan.li · June 14, 2024, 9:18am

@omarc

Yes ,it is the right way.

eg.
on PC

scp linux-dtb-amlogic-5.15_1.6.8_arm64 khadas@192.xxx.xx.xx:~

on VIM4

sudo dpkg -i linux-dtb-amlogic-5.15_1.6.8_arm64

omarc · June 14, 2024, 9:38am

@ivan.li , I have tried the modified dtb deb, and it works! Well, the IRCUT is disabled now. Thanks.

You mentioned that I can set the other parameters in the amlisp driver, do you mean in common_drivers/drivers/amlisp/amlcam or /amllens?

ivan.li · June 14, 2024, 9:50am

@omarc

You can try this.

amlcam/amlsens/ov08a10/aml_ov08a10.c

omarc · June 18, 2024, 12:12pm

Hi @ivan.li I will look into that c file and try to change the camera’s parameters. Then, recompile the common_drivers module and install the OTA deb file.

I will let you know how it goes.

ivan.li · June 18, 2024, 12:24pm

Hi @omarc

If you have any questions, please leave a message in time.

omarc · July 22, 2024, 4:16pm

Hi @ivan.li , @numbqq

Sorry for the late reply.

I have tried to work on the driver file (aml_ov08a10.c) you suggested to change the controls of the v4l2 device. I managed to set different default values. However, it seems those values are ignored when using the gst-launch command.

Here is the output of the logs when recording a video using gst-launch-1.0 and some control functions passed to the v4l2src.

The logs show that the WDR setting is changed to 4 even though it was set to 0 by default.

media_set_wdrMode[420]: media_set_wdrMode ++ wdr_mode : 4

Then, the exposure and gain controls are computed and set which will override the controls set using v4l2-ctl.

I tried to look where in the driver file (aml_ov08a10.c) this feature is set, but I could not find it to disable it. I think it is not coming from the driver itself. It seems the log is coming from the gstreamer or the v4l2scr component. Can you help me locate this setting so that I can finally use the camera’s controls?

Thank you in advance,

LOGS:

gst-launch-1.0 -e v4l2src device=/dev/media0 io-mode=mmap extra-controls="a,wdr_mode=0,exposure=42000,gain=200" ! video/x-raw,format=NV21,width=1920,height=1080,framerate=30/1 ! queue ! amlvenc bitrate=4000 ! h264parse ! mux. alsasrc device="hw:AMLAUGESOUND,4" ! queue ! audio/x-raw,format=S16LE,channels=2,rate=48000 ! volume volume=2.0 ! audioconvert ! audioresample ! voaacenc ! aacparse ! qtmux name=mux ! filesink location=preview.mp4 sync=true
Setting pipeline to PAUSED ...
[API:aml_v4l2src_connect:271]Enter, devname : /dev/media0
func_name: aml_src_get_cam_method
initialize func addr: 0x7fa728167c
finalize func addr: 0x7fa7281948
start func addr: 0x7fa728199c
stop func addr: 0x7fa7281a4c
[2024-07-19 16:13:57]  DEBUG  [amlv4l2src camsrc.c:235:carm_src_is_usb]carm_src_is_usb:info(id 1, type 0x20000, name isp-csiphy), ret 0
[2024-07-19 16:13:57]  DEBUG  [amlv4l2src camsrc.c:235:carm_src_is_usb]carm_src_is_usb:info(id 4, type 0x20000, name isp-adapter), ret 0
[2024-07-19 16:13:57]  DEBUG  [amlv4l2src camsrc.c:235:carm_src_is_usb]carm_src_is_usb:info(id 7, type 0x20000, name isp-test-pattern-gen), ret 0
[2024-07-19 16:13:57]  DEBUG  [amlv4l2src camsrc.c:235:carm_src_is_usb]carm_src_is_usb:info(id 9, type 0x20000, name isp-core), ret 0
[2024-07-19 16:13:57]  DEBUG  [amlv4l2src camsrc.c:235:carm_src_is_usb]carm_src_is_usb:info(id 20, type 0x20001, name ov08a10-0), ret 0
[2024-07-19 16:13:57]  DEBUG  [amlv4l2src camsrc.c:235:carm_src_is_usb]carm_src_is_usb:info(id 24, type 0x10001, name isp-ddr-input), ret 0
[2024-07-19 16:13:57]  DEBUG  [amlv4l2src camsrc.c:235:carm_src_is_usb]carm_src_is_usb:info(id 28, type 0x10001, name isp-param), ret 0
[2024-07-19 16:13:57]  DEBUG  [amlv4l2src camsrc.c:235:carm_src_is_usb]carm_src_is_usb:info(id 32, type 0x10001, name isp-stats), ret 0
[2024-07-19 16:13:57]  DEBUG  [amlv4l2src camsrc.c:235:carm_src_is_usb]carm_src_is_usb:info(id 36, type 0x10001, name isp-output0), ret 0
[2024-07-19 16:13:57]  DEBUG  [amlv4l2src camsrc.c:235:carm_src_is_usb]carm_src_is_usb:info(id 40, type 0x10001, name isp-output1), ret 0
[2024-07-19 16:13:57]  DEBUG  [amlv4l2src camsrc.c:235:carm_src_is_usb]carm_src_is_usb:info(id 44, type 0x10001, name isp-output2), ret 0
[2024-07-19 16:13:57]  DEBUG  [amlv4l2src camsrc.c:235:carm_src_is_usb]carm_src_is_usb:info(id 48, type 0x10001, name isp-output3), ret 0
[2024-07-19 16:13:57]  DEBUG  [amlv4l2src camsrc.c:235:carm_src_is_usb]carm_src_is_usb:info(id 52, type 0x10001, name isp-raw), ret 0
[2024-07-19 16:13:57]  DEBUG  [amlv4l2src camsrc.c:235:carm_src_is_usb]carm_src_is_usb:info(id -2147483596, type 0x0, name ), ret -1
[2024-07-19 16:13:57]  DEBUG  [amlv4l2src camsrc.c:239:carm_src_is_usb]carm_src_is_usb:error Invalid argument
[2024-07-19 16:13:57]  DEBUG  [amlv4l2src camsrc.c:79:cam_src_select_socket]select socket:/tmp/camctrl0.socket
[2024-07-19 16:13:57]  DEBUG  [amlv4l2src camsrc.c:103:cam_src_obtain_devname]fork ok, pid:2289
[2024-07-19 16:13:57]  DEBUG  [amlv4l2src camsrc.c:103:cam_src_obtain_devname]fork ok, pid:0
[2024-07-19 16:13:57]  DEBUG  [amlv4l2src camsrc.c:107:cam_src_obtain_devname]execl /usr/bin/camctrl
[2024-07-19 16:13:57]  DEBUG  [amlv4l2src camctrl.cc:925:main][camctrl.cc:main:925]

[2024-07-19 16:13:57]  DEBUG  [amlv4l2src camctrl.cc:889:parse_opt]media device name: /dev/media0
[2024-07-19 16:13:57]  DEBUG  [amlv4l2src camctrl.cc:898:parse_opt]Server socket: /tmp/camctrl0.socket
Opening media device /dev/media0
Enumerating entities
Found 13 entities
Enumerating pads and links
mediaStreamInit[35]: mediaStreamInit ++.

mediaStreamInit[39]: media devnode: /dev/media0
mediaStreamInit[56]: ent 0, name isp-csiphy
mediaStreamInit[56]: ent 1, name isp-adapter
mediaStreamInit[56]: ent 2, name isp-test-pattern-gen
mediaStreamInit[56]: ent 3, name isp-core
mediaStreamInit[56]: ent 4, name ov08a10-0
mediaStreamInit[56]: ent 5, name isp-ddr-input
mediaStreamInit[56]: ent 6, name isp-param
mediaStreamInit[56]: ent 7, name isp-stats
mediaStreamInit[56]: ent 8, name isp-output0
mediaStreamInit[56]: ent 9, name isp-output1
mediaStreamInit[56]: ent 10, name isp-output2
mediaStreamInit[56]: ent 11, name isp-output3
mediaStreamInit[56]: ent 12, name isp-raw
mediaStreamInit[96]: get  lens_ent fail
mediaLog[30]: v4l2_video_open: open subdev device node /dev/video63 ok, fd 9

mediaStreamInit[151]: mediaStreamInit open video0 fd 9
mediaLog[30]: v4l2_video_open: open subdev device node /dev/video64 ok, fd 10

mediaStreamInit[155]: mediaStreamInit open video1 fd 10
mediaLog[30]: v4l2_video_open: open subdev device node /dev/video65 ok, fd 11

mediaStreamInit[159]: mediaStreamInit open video2 fd 11
mediaLog[30]: v4l2_video_open: open subdev device node /dev/video66 ok, fd 12

mediaStreamInit[163]: mediaStreamInit open video3 fd 12
mediaStreamInit[172]: media stream init success
fetchPipeMaxResolution[27]: find matched sensor configs 3840x2160
media_set_wdrMode[420]: media_set_wdrMode ++ wdr_mode : 0

media_set_wdrMode[444]: media_set_wdrMode success --

media_set_wdrMode[420]: media_set_wdrMode ++ wdr_mode : 4

mediaLog[30]: v4l2_subdev_open: fd 14

mediaLog[30]: v4l2_subdev_open: fd 15

media_set_wdrMode[444]: media_set_wdrMode success --

mediaLog[30]: v4l2_subdev_open: fd 13

mediaLog[30]: v4l2_subdev_open: fd 16

mediaLog[30]: v4l2_subdev_open: fd 14

mediaLog[30]: v4l2_subdev_open: fd 14

mediaLog[30]: v4l2_subdev_open: fd 15

[2024-07-19 16:13:57]  DEBUG  [amlv4l2src camctrl.cc:374:link_and_activate_subdev]link and activate subdev successfully
[2024-07-19 16:13:57]  DEBUG  [amlv4l2src camctrl.cc:407:media_stream_config]config media stream successfully
mediaLog[30]: v4l2_video_open: open subdev device node /dev/video62 ok, fd 17

mediaLog[30]: VIDIOC_QUERYCAP: success

[2024-07-19 16:13:57]  DEBUG  [amlv4l2src camctrl.cc:172:check_capability]entity[isp-stats] -> video[/dev/video62], cap.driver:aml-camera, capabilities:0x85200001, device_caps:0x5200001
mediaLog[30]: v4l2_video_open: open subdev device node /dev/video61 ok, fd 18

mediaLog[30]: VIDIOC_QUERYCAP: success

[2024-07-19 16:13:57]  DEBUG  [amlv4l2src camctrl.cc:172:check_capability]entity[isp-param] -> video[/dev/video61], cap.driver:aml-camera, capabilities:0x85200001, device_caps:0x5200001
mediaLog[30]: set format ok, ret 0.

mediaLog[30]: set format ok, ret 0.

mediaLog[30]:  request buf ok

mediaLog[30]:  request buf ok

mediaLog[30]: query buffer success

[2024-07-19 16:13:57]  DEBUG  [amlv4l2src camctrl.cc:546:isp_alg_param_init]isp stats query buffer, length: 262144, offset: 0
mediaLog[30]: query buffer success

[2024-07-19 16:13:57]  DEBUG  [amlv4l2src camctrl.cc:546:isp_alg_param_init]isp stats query buffer, length: 262144, offset: 262144
mediaLog[30]: query buffer success

[2024-07-19 16:13:57]  DEBUG  [amlv4l2src camctrl.cc:546:isp_alg_param_init]isp stats query buffer, length: 262144, offset: 524288
mediaLog[30]: query buffer success

[2024-07-19 16:13:57]  DEBUG  [amlv4l2src camctrl.cc:546:isp_alg_param_init]isp stats query buffer, length: 262144, offset: 786432
mediaLog[30]: query buffer success

[2024-07-19 16:13:57]  DEBUG  [amlv4l2src camctrl.cc:568:isp_alg_param_init]isp param query buffer, length: 262144, offset: 0
alg2User func addr: 0x7f8f728ed8
alg2Kernel func addr: 0x7f8f728f08
algEnable func addr: 0x7f8f728d70
algDisable func addr: 0x7f8f728e90
algFwInterface func addr: 0x7f8f729008
matchLensConfig[43]: LKK: fail to match lensConfig

cmos_get_ae_default_ov08a10[64]: cmos_get_ae_default

cmos_get_ae_default_ov08a10[64]: cmos_get_ae_default

aisp_enable[984]: tuning device not exist!

aisp_enable[987]: 3a commit b56e430e80b995bb88cecff66a3a6fc17abda2c7

cmos_fps_set_ov08a10[206]: cmos_fps_set: 7680.000000

mediaLog[30]: v4l2_subdev_open: fd 13

cmos_inttime_calc_table_ov08a10[180]: expo: 1741

mediaLog[30]: queue buffer success

mediaLog[30]: queue buffer success

mediaLog[30]: queue buffer success

mediaLog[30]: queue buffer success

mediaLog[30]: queue buffer success

mediaLog[30]: streamon   success

mediaLog[30]: streamon   success

[2024-07-19 16:13:57]  DEBUG  [amlv4l2src camctrl.cc:650:isp_alg_param_init]Finish initializing amlgorithm parameter ...
[2024-07-19 16:13:57]  DEBUG  [amlv4l2src camctrl.cc:971:main]UNIX domain socket bound
[2024-07-19 16:13:57]  DEBUG  [amlv4l2src camctrl.cc:977:main]Accepting connections ...
[2024-07-19 16:13:58]  DEBUG  [amlv4l2src camsrc.c:122:cam_src_obtain_devname]udp_sock_create
[2024-07-19 16:13:58]  DEBUG  [amlv4l2src common/common.c:70:udp_sock_create][93977120][/tmp/camctrl0.socket] start connect
[2024-07-19 16:13:58]  DEBUG  [amlv4l2src camsrc.c:124:cam_src_obtain_devname]udp_sock_recv
[2024-07-19 16:13:58]  DEBUG  [amlv4l2src camctrl.cc:985:main]connected_sockfd: 20
[2024-07-19 16:13:58]  DEBUG  [amlv4l2src camctrl.cc:989:main]video_dev_name: /dev/video63
[2024-07-19 16:13:58]  DEBUG  [amlv4l2src camsrc.c:282:cam_src_initialize]obtain devname: /dev/video63
devname : /dev/video63
driver : aml-camera
device : Amlogic Camera Card
bus_info : platform:aml-cam
version : 331639
error tvin-port use -1
[API:aml_v4l2src_streamon:373]Enter
[2024-07-19 16:13:58]  DEBUG  [amlv4l2src camsrc.c:298:cam_src_start]start ...
[API:aml_v4l2src_streamon:376]Exit
[2024-07-19 16:13:58]  DEBUG  [amlv4l2src camctrl.cc:860:process_socket_thread]receive streamon notification
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
Set log level to 1
[selectHW:92] selectEncoder
[selectHW:96] amvenc_multi 264 present
[amvenc_multi_load_Module:40] amvenc_ports multi loadModule!, load_count:0

Set log level to 4
[initEncParams:205] enc_feature_opts is 0x1 , GopPresetis 0x0

New clock: GstAudioSrcClock
Redistribute latency...
[vdi_sys_sync_inst_param:673] [VDI] fail to deliver sync instance param inst_idx=0

[AML_MultiEncInitialize:1388] VPU instance param sync with open param failed

[SetSequenceInfo:989] Required buffer fb_num=3, src_num=1, actual src=3 1920x1080

Redistribute latency...
mediaLog[30]: dq buffer success

mediaLog[30]: dq buffer success

[vl_multi_encoder_adjust_h264_header:529] sps_start=0

again_reg = 2
cmos_inttime_calc_table_ov08a10[180]: expo: 1386

cmos_alg_update_ov08a10[233]: gain value = 2

mediaLog[30]: v4l2_subdev_open: fd 13

cmos_alg_update_ov08a10[245]: expo.value = 1386

mediaLog[30]: v4l2_subdev_open: fd 13

mediaLog[30]: queue buffer success

mediaLog[30]: queue buffer success

[2024-07-19 16:13:58]  DEBUG  [amlv4l2src camctrl.cc:758:isp_alg_process_one]process one frame successfully
Redistribute latency...
mediaLog[30]: dq buffer success

mediaLog[30]: dq buffer success

again_reg = 0
cmos_inttime_calc_table_ov08a10[180]: expo: 1466

cmos_alg_update_ov08a10[233]: gain value = 0

mediaLog[30]: v4l2_subdev_open: fd 13

cmos_alg_update_ov08a10[245]: expo.value = 1466

mediaLog[30]: v4l2_subdev_open: fd 13

mediaLog[30]: queue buffer success

mediaLog[30]: queue buffer success

[2024-07-19 16:13:58]  DEBUG  [amlv4l2src camctrl.cc:758:isp_alg_process_one]process one frame successfully
mediaLog[30]: dq buffer success

mediaLog[30]: dq buffer success

cmos_inttime_calc_table_ov08a10[180]: expo: 1366

cmos_alg_update_ov08a10[245]: expo.value = 1366

mediaLog[30]: v4l2_subdev_open: fd 13

mediaLog[30]: queue buffer success

mediaLog[30]: queue buffer success

[2024-07-19 16:13:58]  DEBUG  [amlv4l2src camctrl.cc:758:isp_alg_process_one]process one frame successfully
mediaLog[30]: dq buffer success

mediaLog[30]: dq buffer success

cmos_inttime_calc_table_ov08a10[180]: expo: 1265

cmos_alg_update_ov08a10[245]: expo.value = 1265

mediaLog[30]: v4l2_subdev_open: fd 13

mediaLog[30]: queue buffer success

mediaLog[30]: queue buffer success

[2024-07-19 16:13:58]  DEBUG  [amlv4l2src camctrl.cc:758:isp_alg_process_one]process one frame successfully
mediaLog[30]: dq buffer success

mediaLog[30]: dq buffer success

mediaLog[30]: queue buffer success

mediaLog[30]: queue buffer success

[2024-07-19 16:13:58]  DEBUG  [amlv4l2src camctrl.cc:758:isp_alg_process_one]process one frame successfully
mediaLog[30]: dq buffer success

mediaLog[30]: dq buffer success

mediaLog[30]: queue buffer success

mediaLog[30]: queue buffer success

omarc · August 12, 2024, 8:43am

Hello @ivan.li , @numbqq ,

I have tried to trace in the driver why and when the controls are ignored when recording.

I could not find out where the gain, exposure, and fps are computed (maybe created to the WDR), but I found where those parameters are finally set.
In the aml_ov08a10.c file, there are three utility functions to set the exposure, gain, and fps. I had to ignore the values (computed elsewhere) and set my values there.

This is not a clean fix, but I could only trace the driver’s interaction with the V4L2 kernel module. Do you know where these values are computed?

Note: I have also tried to set the values by modifying the register maps you have set in aml_ov08a10.h, but doing this conflicts with some other module, and the results are corrupted recordings.

Best regards,

ivan.li · August 22, 2024, 8:41am

Hello @omarc
Sorry for replying to you so late,we have started to deal with these issues ,I will promptly feedback the results to you.

omarc · August 27, 2024, 4:37pm

Hello @ivan.li ,

It’s great to hear that you are working on the issue. I look forward to your feedback.

Just an update: I did an additional test with my VIM4 NPU + OS08A10 camera using the new Ubuntu 22.04 kernel 5.15.1 available from the Khadas repository. This version comes with the fenix 1.6.9.1 common drivers (where the camera driver is). Although the video recording via gst-launch-1.0 has fewer debug messages, the issue still persists.

steely-glint · August 28, 2024, 2:40pm

I think this static is from the cpu heatsink fan which blows directly onto the microphone. - You will probably hear the fan changing speeds

The onboard microphone is only useful on passively cooled installs, or cases with an external heatsink/fan.