Vim3l ov5640 sensor bringup help needed

VIM3
2

Update from my side:
Now i can capture image using OV5640 but image is too bad and facing repeated data issue in raw frame data.

Command to capture

v4l2-ctl --device /dev/video0 --set-fmt-video=width=1920,height=1080,pixelformat=YUV --stream-mmap --stream-to=frame.raw --stream-count=1

The frame is attached.

Screenshot from 2022-03-07 10-58-34
Screenshot from 2022-03-07 10-58-341920×1080 607 KB

Below are the dmesg logs,


[   52.738645] DEBUG ov5640_open.
[   52.738656] DEBUG OPEN : vm_init_resource, 2091, info->vdin_id : 0
[   52.738660] DEBUG OPEN : vm_init_resource, 2109
[   52.738665] DEBUG OPEN : dma_alloc_from_contiguous, 193, align = 0, count = 6144
[   52.738668] DEBUG OPEN : dma_alloc_from_contiguous, 197, align = 0, count = 6144
[   52.738673] cma: cma_alloc(cma ffffff800ad272e8, count 6144, align 0)
[   52.761854] cma: cma_alloc(): returned ffffffbf01570000
[   52.761865] DEBUG OPEN : vm_init_resource, 2119
[   52.761868] DEBUG OPEN : vm_init_resource, 2123
[   52.761871] DEBUG OPEN : vm_init_resource, 2125
[   52.761875] DEBUG OPEN : vm_init_resource, 2137
[   52.762011] DEBUG : clk_rate : 24000000 
[   52.902032] aml_cams: ov5640 init OK
[   52.942703] OV5640_init_regs-success in initial OV5640
[   52.970141] DEBUG ov5640_open-3446 Open compelte 
[   52.970242] res_para->size_type = 19, res_param->frmsize.width = 1920, height = 1080
[   53.004233] setting resolutin param complete
[   53.005036] DEBUG vidioc_streamon-3101
[   53.005044] vidioc_streamon: h_active=1920; v_active=1080, frame_rate=30,vdin_path=0
[   53.005047] DEBUG ov5640 stream on.
[   53.005052] ov5640 start tvin service.
[   53.005056] **[start_tvin_service]cfmt:1;dfmt:1;dest_hactive:0;
[   53.005060] dest_vactive:0;frame_rate:30;h_active:1920,
[   53.005063] v_active:1080;scan_mode:1**
[   53.005338] settle = 0x001a
[   53.005343] info->lanes = 2
[   53.005457] DEBUG : vdin_start_dec, 471, vdin_dbg_en : 1
[   53.005460] DEBUG : vdin_start_dec, 478
[   53.005464] devp->para.cfmt=1, devp->para.dfmt=1
[   53.005467] DEBUG : vdin_start_dec, 504
[   53.005471] DEBUG : vdin_start_dec, 514
[   53.005480] DEBUG : vdin_start_dec, 521
[   53.005485] DEBUG : vdin_start_dec, 545
[   53.005489] DEBUG : vdin_start_dec, 550
[   53.005492] DEBUG : vdin_start_dec, 555
[   53.005496] vdin0 init afbce_mode: 0
[   53.005499] DEBUG : vdin_start_dec, 563
[   53.005502] DEBUG : vdin_start_dec,566
[   53.005507] DEBUG OPEN : dma_alloc_from_contiguous, 193, align = 0, count = 16384
[   53.005511] DEBUG OPEN : dma_alloc_from_contiguous, 197, align = 0, count = 16384
[   53.005515] cma: cma_alloc(cma ffffff800ad27238, count 16384, align 0)
[   53.050788] cma: cma_alloc(): returned ffffffbf016f0000
[   53.050798] vdin0 mem_start = 0x5bc00000, mem_size = 0x1db1000
[   53.050801] vdin0 cma alloc ok!
[   53.050805] DEBUG : vdin_start_dec, 574
[   53.050819] DEBUG : vdin_start_dec, 586
[   53.050833] vdin_game_mode_check: game_mode flag=0, game_mode=0
[   53.050837] vdin.0 vframe initial information table: (5 of 9)
[   53.050844] 	 0: 0x26 1920x1080, duration = 3200
[   53.050849] 	 1: 0x27 1920x1080, duration = 3200
[   53.050853] 	 2: 0x28 1920x1080, duration = 3200
[   53.050857] 	 3: 0x29 1920x1080, duration = 3200
[   53.050861] 	 4: 0x2a 1920x1080, duration = 3200
[   53.050864] DEBUG : vdin_start_dec, 616
[   53.050867] DEBUG : vdin_start_dec, 626
[   53.050873] DEBUG : vdin_start_dec, 634
[   53.050891] DEBUG : vdin_start_dec, 639
[   53.050895] start_amvdec_csi.
[   53.050898] start_amvdec_csi port = 10000
[   53.050901] init_csi_dec_parameter.
[   53.050904] reinit_csi_dec, 80
[   53.050908] DEBUG : vdin_start_dec, 649
[   53.051049] DEBUG : vdin_start_dec, 673
[   53.051053] ****[vdin_start_dec]ok!****
[   53.051056] DEBUG : vdin_start_dec, 692
[   53.051059] DEBUG : vdin_start_dec, 698
[   53.057996] DEBUG CAM: vm_fill_this_buffer, 927
[   53.058003] DEBUG CAM: vm_fill_this_buffer, 934
[   53.058006] DEBUG CAM: vm_fill_this_buffer, 949
[   53.058010] DEBUG CAM: vm_fill_this_buffer, 956
[   53.058013] DEBUG CAM: vm_fill_this_buffer, 959
[   53.058016] DEBUG CAM: vm_fill_this_buffer, 972
[   53.058019] DEBUG CAM: vm_fill_this_buffer, 982
[   53.058027] DEBUG CAM: vm_fill_this_buffer, 1013
[   53.063669] DEBUG CAM: vm_fill_this_buffer, 1029
[   53.063677] DEBUG CAM: vm_fill_this_buffer, 1035
[   53.070051] DEBUG CAM: vm_fill_this_buffer, 927
[   53.070058] DEBUG CAM: vm_fill_this_buffer, 934
[   53.070061] DEBUG CAM: vm_fill_this_buffer, 949
[   53.070064] DEBUG CAM: vm_fill_this_buffer, 956
[   53.070067] DEBUG CAM: vm_fill_this_buffer, 959
[   53.070070] DEBUG CAM: vm_fill_this_buffer, 972
[   53.070073] DEBUG CAM: vm_fill_this_buffer, 982
[   53.070081] DEBUG CAM: vm_fill_this_buffer, 1013
[   53.075736] DEBUG CAM: vm_fill_this_buffer, 1029
[   53.075744] DEBUG CAM: vm_fill_this_buffer, 1035
[   53.080450] DEBUG ov5640 stream off.
[   53.080466] DEBUG : vdin_stop_dec, 716
[   53.080472] reset_btcsi_module, 75
[   53.080556] cma: cma_release(page ffffffbf016f0000)
[   53.081079] vdin0 cma release ok!
[   53.081091] vdin_stop_dec ok
[   53.081689] aml_cams: ov5640 uninit.
[   53.133982] Success disable mclk
[   53.133991] cma: cma_release(page ffffffbf01570000)
[   53.134182] DEBUG ov5640_close-3528 Close compelte

Also wanted to understand if I can generate test pattern in CSI or CSI decoder ?

Any help will be appreciated.

3

Update from my side,

This seems to capture vim3l connected HDMI display’s screen,
Surprisingly its c screen capture/screenshot mode.

image
image1920×1080 564 KB

Also my dts file has diffrenet node here in cam_0 as per below for ov5640 sensor,

   cam_0{
           cam_name = "ov5640";
           front_back = <0>;
           camera-i2c-bus = <&i2c_AO>;
           gpio_pwdn-gpios = <&gpio_expander 3 GPIO_ACTIVE_HIGH>;
           gpio_rst-gpios = <&gpio_expander 2 GPIO_ACTIVE_HIGH>;
           mirror_flip = <1>;
           vertical_flip = <0>;
           spread_spectrum = <0>;
           bt_path = "csi";
           interface = "mipi";
           vdin_path = <0>;
           clk_channel = "a";
           bt_path_count = <1>;
           status = "okay";
   };

@numbqq can you help or tag someone from khadas team to help ?

Really appreciate

4

Sorry, I don’t have much experience about OV5640 on VIM3L, so you have to debug youself.
As the isp driver from Ubuntu linux kernel is not latest, maybe you can try to sync latest driver from Android source code to check.

5

Hi @numbqq,

Thanks for reply.

As VIM3L, there is no ISP, do we still need ISP driver from the mentioned source ?

Can you help me with the link from where can i get it ?

Also I have seen many camera forums is being addressed by @Gouwa, @chewitt, @Spikerguy, @Electr1 and @Frank , So I would like to ask them as well for any pointers.

  1. I am also getting confused in DTS between sensor(vim3) node,
	sensor: sensor {
		compatible = "soc, sensor";
                .............
	};

and cam_X node (vim3l) being on a same BSP.

aml_cams {
         compatible = "amlogic, cams_prober";
         ............
          cam_0{
                  cam_name = "ov5640";
                  .......
            };
    };
  1. After adding cam_X node and enabling ov5640 , why i am seeing screen shot and not the capture image ?
6

目前 我也需要再VIM3L调试camera 以下是我理解：
1.S905D3不内置ISP模块，需要sensor自带ISP模块，所以camera底层不能走vendor arm_isp驱动，走cams_prober + amvdec_csi框架;，同时I2C配置ISP模块。
2.数据格式。OV5640是支持YUV RGB等格式，S905D3 CSI采集支持哪些格式？从以下寄存器看是支持YUV RGB采集的。

image
image939×100 21.5 KB

3.A311D 模块CSI data path: CSI DPHY–>mipi adapter,S905D3 手册没有这块的介绍。从代码来看
vidioc_streamon -->start_tvin_service会调用VDIN模块，data path是 CSI DPHY->VDIN?

不知道我的理解是否正确 @ KD_1993 @numbqq

2 Likes
7

@seekdream1990 , It seems you and I are having same assumptions.
We have two different kind of frameworks for Camera.

  1. Framework with ISP - soc sensor :
  • This framework seems like it can work with only VIM3 and we can not use it with VIM3L. Here we have many registers which are not present in Amlogic S905D3.
  • The framework is not as per v4l2 complaint.
    I.e stream-on IOCTL is not bind with v4l2_ioctl_ops, can not see videobuffer events like VIDEOBUF_DONE
  • In s905d3 in video path, video path is provided and have different register sets for mipi adapter, I believe they want developer to assume how hardware will work.
  1. Framework without ISP - cam prober :
  • Here in (as you mentioned ) streamon I could see starting csi but not from hardware level , no hardware registers are being written,
  • No communication seems to be happening between sensor and CSI
  • Getting ISR in decoder but nothing from CSI (because nobody told him to do his job via register writing) hence may be I am getting wrong information in frames and seeing own screenshot.

I may be wrong on these but currently it seems so.

Gc2145摄像头显示花屏,如何确定问题, here is also same thing happening (of course in android ),

At the end of the story, lack of documentation and proper implementations is making people suffer.

What bothering me is, this is going to be our production board and not the hobby board, but this lack of support from community, documentation and other relevant thing is not helping us.

Khadas team, @numbqq , @Frank @Gouwa , at least provide some pointers, we are not asking you to develop for us. lets try to make a batter world by helping each other.

Thanking you in anticipation.

8

1.关于S905D3 sensor与CSI mipi的通信疑问：
实际上，在对应sensor的驱动中（例如ov5640），打开camera后vidioc_streamon 接口会调用VDIN驱动模块的start_tvin_service，最终通过amcsi_feopen–>am_mipi_csi2_init(该接口最终配置adapter以及PHY寄存器配置)。但是VDIN模块与sensor的关系还是有些不明白的地方，就如上面所说，难道data数据patch：
CSI DPHY->VDIN

9

@ KD_1993

10

Hi @seekdream1990

I agree on this.

Moving forward, from vdin after start_tvin_service, it is registering in vdin_v4l2_isr for mipi port. sometime in future from vdin_v4l2_isr will call decoder_isr and in decoder_isr, it seems to it read the frame infos in amcsi_isr (afifo).

In my case
amcsi_isr, data1- 0x00, frame.w-0, frame.h -0,ddr_address:0x0, index:0, status-0, read_cnt-0

everything is zero.

I suspect, here i should get a frame or at least its height width and other information.

However as you rightly mentioned, I also dont see a way to pass this info to vdin.

@numbqq , at least can we check for any missing patch from amlogic repo for this ?

11

For Android source has latest drivers & patches. For Ubuntu source code we only test our camera module OS08A10 on VIM3, we haven’t checked other camera modules. So for new camera bringup, you can use android.

12

@numbqq
I have already asked for the link. kindly provide it ASAP.

14
15

@numbqq
Thanks for the link.

I had clone the android source immediately. But, I can not find csi driver, dts of vim3L, registers of MIPI(i.e. MIPI_PHY_CLK_LANE_CTRL) in branch khadas_vims_pie and naught.

Can you guide where to find them?

16

Dear @numbqq ,

This situation forced us to move from vim3l to vim3, but at least on that do we see ov5640 running with vim3 with ISP with some sort of modification ?

Do khadas team has proper documentation on vim3 ?

Definitely development is our responsibility but as this is “non v4l2 compliant”, we may need your pointers in moving further with vim3.

Also I hope that there not any closed source blobs running under the hood for ISP under GPL licensing ?

Kindly share your inputs, we have spent lot financial efforts till now, for us its now a crucial moment.

17

We don’t have OV5640 module, so we don’t know the status on VIM3, we only tested OS08A10 on VIM3.

There is no documentation for this, all the documentation is the source code.

18

from compatible

amlogic, amvdec_csi
from
image

I dont see any entries which mean still no CSI driver present
image

My grep result

I can see it in arm, i believe our arch is arm64, I assume we can modify this as a reference in arch64 but still CSI driver is a question ?

Also I will be able and glad to ship one ov5640 with custom made connector especially for Khadas board, in hope of getting at least some sort of help pointers.(of course with proper NDA and documentation in place )

19

Android use the 32bit kernel.

20

@numbqq , and what about csi driver ?

How compatible is going to probe csi drver from amlogic, amvdec_csi ?

21
22

hi @numbqq
Thanks for the reply.

I had tried to use android on vim3l and the process stuck on the android Logo.

I am using following steps below:

  1. Build android using Building Android Source Code | Khadas Documentation.
  2. Burn OS using Upgrade Android Via An SD-Card | Khadas Documentation
  3. Also, Find similar issue in Vim3 fastboot stuck at logo. so, tried the solution given here. But, still it is stuck.

Last lines of adb log

01-01 12:01:36.113  3476  3501 D MesonHwc: wait for vsync success, (0xa71a1590) peroid: 19963625
01-01 12:01:37.244  3516  3516 I sh      : type=1400 audit(0.0:34): avc: denied { getattr } for path="/vendor/bin/toybox_vendor" dev="mmcblk0p16" ino=37 scontext=u:r:shell:s0 tcontext=u:object_r:vendor_toolbox_exec:s0 tclass=file permissive=1
01-01 12:01:37.304  3516  3516 I sh      : type=1400 audit(0.0:36): avc: denied { getattr } for path="/vendor/xbin/busybox" dev="mmcblk0p16" ino=516 scontext=u:r:shell:s0 tcontext=u:object_r:vendor_file:s0 tclass=file permissive=1
01-01 12:01:37.373  3476  3501 D MesonHwc: wait for vsync success, (0xa71a1590) peroid: 20078125
01-01 12:01:37.304  3516  3516 I sh      : type=1400 audit(0.0:37): avc: denied { execute } for name="busybox" dev="mmcblk0p16" ino=516 scontext=u:r:shell:s0 tcontext=u:object_r:vendor_file:s0 tclass=file permissive=1
01-01 12:01:37.456  3645  3645 I dmesg   : type=1400 audit(0.0:38): avc: denied { syslog_read } for scontext=u:r:shell:s0 tcontext=u:r:kernel:s0 tclass=system permissive=1
01-01 12:01:38.053  3476  3501 D MesonHwc: wait for vsync success, (0xa71a1590) peroid: 20057834
01-01 12:01:38.973  3476  3501 D MesonHwc: wait for vsync success, (0xa71a1590) peroid: 420066959
01-01 12:01:39.513  3476  3501 D MesonHwc: wait for vsync success, (0xa71a1590) peroid: 19934375
01-01 12:01:39.973  3476  3501 D MesonHwc: wait for vsync success, (0xa71a1590) peroid: 20066291
01-01 12:01:40.773  3476  3501 D MesonHwc: wait for vsync success, (0xa71a1590) peroid: 20060916
01-01 12:01:41.253  3476  3501 D MesonHwc: wait for vsync success, (0xa71a1590) peroid: 20061458
01-01 12:01:42.133  3476  3501 D MesonHwc: wait for vsync success, (0xa71a1590) peroid: 20062791
01-01 12:01:42.774  3476  3501 D MesonHwc: wait for vsync success, (0xa71a1590) peroid: 20067083
01-01 12:01:43.693  3476  3501 D MesonHwc: wait for vsync success, (0xa71a1590) peroid: 420071666
01-01 12:01:44.133  3476  3501 D MesonHwc: wait for vsync success, (0xa71a1590) peroid: 20054125
01-01 12:01:44.813  3476  3501 D MesonHwc: wait for vsync success, (0xa71a1590) peroid: 20054541
01-01 12:01:45.773  3476  3501 D MesonHwc: wait for vsync success, (0xa71a1590) peroid: 20051125
01-01 12:01:46.213  3476  3501 D MesonHwc: wait for vsync success, (0xa71a1590) peroid: 20064167
01-01 12:01:47.253  3476  3501 D MesonHwc: wait for vsync success, (0xa71a1590) peroid: 20056458
01-01 12:01:47.833  3476  3501 D MesonHwc: wait for vsync success, (0xa71a1590) peroid: 19921334
01-01 12:01:48.713  3476  3501 D MesonHwc: wait for vsync success, (0xa71a1590) peroid: 379963875
01-01 12:01:49.413  3476  3501 D MesonHwc: wait for vsync success, (0xa71a1590) peroid: 20046958

Logs in console

[    9.906080] <7>[    9.906080@1] audio_ddr_mngr: toddrs[0] released by device ff660000.audiobus:spdif@0
[    9.906532] <6>[    9.906532@1] aml_tdm_open
[    9.909180] <7>[    9.909180@0] audio_ddr_mngr: toddrs[0] registered by device ff660000.audiobus:tdm@0
[    9.909496] <7>[    9.909496@0] tdm capture mute: 1
[    9.909566] <7>[    9.909566@0] audio_ddr_mngr: toddrs[0] released by device ff660000.audiobus:tdm@0
[   11.342588] <6>[   11.342588@1] fb: osd[0] enable: 1 scale:0x10001 (composer@2.2-se)
[   11.343128] <6>[   11.343128@1] fb: osd[0] enable: 1 scale:0x10001 (composer@2.2-se)
[   11.350479] <6>[   11.350479@1] fb: free_scale_switch to fb0, mode: 0x10001
[   11.354757] <6>[   11.354757@2] fb: set logo loaded
[   12.276365] <6>[   12.276365@0] vout: aml_tvout_mode_work: monitor_timeout
[   28.738727] <31>[   28.738727@2] logd: logdr: UID=2000 GID=2000 PID=3640 b tail=0 logMask=19 pid=0 start=0ns timeout=0ns
[   62.804338] <6>[   62.804338@0] fb: mem_free_work, free memory: addr:800000
[   99.105479] <38>[   99.105479@0] type=1400 audit(1293883207.448:19): avc: denied { open } for pid=3470 comm="HwBinder:3470_1" path="/dev/__properties__/u:object_r:system_prop:s0" 1
[   99.125437] <38>[   99.125437@0] type=1400 audit(1293883297.244:34): avc: denied { getattr } for pid=3516 comm="sh" path="/vendor/bin/toybox_vendor" dev="mmcblk0p16" ino=37 sconte1
[   99.149335] <38>[   99.149335@0] type=1400 audit(1293883297.244:34): avc: denied { getattr } for pid=3516 comm="sh" path="/vendor/bin/toybox_vendor" dev="mmcblk0p16" ino=37 sconte1
[   99.167928] <38>[   99.167928@3] type=1400 audit(1293883297.244:35): avc: denied { execute } for pid=3516 comm="sh" name="toybox_vendor" dev="mmcblk0p16" ino=37 scontext=u:r:shell1
[   99.188428] <38>[   99.188428@3] type=1400 audit(1293883297.244:35): avc: denied { execute } for pid=3516 comm="sh" name="toybox_vendor" dev="mmcblk0p16" ino=37 scontext=u:r:shell1
[   99.209122] <38>[   99.209122@3] type=1400 audit(1293883297.304:36): avc: denied { getattr } for pid=3516 comm="sh" path="/vendor/xbin/busybox" dev="mmcblk0p16" ino=516 scontext=u1
[   99.229206] <38>[   99.229206@3] type=1400 audit(1293883297.304:36): avc: denied { getattr } for pid=3516 comm="sh" path="/vendor/xbin/busybox" dev="

Can you guide me here?