VIM4 OS08A10 Very Very Slow

VIM4
1

Which system do you use? Android, Ubuntu, OOWOW or others?

Ubuntu

Which version of system do you use? Please provide the version of the system here:

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 22.04.5 LTS
Release:        22.04
Codename:       jammy
$ uname -a
Linux Khadas 5.15.137 #1.6.9.1 SMP PREEMPT Thu Aug 15 15:08:21 CST 2024 aarch64 aarch64 aarch64 GNU/Linux

Please describe your issue below:

Following the instructions found here VIM4 OS08A10 MIPI Camera [Khadas Docs]

I have enabled the overlay. The further instructions to use /dev/media0 do not work. I can display the camera using the following but the frame rate is very slow, maybe 2fps, and the color depth does not look correct.

gst-launch-1.0 -e v4l2src device=/dev/video65 io-mode=mmap num-buffers=300 ! video/x-raw,format=NV21,width=3840,height=2160,framerate=60/1 ! videoconvert ! waylandsink

gst-launch-1.0 -e v4l2src device=/dev/video63 io-mode=mmap num-buffers=300 ! video/x-raw,format=NV21,width=3840,height=2160,framerate=60/1 ! videoconvert ! waylandsink

both of those will display the camera but very slowly. Using /dev/media0 does not display anything.

Post a console log of your issue below:

mediaStreamInit[39]: media devnode: /dev/media0
mediaStreamInit[56]: ent 0, name PureThermal (fw:v1.3.0): PureTh
mediaStreamInit[56]: ent 1, name PureThermal (fw:v1.3.0): PureTh
mediaStreamInit[56]: ent 2, name Extension 3
mediaStreamInit[56]: ent 3, name Processing 2
mediaStreamInit[56]: ent 4, name Extension 4
mediaStreamInit[56]: ent 5, name Extension 5
mediaStreamInit[56]: ent 6, name Extension 6
mediaStreamInit[56]: ent 7, name Extension 7
mediaStreamInit[56]: ent 8, name Extension 21
mediaStreamInit[56]: ent 9, name Extension 254
mediaStreamInit[56]: ent 10, name Input 1
mediaStreamInit[90]: get  sensor_ent fail
matchSensorConfig[105]: fail to match sensorConfig
fetchPipeMaxResolution[30]: do not find matched sensor configs
media_set_wdrMode[420]: media_set_wdrMode ++ wdr_mode : 0
2

If you want to use waylandsink you can try Ubuntu 24.04, it will have better performance.

For Ubuntu 22.04, it is better to use clutterautovideosink.

gst-launch-1.0 -e v4l2src device=/dev/video65 io-mode=mmap num-buffers=300 ! video/x-raw,format=NV12,width=3840,height=2160,framerate=60/1 ! clutterautovideosink
3

My bad, I thought I had installed Ubuntu 24.04. So here are some more notes:

  1. while under 22.04 I did try the clutterautovideosink which did run faster but I can still see something not right in the camera video. It appears as though there is some time based filtering of the video frames, almost as thought the frames add up. If you move your hand in front of the camera you can see the hand blended with the background.

  2. I reinstalled the latest Ubuntu 24.04 and the waylandsink behaves the same as in 22.04, only showing about 4.5fps. Also displays the strange time filtering as described above.

  3. I notice now that I am in 24.04 there is no clutterautovideosink available.

  4. I have tried a python appsink to see if it was any faster and no, it is also about 4.5fps and with the same strange additive filtering on the frames.

  5. I have tried other formats, UYVY, RGB, GRAY and they are all 4.5fps

I have noticed that I can get the video on /dev/video63,65,&66 why is that?

v4l2-ctrls show the following yet set-ctrl of exposure has no effect:

vim4:~$ v4l2-ctl -d 65 --list-ctrls

User Controls

                     brightness 0x00980900 (int)    : min=0 max=255 step=1 default=128 value=128 flags=slider
                       exposure 0x00980911 (int)    : min=0 max=65535 step=1 default=0 value=100
                           gain 0x00980913 (int)    : min=100 max=3200 step=1 default=100 value=100
           isp_sensor_input_fps 0x00981900 (int)    : min=0 max=120 step=1 default=30 value=60 flags=execute-on-write
                 isp_output_fps 0x00981901 (int)    : min=0 max=120 step=1 default=30 value=60 flags=execute-on-write
4

Hello @DoomMonkey

Could you provide the below information to us

cat /proc/cpu_chipid 
cat /etc/fenix-release 
cat /etc/lsb-release 
uname -a

@ivan.li please help to check.

6 
maia:~$ cat /proc/cpu_chipid
Serial:          360c010200000000051470846dc05910
Hardware:        Khadas VIM4

maia:~$ cat /etc/fenix-release
# PLEASE DO NOT EDIT THIS FILE
BOARD=VIM4
VENDOR=Amlogic
VERSION=1.6.9
ARCH=arm64
INITRD_ARCH=arm64
IMAGE_VERSION=1.6.9-240618
################ GIT VERSION ################
UBOOT_GIT_VERSION=khadas-vims-u-boot-2019.01-v1.6.9-release
LINUX_GIT_VERSION=khadas-vims-linux-5.15-v1.6.9-release
FENIX_GIT_VERSION=v1.6.9
#############################################

maia:~$ cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=24.04
DISTRIB_CODENAME=noble
DISTRIB_DESCRIPTION="Ubuntu 24.04.1 LTS"

maia:~$ uname -a
Linux maia 5.15.119 #1.6.9.1 SMP PREEMPT Thu Jun 20 09:22:43 CST 2024 aarch64 aarch64 aarch64 GNU/Linux
7

Hello @DoomMonkey

You can try this command:

gst-launch-1.0 -v v4l2src device=/dev/media0 io-mode=mmap  ! video/x-raw,format=NV12,width=3840,height=2160,framerate=30/1 ! videoconvert ! waylandsink
8

Ok, thank you @numbqq , this works. I see 30fps. However, I am still full of questions.

Even at this higher frame rate I can still see this kind of “ghosting” artifact as though the frames are being smeared together. Is this due to some HDR/WDR processing? If so is there a way to turn this off or to handle it differently?

Also, how do we control the exposure of this camera? It appears that v4l2-ctrl does not control the exposure when pointed at /dev/video63/65/66.

Could someone explain the relationship of the different devices /dev/video60-66 and /dev/media0?

Thank you

9

Hello @DoomMonkey

We will double check this issue.

10

Any progress or updates on this topic?

11

Hello @DoomMonkey

We are still checking this issue. Do you have issues when you just record the video and not preview?

12

I have not tried recording video. I have used gstreamer to directly display or I have used Python+OpenCV to capture and display. I will check a recording and report back.

13

So I do not see the ghosting problem in well lit or bright scenes. This makes me think it has something to do with the HDR framing. How do we control this camera, specifically the HDR controls and exposure?

14

Hello @DoomMonkey

Yes, we have reproduced this issue and will fix it in next release.