VIM3 cameras inquiry

Alex999 · January 23, 2024, 10:44am

Which Khadas SBC do you use?

VIM3 PRO

Which system do you use? Android, Ubuntu, OOWOW or others?

UBUNTU

Which version of system do you use? Khadas official images, self built images, or others?

official emmc

Please describe your issue below:

Hey!
I’ve got VIM3 PRo. Ordered it in rush, and didn’t notice that camera connector is 30pin. So i cant test quickly our current solution that uses IMX219, in particular ArduCam IMX219 8Mpx 1/4" camera - wide angle - for NVIDIA Jetson Nano - ArduCam B0179 Botland - Robotic Shop

The questions:

can i use this camera with VIM3 Pro some how? Is Imx219 supported? are there plans to support it?
this camera satisfies our requirements, but i saw even better offers from khadas shop https://www.khadas.com/product-page/os08a10-8mp-camera and https://www.khadas.com/product-page/imx415-camera For our tasks, which is tracking via dynamically moving camera, global shutter and high fps are crucial, but our solution works well with 30fps as well. What is the key is FOV. What FOV of those cameras (os08a10 and imx415), what lens mount is used? can we use another lenses?
do os08a10 and imx415 allow to change cable, to more longer one? it is not clear from photos. If not, could you offer some extender?
maybe you could advice some other VIM3 compatible products for us? 1900x1200 >=30fps, vFOV>=110deg, hFOV>=150deg, global shutter is optional but nice to have.
TNX

Jacobe · January 24, 2024, 1:51am

Hi @Alex999 ,

can i use this camera with VIM3 Pro some how? Is Imx219 supported? are there plans to support it?
maybe you could advice some other VIM3 compatible products for us?

VIM3 Pro currently only supports imx415 and os08a10.

do os08a10 and imx415 allow to change cable, to more longer one? it is not clear from photos. If not, could you offer some extender?

Currently there is only one cable on the official website. But we don’t recommend to replace the longer cable by yourself, maybe it will affect the mipi signal

this camera satisfies our requirements, but i saw even better offers from khadas shop https://www.khadas.com/product-page/os08a10-8mp-camera and https://www.khadas.com/product-page/imx415-camera For our tasks, which is tracking via dynamically moving camera, global shutter and high fps are crucial, but our solution works well with 30fps as well. What is the key is FOV. What FOV of those cameras (os08a10 and imx415), what lens mount is used? can we use another lenses?
1900x1200 >=30fps, vFOV>=110deg, hFOV>=150deg, global shutter is optional but nice to have.

You can check the details of imx415 camera on https://dl.khadas.com/products/add-ons/imx415/khadas-imx415-details.pdf

Alex999 · January 24, 2024, 10:10am

Thanks!
I still can’t find mount type used on os08a10 and imx415
Can I use different lens with these cameras and what are mount types?

Jacobe · January 25, 2024, 2:24am

Currently, this is the only lens we have on these cameras

Alex999 · January 25, 2024, 8:30am

That’s ok. But we have different lenses. And we have lens suppliers. I need to know mount type, and wether I can try different lens with these camera modules.

Jacobe · January 25, 2024, 11:16am

Both cameras are twisted to the motor through the lens screw thread, screw thread M12 * P0.5, the size of the screw hole will vary

Alex999 · January 25, 2024, 11:50am

S type? Lens mount - Wikipedia

Jacobe · January 26, 2024, 12:46am

Yes

Alex999 · January 30, 2024, 10:51am

Ok! Got the os08a10 for testing. Have some questions.
It is not clear from VIM3/3L OS08A10 MIPI Camera [Khadas Docs] how to setup camera for official emmc Ubuntu 22.04

$ ls /dev/vid*
/dev/video10  /dev/video16  /dev/video23  /dev/video34  /dev/video_composer.0
/dev/video11  /dev/video17  /dev/video26  /dev/video35  /dev/video_composer.1
/dev/video12  /dev/video18  /dev/video30  /dev/video36
/dev/video13  /dev/video19  /dev/video31  /dev/video37
/dev/video14  /dev/video20  /dev/video32  /dev/video38
/dev/video15  /dev/video21  /dev/video33  /dev/video50

There is no /dev/video0. But

$ v4l2-ctl --list-devices
amlvideo (amlvideo-000):
	/dev/video10

amlvideo (amlvideo-001):
	/dev/video23

amlvideo2 (amlvideo2):
	/dev/video11
	/dev/video12

platform:amlogic (platform:amlogic):
	/dev/video26

ionvideo (platform:ionvideo-000):
	/dev/video13

ionvideo (platform:ionvideo-001):
	/dev/video14

ionvideo (platform:ionvideo-002):
	/dev/video15

ionvideo (platform:ionvideo-003):
	/dev/video16

ionvideo (platform:ionvideo-004):
	/dev/video17

ionvideo (platform:ionvideo-005):
	/dev/video18

ionvideo (platform:ionvideo-006):
	/dev/video19

ionvideo (platform:ionvideo-007):
	/dev/video20

ionvideo (platform:ionvideo-008):
	/dev/video21

juno R2 (platform:isp-v4l2-0):
	/dev/video50

v4lvideo (platform:v4lvideo-000):
	/dev/video30

v4lvideo (platform:v4lvideo-001):
	/dev/video31

v4lvideo (platform:v4lvideo-002):
	/dev/video32

v4lvideo (platform:v4lvideo-003):
	/dev/video33

v4lvideo (platform:v4lvideo-004):
	/dev/video34

v4lvideo (platform:v4lvideo-005):
	/dev/video35

v4lvideo (platform:v4lvideo-006):
	/dev/video36

v4lvideo (platform:v4lvideo-007):
	/dev/video37

v4lvideo (platform:v4lvideo-008):
	/dev/video38

Cannot open device /dev/video0, exiting.

There is /dev/video50. But I didn’t manage quickly to get live stream:

$ gst-launch-1.0 v4l2src name=vsrc device=/dev/video50 ! video/x-raw,width=1920,height=1080,framerate=60/1,format=RGB ! autovideoconvert ! autovideosink
Setting pipeline to PAUSED ...
[API:aml_v4l2src_connect:271]Enter, devname : /dev/video50
driver : ARM-camera-isp
device : juno R2
bus_info : platform:isp-v4l2-0
version : 331639
strcmp cap.driver: ARM-camera-isp
[API:aml_v4l2src_connect:304]Exit, ARM-camera-isp
devname : /dev/video50
driver : ARM-camera-isp
device : juno R2
bus_info : platform:isp-v4l2-0
version : 331639
error tvin-port use -1 
[API:aml_v4l2src_streamon:373]Enter
[API:aml_v4l2src_streamon:376]Exit
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:vsrc: Device '/dev/video50' cannot capture in the specified format
Additional debug info:
../sys/v4l2/gstv4l2object.c(4162): gst_v4l2_object_set_format_full (): /GstPipeline:pipeline0/GstV4l2Src:vsrc:
Tried to capture in RGB3, but device returned format META
Execution ended after 0:00:00.011765190
Setting pipeline to NULL ...
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:vsrc: Internal data stream error.
Additional debug info:
../libs/gst/base/gstbasesrc.c(3127): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:vsrc:
streaming stopped, reason not-negotiated (-4)
[API:aml_v4l2src_streamoff:381]Enter
[API:aml_v4l2src_streamoff:384]Exit
[API:aml_v4l2src_disconnect:363]Enter
change_divide_framerate, L69, dev , change_frame 0
[API:aml_v4l2src_disconnect:368]Exit
Freeing pipeline ...

Am i missing some steps in configuration?

Alex999 · January 30, 2024, 2:18pm

Same thing with python

$ python3 vim3.py 
[ WARN:0@0.131] global cap_v4l.cpp:997 open VIDEOIO(V4L2:/dev/video50): can't open camera by index

although, guvcview does record 1fps video from /dev/video50

Jacobe · January 31, 2024, 2:19am

Oops , there is something wrong in our docs and now we have update it. You can follow this page to check OS08A10 mipi camera.
If you have update 5.15 kernel already, you can also go strightly here.

Alex999 · February 1, 2024, 11:09am

Ok. I managed to test it with opencv

if __name__ == '__main__':
    capture = f"v4l2src device=/dev/video50 ! video/x-raw,format=NV21,width=1920,height=1080,framerate=60/1 ! appsink"
    cap = cv2.VideoCapture(capture, cv2.CAP_GSTREAMER)
    fps_counter = FPSCounter()
    if cap.isOpened():
        while True:
            ret, frame = cap.read()
            if ret:
                if frame is None:
                    print("frame is None")        
                else:
                    fps_counter.update()
                    frameCopy = frame.copy()
                    #cv2.imwrite("frame.jpg", frameCopy)
                    print(f"fps: {fps_counter.get_fps()}")
            else:
                print("RET is None")        
                    
        cap.release()
    else:
        print("Error opening cap")

The best FPS i get from it is 30:

$ python3 vim3.py
[API:aml_v4l2src_connect:271]Enter, devname : /dev/video50
driver : ARM-camera-isp
device : juno R2
bus_info : platform:isp-v4l2-0
version : 331639
strcmp cap.driver: ARM-camera-isp
[API:aml_v4l2src_connect:304]Exit, ARM-camera-isp
devname : /dev/video50
driver : ARM-camera-isp
device : juno R2
bus_info : platform:isp-v4l2-0
version : 331639
error tvin-port use -1 
[API:aml_v4l2src_streamon:373]Enter
[API:aml_v4l2src_streamon:376]Exit
[ WARN:0@0.317] global cap_gstreamer.cpp:1777 open OpenCV | GStreamer warning: Cannot query video position: status=0, value=-1, duration=-1
fps: 29.783823741398006
fps: 30.004964696287924
fps: 30.004964696287924
fps: 30.001745325531825
fps: 30.010568048717488
fps: 29.929470328158494
...

And this is without videoconvert, so i get just grayscale. How can i get RGB 1920x1080@60fps from this camera on VIM3 Pro?

Alex999 · February 2, 2024, 11:21am

Any thoughts for this one? I need at least RGB 1920x1080@30fps to go further. Sensor specs states much more, though…

Jacobe · February 4, 2024, 3:07am

We are working on fixing the same and will get back to you once the fix
is ready

Alex999 · February 4, 2024, 5:48pm

Do you have expected time of arrival for this fix? Planned maybe?

ivan.li · February 5, 2024, 1:57am

We will give you a reply after the Spring Festival，about half a month later.

Alex999 · March 1, 2024, 10:14am

Hey, Guys! Any news with this? My testing phase is still on pause, waiting for fps issue fix.

ivan.li · March 4, 2024, 9:44am

@Alex999
Try this firmware :
https://dl.khadas.com/products/vim3/firmware/ubuntu/emmc/vim3-ubuntu-22.04-gnome-linux-5.15-fenix-1.6.3-240112-emmc.img.xz
Reference documents:
http://docs.khadas.com/products/sbc/vim3/add-ons/vim3-mipi-camera

Alex999 · March 4, 2024, 11:40am

i stopped my current work and tested vim3-ubuntu-22.04-gnome-linux-5.15-fenix-1.6.3-240112-emmc.img.xz as you suggested, although it is the same FW i was using from the beginning. I hoped that you made changes in the code, but left 1.6.3-240112, but nope: it is the same firmware.
The issue completely persists: 30fps max without conversion, 14fps max with conversion for RGB 1920x1080. I’ve used the same testing code as before.

So. You said that your team is working on this issue. Is it still correct?
What are my next steps?

ivan.li · March 5, 2024, 2:39am

@Alex999
What if we directly output RGB format

$ gst-launch-1.0 v4l2src device=/dev/video50 io-mode=mmap num-buffers=300 ! video/x-raw,format=RGB,width=1920,height=1080,framerate=30/1 ! amlvenc bitrate=8000 ! h264parse ! qtmux ! filesink location=test_50.mp4