Infrared camera setup in Edge2

Edge2
1

Which system do you use? Android, Ubuntu, OOWOW or others?

I use Ubuntu.

Which version of system do you use? Khadas official images, self built images, or others?

Ubuntu 20.04

Please describe your issue below:

I want to use an infrared camera setup for night recordings. I have explored the available and compatible (with Edge2) cameras in the khadas store. Unfortunately, non provides this capability out of the box.

I have some raspberry pi cameras but then the issues come to i) connecting them (they are csi 22pin) and ii) making them work on the Edge2.

I have seen that Edge2 supports the OS08A10 8MP HDR camera (https://www.khadas.com/product-page/os08a10-8mp-camera), ideally allowing for the IR-CUT to be disabled. My question is, can I connect some IR LEDs to the Edge2 GPIO, and disable the IR-CUT on the OS08A10 to have a night mode camera setup?

Have someone else work on something similar, or can you give me some pointers?

Help much appreciated.

Post a console log of your issue below:

2

Hi @omarc, welcome to the forum.

In principle, yes, your idea should work. By disabling the IR Cut it should turn the OS08A10 into a infrared night camera.

Let me try this and let you know.

Regards.

3

Thanks, @Electr1

I would really appreciate it if you could test it.

Best regards,

4

@omarc apologies, the Edge2 software doesn’t currently doesn’t support any ways to control the IR-CUT at the moment, so it isn’t possible to disable the IR filter.

5

Thanks, @Electr1, for your update. That is unfortunate.

I also have a VIM4 I can use to disable the IR-CUT if the issue is only with the software. Do you think this might work? Or does the software feature’s unavailability extend to all the compatible Khadas boards?

Best regards

6

@omarc it hasn’t been tested on Edge2, but a solution might be available for it.

8

Hello @omarc
In Edge2 the IR-CUT gpio was disabled in dts by default, I have test OS08A10 in python by opencv and it can capture the IR LED

import cv2

cap = cv2.VideoCapture(51)

if not cap.isOpened():
    print("can't open")
    exit()

while True:
    ret, frame = cap.read()

    if not ret:
        print("cant't read")
        break

    cv2.imshow('Night Vision Preview', frame)

    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

cap.release()
cv2.destroyAllWindows()

You can use by this file, and test whether it would be effective for your demand.

Regrads.

9

Hey @Jacobe ,

thanks for trying the setup.

If I understood correctly, is the IR-CUT of the OS08A10 camera disabled by default? or is the IR-CUT disabled (somehow) in the GPIO?

I don’t have the OS08A10 camera yet, so I wanted to ask you before buying it.

I tried setting up IR LEDs with a Khadas IMX415 Camera I have, but I could not make it work. The camera sees almost nothing in the dark even with the IR LEDS on. I can see a lot of noise and a little bit of light patterns; maybe the IMX415 has an IR cut by default.

Best regards,

10

Hello @omarc

In dts file, the IR-CUT disabled in the GPIO by default. So if one day you need the function you can open the node manully. We only set the reset-gpio.

IMX415 don’t have the IR-CUT :wink:

Regards.

11

Thank you, @Jacobe
I will try this setup when my OS08A10 camera arrives.

Following your information that the IMX415 camera does not have an IR filter, I tested with the Edge2. I updated the Linux kernel (as suggested in the documentation) and recorded some videos using gstreamer or v4l. However, when I record in dark conditions (and using a pair of 850nm IR LEDS for illumination), the camera records noise (as seen in the picture). In light conditions, the IMX415 works great. Do you know what could be the issue?

Screenshot from 2024-05-12 23-22-59
Screenshot from 2024-05-12 23-22-591572Ă—931 224 KB

12

Hello @omarc

Because of IMX415 doesn’t support night recordings, so we test it in light conditions before. Maybe you can give the specific steps of how to capture this picture, I will try to reproduce the issus :wink:

13

Hello @Jacobe

I have an Edge2 and the IMX415 camera connected to the CAM1 port. To record the videos, I use gstreamer as such:

gst-launch-1.0 v4l2src device=/dev/video42 io-mode=dmabuf num-buffers=7000 ! mpph264enc ! h264parse ! qtmux ! filesink location=/tmp/video42.mp4

I have tried these light setups:

  • I start recording with light, and after a couple of seconds, I turn off the lights completely and turn the IR-LEDS.
  • I start the recording without lights and the IR-LEDS, which is already on.
  • I start the recording without lights and the IR-LEDS; after a couple of seconds, I turn on the lights.
14

@omarc the IMX415 does indeed have an IR filter on top of the lens, it doesn’t have the ability to manipulate it like the OS08A10.

It however makes up for the low light performance (low light != No light :wink:) with extremely high sensor gain. This is a feature of the sony starvis sensor this is why the camera will exaggerate in situation such as yours and produce a high amount of noise on the camera screen.

Regards

15

@Electr1 Yes, looking at the low-light and no-light images of the IMX415 camera that makes sense.

Best regards

1 Like
16

Hello @Jacobe

My OS08A10 camera finally arrived. I connected the camera to the CAM3 port and tested the OpenCV script for video showing and recording. Out-of-the-box, the camera seems not to have the IR filter enabled, which is what I was looking for. Thanks.

I tested the camera with two very (dimmed) 850nm IR-LEDS, and the results are promising! I will look for a better IR illuminator. I need to check the OS08A10 sensitivity graph to see which kind of IR-LEDS are better for the camera, 850nm or 940nm.

Best regards,

1 Like
17

Hi @Jacobe @Electr1 ,

related to the IR capabilities of the OS08A10 camera. Do you know if it is possible to change the camera exposure time or the gain?

I have tried using v4l2-ctl, but it seems the control parameter for the camera driver is not present. Do you have any tips regarding how to change these parameters?

I also tried via OpenCV, but the change seems not to have an effect.

Best regards,

1 Like
18

I tested the camera with two very (dimmed) 850nm IR-LEDS, and the results are promising! I will look for a better IR illuminator. I need to check the OS08A10 sensitivity graph to see which kind of IR-LEDS are better for the camera, 850nm or 940nm.

Well, now I don’t have the professional IR LED, so last time I test with my mobile phone which have the remove IR function. I used the cluttervideosink to preview, in the software windows I can see the IR light send by my phone.

Do you know if it is possible to change the camera exposure time or the gain?

First you need to check the media information.

v4l2-ctl -d /dev/v4l-subdev2 -l
User Controls

                       exposure 0x00980911 (int)    : min=4 max=2243 step=1 default=2242 value=1351
                horizontal_flip 0x00980914 (bool)   : default=0 value=0
                  vertical_flip 0x00980915 (bool)   : default=0 value=0

Image Source Controls

              vertical_blanking 0x009e0901 (int)    : min=59 max=30575 step=1 default=59 value=59
            horizontal_blanking 0x009e0902 (int)    : min=536 max=536 step=1 default=536 value=536 flags=read-only
                  analogue_gain 0x009e0903 (int)    : min=0 max=240 step=1 default=0 value=32

Image Processing Controls

                 link_frequency 0x009f0901 (intmenu): min=0 max=3 default=0 value=2 (743000000 0x2c4947c0)
                     pixel_rate 0x009f0902 (int64)  : min=0 max=712800000 step=1 default=297200000 value=297200000 flags=read-only

You can see the settable exposure range of exposure, and then set this node.

$ v4l2-ctl -d /dev/v4l-subdev2 --set-ctrl exposure=2000
$ v4l2-ctl -d /dev/v4l-subdev2 --set-ctrl exposure=200
1 Like
19

Hello @Jacobe ,

Thanks for the information. I can now see and change the parameters in the /v4l2-subdev device.

I have a followup question. When I change the exposure or gain using v4l2-ctl and I run my python code, the exposure and gain values are reset to the previous values.

I notice that this issue happens when I use the opencv cv2.set functions and when I release the video capture. Is this behavior intended? Should I do an additional step to fix the values? Maybe there is an auto mode or the hdr that I need to disable.

Thank you in advance.

20

Hello @omarc

You’re right. It means you need to set the value you want in the code with a init function.

21

Thank you @Jacobe
Do you mean an init system file or an init in the python code?

Another related question:

I am trying to get the most out of the OS08A10 camera configuration. I read in the camera datasheet that it has two binning modes to provide lower-resolution outputs while keeping the same field of view. I have checked the controls in both /dev/video51 and /dev/subdev6 devices but cannot find how to enable the mono binning. I would like to use the mono binning to have a grayscale image (as I mainly use the camera setup under IR illumination).

Is this binning mode exposed? Maybe it is set under the hood when a specific video format is selected?

Thanks in advance