NPU camera demo noise output

[osos55]

I am trying to run the camera demos, I installed latest Ubuntu 20 v0.9 20200530 and did apt upgrade & do-fenix-full-upgrade.

Now I can run inception & image but when I run the camera detect demo, I get noise as in the picture.

I tried in both x11 & fb modes.

When I use the guvcview app I can see the camera output and can play with settings

Sometimes I change the settings and try again then I notice that the output change as I get corrupted still correct images with noisy sections of the image

I use Logitech C505 USB camera and Microsoft LifeCam Studio FHD Webcam

I also tried tengine with no success with no video output at all.

I generate the yolov3_UINT8.tmfile as with wiki and I also tried an already generated model from tengine models zoo but only image recognition works. Video fails. I attached a pic too

Please help as I am out of options to test

[Frank]

@osos55 What resolution is your camera ? you can use -w and -h to setup the resolution.

 ./detect_demo_x11_usb --help
./detect_demo_x11_usb [-d device] [-w width] [-h height] [-m model type] [-H]

[osos55]

Thank you for the info. It is 720p c505 Logitech Cam connected to usb3 port. I noticed that the x11 accept the offered width -w 1280 -h 720 where the fb version always init at 1920x1080 but both fail unfortunately. I tried with the Microsoft FHD 1920x1080 lifecam USB camera and it is the same. I attached a picture for fb.

Another screen for x11 with w h specified. But Here I noticed something:

• I set the camera on guvcview with correct width and height and put my hand over it.
• I remove my hand and run the demo again and it revealed a still img of parts of that last frame in buffer with my hand.
• I noticed whenever I run the demo again the output is always black (buffer cleared?)
  
  
  
  What do you think about the “output_format not support imperfect” msg? I also should mention that I use guvcview to set the resolution before running these scripts

[Frank]

@osos55 I have try it . itwrok fine with 1280x720

Maybe this is a problem with your camera. I think you can yse guvcview to check .

[osos55]

Thank you for taking the time to do the test

Both my cameras actually work in guvcview but not the demo. I also test my camera on simple opencv code yesterday after building opencv4.1

• I use the Logitech c505 720p which was tested according to the wiki. I will buy another camera today. What camera did you use?
• what exact Ubuntu image and steps you follow. I followed this NPU Demo and source code, with latest Ubuntu 20 v0.9 20200530
• did you download any v4l or gstreamer libraries?

Regards

[Frank]

@osos55 Just the least relese firmware,don’t need anymore . We use logitech camera

[osos55]

I tried that. New Ubuntu 20 desktop image 4.9.
I just cloned the binaries.

• sudo ./INSTALL
  It throw an error regarding libge2d not found
• I do sudo apt-get update
• sudo apt full-upgrade
  It ask me to update or keep old /etc/system d/resolved.conf
  I tried both
• sudo do-fenix-full-upgrade && sync
• reboot

Now the libge2d missing error disappeared

I try to run the demo and it has the same noise
Guvcview works fine. I even rolled out some old git binaries

• should I buy the exact c920 pro camera you use? I use c505 but the one tested in the wiki is c525

• I also wanna give Ubuntu 18 more tries as it might be more stable. Could you point me to the latest working git binary hash with Ubuntu 18 & the libge2d.so libion.so

• should I try more with tengine? Since the camera works in opencv and guvcview it might be a software or missing libraries issue. What are your thoughts

[Frank]

@osos55

1. We will buy a c505 camera to test
2. 18.04 has been suspended for maintenance, so this support is not provided

[osos55]

thanks, waiting for your feedback.
I also built the NPU app and its libraries and copyed them to /usr/lib folder and run the video yolo app but it fails.

• I ran the command : $v4l2_test -c 1 -p 0 -F 0 -f 0 -D 0 -R 1 -r 2 -d 2 -N 1000 -n 800 -w 0 -e 1 -b /dev/fb0 -v /dev/video0

and THE SAME NOISE Screen appeared. does this means that the app v4l call settings need to be tweaked?

• Is there a way to find the v4l setting for my camera since guvcview and other apps can use it. Please advise

• $cat /sys/class/video4linux/video0/name
  Microsoft® LifeCam Studio™

• I ran the v4l2-ctl --all and below is the output:
  Driver Info:
  Driver name : uvcvideo
  Card type : Microsoft® LifeCam Studio™
  Bus info : usb-xhci-hcd.0.auto-1.1
  Driver version : 4.9.241
  Capabilities : 0x84200001
  Video Capture
  Streaming
  Extended Pix Format
  Device Capabilities
  Device Caps : 0x04200001
  Video Capture
  Streaming
  Extended Pix Format
  Priority: 2
  Video input : 0 (Camera 1: ok)
  Format Video Capture:
  Width/Height : 640/480
  Pixel Format : ‘YUYV’ (YUYV 4:2:2)
  Field : None
  Bytes per Line : 1280
  Size Image : 3670197192
  Colorspace : sRGB
  Transfer Function : Default (maps to sRGB)
  YCbCr/HSV Encoding: Default (maps to ITU-R 601)
  Quantization : Default (maps to Limited Range)
  Flags :
  Crop Capability Video Capture:
  Bounds : Left 0, Top 0, Width 640, Height 480
  Default : Left 0, Top 0, Width 640, Height 480
  Pixel Aspect: 1/1
  Selection Video Capture: crop_default, Left 0, Top 0, Width 640, Height 480, Flags:
  Selection Video Capture: crop_bounds, Left 0, Top 0, Width 640, Height 480, Flags:
  Streaming Parameters Video Capture:
  Capabilities : timeperframe
  Frames per second: 5.000 (5/1)
  Read buffers : 0
  brightness 0x00980900 (int) : min=30 max=255 step=1 default=133 value=133
  contrast 0x00980901 (int) : min=0 max=10 step=1 default=5 value=6
  saturation 0x00980902 (int) : min=0 max=200 step=1 default=103 value=79
  white_balance_temperature_auto 0x0098090c (bool) : default=1 value=1
  power_line_frequency 0x00980918 (menu) : min=0 max=2 default=2 value=2
  0: Disabled
  1: 50 Hz
  2: 60 Hz
  white_balance_temperature 0x0098091a (int) : min=2500 max=10000 step=1 default=4500 value=4500 flags=inactive
  sharpness 0x0098091b (int) : min=0 max=50 step=1 default=25 value=22
  backlight_compensation 0x0098091c (int) : min=0 max=10 step=1 default=0 value=0
  exposure_auto 0x009a0901 (menu) : min=0 max=3 default=3 value=3
  1: Manual Mode
  3: Aperture Priority Mode
  exposure_absolute 0x009a0902 (int) : min=1 max=10000 step=1 default=156 value=156 flags=inactive
  pan_absolute 0x009a0908 (int) : min=-529200 max=529200 step=3600 default=0 value=0
  tilt_absolute 0x009a0909 (int) : min=-432000 max=432000 step=3600 default=0 value=0
  focus_absolute 0x009a090a (int) : min=0 max=40 step=1 default=0 value=0 flags=inactive
  focus_auto 0x009a090c (bool) : default=1 value=1
  zoom_absolute 0x009a090d (int) : min=0 max=317 step=1 default=0 value=0

[numbqq]

@osos55

• I ran the command : $v4l2_test -c 1 -p 0 -F 0 -f 0 -D 0 -R 1 -r 2 -d 2 -N 1000 -n 800 -w 0 -e 1 -b /dev/fb0 -v /dev/video0
  and THE SAME NOISE Screen appeared. does this means that the app v4l call settings need to be tweaked?

Why you use this command? This command is only for our MIPI camera,not for UVC.

[osos55]

Well, I ran out of options and I am a noob
Actually I noticed that yolo_demo_fb_usb/main.cpp
Use v4l , e.g:

// Start streaming
int buffer_type =V4L2_BUF_TYPE_VIDEO_CAPTURE;
ret = ioctl(capture_fd, VIDIOC_STREAMON, &buffer_type);

Should the command v4l2_test above be changed to test or you guys don’t use v4l at all

Please teach me how you access the USB Cam in the demo

[numbqq]

@osos55

Actually I noticed that yolo_demo_fb_usb/main.cpp

You already find the source code to access the USB camera what else you need?

It’s all standard V4L2 framework, maybe you need to search for the V4L2 usage.

[Frank]

@osos55 We have fixup this error , you can get the leaset code.

khadas / aml_npu_app

khadas / aml_npu_demo_binaries

[osos55]

wow, thank you Frank
that is awesome, it is working now

for different resolutions and both X11, FB

thank you for the great support, looking at new changes, there is no way I could do it myself