NPU Usage with OpenCV DNN

VIM3
1

Which Khadas SBC do you use?

VIM3 Pro

Which system do you use? Android, Ubuntu, OOWOW or others?

Ubuntu 20.04

Hi @Frank @numbqq ,

I ran my Yolov5 detection model with the OpenCv DNN NPU module (OpenCV DNN Usage [Khadas Docs]). The results are correct. However, the fps value I get when I use NPU is almost the same value when I use CPU. Shouldn’t it be faster when I run it with NPU? What is the reason for slow running on NPU? I used the yolov5n.onnx model directly without the .nb and .so conversions. Is it slow because I don’t quantify the Onnx model (i.e. model conversion)?

2

@ahmet_karazor When OpenCV DNN is running, only operators that support conversion will be converted to run on the NPU, and operators that do not support conversion will run on the CPU. This is officially maintained by OpenCV, and you can consult OpenCV for specific implementation technical details.

3

thanks your answer @Frank,
What does “only operators that support conversion will be converted to run on the NPU” mean? what are operators that support conversion.

4

@ahmet_karazor We will ask the opencv side to issue a document or description.

5

@Frank,
thanks your interest

6

Hi @Frank,

Is there any progress?