How to Use NPU via OpenCV | VIM3

The good news here is that VIM3 already supports running models directly on the NPU via OpenCV.
Let’s take a quick look at how this can be achieved :crazy_face:

Just support VIM3 !!!

Preparatory work

1. Please make sure your system version number is greater than 200530.

$ cat /etc/fenix-release | grep IMAGE_VERSION
IMAGE_VERSION=1.0.11-220526

2. Follow this page to upgrade your system.

3. Check package version.

$ dpkg -l | grep aml-npu
ii  aml-npu                               6.4.8.7.1.1.1-1                                              arm64        Amlogic NPU libraries.

Only 6.4.8.7.1.1.1-1 is supported, 6.4.8.7.1.1.1 is not supported

4. Install dependencies

$ sudo apt install python3 python3-dev python3-numpy cmake
$ sudo apt-get install libgtk2.0-dev

Compile OpenCV

1. Clone

$ mkdir ~/opencv_npu && cd ~/opencv_npu
$ git clone https://github.com/opencv/opencv

2. Build

$ ls
opencv
$ pwd
/home/khadas/opencv_npu
$ cmake -B opencv-build \
     -D CMAKE_BUILD_TYPE=RELEASE \
     -D CMAKE_INSTALL_PREFIX=opencv-install \
     -D BUILD_opencv_python3=ON \
     -D BUILD_opencv_gapi=OFF \ -D VIVANTE_SDK_DIR=/usr \
     -D WITH_TIMVX=ON opencv
$ cmake --build opencv-build --target install -j 8

Run Demo

1. Clone test demo

$ export VIVANTE_SDK_DIR=/usr
$ cd /home/khadas/opencv_npu
$ git clone https://github.com/khadas/OpenCV_NPU_Demo.git

2. Compile and run C/C++ demo

$ cd ~/opencv_npu/OpenCV_NPU_Demo/cpp_demo
$ mkdir build && cd build
$ cmake .. -DCMAKE_PREFIX_PATH=~/opencv_npu/opencv-install
$ make

Plugin a USB camera,

$ cd ~/opencv_npu/OpenCV_NPU_Demo/cpp_demo/build
$ ./detect-camera 1 ../../model/yunet_int8.onnx # My cam is /dev/video1, so there is 1

You can chooise different demo:

$ ./detect-camera --help  #Camera detect demo
Usage: ./detect-camera <camera index>
$ ./detect-image --help  #Image detect demo
Usage: ./detect-image <image_file_name> <net_file_name> [<visualization_flag> default false]

3. Run python demo

$ export PYTHONPATH=/home/khadas/opencv_npu/opencv-build/python_loader:$PYTHONPATH
$ cd ~/opencv_npu/OpenCV_NPU_Demo/python_demo
$ python python/detect.py --image=../image/face.gpeg --model=../model/yunet_int8.onnx

image1

About Swap Space

If you run out of memory when compiling Opencv, you can refer to my steps to increase swap space for compilation.

$ sudo dd if=/dev/zero of=/swapfile bs=1024 count=4097152
$ sudo chmod 600 /swapfile
$ sudo mkswap /swapfile
$ sudo swapon /swapfile

About the model

  1. If you want to run the model completely on the NPU, you need to design the int8 model, just like the model in the instance.
  2. You can also directly run the model without redesign. This will be a hybrid operation, partly running on the CPU, partly running on the NPU.
  3. You can find some quantize demo in https://github.com/opencv/opencv_zoo.
2 Likes

我在编译opencv时,一直报错:[modules/dnn/CmakeFiles/opencv_dnn.dir/build.make:2173:lib/libopencv_dnn.sp.4.6.0] 错误1,有遇到过吗

@zxl1109007773 这一行我看不出问题,请贴出具体报错的地方。另外

  1. 确保你的系统是最新的。
  2. 确保你的操作是正确的。

我在编译opencv时,第一步是没问题的,第二部出错了,错误如下:

@zxl1109007773 请确保你使用了最新的系统

$ cat /etc/fenix-release

将结果贴在这里

系统是最新的

@zxl1109007773 我找时间验证下

VIM3用fenix编译5.18内核的版本试了,跑opencv_zoo,修改相关config配置和模型采用int8,开启NPU,跑benchmark中的例子,效果和cpu差不多,是不是5.18内核不支持呢?

@algoideas 你运行我上面的demo了么?

试了例子,可以成功,但是opencv benckmark,开启NPU没有预期的好(耗时大部分与CPU推理差不多,但发现开启NPU时CPU的确占用会低很多),有点纳闷,不清楚什么原因

试了例子没问题,能否同步帮忙确认下,跑opencv benchmark复现不出以下文章的结果:https://mp.weixin.qq.com/s/7AVB8ly89ElPvGEduLqSDw

为什么我在编译opencv时一直出错,你碰到过吗

@algoideas OpenCV这边,不是所有算子都支持跑在NPU上,不支持的算子就会放到CPU上面跑,以我们提供的demo为例,是全算子可以跑在NPU上的。

@zxl1109007773 我这边昨晚测试了,是正常的,不会报错

这个教程需不需要先编译安装TIM-VX,然后再编译安装opencv

@zxl1109007773 不用,编译opencv的过程中会下载,但是你要确保,TIM-VX的包下载成功了,在cmake这一步的log可以看出来

编译opencv 4.6.0 完全没问题的

我看到opencv官方教程就是还要替换galcore.ko这个文件,需要替换吗

你直接参考的opencv官方提供的教程吗,所有步骤都一样的吗

参考官方提供的教程即可,如果需要调用opencv的imshow等函数,需要先额外装些库,如gtk3