How to Use NPU via OpenCV | VIM3

The good news here is that VIM3 already supports running models directly on the NPU via OpenCV.
Let’s take a quick look at how this can be achieved :crazy_face:

Just support VIM3 !!!

Preparatory work

1. Please make sure your system version number is greater than 200530.

$ cat /etc/fenix-release | grep IMAGE_VERSION
IMAGE_VERSION=1.0.11-220526

2. Follow this page to upgrade your system.

3. Check package version.

$ dpkg -l | grep aml-npu
ii  aml-npu                               6.4.8.7.1.1.1-1                                              arm64        Amlogic NPU libraries.

Only 6.4.8.7.1.1.1-1 is supported, 6.4.8.7.1.1.1 is not supported

4. Install dependencies

$ sudo apt install python3 python3-dev python3-numpy cmake
$ sudo apt-get install libgtk2.0-dev

Compile OpenCV

1. Clone

$ mkdir ~/opencv_npu && cd ~/opencv_npu
$ git clone https://github.com/opencv/opencv

2. Build

$ ls
opencv
$ pwd
/home/khadas/opencv_npu
$ cmake -B opencv-build \
     -D CMAKE_BUILD_TYPE=RELEASE \
     -D CMAKE_INSTALL_PREFIX=opencv-install \
     -D BUILD_opencv_python3=ON \
     -D BUILD_opencv_gapi=OFF \ -D VIVANTE_SDK_DIR=/usr \
     -D WITH_TIMVX=ON opencv
$ cmake --build opencv-build --target install -j 8

Run Demo

1. Clone test demo

$ export VIVANTE_SDK_DIR=/usr
$ cd /home/khadas/opencv_npu
$ git clone https://github.com/khadas/OpenCV_NPU_Demo.git

2. Compile and run C/C++ demo

$ cd ~/opencv_npu/OpenCV_NPU_Demo/cpp_demo
$ mkdir build && cd build
$ cmake .. -DCMAKE_PREFIX_PATH=~/opencv_npu/opencv-install
$ make

Plugin a USB camera,

$ cd ~/opencv_npu/OpenCV_NPU_Demo/cpp_demo/build
$ ./detect-camera 1 ../../model/yunet_int8.onnx # My cam is /dev/video1, so there is 1

You can chooise different demo:

$ ./detect-camera --help  #Camera detect demo
Usage: ./detect-camera <camera index>
$ ./detect-image --help  #Image detect demo
Usage: ./detect-image <image_file_name> <net_file_name> [<visualization_flag> default false]

3. Run python demo

$ export PYTHONPATH=/home/khadas/opencv_npu/opencv-build/python_loader:$PYTHONPATH
$ cd ~/opencv_npu/OpenCV_NPU_Demo/python_demo
$ python python/detect.py --image=../image/face.gpeg --model=../model/yunet_int8.onnx

image1

About Swap Space

If you run out of memory when compiling Opencv, you can refer to my steps to increase swap space for compilation.

$ sudo dd if=/dev/zero of=/swapfile bs=1024 count=4097152
$ sudo chmod 600 /swapfile
$ sudo mkswap /swapfile
$ sudo swapon /swapfile

About the model

  1. If you want to run the model completely on the NPU, you need to design the int8 model, just like the model in the instance.
  2. You can also directly run the model without redesign. This will be a hybrid operation, partly running on the CPU, partly running on the NPU.
  3. You can find some quantize demo in https://github.com/opencv/opencv_zoo.
2 Likes