基于NPU 6.4.8.7建立YOLOv3-Tiny模型

VIM3
1

前言：基于NPU 6.4.6.2的模型转换参考零基础导入自己YOLOv3或YOLOv3-Tiny模型步骤基本一致没有什么变化，但是基于NPU 6.4.8.7版本而言在转换模型上有些许改变，转换工具acuity-toolkit的更新和armeabi-v7a跟arm64-v8a ABI的使用方法更新，相比旧的转换工具来说新的acuity-toolkit更智能化了需要转换自己的模型时需要修改的地方更精简了，但是还是有相同的部分，下面我会将差异点及使用方法描述给你。此文档只适用VIM3/VIM3L Android平台。

一，转换模型

模型转换流程图：

image
image1035×321 78.3 KB

1，导入模型

当前模型转换过程都是在 acuity-toolkit 目录下进行，将训练好的darknet模型放入model/目录中

cd {workspace}/khadas_android_npu_librar/acuity-toolkit/demo
cp {workspace}/yolov3-khadas_ai_tiny.cfg_train model/
cp {workspace}/yolov3-khadas_ai_tiny_last.weights model/
cp {workspace}/test.jpg data/

修改0_import_model.sh脚本：

diff --git a/acuity-toolkit/demo/0_import_model.sh b/acuity-toolkit/demo/0_import_model.sh
index 8d3812e..277815b 100644
--- a/acuity-toolkit/demo/0_import_model.sh
+++ b/acuity-toolkit/demo/0_import_model.sh
@@ -1,6 +1,6 @@
 #!/bin/bash
 
-NAME=mobilenet_tf
+NAME=yolotiny
 ACUITY_PATH=../bin/
 
 pegasus=${ACUITY_PATH}pegasus
@@ -9,19 +9,26 @@ if [ ! -e "$pegasus" ]; then
 fi
 
 #Tensorflow
-$pegasus import tensorflow  \
-               --model ./model/mobilenet_v1.pb \
-               --inputs input \
-               --outputs MobilenetV1/Predictions/Reshape_1 \
-               --input-size-list '224,224,3' \
-               --output-data ${NAME}.data \
-               --output-model ${NAME}.json
+#$pegasus import tensorflow  \
+#              --model ./model/mobilenet_v1.pb \
+#              --inputs input \
+#              --outputs MobilenetV1/Predictions/Reshape_1 \
+#              --input-size-list '224,224,3' \
+#              --output-data ${NAME}.data \
+#              --output-model ${NAME}.json
+
+#Darknet
+$pegasus import darknet\
+    --model  ./model/yolov3-khadas_ai_tiny.cfg_train \
+    --weights  ./model/yolov3-khadas_ai_tiny_last.weights \
+    --output-model ${NAME}.json \
+    --output-data ${NAME}.data \

#generate inpumeta  --source-file dataset.txt
 $pegasus generate inputmeta \
        --model ${NAME}.json \
        --input-meta-output ${NAME}_inputmeta.yml \
-       --channel-mean-value "128 128 128 0.0078125"  \
+       --channel-mean-value "0 0 0 0.0039215"  \
        --source-file dataset.txt
 

diff --git a/acuity-toolkit/demo/data/space_shuttle_224.jpg b/acuity-toolkit/demo/data/space_shuttle_224.jpg
deleted file mode 100644
index 97bbb2c..0000000
Binary files a/acuity-toolkit/demo/data/space_shuttle_224.jpg and /dev/null differ
diff --git a/acuity-toolkit/demo/data/validation_tf.txt b/acuity-toolkit/demo/data/validation_tf.txt
index 5b8dd9d..c2b8f96 100644
--- a/acuity-toolkit/demo/data/validation_tf.txt
+++ b/acuity-toolkit/demo/data/validation_tf.txt
@@ -1 +1 @@
-./space_shuttle_224.jpg
+./test.jpg
diff --git a/acuity-toolkit/demo/dataset.txt b/acuity-toolkit/demo/dataset.txt
index d2c7ae0..8b83182 100644
--- a/acuity-toolkit/demo/dataset.txt
+++ b/acuity-toolkit/demo/dataset.txt
@@ -1 +1 @@
-./data/space_shuttle_224.jpg
+./data/test.jpg

执行对应脚本：

bash 0_import_model.sh

image
image1035×427 108 KB

2，对模型进行量化

diff --git a/acuity-toolkit/demo/1_quantize_model.sh b/acuity-toolkit/demo/1_quantize_model.sh
index c3165ec..b592465 100644
--- a/acuity-toolkit/demo/1_quantize_model.sh
+++ b/acuity-toolkit/demo/1_quantize_model.sh
@@ -1,6 +1,6 @@
 #!/bin/bash
 
-NAME=mobilenet_tf
+NAME=yolotiny
 ACUITY_PATH=../bin/
 
 pegasus=${ACUITY_PATH}pegasus
@@ -13,7 +13,7 @@ fi
 # --quantizer perchannel_symmetric_affine --qtype int8(int16, note only T3(0xBE) can support perchannel quantize)
 $pegasus  quantize \
        --quantizer dynamic_fixed_point \
-       --qtype int16 \
+       --qtype int8 \
        --rebuild \
        --with-input-meta  ${NAME}_inputmeta.yml \
        --model  ${NAME}.json \

执行对应脚本：

bash 1_quantize_model.sh

image
image1127×211 29.9 KB

3，生成 case 代码

diff --git a/acuity-toolkit/demo/2_export_case_code.sh b/acuity-toolkit/demo/2_export_case_code.sh
index 80780f6..c86f8dc 100644
--- a/acuity-toolkit/demo/2_export_case_code.sh
+++ b/acuity-toolkit/demo/2_export_case_code.sh
@@ -1,6 +1,6 @@
 #!/bin/bash
 
-NAME=mobilenet_tf
+NAME=yolotiny
 ACUITY_PATH=../bin/
 
 pegasus=$ACUITY_PATH/pegasus
@@ -14,7 +14,7 @@ $pegasus export ovxlib\
     --model-quantize ${NAME}.quantize \
     --with-input-meta ${NAME}_inputmeta.yml \
     --dtype quantized \
-    --optimize VIPNANOQI_PID0X1E  \
+    --optimize VIPNANOQI_PID0X88  \
     --viv-sdk ${ACUITY_PATH}vcmdtools \
     --pack-nbg-unify

注意：VIM3对应VIPNANOQI_PID0X88 ，VIM3L对应VIPNANOQI_PID0X99

执行对应脚本：

bash 2_export_case_code.sh

image
image1064×375 50.2 KB

当然只要运行脚本没报错，后续你可以按照上面修改完成后，一次性执行脚本：

bash 0_import_model.sh && bash 1_quantize_model.sh  && bash 2_export_case_code.sh

image
image825×748 104 KB

最终会产生一个yolotiny_nbg_unify目录：

image
image1035×526 193 KB

image
image1035×550 192 KB

二，导入到VIM3 android平台的demo app运行

1， 安装ndk 编译环境

wget https://dl.google.com/android/repository/android-ndk-r17-linux-x86_64.zip
unzip android-ndk-r17-linux-x86_64.zip
vim ~/.bashrc
##添加下面两行代码到文件末尾
##export NDKROOT=/path/to/android-ndk-r17
##export PATH=$NDKROOT:$PATH

如图所示：

image
image1223×130 17.4 KB

然后在你要使用的ssh上执行此命令：

source ~/.bashrc

2， 编译相关so库

(1) 下载khadas_android_npu_library

git clone https://gitlab.com/khadas/khadas_android_npu_library -b khadas_ai

(2) 替换nbg_unify_yolotiny目录 vnn_pre_process.h，vnn_post_process.h，vnn_yolotiny.h，vnn_yolotiny.c文件到如下对应目录。

cp {workspace}/yolotiny_nbg_unify/vnn_pre_process.h {workspace}/khadas_android_npu_library/model_code/detect_yolo_tiny/jni/include/
cp {workspace}/yolotiny_nbg_unify/vnn_post_process.h {workspace}/khadas_android_npu_library/model_code/detect_yolo_tiny/jni/include/
cp {workspace}/yolotiny_nbg_unify/vnn_yolotiny.h {workspace}/khadas_android_npu_library/model_code/detect_yolo_tiny/jni/include/
cp {workspace}/yolotiny_nbg_unify/vnn_yolotiny.c {workspace}/khadas_android_npu_library/model_code/detect_yolo_tiny/jni/

(3) 修改yolo_tiny_process.c
我们demo是两个类（KuLi, DuLanTe）, 修改class数组并修改num_class及LISTSIZE

static char *coco_names[] = {"KuLi","DuLanTe"};

image
image1035×694 142 KB

其中LISTSIZE = (num_class + 5 ) = 2+5 = 7

(4) 编译出libnn_yolo_tiny.so

基于ABI：armeabi-v7a
我们现在是默认为arm64-v8a，如果需要使用armeabi-v7a要做如下修改使用32位的libs

image
image1035×405 140 KB 

diff --git a/model_code/detect_yolo_tiny/jni/Android.mk b/model_code/detect_yolo_tiny/jni/Android.mk
old mode 100755
new mode 100644
index 8ba2dbc..b47f5a9
--- a/model_code/detect_yolo_tiny/jni/Android.mk
+++ b/model_code/detect_yolo_tiny/jni/Android.mk
@@ -30,7 +30,7 @@ CFLAG+=-Wno-error=typedef-redefinition
 LOCAL_CFLAGS += $(CFLAG)
 
 
-LOCAL_LDLIBS +=  -L$(LOCAL_PATH)/libs -lovxlib 
+LOCAL_LDLIBS +=  -L$(LOCAL_PATH)/lib32 -lovxlib
 
 LOCAL_MODULE := libnn_yolo_tiny
 
diff --git a/model_code/detect_yolo_tiny/jni/Application.mk b/model_code/detect_yolo_tiny/jni/Application.mk
index ca93d62..67d2bac 100644
--- a/model_code/detect_yolo_tiny/jni/Application.mk
+++ b/model_code/detect_yolo_tiny/jni/Application.mk
@@ -4,7 +4,7 @@ APP_STL      := gnustl_static
 
 APP_CFLAGS   := -Wno-error=format-security 
 APP_CPPFLAGS := -frtti -fexceptions
-APP_ABI      := arm64-v8a
+APP_ABI      := armeabi-v7a
 APP_PLATFORM := android-9
 APP_CPPFLAGS := -std=c++11
 #APP_ABI      := armeabi armeabi-v7a arm64-v8a

cd {workspace}/khadas_android_npu_library/model_code/detect_yolo_tiny
ndk-build

image
image1035×481 187 KB

基于ABI：arm64-v8a
我们现在是默认为arm64-v8a，如果需要使用arm64-v8a无须任何改动直接编译生成so文件即可

cd {workspace}/khadas_android_npu_library/model_code/detect_yolo_tiny
ndk-build

image
image1035×334 137 KB

(5) 编译出libnn_yolo_tiny.so
基于ABI：armeabi-v7a
我们现在是默认为arm64-v8a，如果需要使用armeabi-v7a要做如下修改使用32位的libs

image
image1035×297 117 KB 

diff --git a/detect_code/jni/Android.mk b/detect_code/jni/Android.mk
index 0e61a5d..53f4288 100644
--- a/detect_code/jni/Android.mk
+++ b/detect_code/jni/Android.mk
@@ -33,7 +33,7 @@ CFLAG+=-Wno-error=int-conversion
 
 LOCAL_CFLAGS += $(CFLAG)
 LOCAL_LDLIBS += -llog -lz -lm -ldl
-LOCAL_LDLIBS += -L$(LOCAL_PATH)/opencv_libs -lopencv_java4  
+LOCAL_LDLIBS += -L$(LOCAL_PATH)/opencv_lib32 -lopencv_java4
 
 LOCAL_MODULE := libkhadas_npu_jni
 
diff --git a/detect_code/jni/Application.mk b/detect_code/jni/Application.mk
index ca93d62..67d2bac 100644
--- a/detect_code/jni/Application.mk
+++ b/detect_code/jni/Application.mk
@@ -4,7 +4,7 @@ APP_STL      := gnustl_static
 
 APP_CFLAGS   := -Wno-error=format-security 
 APP_CPPFLAGS := -frtti -fexceptions
-APP_ABI      := arm64-v8a
+APP_ABI      := armeabi-v7a
 APP_PLATFORM := android-9
 APP_CPPFLAGS := -std=c++11
 #APP_ABI      := armeabi armeabi-v7a arm64-v8a

cd {workspace}/khadas_android_npu_library/detect_code
ndk-build

image
image1035×96 29.7 KB

基于ABI：arm64-v8a
我们现在是默认为arm64-v8a，如果需要使用arm64-v8a无须任何改动直接编译生成so文件即可

cd {workspace}/khadas_android_npu_library/detect_code
ndk-build

image
image1035×94 27.8 KB

4， 导入相关库到demo app运行

(1) 下载demo app

git clone https://github.com/khadas/khadas_android_npu_app -b khadas_ai

(2) 替换libnn_yolo_tiny.so

基于ABI：armeabi-v7a

cp {workspace}/khadas_android_npu_library/model_code/detect_yolo_tiny/libs/armeabi-v7a/libnn_yolo_tiny.so {workspace}/khadas_android_npu_app/app/libs/armeabi-v7a/

基于ABI：arm64-v8a

cp {workspace}/khadas_android_npu_library/model_code/detect_yolo_tiny/libs/arm64-v8a/libnn_yolo_tiny.so {workspace}/khadas_android_npu_app/app/libs/arm64-v8a/

(3) 替换libkhadas_npu_jni.so

基于ABI：armeabi-v7a

cp {workspace}/khadas_android_npu_library/detect_code/libs/armeabi-v7a/libkhadas_npu_jni.so {workspace}/khadas_android_npu_app/app/libs/armeabi-v7a/

基于ABI：arm64-v8a

cp {workspace}/khadas_android_npu_library/detect_code/libs/arm64-v8a/khadas_npu_jni.so {workspace}/khadas_android_npu_app/app/libs/arm64-v8a/

(4) 替换nb文件
ABI armeabi-v7a和arm64-v8a的nb文件一致

cp {workspace}/acuity-toolkit/demo/yolotiny_nbg_unify/yolotiny.nb {workspace}/khadas_android_npu_app/app/src/main/assets/yolotiny_88.nb

注意：VIM3为：yolotiny_88.nb。VIM3L为：yolotiny_99.nb

(5) 大功告成，把khadas_android_npu_app工程代码导入android studio中编译运行在VIM3板子上。
也可自行下载demo apk

image
image1035×582 46.4 KB

1 Like