Preface: NPU 6.4.6.2-based model conversion reference [zero-based import your own YOLOv3 or YOLOv3-Tiny model] (零基础导入自己YOLOv3或YOLOv3-Tiny模型) The steps are basically the same and there is no change , but based on the NPU 6.4.8.7 version, there are some changes in the conversion model. The conversion tool acuity-toolkit is updated and the use of armeabi-v7a and arm64-v8a ABI is updated. Compared with the old conversion tool, the new acuity - The toolkit is more intelligent and the places that need to be modified are more streamlined when you need to convert your own model, but there are still the same parts. I will describe the differences and usage methods to you below. This document is only applicable to VIM3/VIM3L Android platform.
一，Conversion Model
Model Conversion Flowchart：
1，Import Model
The current model conversion process is carried out in the acuity-toolkit directory, put the trained darknet model into the model/ directory
cd {workspace}/khadas_android_npu_librar/acuity-toolkit/demo
cp {workspace}/yolov3-khadas_ai_tiny.cfg_train model/
cp {workspace}/yolov3-khadas_ai_tiny_last.weights model/
cp {workspace}/test.jpg data/
Modify the 0_import_model.sh script：
diff --git a/acuity-toolkit/demo/0_import_model.sh b/acuity-toolkit/demo/0_import_model.sh
index 8d3812e..277815b 100644
--- a/acuity-toolkit/demo/0_import_model.sh
+++ b/acuity-toolkit/demo/0_import_model.sh
@@ -1,6 +1,6 @@
#!/bin/bash
-NAME=mobilenet_tf
+NAME=yolotiny
ACUITY_PATH=../bin/
pegasus=${ACUITY_PATH}pegasus
@@ -9,19 +9,26 @@ if [ ! -e "$pegasus" ]; then
fi
#Tensorflow
-$pegasus import tensorflow \
- --model ./model/mobilenet_v1.pb \
- --inputs input \
- --outputs MobilenetV1/Predictions/Reshape_1 \
- --input-size-list '224,224,3' \
- --output-data ${NAME}.data \
- --output-model ${NAME}.json
+#$pegasus import tensorflow \
+# --model ./model/mobilenet_v1.pb \
+# --inputs input \
+# --outputs MobilenetV1/Predictions/Reshape_1 \
+# --input-size-list '224,224,3' \
+# --output-data ${NAME}.data \
+# --output-model ${NAME}.json
+
+#Darknet
+$pegasus import darknet\
+ --model ./model/yolov3-khadas_ai_tiny.cfg_train \
+ --weights ./model/yolov3-khadas_ai_tiny_last.weights \
+ --output-model ${NAME}.json \
+ --output-data ${NAME}.data \
#generate inpumeta --source-file dataset.txt
$pegasus generate inputmeta \
--model ${NAME}.json \
--input-meta-output ${NAME}_inputmeta.yml \
- --channel-mean-value "128 128 128 0.0078125" \
+ --channel-mean-value "0 0 0 0.0039215" \
--source-file dataset.txt
diff --git a/acuity-toolkit/demo/data/space_shuttle_224.jpg b/acuity-toolkit/demo/data/space_shuttle_224.jpg
deleted file mode 100644
index 97bbb2c..0000000
Binary files a/acuity-toolkit/demo/data/space_shuttle_224.jpg and /dev/null differ
diff --git a/acuity-toolkit/demo/data/validation_tf.txt b/acuity-toolkit/demo/data/validation_tf.txt
index 5b8dd9d..c2b8f96 100644
--- a/acuity-toolkit/demo/data/validation_tf.txt
+++ b/acuity-toolkit/demo/data/validation_tf.txt
@@ -1 +1 @@
-./space_shuttle_224.jpg
+./test.jpg
diff --git a/acuity-toolkit/demo/dataset.txt b/acuity-toolkit/demo/dataset.txt
index d2c7ae0..8b83182 100644
--- a/acuity-toolkit/demo/dataset.txt
+++ b/acuity-toolkit/demo/dataset.txt
@@ -1 +1 @@
-./data/space_shuttle_224.jpg
+./data/test.jpg
Execute the corresponding script：
bash 0_import_model.sh
2，Quantify the model
diff --git a/acuity-toolkit/demo/1_quantize_model.sh b/acuity-toolkit/demo/1_quantize_model.sh
index c3165ec..b592465 100644
--- a/acuity-toolkit/demo/1_quantize_model.sh
+++ b/acuity-toolkit/demo/1_quantize_model.sh
@@ -1,6 +1,6 @@
#!/bin/bash
-NAME=mobilenet_tf
+NAME=yolotiny
ACUITY_PATH=../bin/
pegasus=${ACUITY_PATH}pegasus
@@ -13,7 +13,7 @@ fi
# --quantizer perchannel_symmetric_affine --qtype int8(int16, note only T3(0xBE) can support perchannel quantize)
$pegasus quantize \
--quantizer dynamic_fixed_point \
- --qtype int16 \
+ --qtype int8 \
--rebuild \
--with-input-meta ${NAME}_inputmeta.yml \
--model ${NAME}.json \
Execute the corresponding script：
bash 1_quantize_model.sh
3，Generate case code
diff --git a/acuity-toolkit/demo/2_export_case_code.sh b/acuity-toolkit/demo/2_export_case_code.sh
index 80780f6..c86f8dc 100644
--- a/acuity-toolkit/demo/2_export_case_code.sh
+++ b/acuity-toolkit/demo/2_export_case_code.sh
@@ -1,6 +1,6 @@
#!/bin/bash
-NAME=mobilenet_tf
+NAME=yolotiny
ACUITY_PATH=../bin/
pegasus=$ACUITY_PATH/pegasus
@@ -14,7 +14,7 @@ $pegasus export ovxlib\
--model-quantize ${NAME}.quantize \
--with-input-meta ${NAME}_inputmeta.yml \
--dtype quantized \
- --optimize VIPNANOQI_PID0X1E \
+ --optimize VIPNANOQI_PID0X88 \
--viv-sdk ${ACUITY_PATH}vcmdtools \
--pack-nbg-unify
Note: VIM3 corresponds to VIPNANOQI_PID0X88, and VIM3L corresponds to VIPNANOQI_PID0X99
Execute the corresponding script：
bash 2_export_case_code.sh
Of course, as long as there is no error when running the script, you can execute the script at one time after the above modification is completed.：
bash 0_import_model.sh && bash 1_quantize_model.sh && bash 2_export_case_code.sh
Eventually a yolotiny_nbg_unify directory will be generated：
二，The demo app imported to the VIM3 android platform runs
1， Install ndk compilation environment
wget https://dl.google.com/android/repository/android-ndk-r17-linux-x86_64.zip
unzip android-ndk-r17-linux-x86_64.zip
vim ~/.bashrc
##Add the following two lines of code to the end of the file
##export NDKROOT=/path/to/android-ndk-r17
##export PATH=$NDKROOT:$PATH
As the picture shows：
Then execute this command on the ssh you want to use：
source ~/.bashrc
2， Compile related so libraries
(1) Downloadkhadas_android_npu_library
git clone https://gitlab.com/khadas/khadas_android_npu_library -b khadas_ai
(2) Replace the nbg_unify_yolotiny directory vnn_pre_process.h, vnn_post_process.h, vnn_yolotiny.h, vnn_yolotiny.c files to the following corresponding directories.
cp {workspace}/yolotiny_nbg_unify/vnn_pre_process.h {workspace}/khadas_android_npu_library/model_code/detect_yolo_tiny/jni/include/
cp {workspace}/yolotiny_nbg_unify/vnn_post_process.h {workspace}/khadas_android_npu_library/model_code/detect_yolo_tiny/jni/include/
cp {workspace}/yolotiny_nbg_unify/vnn_yolotiny.h {workspace}/khadas_android_npu_library/model_code/detect_yolo_tiny/jni/include/
cp {workspace}/yolotiny_nbg_unify/vnn_yolotiny.c {workspace}/khadas_android_npu_library/model_code/detect_yolo_tiny/jni/
(3) Reviseyolo_tiny_process.c
Our demo is two classes (KuLi, DuLanTe), modify the class array and modify num_class and LISTSIZE
static char *coco_names[] = {"KuLi","DuLanTe"};
in
LISTSIZE = (num_class + 5 ) = 2+5 = 7
(4) compiled outlibnn_yolo_tiny.so
ABI-based: armeabi-v7a
We now default to arm64-v8a. If you need to use armeabi-v7a, you need to make the following modifications to use 32-bit libs
diff --git a/model_code/detect_yolo_tiny/jni/Android.mk b/model_code/detect_yolo_tiny/jni/Android.mk
old mode 100755
new mode 100644
index 8ba2dbc..b47f5a9
--- a/model_code/detect_yolo_tiny/jni/Android.mk
+++ b/model_code/detect_yolo_tiny/jni/Android.mk
@@ -30,7 +30,7 @@ CFLAG+=-Wno-error=typedef-redefinition
LOCAL_CFLAGS += $(CFLAG)
-LOCAL_LDLIBS += -L$(LOCAL_PATH)/libs -lovxlib
+LOCAL_LDLIBS += -L$(LOCAL_PATH)/lib32 -lovxlib
LOCAL_MODULE := libnn_yolo_tiny
diff --git a/model_code/detect_yolo_tiny/jni/Application.mk b/model_code/detect_yolo_tiny/jni/Application.mk
index ca93d62..67d2bac 100644
--- a/model_code/detect_yolo_tiny/jni/Application.mk
+++ b/model_code/detect_yolo_tiny/jni/Application.mk
@@ -4,7 +4,7 @@ APP_STL := gnustl_static
APP_CFLAGS := -Wno-error=format-security
APP_CPPFLAGS := -frtti -fexceptions
-APP_ABI := arm64-v8a
+APP_ABI := armeabi-v7a
APP_PLATFORM := android-9
APP_CPPFLAGS := -std=c++11
#APP_ABI := armeabi armeabi-v7a arm64-v8a
cd {workspace}/khadas_android_npu_library/model_code/detect_yolo_tiny
ndk-build
ABI-based: arm64-v8a
We now default to arm64-v8a. If you need to use arm64-v8a without any modification, you can directly compile and generate the so file
cd {workspace}/khadas_android_npu_library/model_code/detect_yolo_tiny
ndk-build
(5) Compile libnn_yolo_tiny.so
ABI-based: armeabi-v7a
We now default to arm64-v8a. If you need to use armeabi-v7a, you need to make the following modifications to use 32-bit libs
diff --git a/detect_code/jni/Android.mk b/detect_code/jni/Android.mk
index 0e61a5d..53f4288 100644
--- a/detect_code/jni/Android.mk
+++ b/detect_code/jni/Android.mk
@@ -33,7 +33,7 @@ CFLAG+=-Wno-error=int-conversion
LOCAL_CFLAGS += $(CFLAG)
LOCAL_LDLIBS += -llog -lz -lm -ldl
-LOCAL_LDLIBS += -L$(LOCAL_PATH)/opencv_libs -lopencv_java4
+LOCAL_LDLIBS += -L$(LOCAL_PATH)/opencv_lib32 -lopencv_java4
LOCAL_MODULE := libkhadas_npu_jni
diff --git a/detect_code/jni/Application.mk b/detect_code/jni/Application.mk
index ca93d62..67d2bac 100644
--- a/detect_code/jni/Application.mk
+++ b/detect_code/jni/Application.mk
@@ -4,7 +4,7 @@ APP_STL := gnustl_static
APP_CFLAGS := -Wno-error=format-security
APP_CPPFLAGS := -frtti -fexceptions
-APP_ABI := arm64-v8a
+APP_ABI := armeabi-v7a
APP_PLATFORM := android-9
APP_CPPFLAGS := -std=c++11
#APP_ABI := armeabi armeabi-v7a arm64-v8a
cd {workspace}/khadas_android_npu_library/detect_code
ndk-build
ABI-based: arm64-v8a
We now default to arm64-v8a. If you need to use arm64-v8a without any modification, you can directly compile and generate the so file
cd {workspace}/khadas_android_npu_library/detect_code
ndk-build
4， Import related libraries to demo app to run
(1) Downloaddemo app
git clone https://github.com/khadas/khadas_android_npu_app -b khadas_ai
(2) Replacelibnn_yolo_tiny.so
Based on ABI：armeabi-v7a
cp {workspace}/khadas_android_npu_library/model_code/detect_yolo_tiny/libs/armeabi-v7a/libnn_yolo_tiny.so {workspace}/khadas_android_npu_app/app/libs/armeabi-v7a/
Based on ABI：arm64-v8a
cp {workspace}/khadas_android_npu_library/model_code/detect_yolo_tiny/libs/arm64-v8a/libnn_yolo_tiny.so {workspace}/khadas_android_npu_app/app/libs/arm64-v8a/
(3) Replacelibkhadas_npu_jni.so
Based on ABI：armeabi-v7a
cp {workspace}/khadas_android_npu_library/detect_code/libs/armeabi-v7a/libkhadas_npu_jni.so {workspace}/khadas_android_npu_app/app/libs/armeabi-v7a/
Based on ABI：arm64-v8a
cp {workspace}/khadas_android_npu_library/detect_code/libs/arm64-v8a/khadas_npu_jni.so {workspace}/khadas_android_npu_app/app/libs/arm64-v8a/
(4) replace nb file
The nb files of ABI armeabi-v7a and arm64-v8a are consistent
cp {workspace}/acuity-toolkit/demo/yolotiny_nbg_unify/yolotiny.nb {workspace}/khadas_android_npu_app/app/src/main/assets/yolotiny_88.nb
Note: VIM3 is: yolotiny_88.nb. VIM3L is: yolotiny_99.nb
(5) You’re done, import the khadas_android_npu_app project code into android studio to compile and run on the VIM3 board.
You can also download it yourselfdemo apk
