VIM3 NPU demo for SSD MobileNet

New image_classify could not read the binary file and throws an error

E NN_SDK: [read_binary_file:95]Error: NOT A VALID GRAPH BINARY FILE

I used the binary file from AML_NN_SDK/Model/DDK6.4.4.3/88/image_classify_88.nb at master · Amlogic-NN/AML_NN_SDK · GitHub

@Vignesh_Raja

khadas@Khadas:~/aml_npu_nnsdk_app/image_classify_224x224$ ./build-cv3.sh 
  COMPILE /home/khadas/aml_npu_nnsdk_app/image_classify_224x224/image_classify_224x224.cpp
image_classify_224x224.cpp: In function ‘int postpress_network()’:
image_classify_224x224.cpp:263:6: warning: unused variable ‘i’ [-Wunused-variable]
  263 |  int i;
      |      ^
image_classify_224x224.cpp:265:22: warning: unused variable ‘pout’ [-Wunused-variable]
  265 |  img_classify_out_t *pout = NULL;
      |                      ^~~~
image_classify_224x224.cpp:266:15: warning: variable ‘modelType’ set but not used [-Wunused-but-set-variable]
  266 |  aml_module_t modelType;
      |               ^~~~~~~~~
khadas@Khadas:~/aml_npu_nnsdk_app/image_classify_224x224$ cd cv3_output/
khadas@Khadas:~/aml_npu_nnsdk_app/image_classify_224x224/cv3_output$ wget https://github.com/Amlogic-NN/AML_NN_SDK/raw/master/Model/DDK6.4.4.3/88/image_classify_88.nb
--2021-05-26 08:58:10--  https://github.com/Amlogic-NN/AML_NN_SDK/raw/master/Model/DDK6.4.4.3/88/image_classify_88.nb
Resolving github.com (github.com)... 192.30.255.112
Connecting to github.com (github.com)|192.30.255.112|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://raw.githubusercontent.com/Amlogic-NN/AML_NN_SDK/master/Model/DDK6.4.4.3/88/image_classify_88.nb [following]
--2021-05-26 08:58:11--  https://raw.githubusercontent.com/Amlogic-NN/AML_NN_SDK/master/Model/DDK6.4.4.3/88/image_classify_88.nb
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.110.133, 185.199.108.133, 185.199.109.133, ...
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.110.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 5765169 (5.5M) [application/octet-stream]
Saving to: ‘image_classify_88.nb’

image_classify_88.nb                            100%[=======================================================================================================>]   5.50M  3.17MB/s    in 1.7s    

2021-05-26 08:58:14 (3.17 MB/s) - ‘image_classify_88.nb’ saved [5765169/5765169]

khadas@Khadas:~/aml_npu_nnsdk_app/image_classify_224x224/cv3_output$ ./image_classify_224x224 image_classify_88.nb ../goldfish_224.jpg 
  2: 16.121683
116: 13.008531
843: 9.895378
444: 9.339458
795: 8.894722

It work for me

Sorry, it works for me as well.

@Frank
In this video (VIM3 - Realtime Object Detection Using Yolo v3 - YouTube), I noticed that you previously had instructions on how to retrain yolov3 to custom objects. Can you tell me where I can find this guide now

@Frank
I found this guide. Can you tell me if it is possible to freeze layers during training?

@ilya71 You can follow this guide

https://pjreddie.com/darknet/yolo/

@Frank I am using “ssd_mobilenet_v2_coco_2018_03_29” model for my use case.

What do you mean by saying this cannot be converted using conversion tools? Is this model not supported?

Also, I have come across the below mentioned links on khadas documentation.

What is the difference between these? Are there any specialities among each of them? Do all the three use NPU?

I trained and converted my yolov3 model and got the files as for mobile net_tf. I used them in the tutorial and ran the binary file from demo_binaries. However, I want to work with the source code that runs the neural network for integration into my application. I tried to compile my binary files from nbg_unify_my_yolov3 using the following cmake:

cmake_minimum_required(VERSION 3.1)
project(a LANGUAGES C CXX)

add_executable(person_detect "")
find_package(OpenCV REQUIRED)

target_include_directories(person_detect PRIVATE
        ${CMAKE_CURRENT_LIST_DIR}/
        /root/NeuralNetworks/aml_npu_sdk_6.4.3/linux_sdk/linux_sdk_6.4.3/acuity-ovxlib-dev/include/
)

target_link_libraries(person_detect PRIVATE
  opencv_core
  opencv_highgui
  /lib/libnnsdk.so
    /root/NeuralNetworks/aml_npu_sdk_6.4.3/linux_sdk/linux_sdk_6.4.3/acuity-ovxlib-dev/lib/libovxlib.so
/root/NeuralNetworks/aml_npu_sdk_6.4.3/linux_sdk/linux_sdk_6.4.3/build/sdk/drivers_64/libCLC.so
  /root/NeuralNetworks/aml_npu_sdk_6.4.3/linux_sdk/linux_sdk_6.4.3/acuity-ovxlib-dev/lib/libjpeg_64.a
  /root/NeuralNetworks/aml_npu_sdk_6.4.3/linux_sdk/linux_sdk_6.4.3/build/sdk/drivers_64/libOpenVX.so
  /root/NeuralNetworks/aml_npu_sdk_6.4.3/linux_sdk/linux_sdk_6.4.3/build/sdk/drivers_64
/root/NeuralNetworks/aml_npu_sdk_6.4.3/linux_sdk/linux_sdk_6.4.3/build/sdk/drivers_64/libOpenVX.so.1.2
  )

target_sources(person_detect PRIVATE
        main.c
        vnn_myyolov3.c
        vnn_post_process.c
        vnn_pre_process.c

)

It is going to but when I run it (LD_LIBRARY_PATH=/root/NeuralNetworks/aml_npu_sdk_6.4.3/linux_sdk/linux_sdk_6.4.3/build/sdk/drivers_64/ ./person_detect ../my_yolov3.nb /root/NeuralNetworks/test.bmp)
[ 1] HAL user version 6.4.3.279124
[ 2] HAL kernel version 6.4.3.279124
E [vnn_CreateMyYolov3:178]Create graph fail.
E [vnn_CreateNeuralNetwork:179]CHECK PTR 179
E [main:212]CHECK PTR 212

Please tell me what I did wrong

1 Like

@ilya71 Sorry . I am not familiar with cmake.

and what can be the reason for the error " E [vn_CreateMyYolov3:178]Create graph fail."?
or how do you run the files after converting in addition to copying them to the demo

I tried using build_vx.sh and makefile. linux replacing some library paths that were not present at compilation (build_vx.sh and makefile. linux took from. sample_demo_fb). And it helped me to go further, now the network is being created, but I have the following error:
Create Neural Network: 85ms or 85296us
Wrong JPEG library version: library is 90, caller expects 80

libjpeg. a was taken from linux_sdk_6. 4. 3/acuity-ovxlib-dev/lib

@ilya71 You need to use least release SDK. It’s 6.4.4.3. And you can follow this

I downloaded the SDK 6.4.4.3, but I do not see the linux_sdk folder from which I took the necessary libraries

i solved the problem on the old sdk by taking libjpeg from opencv

@ilya71 , In 6.4.4.3 and later versions, we are not integrating linux_sdk. we just support local compile, it don’t need linux_sdk

Please tell me in the manual (How to convert and call your own model through NPU | Khadas Documentation) “MOdify 1_quantize_model.sh” there is such an action:

Replace the image inside

1 $ cat ./data/validation_tf.txt
2 ./space_shuttle_224.jpg, 813
Modify to,

1 path/to/416x416.jpg

What did 813 mean in this file? And what is it used for?
Also I see in conversion_script datasets.txt with the same content. Do I need to change it?

@ilya71 Why don’t you read the documentation in the SDK?

But it doesn’t explain what the magic number 813 is, which is used in the demo

@ilya71 The mobilent model use imagenet dataset. the space shuttle in the NO.814 in this dataset.

@Frank @alcohol I want to use NPU for MobileNet v1 SSD model. As you said, since NPU SDK is not compatible for SSD, I decided to use Tengine SDK.

During my conversion process to .tmfile, I got the following error. Can you please share your input on this? I am not able to create graph! :frowning: