关于npu使用该用哪种接口方案

ideafold · November 8, 2021, 3:49am

目测有几种方法：
1 /usr/lib/libnnsdk，包含aml_module_create等接口
2 acuity-toolkit/conversion_scripts产生的case代码
3 oaid 的tengine 、tvm fork等第三方框架

请问，推荐用哪种方法呢？
@numbqq @Frank

备注：
用/usr/lib/libnnsdk的aml_module_create 加载 acuity-toolkit/conversion_scripts 产生的nb文件报错哦，
E NN_SDK: [amlnn_init:858]Error:network verifygraph fail
E NN_SDK: [amlnn_init:904]Error:amlnn_init fail
E NN_SDK: [aml_module_create:71]Error:amlnn_init is fail, context is null

我的模型时onnx，用的是 https://github.com/khadas/aml_npu_sdk 这个转换docker

Frank · November 8, 2021, 5:39am

使用那个方案都可以，这个你自行选择，个人推荐使用原生的，或者是tengine的，我们也有自己的python的接口.

请把报错信息详细贴出来。同时转换是需要在x86，也就是你自己的PC上进行的

ideafold · November 8, 2021, 6:20am

是在pc上转的。转换没有报错
转出的nb模型，用vsi_nn_graph这一套搞，在status = vsi_nn_VerifyGraph( graph );
提示 “E [vnn_VerifyGraph:51]CHECK STATUS 51”
目测是不通过。

onnx模型以及样例图片数据
链接：百度网盘请输入提取码
提取码：s1h2

NAME=dddnuscenes_v1
ACUITY_PATH=../bin/

convert_caffe=${ACUITY_PATH}convertcaffe
convert_tf=${ACUITY_PATH}convertensorflow
convert_tflite=${ACUITY_PATH}convertflite
convert_darknet=${ACUITY_PATH}convertdarknet
convert_onnx=${ACUITY_PATH}convertonnx
convert_torch=${ACUITY_PATH}convertpytorch

$convert_onnx \
    --onnx-model  /model/ddd_nuscenes.onnx \
    --net-output ${NAME}.json \
    --data-output ${NAME}.data 

NAME=dddnuscenes_v1
ACUITY_PATH=../bin/

tensorzone=${ACUITY_PATH}tensorzonex

#asymmetric_quantized-u8 dynamic_fixed_point-8 dynamic_fixed_point-16
$tensorzone \
    --action quantization \
    --source text \
    --source-file ./data/nuscenes_validation_tf.txt \
    --channel-mean-value '0 0 0 1' \
    --model-input ${NAME}.json \
    --model-data ${NAME}.data \
    --quantized-dtype asymmetric_quantized-u8 \
        --quantized-rebuild

$tensorzone \
    --action inference \
    --source text \
    --source-file ./data/nuscenes_validation_tf.txt \
    --channel-mean-value '0 0 0 1' \
    --model-input ${NAME}.json \
    --model-data ${NAME}.data \
    --dtype quantized


NAME=dddnuscenes_v1
ACUITY_PATH=../bin/

export_ovxlib=${ACUITY_PATH}ovxgenerator

$export_ovxlib \
    --model-input ${NAME}.json \
    --data-input ${NAME}.data \
    --reorder-channel '0 1 2' \
    --channel-mean-value '0 0 0 1' \
    --export-dtype quantized \
    --model-quantize ${NAME}.quantize \
    --optimize VIPNANOQI_PID0X88  \
    --viv-sdk ../bin/vcmdtools \
    --pack-nbg-unify 

rm -rf *.h *.c .project .cproject *.vcxproj *.lib BUILD *.linux
mv ../*_nbg_unify nbg_unify_${NAME}
cd nbg_unify_${NAME}
mv network_binary.nb ${NAME}.nb

@Frank

Frank · November 8, 2021, 6:21am

@ideafold 那你的报错是指什么报错

ideafold · November 8, 2021, 6:23am

aml_module_create 这一套的出错：
nbfile: [dddnuscenes_v1.nb]
E NN_SDK: [amlnn_init:858]Error:network verifygraph fail
E NN_SDK: [amlnn_init:904]Error:amlnn_init fail
E NN_SDK: [aml_module_create:71]Error:amlnn_init is fail, context is null

用vsi_nn_graph这一套搞，在status = vsi_nn_VerifyGraph( graph ); 的出错
提示 “E [vnn_VerifyGraph:51]CHECK STATUS 51”
目测是不通过。

@Frank

Frank · November 8, 2021, 6:25am

@ideafold 你的代码是你自己写的么，这个报错看起来是API没有正确使用

ideafold · November 8, 2021, 6:25am

训练代码是我自己写的，模型也是我自己训练的。

Frank · November 8, 2021, 6:31am

@ideafold 首先要确保环境没有问题，建议你先用我们的demo跑一下，确定环境是没有问题的，再去跑你自己的模型，这个错误应该是代码的问题

ideafold · November 8, 2021, 6:39am

demo 要跑起来也不容易啊。
cd aml_npu_app-master/DDK_6.3.3.4/detect_library/model_code/detect_yolo_v3
sh build_vx.sh

平台：
vim3 pro

g++ (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

khadas@Khadas:~/projects/aml_npu_app-master$ gcc --version
gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

khadas@Khadas:~/projects/aml_npu_app-master$ make --version
GNU Make 4.2.1
Built for aarch64-unknown-linux-gnu
Copyright (C) 1988-2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

报一堆错
khadas@Khadas:~/projects/aml_npu_app-master/DDK_6.3.3.4/detect_library/model_code/detect_yolo_v3$ ./build_vx.sh
COMPILE /home/khadas/projects/aml_npu_app-master/DDK_6.3.3.4/detect_library/model_code/detect_yolo_v3/yolov3_process.c
In file included from ./include/yolov3_process.h:5,
from yolov3_process.c:7:
./include/nn_detect_common.h:1:1: error: expected identifier or ‘(’ before ‘.’ token
1 | …/…/…/…/…/detect_library/source_code/include/nn_detect_common.h
| ^
In file included from yolov3_process.c:7:
./include/yolov3_process.h:10:24: error: unknown type name ‘input_image_t’
10 | void yolov3_preprocess(input_image_t imageData, uint8_t *ptr);
| ^~~~~~~~~~~~~
./include/yolov3_process.h:11:48: error: unknown type name ‘pDetResult’
11 | void yolov3_postprocess(vsi_nn_graph_t *graph, pDetResult resultData);
| ^~~~~~~~~~
In file included from ./include/yolo_v3.h:5,
from yolov3_process.c:8:
./include/nn_detect_common.h:1:1: error: expected identifier or ‘(’ before ‘.’ token
1 | …/…/…/…/…/detect_library/source_code/include/nn_detect_common.h
| ^
In file included from yolov3_process.c:8:
./include/yolo_v3.h:10:21: error: unknown type name ‘input_image_t’
10 | void model_setinput(input_image_t imageData, uint8_t* data);
| ^~~~~~~~~~~~~
./include/yolo_v3.h:11:1: error: unknown type name ‘det_status_t’
11 | det_status_t model_getresult(pDetResult resultData, uint8_t* data);
| ^~~~~~~~~~~~
./include/yolo_v3.h:11:30: error: unknown type name ‘pDetResult’
11 | det_status_t model_getresult(pDetResult resultData, uint8_t* data);
| ^~~~~~~~~~
./include/yolo_v3.h:12:20: error: unknown type name ‘dev_type’; did you mean ‘dev_t’?
12 | void model_release(dev_type type);
| ^~~~~~~~
| dev_t
yolov3_process.c:13:24: error: unknown type name ‘input_image_t’
13 | void yolov3_preprocess(input_image_t imageData, uint8_t *ptr)
| ^~~~~~~~~~~~~
yolov3_process.c:62:31: error: unknown type name ‘box’
62 | static float box_intersection(box a, box b)
| ^~~

ideafold · November 8, 2021, 6:47am

想找一份可以在vim3 pro下，下载回来make一下就可以跑的。但的确没找到。

Frank · November 8, 2021, 6:51am

@ideafold

这个文档的跑不起来？

ideafold · November 8, 2021, 6:58am

也跑不起

cv4_output$
./image_classify_224x224 image_classify_88.nb /home/khadas/t/aml_1532707724012404.jpg

报错如下：
E NN_SDK: [amlnn_init:858]Error:network verifygraph fail
E NN_SDK: [amlnn_init:904]Error:amlnn_init fail
E NN_SDK: [aml_module_create:71]Error:amlnn_init is fail, context is null
E NN_SDK: [aml_module_input_set:85]Error:aml_module_input_set is fail, context is null
E NN_SDK: [aml_module_output_get:120]Error:aml_module_output_get is fail, context is null

dmesg | grep Galcore
[ 19.540365] Galcore version 6.4.6.2

ideafold · November 8, 2021, 7:00am

我的Galcore 升级过，用的tim vx中最新的galcore。
之前为了跑tengine-lite升级的。

并且/usr/lib 的部分跟tim vx相关的库也升级过

Frank · November 8, 2021, 7:00am

@ideafold 你这个驱动版本，我们没有支持，你应该用我们官方的固件去试，不然我们提供的所有demo都是跑不起来的

ideafold · November 8, 2021, 7:01am

额。
旧的库文件和驱动文件没有保存
哪里可以下载吗。
刷系统稍微麻烦

Frank · November 8, 2021, 7:05am

@ideafold 你找到npu的模块，替换回去。相应的库也要重新装

这里可以找到

ideafold · November 8, 2021, 7:10am

请问下 Galcore 驱动，在那个deb包里呢？

Frank · November 8, 2021, 7:12am

@ideafold 通过fenix编译内核，然后手动替换这个驱动

ideafold · November 8, 2021, 7:14am

你方便copy一个你电脑的给我吗，编译目测有点麻烦啊。
谢谢了

Frank · November 8, 2021, 7:28am

@ideafold 你邮箱多少，私信给我，我通过邮箱附件发一个给你