怎么转换pytorch模型到板子运行？

zhakangning · January 6, 2022, 9:13am

我测试了convert_pytorch工具，会报错
Traceback (most recent call last):
File “convertpytorch.py”, line 96, in
File “convertpytorch.py”, line 86, in main
File “acuitylib/vsi_nn.py”, line 225, in load_pytorch_by_onnx_backend
File “acuitylib/onnx_ir/frontend/pytorch_frontend/pytorch_frontend.py”, line 45, in init
File “torch/jit/init.py”, line 162, in load
RuntimeError: version_number <= kMaxSupportedFileFormatVersion INTERNAL ASSERT FAILED at /pytorch/caffe2/serialize/inline_container.cc:131, please report a bug to PyTorch. Attempted to read a PyTorch file with version 3, but the maximum supported version for reading is 1. Your PyTorch installation may be too old. (init at /pytorch/caffe2/serialize/inline_container.cc:131)
frame #0: c10::Error::Error(c10::SourceLocation, std::string const&) + 0x33 (0x7f5a10545273 in /media/dd/Data/重大项目/目标检测项目/khadas_VIM3/aml_npu_sdk_new/acuity-toolkit/bin/acuitylib/libc10.so)
frame #1: caffe2::serialize::PyTorchStreamReader::init() + 0x1e9a (0x7f5a134d400a in /media/dd/Data/重大项目/目标检测项目/khadas_VIM3/aml_npu_sdk_new/acuity-toolkit/bin/acuitylib/libtorch.so)
frame #2: caffe2::serialize::PyTorchStreamReader::PyTorchStreamReader(std::string const&) + 0x60 (0x7f5a134d5270 in /media/dd/Data/重大项目/目标检测项目/khadas_VIM3/aml_npu_sdk_new/acuity-toolkit/bin/acuitylib/libtorch.so)
frame #3: torch::jit::import_ir_module(std::shared_ptrtorch::jit::script::CompilationUnit, std::string const&, c10::optionalc10::Device, std::unordered_map<std::string, std::string, std::hashstd::string, std::equal_tostd::string, std::allocator<std::pair<std::string const, std::string> > >&) + 0x38 (0x7f5a145b4088 in /media/dd/Data/重大项目/目标检测项目/khadas_VIM3/aml_npu_sdk_new/acuity-toolkit/bin/acuitylib/libtorch.so)
frame #4: + 0x4d6abc (0x7f5a5a8b5abc in /media/dd/Data/重大项目/目标检测项目/khadas_VIM3/aml_npu_sdk_new/acuity-toolkit/bin/acuitylib/libtorch_python.so)
frame #5: + 0x1d3f04 (0x7f5a5a5b2f04 in /media/dd/Data/重大项目/目标检测项目/khadas_VIM3/aml_npu_sdk_new/acuity-toolkit/bin/acuitylib/libtorch_python.so)

frame #26: …/bin/convertpytorch() [0x402ca1]
frame #27: …/bin/convertpytorch() [0x403087]
frame #28: __libc_start_main + 0xf3 (0x7f5aa91f80b3 in /lib/x86_64-linux-gnu/libc.so.6)
frame #29: …/bin/convertpytorch() [0x401a9e]

[670662] Failed to execute script convertpytorch

zhakangning · January 6, 2022, 9:14am

@Frank
0_import_model.sh内容如下

$convert_pytorch --pytorch-model model/yolov5s.pt
–net-output ${NAME}.json
–data-output ${NAME}.data
–input-size-list ‘1,480,854’

Frank · January 6, 2022, 9:15am

@zhakangning 因为工具本身支持的python版本很低，建议你这边先导出成onnx，再使用onnx的工具去转换

zhakangning · January 10, 2022, 12:48am

我用pytorch自定义了一个和yolov5模型输出一样的模型，通过转换到ONNX，再转换到NB文件使用，发现vsi_nn_VerifyGraph接口会返回-1。

请问这种情况怎么处理？

附上源码：
vsi_nn_context_t ctx = vsi_nn_CreateContext();

graph = vsi_nn_CreateGraph(ctx, NET_TOTAL_TENSOR_NUM, NET_NODE_NUM);
// printf("graph = %x\n", graph);

vsi_nn_SetGraphVersion( graph, VNN_VERSION_MAJOR, VNN_VERSION_MINOR, VNN_VERSION_PATCH );
vsi_nn_SetGraphInputs( graph, NULL, 1 );
vsi_nn_SetGraphOutputs( graph, NULL, 3 );

/*-----------------------------------------

Node definitions
-----------------------------------------*/

/*-----------------------------------------
  lid       - nbg_0
  var       - node[0]
  name      - nbg
  operation - nbg
  input     - [416, 416, 3, 1]
  output    - [13, 13, 33, 1]
              [26, 26, 33, 1]
              [52, 52, 33, 1]
-----------------------------------------*/
vsi_nn_node_t *node[NET_NODE_NUM];

NEW_VXNODE(node[0], VSI_NN_OP_NBG, 1, 3, 0);
node[0]->nn_param.nbg.type = VSI_NN_NBG_FILE;
node[0]->nn_param.nbg.url = model_path.c_str();

/*-----------------------------------------

Tensor initialize
-----------------------------------------/
vsi_nn_tensor_attr_t attr={0};
attr.dtype.fmt = VSI_NN_DIM_FMT_NCHW;
/ @input_0:out0 */
attr.size[0] = input_h;
attr.size[1] = input_w;
attr.size[2] = 3;
attr.size[3] = 1;
attr.dim_num = 4;
attr.dtype.fl = 7;
attr.dtype.qnt_type = VSI_NN_QNT_TYPE_DFP;

vsi_nn_tensor_id_t norm_tensor[NET_NORM_TENSOR_NUM];
NEW_NORM_TENSOR(norm_tensor[0], attr, VSI_NN_TYPE_INT8);

/* @output_82:out0 */
attr.size[0] = input_h/32;
attr.size[1] = input_w/32;
attr.size[2] = (num_class+5)*3;
attr.size[3] = 1;
attr.dim_num = 4;
attr.dtype.fl = 2;
attr.dtype.qnt_type = VSI_NN_QNT_TYPE_DFP;
NEW_NORM_TENSOR(norm_tensor[1], attr, VSI_NN_TYPE_INT8);

/* @output_94:out0 */
attr.size[0] = input_h/16;
attr.size[1] = input_w/16;
attr.size[2] = (num_class+5)*3;
attr.size[3] = 1;
attr.dim_num = 4;
attr.dtype.fl = 2;
attr.dtype.qnt_type = VSI_NN_QNT_TYPE_DFP;
NEW_NORM_TENSOR(norm_tensor[2], attr, VSI_NN_TYPE_INT8);

/* @output_106:out0 */
attr.size[0] = input_h/8;
attr.size[1] = input_w/8;
attr.size[2] = (num_class+5)*3;
attr.size[3] = 1;
attr.dim_num = 4;
attr.dtype.fl = 2;
attr.dtype.qnt_type = VSI_NN_QNT_TYPE_DFP;
NEW_NORM_TENSOR(norm_tensor[3], attr, VSI_NN_TYPE_INT8);


/*-----------------------------------------

Connection initialize
-----------------------------------------*/
node[0]->input.tensors[0] = norm_tensor[0];
node[0]->output.tensors[0] = norm_tensor[1];
node[0]->output.tensors[1] = norm_tensor[2];
node[0]->output.tensors[2] = norm_tensor[3];

/* nbg_0 */
graph->input.tensors[0] = norm_tensor[0];
graph->output.tensors[0] = norm_tensor[1];
graph->output.tensors[1] = norm_tensor[2];
graph->output.tensors[2] = norm_tensor[3];

// char *use_img_process_s=getenv( "VSI_USE_IMAGE_PROCESS" );

vsi_status status = vsi_nn_SetupGraph( graph, FALSE );
    // TEST_CHECK_STATUS( status, error );
cout<<"vsi_nn_SetupGraph() return "<<status<<endl;
status = vsi_nn_VerifyGraph(graph);
cout<<"vsi_nn_VerifyGraph() return "<<status<<endl;

zhakangning · January 10, 2022, 12:49am

@Frank 请求支援。。。。。。。凑字数字数字数字数字数字数

Frank · January 10, 2022, 1:24am

@zhakangning

github.com

khadas/aml_npu_app/blob/master/DDK_6.3.3.4/detect_library/model_code/detect_yolo_v3/yolo_v3.c

#include "yolo_v3.h"
#include "vnn_yolov3.h"
#include "yolov3_process.h"
#include "vnn_pre_process.h"
#include "vnn_post_process.h"

vsi_nn_graph_t * g_graph = NULL;

const static vsi_nn_postprocess_map_element_t* postprocess_map = NULL;
const static vsi_nn_preprocess_map_element_t* preprocess_map = NULL;

const vsi_nn_preprocess_map_element_t * vnn_GetPrePorcessMap()
{
	return preprocess_map;
}

uint32_t vnn_GetPrePorcessMapCount()
{
	if (preprocess_map == NULL)
		return 0;

This file has been truncated. show original

你可以参考这个代码，怎么使用转换出来的代码

zhakangning · January 10, 2022, 1:26am

我之前就是参考的这个代码，用darknet框架转出来的nb文件就可以使用。我现在改pytorch框架（同样的输入输出型状），代码也没有改，就是不行

Frank · January 10, 2022, 1:31am

@zhakangning 生成的文件里面已经给你生成好函数了，请问你自己修改成上面代码的原因是什么，从你给我的代码来看，我没看到你哪里申请了内存。对内存操作不熟悉的话，最好是直接使用生成的代码。

zhakangning · January 10, 2022, 1:36am

这个代码就是从自动生成的代码里复制出来的，就是稍微按照自己的风格封装了一下，并且我测试之前darknet转换出的nb文件，是可以执行的。现在就是换了一个nb文件（通过pytorch转onnx转出来的）就不行了。

Frank · January 10, 2022, 1:42am

@zhakangning 那你试试看，不要封装，就直接用转换出来的代码测试呢，验证graph失败，基本都是内存的问题。如果是nb和graph不匹配，再创建的时候就会失败

zhakangning · January 17, 2022, 6:44am

我在尝试编译那个转换onnx模型之后，自动生成的demo代码，发现缺少jpeglib库

g++ -o bin/demo.out main.c vnn_yolov5s.c vnn_pre_process.c
-lovxlib
vnn_pre_process.c:14:10: fatal error: jpeglib.h: No such file or directory
14 | #include “jpeglib.h”
| ^~~~~~~~~~~
compilation terminated.
make: *** [Makefile:2: demo] Error 1
@Frank

Frank · January 17, 2022, 7:04am

@zhakangning sudo apt install libjpeg9-dev