Hello khadas team,
For using NPU in Edge2, you have reported this doc:
https://docs.khadas.com/products/sbc/edge2/npu/npu-applications
It is mentioned in this document that use the below github link:
In this project and under edge2-npu/C++/yolov8n_cap/data/model there is a yolov8n rknn model file that properly works on edge2.
But in the below link there is no way to convert yolov8n to rknn:
https://docs.khadas.com/products/sbc/edge2/npu/npu-convert
My question is that how do you convert the yolov8n model to rknn?
Thank you.
Hello @aliliaei ,
First, download yolov8 official code.
git clone https://github.com/ultralytics/ultralytics.git
After training, modify ultralytics/ultralytics/nn/modules.py as follows.
diff --git a/ultralytics/nn/modules.py b/ultralytics/nn/modules.py
old mode 100644
new mode 100755
index 4c198dc..962504e
--- a/ultralytics/nn/modules.py
+++ b/ultralytics/nn/modules.py
@@ -466,6 +466,9 @@ class Detect(nn.Module):
def forward(self, x):
"""Concatenates and returns predicted bounding boxes and class probabilities."""
+ if torch.onnx.is_in_onnx_export():
+ return self.forward_export(x)
+
shape = x[0].shape # BCHW
for i in range(self.nl):
x[i] = torch.cat((self.cv2[i](x[i]), self.cv3[i](x[i])), 1)
@@ -485,6 +488,18 @@ class Detect(nn.Module):
y = torch.cat((dbox, cls.sigmoid()), 1)
return y if self.export else (y, x)
+ def forward_export(self, x):
+ results = []
+ for i in range(self.nl):
+ dfl = self.cv2[i](x[i]).contiguous()
+ cls = self.cv3[i](x[i]).contiguous()
+ results.append(torch.cat([cls, dfl], 1).permute(0, 2, 3, 1).unsqueeze(1))
+ return tuple(results)
Create a python file written as follows to export onnx model.
from ultralytics import YOLO
model = YOLO("./runs/detect/train/weights/best.pt")
results = model.export(format="onnx")
Put onnx model in example/onnx/yolov5 and run test.py (remember to choose rk3588 platform). You will get yolov8n rknn model. In test.py, there are yolov5 infer code. Maybe it will report error when you run but it does not affect converting model. If the rknn model is generated under the file, you succeed.
Thank you for your reply.
It worked for the original yolov8n model.
But my model contains one class (car). I did the above instruction for my model and created the rknn model. I also changed CLASSES on test.py in yolov5 folder in rknn toolkit and the label list on edg2 under data folder. But when i run the yolov8 algorithm on edg2 with my model it result in “Segmentation fault”.
Hello @aliliaei ,
Change OBJ_CLASS_NUM in include/postprocess.h from 80 to 1.
And modify src/postprocess.cc as follows.
diff --git a/C++/yolov8n_cap/src/postprocess.cc b/C++/yolov8n_cap/src/postprocess.cc
index 54d430c..46193e1 100644
--- a/C++/yolov8n_cap/src/postprocess.cc
+++ b/C++/yolov8n_cap/src/postprocess.cc
@@ -223,7 +223,7 @@ static int process(int8_t* input, int grid_h, int grid_w, int height, int width,
}
}
if (maxClassId >= 0) {
- int offset = (i * grid_h + j) * (OBJ_CLASS_NUM + 4 * 16) + 80;
+ int offset = (i * grid_h + j) * (OBJ_CLASS_NUM + 4 * 16) + OBJ_CLASS_NUM;
float box_x1 = (j + 0.5 - decode_box(&input[offset], 16, zp, scale)) * (float)stride;
float box_y1 = (i + 0.5 - decode_box(&input[offset + 16], 16, zp, scale)) * (float)stride;
float box_x2 = (j + 0.5 + decode_box(&input[offset + 16 * 2], 16, zp, scale)) * (float)stride;
If still run fail, please provide your original model and rknn model for us to find the cause of problem.
Hello @Louis-Cheng-Liu
I did it but i get segmentation fault again.
My trained model (.pt) and its onnx and rknn files can be download from below link:
Hello @aliliaei ,
I have received your model. We will solve the problem next week.
Hello @aliliaei ,
You maybe did not modify modules.py to change model output. However i find that official codes has updated. The latest codes splits modules.py. The latest codes should modify ultralytics/ultralytics/nn/modules/head.py.
diff --git a/ultralytics/nn/modules/head.py b/ultralytics/nn/modules/head.py
index 0b02eb3..0a6e43a 100644
--- a/ultralytics/nn/modules/head.py
+++ b/ultralytics/nn/modules/head.py
@@ -42,6 +42,9 @@ class Detect(nn.Module):
def forward(self, x):
"""Concatenates and returns predicted bounding boxes and class probabilities."""
+ if torch.onnx.is_in_onnx_export():
+ return self.forward_export(x)
+
shape = x[0].shape # BCHW
for i in range(self.nl):
x[i] = torch.cat((self.cv2[i](x[i]), self.cv3[i](x[i])), 1)
@@ -80,6 +83,15 @@ class Detect(nn.Module):
a[-1].bias.data[:] = 1.0 # box
b[-1].bias.data[:m.nc] = math.log(5 / m.nc / (640 / s) ** 2) # cls (.01 objects, 80 classes, 640 img)
+ def forward_export(self, x):
+ results = []
+ for i in range(self.nl):
+ dfl = self.cv2[i](x[i]).contiguous()
+ cls = self.cv3[i](x[i]).contiguous()
+ results.append(torch.cat([cls, dfl], 1).permute(0, 2, 3, 1).unsqueeze(1))
+ # results.append(torch.cat([cls, dfl], 1))
+ return tuple(results)
After converting onnx model, you can check if your model agrees with mine with using this tool.
Netron
Model should have three outputs.
Thank you @Louis-Cheng-Liu
I had a mistak. I was using another modules.py file.
I converted my model to onnx with the modules.py file that was modified as you said.
Here is my model:
Apparently the output onnx file is similar with your model.
Then i converted the onnx file to rknn with test.py with adding rk3588 platform.
But i get segmentation fault again:
I think there is a problem here. In the above screen, model output num is 1. It should be 3
When i convert the original yolov8n to rknn with the above instruction, the model output is 3.
I uploaded the new model:
Hello @aliliaei ,
There are two questions.
First, please check your model if have been replaced or not. The right output should like this.
So, we guess you did not replace old model. If you put new model in yolov8n_cap/data/model, please remember to recompile to copy new model into yolov8n_cap/install/yolov8n_cap/data/model.
Secord, the version of your tool is v1.5.0. This version now is unsupported. Please downgrade the version to v1.4.0.
Hello @Louis-Cheng-Liu,
About your first question. your are right. I was not recompiling the code. After i recompiled, the output was 3 as below:
About your second question, when i use toolkit version 1.3 and 1.4 i encountered with these errors:
for toolkit 1.3:
pkg_resources.extern.packaging.version.InvalidVersion: Invalid version: ‘1.3.0-11912b58’
for toolkit 1.4:
pkg_resources.extern.packaging.version.InvalidVersion: Invalid version: ‘1.4.0-22dcfef4’
Only toolkit version 1.5 works for me.
Hello @aliliaei ,
I did not encountered this mistake. But i find a post about it.
pkg_resources.extern.packaging.version.InvalidVersion: Invalid version: ‘1.4.0-22dcfef4’ · Issue #161 · rockchip-linux/rknn-toolkit2 (github.com)
The solution is as follows in Chinese.
我第一次做的时候，报了错误，当时提示的是没有这个2-1.4.0-22dcfef4版本，我就跑到conda的环境包下，把所有的2-1.4.0-22dcfef4版本改成了2-1.4.0，才解决这个问题。
This means that modify rknn library name from 2-1.4.0-22dcfef4 to 2-1.4.0 in conda environment. You can try it.
Hello @Louis-Cheng-Liu,
Thank you very much
It worked by renaming rknn_toolkit2-1.4.0-22dcfef4.dist-info to rknn_toolkit2-1.4.0.dist-info under conda/envs/myenv/lib/python3.8/site-packages
Hello, @Louis-Cheng-Liu
I follow your instruction and take yolov8.pt from ultralytics repo to modify the code in nn/modules/head.py .
I get the output dimension of ONNX with [1, 1, 80, 80, 114] [1, 1, 40, 40, 114] [1, 1, 20, 20, 114].
Then, I convert the ONNX to RKNN with yolov8 rk3588 · GitHub and turn off the quantization.
I can run the code from edge2-npu/yolov8n to perform detection.
However, if I change the code to use my RKNN file, I cannot perform the detection.
Can you help me on that? Thanks
I have updated related file to the following link.
Hello @hweiminc ,
I convert youur ONNX model to RKNN and it can run right result. But i run your RKNN model, it performs bad, too. So i think maybe something wrong when you convert RKNN model.
And there are some differences. First is yours and second is mine.
Here is my test.py, you can refer it.
# Create RKNN object
rknn = RKNN(verbose=True)
# pre-process config
print('--> Config model')
rknn.config(mean_values=[[0, 0, 0]], std_values=[[255, 255, 255]], target_platform='rk3588')
print('done')
# Load ONNX model
print('--> Loading model')
ret = rknn.load_onnx(model='yolov8n_hweiminc.onnx')
if ret != 0:
print('Load model failed!')
exit(ret)
print('done')
# Build model
print('--> Building model')
ret = rknn.build(do_quantization=True, dataset='./dataset.txt')
if ret != 0:
print('Build model failed!')
exit(ret)
print('done')
# Export RKNN model
print('--> Export rknn model')
ret = rknn.export_rknn('yolov8n_hweiminc.rknn')
if ret != 0:
print('Export rknn model failed!')
exit(ret)
print('done')
If still fail, please show your convert file and the converting log.
Hello @Louis-Cheng-Liu,
Why are the results of .pt model and converted rknn model different?
.pt model on original ultralytics yolov8n algorithm:
rknn model on khadas edge2-npu algorithm:
The results of converted rknn model are weaker.
Hello @aliliaei ,
Because of quantification when convert model, the result of rknn model is a little worse than original model. You can try to lower threshold to improve result.
The set of threshold is in include/postprocess.h, BOX_THRESH.
