Severe Accuracy Drop After Model Conversion on Khadas VIM3

VIM3
1

Which system do you use? Android, Ubuntu, OOWOW or others?

Ubuntu

Which version of system do you use? Please provide the version of the system here:

Linux Khadas 4.9.241 #4 SMP PREEMPT Tue Apr 25 08:04:06 UTC 2023 aarch64 aarch64 aarch64 GNU/Linux

Please describe your issue below:

After converting the model using the convert tool, the model's accuracy has significantly dropped. How can this issue be resolved?

Except(Before convert):

except
except1920×1080 290 KB

Reality(After convert):

result
result1920×1080 211 KB

Post a console log of your issue below:


|---+ KSNN Version: v1.4 +---| 
Start init neural network ...
Done.
Get input data ...
Done.
Start inference ...
Done. inference time:  0.14252567291259766
class: black, score: 0.9248914122581482
box coordinate left,top,right,down: [1.2979914532351699e-05, 0.5121805174695353, 0.14087214256827352, 0.7768741957976658]
class: black, score: 0.5774953961372375
box coordinate left,top,right,down: [0.466750867045931, 0.4039581094152929, 0.5154227235715428, 0.4798818885531594]
class: blue, score: 0.7632191181182861
box coordinate left,top,right,down: [0.1632609927355418, 0.4736040097388326, 0.23753008806830564, 0.5739108720924456]
class: yellow, score: 0.7489954233169556
box coordinate left,top,right,down: [0.2471383326601398, 0.44844722907044454, 0.28604165968659745, 0.5034682774283155]
class: yellow, score: 0.6720089316368103
box coordinate left,top,right,down: [0.3117288200151421, 0.41255611916804613, 0.373750039175043, 0.5113955799955505]
Traceback (most recent call last):
  File "yolov8n-picture.py", line 252, in <module>
    cv.imshow("results", img)
cv2.error: OpenCV(4.11.0) /io/opencv/modules/core/src/array.cpp:2494: error: (-206:Bad flag (parameter or structure field)) Unrecognized or unsupported array type in function 'cvGetMat'

ExtraBackgroud

Model Conversion Platform: WSL2 Ubuntu 20.04
Model Conversion Parameters:
./convert --model-name eac_model \
  --platform onnx \
  --model best.onnx \
  --mean-values '0 0 0 0.00392156' \
  --quantized-dtype dynamic_fixed_point \
  --qtype int16 \
  --source-files ./test_data.txt \
  --kboard VIM3 --print-level 1
2

Here is my original model

origin model

3

@unnamed during the model training/finetuning please try QAT (Quantized Aware Training), It can provide significant improvement to accuracy compared to the quantization conversion done by the tool.

4

I modified the YOLOv8 training code to insert QuantStub and DeQuantStub for QAT.
However, when exporting with YOLO’s export method, the QAT nodes are not preserved.

Most online tutorials use PyTorch’s torch.onnx.export to retain QAT nodes (e.g., QuantizeLinear and DequantizeLinear).
However, Khadas documentation requires using YOLO’s built-in export method, which does not seem to retain QAT nodes in the exported ONNX model.

5

Maybe there is a work around for this, I’ll will take a look.

6

Have you got any progress?
Or can you give me some QAT example?

7

@unnamed I did come across this, but I haven’t been able to find a definite solution for you.