Do you have a document on measuring npu performance?

General Discussion
1

base Khadas Edge2
Is there any data comparing the difference in inference speed between other models (pt, tflite …etc) and models converted to rknn?

2

Hello @re_roy

We have some tflite benchmark comparisons against CPU, GPU and RKNN for Edge2.

These are the examples compared.

Sine model

CPU : Execution time: 0.29 ms.
GPU : Execution time: 3.10 ms.
NPU : Execution time: 0.64 ms.

Digit recognization model

CPU : Execution time: 0.51 ms.
GPU : Execution time: 1.46 ms.
NPU : Execution time: 0.67 ms.

Audio classification model

CPU: Execution time: 44.03 ms.
GPU: Execution time: 22.36 ms.
NPU: Execution time: 5.10 ms.

Mobilenet quant v1 model

CPU : Execution time: 60.53 ms.
GPU : Execution time: 21.30 ms.
NPU : Execution time: 2.920 ms.

Yolov8n model

CPU : Execution time: 206.60 ms
GPU: Execution time: 216.82 ms
NPU: Execution time: 106.84 ms

Regards.

3

Thank you !!
You’ve been a great help.

4

Did you apply quantization to the rknn model test?

5

@re_roy except the mobilenet model, none of the other models are quantized.