KSNN convert Yolov5s

Agent_kapo · December 7, 2023, 3:27pm

I’m using Vim3 Pro model. Decided to move from TEngine framework to ksnn.
I’ve cloned ksnn git and aml_npu_sdk git
but I cant understand what params I should give to convertor:

$ .convert \

--model-name detector --platform pytorch \ (OK)

--model ./yolov5s.pt \ (OK)

--input-size-list '3,640,640' --inputs input \ (OK)

--mean-values '103.94 116.78 123.68 0.01700102' \ (OK)

--quantized-dtype asymmetric_affine \ (OK)

--source-files ./data/dataset/dataset0.txt \ (Whats inside of txt file?)

--kboard VIM3 --print-level 1 (And what is it for?)

Also in few topics I saw that .pt weights doesn’t work and you should use .onnx is that right?

Electr1 · December 8, 2023, 4:19am

Hello @Agent_kapo

It needs to contain the relative path of a sample image for the model conversion.
Content can be as simple as ./sample.png, where sample.png is just any simple image to test the model.

This is for printing the diagnostic information of the model conversion parameters, just more verbose during the conversion process.

Yes, you can use the ONNX model to do the conversion it will work.

Agent_kapo · December 8, 2023, 6:37am

Thank you very much, Im bit confused because inside onnx folder is resnet50.py file, I need to use this file?

And why I cant use pytorch?

Agent_kapo · December 8, 2023, 6:41am

I mean the resnet50.py is to convert my yolov5s.onnx model into .nb file? The file name doesnt mean anything?

Electr1 · December 8, 2023, 6:47am

@Agent_kapo you need to look at the docs regarding conversion, you are looking at the ksnn examples.

the conversion method is here:
https://docs.khadas.com/products/sbc/vim3/npu/ksnn/ksnn-convert

Agent_kapo · December 8, 2023, 9:38am

No, I understand how to convert)
After the convert operation you get model.nb libnn_model.so

But how to run it?
python3 resnet50.py \ (This file contains ksnn library and I don’t understand - can I use yours, or should I make mine?)
--model ./models/VIM3/model.nb (OK, I got it in last step)
--library ./libs/libnn_model.so (OK, I got it in last step)
--picture pic.jpg --level 0 (OK, just parametr)
Here I dont understand

I bit confused about what to do with this file

github.com

khadas/ksnn/blob/master/examples/onnx/resnet50.py

import numpy as np
import os      
import argparse
import sys
from ksnn.api import KSNN
from ksnn.types import *
import cv2 as cv
import time

def show_top5(output):
    output_sorted = sorted(output, reverse=True)
    top5_str = '----Resnet50----\n-----TOP 5-----\n'
    for i in range(5):
        value = output_sorted[i]
        index = np.where(output == value)
        for j in range(len(index)):
            if (i + j) >= 5:
                break
            if value > 0:
                topi = '{}: {}\n'.format(index[j], value)

This file has been truncated. show original

If I need to create my own, how I should do it?

Electr1 · December 8, 2023, 10:10am

@Agent_kapo that is an example script to use the resnet model, for yolov5s, you need to make a similar kind of script that does the input and output processing,

You can refer to the resnet example on how to load the KSNN ONNX converted model, and you can write your own post-processing to get the results from the output tensor data.

You can follow this example code to do post-processing from a yolo model:

github.com

sravansenthiln1/armnn_tflite/blob/main/yolov8n/run_inference.py#L63-L144


      
          def postprocess(output, confidence_threshold=0.5, iou_threshold=0.5):
              # Extracting the output tensor and arranging it for further processing
              outputs = np.transpose(np.squeeze(output[0]))
          
          
    # Get the number of rows in the output tensor
              rows = outputs.shape[0]
          
          
    # Lists to store information about each detected object
              boxes = []
              scores = []
              class_ids = []
          
          
    # Extract factors for scaling back to the original image size
              x_factor, y_factor = IMGSZ
          
          
    # Iterate through each row in the output tensor
              for i in range(rows):
                  # Extract scores for each class for the current row
                  classes_scores = outputs[i][4:]

This file has been truncated. show original

Agent_kapo · December 8, 2023, 12:23pm

As I understand - I take script for loading .nb model from resnet50.py and then I take the part from run_inference.py (of yolov8n) to upload the results?

Agent_kapo · December 8, 2023, 1:05pm

I just found this Yolov7 implementation in your github, as far I understood this topic, I can use it for Yolov5s, am I correct?

github.com

khadas/ksnn/blob/master/examples/yolov7_tiny/yolov7_tiny-picture.py

import numpy as np
import os
import urllib.request
from matplotlib import gridspec
from matplotlib import pyplot as plt
from PIL import Image
import argparse
import sys
import math
from ksnn.api import KSNN
from ksnn.types import *
import cv2 as cv
import time

GRID0 = 20
GRID1 = 40
GRID2 = 80
LISTSIZE = 85
SPAN = 3
NUM_CLS = 80

This file has been truncated. show original

numbqq · December 9, 2023, 1:35am

Hello @Agent_kapo

@Louis-Cheng-Liu will help you then.

Agent_kapo · December 9, 2023, 10:46am

Okay, Im waiting for the response

Louis-Cheng-Liu · December 11, 2023, 1:21am

Hello @Agent_kapo ,

This YOLOv7_Tiny demo is the version of bounding boxes. If your model does not use bounding boxes, this demo you can not use.

We are plannning to add YOLOv8n KSNN demo. This is the non-bounding boxes version. You should wait some weeks.

If you do not know whether use or not, you can provide your ONNX model for me. I will check.

Louis-Cheng-Liu · December 22, 2023, 10:22am

Hello @Agent_kapo ,

YOLOv8 KSNN demo is released. This is the non-bounding boxes version. You can try it.
khadas/ksnn: Khadas Software Neural Network (github.com)

Docs
YOLOv8n KSNN Demo - 2 [Khadas Docs]

Agent_kapo · December 22, 2023, 2:35pm

Thank you very much!
But I’m also interested what is ListSize and Span parameters are:

Louis-Cheng-Liu · December 25, 2023, 2:35am

Hello @Agent_kapo ,

Model’s output are [1, 144, 80, 80], [1, 144, 40, 40] and [1, 144, 20, 20]. You can simply understand that they ony match the dimension number. The SPAN is the number of boxes for each based prediction. The LISTSIZE is each box information number. [:80] is the information of classes confidence. [80:] is the information of location.

So, if your model’s classes is not 80, you need to modify the LISTSIZE.

LISTSIZE = classes + 64

Agent_kapo · December 25, 2023, 7:04pm

Thank you very much!