Accelerate Onnx Model

umutt · January 5, 2022, 12:00pm

Hello everyone,

I am trying to accelerate my onnx model. Now my speed is very slow. Is there anyone who faced this problem and solved this problem? If yes, can you explain the solution?

Frank · January 6, 2022, 1:08am

@umutt What model is your onnx, and how slow is it? What are your conversion parameters?

umutt · January 6, 2022, 11:48am

Hello Frank,

I am working on YOLOX + ByteTrack. On my computer, my solution works at about 15 fps. But when I worked on Edge-V Pro, codes work at about 1 fps. You can see the code that generates Onnx model from torch.

github.com

ifzhang/ByteTrack/blob/main/tools/export_onnx.py

from loguru import logger

import torch
from torch import nn

from yolox.exp import get_exp
from yolox.models.network_blocks import SiLU
from yolox.utils import replace_module

import argparse
import os


def make_parser():
    parser = argparse.ArgumentParser("YOLOX onnx deploy")
    parser.add_argument(
        "--output-name", type=str, default="bytetrack_s.onnx", help="output name of models"
    )
    parser.add_argument(
        "--input", default="images", type=str, help="input node name of onnx model"

This file has been truncated. show original

Frank · January 7, 2022, 12:57am

@umutt Edge don’t have NPU, you just run your model in CPU. Speed is very slow

umutt · January 10, 2022, 11:37am

@Frank which board should be preferred for Yolo models in Khadas Products?

Frank · January 11, 2022, 12:48am

@umutt Just vim3 and vim3l support NPU

umutt · January 11, 2022, 12:43pm

@Frank do you have any information about FPS? 15 fps is possible for Yolo- on vim3 or vim3l?

Frank · January 12, 2022, 1:15am

@umutt Both yolov2 and tiny series can reach 15FPS