Multiple Model Execution Errors on NPU

VIM4
1

Which system do you use? Android, Ubuntu, OOWOW or others?

Ubuntu

Which version of system do you use? Please provide the version of the system here:

Ubuntu 22.04

Please describe your issue below:

I’ve been using your great examples to get useful models like YOLO working in VIM4.

To take it a step further, I envisioned running two YOLO models simultaneously.

I created two networks to make the model work, just as I did to make a single YOLO work.
At this point, the detection box was very unstable and bouncing around.

It was as if the two networks were interfering with each other, with one model’s result box popping into the other model’s result box.

Question
  • Should the NPU only operate on one model at a time?
  • How can I use NPU resources stably when running multiple models?

Post a console log of your issue below:

2

Hello @GHdevlog ,

Two model will not affect each other if they are same input. But if one model result determines the other model input, the former will affect the latter.

For model result stable, it depends more on the precision of the model. The higher precision, the higher the stability.

About how to improve two related models, usually train the first model and then use trained model to make second model training dataset. Moeover, according to workflow, you can add check method to filter out some errors.

3

Thank you for your response!

Just to clarify, I’m not running the two YOLO models sequentially — I’m trying to run them in parallel, at the same time.

Here’s a simplified outline of what I’m doing:

// Pseudo-code
thread1: run network1 with yolov5s
thread2: run network2 with yolov8n
// Both run in parallel

Each YOLO model is loaded into a separate network, and I execute both networks simultaneously using multithreading. The issue I’m seeing is that the detection boxes from each model are unstable and seem to interfere with each other, as if the NPU is not handling both tasks cleanly in parallel.

So my main questions are:

  • Does the NPU on the VIM4 support parallel execution of multiple models?
  • If not, is there a recommended approach for scheduling or switching between multiple models efficiently?
4

Hello @GHdevlog ,

NPU supports to run model parallel execution.

Have you ever run the two models separately? Still unstable?

The postprocess of YOLOv5 is different from YOLOv8. It is the same as YOLOv7. Have you modified the postprocess?