Npu inference speed slows down after running for a while

VIM3
1

Which Khadas SBC do you use?

Khadas VIM3 Pro

Which system do you use? Android, Ubuntu, OOWOW or others?

Ubuntu

Which version of system do you use? Khadas official images, self built images, or others?

vim3-ubuntu-20.04-gnome-linux-4.9-fenix-1.1.1-220725-emmc.img

Please describe your issue below:

We run a large object detection model on images and after about 60 inferences, the inference time went from 1 sec/images to 60 sec/images and we got some messages in logs via dmesg.

Post a console log of your issue below:

image
image765×671 151 KB

2

@JJ1997 This is that the memory allocated to the NPU is not enough, maybe your model is too large

3

Hello @Frank,

Thanks for the quick response. I thought of this as well. I am still a bit confused tho: It would mean the memory needs for the NPU increases over time, if so why and is there a limit ?
It is indeed a large model, but since I successfully load the model and infer at an acceptable frame rate for around 60 frame before the issue occures, do you have an idea why suddenly it would not be enough anymore ? I checked my code and added garbage collection to clear the RAM as I go so I can garanty free memory available but the issue still occures.

Thanks in advance,

4

I think this is caused by memory fragmentation in your post-processing code.From the log I can’t analyze the reason