Npu inference speed slows down after running for a while

Which Khadas SBC do you use?

Khadas VIM3 Pro

Which system do you use? Android, Ubuntu, OOWOW or others?


Which version of system do you use? Khadas official images, self built images, or others?


Please describe your issue below:

We run a large object detection model on images and after about 60 inferences, the inference time went from 1 sec/images to 60 sec/images and we got some messages in logs via dmesg.

Post a console log of your issue below:

@JJ1997 This is that the memory allocated to the NPU is not enough, maybe your model is too large

Hello @Frank,

Thanks for the quick response. I thought of this as well. I am still a bit confused tho: It would mean the memory needs for the NPU increases over time, if so why and is there a limit ?
It is indeed a large model, but since I successfully load the model and infer at an acceptable frame rate for around 60 frame before the issue occures, do you have an idea why suddenly it would not be enough anymore ? I checked my code and added garbage collection to clear the RAM as I go so I can garanty free memory available but the issue still occures.

Thanks in advance,

I think this is caused by memory fragmentation in your post-processing code.From the log I can’t analyze the reason