Inference Profiling

Technical Support
1

Are there any official/open-source softwares for profiling the NPU provided by khadas VIM3 board?
The logs generated during inference (level 2) are too less for understanding the profiles. Can we get the same data on a layer basis? (i.e. time spent on each layer, number of MAC ops per second, etc.)

2

There’s a method to profile layer-wise inference time using the NPU SDK (using C++). Is there any support for python as well?

3

@johndoe This method does not support python, at least not currently, I will try to support

1 Like