NPU Demo and source code

Dhruv_Gaba · April 20, 2021, 4:14pm

@Frank thanks for the GPU clarification.

Dhruv_Gaba · April 20, 2021, 4:55pm

@Frank I ran the following commands and no errors came, but when I run my model I don’t see any logs of bandwidth as explained in the docs

rmmod galcore
cd /lib/modules/4.9.241.kernel/drivers/amlogic/npu
sudo insmod galcore.ko gpuProfiler=1 showArgs=1
export VIV_VX_PROFILE=1
export VIV_VX_DEBUG_LEVEL=1
cd {workspace}/aml_npu_demo_binaries/detect_demo_picture/
sudo ./INSTALL
./detect_demo 0 1080.bmp

But it doesn’t show any logs with the bandwidth in the terminal??
is there anything I am doing wrong??

Dhruv_Gaba · April 20, 2021, 5:53pm

also, if I try to do rmmod galcore I get the following

Frank · April 21, 2021, 1:22am

@Dhruv_Gaba You need to check the background processes, which processes are using NPU, please terminate these processes

smartadpole · April 21, 2021, 9:41am

看起来像是显示有问题，如果是远程连接，可以加上参数 ssh -X

Dhruv_Gaba · April 24, 2021, 2:14am

@Frank the command worked for me.

I can get the read and write bandwidth information

Can you please tell what does the total, axi and ddr bandwidths mean??
And, how can I calculate the percentage utilization of the npu from this ??

RKroells · April 25, 2021, 6:19pm

@Frank please answer my question how to add external urls to test with the npu demo

Frank · April 26, 2021, 1:01am

@RKroells I don’t know how to add external urls . I am not familiar with web programming and have never used external urls. The code is open source, you can see all the content

RKroells · April 26, 2021, 5:32am

So the vim3 will only work with usb camera?
This cannot be.

Please contact your team to provide thisinformation

numbqq · April 26, 2021, 5:38am

It should be common solution, has nothing to do with the NPU, so please google for it yourself.

Frank · April 26, 2021, 5:45am

@RKroells The NPU demo just provides a template for you to use the NPU interface. It has nothing to do with how the application layer processes it. If you want to use a web camera, or other cameras, or other ways to transfer data, you can implement it yourself, which has nothing to do with the NPU.

RKroells · April 26, 2021, 6:16am

So what is this forum for?

Where would I find the required information as
Google is not help me.

One of the key selling points of your vim3 is the npu, so adding an external cam your should be included in your documentation, as you can’t expect every user to buy a 100% compatible camera.

Dhruv_Gaba · April 26, 2021, 3:12pm

@Frank Any suggestions on understanding the NPU Utilization? I can see TOTAL_READ_BANDWIDTH and TOTAL_WRITE_BANDWIDTH printed into the terminal logs for my model.
What is the maximum of this BANDWIDTH for the VIM3 board? so that I can calcualte the percentage?

For E.g. for my model it is ~ 50 MByte (TOTAL_READ_BANDWIDTH+ TOTAL_WRITE_BANDWIDTH)

And in the Khadas Documentation it says to subtract OCR_BANDWIDTH but I am not able too that being printed onto my screen. Please help!

Frank · April 27, 2021, 1:07am

@Dhruv_Gaba In the document, the content of this part has not been updated yet, I will consult the relevant part, and then feedback to you.

In my understanding, total here is the total time, axi is the time to configure the model layer, and ddr time is the time to input data and obtain results. I’m not sure, I will confirm a correct result and feedback to you

Vignesh_Raja · April 27, 2021, 2:45pm

@Frank I got questions about the capabilities of the NPU. Everything in the NPU demo works well for me. I would like to know

If NPU can handle more processing techniques like semantic segmentation besides the object classification and detection? I could not find anything on this.
Is there any performance bench marking among different neural networks/topologies? Since NPU is 5 TOPS, it would be interesting to know if it can do better or on par with other edge AI devices as well.
How much CPU will be consumed when running operations on NPU?

Thanks in advance.

Dhruv_Gaba · April 27, 2021, 3:01pm

Sure, @Frank I am eagerly waiting for your response.

Dhruv_Gaba · April 27, 2021, 4:23pm

@Frank What is the **AXI and DDR clock frequencies maximum (e.g. DDR data transfer max frequency and AXI data transfer frequency maximum) **? I think if we know the maximum we can calculate the percentage utilization of the NPU.
Also, after reading on AXI I found that it is the Data transfer rate between the internal blocks/components on the SOC board and it is called Advanced eXtensible Interface. Which would be used for the processing of the model on the NPU
And, for DDR it’s the read and write of data speed (e.g. Mbytes/second) with the ram of the board. Which would be used to input and output data from the ram memory of the board.
Thanks for the hint, will be waiting for your response.

Frank · April 28, 2021, 1:11am

@Dhruv_Gaba I have new news, I will feed back here

Frank · April 28, 2021, 1:17am

@Vignesh_Raja At present, it is mainly in the CV direction. The input after the model conversion is picture information, so as long as the input is a picture model, it is theoretically possible. The model in the NLP direction is not possible with a high probability

We have the documentation for the supported layers, but there is no description of the relevant performance, you can find it in the SDK

This is related to your model. You can view the background process and see the CPU usage during the running process.

Vignesh_Raja · April 28, 2021, 7:29am

Thanks for the clear explanation. It would be great to see more future developments in the NPU area.