Simple NPU demo on VIM3 Ubuntu Bionic

numbqq · September 12, 2019, 8:17am

Preparation

You need to flash ubuntu image V190830.

Update kernel

You need to update the kernel to add NPU module support.

khadas@Khadas:~$ sudo -i
root@Khadas:~# apt-get update
root@Khadas:~# apt-get upgrade
root@Khadas:~# echo galcore >> /etc/modules
root@Khadas:~# sync
root@Khadas:~# reboot

After reboot, you can check whether the NPU module exist or not.

root@Khadas:~# lsmod | grep galcore
galcore               442368  0

If you find the galcore module, indicate that the NPU module is loaded.

Install NPU libraries and demo

khadas@Khadas:~$ sudo apt-get update
khadas@Khadas:~$ sudo apt-get install aml-npu aml-npu-demo

There is a Inception V3 demo, you can run it.

khadas@Khadas:~$ sudo -i
root@Khadas:~# cd /usr/share/npu/inceptionv3
root@Khadas:/usr/share/npu/inceptionv3# ./run.sh
Create Neural Network: 82ms or 82537us
Verify...



********pid is 136 *********


Verify Graph: 181ms or 181495us
Start run graph [1] times...
Run the 1 time: 22ms or 22021us
vxProcessGraph execution time:
Total   22ms or 22050us
Average 22.00ms or 22050.00us
 --- Top5 ---
  2: 0.897461
795: 0.004154
974: 0.001733
408: 0.001349
399: 0.001191

You can check the result with labels in imagenet_slim_labels.txt .

In the End

We are working on the documentation and model convert tools, and we will release them soon.

Thanks.

blitzfr1 · September 12, 2019, 12:22pm

great!

will you support accelerated NPU for object detection too ? (with Tensorflow)

numbqq · September 12, 2019, 1:50pm

Yes, but we need to wait for the camera.

Need to convert the TF model to case code.

blitzfr1 · September 12, 2019, 2:08pm

I am waiting for the camera too any ETA ?

Do you realise that if you can deliver 5 TOPS NPU + CS mount 1080p@60FPS, the VIM3 will be hands down the best SBC.

pitchaya · September 18, 2019, 4:30pm

Thank you very much. The performance looks very promising.
Will there be document for a bit lower level library as well. I notice that the example use library from ovxlib. I do not mind interfacing with it directly.

pitchaya · September 27, 2019, 5:30am

Hi,
Based from the demo program, I am able to write a code to create my own graph and execute a simple convolution operations (VSI_NN_OP_CONV_RELU) . However, the parameter of the operation seems to be coded in a special format (VSI_NN_TYPE_VDATA) and I do not know how to make one myself. It seems to compress somehow. Right now, I pulled the data out from the demo program and used it in my testing.

While I understand that there is no model conversion tool, yet, I don’t mind writing my own tool to convert the network myself if there is a way to create the parameters (VDATA). Would it be possible for you to provide that ?

Thanks!!

Frank · September 28, 2019, 1:24am

@pitchaya

Yes, we need conversion tools to achieve conversion. We have been preparing the relevant content, and we need some time to prepare all the tools and documents we need, and then we will open them to users.

alcohol · October 18, 2019, 5:51pm

hi Frank, is there any new progress of the Big NPU Package?

Thanks.

Gouwa · October 18, 2019, 6:15pm

Hi @alcohol:
Please sent email to hello@khadas.com and I will sent to you first.

Good day!

alcohol · October 22, 2019, 5:12pm

Thanks for your attention, an email with my order number has been sent out.

Gouwa · October 22, 2019, 5:42pm

Replied already, I just noticed you sent two emails

ahmetkemal · October 22, 2019, 9:06pm

Thanks Gouwa, its done ,very useful

Ehsan · May 15, 2021, 11:48pm

Hi,

I compiled case code for inception demo with ndk-build (v17 and v21). Then I push the vnn_inceptionv3 file into khadas vim3 (Android OS). When I run it give error:
CANNOT LINK EXECUTABLE “./vnn_inceptionv3”: cannot locate symbol “vxCreateTensorFromHandle2” referenced by “/vendor/lib/libovxlib.so”…

Why it cannot locate symbol?
Does my libovxlib.so has problem? where is the original and working libovxlib.so file located ? (I push it from aml_npu_sdk_6.4.3/Android-sdk/lib/libovxlib.so to /vendor/lib)
Best,
Ehsan

Frank · May 17, 2021, 1:30am

@Ehsan @jasonl Please help about this.

xingye666 · May 20, 2021, 6:39am

sudo apt-get install aml-npu aml-npu-demo

when i work at root mode ,It did not work

E:unable to locate package aml_npu_demo
khadas@Khadas:~$ sudo apt-get install aml-npu aml-npu-demo

numbqq · May 20, 2021, 6:41am

Why you do this ? Which image you used? You just need to follow this docs to upgrade your system, you don’t need to install the npu package manually.

https://docs.khadas.com/vim3/HowToUpgradeTheSystem.html

xingye666 · May 20, 2021, 7:55am

I did as you said,but it could do nothing

numbqq · May 20, 2021, 7:56am

Which image you used?

xingye666 · May 20, 2021, 8:00am

the first or third one , I can not remember it clearly

numbqq · May 20, 2021, 8:01am

Please use this one:

https://dl.khadas.com/Firmware/VIM3/Ubuntu/EMMC/VIM3_Ubuntu-gnome-focal_Linux-4.9_arm64_EMMC_V1.0.5-210430.img.xz

And update to latest version: