NPU Demo and source code

There are some ready to use NPU demo binaries for VIM3/VIM3L.

This documentation is about how to run these demos on VIM3/VIM3L.

Prepare

You need to run Ubuntu image V20190830 on board, and update the system.

Note: If the image is V20191231 or newer, you don’t need to do this anymore.

khadas@Khadas:~$ sudo apt-get update
khadas@Khadas:~$ sudo apt-get upgrade
khadas@Khadas:~$ sudo apt-get install -y gstreamer1.0-libav gstreamer1.0-plugins-bad gstreamer1.0-tools gstreamer1.0-plugins-base gstreamer1.0-alsa gstreamer1.0-plugins-good libconfig++9v5 libconfig9
khadas@Khadas:~$ sudo apt-get install -y aml-npu
khadas@Khadas:~$ sudo apt-get install -y gstreamer-aml libplayer-aml opencv3
khadas@Khadas:~$ sync
khadas@Khadas:~$ sudo reboot

Note: You can choose server image to run these demos. If you choose desktop image, you need to switch to the framebuffer console by Ctrl+Alt+F1 to run these demos.

Download NPU demo binaries

khadas@Khadas:~$ sudo -i
root@Khadas:~# git clone https://gitlab.com/khadas/aml_npu_demo_binaries --depth=1

Directory description:

  • detect_demo - Yoloface/YoloV2/YoloV3 demo with video
  • detect_demo_picture - Yoloface/YoloV2/YoloV3 demo with picture
  • inceptionv3 - Inceptionv3

Run Yolo demo with USB camera

Install the libraries:

root@Khadas:~# cd aml_npu_demo_binaries/detect_demo
root@Khadas:~/aml_npu_demo_binaries/detect_demo# ./INSTALL

Run UVC demo:

root@Khadas:~/aml_npu_demo_binaries/detect_demo# ./detect_demo_uvc /dev/videoX 1 > /dev/null

Note: Replace /dev/videoX to the correct node.

You can run ./detect_demo_uvc to get help information and read README to get more information.

Tested USB camera:

  • Logitech C920 PRO
  • Logitech C525

Run Yolo demo with MIPI camera

root@Khadas:~/aml_npu_demo_binaries/detect_demo# ./detect_demo_mipi /dev/videoX  1 > /dev/null

Note: MIPI camera for VIM3 has not launched yet.

Source Code

The source code of the demos:

In The End

The performance of the video display still need to improve.

8 Likes

Thank you @numbqq . I was looking out for this.
I’ll check it out when I find the time and make a video about it.
Just got the NVIDIA Jetson Nano. So could be nice to do a comparison of both.
The Nano may have a good GPU, but its CPU is mighty slow. Can’t be said about the VIM3.
Greetings.

I got the following error when I run detect_demo in the detect_demo_picture directory:

./detect_demo: error while loading shared libraries: libgtk-3.so.0: cannot open shared object file: No such file or directory

Where can I install libgtk-3.so?

Can you provide this for me? ldd detect_demo ?

    linux-vdso.so.1 (0x0000007f89f97000)
    libovxlib.so => /lib/libovxlib.so (0x0000007f89e53000)
    libm.so.6 => /lib/aarch64-linux-gnu/libm.so.6 (0x0000007f89d99000)
    libpthread.so.0 => /lib/aarch64-linux-gnu/libpthread.so.0 (0x0000007f89d6d000)
    libdl.so.2 => /lib/aarch64-linux-gnu/libdl.so.2 (0x0000007f89d58000)
    libz.so.1 => /lib/aarch64-linux-gnu/libz.so.1 (0x0000007f89d2b000)
    libopencv_imgproc.so.3.4 => /home/khadas/USBDrive/nanopi/opencv/build/lib/libopencv_imgproc.so.3.4 (0x0000007f8993b000)
    libopencv_core.so.3.4 => /home/khadas/USBDrive/nanopi/opencv/build/lib/libopencv_core.so.3.4 (0x0000007f895e5000)
    libopencv_videoio.so.3.4 => /home/khadas/USBDrive/nanopi/opencv/build/lib/libopencv_videoio.so.3.4 (0x0000007f89593000)
    libopencv_imgcodecs.so.3.4 => /home/khadas/USBDrive/nanopi/opencv/build/lib/libopencv_imgcodecs.so.3.4 (0x0000007f89397000)
    libopencv_highgui.so.3.4 => /home/khadas/USBDrive/nanopi/opencv/build/lib/libopencv_highgui.so.3.4 (0x0000007f89372000)
    libnn_detect.so => /usr/lib/libnn_detect.so (0x0000007f8935e000)
    libstdc++.so.6 => /usr/lib/aarch64-linux-gnu/libstdc++.so.6 (0x0000007f891cb000)
    librt.so.1 => /lib/aarch64-linux-gnu/librt.so.1 (0x0000007f891b4000)
    libgcc_s.so.1 => /lib/aarch64-linux-gnu/libgcc_s.so.1 (0x0000007f89190000)
    libc.so.6 => /lib/aarch64-linux-gnu/libc.so.6 (0x0000007f89037000)
    libOpenVX.so => /lib/libOpenVX.so (0x0000007f88eab000)
    libOpenVXU.so => /lib/libOpenVXU.so (0x0000007f88e95000)
    libCLC.so => /lib/libCLC.so (0x0000007f88c84000)
    libVSC.so => /lib/libVSC.so (0x0000007f87dde000)
    /lib/ld-linux-aarch64.so.1 (0x0000007f89f6c000)
    libavcodec.so.57 => /usr/lib/aarch64-linux-gnu/libavcodec.so.57 (0x0000007f86b98000)
    libavformat.so.57 => /usr/lib/aarch64-linux-gnu/libavformat.so.57 (0x0000007f86961000)
    libavutil.so.55 => /usr/lib/aarch64-linux-gnu/libavutil.so.55 (0x0000007f868d1000)
    libswscale.so.4 => /usr/lib/aarch64-linux-gnu/libswscale.so.4 (0x0000007f8685d000)
    libjpeg.so.8 => /usr/lib/aarch64-linux-gnu/libjpeg.so.8 (0x0000007f86813000)
    libpng16.so.16 => /usr/lib/aarch64-linux-gnu/libpng16.so.16 (0x0000007f867d8000)
    libtiff.so.5 => /usr/lib/aarch64-linux-gnu/libtiff.so.5 (0x0000007f8675d000)
    libgtk-3.so.0 => /usr/lib/aarch64-linux-gnu/libgtk-3.so.0 (0x0000007f86101000)
    libgdk-3.so.0 => /usr/lib/aarch64-linux-gnu/libgdk-3.so.0 (0x0000007f86019000)
    libcairo.so.2 => /usr/lib/aarch64-linux-gnu/libcairo.so.2 (0x0000007f85f1f000)
    libgdk_pixbuf-2.0.so.0 => /usr/lib/aarch64-linux-gnu/libgdk_pixbuf-2.0.so.0 (0x0000007f85ef0000)
    libgobject-2.0.so.0 => /usr/lib/aarch64-linux-gnu/libgobject-2.0.so.0 (0x0000007f85e92000)
    libglib-2.0.so.0 => /usr/lib/aarch64-linux-gnu/libglib-2.0.so.0 (0x0000007f85d84000)
    libGAL.so => /lib/libGAL.so (0x0000007f85c00000)
    libLLVM_viv.so => /lib/libLLVM_viv.so (0x0000007f85768000)
    libswresample.so.2 => /usr/lib/aarch64-linux-gnu/libswresample.so.2 (0x0000007f85740000)
    libwebp.so.6 => /usr/lib/aarch64-linux-gnu/libwebp.so.6 (0x0000007f856e8000)
    libva.so.2 => /usr/lib/aarch64-linux-gnu/libva.so.2 (0x0000007f856b9000)
    libzvbi.so.0 => /usr/lib/aarch64-linux-gnu/libzvbi.so.0 (0x0000007f8562a000)
    libxvidcore.so.4 => /usr/lib/aarch64-linux-gnu/libxvidcore.so.4 (0x0000007f85543000)
    libx265.so.146 => /usr/lib/aarch64-linux-gnu/libx265.so.146 (0x0000007f852e5000)
    libx264.so.152 => /usr/lib/aarch64-linux-gnu/libx264.so.152 (0x0000007f8517f000)
    libwebpmux.so.3 => /usr/lib/aarch64-linux-gnu/libwebpmux.so.3 (0x0000007f85166000)
    libwavpack.so.1 => /usr/lib/aarch64-linux-gnu/libwavpack.so.1 (0x0000007f85134000)
    libvpx.so.5 => /usr/lib/aarch64-linux-gnu/libvpx.so.5 (0x0000007f84f89000)
    libvorbisenc.so.2 => /usr/lib/aarch64-linux-gnu/libvorbisenc.so.2 (0x0000007f84eda000)
    libvorbis.so.0 => /usr/lib/aarch64-linux-gnu/libvorbis.so.0 (0x0000007f84ea4000)
    libtwolame.so.0 => /usr/lib/aarch64-linux-gnu/libtwolame.so.0 (0x0000007f84e73000)
    libtheoraenc.so.1 => /usr/lib/aarch64-linux-gnu/libtheoraenc.so.1 (0x0000007f84e31000)
    libtheoradec.so.1 => /usr/lib/aarch64-linux-gnu/libtheoradec.so.1 (0x0000007f84e0a000)
    libspeex.so.1 => /usr/lib/aarch64-linux-gnu/libspeex.so.1 (0x0000007f84de5000)
    libsnappy.so.1 => /usr/lib/aarch64-linux-gnu/libsnappy.so.1 (0x0000007f84dcd000)
    libshine.so.3 => /usr/lib/aarch64-linux-gnu/libshine.so.3 (0x0000007f84db3000)
    librsvg-2.so.2 => /usr/lib/aarch64-linux-gnu/librsvg-2.so.2 (0x0000007f84d73000)
    libopus.so.0 => /usr/lib/aarch64-linux-gnu/libopus.so.0 (0x0000007f84d26000)
    libopenjp2.so.7 => /usr/lib/aarch64-linux-gnu/libopenjp2.so.7 (0x0000007f84ccb000)
    libmp3lame.so.0 => /usr/lib/aarch64-linux-gnu/libmp3lame.so.0 (0x0000007f84c4f000)
    libgsm.so.1 => /usr/lib/aarch64-linux-gnu/libgsm.so.1 (0x0000007f84c35000)
    liblzma.so.5 => /lib/aarch64-linux-gnu/liblzma.so.5 (0x0000007f84c05000)
    libssh-gcrypt.so.4 => /usr/lib/aarch64-linux-gnu/libssh-gcrypt.so.4 (0x0000007f84b8d000)
    libopenmpt.so.0 => /usr/lib/aarch64-linux-gnu/libopenmpt.so.0 (0x0000007f849c5000)
    libbluray.so.2 => /usr/lib/aarch64-linux-gnu/libbluray.so.2 (0x0000007f8496e000)
    libgnutls.so.30 => /usr/lib/aarch64-linux-gnu/libgnutls.so.30 (0x0000007f84811000)
    libxml2.so.2 => /usr/lib/aarch64-linux-gnu/libxml2.so.2 (0x0000007f84672000)
    libgme.so.0 => /usr/lib/aarch64-linux-gnu/libgme.so.0 (0x0000007f84619000)
    libchromaprint.so.1 => /usr/lib/aarch64-linux-gnu/libchromaprint.so.1 (0x0000007f845f7000)
    libbz2.so.1.0 => /lib/aarch64-linux-gnu/libbz2.so.1.0 (0x0000007f845d7000)
    libX11.so.6 => /usr/lib/aarch64-linux-gnu/libX11.so.6 (0x0000007f844ae000)
    libdrm.so.2 => /usr/lib/aarch64-linux-gnu/libdrm.so.2 (0x0000007f8448f000)
    libvdpau.so.1 => /usr/lib/aarch64-linux-gnu/libvdpau.so.1 (0x0000007f8447b000)
    libva-x11.so.2 => /usr/lib/aarch64-linux-gnu/libva-x11.so.2 (0x0000007f84464000)
    libva-drm.so.2 => /usr/lib/aarch64-linux-gnu/libva-drm.so.2 (0x0000007f84451000)
    libjbig.so.0 => /usr/lib/aarch64-linux-gnu/libjbig.so.0 (0x0000007f84434000)
    libgmodule-2.0.so.0 => /usr/lib/aarch64-linux-gnu/libgmodule-2.0.so.0 (0x0000007f84420000)
    libpangocairo-1.0.so.0 => /usr/lib/aarch64-linux-gnu/libpangocairo-1.0.so.0 (0x0000007f84404000)
    libXi.so.6 => /usr/lib/aarch64-linux-gnu/libXi.so.6 (0x0000007f843e4000)
    libXfixes.so.3 => /usr/lib/aarch64-linux-gnu/libXfixes.so.3 (0x0000007f843ce000)
    libcairo-gobject.so.2 => /usr/lib/aarch64-linux-gnu/libcairo-gobject.so.2 (0x0000007f843b5000)
    libatk-1.0.so.0 => /usr/lib/aarch64-linux-gnu/libatk-1.0.so.0 (0x0000007f84381000)
    libatk-bridge-2.0.so.0 => /usr/lib/aarch64-linux-gnu/libatk-bridge-2.0.so.0 (0x0000007f84343000)
    libepoxy.so.0 => /usr/lib/aarch64-linux-gnu/libepoxy.so.0 (0x0000007f8424f000)
    libpangoft2-1.0.so.0 => /usr/lib/aarch64-linux-gnu/libpangoft2-1.0.so.0 (0x0000007f8422a000)
    libpango-1.0.so.0 => /usr/lib/aarch64-linux-gnu/libpango-1.0.so.0 (0x0000007f841d3000)
    libfontconfig.so.1 => /usr/lib/aarch64-linux-gnu/libfontconfig.so.1 (0x0000007f84183000)
    libgio-2.0.so.0 => /usr/lib/aarch64-linux-gnu/libgio-2.0.so.0 (0x0000007f84007000)
    libXinerama.so.1 => /usr/lib/aarch64-linux-gnu/libXinerama.so.1 (0x0000007f83ff4000)
    libXrandr.so.2 => /usr/lib/aarch64-linux-gnu/libXrandr.so.2 (0x0000007f83fda000)
    libXcursor.so.1 => /usr/lib/aarch64-linux-gnu/libXcursor.so.1 (0x0000007f83fbf000)
    libXcomposite.so.1 => /usr/lib/aarch64-linux-gnu/libXcomposite.so.1 (0x0000007f83fac000)
    libXdamage.so.1 => /usr/lib/aarch64-linux-gnu/libXdamage.so.1 (0x0000007f83f99000)
    libxkbcommon.so.0 => /usr/lib/aarch64-linux-gnu/libxkbcommon.so.0 (0x0000007f83f4f000)
    libwayland-cursor.so.0 => /usr/lib/aarch64-linux-gnu/libwayland-cursor.so.0 (0x0000007f83f37000)
    libwayland-egl.so.1 => /usr/lib/aarch64-linux-gnu/libwayland-egl.so.1 (0x0000007f83f25000)
    libwayland-client.so.0 => /usr/lib/aarch64-linux-gnu/libwayland-client.so.0 (0x0000007f83f06000)
    libXext.so.6 => /usr/lib/aarch64-linux-gnu/libXext.so.6 (0x0000007f83ee6000)
    libpixman-1.so.0 => /usr/lib/aarch64-linux-gnu/libpixman-1.so.0 (0x0000007f83e82000)
    libfreetype.so.6 => /usr/lib/aarch64-linux-gnu/libfreetype.so.6 (0x0000007f83dd9000)
    libxcb-shm.so.0 => /usr/lib/aarch64-linux-gnu/libxcb-shm.so.0 (0x0000007f83dc6000)
    libxcb.so.1 => /usr/lib/aarch64-linux-gnu/libxcb.so.1 (0x0000007f83d96000)
    libxcb-render.so.0 => /usr/lib/aarch64-linux-gnu/libxcb-render.so.0 (0x0000007f83d79000)
    libXrender.so.1 => /usr/lib/aarch64-linux-gnu/libXrender.so.1 (0x0000007f83d60000)
    libffi.so.6 => /usr/lib/aarch64-linux-gnu/libffi.so.6 (0x0000007f83d48000)
    libpcre.so.3 => /lib/aarch64-linux-gnu/libpcre.so.3 (0x0000007f83cd6000)
    libsoxr.so.0 => /usr/lib/aarch64-linux-gnu/libsoxr.so.0 (0x0000007f83c75000)
    libnuma.so.1 => /usr/lib/aarch64-linux-gnu/libnuma.so.1 (0x0000007f83c55000)
    libogg.so.0 => /usr/lib/aarch64-linux-gnu/libogg.so.0 (0x0000007f83c3e000)
    libcroco-0.6.so.3 => /usr/lib/aarch64-linux-gnu/libcroco-0.6.so.3 (0x0000007f83bfa000)
    libgcrypt.so.20 => /lib/aarch64-linux-gnu/libgcrypt.so.20 (0x0000007f83b3f000)
    libgssapi_krb5.so.2 => /usr/lib/aarch64-linux-gnu/libgssapi_krb5.so.2 (0x0000007f83aec000)
    libmpg123.so.0 => /usr/lib/aarch64-linux-gnu/libmpg123.so.0 (0x0000007f83a8f000)
    libvorbisfile.so.3 => /usr/lib/aarch64-linux-gnu/libvorbisfile.so.3 (0x0000007f83a77000)
    libp11-kit.so.0 => /usr/lib/aarch64-linux-gnu/libp11-kit.so.0 (0x0000007f83965000)
    libidn2.so.0 => /usr/lib/aarch64-linux-gnu/libidn2.so.0 (0x0000007f83939000)
    libunistring.so.2 => /usr/lib/aarch64-linux-gnu/libunistring.so.2 (0x0000007f837b6000)
    libtasn1.so.6 => /usr/lib/aarch64-linux-gnu/libtasn1.so.6 (0x0000007f83795000)
    libnettle.so.6 => /usr/lib/aarch64-linux-gnu/libnettle.so.6 (0x0000007f83752000)
    libhogweed.so.4 => /usr/lib/aarch64-linux-gnu/libhogweed.so.4 (0x0000007f83711000)
    libgmp.so.10 => /usr/lib/aarch64-linux-gnu/libgmp.so.10 (0x0000007f83694000)
    libicuuc.so.60 => /usr/lib/aarch64-linux-gnu/libicuuc.so.60 (0x0000007f834bf000)
    libdbus-1.so.3 => /lib/aarch64-linux-gnu/libdbus-1.so.3 (0x0000007f8346b000)
    libatspi.so.0 => /usr/lib/aarch64-linux-gnu/libatspi.so.0 (0x0000007f8342f000)
    libharfbuzz.so.0 => /usr/lib/aarch64-linux-gnu/libharfbuzz.so.0 (0x0000007f8338f000)
    libthai.so.0 => /usr/lib/aarch64-linux-gnu/libthai.so.0 (0x0000007f83377000)
    libexpat.so.1 => /lib/aarch64-linux-gnu/libexpat.so.1 (0x0000007f83338000)
    libselinux.so.1 => /lib/aarch64-linux-gnu/libselinux.so.1 (0x0000007f83303000)
    libresolv.so.2 => /lib/aarch64-linux-gnu/libresolv.so.2 (0x0000007f832de000)
    libmount.so.1 => /lib/aarch64-linux-gnu/libmount.so.1 (0x0000007f83281000)
    libXau.so.6 => /usr/lib/aarch64-linux-gnu/libXau.so.6 (0x0000007f8326e000)
    libXdmcp.so.6 => /usr/lib/aarch64-linux-gnu/libXdmcp.so.6 (0x0000007f83259000)
    libgomp.so.1 => /usr/lib/aarch64-linux-gnu/libgomp.so.1 (0x0000007f8321a000)
    libgpg-error.so.0 => /lib/aarch64-linux-gnu/libgpg-error.so.0 (0x0000007f831f8000)
    libkrb5.so.3 => /usr/lib/aarch64-linux-gnu/libkrb5.so.3 (0x0000007f83129000)
    libk5crypto.so.3 => /usr/lib/aarch64-linux-gnu/libk5crypto.so.3 (0x0000007f830eb000)
    libcom_err.so.2 => /lib/aarch64-linux-gnu/libcom_err.so.2 (0x0000007f830d7000)
    libkrb5support.so.0 => /usr/lib/aarch64-linux-gnu/libkrb5support.so.0 (0x0000007f830bd000)
    libicudata.so.60 => /usr/lib/aarch64-linux-gnu/libicudata.so.60 (0x0000007f81702000)
    libsystemd.so.0 => /lib/aarch64-linux-gnu/libsystemd.so.0 (0x0000007f8167e000)
    libgraphite2.so.3 => /usr/lib/aarch64-linux-gnu/libgraphite2.so.3 (0x0000007f8164d000)
    libdatrie.so.1 => /usr/lib/aarch64-linux-gnu/libdatrie.so.1 (0x0000007f81637000)
    libblkid.so.1 => /lib/aarch64-linux-gnu/libblkid.so.1 (0x0000007f815e2000)
    libbsd.so.0 => /lib/aarch64-linux-gnu/libbsd.so.0 (0x0000007f815be000)
    libkeyutils.so.1 => /lib/aarch64-linux-gnu/libkeyutils.so.1 (0x0000007f815aa000)
    liblz4.so.1 => /usr/lib/aarch64-linux-gnu/liblz4.so.1 (0x0000007f8157e000)
    libuuid.so.1 => /lib/aarch64-linux-gnu/libuuid.so.1 (0x0000007f81567000)

Install gtk3:
sudo apt-get install libgtk-3-0 libgtk-3-bin libgtk-3-common

@kaishi Would you install the libgtk-3-0 libgtk-3-bin libgtk-3-common ?

After installing libgtk, it works now,
Adding time duration code to profile performance of yolo3, it takes 100ms to finish one picture. Does this number sound right?

Thanks,

how to make the sources?

@RichardG You can refer to those docs.

Hi
I am able to train Yolo on intel / linux / nvidia platform.
I am able to modify darknet also image.c
I am able to download demo binaries and run the picture demo.
but after downloading https://gitlab.com/khadas/aml_npu_app
i want to modify the khadas demo to accept rtsp input
but I cant make the sources.
Any advise will be greatly appreciated.

https://youtu.be/jkMlSN06iQ0 inline display

@RichardG You need to Application for DDK by email.

ok email application sent

how did you modify and compile the source code?
appreciate any advise.
thanks

Have applied a few days back, pls respond.
Need to get the 5 TOP npu working urgently
thanks

@RichardG Sorry for that . We will check it. Maybe it’s an automated response .
What is your email address. @tsangyoujun can reply to you manually.

thanks
richard@optasia.com.sg

@tsangyoujun Please help him . THX

Yes already PMed him :smile:

1 Like

It’s weird. Anyone that ran the inception_v3 example successfully? How did you enable /dev/galcore (device node file) to run the inception_v3 (ELF file)?

khadas@Khadas:~/mywork/aml_npu_demo_binaries/inceptionv3/VIM3$ ./run.sh
[ 1] Failed to open device: No such file or directory, Try again…
[ 2] Failed to open device: No such file or directory, Try again…
[ 3] Failed to open device: No such file or directory, Try again…
[ 4] Failed to open device: No such file or directory, Try again…
[ 5] _OpenDevice(1205): FATAL: Failed to open device, errno=No such file or directory.