NPU Demo and source code

There are some ready to use NPU demo binaries for VIM3/VIM3L.

This documentation is about how to run these demos on VIM3/VIM3L.

Prepare

You need to run Ubuntu image V20190830 on board, and update the system.

khadas@Khadas:~$ sudo apt-get update
khadas@Khadas:~$ sudo apt-get upgrade
khadas@Khadas:~$ sudo apt-get install -y gstreamer1.0-libav gstreamer1.0-plugins-bad gstreamer1.0-tools gstreamer1.0-plugins-base gstreamer1.0-alsa gstreamer1.0-plugins-good libconfig++9v5 libconfig9
khadas@Khadas:~$ sudo apt-get install -y aml-npu
khadas@Khadas:~$ sudo apt-get install -y gstreamer-aml libplayer-aml opencv3
khadas@Khadas:~$ sync
khadas@Khadas:~$ sudo reboot

Note: You can choose server image to run these demos. If you choose desktop image, you need to switch to the framebuffer console by Ctrl+Alt+F1 to run these demos.

Download NPU demo binaries

khadas@Khadas:~$ sudo -i
root@Khadas:~# git clone https://gitlab.com/khadas/aml_npu_demo_binaries --depth=1

Directory description:

  • detect_demo - Yoloface/YoloV2/YoloV3 demo with video
  • detect_demo_picture - Yoloface/YoloV2/YoloV3 demo with picture
  • inceptionv3 - Inceptionv3

Run Yolo demo with USB camera

Install the libraries:

root@Khadas:~# cd aml_npu_demo_binaries/detect_demo
root@Khadas:~/aml_npu_demo_binaries/detect_demo# ./INSTALL

Run UVC demo:

root@Khadas:~/aml_npu_demo_binaries/detect_demo# ./detect_demo_uvc /dev/videoX 1 > /dev/null

Note: Replace /dev/videoX to the correct node.

You can run ./detect_demo_uvc to get help information and read README to get more information.

Tested USB camera:

  • Logitech C920 PRO
  • Logitech C525

Run Yolo demo with MIPI camera

root@Khadas:~/aml_npu_demo_binaries/detect_demo# ./detect_demo_mipi /dev/videoX  1 > /dev/null

Note: MIPI camera for VIM3 has not launched yet.

Source Code

The source code of the demos:

In The End

The performance of the video display still need to improve.

6 Likes

Thank you @numbqq . I was looking out for this.
I’ll check it out when I find the time and make a video about it.
Just got the NVIDIA Jetson Nano. So could be nice to do a comparison of both.
The Nano may have a good GPU, but its CPU is mighty slow. Can’t be said about the VIM3.
Greetings.

I got the following error when I run detect_demo in the detect_demo_picture directory:

./detect_demo: error while loading shared libraries: libgtk-3.so.0: cannot open shared object file: No such file or directory

Where can I install libgtk-3.so?

Can you provide this for me? ldd detect_demo ?

    linux-vdso.so.1 (0x0000007f89f97000)
    libovxlib.so => /lib/libovxlib.so (0x0000007f89e53000)
    libm.so.6 => /lib/aarch64-linux-gnu/libm.so.6 (0x0000007f89d99000)
    libpthread.so.0 => /lib/aarch64-linux-gnu/libpthread.so.0 (0x0000007f89d6d000)
    libdl.so.2 => /lib/aarch64-linux-gnu/libdl.so.2 (0x0000007f89d58000)
    libz.so.1 => /lib/aarch64-linux-gnu/libz.so.1 (0x0000007f89d2b000)
    libopencv_imgproc.so.3.4 => /home/khadas/USBDrive/nanopi/opencv/build/lib/libopencv_imgproc.so.3.4 (0x0000007f8993b000)
    libopencv_core.so.3.4 => /home/khadas/USBDrive/nanopi/opencv/build/lib/libopencv_core.so.3.4 (0x0000007f895e5000)
    libopencv_videoio.so.3.4 => /home/khadas/USBDrive/nanopi/opencv/build/lib/libopencv_videoio.so.3.4 (0x0000007f89593000)
    libopencv_imgcodecs.so.3.4 => /home/khadas/USBDrive/nanopi/opencv/build/lib/libopencv_imgcodecs.so.3.4 (0x0000007f89397000)
    libopencv_highgui.so.3.4 => /home/khadas/USBDrive/nanopi/opencv/build/lib/libopencv_highgui.so.3.4 (0x0000007f89372000)
    libnn_detect.so => /usr/lib/libnn_detect.so (0x0000007f8935e000)
    libstdc++.so.6 => /usr/lib/aarch64-linux-gnu/libstdc++.so.6 (0x0000007f891cb000)
    librt.so.1 => /lib/aarch64-linux-gnu/librt.so.1 (0x0000007f891b4000)
    libgcc_s.so.1 => /lib/aarch64-linux-gnu/libgcc_s.so.1 (0x0000007f89190000)
    libc.so.6 => /lib/aarch64-linux-gnu/libc.so.6 (0x0000007f89037000)
    libOpenVX.so => /lib/libOpenVX.so (0x0000007f88eab000)
    libOpenVXU.so => /lib/libOpenVXU.so (0x0000007f88e95000)
    libCLC.so => /lib/libCLC.so (0x0000007f88c84000)
    libVSC.so => /lib/libVSC.so (0x0000007f87dde000)
    /lib/ld-linux-aarch64.so.1 (0x0000007f89f6c000)
    libavcodec.so.57 => /usr/lib/aarch64-linux-gnu/libavcodec.so.57 (0x0000007f86b98000)
    libavformat.so.57 => /usr/lib/aarch64-linux-gnu/libavformat.so.57 (0x0000007f86961000)
    libavutil.so.55 => /usr/lib/aarch64-linux-gnu/libavutil.so.55 (0x0000007f868d1000)
    libswscale.so.4 => /usr/lib/aarch64-linux-gnu/libswscale.so.4 (0x0000007f8685d000)
    libjpeg.so.8 => /usr/lib/aarch64-linux-gnu/libjpeg.so.8 (0x0000007f86813000)
    libpng16.so.16 => /usr/lib/aarch64-linux-gnu/libpng16.so.16 (0x0000007f867d8000)
    libtiff.so.5 => /usr/lib/aarch64-linux-gnu/libtiff.so.5 (0x0000007f8675d000)
    libgtk-3.so.0 => /usr/lib/aarch64-linux-gnu/libgtk-3.so.0 (0x0000007f86101000)
    libgdk-3.so.0 => /usr/lib/aarch64-linux-gnu/libgdk-3.so.0 (0x0000007f86019000)
    libcairo.so.2 => /usr/lib/aarch64-linux-gnu/libcairo.so.2 (0x0000007f85f1f000)
    libgdk_pixbuf-2.0.so.0 => /usr/lib/aarch64-linux-gnu/libgdk_pixbuf-2.0.so.0 (0x0000007f85ef0000)
    libgobject-2.0.so.0 => /usr/lib/aarch64-linux-gnu/libgobject-2.0.so.0 (0x0000007f85e92000)
    libglib-2.0.so.0 => /usr/lib/aarch64-linux-gnu/libglib-2.0.so.0 (0x0000007f85d84000)
    libGAL.so => /lib/libGAL.so (0x0000007f85c00000)
    libLLVM_viv.so => /lib/libLLVM_viv.so (0x0000007f85768000)
    libswresample.so.2 => /usr/lib/aarch64-linux-gnu/libswresample.so.2 (0x0000007f85740000)
    libwebp.so.6 => /usr/lib/aarch64-linux-gnu/libwebp.so.6 (0x0000007f856e8000)
    libva.so.2 => /usr/lib/aarch64-linux-gnu/libva.so.2 (0x0000007f856b9000)
    libzvbi.so.0 => /usr/lib/aarch64-linux-gnu/libzvbi.so.0 (0x0000007f8562a000)
    libxvidcore.so.4 => /usr/lib/aarch64-linux-gnu/libxvidcore.so.4 (0x0000007f85543000)
    libx265.so.146 => /usr/lib/aarch64-linux-gnu/libx265.so.146 (0x0000007f852e5000)
    libx264.so.152 => /usr/lib/aarch64-linux-gnu/libx264.so.152 (0x0000007f8517f000)
    libwebpmux.so.3 => /usr/lib/aarch64-linux-gnu/libwebpmux.so.3 (0x0000007f85166000)
    libwavpack.so.1 => /usr/lib/aarch64-linux-gnu/libwavpack.so.1 (0x0000007f85134000)
    libvpx.so.5 => /usr/lib/aarch64-linux-gnu/libvpx.so.5 (0x0000007f84f89000)
    libvorbisenc.so.2 => /usr/lib/aarch64-linux-gnu/libvorbisenc.so.2 (0x0000007f84eda000)
    libvorbis.so.0 => /usr/lib/aarch64-linux-gnu/libvorbis.so.0 (0x0000007f84ea4000)
    libtwolame.so.0 => /usr/lib/aarch64-linux-gnu/libtwolame.so.0 (0x0000007f84e73000)
    libtheoraenc.so.1 => /usr/lib/aarch64-linux-gnu/libtheoraenc.so.1 (0x0000007f84e31000)
    libtheoradec.so.1 => /usr/lib/aarch64-linux-gnu/libtheoradec.so.1 (0x0000007f84e0a000)
    libspeex.so.1 => /usr/lib/aarch64-linux-gnu/libspeex.so.1 (0x0000007f84de5000)
    libsnappy.so.1 => /usr/lib/aarch64-linux-gnu/libsnappy.so.1 (0x0000007f84dcd000)
    libshine.so.3 => /usr/lib/aarch64-linux-gnu/libshine.so.3 (0x0000007f84db3000)
    librsvg-2.so.2 => /usr/lib/aarch64-linux-gnu/librsvg-2.so.2 (0x0000007f84d73000)
    libopus.so.0 => /usr/lib/aarch64-linux-gnu/libopus.so.0 (0x0000007f84d26000)
    libopenjp2.so.7 => /usr/lib/aarch64-linux-gnu/libopenjp2.so.7 (0x0000007f84ccb000)
    libmp3lame.so.0 => /usr/lib/aarch64-linux-gnu/libmp3lame.so.0 (0x0000007f84c4f000)
    libgsm.so.1 => /usr/lib/aarch64-linux-gnu/libgsm.so.1 (0x0000007f84c35000)
    liblzma.so.5 => /lib/aarch64-linux-gnu/liblzma.so.5 (0x0000007f84c05000)
    libssh-gcrypt.so.4 => /usr/lib/aarch64-linux-gnu/libssh-gcrypt.so.4 (0x0000007f84b8d000)
    libopenmpt.so.0 => /usr/lib/aarch64-linux-gnu/libopenmpt.so.0 (0x0000007f849c5000)
    libbluray.so.2 => /usr/lib/aarch64-linux-gnu/libbluray.so.2 (0x0000007f8496e000)
    libgnutls.so.30 => /usr/lib/aarch64-linux-gnu/libgnutls.so.30 (0x0000007f84811000)
    libxml2.so.2 => /usr/lib/aarch64-linux-gnu/libxml2.so.2 (0x0000007f84672000)
    libgme.so.0 => /usr/lib/aarch64-linux-gnu/libgme.so.0 (0x0000007f84619000)
    libchromaprint.so.1 => /usr/lib/aarch64-linux-gnu/libchromaprint.so.1 (0x0000007f845f7000)
    libbz2.so.1.0 => /lib/aarch64-linux-gnu/libbz2.so.1.0 (0x0000007f845d7000)
    libX11.so.6 => /usr/lib/aarch64-linux-gnu/libX11.so.6 (0x0000007f844ae000)
    libdrm.so.2 => /usr/lib/aarch64-linux-gnu/libdrm.so.2 (0x0000007f8448f000)
    libvdpau.so.1 => /usr/lib/aarch64-linux-gnu/libvdpau.so.1 (0x0000007f8447b000)
    libva-x11.so.2 => /usr/lib/aarch64-linux-gnu/libva-x11.so.2 (0x0000007f84464000)
    libva-drm.so.2 => /usr/lib/aarch64-linux-gnu/libva-drm.so.2 (0x0000007f84451000)
    libjbig.so.0 => /usr/lib/aarch64-linux-gnu/libjbig.so.0 (0x0000007f84434000)
    libgmodule-2.0.so.0 => /usr/lib/aarch64-linux-gnu/libgmodule-2.0.so.0 (0x0000007f84420000)
    libpangocairo-1.0.so.0 => /usr/lib/aarch64-linux-gnu/libpangocairo-1.0.so.0 (0x0000007f84404000)
    libXi.so.6 => /usr/lib/aarch64-linux-gnu/libXi.so.6 (0x0000007f843e4000)
    libXfixes.so.3 => /usr/lib/aarch64-linux-gnu/libXfixes.so.3 (0x0000007f843ce000)
    libcairo-gobject.so.2 => /usr/lib/aarch64-linux-gnu/libcairo-gobject.so.2 (0x0000007f843b5000)
    libatk-1.0.so.0 => /usr/lib/aarch64-linux-gnu/libatk-1.0.so.0 (0x0000007f84381000)
    libatk-bridge-2.0.so.0 => /usr/lib/aarch64-linux-gnu/libatk-bridge-2.0.so.0 (0x0000007f84343000)
    libepoxy.so.0 => /usr/lib/aarch64-linux-gnu/libepoxy.so.0 (0x0000007f8424f000)
    libpangoft2-1.0.so.0 => /usr/lib/aarch64-linux-gnu/libpangoft2-1.0.so.0 (0x0000007f8422a000)
    libpango-1.0.so.0 => /usr/lib/aarch64-linux-gnu/libpango-1.0.so.0 (0x0000007f841d3000)
    libfontconfig.so.1 => /usr/lib/aarch64-linux-gnu/libfontconfig.so.1 (0x0000007f84183000)
    libgio-2.0.so.0 => /usr/lib/aarch64-linux-gnu/libgio-2.0.so.0 (0x0000007f84007000)
    libXinerama.so.1 => /usr/lib/aarch64-linux-gnu/libXinerama.so.1 (0x0000007f83ff4000)
    libXrandr.so.2 => /usr/lib/aarch64-linux-gnu/libXrandr.so.2 (0x0000007f83fda000)
    libXcursor.so.1 => /usr/lib/aarch64-linux-gnu/libXcursor.so.1 (0x0000007f83fbf000)
    libXcomposite.so.1 => /usr/lib/aarch64-linux-gnu/libXcomposite.so.1 (0x0000007f83fac000)
    libXdamage.so.1 => /usr/lib/aarch64-linux-gnu/libXdamage.so.1 (0x0000007f83f99000)
    libxkbcommon.so.0 => /usr/lib/aarch64-linux-gnu/libxkbcommon.so.0 (0x0000007f83f4f000)
    libwayland-cursor.so.0 => /usr/lib/aarch64-linux-gnu/libwayland-cursor.so.0 (0x0000007f83f37000)
    libwayland-egl.so.1 => /usr/lib/aarch64-linux-gnu/libwayland-egl.so.1 (0x0000007f83f25000)
    libwayland-client.so.0 => /usr/lib/aarch64-linux-gnu/libwayland-client.so.0 (0x0000007f83f06000)
    libXext.so.6 => /usr/lib/aarch64-linux-gnu/libXext.so.6 (0x0000007f83ee6000)
    libpixman-1.so.0 => /usr/lib/aarch64-linux-gnu/libpixman-1.so.0 (0x0000007f83e82000)
    libfreetype.so.6 => /usr/lib/aarch64-linux-gnu/libfreetype.so.6 (0x0000007f83dd9000)
    libxcb-shm.so.0 => /usr/lib/aarch64-linux-gnu/libxcb-shm.so.0 (0x0000007f83dc6000)
    libxcb.so.1 => /usr/lib/aarch64-linux-gnu/libxcb.so.1 (0x0000007f83d96000)
    libxcb-render.so.0 => /usr/lib/aarch64-linux-gnu/libxcb-render.so.0 (0x0000007f83d79000)
    libXrender.so.1 => /usr/lib/aarch64-linux-gnu/libXrender.so.1 (0x0000007f83d60000)
    libffi.so.6 => /usr/lib/aarch64-linux-gnu/libffi.so.6 (0x0000007f83d48000)
    libpcre.so.3 => /lib/aarch64-linux-gnu/libpcre.so.3 (0x0000007f83cd6000)
    libsoxr.so.0 => /usr/lib/aarch64-linux-gnu/libsoxr.so.0 (0x0000007f83c75000)
    libnuma.so.1 => /usr/lib/aarch64-linux-gnu/libnuma.so.1 (0x0000007f83c55000)
    libogg.so.0 => /usr/lib/aarch64-linux-gnu/libogg.so.0 (0x0000007f83c3e000)
    libcroco-0.6.so.3 => /usr/lib/aarch64-linux-gnu/libcroco-0.6.so.3 (0x0000007f83bfa000)
    libgcrypt.so.20 => /lib/aarch64-linux-gnu/libgcrypt.so.20 (0x0000007f83b3f000)
    libgssapi_krb5.so.2 => /usr/lib/aarch64-linux-gnu/libgssapi_krb5.so.2 (0x0000007f83aec000)
    libmpg123.so.0 => /usr/lib/aarch64-linux-gnu/libmpg123.so.0 (0x0000007f83a8f000)
    libvorbisfile.so.3 => /usr/lib/aarch64-linux-gnu/libvorbisfile.so.3 (0x0000007f83a77000)
    libp11-kit.so.0 => /usr/lib/aarch64-linux-gnu/libp11-kit.so.0 (0x0000007f83965000)
    libidn2.so.0 => /usr/lib/aarch64-linux-gnu/libidn2.so.0 (0x0000007f83939000)
    libunistring.so.2 => /usr/lib/aarch64-linux-gnu/libunistring.so.2 (0x0000007f837b6000)
    libtasn1.so.6 => /usr/lib/aarch64-linux-gnu/libtasn1.so.6 (0x0000007f83795000)
    libnettle.so.6 => /usr/lib/aarch64-linux-gnu/libnettle.so.6 (0x0000007f83752000)
    libhogweed.so.4 => /usr/lib/aarch64-linux-gnu/libhogweed.so.4 (0x0000007f83711000)
    libgmp.so.10 => /usr/lib/aarch64-linux-gnu/libgmp.so.10 (0x0000007f83694000)
    libicuuc.so.60 => /usr/lib/aarch64-linux-gnu/libicuuc.so.60 (0x0000007f834bf000)
    libdbus-1.so.3 => /lib/aarch64-linux-gnu/libdbus-1.so.3 (0x0000007f8346b000)
    libatspi.so.0 => /usr/lib/aarch64-linux-gnu/libatspi.so.0 (0x0000007f8342f000)
    libharfbuzz.so.0 => /usr/lib/aarch64-linux-gnu/libharfbuzz.so.0 (0x0000007f8338f000)
    libthai.so.0 => /usr/lib/aarch64-linux-gnu/libthai.so.0 (0x0000007f83377000)
    libexpat.so.1 => /lib/aarch64-linux-gnu/libexpat.so.1 (0x0000007f83338000)
    libselinux.so.1 => /lib/aarch64-linux-gnu/libselinux.so.1 (0x0000007f83303000)
    libresolv.so.2 => /lib/aarch64-linux-gnu/libresolv.so.2 (0x0000007f832de000)
    libmount.so.1 => /lib/aarch64-linux-gnu/libmount.so.1 (0x0000007f83281000)
    libXau.so.6 => /usr/lib/aarch64-linux-gnu/libXau.so.6 (0x0000007f8326e000)
    libXdmcp.so.6 => /usr/lib/aarch64-linux-gnu/libXdmcp.so.6 (0x0000007f83259000)
    libgomp.so.1 => /usr/lib/aarch64-linux-gnu/libgomp.so.1 (0x0000007f8321a000)
    libgpg-error.so.0 => /lib/aarch64-linux-gnu/libgpg-error.so.0 (0x0000007f831f8000)
    libkrb5.so.3 => /usr/lib/aarch64-linux-gnu/libkrb5.so.3 (0x0000007f83129000)
    libk5crypto.so.3 => /usr/lib/aarch64-linux-gnu/libk5crypto.so.3 (0x0000007f830eb000)
    libcom_err.so.2 => /lib/aarch64-linux-gnu/libcom_err.so.2 (0x0000007f830d7000)
    libkrb5support.so.0 => /usr/lib/aarch64-linux-gnu/libkrb5support.so.0 (0x0000007f830bd000)
    libicudata.so.60 => /usr/lib/aarch64-linux-gnu/libicudata.so.60 (0x0000007f81702000)
    libsystemd.so.0 => /lib/aarch64-linux-gnu/libsystemd.so.0 (0x0000007f8167e000)
    libgraphite2.so.3 => /usr/lib/aarch64-linux-gnu/libgraphite2.so.3 (0x0000007f8164d000)
    libdatrie.so.1 => /usr/lib/aarch64-linux-gnu/libdatrie.so.1 (0x0000007f81637000)
    libblkid.so.1 => /lib/aarch64-linux-gnu/libblkid.so.1 (0x0000007f815e2000)
    libbsd.so.0 => /lib/aarch64-linux-gnu/libbsd.so.0 (0x0000007f815be000)
    libkeyutils.so.1 => /lib/aarch64-linux-gnu/libkeyutils.so.1 (0x0000007f815aa000)
    liblz4.so.1 => /usr/lib/aarch64-linux-gnu/liblz4.so.1 (0x0000007f8157e000)
    libuuid.so.1 => /lib/aarch64-linux-gnu/libuuid.so.1 (0x0000007f81567000)

Install gtk3:
sudo apt-get install libgtk-3-0 libgtk-3-bin libgtk-3-common

@kaishi Would you install the libgtk-3-0 libgtk-3-bin libgtk-3-common ?

After installing libgtk, it works now,
Adding time duration code to profile performance of yolo3, it takes 100ms to finish one picture. Does this number sound right?

Thanks,