Tengine on VIM3

Announcements
1

Prepare

  1. Reburn the Ubunru image into VIM3.

  2. Upgrade your system to least OTA version. Then reboot.

  1. Check Galcore version
$ dmesg | grep Galcore
[   14.541908] Galcore version 6.4.8.7.1.1.1
  1. Install cmake
$ sudo apt-get install cmake

Compile Tengine

  1. Get TIM-VX
$ git clone https://github.com/VeriSilicon/TIM-VX.git
  1. Get Tengine-lite
$ git clone https://github.com/OAID/Tengine.git
  1. Copy TIM-VX files into Teinge-lite
$ cp -rf ../TIM-VX/include  ./source/device/tim-vx/
$ cp -rf ../TIM-VX/src      ./source/device/tim-vx/
  1. Get 3rdparty
$ wget -c https://github.com/VeriSilicon/TIM-VX/releases/download/v1.1.34.fix/aarch64_A311D_6.4.8.tgz
$ tar zxvf aarch64_A311D_6.4.8.tgz
$ cd Tengine && mkdir -p ./3rdparty/tim-vx/include
$ cp -rf ../prebuild-sdk-s905d3/include/*  ./3rdparty/tim-vx/include/
  1. Compile
$ cd Tengine
$ mkdir build && cd build
$ cmake -DTENGINE_ENABLE_TIM_VX=ON ..
$ make -j4 && make install

Test Tengine

  1. Download mobilenet_uint8.tmfile from Google Drive.

  2. wget -c https://github.com/khadas/aml_npu_sdk/raw/master/acuity-toolkit/demo/data/cat.jpg .

  3. Set environment variables.

$ cd Tengine
$ export LD_LIBRARY_PATH=./build/install/lib
$ ./build/install/bin/tm_classification_timvx -m path/to/mobilenet_uint8.tmfile -i path/to/cat.jpg -g 224,224 -s 0.017,0.017,0.017 -w 104.007,116.669,122.679
tengine-lite library version: 1.5-dev

model file : build/install/bin/mobilenet_uint8.tmfile
image file : cat.jpg
img_h, img_w, scale[3], mean[3] : 224 224 , 0.017 0.017 0.017, 104.0 116.7 122.7
Repeat 1 times, thread 1, avg time 4.47 ms, max_time 4.47 ms, min_time 4.47 ms
--------------------------------------
5.520552, 283
5.200520, 285
5.040504, 281
4.880488, 282
4.720472, 287
--------------------------------------
1 Like
2 
model file : build/install/bin/mobilenet_uint8.tmfile
image file : cat.jpg
img_h, img_w, scale[3], mean[3] : 224 224 , 0.017 0.017 0.017, 104.0 116.7 122.7
Repeat 1 times, thread 1, avg time 4.47 ms, max_time 4.47 ms, min_time 4.47 ms
--------------------------------------
5.520552, 283
5.200520, 285
5.040504, 281
4.880488, 282
4.720472, 287

What does this numbers mean? Can u tell about output

3

@HENNESSYxie This is related to the model itself. The numbers behind here refer to the classification of the recognition results in the original model.

4

And question not about this topic, Is there any way to install python openvino module to Khadas VIM3?

5

@HENNESSYxie Currently not supported, only Intel hardware is supported at this stage

1 Like
6

@Frank Can you please update the link to the Ubuntu image in step 1?
Following the index of the link “dl.khadas.com - Index of /firmware/VIM3/Ubuntu/EMMC/”, should I be using https://dl.khadas.com/firmware/VIM3/Ubuntu/EMMC/vim3-ubuntu-20.04-gnome-linux-4.9-fenix-1.1.1-220725-emmc.img.xz?

7

@Prateek_Arora Yes, you can use 220725 firmware.

8

@Frank, can u please help me with run inference tm_mobilenet_ssd_uint8_timvx

Firmware version: vim3-ubuntu-20.04-server-linux-4.9-fenix-1.1.1-220725.

khadas@khadas:~/tengine$ cat /etc/fenix-release
# PLEASE DO NOT EDIT THIS FILE
BOARD=VIM3
VENDOR=Amlogic
VERSION=1.1.1
ARCH=arm64
INITRD_ARCH=arm64
INSTALL_TYPE=EMMC
IMAGE_VERSION=1.1.1-220725
################ GIT VERSION ################
UBOOT_GIT_VERSION=khadas-vims-u-boot-v2015.01-v1.1.1-release
LINUX_GIT_VERSION=khadas-vims-linux-4.9-v1.1.1-release
FENIX_GIT_VERSION=v1.1.1
#############################################

khadas@khadas:~$ sudo apt list --installed | grep aml-npu
aml-npu/focal,now 6.4.8.7.1.1.1-1 arm64 [installed]

khadas@khadas:~$ dmesg | grep Galcore
[   19.552904] Galcore version 6.4.8.7.1.1.1

Build Tengine with TIM-VX support:

    git clone https://github.com/OAID/Tengine.git
    git clone https://github.com/VeriSilicon/TIM-VX.git
    cp -r ./TIM-VX/include ./Tengine/source/device/tim-vx
    cp -r ./TIM-VX/src     ./Tengine/source/device/tim-vx
    cd Tengine && mkdir build && cd build
    cmake -DTENGINE_ENABLE_TIM_VX=ON -DTENGINE_BUILD_EXAMPLES=ON ..
    make -j4 && make install

Build was success.

But when i run inference with tm_mobilenet_ssd_uint8_timvx i get error:

khadas@khadas:~/tengine$ ./Tengine/build/install/bin/tm_mobilenet_ssd_uint8_timvx -m mobilenet_ssd.tmfile -i bike.jpg
Tengine: Size of tensor != size of buffer(1080000 vs 270000).
Set input tensor buffer failed

Can u give some ideas?

Classification example with NPU work well:

khadas@khadas:~/tengine$ ./Tengine/build/examples/tm_classification_timvx -m mobilenet_uint8.tmfile -i cat.jpg 
Image height not specified, use default 227
Image width not specified, use default  227
Scale value not specified, use default  1.0, 1.0, 1.0
Mean value not specified, use default   104.0, 116.7, 122.7
tengine-lite library version: 1.5-dev

model file : mobilenet_uint8.tmfile
image file : cat.jpg
img_h, img_w, scale[3], mean[3] : 227 227 , 1.000 1.000 1.000, 104.0 116.7 122.7
Repeat 1 times, thread 1, avg time 5.20 ms, max_time 5.20 ms, min_time 5.20 ms
--------------------------------------
5.840584, 281
5.760576, 611
5.200520, 282
5.120512, 326
4.560456, 285
--------------------------------------
9

Hi @ DanilZittser：

the info of “Tengine: Size of tensor != size of buffer(1080000 vs 270000)” means that input tensor buffer size is 108000, 4 times of 27000, so the model is a float model not an uint8 model.
the uint8 model of “mobilenet_ssd.tmfile” may named “mobilenet_ssd_uint8.tmfile”.