After a few days of struggle I managed to create a sample app for mobilenet ssd v2 and test VIM3 NPU with it. I had to hack around a few things to get it working and while this code might not be ideal I think it does the job to evaluate the hardware capabilities.
Performance seems to be reasonable, on par with Google’s TPU. Inference takes about 14ms/72 Inference FPS. Power consumption is around, estimated at 1.5W for NPU.
I uploaded this on github, source and binary with some notes if anyone want to give it a try.
There is still an open question about ops not supported in Acuity, that would had made my life easier
As a side note, I noticed that the load/power usage on VIM3 in idle went up from initially (when I bought it two months ago) 1.4W to 2.1W - 2.2W. Also I noticed load average now in idle is at 2 and it should normally be 0. Any idea what is the cause of this? A recent update?
I am not sure as to which image input size you refer? The app can use webcam feed (you can change the cam video resolution just be aware that logitech cams in high res usually have a low FPS), video file or image as input. The mobilenet ssd v2 used has a 300x300 input but conversion, crop/resize is done by the app.
I tried your code and it works on my VIM3 board.
The speed is about 65frames/sec, ie, 15ms/frame.
Do you plan to convert ADAS model, like lane detection to evaluate the performance because your github mentions about ADAS?
I might, I started the project with the idea to make a rear drive camera with an led matrix display made out of 4 x MAX7219 Dot Matrix Module 4-in-1 to warn drivers tailgating of distance/speed and for that an easy way to measure approximate distance will be detect car and calculate distance using lens focal/camera sensor size. For that I would only need a detector say for various vehicles class sizes, and perhaps cyclists/motorcycles. Then on the same note I thought why not use a forward system as well with a larger class pedestrians etc. and connect the two over gigabit network so the two can share intelligence, say for example forward system can pass info on hazards ahead and display the info on the led display. Will see, it’s work in progress, I started working on a system with xu4 and two Intel movidius ncs to barely get 16 FPS, then I got the Gyrfalcon 2801 to jetson and coral and now VIM which seems promising and even better if they solve a few bits, like the pcie m.2 Or even better if they make an M2X Extension Board specialized for robotics similar to Qualcomm Robotics RB3
with gps/accel/gyro/etc external battery and/or supercaps for nice shutdown.
Lary i didnt compile your makefile code on Vim3
i installed from source opencv 4.1 ,its okey.
But i cant connect library header file on the Makefile
they says root@Khadas:/home/khadas/Desktop/Folder/aml_npu_sdk/linux_sdk/demo/vim3# make aarch64-linux-gnu-g++ -o obj/aml_obj_detect.o -O3 --std=c++11 -mcpu=cortex-a73 -funsafe-math-optimizations -ftree-vectorize -fPIC -I/usr/local/include/opencv4/opencv -I/usr/local/include/opencv4 -c aml_obj_detect.cpp In file included from include/ovxlib/vsi_nn_context.h:27:0, from include/ovxlib/vsi_nn_pub.h:13, from aml_worker.h:25, from aml_obj_detect.cpp:18: include/ovxlib/vsi_nn_platform.h:27:10: fatal error: VX/vx_khr_cnn.h: No such file or directory #include <VX/vx_khr_cnn.h> ^~~~~~~~~~~~~~~~~ compilation terminated. Makefile:61: recipe for target 'obj/aml_obj_detect.o' failed make: *** [obj/aml_obj_detect.o] Error 1
my file and path looks like ,how can i do ?
/home/khadas/Desktop/Folder/aml_npu_sdk/
İs this can be problem path??
i changed many thinks,but every time i got error ,normaly i use -mcpu=a73 for Aarch64 .this image is wrong
Thanks for sharing your plan.
My plan is to use VIM3 and tourchscreen to emulate commaai-openpilot (open source code in github) without car controlling part. So it can do lane detection and departure warning.
…
…
Fold/bias:out0’, ‘FeatureExtractor/MobilenetV1/MobilenetV1/Conv2d_3_depthwise/mul_fold:out0’, ‘FeatureExtractor/MobilenetV1/MobilenetV1/Conv2d_13_pointwise/mul_fold:out0’, ‘FeatureExtractor/MobilenetV1/Conv2d_13_pointwise_2_Conv2d_5_3x3_s2_32/mul_fold:out0’]
2019-12-03 17:57:30.881831: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
I Convert To TFLite to Import quantized model.
Traceback (most recent call last):
File “convertensorflow.py”, line 62, in
File “convertensorflow.py”, line 58, in main
File “acuitylib/app/importer/import_tensorflow.py”, line 125, in run
IndexError: list index out of range
[18365] Failed to execute script convertensorflow
I used the export_tflite_ssd_graph.py as explained in the repo readme, it actually creates a .pb file and not a .tflite file, but that seems to follow the steps as I understood them
thanks for your helping, you are brillliant people, tomorrow i Will add empty Folder. Every body make a little mistake,
thanks again larry