VIM3 NPU using Tengine SDK for MobileNet

I have used tengine SDK for my custom yolov3 model. I could tranform the model and could do the inference as there was a special repo for yolo framework. https://gitlab.com/khadas/tengine_khadas_app.git
This repository had " tengine_khadas_yolov3_picture.cpp" which helped me to do inference on my model.

Similary, I would like to know, where is the repository for other frameworks, like tensorflow/ caffe etc.
I have already converted my caffe model to (.tmfile)int8 format. Now I want to further implement this on the VIM3 and infer the model.

Please help me with this issue.

@Akkisony Hello, how to process the post-processing data is basically the same. It is calculated by zero_ponter and scare. You need to have basic knowledge. It is impossible for us to make a demo for every model. Whether it is TF or CAFFE or PT, there will only be a difference before the conversion, and the same after the conversion.

@Frank

What do you mean by the above line? I am new to this and hence, I am questioning you about it. Can you give me some clarity?

After converting the model to (.tmfile)int8 format, please tell me what are the next procedure that needs to be followed for caffe model? Do we I have to write a post-processing code in C?

Do you mean the existing “tengine_khadas_yolov3_picture.cpp” works for other frameworks as well? I would like to learn from an expert like you. Please feel free to share some information.

@Akkisony The post-processing method is similar, convert the data of uint8_t into floating-point data, and then process the floating-point data according to different models. But according to each model, the details of post-processing are different

1 Like

@Frank Thank you for the explanation.

  1. Can you tell me which are the post processing files that needs to be altered in the aml_npu_app repo?

  2. Please give me one more clarity, the post processing also does the inference for the model in tengine_khadas_yolov3_picture.cpp?

  3. I am confused because in AML NPU APP, there are several files involved and in tengine its just one file - tengine_khadas_yolov3_picture.cpp

@Frank Please can you tell me if there is any document to write Post - Processing for other frameworks? I would atleast look into it and try implementing to my caffe model.

@Akkisony Sorry, I have no more information to provide you. Maybe you should understand the structure of the output layer of the SSD model

@Frank I tried analysing AML_NPU_APP repo and found many files. So, it would be really helpful for me if you tell me which files needs to be modified to get caffe model running.

In terms of Tengine SDK, I have converted the model to .tmfile(uint8) format. The next step is to create an object (CaffeSSD.o file). Inthe demo, tengine_khadas_yolov3_picture.cpp creates the object file for us. Will you give me any inputs for me to alter this file and get caffe ssd working, please? It would really help me! I am new to this and hence, I have to learn from scratch.

@Akkisony There is no general solution here, and it has nothing to do with caffe and tensorflow. What you need to do is to understand the original composition of the SDD model

@Frank

  1. Can you tell me other than writing post processing code, do I also have to make any changes in makefile-cv4.linux?
  2. I found one example on Tengine GIT repossitory. Tengine/tm_mobilenet_ssd_uint8.cpp at tengine-lite · OAID/Tengine · GitHub

When I try doing ./build-cv4.sh, I get a very huge log file.


In file included from /usr/include/c++/9/istream:991,
from /usr/include/c++/9/iostream:40,
from tm_mobilenet_ssd_uint8.cpp:25:
/usr/include/c++/9/bits/istream.tcc:500:5: error: ‘std::basic_istream<_CharT, _Traits>& std::basic_istream<_CharT, _Traits>::ignore’ is not a static data member of ‘class std::basic_istream<_CharT, _Traits>’
500 | basic_istream<_CharT, _Traits>::
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/include/c++/9/bits/istream.tcc:501:12: error: template definition of non-template ‘std::basic_istream<_CharT, _Traits>& std::basic_istream<_CharT, _Traits>::ignore’
501 | ignore(streamsize __n)
| ^~~~~~~~~~
/usr/include/c++/9/bits/istream.tcc:501:12: error: ‘streamsize’ was not declared in this scope
/usr/include/c++/9/bits/istream.tcc:562:5: error: ‘std::basic_istream<_CharT, _Traits>& std::basic_istream<_CharT, _Traits>::ignore’ is not a static data member of ‘class std::basic_istream<_CharT, _Traits>’
562 | basic_istream<_CharT, _Traits>::
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/include/c++/9/bits/istream.tcc:563:12: error: template definition of non-template ‘std::basic_istream<_CharT, _Traits>& std::basic_istream<_CharT, _Traits>::ignore’
563 | ignore(streamsize __n, int_type __delim)
| ^~~~~~~~~~
/usr/include/c++/9/bits/istream.tcc:563:12: error: ‘streamsize’ was not declared in this scope
/usr/include/c++/9/bits/istream.tcc:563:37: error: expected primary-expression before ‘__delim’
563 | ignore(streamsize __n, int_type __delim)
| ^~~~~~~
/usr/include/c++/9/bits/istream.tcc: In member function ‘std::basic_istream<_CharT, _Traits>::int_type std::basic_istream<_CharT, _Traits>::peek()’:
/usr/include/c++/9/bits/istream.tcc:631:7: error: ‘_M_gcount’ was not declared in this scope
631 | _M_gcount = 0;
| ^~~~~~~~~
/usr/include/c++/9/bits/istream.tcc: At global scope:
/usr/include/c++/9/bits/istream.tcc:658:26: error: ‘streamsize’ has not been declared
658 | read(char_type* __s, streamsize __n)
| ^~~~~~~~~~
/usr/include/c++/9/bits/istream.tcc: In member function ‘std::basic_istream<_CharT, _Traits>& std::basic_istream<_CharT, _Traits>::read(std::basic_istream<_CharT, _Traits>::char_type*, int)’:
/usr/include/c++/9/bits/istream.tcc:660:7: error: ‘_M_gcount’ was not declared in this scope
660 | _M_gcount = 0;
| ^~~~~~~~~
/usr/include/c++/9/bits/istream.tcc: At global scope:
/usr/include/c++/9/bits/istream.tcc:685:5: error: ‘streamsize’ does not name a type
685 | streamsize
| ^~~~~~~~~~
/usr/include/c++/9/bits/istream.tcc: In member function ‘std::basic_istream<_CharT, _Traits>& std::basic_istream<_CharT, _Traits>::putback(std::basic_istream<_CharT, _Traits>::char_type)’:
/usr/include/c++/9/bits/istream.tcc:723:7: error: ‘_M_gcount’ was not declared in this scope
723 | _M_gcount = 0;
| ^~~~~~~~~
/usr/include/c++/9/bits/istream.tcc: In member function ‘std::basic_istream<_CharT, _Traits>& std::basic_istream<_CharT, _Traits>::unget()’:
/usr/include/c++/9/bits/istream.tcc:758:7: error: ‘_M_gcount’ was not declared in this scope
758 | _M_gcount = 0;
| ^~~~~~~~~
/usr/include/c++/9/bits/istream.tcc: In function ‘std::basic_istream<_CharT, _Traits>& std::operator>>(std::basic_istream<_CharT, _Traits>&, _CharT2*)’:
/usr/include/c++/9/bits/istream.tcc:971:7: error: ‘streamsize’ was not declared in this scope
971 | streamsize __extracted = 0;
| ^~~~~~~~~~
/usr/include/c++/9/bits/istream.tcc:979:18: error: expected ‘;’ before ‘__num’
979 | streamsize __num = __in.width();
| ^~~~~~
| ;
/usr/include/c++/9/bits/istream.tcc:980:12: error: ‘__num’ was not declared in this scope
980 | if (__num <= 0)
| ^~~~~
/usr/include/c++/9/bits/istream.tcc:989:15: error: ‘__extracted’ was not declared in this scope
989 | while (__extracted < __num - 1
| ^~~~~~~~~~~
/usr/include/c++/9/bits/istream.tcc:989:29: error: ‘__num’ was not declared in this scope
989 | while (__extracted < __num - 1
| ^~~~~
/usr/include/c++/9/bits/istream.tcc:1014:12: error: ‘__extracted’ was not declared in this scope
1014 | if (!__extracted)
| ^~~~~~~~~~~
tm_mobilenet_ssd_uint8.cpp: In function ‘void get_input_uint_data_ssd(const char*, uint8_t*, int, int, float, int)’:
tm_mobilenet_ssd_uint8.cpp:57:5: error: ‘image’ was not declared in this scope
57 | image img = imread_process(image_file, img_w, img_h, mean, scales);
| ^~~~~
tm_mobilenet_ssd_uint8.cpp:59:35: error: ‘img’ was not declared in this scope; did you mean ‘img_h’?
59 | float* image_data = ( float* )img.data;
| ^~~
| img_h
tm_mobilenet_ssd_uint8.cpp:65:13: error: ‘married’ was not declared in this scope
65 | married = 255 ;
| ^~~~~~~
tm_mobilenet_ssd_uint8.cpp:67:13: error: ‘married’ was not declared in this scope
67 | married = 0 ;
| ^~~~~~~
tm_mobilenet_ssd_uint8.cpp:72:5: error: ‘free_image’ was not declared in this scope
72 | free_image(img);
| ^~~~~~~~~~
tm_mobilenet_ssd_uint8.cpp:55:11: warning: unused variable ‘mean’ [-Wunused-variable]
55 | float mean[3] = {127.5f, 127.5f, 127.5f};
| ^~~~
tm_mobilenet_ssd_uint8.cpp:56:11: warning: unused variable ‘scales’ [-Wunused-variable]
56 | float scales[3] = {1 / 127.5f, 1 / 127.5f, 1 / 127.5f};
| ^~~~~~
tm_mobilenet_ssd_uint8.cpp: In function ‘void post_process_ssd(const char*, float, float*, int)’:
tm_mobilenet_ssd_uint8.cpp:82:5: error: ‘image’ was not declared in this scope
82 | image im = imread(image_file);
| ^~~~~
tm_mobilenet_ssd_uint8.cpp:84:17: error: ‘im’ was not declared in this scope; did you mean ‘tm’?
84 | int raw_h = im.h;
| ^~
| tm
In file included from /usr/include/c++/9/bits/basic_string.h:40,
from /usr/include/c++/9/string:55,
from /usr/include/c++/9/bits/locale_classes.h:40,
from /usr/include/c++/9/bits/ios_base.h:41,
from /usr/include/c++/9/ios:42,
from /usr/include/c++/9/ostream:38,
from /usr/include/c++/9/iostream:39,
from tm_mobilenet_ssd_uint8.cpp:25:
/usr/include/c++/9/ext/alloc_traits.h: In instantiation of ‘struct __gnu_cxx::__alloc_traits<std::allocator, Box>’:
/usr/include/c++/9/bits/stl_vector.h:84:21: required from ‘struct std::_Vector_base<Box, std::allocator >’
/usr/include/c++/9/bits/stl_vector.h:386:11: required from ‘class std::vector’
tm_mobilenet_ssd_uint8.cpp:86:22: required from here
/usr/include/c++/9/ext/alloc_traits.h:61:53: error: no type named ‘size_type’ in ‘struct std::allocator_traits<std::allocator >’
61 | typedef typename _Base_type::size_type size_type;
| ^~~~~~~~~
/usr/include/c++/9/ext/alloc_traits.h:62:53: error: no type named ‘difference_type’ in ‘struct std::allocator_traits<std::allocator >’
62 | typedef typename _Base_type::difference_type difference_type;
| ^~~~~~~~~~~~~~~
/usr/include/c++/9/ext/alloc_traits.h:70:23: error: no members matching ‘__gnu_cxx::__alloc_traits<std::allocator, Box>::_Base_type {aka std::allocator_traits<std::allocator >}::max_size’ in ‘__gnu_cxx::__alloc_traits<std::allocator, Box>::_Base_type’ {aka ‘struct std::allocator_traits<std::allocator >’}
70 | using _Base_type::max_size;
| ^~~~~~~~
tm_mobilenet_ssd_uint8.cpp:108:9: error: ‘draw_box’ was not declared in this scope
108 | draw_box(im, box.x0, box.y0, box.x1, box.y1, 2, 125, 0, 125);
| ^~~~~~~~
tm_mobilenet_ssd_uint8.cpp:111:5: error: ‘save_image’ was not declared in this scope
111 | save_image(im, “mobilenet_ssd_uint8_out”);
| ^~~~~~~~~~
tm_mobilenet_ssd_uint8.cpp:112:5: error: ‘free_image’ was not declared in this scope
112 | free_image (im);
| ^~~~~~~~~~
tm_mobilenet_ssd_uint8.cpp: In function ‘int main(int, char**)’:
tm_mobilenet_ssd_uint8.cpp:151:37: error: ‘OPTARG’ was not declared in this scope
151 | num_thread = atoi ( OPTARG );
| ^~~~~~
tm_mobilenet_ssd_uint8.cpp:176:10: error: ‘check_file_exist’ was not declared in this scope
176 | if (!check_file_exist(model_file) || !check_file_exist(image_file))
| ^~~~~~~~~~~~~~~~
tm_mobilenet_ssd_uint8.cpp:280:5: error: expected primary-expression before ‘/’ token
280 | / * release tengine * /
| ^
tm_mobilenet_ssd_uint8.cpp:280:9: error: ‘release’ was not declared in this scope
280 | / * release tengine * /
| ^~~~~~~
tm_mobilenet_ssd_uint8.cpp:133:11: warning: unused variable ‘mean’ [-Wunused-variable]
133 | float mean[3] = {127.5f, 127.5f, 127.5f};
| ^~~~
tm_mobilenet_ssd_uint8.cpp:134:11: warning: unused variable ‘scale’ [-Wunused-variable]
134 | float scale[3] = {0.007843f, 0.007843f, 0.007843f};
| ^~~~~
In file included from /usr/include/c++/9/string:55,
from /usr/include/c++/9/bits/locale_classes.h:40,
from /usr/include/c++/9/bits/ios_base.h:41,
from /usr/include/c++/9/ios:42,
from /usr/include/c++/9/ostream:38,
from /usr/include/c++/9/iostream:39,
from tm_mobilenet_ssd_uint8.cpp:25:
/usr/include/c++/9/bits/basic_string.h: In instantiation of ‘std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::basic_string(std::__cxx11::basic_string<_CharT, _Traits, _Alloc>&&) [with _CharT = char; _Traits = std::char_traits; _Alloc = std::allocator]’:
/usr/include/c++/9/system_error:189:40: required from here
/usr/include/c++/9/bits/basic_string.h:563:24: error: ‘class std::__cxx11::basic_string’ has no member named ‘_M_allocated_capacity’; did you mean ‘_S_local_capacity’?
563 | _M_capacity(__str._M_allocated_capacity);
| ^~~~~~~~~~~~~~~
| _S_local_capacity
/usr/include/c++/9/bits/basic_string.h: In instantiation of ‘std::__cxx11::basic_string<_CharT, _Traits, _Alloc> std::operator+(std::__cxx11::basic_string<_CharT, _Traits, _Alloc>&&, std::__cxx11::basic_string<_CharT, _Traits, _Alloc>&&) [with _CharT = char; _Traits = std::char_traits; _Alloc = std::allocator]’:
/usr/include/c++/9/system_error:351:50: required from here
/usr/include/c++/9/bits/basic_string.h:6100:33: error: ‘class std::__cxx11::basic_string’ has no member named ‘size’
6100 | const auto __size = __lhs.size() + __rhs.size();
| ^~
/usr/include/c++/9/bits/basic_string.h:6100:48: error: ‘class std::__cxx11::basic_string’ has no member named ‘size’
6100 | const auto __size = __lhs.size() + __rhs.size();
| ^~
/usr/include/c++/9/bits/basic_string.h:6101:43: error: ‘class std::__cxx11::basic_string’ has no member named ‘capacity’
6101 | const bool __cond = (__size > __lhs.capacity()
| ^~
/usr/include/c++/9/bits/basic_string.h:6102:26: error: ‘class std::__cxx11::basic_string’ has no member named ‘capacity’
6102 | && __size <= __rhs.capacity());
| ^~
/usr/include/c++/9/bits/basic_string.h:6103:32: error: no matching function for call to ‘std::__cxx11::basic_string::insert(int, std::__cxx11::basic_string&)’
6103 | return __cond ? std::move(__rhs.insert(0, __lhs))
| ~^
~~~~~~~
/usr/include/c++/9/bits/basic_string.h:1578:9: note: candidate: ‘template<class _InputIterator, class> std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::iterator std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::insert(std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::const_iterator, _InputIterator, _InputIterator) [with _InputIterator = _InputIterator; = ; _CharT = char; _Traits = std::char_traits; _Alloc = std::allocator]’
1578 | insert(const_iterator __p, _InputIterator __beg, _InputIterator __end)
| ^~~~~~
/usr/include/c++/9/bits/basic_string.h:1578:9: note: template argument deduction/substitution failed:
/usr/include/c++/9/bits/basic_string.h:6103:32: note: candidate expects 3 arguments, 2 provided
6103 | return __cond ? std::move(__rhs.insert(0, __lhs))
| ~^~~~~~~~~~~~~~~~
/usr/include/c++/9/bits/basic_string.h:1612:7: note: candidate: ‘std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::iterator std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::insert(std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::const_iterator, std::initializer_list<_Tp>) [with _CharT = char; _Traits = std::char_traits; _Alloc = std::allocator; std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::iterator = __gnu_cxx::__normal_iterator<char*, std::__cxx11::basic_string >; typename __gnu_cxx::__alloc_traits<typename __gnu_cxx::__alloc_traits<_Alloc>::rebind<_CharT>::other>::pointer = char*; std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::const_iterator = __gnu_cxx::__normal_iterator<const char*, std::__cxx11::basic_string >; typename __gnu_cxx::__alloc_traits<typename __gnu_cxx::__alloc_traits<_Alloc>::rebind<_CharT>::other>::const_pointer = const char*]’
1612 | insert(const_iterator __p, initializer_list<_CharT> __l)
| ^~~~~~
/usr/include/c++/9/bits/basic_string.h:1612:29: note: no known conversion for argument 1 from ‘int’ to ‘std::__cxx11::basic_string::const_iterator’ {aka ‘__gnu_cxx::__normal_iterator<const char*, std::__cxx11::basic_string >’}
1612 | insert(const_iterator __p, initializer_list<_CharT> __l)
| ~^
/usr/include/c++/9/bits/basic_string.h:1746:7: note: candidate: ‘std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::iterator std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::insert(std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::__const_iterator, _CharT) [with _CharT = char; _Traits = std::char_traits; _Alloc = std::allocator; std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::iterator = __gnu_cxx::__normal_iterator<char*, std::__cxx11::basic_string >; typename __gnu_cxx::__alloc_traits<typename __gnu_cxx::__alloc_traits<_Alloc>::rebind<_CharT>::other>::pointer = char*; std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::__const_iterator = __gnu_cxx::__normal_iterator<const char*, std::__cxx11::basic_string >; typename __gnu_cxx::__alloc_traits<typename __gnu_cxx::__alloc_traits<_Alloc>::rebind<_CharT>::other>::const_pointer = const char*]’
1746 | insert(__const_iterator __p, _CharT __c)
| ^
~
/usr/include/c++/9/bits/basic_string.h:1746:31: note: no known conversion for argument 1 from ‘int’ to ‘std::__cxx11::basic_string::__const_iterator’ {aka ‘__gnu_cxx::__normal_iterator<const char*, std::__cxx11::basic_string >’}
1746 | insert(__const_iterator __p, _CharT __c)
| ~^
/usr/include/c++/9/bits/basic_string.h: In instantiation of ‘std::__cxx11::basic_string<_CharT, _Traits, _Alloc> std::operator+(const _CharT*, std::__cxx11::basic_string<_CharT, _Traits, _Alloc>&&) [with _CharT = char; _Traits = std::char_traits; _Alloc = std::allocator]’:
/usr/include/c++/9/system_error:354:51: required from here
/usr/include/c++/9/bits/basic_string.h:6111:23: error: no matching function for call to ‘std::__cxx11::basic_string::insert(int, const char*&)’
6111 | { return std::move(__rhs.insert(0, __lhs)); }
| ~^
~
/usr/include/c++/9/bits/basic_string.h:1578:9: note: candidate: ‘template<class _InputIterator, class> std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::iterator std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::insert(std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::const_iterator, _InputIterator, _InputIterator) [with _InputIterator = _InputIterator; = ; _CharT = char; _Traits = std::char_traits; _Alloc = std::allocator]’
1578 | insert(const_iterator __p, _InputIterator __beg, _InputIterator __end)
| ^
~
/usr/include/c++/9/bits/basic_string.h:1578:9: note: template argument deduction/substitution failed:
/usr/include/c++/9/bits/basic_string.h:6111:23: note: candidate expects 3 arguments, 2 provided
6111 | { return std::move(__rhs.insert(0, __lhs)); }
| ~^
~~~~~~~~~~~
/usr/include/c++/9/bits/basic_string.h:1612:7: note: candidate: ‘std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::iterator std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::insert(std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::const_iterator, std::initializer_list<_Tp>) [with _CharT = char; _Traits = std::char_traits; _Alloc = std::allocator; std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::iterator = __gnu_cxx::__normal_iterator<char*, std::__cxx11::basic_string >; typename __gnu_cxx::__alloc_traits<typename __gnu_cxx::__alloc_traits<_Alloc>::rebind<_CharT>::other>::pointer = char*; std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::const_iterator = __gnu_cxx::__normal_iterator<const char*, std::__cxx11::basic_string >; typename __gnu_cxx::__alloc_traits<typename __gnu_cxx::__alloc_traits<_Alloc>::rebind<_CharT>::other>::const_pointer = const char*]’
1612 | insert(const_iterator __p, initializer_list<_CharT> __l)
| ^~~~~~
/usr/include/c++/9/bits/basic_string.h:1612:29: note: no known conversion for argument 1 from ‘int’ to ‘std::__cxx11::basic_string::const_iterator’ {aka ‘__gnu_cxx::__normal_iterator<const char*, std::__cxx11::basic_string >’}
1612 | insert(const_iterator __p, initializer_list<_CharT> __l)
| ~^
/usr/include/c++/9/bits/basic_string.h:1746:7: note: candidate: ‘std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::iterator std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::insert(std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::__const_iterator, _CharT) [with _CharT = char; _Traits = std::char_traits; _Alloc = std::allocator; std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::iterator = __gnu_cxx::__normal_iterator<char*, std::__cxx11::basic_string >; typename __gnu_cxx::__alloc_traits<typename __gnu_cxx::__alloc_traits<_Alloc>::rebind<_CharT>::other>::pointer = char*; std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::__const_iterator = __gnu_cxx::__normal_iterator<const char*, std::__cxx11::basic_string >; typename __gnu_cxx::__alloc_traits<typename __gnu_cxx::__alloc_traits<_Alloc>::rebind<_CharT>::other>::const_pointer = const char*]’
1746 | insert(__const_iterator __p, _CharT __c)
| ^
~
/usr/include/c++/9/bits/basic_string.h:1746:31: note: no known conversion for argument 1 from ‘int’ to ‘std::__cxx11::basic_string::__const_iterator’ {aka ‘__gnu_cxx::__normal_iterator<const char*, std::__cxx11::basic_string >’}
1746 | insert(__const_iterator __p, _CharT __c)
| ~^
In file included from /usr/include/aarch64-linux-gnu/c++/9/bits/c++allocator.h:33,
from /usr/include/c++/9/bits/allocator.h:46,
from /usr/include/c++/9/string:41,
from /usr/include/c++/9/bits/locale_classes.h:40,
from /usr/include/c++/9/bits/ios_base.h:41,
from /usr/include/c++/9/ios:42,
from /usr/include/c++/9/ostream:38,
from /usr/include/c++/9/iostream:39,
from tm_mobilenet_ssd_uint8.cpp:25:
/usr/include/c++/9/ext/new_allocator.h: In instantiation of ‘void __gnu_cxx::new_allocator<_Tp>::construct(_Up*, _Args&& …) [with _Up = Box; _Args = {const Box&}; _Tp = Box]’:
/usr/include/c++/9/bits/alloc_traits.h:482:2: required from ‘static void std::allocator_traits<std::allocator<_CharT> >::construct(std::allocator_traits<std::allocator<_CharT> >::allocator_type&, _Up*, _Args&& …) [with _Up = Box; _Args = {const Box&}; _Tp = Box; std::allocator_traits<std::allocator<_CharT> >::allocator_type = std::allocator]’
/usr/include/c++/9/bits/stl_vector.h:1189:30: required from ‘void std::vector<_Tp, _Alloc>::push_back(const value_type&) [with _Tp = Box; _Alloc = std::allocator; std::vector<_Tp, _Alloc>::value_type = Box]’
tm_mobilenet_ssd_uint8.cpp:99:32: required from here
/usr/include/c++/9/ext/new_allocator.h:145:20: error: no matching function for call to ‘operator new(sizetype, void*)’
145 | noexcept(noexcept(::new((void *)__p)
| ^
~
146 | _Up(std::forward<_Args>(__args)…)))
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
: note: candidate: ‘void* operator new(long unsigned int)’
: note: candidate expects 1 argument, 2 provided
In file included from /usr/include/c++/9/string:55,
from /usr/include/c++/9/bits/locale_classes.h:40,
from /usr/include/c++/9/bits/ios_base.h:41,
from /usr/include/c++/9/ios:42,
from /usr/include/c++/9/ostream:38,
from /usr/include/c++/9/iostream:39,
from tm_mobilenet_ssd_uint8.cpp:25:
/usr/include/c++/9/bits/basic_string.h: In instantiation of ‘void std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::_M_dispose() [with _CharT = char; _Traits = std::char_traits; _Alloc = std::allocator]’:
/usr/include/c++/9/bits/basic_string.h:658:9: required from ‘std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::~basic_string() [with _CharT = char; _Traits = std::char_traits; _Alloc = std::allocator]’
/usr/include/c++/9/system_error:189:40: required from here
/usr/include/c++/9/bits/basic_string.h:232:15: error: using invalid field ‘std::__cxx11::basic_string<_CharT, _Traits, _Alloc>::::_M_allocated_capacity’
232 | _M_destroy(_M_allocated_capacity);
| ^
~~~~~~~~~~~~~~
In file included from tm_mobilenet_ssd_uint8.cpp:30:
common.h: At global scope:
common.h:70:13: warning: ‘void split(float*, char*, const char*)’ defined but not used [-Wunused-function]
70 | static void split(float* array, char* str, const char* del)
| ^~~~~
make: *** [makefile-cv4.linux:51: cv5_output/tm_mobilenet_ssd_uint8.o] Fehler 1

I am not able to generate the execuable file :confused:

Can you help me analyse the issue?

@Akkisony I think I have told you everything I can help you

@Frank The below is my post processing function.

void post_process_ssd(const char* image_file, float threshold, float* outdata, int num)
{
const char* class_names[] = {“background”, “aeroplane”, “bicycle”, “bird”, “boat”, “bottle”,
“bus”, “car”, “cat”, “chair”, “cow”, “diningtable”,
“dog”, “horse”, “motorbike”, “person”, “pottedplant”, “sheep”,
“sofa”, “train”, “tvmonitor”};

image im = imread(image_file);

int raw_h = im.h;
int raw_w = im.w;
std::vector<Box> boxes;
printf("detect result num: %d \n", num);
for (int i = 0; i < num; i++)
{
    if (outdata[1] >= threshold)
    {
        Box box;
        box.class_idx = round(outdata[0]);
        box.score = outdata[1];
        box.x0 = outdata[2] * raw_w;
        box.y0 = outdata[3] * raw_h;
        box.x1 = outdata[4] * raw_w;
        box.y1 = outdata[5] * raw_h;
        boxes.push_back(box);
        printf("%s\t:%.2f\n", class_names[box.class_idx], box.score * 100.f);
        printf("BOX:( %d , %d ),( %d , %d )\n", ( int )box.x0, ( int )box.y0, ( int )box.x1, ( int )box.y1);
    }
    outdata += 6;
}
for (int i = 0; i < ( int )boxes.size(); i++)
{
    Box box = boxes[i];
    draw_box(im, box.x0, box.y0, box.x1, box.y1, 2, 125, 0, 125);
}

save_image(im, "mobilenet_ssd_uint8_out");
free_image (im);
std::cout << "======================================\n";
std::cout << "[DETECTED IMAGE SAVED]:\t"
          << "Mobilenet_SSD"
          << "\n";
std::cout << "======================================\n";

}

Is the post processing right?

@Akkisony Sorry, I have not tried the SSD model, not sure if this post-processing is feasible

@frank @alcohol
I am getting the following error. “Segmentation Fault” when I try to do inference using Tengine SDK.
Do you have any idea about the possible solutions for this?
Please find the attached screenshot.

@Akkisony You should add printing information in the code to confirm where this error occurred. This is a very common error, and I can’t analyze it.

@Frank Can you please tell me the significance of these parameters.

  • float mean[3] = {127.5f, 127.5f, 127.5f};
  • float scale[3] = {0.007843f, 0.007843f, 0.007843f};

Is this default for Uint8 data type, if not how is it calculated?

Another file, tm_classification_uint8.c has the following parameter:

#define DEFAULT_SCALE1 0.017f
#define DEFAULT_SCALE2 0.017f
#define DEFAULT_SCALE3 0.017f
#define DEFAULT_MEAN1 104.007
#define DEFAULT_MEAN2 116.669
#define DEFAULT_MEAN3 122.679

I just want to know how is it calculated?
Thank you for your input in advance! :slight_smile:

@Akkisony This is to normalize the pre-processing data to [-1,1]

@Frank Thank you for the answer. But, why is it different in different files? I am just curious to know as both of them does the same thing.
Script 1: tm_mobilenet_ssd_uint8.cpp has the following below parameters.

Script 2: tm_classification_uint8.c has the below parameters.

  1. Is there any reason for different values?
  2. How do you set values to the range of [0, 1]?

@Akkisony This is based on the structure of the model itself, I think you need to understand the most basic AI-related knowledge, otherwise you will not be able to solve your problem