Unable to convert tflite model ( efficientlite architecture) to .nb. Does efficient lite architecture is supported by acuity-models?

VIM3
1

Which Khadas SBC do you use?

VIM3 A311D

Which system do you use? Android, Ubuntu, OOWOW or others?

Ubantu

Please describe your issue below:

I was running the following script to convert efficient_lite0.tflite model to .nb.
Script is
./convert --model-name efficientlite
–platform tflite
–model efficient_lite0.tflite
–mean-values ‘128 128 128 128’
–quantized-dtype asymmetric_affine
–source-files Desktop/aml_npu_sdk/acuity-toolkit/python/data_1/dataset/dataset0.txt --kboard VIM3
–print-level 1

Post a console log of your issue below:

er
er1073×476 47.4 KB 

Can you please guide on this.
Link of model: [https://tfhub.dev/tensorflow/lite-model/efficientdet/lite0/detection/default/1](https://tfhub.dev/tensorflow/lite-model/efficientdet/lite0/detection/default/1)
2

Hello @Chetan_Deshmukh

@Louis-Cheng-Liu will help you about this issue.

3

Eagerly waiting for reply from @Louis-Cheng-Liu :slight_smile:

4

Still waiting @Louis-Cheng-Liu :sweat_smile:

5

Hello @Chetan_Deshmukh ,

First of all, i am very sorry for you that i forgot to reply you.

This week Monday, i have reproduce your problem successfully. I have given feedback to our KSNN API engineer. He replied he need time to find the cause of problem and fixed it. Maybe next week.

2 Likes
6

Thanks for reply :slight_smile: Please update here as you come to know the solution, till then I will be waiting for a reply from your side. Thank you once again.

7

Hello @Chetan_Deshmukh ,

In the end of this model, a layer named TFLite_Detection_PostProcess is a unsupport operation. From its name, we guess it is a customized postprocess operation.

To convert it, you should specify input and output and the output is the input of TFLite_Detection_PostProcess. The script is as follows.

./convert --model-name efficientlite \
               --platform tflite \
               --model efficientdet_lite0.tflite \
               --inputs serving_default_images:0 \
               --input-size-list '320,320,3' \
               --outputs  "'Sigmoid1 tfl.dequantize'" \
               --mean-values "128 128 128 0.0078125" \
               --quantized-dtype asymmetric_affine \
               --source-files ./data/dataset/dataset0.txt \
               --kboard VIM3 --print-level 1

Then, you should do postprocess by yourself if you want to run model on the borad.

Moreover, this model type is tflite int8. Now, our tool int8 type is not the same as tflite int8. So, we can not make sure the performance of converted model is as good as the original model. But our tool is support tflite uint8.

1 Like
8

Thank you @Louis-Cheng-Liu for your fruitful response. Only the the point is left, how I can do the post process after running nn_inference on this model using ksnn tool. I know you have mentioned that “you should do postprocess by yourself if you want to run model on the borad.” but there is no camera demo examples for tflite in ksnn ( only available for yolo-darknet.) and also there is no explanation are available for this post process in documentation or in examples. Again, if you provide camera demo link for tflite platform or any documentation related to it will be very helpful for us, because conversion of above model is only then meaningful.

9

Hello @Chetan_Deshmukh ,

Sorry, maybe i did not describe clearly. The postprocess i mean is the operation of TFLite_Detection_PostProcess. This layer is to decode data from the model outputs to actual results. Because of removing the layer, you should do the same operation by python after getting output from running nn_inference. Each model has its own decoding method. So, we can not do all model’s postprocess. In a word, you should decode data by yourself.

The way, decoding, you can try to search on website.

Camera demo you can refer this.
ksnn/examples/darknet/yolov3-cap.py at master · khadas/ksnn · GitHub
After converting model, it is the same whatever the original platform.

1 Like
10

Thank You @Louis-Cheng-Liu for your pin pointed response. I went through Camera demo and implementing the same with above converted tflite model. Right now I am stuck to a input0_data = input0_data.reshape(SPAN, LISTSIZE, GRID0, GRID0)
input1_data = input1_data.reshape(SPAN, LISTSIZE, GRID1, GRID1)
input2_data = input2_data.reshape(SPAN, LISTSIZE, GRID2, GRID2). I know that length of input_data is nothing but multiplication of spanlistsizegrid*grid. But can you please tell me on which criteria or formula these four parameters are based on ?

11

Hello @Chetan_Deshmukh ,

These are part of yolov3 postpocess. The name, input_data, may mislead you. output_data is more suitable for this variable. All of them are model output and yolov3 has three kinds of output.

SPAN is 3. It is the number of bounding boxes. LISTSIZE is 85. Its number is related to how many classes the model can detect. Index 0 to 3 is the information of box. Index 4 is the confidence of the box. Index 5 to last is each class confidence. This model can detect 80 different types of objects. So LISTSIZE is 4+1+80=85. yolov3 has three different sizes of output, 13×13, 26×26 and 52×52. GRID corresponds to those.

1 Like
12

Thank you @Louis-Cheng-Liu for your informative response. We are using Vim3 board for our python application on ubuntu 18 os. Vim3 has mali G52 gpu, How we can use this available gpu to accelerate our application ? Or can you guide how we can accelerate python applications on this board.

13

If want to use the GPU you can try OpenCL.

14

Thank you @Louis-Cheng-Liu @numbqq for support. This query is regarding the post process of above model. We have converted it to .nb file, but are unable to postprocess. I know you have replied us as: " So, we can not do all model’s postprocess. In a word, you should decode data by yourself. The way, decoding, you can try to search on websites". But after refering all websites and trying all possible solutions with my team , we are not able to get bounding box, scores and classes for above model. We have also refered all the KSNN examples and post process after nn_inference but still we are not getting bounding box, scores and classes for object detection using camera for above mentioned model. So, requesting you to provide post process for this model as this procedure is very important to run the model on NPU. Please provide us with sample codes and documentation for the same. Thank you in advance.

15

Hello @Chetan_Deshmukh ,

I am very sorry that we have not available time to do it. But i can give you some suggestions.

First, you can refer EfficientDet paper and origin code. The paper link.
[1911.09070] EfficientDet: Scalable and Efficient Object Detection (arxiv.org)

Second, run the codes to train a new model and inference the model. Find the codes from getting model output to drawing the result. Try to understand what it does.
I recommend a online tool which can isualize the model structure and help you understand easier.
Netron

At last, all roads lead to Rome. This model is a object detection model. You can try to use other model. For example, YOLO is the object detection, too.