Tensorflow Movement

VIM3
1

@numbqq, @Frank
HI,
WIll NPU convert tool supports tensorflow movement?
I am using Khadas VIM3 with android,
and i need fast pose detection,
open pose and alpha pose is not enough.

Model:
https://tfhub.dev/google/movenet/multipose/lightning/1

Or at least one of these:
https://tfhub.dev/google/movenet/singlepose/thunder/4
https://tfhub.dev/google/movenet/singlepose/lightning/4

2

@banak It seems to be using the tensorflow api, so it should be convertible

3

@Frank
I used tutorial:

Command (0_import_model.sh) WORKS:

$convert_tflite \
    --tflite-mode  model/lite-model_movenet_singlepose_thunder_3.tflite \
    --net-output ${NAME}.json \
    --data-output ${NAME}.data

Command (1_quantize_model.sh) ERROR:

$tensorzone \
    --action quantization \
    --dtype float32 \
    --source text \
    --source-file data/validation_tf.txt \
    --channel-mean-value '128 128 128 128' \
    --reorder-channel '0 1 2' \
    --model-input ${NAME}.json \
    --model-data ${NAME}.data \
    --quantized-dtype asymmetric_affine-u8 \
    --quantized-rebuild

Error:

Traceback (most recent call last):
  File "tensorflow/python/framework/op_def_library.py", line 470, in _apply_op_helper
  File "tensorflow/python/framework/ops.py", line 1499, in convert_to_tensor
  File "tensorflow/python/ops/resource_variable_ops.py", line 1909, in _dense_var_to_tensor
  File "tensorflow/python/ops/resource_variable_ops.py", line 1322, in _dense_var_to_tensor
ValueError: Incompatible type conversion requested to type 'int64' for variable of type 'int32'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "tensorzonex.py", line 445, in <module>
  File "tensorzonex.py", line 386, in main
  File "acuitylib/app/tensorzone/quantization.py", line 176, in run
  File "acuitylib/app/tensorzone/quantization.py", line 116, in _run_quantization
  File "acuitylib/app/tensorzone/workspace.py", line 184, in _setup_graph
  File "acuitylib/app/tensorzone/graph.py", line 59, in generate
  File "acuitylib/acuitynetbuilder.py", line 327, in build
  File "acuitylib/acuitynetbuilder.py", line 358, in build_layer
  File "acuitylib/acuitynetbuilder.py", line 358, in build_layer
  File "acuitylib/acuitynetbuilder.py", line 358, in build_layer
  File "acuitylib/acuitynetbuilder.py", line 358, in build_layer
  File "acuitylib/acuitynetbuilder.py", line 358, in build_layer
  File "acuitylib/acuitynetbuilder.py", line 358, in build_layer
  File "acuitylib/acuitynetbuilder.py", line 358, in build_layer
  File "acuitylib/acuitynetbuilder.py", line 358, in build_layer
  File "acuitylib/acuitynetbuilder.py", line 358, in build_layer
  File "acuitylib/acuitynetbuilder.py", line 358, in build_layer
  File "acuitylib/acuitynetbuilder.py", line 358, in build_layer
  File "acuitylib/acuitynetbuilder.py", line 358, in build_layer
  File "acuitylib/acuitynetbuilder.py", line 358, in build_layer
  File "acuitylib/acuitynetbuilder.py", line 358, in build_layer
  File "acuitylib/acuitynetbuilder.py", line 358, in build_layer
  File "acuitylib/acuitynetbuilder.py", line 358, in build_layer
  File "acuitylib/acuitynetbuilder.py", line 358, in build_layer
  File "acuitylib/acuitynetbuilder.py", line 358, in build_layer
  File "acuitylib/acuitynetbuilder.py", line 393, in build_layer
  File "acuitylib/layer/acuitylayer.py", line 306, in compute_tensor
  File "acuitylib/layer/math_binary_op.py", line 129, in compute_out_tensor
  File "tensorflow/python/util/dispatch.py", line 201, in wrapper
  File "tensorflow/python/ops/math_ops.py", line 1408, in floordiv
  File "tensorflow/python/ops/gen_math_ops.py", line 3817, in floor_div
  File "tensorflow/python/framework/op_def_library.py", line 506, in _apply_op_helper
TypeError: Input 'y' of 'FloorDiv' Op has type int32 that does not match type int64 of argument 'x'.
4

@banak

  1. The tool has requirements for the version of tensorflow, and not all versions of tensorflow can be converted. You need to know the version of your model when it was trained. Try not to be higher than the version supported by the tool.

  2. There is a document of all supported layers in the document. You should compare and check whether your model has unsupported layers and operators.

From my experience, this problem is the API problem of operators

5

Version of tensorflow in tool is 2.0.0,
thet is more then 2 years old.

I search operation of “FloorDiv”,
it seams to be caused by incompatibility of old tensorvlow, but i am not sure.

My origin question, calculate differently:
Will convert tool supports newer version of tensorflow in next 6 months?
Because is useless buy more then 300 pieses khadas, when development is stopped.
:confused:

Khadas is very good singleboard for my tasks,
but with old NPU stuff i can not complete tasks which starts in 2 months.

In fact some things even currently must run on intel i7 (Cheaper then Jetson Xavier :smiley: ),
because of tensorflow incompatibility on khadas.

6

@banak The release time of the next new version is uncertain, if it is a version problem. It may be difficult to solve. Even if the next version can be upgraded, it will not be upgraded to the latest tengsorflow. Generally, only about 2 versions will be upgraded.

7

wait… the next version…

8

@banak did you get any pose estimation model working at >5 FPS? I have the exact same application (fast-ish human pose estimation) with the same Multipose Lightning model on a Pi 4 and can’t seem to pass 4 FPS. I was considering getting the VIM3 until I saw this thread. Any update on getting it to work? I think I remember there being an issue with running the current Multipose model on TPU’s because it does a processing step on the CPU (not TPU).

9

@jakabo27

On CPU:
movenet/multipose/lightning up to 6-FPS
movenet/singlepose/lightning up to 14-FPS

On NPU:
OpenPose low precision up to 20FPS,
with self-correcting codes usable but FPS max 16 and only snglepose

  • all cases tested on Android 9.0
  • image preprocesing by OpenCV → not part of FPS
2 Likes
10

@banak
Hello, could I please get access to the model you converted?