How to run the person semantic segmentation or masking model on the VIM3 NPU using the vim3_npu_applications_lite SDK

VIM3
1

Which system do you use? Android, Ubuntu, OOWOW or others?

Ubuntu

Which version of system do you use? Please provide the version of the system here:

vim3-ubuntu-20.04-gnome-linux-4.9-fenix-1.4-221229-emmc.img

Please describe your issue below:

Hi everyone,

I’m working on a person semantic segmentation or person masking project on a Khadas VIM3 and need some help running my segmentation model on the NPU using the vim3_npu_applications_lite SDK.

I have a YOLOv8 segmentation model that runs perfectly in Python using the Khadas NPU Python SDK with KSNN. I have:

  • The compiled yolov8n_seg.nb model

  • The corresponding .so library

  • A working Python script that does person segmentation on the NPU

What I’d like to do now is:

Run this same yolov8n_seg.nb segmentation model on the VIM3 NPU using the vim3_npu_applications_lite SDK (not KSNN).

My problem is:

  • The vim3_npu_applications_lite SDK examples only cover detection / classification (no segmentation examples that I can see).

  • The vim3_npu_applications_lite SDK doesn’t use KSNN, so I’m not sure how to correctly set up the graph / tensors for a segmentation model and map the output to a mask.

Could someone please guide me on:

  1. Whether it’s actually possible to run this .nb segmentation model directly with the vim3_npu_applications_lite SDK.

  2. What extra files / configs are needed (e.g. JSON, BIN, tensor layout, etc.), given that I currently only have the .nb and .so.

  3. Any sample C++ code, repo, or documentation that shows how to handle segmentation outputs on the VIM3 NPU in C++.

If you need any more details (model input/output shapes, my Python code, logs, etc.), I’m happy to share them.
Thanks a lot in advance
– this is an important part of my project. :folded_hands:

2

Hello @OM.KOTHALKAR

You can follow the documentation to convert your own model.