Media gateway with AI

VIM3
1

Hi all,

I’m new to VIM3 but seems as a good fit for a project I’m involved. At the same time, I have found other RK3399Pro alternative that also makes sense.

The project is about to open a RTSP/RMTP stream, do some object detection / tracking stuff, crop the original video and reencode it in RTMP. The fact I need to make video decoding / encoding plus some AI on top of it makes VIM3 as a great candidate. If not for the video assisted stuff, it might make more sense to do this on x86 device with a Coral card or similar.

In order to make a decision I would like to clarify some points:

  1. Is the video decoding / encoding assisted by hardware in Linux? This is critical, as it is the main reason to opt for an ARM based system instead of a x86 + Coral. What applications do support it, maybe FFMPEG?

  2. I have seen in some forums Wifi is not supported in Linux. Ths is not critical for the project, just an inconvenience. Please confirm.

  3. In order to use the NPU, I need to request access for it. This is ok, but please confirm it works under Linux.

As for the project itself, if anybody is interested in participating or any company with experience in this field, please contact me

2

Hello @jnebrera
Nice to hear about your project, here is some answers to your question…

This one is still wip, panfrost drivers for the GPU in mainline linux are seen to be around the corner, at the moment they are software accelerated, which is very intensive, and slow…

Wifi works well, fully supports AC wifi, shouldn’t be facing a problem there…

Yes the NPU currently works, but only with the legacy kernel (4.9)

sorry that some of the things are still hit or miss, but that is mostly due to the reason that most of the stuff is proprietary and locked by Amlogic…

If there is anything else the community can do help please ask here,

Cheers

3

Well then, I think this alternative should be discarded at this moment due to lack of proper Linux support. It’s a pitty. Thanks for the clarification

4

I think the drivers for decoders are there, but you cannot use them through FFMPEG or gstreamer… @numbqq maybe you can confirm on this.

The fact is even if GPU drivers are there FFMPEG and gstreamer would not support it… You will need to manually add support for it. This goes for RK3399 pro also

since vim3 has 6 cores you can easily do 1080 soft enc/dec at 30 FPS

I would say you go with jetson nano/NX they have support for hardware enc/dec through gstreamer

5

Yep, that’s a better option for the application desired