OSA810 running at 15fps

steely-glint · August 29, 2023, 9:13am

Which system do you use? Android, Ubuntu, OOWOW or others?

Ubuntu

Which version of system do you use? Khadas official images, self built images, or others?

Official
Linux k4 5.4.180 #1.5 SMP PREEMPT Tue Apr 25 12:18:15 UTC 2023 aarch64 aarch64 aarch64 GNU/Linux

Please describe your issue below:

When using the OSA810 camera with gstreamer I only get 15 fps.

gst-launch-1.0 -v
v4l2src device=/dev/video50 io-mode=mmap
! video/x-raw,width=1920,height=1080,framerate=30/1,format=NV12
! videoconvert ! queue ! fpsdisplaysink

Post a console log of your issue below:

/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 523, dropped: 0, current: 15.01, average: 15.03

numbqq · August 29, 2023, 10:43am

Hello @steely-glint

@Jacobe will check this issue and update here.

steely-glint · August 29, 2023, 11:01am

Thanks. This is an original VIM4 - not a new VIM4

Jacobe · August 30, 2023, 3:30am

Hello @steely-glint

Apologize for getting back to you so late~
Here I have tested the issue you raised, and indeed, this situation exists. Moving forward, we will conduct further debugging.
If there is any progress, I will provide you with updates. Thank you for your patience during this time of waiting

Jacobe · September 5, 2023, 3:09am

Hello, @steely-glint

Good news!

We solved this problem, after testing it can not only be stabilized at 1920x1080 30 fps, but also at 3840x2160 30 fps.

But the steps can be a little complicated Please follow these steps.

Clone multimedia debs

git clone https://github.com/numbqq/multimedia_debs_aml.git

Install these(not all)

sudo dpkg -i multimedia_debs_aml/focal/arm64/gst-aml-drmbufferpool-plugins_0.1-202307_arm64.deb
sudo dpkg -i multimedia_debs_aml/focal/arm64/mediactrlsrc_0.1-202307_arm64.deb

Then clone gst related debs

git clone https://github.com/numbqq/gstreamer_aml.git

Install them in order

sudo dpkg -i gstreamer-aml_0.3-202307_arm64.deb

cd gstreamer1.0-debs/
sudo dpkg -i libgstreamer1.0-0_1.20.3-0ubuntu1_arm64.deb
sudo dpkg -i gir1.2-gstreamer-1.0_1.20.3-0ubuntu1_arm64.deb
sudo dpkg -i gstreamer1.0-tools_1.20.3-0ubuntu1_arm64.deb

cd ../gst-plugins-base1.0-debs/
sudo dpkg -i libgstreamer-plugins-base1.0-0_1.20.1-1_arm64.deb
sudo dpkg -i libgstreamer-gl1.0-0_1.20.1-1_arm64.deb
sudo dpkg -i gstreamer1.0-plugins-base_1.20.1-1_arm64.deb
sudo dpkg -i gstreamer1.0-plugins-base-apps_1.20.1-1_arm64.deb
sudo dpkg -i gstreamer1.0-alsa_1.20.1-1_arm64.deb
sudo dpkg -i gir1.2-gst-plugins-base-1.0_1.20.1-1_arm64.deb

cd ../gst-plugins-good1.0-debs/
sudo dpkg -i libgstreamer-plugins-good1.0-0_1.20.3-0ubuntu1_arm64.deb
sudo dpkg -i gstreamer1.0-plugins-good_1.20.3-0ubuntu1_arm64.deb
sudo dpkg -i gstreamer1.0-pulseaudio_1.20.3-0ubuntu1_arm64.deb

cd ../gst-plugins-bad1.0-debs/
sudo dpkg -i libgstreamer-plugins-bad1.0-0_1.20.3-0ubuntu1_arm64.deb
sudo dpkg -i gstreamer1.0-plugins-bad_1.20.3-0ubuntu1_arm64.deb

sync
sudo reboot

Download this file.

https://dl.khadas.com/.test/libgstvideo4linux2.so

Replace old one

sudo cp libgstvideo4linux2.so /usr/lib/aarch64-linux-gnu/gstreamer-1.0/

Then clear the cache directory

rm .cache/gstreamer-1.0/ -rf

Final you can run it!

gst-launch-1.0 -v v4l2src device=/dev/video50 io-mode=mmap ! video/x-raw,width=1920,height=1080,framerate=30/1,format=NV12 ! videoconvert ! queue ! fpsdisplaysink

The followings are the result that run by me.

/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 64, dropped: 0, current: 30.01, average: 30.25

Last but not least, if you think these steps too complicated, you can also wait for the new kernel released by Khadas Team in Q3

steely-glint · September 5, 2023, 7:04am

Thanks for the quick work.

I’ve done that and it works!

I had to add
cd ./gstreamer_aml/jammy/arm64/VIM4/5.15/
before
sudo dpkg -i gstreamer-aml_0.3-202307_arm64.deb

One problem however - In my application I’m using the amlvenc to encode to h264 and it’s no longer available - only decoders.

khadas@k4:~$ gst-inspect-1.0 | grep aml
amljpegenc:  amljpegenc: Amlogic JPEG image encoder
amlv4l2:  amlv4l2av1dec: AML V4L2 AV1 Decoder
amlv4l2:  amlv4l2h264dec: AML V4L2 H264 Decoder
amlv4l2:  amlv4l2h265dec: AML V4L2 H265 Decoder
amlv4l2:  amlv4l2jpegdec: AML V4L2 JPEG Decoder
amlv4l2:  amlv4l2mpeg4dec: AML V4L2 MPEG4 Decoder
amlv4l2:  amlv4l2vp9dec: AML V4L2 VP9 Decoder

Can I get it back somehow ?

Thanks.

steely-glint · September 5, 2023, 7:11am

@Jacobe Oh, and when is Q3 ? (I mean by which calendar - Chinese or Western or Western financial ?)
T.

Jacobe · September 5, 2023, 10:27am

Western financial.

Before 10.31 the new kernel will release.

About the encoder problem, still fixing

numbqq · September 6, 2023, 1:15am

Hello @steely-glint

We are not developing the new 5.15 kernel for VIM4, and will release testing images of 5.15 kernel for VIM4 in Q3, about the final release will be in Q4.

@Jacobe will provide you a solution later.

Jacobe · September 6, 2023, 6:23am

Hello @steely-glint

According to the h264 encoder issue, I’ve fixed.

First you should reflash firmware into the SBC.Then follow these steps.

sudo apt update
sudo apt-get install libgstreamer-plugins-good1.0-0

Download the .deb from this link.
https://dl.khadas.com/.test/gstreamer1.0-plugins-good_1.20.3-0ubuntu1.1_arm64.deb
Install it.

sudo dpkg -i gstreamer-plugin-good_*.deb

After these you can find the h264 encoder as well your program runs at 30 fps.

gst-inspect-1.0 | grep aml
amljpegenc:  amljpegenc: Amlogic JPEG image encoder
amlvenc:  amlvenc: Amlogic h264/h265 Multi-Encoder

Cheers!

steely-glint · September 6, 2023, 11:55am

Thanks, I tried that and it seemed to make things worse.

gst-launch-1.0 -v v4l2src device=/dev/video50 io-mode=mmap ! video/x-raw,width=1920,height=1080,framerate=30/1,format=NV12 ! videoconvert ! queue ! fpsdisplaysink

Gives me

WARNING: from element /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstAutoVideoSink:fps-display-video_sink/GstKMSSink:fps-display-video_sink-actual-sink-kms: A lot of buffers are being dropped.
Additional debug info:
../libs/gst/base/gstbasesink.c(3143): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstAutoVideoSink:fps-display-video_sink/GstKMSSink:fps-display-video_sink-actual-sink-kms:
There may be a timestamping problem, or this computer is too slow.
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 21, dropped: 164, fps: 1.81, drop rate: 12.70

But I do have amlvenc

Here’s what I did

reinstall OS from OWOWOO
Vim4-ubuntu-22.04-server-linux-5.4-fenix-1.5.230425.img.xz

sudo apt update
sudo apt-get install libgstreamer-plugins-good1.0-0
wget https://dl.khadas.com/.test/gstreamer1.0-plugins-good_1.20.3-0ubuntu1.1_arm64.deb
sudo dpkg -i gstreamer1.0-plugins-good_1.20.3-0ubuntu1.1_arm64.deb

What did I do wrong ?

Thanks.

Tim.

Jacobe · September 7, 2023, 1:57am

I make sure my platform has the same environment as yours and tried all about these, but it still works well.

Please post more logs both about kernel and gstreamer.

steely-glint · September 7, 2023, 12:10pm

Jacobe I have re-built it and now it works,
I think the problem was that I did
apt-get upgrade after installing the new deb and it over wrote it.

Thanks for your help.

I’m happy to do any tests on new versions.

Tim.