I would also like to voice a need for this functionality. Using the HDMI input with ffmpeg is the reason I purchased the boards. I can’t seem to use the input with /anything/. Seeing the video input working in Android is encouraging, but I’m disappointed that there seems to be no way to use the video input for anything useful at the time of release. Will be following this thread very closely and posting any updates I might find.
2 Likes
+1 for gstreamer src
Hello @grbur5t @Farwisderax @blckcoffee
May I know the specific usage scenarios of HDMI IN and ffmpeg?
Thank you for your reply!
I was unfamiliar with gstreamer, but it looks as though that will suit my needs as well.
Basically if the board is on I want it to be broadcasting a stream of whatever is coming into the input at any given time with no need for human interaction if the input is removed or swapped. So ideally it would continue to stream even if there is no input and maybe have an OSD that reflects as much. But I digress.
TLDR: HDMI to Stream with as little latency as possible. Trying to go webrtc unless I have no other option.
Cheers!
I’d use it as a gateway to other apps.
My specific use case would be to send it thru some custom opengl mapping shaders and output the result to an udp stream.
(hdmi source would be video synth (computer, hardware or even cameras such as a gopro)
I can see another usecase feeding hdmi to opencv (but that’s not my primary usecase)
I will add my hopes here. Similar to what others have said, I want to stream the HDMI In via gstreamer via RTSP. I do this right on other SBC using usb capture devices, but this seems like one less thing to worry about. So having v4l2 input, and preferably also hardware encoding in gstreamer would be ideal.
1 Like
I do agree here.
In my opinion a product which is advertised to have a certain feature (HDMI, H.264/H.265 HW encode) should support this through a standard interface on the supported OS(s).
So for hdmi in v4l2 backend so that it appears as a video input device and for the encoders a lib for ffmpeg/gstreamer.
What happens in between is up to the customer
4 Likes
Hello @numbqq my use case is the following.
I need to stream data from a HDMI live source (camera) over a encoded (h.264/h.265) MPEG2 TS.
1 Like
Hi @numbqq ,
As others have mentioned building a driver to support v4l2 would be good enough. From there multiple applications would be able to pull frames and audio from it (not limited to gst or ffmpeg only).
Wonder if you have any timeline for gstreamer implementation? Do you need beta testers? Do reach out if you do.
Thanks for working on this!
Yes, the driver already supports V4L2.
Gstreamer decoder can work on my side now, but still have some issues, e.g display can only on Wayland Weston, and still have some bugs.
I will publish a test image for you guys to check this month.
4 Likes
Cool thanks so much for working hard on this (= Probably have to build GStreamer from scratch through meson and ninja rather than use the packages from ubuntu if you need additional Wayland support. Minimally the core libraries though you’d probably want a clean install of gst cos the plugins might interfere if you just rebuild the core.
Can you document how to open the v4l2 device? Then I (and assumed many others) could already try some parts of the Pipeline (without encoding)
2 Likes
v4l2 device would indeed be perfect
Good to hear!! Thanks for your time spent solving this issue.
Here is a guide about gstreamer hardware decoding based on latest image 220620, you can OOWOW to install it, here are the steps.
- Install weston & gstreamer
$ sudo apt update
$ sudo apt install weston gstreamer1.0-plugins-bad
- Reboot and login with weston
You need to choose weston im the login prompt not ubuntu.
- Setup parameters
$ echo 1 | sudo tee /sys/module/amvdec_ports/parameters/multiplanar
$ echo 1 | sudo tee /sys/module/amvdec_ports/parameters/bypass_vpp
$ echo 0 | sudo tee /sys/module/amvdec_ports/parameters/enable_drm_mode
- Open the terminal and execute the gstreamer pipe below
$ gst-launch-1.0 playbin uri=file:///home/khadas/bbb_sunflower_1080p_60fps_normal.mp4 video-sink=waylandsink
or
$ gst-launch-1.0 filesrc location=/home/khadas/bbb_sunflower_1080p_60fps_normal.mp4 ! qtdemux name=d d.video_0 ! h264parse ! v4l2h264dec capture-io-mode=4 ! video/x-raw,format=NV12 ! waylandsink
NOTE1: If you got a black screen, please move the mouse to top panel and move back you will see the video.
NOTE2: If you want to stop the video, you have to use uart console or ssh to kill the gst process.
$ killall -9 gst-launch-1.0
Hi Nick thanks for this. This is for hw decoding, wonder if you happen to know which V4L2 device HDMI IN is? I see a bunch of them when trying to tap the various devices. The board enumerates devices up to video70
It seems that the HDMI IN is not based on V4L2.
What would be the right way to access it then?
Good to know we can access h264 decoder thru v4l2
Does encoder works as well?
Thanks for letting us know @numbqq. If v4l2 is not going to be implemented, any chance that aml_enc can be passed to stdout instead of to a h264 file dump?