How use gstreamer to play/stream youtube video in 1080/60 or higher with audio

ARM
1

Device: Opi5-Plus (RK3588)
Kernel: linux-rc-6.14-rc7-1

Per my understanding kernel-linux-rc-6.14-rc7-1 does have vpu hardware acceleration support.

[jfl@opi5plus ~]$ lsmod | grep v4l2
v4l2_vp9               24576  1 hantro_vpu
v4l2_jpeg              16384  1 hantro_vpu
v4l2_h264              16384  2 hantro_vpu,rockchip_vdec2
v4l2_mem2mem           36864  3 hantro_vpu,rockchip_rga,rockchip_vdec2
videobuf2_v4l2         28672  4 hantro_vpu,rockchip_rga,rockchip_vdec2,v4l2_mem2mem
videodev              270336  5 videobuf2_v4l2,hantro_vpu,rockchip_rga,rockchip_vdec2,v4l2_mem2mem
videobuf2_common       65536  8 videobuf2_dma_contig,videobuf2_v4l2,hantro_vpu,rockchip_rga,videobuf2_dma_sg,rockchip_vdec2,v4l2_mem2mem,videobuf2_memops
mc                     69632  6 videodev,videobuf2_v4l2,hantro_vpu,videobuf2_common,rockchip_vdec2,v4l2_mem2mem
[jfl@opi5plus ~]$

Read that gstreamer does have v4l2-request support and it most likely have vpu hardware acceleration with kernel-linux-rc-6.14-rc7-1. I have limited or no knowledge of using gstreamer to play or stream youtube video with audio. I have tried the follow gstreamer command to stream youtube video but t he video output with a window around 360p or so.

gst-launch-1.0 souphttpsrc is-live=true location="$(yt-dlp --format "best[ext=mp4][height<=?1080][fps<=?60][protocol=https]" --get-url https://www.youtube.com/watch?v=7PIji8OubXU)" ! qtdemux name=demuxer  demuxer. ! queue ! avdec_h264 ! autovideosink  demuxer.audio_0 ! queue ! avdec_aac ! audioconvert ! audioresample ! autoaudiosink
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Got context from element 'souphttpsrc0': gst.soup.session=context, session=(GstSoupSession)NULL;
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Redistribute latency...
Redistribute latency...
Redistribute latency...
Redistribute latency...
Redistribute latency...
^Chandling interrupt. (1.2 %)
Interrupt: Stopping pipeline ...
Execution ended after 0:00:08.683770572
Setting pipeline to NUL

The above gstreamer command is basically from How to watch Youtube videos with Gstreamer - LifeStyleTransfer

Wanted to gstreamer to output or select youtube video at up to 1080p/60fps instead of the default small window resolution. But have no knowledge how to achieve the goal.

Appreciate some tips and guidance.

2

with firefox you can add enhanced-h264ify , and select format h264(avc) and AV1 ( block others formats and keep 60 fps )
for exemple see with

3

Thanks. Yes, am familiar with enhanced-h264ify. Firefox on alarm/manjaro-repo does not support v4l2-request out of the box on Opi5-Plus as such no vpu hardware acceleration.

RK3588 can stream 1080p/60 or even 4K/30 youtube with software decoder on Firefox or Chromium.

4

On that page it also uses yt-dlp -F to get available formats so my internet/pi5 will not play that resolution here but will play a lower resolution. Try this:

gst-launch-1.0 souphttpsrc is-live=true location="$(yt-dlp --format "299" --get-url https://www.youtube.com/watch?v=7PIji8OubXU)" ! qtdemux name=demuxer  demuxer. ! queue ! avdec_h264 ! autovideosink  demuxer.audio_0 ! queue ! avdec_aac ! audioconvert ! audioresample ! autoaudiosink
5

Hi @Darksky, Thanks.
With the above command, gstreamer open up a blank full screen but not streaming any video.

gst-launch-1.0 souphttpsrc is-live=true location="$(yt-dlp --format "299" --get-url https://www.youtube.com/watch?v=7PIji8OubXU)" ! qtdemux name=demuxer  demuxer. ! queue ! avdec_h264 ! autovideosink  demuxer.audio_0 ! queue ! avdec_aac ! audioconvert ! audioresample ! autoaudiosink
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Got context from element 'souphttpsrc0': gst.soup.session=context, session=(GstSoupSession)NULL;
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
WARNING: from element /GstPipeline:pipeline0/GstQTDemux:demuxer: Delayed linking failed.
Additional debug info:
subprojects/gstreamer/gst/parse/grammar.y(918): gst_parse_no_more_pads (): /GstPipeline:pipeline0/GstQTDemux:demuxer:
failed delayed linking pad  audio_0 of GstQTDemux named demuxer to some pad of GstQueue named queue1
Redistribute latency...
Redistribute latency...
Redistribute latency...
0:00:00.0 / 0:12:16.0 (0.0 %)

What is --format “299”?

Edit: ok, yt-dlp -F to list of available format.

6

Same here but if you look real hard it tries. It has some letters in the center of the screen. My internet is to slow and my pi5 will not hw decode hdr mp4. Try substuting 299 with 18 then you will see the letters I am talking about at the first of the video.

18      mp4   640x360     30      2 │ ≈ 40.63MiB   463k https │ avc1.42001E          mp4a.40.2       44k 360p

After further testing it only plays mp4a in the formats.

yt-dlp -F https://www.youtube.com/watch?v=7PIji8OubXU
7

Tried a few. “18” is the only that played with video and sound. The rest 299, 312, 134, 135, 136 all does not stream just a blank window.

8

That is the reason you were getting 360p in your original command. You need to move on and find another stream to test. Look for mp4a. Not mp4_dash.

9

No wonder, all the youtube videos that I tried all default to 360p with gstreamer. But the same youtube videos, mpv can stream them in 1080p or higher.

10

You seem convinced that gstreamer would have a material acceleration. Is this really the case with a local file containing an accelerated codec? Is it really superior to mpv? Personally, I doubt that gstreamer has integrated this compared to the mpv/ffmepg reference, which doesn’t have this as standard but requires a special version of ffmpeg, as is the case with raspberry. Because it’s much easier to configure mpv.

11

I did successfully played a local 1080p/60 video “big buck bunny…” with gstreamer and compared the cpu resources usage eyeballing HTOP as the gstreamer doesn’t seems to have indication whether it is using hardware or software decode.

gst-launch-1.0 playbin uri=file:/run/media/alarm/writable/home/jfl/Videos/bbb_sunflower_1080p_60fps_normal.mp4
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
Redistribute latency...
Redistribute latency...
Redistribute latency...
Redistribute latency...
Redistribute latency...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
Redistribute latency...
New clock: GstPulseSinkClock
Redistribute latency...0.0 %)
Redistribute latency...0.0 %)
Redistribute latency...0.1 %)
Redistribute latency...0.6 %)
Redistribute latency...0.9 %)
Redistribute latency...1.9 %)
^Chandling interrupt. (6.8 %)
Interrupt: Stopping pipeline ...
Execution ended after 0:00:43.371675369
Setting pipeline to NULL ...
Freeing pipeline ...

gstreamer does use consistently lower CPU resources cormpared to mpv.
Unfortunately gstreamer would not play a 4K/60 video.

gst-launch-1.0 playbin uri=file:/home/alarm/Downloads/bbb_sunflower_2160p_60fps_normal.mp4
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
Redistribute latency...
Redistribute latency...
ERROR: from element /GstPlayBin:playbin0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/v4l2slh264dec:v4l2slh264dec0: Failed to configure H264 decoder
Additional debug info:
../gstreamer/subprojects/gst-plugins-bad/sys/v4l2codecs/gstv4l2codech264dec.c(346): gst_v4l2_codec_h264_dec_negotiate (): /GstPlayBin:playbin0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/v4l2slh264dec:v4l2slh264dec0:
gst_v4l2_decoder_set_sink_fmt() failed: Inappropriate ioctl for device
ERROR: pipeline doesn't want to preroll.
Setting pipeline to NULL ...
Freeing pipeline ...

mpv '/run/media/alarm/writable/home/jfl/Videos/bbb_sunflower_1080p_60fps_normal.mp4' 
Option vf-add: 'scale_rkrga' isn't supported.
Error parsing option vf-add (option parameter could not be parsed)
/etc/mpv/mpv.conf:83: setting option vf-add='scale_rkrga=force_yuv=auto' failed.
● Video  --vid=1  (h264 1920x1080 60 fps) [default]
● Audio  --aid=1  (mp3 2ch 48000 Hz 160 kbps) [default]
○ Audio  --aid=2  (ac3 6ch 48000 Hz 320 kbps) [default]
File tags:
 Artist: Blender Foundation 2008, Janus Bager Kristensen 2013
 Comment: Creative Commons Attribution 3.0 - http://bbb3d.renderfarming.net
 Composer: Sacha Goedegebure
 Genre: Animation
 Title: Big Buck Bunny, Sunflower version
[ffmpeg] AVHWDeviceContext: Instance creation failure: VK_ERROR_INCOMPATIBLE_DRIVER
Failed to open VDPAU backend libvdpau_nvidia.so: cannot open shared object file: No such file or directory
[ffmpeg/video] h264_v4l2m2m: Could not find a valid device
[ffmpeg/video] h264_v4l2m2m: can't configure decoder
Could not open codec.
AO: [pipewire] 48000Hz stereo 2ch floatp
VO: [gpu] 1920x1080 yuv420p
AV: 00:01:22 / 00:10:34 (13%) A-V:  0.000 Dropped: 5
Exiting... (Quit)
12

I have used rockchip_vdec2 in Chromium with video acceleration for youtube. To do that disabled hantro_vpu and forced H264 with enhanced-h264ify extension.

13

Hi @SickOS,
What version of Chromium are you using/on? chromium 131.0.6778.264-1? Which kernel are you on? How to disable hantro_vpu?

14

No hardware acceleration for gstreamer.

Do you have rockchip-mpp (AUR) installed ? MPP: Media Process Platform is the video codec parser and Hardware Abstraction Layer library for the Rockchip platforms. It’s mandatory for hardware and gstreamer.

Edit : You’ll probably have to recompile gstreamer and look at the configure options if it doesn’t automatically recognize mpp.

For mpv ,lot of error, make a new config with just

autofit-smaller=90%x90%
autofit-larger=90%x90%
volume-max=400    # 0&9 keys controls the volume
profile=fast
save-position-on-quit
hwdec=auto-safe

And if you want acceleration with it, compil ffmpeg-rockchip.
mpv is more easy with yt-dlp and possible browser acceleration (firefox use local ffmpeg). After it will be time to see yt-dlp.

15

Yes this is true, I do this with bsp-vendor’s kernel like 6.1.75-joshua-git or 6.1.99-armbian-git and use ffmpeg-mpp-git and mpp-git with mpv-0.38.0-y. mpv-0.39.0 doesn’t seems to work with ffmpeg-mpp-git.

Currently testing mainline kernel-linux-rc-6.14-rc7-1 which do have vpu hardware acceleration support but need other packages/applications to make it work.

Hi @tartanpion,

Are referring to when gstreamer playing 4K/60 video or 1080/60 video? Per my understanding gstreamer does not have hardware decode for 4K/60 video but might have hardware decode with 1080/60 mp4 video.

Do you have rockchip-mpp (AUR) installed ? MPP: Media Process Platform is the video codec parser and Hardware Abstraction Layer library for the Rockchip platforms. It’s mandatory for hardware and gstreamer.

I have ffmpeg-mpp-git which if not mistaken is similar or based on ffmpeg-rockchip which. Based on my understanding the mainline kernel does not have rockchip-mpp support. The kernel linux-rc-6.14-rc7-1 which is based on kernel from MiniArch warpme does not have patch to work with mpp but does have patches to work with v4l2-request.

As stated above, I do use ffmpeg-mpp-git, mpp-git with the bsp-vendors kernels and have vpu hardware acceleration with chromium-mpp and mpv.

Have tested armbian kernel-6.14-rc4 on Debian Trixie where upstream chromium v132 and above do have vpu hw acceleration (need --ozone-platform=wayland) but unfortunately these upstream chromium v133 and v134 are not stable with --onzone-platform=wayland on RK3588 devices. As such as of now, per my understanding need a revert a patch in upstream chromium to make it stable which is available for Debian/Ubuntu from https://launchpad.net/~liujianfeng1994/+archive/ubuntu/chromium. Have tested it and chromium-v133 and v134 does support vpu hw acceleration with h264 youtube (test with 1080p videos) but currently armbian-kernel-6.14-rc4-1 does not support hdmi-audio on Opi5-Plus yet so is no fun streaming no sound video.

Firefox out of the box does not work with ffmpeg-mpp-git to enable vpu hw acceleration per my understanding. There is firefox-mpp available but need to compile it which is beyond my skill set.

Have tested latest Vivaldi from Manjaro-Repo which is based on Chromium_v134 with linux-rc-6.14-rc6/7 but unfortunately no vpu hw accelaration on Opi5-Plus.

Edit: Just noticed, Firefox-MPP-136.0.2 is available for download from which works with bsp-vendor’s kernel.

https://github.com/hbiyik/agrrepo/releases/download/alarm-aarch64/firefox-mpp-136.0.2-1-aarch64.pkg.tar.xz