FFmpeg transcoding /w VA-API on AMD hardware

Support AUR
,
1

Hi folks @ the Manjaro Forum,

I spent more than a few days trying to get FFmpeg to transcode various videos from my library to HEVC using VA-API on stock AMD RX 5700 XT. I do not want to use QSV, due to quality/filesize discrepancies.

The problem I’m facing is segfaults with FFmpeg with VA-API. I have exhausted my various search engines options for the time being, I have gone through official documentations, experimentation, user input, software versions, etc.

Below you can find information about my setup, please - see if you can find something wrong or suspicious and let me know how I could approach a possible solution. Thanks!

❯ inxi -v3
System:    Host: manjaro Kernel: 5.13.8-188-tkg-pds x86_64 bits: 64 compiler: gcc v: 11.1.0 
           Desktop: Xfce 4.16.0 Distro: Manjaro Linux base: Arch Linux 
Machine:   Type: Desktop System: Gigabyte product: Z270X-Ultra Gaming v: N/A serial: <superuser required> 
           Mobo: Gigabyte model: Z270X-Ultra Gaming-CF serial: <superuser required> 
           UEFI: American Megatrends v: F9d date: 03/09/2018 
CPU:       Info: Quad Core model: Intel Core i7-6700K bits: 64 type: MT MCP arch: Skylake-S rev: 3 cache: 
           L2: 8 MiB 
           flags: avx avx2 lm nx pae sse sse2 sse3 sse4_1 sse4_2 ssse3 vmx bogomips: 63999 
           Speed: 1000 MHz min/max: 800/4400 MHz Core speeds (MHz): 1: 1000 2: 4358 3: 1712 4: 4380 
           5: 1061 6: 1223 7: 1621 8: 1344 
Graphics:  Device-1: AMD Navi 10 [Radeon RX 5600 OEM/5600 XT / 5700/5700 XT] driver: amdgpu v: kernel 
           bus-ID: 03:00.0 
           Display: x11 server: X.Org 1.20.11 driver: loaded: amdgpu,ati unloaded: modesetting,radeon 
           resolution: 1: 1920x1080 2: 1280x1024~75Hz 
           OpenGL: renderer: AMD Radeon RX 5700 XT (NAVI10 DRM 3.41.0 5.13.8-188-tkg-pds LLVM 12.0.1) 
           v: 4.6 Mesa 21.3.0-devel (git-518b94e8e1) direct render: Yes 
Network:   Device-1: Intel Ethernet I219-V driver: e1000e v: kernel port: f000 bus-ID: 00:1f.6 
           IF: enp0s31f6 state: up speed: 1000 Mbps duplex: full mac: 1c:1b:0d:98:ca:f4 
Drives:    Local Storage: total: 4.53 TiB used: 1.76 TiB (38.8%) 
Info:      Processes: 292 Uptime: 42m Memory: 15.59 GiB used: 4.98 GiB (32.0%) Init: systemd Compilers: 
           gcc: 11.1.0 clang: 12.0.1 Packages: 1889 Shell: Zsh v: 5.8 inxi: 3.3.06 


❯ pacman -Q|grep -E 'amd|vdp|mesa|cuda'|sort -h
ffmpeg-amd-full 4.4-3
lib32-libvdpau 1.4-1
lib32-mesa-demos 8.4.0-2
lib32-mesa-tkg-git 21.3.0_devel.143295.518b94e8e1a-1
libva-vdpau-driver 0.7.4-5
libvdpau 1.4-1
mesa-demos 8.4.0-4
mesa-tkg-git 21.3.0_devel.143295.518b94e8e1a-1
mhwd-amdgpu 19.1.0-1
mpv-amd-full-git 0.33.1.r270.g24357cb7b5-1
vdpauinfo 1.4-1
xf86-video-amdgpu 19.1.0-2


❯ vainfo
vainfo: VA-API version: 1.12 (libva 2.13.0.1)
vainfo: Driver version: Mesa Gallium driver 21.3.0-devel for AMD Radeon RX 5700 XT (NAVI10, DRM 3.41.0, 5.13.8-188-tkg-pds, LLVM 12.0.1)
vainfo: Supported profile and entrypoints
      VAProfileMPEG2Simple            :	VAEntrypointVLD
      VAProfileMPEG2Main              :	VAEntrypointVLD
      VAProfileVC1Simple              :	VAEntrypointVLD
      VAProfileVC1Main                :	VAEntrypointVLD
      VAProfileVC1Advanced            :	VAEntrypointVLD
      VAProfileH264ConstrainedBaseline:	VAEntrypointVLD
      VAProfileH264ConstrainedBaseline:	VAEntrypointEncSlice
      VAProfileH264Main               :	VAEntrypointVLD
      VAProfileH264Main               :	VAEntrypointEncSlice
      VAProfileH264High               :	VAEntrypointVLD
      VAProfileH264High               :	VAEntrypointEncSlice
      VAProfileHEVCMain               :	VAEntrypointVLD
      VAProfileHEVCMain               :	VAEntrypointEncSlice
      VAProfileHEVCMain10             :	VAEntrypointVLD
      VAProfileHEVCMain10             :	VAEntrypointEncSlice
      VAProfileJPEGBaseline           :	VAEntrypointVLD
      VAProfileVP9Profile0            :	VAEntrypointVLD
      VAProfileVP9Profile2            :	VAEntrypointVLD
      VAProfileNone                   :	VAEntrypointVideoProc

❯ vdpauinfo
display: :0.0   screen: 0
API version: 1
Information string: G3DVL VDPAU Driver Shared Library version 1.0

Video surface:

name   width height types
-------------------------------------------
420    16384 16384  NV12 YV12 
422    16384 16384  UYVY YUYV 
444    16384 16384  Y8U8V8A8 V8U8Y8A8 
420_16 16384 16384  
422_16 16384 16384  
444_16 16384 16384  

Decoder capabilities:

name                        level macbs width height
----------------------------------------------------
MPEG1                          --- not supported ---
MPEG2_SIMPLE                    3 65536  4096  4096
MPEG2_MAIN                      3 65536  4096  4096
H264_BASELINE                  52 65536  4096  4096
H264_MAIN                      52 65536  4096  4096
H264_HIGH                      52 65536  4096  4096
VC1_SIMPLE                      1 65536  4096  4096
VC1_MAIN                        2 65536  4096  4096
VC1_ADVANCED                    4 65536  4096  4096
MPEG4_PART2_SP                  3 65536  4096  4096
MPEG4_PART2_ASP                 5 65536  4096  4096
DIVX4_QMOBILE                  --- not supported ---
DIVX4_MOBILE                   --- not supported ---
DIVX4_HOME_THEATER             --- not supported ---
DIVX4_HD_1080P                 --- not supported ---
DIVX5_QMOBILE                  --- not supported ---
DIVX5_MOBILE                   --- not supported ---
DIVX5_HOME_THEATER             --- not supported ---
DIVX5_HD_1080P                 --- not supported ---
H264_CONSTRAINED_BASELINE       0 65536  4096  4096
H264_EXTENDED                  --- not supported ---
H264_PROGRESSIVE_HIGH          --- not supported ---
H264_CONSTRAINED_HIGH          --- not supported ---
H264_HIGH_444_PREDICTIVE       --- not supported ---
VP9_PROFILE_0                  --- not supported ---
VP9_PROFILE_1                  --- not supported ---
VP9_PROFILE_2                  --- not supported ---
VP9_PROFILE_3                  --- not supported ---
HEVC_MAIN                      186 139264  8192  4352
HEVC_MAIN_10                   186 139264  8192  4352
HEVC_MAIN_STILL                --- not supported ---
HEVC_MAIN_12                   --- not supported ---
HEVC_MAIN_444                  --- not supported ---
HEVC_MAIN_444_10               --- not supported ---
HEVC_MAIN_444_12               --- not supported ---

Output surface:

name              width height nat types
----------------------------------------------------
B8G8R8A8         16384 16384    y  NV12 YV12 UYVY YUYV Y8U8V8A8 V8U8Y8A8 P010 P016 A8I8 I8A8 
R8G8B8A8         16384 16384    y  NV12 YV12 UYVY YUYV Y8U8V8A8 V8U8Y8A8 P010 P016 A8I8 I8A8 
R10G10B10A2      16384 16384    y  NV12 YV12 UYVY YUYV Y8U8V8A8 V8U8Y8A8 P010 P016 A8I8 I8A8 
B10G10R10A2      16384 16384    y  NV12 YV12 UYVY YUYV Y8U8V8A8 V8U8Y8A8 P010 P016 A8I8 I8A8 

Bitmap surface:

name              width height
------------------------------
B8G8R8A8         16384 16384
R8G8B8A8         16384 16384
R10G10B10A2      16384 16384
B10G10R10A2      16384 16384
A8               16384 16384

Video mixer:

feature name                    sup
------------------------------------
DEINTERLACE_TEMPORAL             y
DEINTERLACE_TEMPORAL_SPATIAL     -
INVERSE_TELECINE                 -
NOISE_REDUCTION                  y
SHARPNESS                        y
LUMA_KEY                         y
HIGH QUALITY SCALING - L1        y
HIGH QUALITY SCALING - L2        -
HIGH QUALITY SCALING - L3        -
HIGH QUALITY SCALING - L4        -
HIGH QUALITY SCALING - L5        -
HIGH QUALITY SCALING - L6        -
HIGH QUALITY SCALING - L7        -
HIGH QUALITY SCALING - L8        -
HIGH QUALITY SCALING - L9        -

parameter name                  sup      min      max
-----------------------------------------------------
VIDEO_SURFACE_WIDTH              y        48     4096
VIDEO_SURFACE_HEIGHT             y        48     4096
CHROMA_TYPE                      y  
LAYERS                           y         0        4

attribute name                  sup      min      max
-----------------------------------------------------
BACKGROUND_COLOR                 y  
CSC_MATRIX                       y  
NOISE_REDUCTION_LEVEL            y      0.00     1.00
SHARPNESS_LEVEL                  y     -1.00     1.00
LUMA_KEY_MIN_LUMA                y  
LUMA_KEY_MAX_LUMA                y  


ffmpeg -vaapi_device /dev/dri/renderD128 -i input.mkv -vf 'format=nv12,hwupload' -c:v hevc_vaapi -f mp4 -rc_mode 1 -qp 25 output.mp4
......
Input #0, matroska,webm, from 'input.mkv':
  Metadata:
    encoder         : libebml v1.3.1 + libmatroska v1.4.2
    creation_time   : 2016-09-13T15:30:02.000000Z
  Duration: 00:21:08.94, start: 0.000000, bitrate: 15033 kb/s
  Stream #0:0(eng): Audio: eac3, 48000 Hz, 5.1(side), fltp, 640 kb/s (default)
    Metadata:
      BPS             : 640000
      BPS-eng         : 640000
      DURATION        : 00:21:08.896000000
      DURATION-eng    : 00:21:08.896000000
      NUMBER_OF_FRAMES: 39653
      NUMBER_OF_FRAMES-eng: 39653
      NUMBER_OF_BYTES : 101511680
      NUMBER_OF_BYTES-eng: 101511680
      _STATISTICS_WRITING_APP: mkvmerge v8.3.0 ('Over the Horizon') 64bit
      _STATISTICS_WRITING_APP-eng: mkvmerge v8.3.0 ('Over the Horizon') 64bit
      _STATISTICS_WRITING_DATE_UTC: 2016-09-13 15:30:02
      _STATISTICS_WRITING_DATE_UTC-eng: 2016-09-13 15:30:02
      _STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
      _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
  Stream #0:1: Video: h264 (High), yuv420p(tv, bt709, progressive), 1920x1080 [SAR 1:1 DAR 16:9], Closed Captions, 23.98 fps, 23.98 tbr, 1k tbn, 47.95 tbc (default)
    Metadata:
      BPS             : 14390752
      BPS-eng         : 14390752
      DURATION        : 00:21:08.935000000
      DURATION-eng    : 00:21:08.935000000
      NUMBER_OF_FRAMES: 30424
      NUMBER_OF_FRAMES-eng: 30424
      NUMBER_OF_BYTES : 2282616143
      NUMBER_OF_BYTES-eng: 2282616143
      _STATISTICS_WRITING_APP: mkvmerge v8.3.0 ('Over the Horizon') 64bit
      _STATISTICS_WRITING_APP-eng: mkvmerge v8.3.0 ('Over the Horizon') 64bit
      _STATISTICS_WRITING_DATE_UTC: 2016-09-13 15:30:02
      _STATISTICS_WRITING_DATE_UTC-eng: 2016-09-13 15:30:02
      _STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
      _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
  Stream #0:2: Subtitle: subrip (default)
    Metadata:
      BPS             : 96
      BPS-eng         : 96
      DURATION        : 00:20:59.826000000
      DURATION-eng    : 00:20:59.826000000
      NUMBER_OF_FRAMES: 487
      NUMBER_OF_FRAMES-eng: 487
      NUMBER_OF_BYTES : 15181
      NUMBER_OF_BYTES-eng: 15181
      _STATISTICS_WRITING_APP: mkvmerge v8.3.0 ('Over the Horizon') 64bit
      _STATISTICS_WRITING_APP-eng: mkvmerge v8.3.0 ('Over the Horizon') 64bit
      _STATISTICS_WRITING_DATE_UTC: 2016-09-13 15:30:02
      _STATISTICS_WRITING_DATE_UTC-eng: 2016-09-13 15:30:02
      _STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
      _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
Stream mapping:
  Stream #0:1 -> #0:0 (h264 (native) -> hevc (hevc_vaapi))
  Stream #0:0 -> #0:1 (eac3 (native) -> aac (native))
Press [q] to stop, [?] for help
[aac @ 0x560ed176a040] Using a PCE to encode channel layout "5.1(side)"
[hevc_vaapi @ 0x560ed1754280] Driver does not support some wanted packed headers (wanted 0xd, found 0x1).
Output #0, mp4, to 'output.mp4':
  Metadata:
    encoder         : Lavf58.76.100
  Stream #0:0: Video: hevc (Main) (hev1 / 0x31766568), vaapi_vld(tv, bt709, progressive), 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 23.98 fps, 24k tbn (default)
    Metadata:
      BPS             : 14390752
      BPS-eng         : 14390752
      DURATION        : 00:21:08.935000000
      DURATION-eng    : 00:21:08.935000000
      NUMBER_OF_FRAMES: 30424
      NUMBER_OF_FRAMES-eng: 30424
      NUMBER_OF_BYTES : 2282616143
      NUMBER_OF_BYTES-eng: 2282616143
      _STATISTICS_WRITING_APP: mkvmerge v8.3.0 ('Over the Horizon') 64bit
      _STATISTICS_WRITING_APP-eng: mkvmerge v8.3.0 ('Over the Horizon') 64bit
      _STATISTICS_WRITING_DATE_UTC: 2016-09-13 15:30:02
      _STATISTICS_WRITING_DATE_UTC-eng: 2016-09-13 15:30:02
      _STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
      _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
      encoder         : Lavc58.134.100 hevc_vaapi
  Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, 5.1(side), fltp, 394 kb/s (default)
    Metadata:
      BPS             : 640000
      BPS-eng         : 640000
      DURATION        : 00:21:08.896000000
      DURATION-eng    : 00:21:08.896000000
      NUMBER_OF_FRAMES: 39653
      NUMBER_OF_FRAMES-eng: 39653
      NUMBER_OF_BYTES : 101511680
      NUMBER_OF_BYTES-eng: 101511680
      _STATISTICS_WRITING_APP: mkvmerge v8.3.0 ('Over the Horizon') 64bit
      _STATISTICS_WRITING_APP-eng: mkvmerge v8.3.0 ('Over the Horizon') 64bit
      _STATISTICS_WRITING_DATE_UTC: 2016-09-13 15:30:02
      _STATISTICS_WRITING_DATE_UTC-eng: 2016-09-13 15:30:02
      _STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
      _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
      encoder         : Lavc58.134.100 aac
zsh: segmentation fault (core dumped)  ffmpeg -vaapi_device /dev/dri/renderD128 -i  -vf 'format=nv12,hwupload' -c:v 

❯ sudo dmesg | grep -E 'radeon|ffmpeg'
[  205.036533] ffmpeg[3819]: segfault at 7fd0847fb000 ip 00007fd0cdf42cc0 sp 00007ffe033b5628 error 6 in radeonsi_drv_video.so[7fd0cdced000+624000]
[  205.036586] audit: type=1701 audit(1628155681.273:139): auid=1000 uid=1000 gid=1000 ses=2 subj==unconfined pid=3819 comm="ffmpeg" exe="/usr/bin/ffmpeg" sig=11 res=1
[ 3198.674832] ffmpeg[22742]: segfault at 7fe5b0201000 ip 00007fe60eb98cc0 sp 00007ffe0c11ab38 error 6 in radeonsi_drv_video.so[7fe60e943000+624000]
[ 3198.674860] audit: type=1701 audit(1628158674.950:150): auid=1000 uid=1000 gid=1000 ses=2 subj==unconfined pid=22742 comm="ffmpeg" exe="/usr/bin/ffmpeg" sig=11 res=1

Thanks for reading!

2

Please read this, especially the 'Upstream" and “Donwstream” business and then file a bug upstream…

:crossed_fingers:

1 Like
3

This works for me on ffmpeg version n4.4 and Radeon RX 590:

ffmpeg -i input.mkv -vaapi_device /dev/dri/renderD128 -vf format=nv12,hwupload -c:v hevc_vaapi -map 0:v:0  -map 0:a? -c:a copy -map 0:s? -c:s copy -rc_mode 1 -qp 25 -map_chapters 0  output.mp4
1 Like
4

Does not work for me, unfortunately:

[hevc_vaapi @ 0x55e603611700] Driver does not support some wanted packed headers (wanted 0xd, found 0x1).
[mp4 @ 0x55e60365ea80] track 1: codec frame size is not set
[mp4 @ 0x55e60365ea80] Could not find tag for codec subrip in stream #2, codec not currently supported in container
Could not write header for output file #0 (incorrect codec parameters ?): Invalid argument
Error initializing output stream 0:0 --
5

You used the command not correctly. Some options depend on the source file. You need to adapt it to your file.

Maybe start why a sample file that can everyone download. For this example you should download sample_1280x720_surfing_with_audio.mkv

This command worked for my system with this (sample_1280x720_surfing_with_audio.mkv) file.

ffmpeg -i sample_1280x720_surfing_with_audio.mkv -vaapi_device /dev/dri/renderD128 -vf format=nv12,hwupload -c:v hevc_vaapi -qp 25 -c:a copy output.mp4

I tried to remove all not essential options. You will probably see these warnings

[hevc_vaapi @ 0x563b72ac6f80] Driver does not support some wanted packed headers (wanted 0xd, found 0x1).
[mp4 @ 0x563b72adc9c0] track 1: codec frame size is not set

You also might want to directly try

ffmpeg -hwaccel vaapi -hwaccel_output_format vaapi -hwaccel_device /dev/dri/renderD128 -i sample_1280x720_surfing_with_audio.mkv -c:v hevc_vaapi -qp 25 -c:a copy output.mp4
1 Like
6

Thanks for the input @xabbu, both examples do not work on my machine, both examples fail with:

[hevc_vaapi @ 0x56267c021280] Driver does not support some wanted packed headers (wanted 0xd, found 0x1).
[mp4 @ 0x56267c022500] track 1: codec frame size is not set
........
zsh: segmentation fault (core dumped)  ffmpeg -i sample_1280x720_surfing_with_audio.mkv -vaapi_device  -vf  -c:v  -q

6 months ago or so I had similar issues, but somehow managed to get FFmpeg with VAAPI working with some very basic options, leading to some really bad transcodes. I was hoping that in time things would improve, considering the rapid development of MESA and FFmpeg, alas this is not the case.

Fun fact: obs-hevc-vaapi-git works without a hitch… Go figure.

7

Can you try the normal default ffmpeg package. Not any of these weird patched packages form the AUR.

Why?

It looks like you use a lot of not supported packages. You should switch back to only supported repository packages. Reboot and try again.

8

Tried them already, including various combinations between software - mesa, ffmpeg, icd loaders, non-free amd drivers, manjaro kernels, custom kernels, etc. I even removed my second GPU for the time being to make sure there’s no mesa mix ups with multiple devices. The only thing i have not tried yet is booting a clean distro USB, which is actually a good idea - i’ll give that a go later today.

9

Solution: Had to downgrade practically every package related to the GPU to the official packages - mesa, vulkan-radeon, vulkan-icd-loader, etc and still had issues, until the reinstall of the libva-mesa-driver package - this was a step I had missed before with the same GPU driver stack.

Downgrading mesa is a bit of pain if you don’t read the pacman/pamac output and not ideal, since every now and then mesa-git may be needed for running some newer games. However currently FFmpeg with VAAPI transcoding works as expected and Cyberpunk 2077 runs fine.

ffmpeg 2:4.4-4
ffmpegthumbnailer 2.2.2-2
lib32-libva 2.11.0-1
lib32-libva-intel-driver 2.4.1-1
lib32-mesa 21.1.6-1
lib32-mesa-demos 8.4.0-2
lib32-vulkan-icd-loader 1.2.184-1
lib32-vulkan-radeon 21.1.6-1
libva 2.12.0-1
libva-intel-driver 2.4.1-1
libva-mesa-driver 21.1.6-1
libva-utils 2.12.0-1
libva-vdpau-driver 0.7.4-5
mesa 21.1.6-1
mesa-demos 8.4.0-4
mhwd-amdgpu 19.1.0-1
opera-ffmpeg-codecs 92.0.4515.107-1
vulkan-headers 1:1.2.185-1
vulkan-icd-loader 1.2.185-1
vulkan-radeon 21.1.6-1
vulkan-tools 1.2.184-1
xf86-video-amdgpu 21.0.0-1
10

This topic was automatically closed 15 days after the last reply. New replies are no longer allowed.