VFIO GPU passthrough not working anymore

Support Virtualization
1

Been pulling my hair out for an hour or 2 each day trying to get this working again. I’m trying to get GPU passthrough working again on Manjaro with 2 identical 1080TI’s. I had this working previously but something borked my Manjaro install so I had to reinstall from scratch, now I can’t seem to get it going again. I haven’t made any hardware changes in between so I know UEFI is setup correctly.

Following the steps on the Arch Wiki, but jumping from step 3 to 6.1 because of the identical GPUs. IOMMU is working but the 2nd GPU to be passed through is not using the vfio-pci driver which it would be if it were being properly passed through. Not really sure what I’m missing.

$ lspci -nnk
0e:00.0 VGA compatible controller [0300]: NVIDIA Corporation GP102 [GeForce GTX 1080 Ti] [10de:1b06] (rev a1)
Subsystem: Micro-Star International Co., Ltd. [MSI] Device [1462:3604]
Kernel driver in use: nvidia
Kernel modules: nouveau, nvidia_drm, nvidia
0e:00.1 Audio device [0403]: NVIDIA Corporation GP102 HDMI Audio Controller [10de:10ef] (rev a1)
Subsystem: Micro-Star International Co., Ltd. [MSI] Device [1462:3604]
Kernel driver in use: snd_hda_intel
Kernel modules: snd_hda_intel
0f:00.0 VGA compatible controller [0300]: NVIDIA Corporation GP102 [GeForce GTX 1080 Ti] [10de:1b06] (rev a1)
Subsystem: Micro-Star International Co., Ltd. [MSI] Device [1462:3604]
Kernel driver in use: nvidia
Kernel modules: nouveau, nvidia_drm, nvidia
0f:00.1 Audio device [0403]: NVIDIA Corporation GP102 HDMI Audio Controller [10de:10ef] (rev a1)
Subsystem: Micro-Star International Co., Ltd. [MSI] Device [1462:3604]
Kernel driver in use: snd_hda_intel
Kernel modules: snd_hda_intel

% sudo dmesg | grep -i vfio            
[    2.026812] VFIO - User Level meta-driver version: 0.3
2

What changes did you make in order to get vfio-pci to bind to the GPU?

3

Pretty much exactly this as per the Arch wiki (no links in posts?)
##### Passthrough selected GPU

In this case we manually specify the GPU to bind.

#!/bin/sh DEVS="0000:03:00.0 0000:03:00.1" if [ ! -z "$(ls -A /sys/class/iommu)" ]; then for DEV in $DEVS; do echo "vfio-pci" > /sys/bus/pci/devices/$DEV/driver_override done fi modprobe -i vfio-pci

#### Script installation

Edit `/etc/mkinitcpio.conf`:

1. Add `modconf` to the [HOOKS](https://wiki.archlinux.org/index.php/Mkinitcpio#HOOKS) array and `/usr/local/bin/vfio-pci-override.sh` to the [FILES](https://wiki.archlinux.org/index.php/Mkinitcpio#BINARIES_and_FILES) array.

Edit `/etc/modprobe.d/vfio.conf`:

1. Add the following line: `install vfio-pci /usr/local/bin/vfio-pci-override.sh`
2. [Regenerate the initramfs](https://wiki.archlinux.org/index.php/Regenerate_the_initramfs) and reboot.

This isn’t exactly what I did the first time. I can’t find the tutorial I originally used. I don’t recall using the script

4

were you able to fix this? same thing is happening to me.

5

Hello,

please try to replace in your grub config or vfio.conf respectively devices ID from e.g. 10de:1f06 format to 0000:0e:00.0 and 0000:0e:00.1 where 0e:00.0 and 0e:00.1 stands for device group from lspci -nnk command. Regenerate the initramfs using command sudo mkinitcpio -p linuxXXX where XXX stands for kernel version (e.g. for 5.10 kernel it should be sudo mkinitcpio -p linux510) then reboot the system. Check if GPU uses vfio-pci driver.

6

Hey! I was at the same spot.
Assuming you have done everything else properly, I think I know the cause for this.
In /etc/modules-load.d/mhwd-gpu.conf you have to comment the lines “nvidia” and “nvidia-drm” (had to figure thiis out myself)

and then you have to blacklist nvidia drivers in /etc/modprob.d/mhwd-gpu.conf
Here, check this guide- www.tauceti.blog/post/linux-amd-x570-nvidia-gpu-pci-passthrough-2-prepare-linux/