Iommu not enabling

Support Virtualization
1

I am trying to set up a kvm with gpu passthrough to run games and other stuff that wont run on linux. To do that I need to enable iommu, but I am having trouble with it.

I believe my hardware supports it and I have enabled VT-d

I have an i7-7700k, GTX 1080, GT 1030, and an MSI B250 mb

I configure /etc/default/grub to enable iommu

GRUB_CMDLINE_LINUX_DEFAULT="intel_iommu=on iommu=pt quiet apparmor=1 security=apparmor udev.log_priority=3"

run grub-mkconfig

sudo grub-mkconfig -o /boot/grub/grub.fg 
                                                                                                                                                                             
Generating grub configuration file ...
Found theme: /usr/share/grub/themes/manjaro/theme.txt
Found linux image: /boot/vmlinuz-5.13-x86_64
Found initrd image: /boot/intel-ucode.img /boot/initramfs-5.13-x86_64.img
Found initrd fallback image: /boot/initramfs-5.13-x86_64-fallback.img
Warning: os-prober will be executed to detect other bootable partitions.
Its output will be used to detect bootable binaries on them and create new boot entries.
Adding boot menu entry for UEFI Firmware Settings ...
Found memtest86+ image: /boot/memtest86+/memtest.bin
/usr/bin/grub-probe: warning: unknown device type nvme0n1.

reboot
then run dmesg

sudo dmesg | grep -i -e DMAR -e IOMMU 
                                                                                                                                                                   
[    0.006127] ACPI: DMAR 0x000000007ECD0748 000070 (v01 INTEL  KBL      00000001 INTL 00000001)
[    0.006144] ACPI: Reserving DMAR table memory at [mem 0x7ecd0748-0x7ecd07b7]
[    0.137302] DMAR: Host address width 39
[    0.137303] DMAR: DRHD base: 0x000000fed90000 flags: 0x1
[    0.137306] DMAR: dmar0: reg_base_addr fed90000 ver 1:0 cap d2008c40660462 ecap f050da
[    0.137308] DMAR: RMRR base: 0x0000007e79b000 end: 0x0000007e7bafff
[    0.137309] DMAR-IR: IOAPIC id 2 under DRHD base  0xfed90000 IOMMU 0
[    0.137310] DMAR-IR: HPET id 0 under DRHD base 0xfed90000
[    0.137311] DMAR-IR: Queued invalidation will be enabled to support x2apic and Intr-remapping.
[    0.138626] DMAR-IR: Enabled IRQ remapping in x2apic mode
[    0.257938] iommu: Default domain type: Translated 
[    0.321099] AMD-Vi: AMD IOMMUv2 driver by Joerg Roedel <jroedel@suse.de>
[    0.321099] AMD-Vi: AMD IOMMUv2 functionality not available on this system

it doesn’t say that iommu is enabled
then I check my iommu groups and nothing is there

2

Can’t you toggle the BIOS/UEFI setting to enable IOMMU?

3

I did not see a setting anywhere in my bios labeled IOMMU. And of what I have read there is no iommu setting, just a VT-d and VT-x setting which I have both enabled.

4

Give proper information about your hardware.

inxi -Fazy

5

sure

inxi -Fazy                                                                                                                                                                                                    

System:
  Kernel: 5.13.19-2-MANJARO x86_64 bits: 64 compiler: gcc v: 11.1.0
  parameters: BOOT_IMAGE=/boot/vmlinuz-5.13-x86_64
  root=UUID=ee73ff49-9079-411e-8ce6-5314964149cf rw quiet apparmor=1
  security=apparmor udev.log_priority=3
  Desktop: KDE Plasma 5.22.5 tk: Qt 5.15.2 wm: kwin_x11 vt: 1 dm: SDDM
  Distro: Manjaro Linux base: Arch Linux
Machine:
  Type: Desktop Mobo: MSI model: B250 GAMING PRO CARBON (MS-7A64) v: 1.0
  serial: <filter> UEFI: American Megatrends v: 1.00 date: 12/21/2016
CPU:
  Info: Quad Core model: Intel Core i7-7700K bits: 64 type: MT MCP
  arch: Kaby Lake family: 6 model-id: 9E (158) stepping: 9 microcode: EA
  cache: L2: 8 MiB
  flags: avx avx2 lm nx pae sse sse2 sse3 sse4_1 sse4_2 ssse3 vmx
  bogomips: 67224
  Speed: 800 MHz min/max: 800/4500 MHz Core speeds (MHz): 1: 800 2: 800 3: 800
  4: 800 5: 800 6: 800 7: 800 8: 800
  Vulnerabilities: Type: itlb_multihit status: KVM: VMX disabled
  Type: l1tf
  mitigation: PTE Inversion; VMX: conditional cache flushes, SMT vulnerable
  Type: mds mitigation: Clear CPU buffers; SMT vulnerable
  Type: meltdown mitigation: PTI
  Type: spec_store_bypass
  mitigation: Speculative Store Bypass disabled via prctl and seccomp
  Type: spectre_v1
  mitigation: usercopy/swapgs barriers and __user pointer sanitization
  Type: spectre_v2 mitigation: Full generic retpoline, IBPB: conditional,
  IBRS_FW, STIBP: conditional, RSB filling
  Type: srbds mitigation: Microcode
  Type: tsx_async_abort mitigation: Clear CPU buffers; SMT vulnerable
Graphics:
  Device-1: NVIDIA GP104 [GeForce GTX 1080] vendor: ZOTAC driver: nvidia
  v: 470.63.01 alternate: nouveau,nvidia_drm bus-ID: 01:00.0
  chip-ID: 10de:1b80 class-ID: 0300
  Device-2: NVIDIA GP108 [GeForce GT 1030] vendor: Gigabyte driver: nvidia
  v: 470.63.01 alternate: nouveau,nvidia_drm bus-ID: 02:00.0
  chip-ID: 10de:1d01 class-ID: 0300
  Display: x11 server: X.Org 1.20.13 compositor: kwin_x11 driver:
  loaded: nvidia display-ID: :0 screens: 1
  Screen-1: 0 s-res: 5760x2160 s-dpi: 93 s-size: 1573x584mm (61.9x23.0")
  s-diag: 1678mm (66.1")
  Monitor-1: DVI-D-0 res: 1920x1080 hz: 60 dpi: 94
  size: 521x293mm (20.5x11.5") diag: 598mm (23.5")
  Monitor-2: DP-4 res: 3840x2160 hz: 60 dpi: 163 size: 600x340mm (23.6x13.4")
  diag: 690mm (27.2")
  OpenGL: renderer: NVIDIA GeForce GTX 1080/PCIe/SSE2
  v: 4.6.0 NVIDIA 470.63.01 direct render: Yes
Audio:
  Device-1: Intel 200 Series PCH HD Audio vendor: Micro-Star MSI
  driver: snd_hda_intel v: kernel bus-ID: 00:1f.3 chip-ID: 8086:a2f0
  class-ID: 0403
  Device-2: NVIDIA GP104 High Definition Audio vendor: ZOTAC
  driver: snd_hda_intel v: kernel bus-ID: 01:00.1 chip-ID: 10de:10f0
  class-ID: 0403
  Device-3: NVIDIA GP108 High Definition Audio vendor: Gigabyte
  driver: snd_hda_intel v: kernel bus-ID: 02:00.1 chip-ID: 10de:0fb8
  class-ID: 0403
  Sound Server-1: ALSA v: k5.13.19-2-MANJARO running: yes
  Sound Server-2: JACK v: 1.9.19 running: no
  Sound Server-3: PulseAudio v: 15.0 running: yes
  Sound Server-4: PipeWire v: 0.3.38 running: yes
Network:
  Device-1: Intel Ethernet I219-V vendor: Micro-Star MSI driver: e1000e
  v: kernel port: N/A bus-ID: 00:1f.6 chip-ID: 8086:15b8 class-ID: 0200
  IF: enp0s31f6 state: up speed: 1000 Mbps duplex: full mac: <filter>
Drives:
  Local Storage: total: 931.51 GiB used: 12.18 GiB (1.3%)
  SMART Message: Unable to run smartctl. Root privileges required.
  ID-1: /dev/nvme0n1 maj-min: 259:0 vendor: Samsung model: SSD 980 1TB
  size: 931.51 GiB block-size: physical: 512 B logical: 512 B speed: 31.6 Gb/s
  lanes: 4 type: SSD serial: <filter> rev: 1B4QFXO7 temp: 34.9 C scheme: GPT
Partition:
  ID-1: / raw-size: 931.22 GiB size: 915.53 GiB (98.32%)
  used: 12.18 GiB (1.3%) fs: ext4 dev: /dev/nvme0n1p2 maj-min: 259:2
  ID-2: /boot/efi raw-size: 300 MiB size: 299.4 MiB (99.80%)
  used: 288 KiB (0.1%) fs: vfat dev: /dev/nvme0n1p1 maj-min: 259:1
Swap:
  Alert: No swap data was found.
Sensors:
  System Temperatures: cpu: 29.8 C mobo: 27.8 C gpu: nvidia temp: 42 C
  Fan Speeds (RPM): N/A gpu: nvidia fan: 41%
Info:
  Processes: 301 Uptime: 1h 5m wakeups: 2 Memory: 39.14 GiB
  used: 4.06 GiB (10.4%) Init: systemd v: 249 tool: systemctl Compilers:
  gcc: 11.1.0 Packages: pacman: 1222 lib: 334 flatpak: 0 Shell: Zsh v: 5.8
  default: Bash v: 5.1.8 running-in: konsole inxi: 3.3.08
6

OK yeah my bad, for Intel CPU it is called VT-d, not IOMMU. The motherboard and CPU support it. So if you enabled it in UEFI it should be enabled. I don’t really know anything about this technology and passthrough, just a guess, thy without the GRUB forced settings (EDIT: but reading the Arch WIKI PCI passthrough via OVMF - ArchWiki, these parameters should be good actually, so probably wrong guess)?

EDIT: maybe you could update your very outdated BIOS if you know how to do it. You have the initial BIOS released at same time of the first revision of the board, in 2016.

7

I have no experince with updating a BIOS but I guess I could give it a shot.

8

If you know nothing about it, it is doable, but first know that there is a RISK implied in updating the motherboard BIOS, if you power down the computer during the operation, it could simply brick the motherboard. In 20 years of messing around this never happened to me, but this is a possible risk.

Here is the download page for your motherboard BIOS https://www.msi.com/Motherboard/support/B250-GAMING-PRO-CARBON#down-bios

From you motherboard manual you can read instructions on how to do that https://www.msi.com/Motherboard/support/B250-GAMING-PRO-CARBON#down-manual Look at the M-FLASH part of the manual.

EDIT: also, before flashing the BIOS, make sure you have a Manjaro USB stick ready, in case you can’t boot back on Manjaro, it always happens to me (flashing the BIOS erase the motherboard settings and what contains the bootloader information to detect the OS), it never happens to some other people so just in case, be ready to use the Manjaro USB stick to detect your Manjaro installation, boot on it, and restore GRUB to fix this issue.

1 Like
9

VT-d is the iommu setting if i recall correctly.

But it appears you did not properly change your commmandline, according to your inxi output your kernel params do not include the iommu parameters you listed in your OP…

Actually in your OP there’s:

sudo grub-mkconfig -o /boot/grub/grub.fg

But it should be:

sudo grub-mkconfig -o /boot/grub/grub.cfg

(Always use tab to autocomplete pre-existing filenames in the terminal just to be sure you got them right :wink: )

3 Likes
10

Indeed the kernel parameters do not appear in the inxi output. Well spotted I just assumed he did it correctly.

On a side not, just use sudo update-grub so no issues. It will properly update grub. And also now if he did the improper command with the wrong file, there is a /boot/grub/grub.fg lying around :smiley:

11

Thx for the help. I updated my bios and fixed the typo and now iommu seems to be working, thoug havent tested it. I doubt the bois update is what fixed it, but it was probably good to update it anyways since it was so outdated.

sudo dmesg | grep -i -e DMAR -e IOMMU
                                                                                                                                                                          
[    0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-5.13-x86_64 root=UUID=ed74b9c6-7e1c-403f-87cd-903f93295482 rw intel_iommu=on iommu=pt quiet apparmor=1 security=apparmor udev.log_priority=3
[    0.006187] ACPI: DMAR 0x000000007EC473F0 000070 (v01 INTEL  KBL      00000001 INTL 00000001)
[    0.006205] ACPI: Reserving DMAR table memory at [mem 0x7ec473f0-0x7ec4745f]
[    0.058687] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-5.13-x86_64 root=UUID=ed74b9c6-7e1c-403f-87cd-903f93295482 rw intel_iommu=on iommu=pt quiet apparmor=1 security=apparmor udev.log_priority=3
[    0.058713] DMAR: IOMMU enabled
[    0.136038] DMAR: Host address width 39
[    0.136039] DMAR: DRHD base: 0x000000fed90000 flags: 0x1
[    0.136042] DMAR: dmar0: reg_base_addr fed90000 ver 1:0 cap d2008c40660462 ecap f050da
[    0.136044] DMAR: RMRR base: 0x0000007ead2000 end: 0x0000007eaf1fff
[    0.136046] DMAR-IR: IOAPIC id 2 under DRHD base  0xfed90000 IOMMU 0
[    0.136046] DMAR-IR: HPET id 0 under DRHD base 0xfed90000
[    0.136047] DMAR-IR: Queued invalidation will be enabled to support x2apic and Intr-remapping.
[    0.137426] DMAR-IR: Enabled IRQ remapping in x2apic mode
[    0.256898] iommu: Default domain type: Passthrough (set via kernel command line)
[    0.302254] DMAR: No ATSR found
[    0.302254] DMAR: No SATC found
[    0.302255] DMAR: dmar0: Using Queued invalidation
[    0.302279] pci 0000:00:00.0: Adding to iommu group 0
[    0.302287] pci 0000:00:01.0: Adding to iommu group 1
[    0.302292] pci 0000:00:08.0: Adding to iommu group 2
[    0.302300] pci 0000:00:14.0: Adding to iommu group 3
[    0.302305] pci 0000:00:14.2: Adding to iommu group 3
[    0.302311] pci 0000:00:16.0: Adding to iommu group 4
[    0.302316] pci 0000:00:17.0: Adding to iommu group 5
[    0.302322] pci 0000:00:1b.0: Adding to iommu group 6
[    0.302327] pci 0000:00:1c.0: Adding to iommu group 7
[    0.302333] pci 0000:00:1c.7: Adding to iommu group 8
[    0.302340] pci 0000:00:1d.0: Adding to iommu group 9
[    0.302353] pci 0000:00:1f.0: Adding to iommu group 10
[    0.302358] pci 0000:00:1f.2: Adding to iommu group 10
[    0.302363] pci 0000:00:1f.3: Adding to iommu group 10
[    0.302367] pci 0000:00:1f.4: Adding to iommu group 10
[    0.302372] pci 0000:00:1f.6: Adding to iommu group 11
[    0.302375] pci 0000:01:00.0: Adding to iommu group 1
[    0.302377] pci 0000:01:00.1: Adding to iommu group 1
[    0.302386] pci 0000:02:00.0: Adding to iommu group 12
[    0.302392] pci 0000:02:00.1: Adding to iommu group 12
[    0.302398] pci 0000:03:00.0: Adding to iommu group 13
[    0.302403] pci 0000:04:00.0: Adding to iommu group 14
[    0.302409] pci 0000:06:00.0: Adding to iommu group 15
[    0.302427] DMAR: Intel(R) Virtualization Technology for Directed I/O
[    0.323190] AMD-Vi: AMD IOMMUv2 driver by Joerg Roedel <jroedel@suse.de>
[    0.323190] AMD-Vi: AMD IOMMUv2 functionality not available on this system
[    1.011630]     intel_iommu=on

bit of a noob mistake

1 Like
12

Yeah the typo was the issue there :wink: but also updating your BIOS was a good thing regarding all the improvements you can read on the BIOS download page from the 6 past BIOS releases.

13

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.