GPU Passthrough with Proxmox 7.4 (with an AMD or NVidia GPU) (updated)
Once again, things have changed.
First, please reference:
/etc/kernel/cmdline only needs to contain the following:
root=ZFS=rpool/ROOT/pve-1 boot=zfs intel_iommu=on iommu=pt initcall_blacklist=sysfb_initOf course, only modify the part after boot=zfs. Your line before that point may vary. I understand that Proxmox may now automatically define either “intel_iommu=on” or “amd_iommu=on”, but it doesn’t hurt to add the setting either way.
Next, be sure you’re utilizing either a physical and plugged-in monitor or an HDMI or DP dongle. Virtual display or monitor drivers DO NOT WORK. If your installation is sluggish and graphics on the Windows desktop seem distorted or out-of-place, you’ve run into this issue. For me, the start menu would only draw a sliver on the left edge and control panel windows had refresh and write-over issues. Add a physical display or dummy monitor dongle.
Once you have both the “Redhat vfio” adapter and your GPU adapter working correctly in Device Manager, shut it down and set your display to “none” in hardware. When you start back up, you must connect with a remote viewer like RDP, No Machine, or Parsec. For performance and gaming, Parsec or Moonlight are a must.
Have fun!
Update for AMD GPUs on Proxmox 7.4:
Kernel 5.15 causes some issues related to a bug on 400, 500 and 5000 series GPUs with regard to a necessary module called “vendor-reset“. If you find that your passthrough only works for a single time without rebooting your Proxmox host, you’ve encountered this bug. Vendor-reset is necessary to work around this. As of the production kernel in Proxmox 7.4, vendor-reset is broken. To repair it, do the following:
https://github.com/gnif/vendor-reset/issues/46#issuecomment-1295482826
Frustratingly, from what I understand, this will only work with the remaining 5.15+ kernels (5.16, 5.17, etc…) Kernel 6.x will introduce a new wrench into the equation, but we’ll cross that bridge once we get there.
Additionally, the following options should be set:
/etc/kernel/cmdline
root=ZFS=rpool/ROOT/pve-1 boot=zfs intel_iommu=on iommu=pt initcall_blacklist=sysfb_init noats nobar hugepagesz=2M hugepages=1375 default_hugepagesz=1G hugepagesz=1G hugepages=8 textNo, unfortunately, I don’t know why “hugepages” is set twice with different values, or if not, which one is correct.
qm set <VMID> -cpu "host,hidden=1,flags=+pcid"
qm set <VMID> -args '-cpu 'host,+kvm_pv_unhalt,+kvm_pv_eoi,hv_vendor_id=NV43FIX,kvm=off''Play around with these flags, especially “kvm=off” since that may come with a large performance penalty. NV43FIX is in theory just a way to obfuscate machine being run in virtualization, which was only needed for nVidia cards. Test removing that as well.