How to share your intel gpu with multiple VMs on proxmox

How to share your intel gpu with multiple VMs on proxmox
Have you ever wanted to use your GPU on multiple virtual machines because you wanted to do multiple things but on a different OS or wanted to give multiple users the same GPU?

Now you can with the magic that is SR-IOV if you have a intel GPU! (results may vary depending on your GPU, mine is a 12gen intel GPU, UHD Graphics 770)

Do you have NVIDIA or AMD instead? For the first, pay up for enterprise stuff (or use hacks such as this one on supported architectures) and for the latter the answer is get f****d as the GPUs from AMD that support SR-IOV can be counted with two hands and are enterprise ones anyway.

Until a few months ago there was basically no support (and there still is no support) and a custom intel kernel was needed as even if you enabled the feature, the linux i915 driver (which is needed in the host) would not be able to create the multiple virtual devices. Fortunately this has changed and an experimental DKMS module has been made which works on the pve-kernel-6.1.0-1-pve kernel correctly at the time of writing.

Note: The 6.1 kernel seems to be mandatory and it's not yet in production so there could be some bugs, but you can ignore that and do:

- apt install pve-kernel-6.1
- apt install pve-headers-6.1

Of course there was no installation guide on the github page, otherwhise where would the fun be? (Experienced linux developers might know how to install a DKMS module manually, but I didn't)

Let's install prerequisites via apt install dkms build-essential make

Fortunately, the AUR package had an install script so I just reversed it, which boils down to:

1. git clone https://github.com/strongtz/i915-sriov-dkms
2. Edit the @[email protected] and @[email protected] values with the ones provided in the install script in the dkms.conf file
3. Copy the folder to /usr/source as /usr/src/${_pkgbase}-${pkgver}
4. Set permissions for the folder and all files to 755 and permissions for the dkms.conf file to 644
5. dkms install -m i915-sriov-dkms -v ${pkgver}
6. Edit the /etc/kernel/cmdline file (I use zfs on proxmox with systemd-boot, you might have the kernel arguments in a different place) and add intel_iommu=on i915.enable_guc=7 (The repository does not use it but I also have iommu=pt there)
7. Refresh the kernel arguments via proxmox-boot-tool refresh (see the documentation if you're not using systemd-boot)
8. apt install sysfsutils
9. Create the virtual devices permanently echo "devices/pci0000:00/0000:00:02.0/sriov_numvfs = 7" > /etc/sysfs.conf (check if your intel GPU is at 0000:00:02.0 via lspci -v)
10. Reboot

If everything went well by doing lspci you'll now have something like this!

You can now add it to your virtual machine like it was a PCI device.

Sometimes things don't work at the first time

In my case I had to add it like in the image below to a Windows VM because I was getting a code 43 error otherwhise. 🥲  
But after that there were no issues! (Note: I also installed the recommended driver for Windows from this page)

The actual fix

For future updates this would be an interesting thread to follow.

Mastodon