一、前提条件
主板 Bios 中开启虚拟化功能,vt-d,请根据主板型号自行设置
二、开启 iommu,网卡直通
编辑grub,建议先备份一下原文件
vi /etc/default/grub
#在里面找到:
GRUB_CMDLINE_LINUX_DEFAULT="quiet"
intel网卡改为下面的
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt pcie_port_pm=off"
amd网卡改为下面的
GRUB_CMDLINE_LINUX_DEFAULT="quiet amd_iommu=on iommu=pt pcie_port_pm=off"
修改完成后,执行:
update-grub
reboot
三、显卡直通
3.1 屏蔽显卡驱动
#直通AMD显卡,请使用下面命令
echo "blacklist radeon" >> /etc/modprobe.d/blacklist.conf
echo "blacklist amdgpu" >> /etc/modprobe.d/blacklist.conf
#直通NVIDIA显卡,请使用下面命令
echo "blacklist nouveau" >> /etc/modprobe.d/blacklist.conf
echo "blacklist nvidia" >> /etc/modprobe.d/blacklist.conf
echo "blacklist nvidiafb" >> /etc/modprobe.d/blacklist.conf
#直通INTEL核显,请使用下面命令,注意!如果使用Gvt-G,请不要使用下面的命令
echo "blacklist snd_hda_intel" >> /etc/modprobe.d/blacklist.conf
echo "blacklist snd_hda_codec_hdmi" >> /etc/modprobe.d/blacklist.conf
echo "blacklist i915" >> /etc/modprobe.d/blacklist.conf
3.2 绑定PCI
使用lspci 查看自己的显卡PCI地址,如02:00使用lspci -n 查看显卡的did和vid。我这边是02:00,可以看到下面输出
root@pve1:~# lspci -n
00:00.0 0600: 8086:3ec4 (rev 0a)
00:01.0 0604: 8086:1901 (rev 0a)
00:02.0 0300: 8086:3e9b00:14.0 0c03: 8086:a12f (rev 31)
00:17.0 0106: 8086:a102 (rev 31)
00:1c.0 0604: 8086:a114 (rev f1)
00:1d.0 0604: 8086:a118 (rev f1)
00:1d.2 0604: 8086:a11a (rev f1)
00:1d.3 0604: 8086:a11b (rev f1)
00:1f.0 0601: 8086:a148 (rev 31)
00:1f.2 0580: 8086:a121 (rev 31)
00:1f.3 0403: 8086:a170 (rev 31)
00:1f.4 0c05: 8086:a123 (rev 31)
00:1f.6 0200: 8086:15b8 (rev 31)
01:00.0 0200: 15b3:100302:00.0 0100: 10de:1381 (rev 03)
02:00.1 0100: 10de:0fbc (rev 03)
03:00.0 0108: 8086:f1a6 (rev 03)
04:00.0 0604: 1b21:1080 (rev 04)
02:00.0 02:00.1一个是GPU,一个是声卡,两者都要一起直通,所以通过命令,把2者都绑定到vfio-pci上。
echo "options vfio-pci ids=10de:1381,10de:0fbc" > /etc/modprobe.d/vfio.conf
#注意,上面这条命令,ids=后面跟直通组的所有设备。中间以英文逗号隔开。自己的设备自己替换。
上述操作完成之后,再检查一下,是否将例子内容替换成自己的。使用以下命令查看。
root@pve:~# cat /etc/modprobe.d/blacklist.conf
root@pve:~# cat /etc/modprobe.d/vfio.conf
root@pve:~# dmesg | grep -e DMAR -e IOMMU -e AMD-Vi
[ 0.012533] ACPI: DMAR 0x000000007ACBE858 000118 (v01 ALASKA A M I 00000001 INTL 20091013)
[ 0.012569] ACPI: Reserving DMAR table memory at [mem 0x7acbe858-0x7acbe96f]
[ 0.087008] DMAR: IOMMU enabled
[ 0.244511] DMAR: Host address width 46
[ 0.244514] DMAR: DRHD base: 0x000000fbffc000 flags: 0x0
[ 0.244521] DMAR: dmar0: reg_base_addr fbffc000 ver 1:0 cap 8d2078c106f0466 ecap f020df
[ 0.244526] DMAR: DRHD base: 0x000000c7ffd000 flags: 0x0
[ 0.244531] DMAR: dmar1: reg_base_addr c7ffd000 ver 1:0 cap 8d2008c10ef0466 ecap f0205b
[ 0.244535] DMAR: DRHD base: 0x000000c7ffc000 flags: 0x1
[ 0.244539] DMAR: dmar2: reg_base_addr c7ffc000 ver 1:0 cap 8d2078c106f0466 ecap f020df
[ 0.244543] DMAR: RMRR base: 0x0000007b3e0000 end: 0x0000007b3f0fff
[ 0.244546] DMAR: ATSR flags: 0x0
[ 0.244549] DMAR: RHSA base: 0x000000c7ffc000 proximity domain: 0x0
[ 0.244552] DMAR: RHSA base: 0x000000fbffc000 proximity domain: 0x1
[ 0.244556] DMAR-IR: IOAPIC id 3 under DRHD base 0xfbffc000 IOMMU 0
[ 0.244559] DMAR-IR: IOAPIC id 1 under DRHD base 0xc7ffc000 IOMMU 2
[ 0.244562] DMAR-IR: IOAPIC id 2 under DRHD base 0xc7ffc000 IOMMU 2
[ 0.244565] DMAR-IR: HPET id 0 under DRHD base 0xc7ffc000
[ 0.244568] DMAR-IR: x2apic is disabled because BIOS sets x2apic opt out bit.
[ 0.244568] DMAR-IR: Use 'intremap=no_x2apic_optout' to override the BIOS setting.
[ 0.245582] DMAR-IR: Enabled IRQ remapping in xapic mode
[ 0.912919] DMAR: No SATC found
[ 0.912923] DMAR: IOMMU feature sc_support inconsistent
[ 0.912924] DMAR: IOMMU feature dev_iotlb_support inconsistent
[ 0.912926] DMAR: IOMMU feature sc_support inconsistent
[ 0.912928] DMAR: IOMMU feature dev_iotlb_support inconsistent
[ 0.912931] DMAR: dmar1: Using Queued invalidation
[ 0.912938] DMAR: dmar0: Using Queued invalidation
[ 0.912942] DMAR: dmar2: Using Queued invalidation
[ 0.922490] DMAR: Intel(R) Virtualization Technology for Directed I/O
3.3 更新内核
### 对于nvidia显卡:
echo "options kvm ignore_msrs=1" > /etc/modprobe.d/kvm.conf
update-initramfs -k all -u
2 条评论
兄弟你的avhub项目怎么搜不了番号了
demo环境的压力过大,服务器顶不住了