XPDDS18 Design Session: PCI pass-through with de-privileged QEMU - Xin Li, Citrix
Download as PPTX, PDF1 like322 views
The document discusses the implementation of PCI pass-through in a de-privileged qemu environment, detailing the differences between privileged and de-privileged modes. It lists the challenges faced in a chrooted environment, such as restricted access to sysfs and limited user permissions. It also outlines methods to enable privileged modes in different tool stacks to facilitate PCI device management while ensuring security.
![Enable privileged mode (4)
QEMU access /dev/mem to
1. read vector control(per-vector mask) from msix table entry
2. read the PBA after msix table
Remove direct access to /dev/mem
• /sys/…/resource is already parsed to get the MMIO regions info. (addr, size, bar index,
offset).
• Instead of /dev/mem, we can mmap /sys/…/resourceX.
How to verify? Capabilities: [a0] MSI-X: Enable+ Count=17 Masked+](https://image.slidesharecdn.com/pci-passthrough-deprivileged-180629141707/85/XPDDS18-Design-Session-PCI-pass-through-with-de-privileged-QEMU-Xin-Li-Citrix-8-320.jpg)
XPDDS18 Design Session: PCI pass-through with de-privileged QEMU - Xin Li, Citrix
- 1. PCI pass-through with de-privileged QEMU XIN LI, XENSERVER, CITRIX
- 2. Privileged mode • QEMU process is running as root. • QEMU process’s CWD is “/”. • No restrict to the hypercalls that QEMU can call. • …. (can do almost anything) PID UID GID CMD 8265 0 0 qemu-dm-16 –machine ... # pwdx 8265 8265: /
- 3. De-privileged mode • QEMU process is running in generated user-id and group-id. • QEMU process is chrooted. • xen-domid-restrict mode enabled. • …. PID UID GID CMD 11544 65552 1010 qemu-dm- ... # pwdx 11544 11544: /var/xen/qemu/root-17 1. disable arbitrary hypercalls, only a subset hypercalls(like MMAP, DM-Ops ….) are allowed. 2. allow operation on the specified domain only for all existing handlers.
- 4. Fallback to privileged for PCI pass-through Why, what are the problems? • In chrooted environment, both sysfs and device node path can’t be accessed. • Without CAP_SYS_ADMIN, only the first 64B of PCI config file can be read. • Non-root user can’t read all sysfs files needed for the PCI device. • /dev/mem is directly access by QEMU to map physical MSI-X table. • A set of restricted hypercalls are called only for PCI pass-through.
- 5. Enable privileged mode (1) In tool stack (XAPI): Before PCI device is attached: 1. mount sysfs in chroot directory 2. change ownership of the related sysfs resource*, vendor, device, irq, class After PCI device is detached: reset the file owner to root
- 6. Enable privileged mode (2) In tool stack (xl): In pci_add, before sending “device_add”: 1. open PCI config file as root 2. send this fd to QEMU over QMP In QEMU: 1. add new property config_fdset for XenPCIPassthroughState 2. prefer to read from this config fd from config_fdset
- 7. Enable privileged mode (3) Index New DM-OP 1 physdev_map_pirq 2 physdev_map_pirq_msi 3 physdev_unmap_pirq 4 domain_bind_pt_pci_irq 5 domain_unbind_pt_pci_irq 6 domain_update_msi_irq 7 domain_unbind_msi_irq 8 domain_iomem_permission 9 domain_memory_mapping 10 domain_ioport_mapping For xen-domid-restrict mode, add new DM-OPs, let QEMU call them instead.
- 8. Enable privileged mode (4) QEMU access /dev/mem to 1. read vector control(per-vector mask) from msix table entry 2. read the PBA after msix table Remove direct access to /dev/mem • /sys/…/resource is already parsed to get the MMIO regions info. (addr, size, bar index, offset). • Instead of /dev/mem, we can mmap /sys/…/resourceX. How to verify? Capabilities: [a0] MSI-X: Enable+ Count=17 Masked+
- 9. Thank you!