SlideShare a Scribd company logo
Xen on x86, 15 years later
Recent development, future direction
PVH Guests
KConfig
PVShim
PVCalls VM Introspection /
Memaccess
PVH dom0
QEMU Deprivileging
Panopticon
Sub-page protection
NVDIMM
Posted Interrupts
Large guests (288 vcpus)
PV IOMMU ACPI Memory Hotplug
Hypervisor
Multiplexing
Talk approach
• Highlight some key features

• Recently finished

• In progress

• Cool Idea: Should be possible, nobody committed to working on it yet

• Highlight how these work together to create interesting theme
• PVH (with PVH dom0)

• KConfig

• … to disable PV

• PVshim

• Windows in PVH
PVH: Finally here
• Full PVH DomU support in Xen 4.10, Linux 4.15

• First backwards-compatibility hack

• Experimental PVH Dom0 support in Xen 4.11
PVH: What is it?
• Next-generation paravirtualization mode

• Takes advantage of hardware virtualization support

• No need for emulated BIOS or emulated devices

• Lower performance overhead than PV

• Lower memory overhead than HVM

• More secure than either PV or HVM mode
• PVH (with PVH dom0)

• KConfig

• … to disable PV

• PVshim

• Windows in PVH
KConfig
• KConfig for Xen allows…

• Users to produce smaller / more secure binaries

• Makes it easier to merge experimental functionality

• KConfig option to disable PV entirely
• PVH

• KConfig

• … to disable PV

• PVshim

• Windows in PVH
PVShim
• Some older kernels can only run in PV
mode

• Expect to run in ring 1, ask a hypervisor
to perform privileged actions

• “Shim”: A build of Xen designed to allow an
unmodified PV guest to run in PVH mode

• type=‘pvh’ / pvshim=1
Xen
PVH Guest
“Shim” Hypervisor (ring 0)
PV-only kernel (ring 1)
• PVH

• KConfig

• … to disable PV

• PVshim

• Windows in PVH
No-PV

Hypervisors
• PVH

• KConfig

• … to disable PV

• PVshim

• Windows in PVH
Windows in PVH
• Windows EFI should be able to do 

• OVMF (Virtual EFI implementation) already has

• PVH support

• Xen PV disk, network support

• Only need PV Framebuffer support…?
• PVH

• KConfig

• … to disable PV

• PVshim

• Windows in PVH
One guest type

to rule them all
Is PV mode obsolete then?
• KConfig: No HVM 

• PV 9pfs

• PVCalls

• rkt Stage 1 

• Hypervisor Multiplexing
Containers: Passing through “host” OS resources
Container Container Container Container
Host Filesystem
Linux Container HostHost Port Host Port
Containers: Passing through “host” OS resources
• Allows file-based difference tracking rather
than block-based

• Allows easier inspection of container state
from host OS

• Allows setting up multiple isolated services
without needing to mess around with
multiple IP addresses
Container Container Container Container
Host Filesystem
LinuxHost Host
• KConfig: No HVM 

• PV 9pfs

• PVCalls

• rkt Stage 1

• Hypervisor Multiplexing
PV 9pfs
• Allows dom0 to expose files directly to
guests
Xen
Dom0
9pfs
Backend
Host
Filesystem
9pfs
Front
9pfs
Front
• KConfig: No HVM 

• PV 9pfs

• PVCalls

• rkt Stage 1

• Hypervisor Multiplexing
PV Calls
• Pass through specific system calls

• socket()
• listen()
• accept()
• read()
• write()
Xen
Dom0
PVCalls
Backend
Host
Port
PVCalls
Front
PVCalls
Front
Host
Port
• KConfig: No HVM 

• PV 9pfs

• PVCalls

• rkt Stage 1

• Hypervisor Multiplexing
rkt Stage 1
• rkt: “Container abstraction” part of CoreOS

• Running rkt containers (part of CoreOS) under Xen
• KConfig: No HVM 

• PV 9pfs

• PVCalls

• rkt Stage 1

• Hypervisor Multiplexing
Xen as full
Container
Host
• KConfig: No HVM 

• PV 9pfs

• PVCalls

• rkt Stage 1

• Hypervisor Multiplexing
Hypervisor multiplexing
• Xen can run in an HVM guest /without
nested HVM support/

• PV protocols use xenbus + hypercalls

• At the moment, Linux code assumes
only one xenbus / hypervisor

• Host PV drivers 

• OR Guest PV drivers

• Multiplexing: Allow both
L1 Xen
Back FrontFront
L0 Xen
Back
L1 Dom0
L0 Dom0HVM Guest
• KConfig: No HVM 

• PV 9pfs

• PVCalls

• rkt Stage 1

• Hypervisor Multiplexing
Xen as
Cloud-ready
Container
Host
QEMU Deprivileging
• Restricting hypercalls to a single guest

• Restricting what QEMU can do within dom0
Panopticon / No Secrets
• Spectre-style information leaks

• You can only leak what you can see

• Xen has all of physical memory mapped

• But this is not really necessary

• Assume that all guests can read hypervisor memory at all times
PVH Guests
KConfig
PVShim
PVCalls VM Introspection /
Memaccess
PVH dom0
QEMU Deprivileging
Panopticon
Sub-page protection
NVDIMM
Posted Interrupts
Large guests (288 vcpus)
PV IOMMU ACPI Memory Hotplug
Hypervisor
Multiplexing
Questions

More Related Content

PDF
OSSNA18: Xen Beginners Training
PDF
OSSNA18: Xen Beginners Training (exercise script)
PDF
OSSEU18: NVDIMM and Virtualization - George Dunlap, Citrix
PDF
Securing Your Cloud With the Xen Hypervisor by Russell Pavlicek
PPTX
XPDDS18: Windows PV Drivers Project: Status and Updates - Paul Durrant, Citri...
PDF
2018 Genivi Xen Overview Nov Update
PDF
Securing Your Cloud with Xen (CloudOpen NA 2013)
PDF
Xen io
OSSNA18: Xen Beginners Training
OSSNA18: Xen Beginners Training (exercise script)
OSSEU18: NVDIMM and Virtualization - George Dunlap, Citrix
Securing Your Cloud With the Xen Hypervisor by Russell Pavlicek
XPDDS18: Windows PV Drivers Project: Status and Updates - Paul Durrant, Citri...
2018 Genivi Xen Overview Nov Update
Securing Your Cloud with Xen (CloudOpen NA 2013)
Xen io

What's hot (20)

PPTX
Linuxcon EU : Virtualization in the Cloud featuring Xen and XCP
PDF
Xen Project 15 Years down the Line
PDF
Rootlinux17: An introduction to Xen Project Virtualisation
PDF
LCA13: Xen on ARM
PDF
Xen 10th anniversary Status Report (at SELF 2013)
ODP
S4 xen hypervisor_20080622
PPTX
CIF16/Scale14x: The latest from the Xen Project (Lars Kurth, Chairman of Xen ...
PDF
Ian Pratt Nsdi Keynote Apr2008
PDF
XPDS14: Xen and the Art of Certification - Nathan Studer & Robert VonVossen, ...
PPTX
LFCOLLAB15: Xen 4.5 and Beyond
PPTX
Scale14x: Are today's foss security practices robust enough in the cloud era ...
PDF
LF Collaboration Summit: Xen Project 4 4 Features and Futures
PDF
Bare-Metal Hypervisor as a Platform for Innovation
PDF
XPDS13: HVM Dom0 - Any unmodified OS as Dom0 - Will Auld, Intel
PDF
LCEU13: Securing your cloud with Xen's advanced security features - George Du...
PDF
Xen time machine
PDF
Xen, XenServer, and XAPI: What’s the Difference?-XPUS13 Bulpin,Pavlicek
PDF
Cloud Computing Hypervisors and Comparison Xen KVM
PDF
Aplura virtualization slides
PDF
Scale 12x Securing Your Cloud with The Xen Hypervisor
Linuxcon EU : Virtualization in the Cloud featuring Xen and XCP
Xen Project 15 Years down the Line
Rootlinux17: An introduction to Xen Project Virtualisation
LCA13: Xen on ARM
Xen 10th anniversary Status Report (at SELF 2013)
S4 xen hypervisor_20080622
CIF16/Scale14x: The latest from the Xen Project (Lars Kurth, Chairman of Xen ...
Ian Pratt Nsdi Keynote Apr2008
XPDS14: Xen and the Art of Certification - Nathan Studer & Robert VonVossen, ...
LFCOLLAB15: Xen 4.5 and Beyond
Scale14x: Are today's foss security practices robust enough in the cloud era ...
LF Collaboration Summit: Xen Project 4 4 Features and Futures
Bare-Metal Hypervisor as a Platform for Innovation
XPDS13: HVM Dom0 - Any unmodified OS as Dom0 - Will Auld, Intel
LCEU13: Securing your cloud with Xen's advanced security features - George Du...
Xen time machine
Xen, XenServer, and XAPI: What’s the Difference?-XPUS13 Bulpin,Pavlicek
Cloud Computing Hypervisors and Comparison Xen KVM
Aplura virtualization slides
Scale 12x Securing Your Cloud with The Xen Hypervisor
Ad

Similar to XPDDS18: LCC18: Xen Project: After 15 years, What's Next? - George Dunlap, Citrix (20)

PDF
XPDDS17: Recent and Ongoing Xen Related Work in the Linux Kernel - Jürgen Gr...
PPTX
LinuxCon Japan 13 : 10 years of Xen and Beyond
PPTX
Xen Cloud Platform Update
PDF
XCP: The Art of Open Virtualization for the Enterprise and the Cloud
PDF
RHEL5 XEN HandOnTraining_v0.4.pdf
PDF
RMLL / LSM 2009
PDF
Rmll Virtualization As Is Tool 20090707 V1.0
PPTX
Cloud.pptm
PPTX
LinuxTag13: 10 years of Xen and beyond
PDF
Using and Understanding Xen4Centos
PDF
Xen Hypervisor Update 2011
PDF
20 christian ferber xen_server_6_workshop
PDF
XPDS14: Xen 4.5 Roadmap - Konrad Wilk, Oracle
PPTX
17-virtualization.pptx
PDF
Linaro connect : Introduction to Xen on ARM
PPTX
Xen Project Update LinuxCon Brazil
PDF
Oscon 2012 : From Datacenter to the Cloud - Featuring Xen and XCP
PDF
CentOS at Facebook
PPTX
XenServer Design Workshop
PPTX
vBACD July 2012 - Xen Cloud Platform
XPDDS17: Recent and Ongoing Xen Related Work in the Linux Kernel - Jürgen Gr...
LinuxCon Japan 13 : 10 years of Xen and Beyond
Xen Cloud Platform Update
XCP: The Art of Open Virtualization for the Enterprise and the Cloud
RHEL5 XEN HandOnTraining_v0.4.pdf
RMLL / LSM 2009
Rmll Virtualization As Is Tool 20090707 V1.0
Cloud.pptm
LinuxTag13: 10 years of Xen and beyond
Using and Understanding Xen4Centos
Xen Hypervisor Update 2011
20 christian ferber xen_server_6_workshop
XPDS14: Xen 4.5 Roadmap - Konrad Wilk, Oracle
17-virtualization.pptx
Linaro connect : Introduction to Xen on ARM
Xen Project Update LinuxCon Brazil
Oscon 2012 : From Datacenter to the Cloud - Featuring Xen and XCP
CentOS at Facebook
XenServer Design Workshop
vBACD July 2012 - Xen Cloud Platform
Ad

More from The Linux Foundation (20)

PDF
ELC2019: Static Partitioning Made Simple
PDF
XPDDS19: How TrenchBoot is Enabling Measured Launch for Open-Source Platform ...
PDF
XPDDS19 Keynote: Xen in Automotive - Artem Mygaiev, Director, Technology Solu...
PDF
XPDDS19 Keynote: Xen Project Weather Report 2019 - Lars Kurth, Director of Op...
PDF
XPDDS19 Keynote: Unikraft Weather Report
PDF
XPDDS19 Keynote: Secret-free Hypervisor: Now and Future - Wei Liu, Software E...
PDF
XPDDS19 Keynote: Xen Dom0-less - Stefano Stabellini, Principal Engineer, Xilinx
PDF
XPDDS19 Keynote: Patch Review for Non-maintainers - George Dunlap, Citrix Sys...
PDF
XPDDS19: Memories of a VM Funk - Mihai Donțu, Bitdefender
PPTX
OSSJP/ALS19: The Road to Safety Certification: Overcoming Community Challeng...
PPTX
OSSJP/ALS19: The Road to Safety Certification: How the Xen Project is Making...
PDF
XPDDS19: Speculative Sidechannels and Mitigations - Andrew Cooper, Citrix
PDF
XPDDS19: Keeping Coherency on Arm: Reborn - Julien Grall, Arm ltd
PDF
XPDDS19: QEMU PV Backend 'qdevification'... What Does it Mean? - Paul Durrant...
PDF
XPDDS19: Status of PCI Emulation in Xen - Roger Pau Monné, Citrix Systems R&D
PDF
XPDDS19: [ARM] OP-TEE Mediator in Xen - Volodymyr Babchuk, EPAM Systems
PDF
XPDDS19: Bringing Xen to the Masses: The Story of Building a Community-driven...
PDF
XPDDS19: Will Robots Automate Your Job Away? Streamlining Xen Project Contrib...
PDF
XPDDS19: Client Virtualization Toolstack in Go - Nick Rosbrook & Brendan Kerr...
PDF
XPDDS19: Core Scheduling in Xen - Jürgen Groß, SUSE
ELC2019: Static Partitioning Made Simple
XPDDS19: How TrenchBoot is Enabling Measured Launch for Open-Source Platform ...
XPDDS19 Keynote: Xen in Automotive - Artem Mygaiev, Director, Technology Solu...
XPDDS19 Keynote: Xen Project Weather Report 2019 - Lars Kurth, Director of Op...
XPDDS19 Keynote: Unikraft Weather Report
XPDDS19 Keynote: Secret-free Hypervisor: Now and Future - Wei Liu, Software E...
XPDDS19 Keynote: Xen Dom0-less - Stefano Stabellini, Principal Engineer, Xilinx
XPDDS19 Keynote: Patch Review for Non-maintainers - George Dunlap, Citrix Sys...
XPDDS19: Memories of a VM Funk - Mihai Donțu, Bitdefender
OSSJP/ALS19: The Road to Safety Certification: Overcoming Community Challeng...
OSSJP/ALS19: The Road to Safety Certification: How the Xen Project is Making...
XPDDS19: Speculative Sidechannels and Mitigations - Andrew Cooper, Citrix
XPDDS19: Keeping Coherency on Arm: Reborn - Julien Grall, Arm ltd
XPDDS19: QEMU PV Backend 'qdevification'... What Does it Mean? - Paul Durrant...
XPDDS19: Status of PCI Emulation in Xen - Roger Pau Monné, Citrix Systems R&D
XPDDS19: [ARM] OP-TEE Mediator in Xen - Volodymyr Babchuk, EPAM Systems
XPDDS19: Bringing Xen to the Masses: The Story of Building a Community-driven...
XPDDS19: Will Robots Automate Your Job Away? Streamlining Xen Project Contrib...
XPDDS19: Client Virtualization Toolstack in Go - Nick Rosbrook & Brendan Kerr...
XPDDS19: Core Scheduling in Xen - Jürgen Groß, SUSE

Recently uploaded (20)

PDF
7 ChatGPT Prompts to Help You Define Your Ideal Customer Profile.pdf
PDF
Dropbox Q2 2025 Financial Results & Investor Presentation
PPT
“AI and Expert System Decision Support & Business Intelligence Systems”
PPTX
Spectroscopy.pptx food analysis technology
PDF
Approach and Philosophy of On baking technology
PDF
Empathic Computing: Creating Shared Understanding
PPTX
Programs and apps: productivity, graphics, security and other tools
PDF
KodekX | Application Modernization Development
PDF
Mobile App Security Testing_ A Comprehensive Guide.pdf
PDF
cuic standard and advanced reporting.pdf
PPTX
ACSFv1EN-58255 AWS Academy Cloud Security Foundations.pptx
PDF
Profit Center Accounting in SAP S/4HANA, S4F28 Col11
PPTX
KOM of Painting work and Equipment Insulation REV00 update 25-dec.pptx
PDF
Unlocking AI with Model Context Protocol (MCP)
PDF
Build a system with the filesystem maintained by OSTree @ COSCUP 2025
PPTX
sap open course for s4hana steps from ECC to s4
PDF
Machine learning based COVID-19 study performance prediction
PPTX
MYSQL Presentation for SQL database connectivity
PPTX
Understanding_Digital_Forensics_Presentation.pptx
PDF
Agricultural_Statistics_at_a_Glance_2022_0.pdf
7 ChatGPT Prompts to Help You Define Your Ideal Customer Profile.pdf
Dropbox Q2 2025 Financial Results & Investor Presentation
“AI and Expert System Decision Support & Business Intelligence Systems”
Spectroscopy.pptx food analysis technology
Approach and Philosophy of On baking technology
Empathic Computing: Creating Shared Understanding
Programs and apps: productivity, graphics, security and other tools
KodekX | Application Modernization Development
Mobile App Security Testing_ A Comprehensive Guide.pdf
cuic standard and advanced reporting.pdf
ACSFv1EN-58255 AWS Academy Cloud Security Foundations.pptx
Profit Center Accounting in SAP S/4HANA, S4F28 Col11
KOM of Painting work and Equipment Insulation REV00 update 25-dec.pptx
Unlocking AI with Model Context Protocol (MCP)
Build a system with the filesystem maintained by OSTree @ COSCUP 2025
sap open course for s4hana steps from ECC to s4
Machine learning based COVID-19 study performance prediction
MYSQL Presentation for SQL database connectivity
Understanding_Digital_Forensics_Presentation.pptx
Agricultural_Statistics_at_a_Glance_2022_0.pdf

XPDDS18: LCC18: Xen Project: After 15 years, What's Next? - George Dunlap, Citrix

  • 1. Xen on x86, 15 years later Recent development, future direction
  • 2. PVH Guests KConfig PVShim PVCalls VM Introspection / Memaccess PVH dom0 QEMU Deprivileging Panopticon Sub-page protection NVDIMM Posted Interrupts Large guests (288 vcpus) PV IOMMU ACPI Memory Hotplug Hypervisor Multiplexing
  • 3. Talk approach • Highlight some key features • Recently finished • In progress • Cool Idea: Should be possible, nobody committed to working on it yet • Highlight how these work together to create interesting theme
  • 4. • PVH (with PVH dom0) • KConfig • … to disable PV • PVshim • Windows in PVH
  • 5. PVH: Finally here • Full PVH DomU support in Xen 4.10, Linux 4.15 • First backwards-compatibility hack • Experimental PVH Dom0 support in Xen 4.11
  • 6. PVH: What is it? • Next-generation paravirtualization mode • Takes advantage of hardware virtualization support • No need for emulated BIOS or emulated devices • Lower performance overhead than PV • Lower memory overhead than HVM • More secure than either PV or HVM mode
  • 7. • PVH (with PVH dom0) • KConfig • … to disable PV • PVshim • Windows in PVH
  • 8. KConfig • KConfig for Xen allows… • Users to produce smaller / more secure binaries • Makes it easier to merge experimental functionality • KConfig option to disable PV entirely
  • 9. • PVH • KConfig • … to disable PV • PVshim • Windows in PVH
  • 10. PVShim • Some older kernels can only run in PV mode • Expect to run in ring 1, ask a hypervisor to perform privileged actions • “Shim”: A build of Xen designed to allow an unmodified PV guest to run in PVH mode • type=‘pvh’ / pvshim=1 Xen PVH Guest “Shim” Hypervisor (ring 0) PV-only kernel (ring 1)
  • 11. • PVH • KConfig • … to disable PV • PVshim • Windows in PVH No-PV
 Hypervisors
  • 12. • PVH • KConfig • … to disable PV • PVshim • Windows in PVH
  • 13. Windows in PVH • Windows EFI should be able to do • OVMF (Virtual EFI implementation) already has • PVH support • Xen PV disk, network support • Only need PV Framebuffer support…?
  • 14. • PVH • KConfig • … to disable PV • PVshim • Windows in PVH One guest type
 to rule them all
  • 15. Is PV mode obsolete then?
  • 16. • KConfig: No HVM • PV 9pfs • PVCalls • rkt Stage 1 • Hypervisor Multiplexing
  • 17. Containers: Passing through “host” OS resources Container Container Container Container Host Filesystem Linux Container HostHost Port Host Port
  • 18. Containers: Passing through “host” OS resources • Allows file-based difference tracking rather than block-based • Allows easier inspection of container state from host OS • Allows setting up multiple isolated services without needing to mess around with multiple IP addresses Container Container Container Container Host Filesystem LinuxHost Host
  • 19. • KConfig: No HVM • PV 9pfs • PVCalls • rkt Stage 1 • Hypervisor Multiplexing
  • 20. PV 9pfs • Allows dom0 to expose files directly to guests Xen Dom0 9pfs Backend Host Filesystem 9pfs Front 9pfs Front
  • 21. • KConfig: No HVM • PV 9pfs • PVCalls • rkt Stage 1 • Hypervisor Multiplexing
  • 22. PV Calls • Pass through specific system calls • socket() • listen() • accept() • read() • write() Xen Dom0 PVCalls Backend Host Port PVCalls Front PVCalls Front Host Port
  • 23. • KConfig: No HVM • PV 9pfs • PVCalls • rkt Stage 1 • Hypervisor Multiplexing
  • 24. rkt Stage 1 • rkt: “Container abstraction” part of CoreOS • Running rkt containers (part of CoreOS) under Xen
  • 25. • KConfig: No HVM • PV 9pfs • PVCalls • rkt Stage 1 • Hypervisor Multiplexing Xen as full Container Host
  • 26. • KConfig: No HVM • PV 9pfs • PVCalls • rkt Stage 1 • Hypervisor Multiplexing
  • 27. Hypervisor multiplexing • Xen can run in an HVM guest /without nested HVM support/ • PV protocols use xenbus + hypercalls • At the moment, Linux code assumes only one xenbus / hypervisor • Host PV drivers • OR Guest PV drivers • Multiplexing: Allow both L1 Xen Back FrontFront L0 Xen Back L1 Dom0 L0 Dom0HVM Guest
  • 28. • KConfig: No HVM • PV 9pfs • PVCalls • rkt Stage 1 • Hypervisor Multiplexing Xen as Cloud-ready Container Host
  • 29. QEMU Deprivileging • Restricting hypercalls to a single guest • Restricting what QEMU can do within dom0
  • 30. Panopticon / No Secrets • Spectre-style information leaks • You can only leak what you can see • Xen has all of physical memory mapped • But this is not really necessary • Assume that all guests can read hypervisor memory at all times
  • 31. PVH Guests KConfig PVShim PVCalls VM Introspection / Memaccess PVH dom0 QEMU Deprivileging Panopticon Sub-page protection NVDIMM Posted Interrupts Large guests (288 vcpus) PV IOMMU ACPI Memory Hotplug Hypervisor Multiplexing