XPDDS19 Keynote: Secret-free Hypervisor: Now and Future - Wei Liu, Software Engineer, Citrix
1 like953 views
The document discusses the development of a secret-free hypervisor prompted by issues like Meltdown and Spectre, focusing on rethinking Xen's strategy to eliminate sensitive data leaks. Key targets include removing direct memory mapping, implementing per-CPU stacks, and redefining the memory allocation structure within Xen. The project involves extensive code revisions and patch development to enhance security while maintaining performance.
XPDDS19 Keynote: Secret-free Hypervisor: Now and Future - Wei Liu, Software Engineer, Citrix
- 1. Secret-free Hypervisor: Now and Future Wei Liu <wl@xen.org>
- 2. Agenda • Why this project • What and how • Primary targets • Project scope • Project showcase • Project status
- 3. Why this project • Prompted by Meltdown and Spectre • Side-channels are here to stay • Hardware bugs are here to stay • Well-established isolation mechanisms and security boundaries are broken • Rethink Xen’s strategy against a whole class of issues • What if there is no secret to leak in the first place?
- 4. Primary targets • Direct map • Xen has a 1:1 mapping of all physical memory in the machine for performance reason • If you can read from that mapping, you can read everything in the machine • Stack • Stack in Xen is allocated from an always mapped heap (xenheap) visible to all PCPUs • Stack content is considered sensitive
- 5. Fixes? • Direct map • Remove it completely • Map / unmap pages as required when accessing memory • Stack • Switch to per-PCPU stack which is not visible to other PCPUs in the same machine
- 6. Xen code is … J • Xen’s academic origin and organic growth for years brings some challenging aspects: • Messy or not so modular code structure • Missing design documents • Lacking comments • Interesting-looking code and design decisions
- 7. A taste of the scope Secret-free hypervisor Remove direct map Use per-PCPU stack Implement Xenheap with vmap Remove mapping infrastructure’s dependency on direct map Implement per-PCPU mapping infrastructure Implement Persistent MAP (PMAP) infrastructure for bootstrapping Use Domheap for page tables Carve out address space Adjust early boot / AP bringup code Maintain correct mapping while context switching
- 8. Project showcase Secret-free hypervisor Remove direct map Use per-PCPU stack Implement Xenheap with vmap Remove mapping infrastructure’s dependency on direct map Implement per-PCPU mapping infrastructure Implement Persistent MAP (PMAP) infrastructure for bootstrapping Use Domheap for page tables Carve out address space Adjust early boot / AP bringup code Maintain correct mapping while context switching
- 9. Domheap for page tables • Currently page tables are xenheap pages (always mapped pages) • All code is written with the “always mapped” property in mind • Want to use domheap for page tables • Need to (painstakingly) rewrite internal APIs and dependent code without breaking bisection • More importantly, the changes need to be broken down to reviewable patches
- 10. Project showcase Secret-free hypervisor Remove direct map Use per-PCPU stack Implement Xenheap with vmap Remove mapping infrastructure’s dependency on direct map Implement per-PCPU mapping infrastructure Implement Persistent MAP (PMAP) infrastructure for bootstrapping Use Domheap for page tables Carve out address space Adjust early boot / AP bringup code Maintain correct mapping while context switching
- 11. Implement xenheap with vmap • Less secrets in xenheap but keep its property • Currently vmap is implemented on top of xenheap -- need to flip their relationship • Problems so far: • Make sure page <-> virt conversion functions continue working (or remove them entirely with lots of code churn…) • Make vmap able to unmap a portion of a mapping – there is an optimization in xmalloc which depends on such behavior • Need to break a xenheap-vmap call loop in the page table allocation APIs - - this is reason for the previous project • Patches need not break bisection and be reviewable
- 12. Project showcase Secret-free hypervisor Remove direct map Use per-PCPU stack Implement Xenheap with vmap Remove mapping infrastructure’s dependency on direct map Implement per-PCPU mapping infrastructure Implement Persistent MAP (PMAP) infrastructure for bootstrapping Use Domheap for page tables Carve out address space Adjust early boot / AP bringup code Maintain correct mapping while context switching
- 13. Implement PMAP infrastructure • Named (badly) after an infrastructure in Linux • The only similarity is they both have a small-ish address space with statically set up page tables • To solve a bootstrapping problem: • Per-PCPU mapping infrastructure needs some mapping mechanism to set itself up without a direct map • Only used during CPU onlining / offlining
- 14. Status Secret-free hypervisor Remove direct map Use per-PCPU stack Implement Xenheap with vmap Remove mapping infrastructure’s dependency on direct map Implement per-PCPU mapping infrastructure Implement Persistent MAP (PMAP) infrastructure for bootstrapping Use Domheap for page tables Carve out address space Adjust early boot / AP bringup code Maintain correct mapping while context switching 50+ patches ~5 patches 10+ patches (incomplete) ~5 patches ~10 patches ~15 patches Legend: Patches written from scratch Patches from earlier work that are maybe salvageable
- 15. Missing aspect: performance • Can’t really test performance at this stage • May not be too bad (but I could be wrong): • Removing direct map • Xen only needs to access a few pages while a guest is running • PV guest page manipulation code is already very long • Per-PCPU stack • A PCPU only cares about its own stack during normal operation
- 16. More information • https://xenproject.atlassian.net/browse/XEN-119 • https://lists.xenproject.org/archives/html/xen-devel/2019- 01/msg01923.html • https://lists.xenproject.org/archives/html/xen-devel/2018- 10/msg01477.html