ARMvisor, more details
4 likes4,588 views
- Peter Chang is a developer of ARMvisor, which is a KVM implementation for ARM architecture - ARMvisor uses para-virtualization and trap-and-emulate techniques to virtualize sensitive ARM instructions like branches, register access, and memory operations - It traps these instructions and emulates their behavior to enable full system virtualization on ARM platforms
ARMvisor, more details
- 1. ARMvisor Peter Chang This side is licensed under CC-BY-NC-SA 姓名標示─非商業性─相同方式分享 (http://creativecommons.org/licenses/by-nc-sa/3.0/tw/legalcode)
- 2. Who am I?
- 3. Who am I? • Peter Chang
- 4. Who am I? • Peter Chang • One of the current developers of ARMvisor
- 5. Who am I? • Peter Chang • One of the current developers of ARMvisor • http://tw.linkedin.com/in/peterchangtw
- 8. ARM
- 9. ARM
- 10. ARM
- 11. What is KVM?
- 12. What is KVM? • A.k.a. Kernel-based Virtual Machine
- 13. What is KVM? • A.k.a. Kernel-based Virtual Machine • Type-II Virtual Machine Monitor
- 14. What is KVM? • A.k.a. Kernel-based Virtual Machine • Type-II Virtual Machine Monitor • A module of Linux kernel
- 15. What is KVM?
- 16. What is KVM? • Officially support x86/x64, PowerPC, S390
- 17. What is KVM? • Officially support x86/x64, PowerPC, S390 • No official support for ARM architecture
- 19. What is ARMvisor? • ARM架構上的KVM
- 20. What is ARMvisor? • ARM架構上的KVM • Para-virtualization
- 21. What is ARMvisor? • ARM架構上的KVM • Para-virtualization • Trap & Emulation
- 22. What is ARMvisor? • ARM架構上的KVM • Para-virtualization • Trap & Emulation • Dynamic Memory Allocation
- 23. What is ARMvisor? • ARM架構上的KVM • Para-virtualization • Trap & Emulation • Dynamic Memory Allocation • virtio & IRQchip-in-kernel
- 24. Guest OS: Linux 2.6.35 QEMU 0.14 Driver Device Driver ARMvisor Host OS: Linux 2.6.38 Hardware: ARM Cortex-A8
- 25. 2012 2011 2010 2009
- 26. 2012 2011 2010 2009
- 27. 2012 2011 2010 Starting Point 2009
- 28. 2012 2011 2010 Starting Point 2009
- 29. 2012 2011 ARMvisor Prototype 2010 Starting Point 2009
- 30. 2012 2011 ARMvisor Prototype 2010 Starting Point 2009
- 31. 2012 CPU Opt & Mem Opt 2011 ARMvisor Prototype 2010 Starting Point 2009
- 32. 2012 CPU Opt & Mem Opt 2011 ARMvisor Prototype 2010 Starting Point 2009
- 33. I/O Opt 2012 CPU Opt & Mem Opt 2011 ARMvisor Prototype 2010 Starting Point 2009
- 34. Supported Hardware ARM Realview-eb ARM11 ARMv6 ISA
- 35. Supported Hardware TI BeagleBoard Cortex-A8 ARMv7 ISA
- 36. Supported Software • Patched Host OS: • Linaro Linux 2.6.38 • Host Root Filesystem • Ubuntu/Debian RFS (CLI or GUI) • QEMU 0.14
- 37. Supported Software • Patched Guest OS: • Linux 2.6.35 (running on ARMv6 ISA) • Guest Root Filesystem: • Ubuntu/Debian RFS (CLI or GUI)
- 38. System model of ARMvisor
- 40. CPU virtualization • ARM is non-virtualizable CPU • Patch guest OS • “Trap and emulation”
- 41. ARMv6 ISA 1.Branch instructions 2.Data-processing instructions 3.Multiply instructions 4.Parallel addition and subtraction instructions 5.Extend instructions 6.Miscellaneous arithmetic instructions 7.Other miscellaneous instructions 8.Status register access instructions 9.Load and store instructions 10.Load and Store Multiple instructions 11.Semaphore instructions 12.Exception-generating instructions 13.Coprocessor instructions
- 42. ARMv6 ISA 1.Branch instructions Sensitive 2.Data-processing instructions Instructions ? 3.Multiply instructions 4.Parallel addition and subtraction instructions 5.Extend instructions 6.Miscellaneous arithmetic instructions 7.Other miscellaneous instructions 8.Status register access instructions 9.Load and store instructions 10.Load and Store Multiple instructions 11.Semaphore instructions 12.Exception-generating instructions 13.Coprocessor instructions
- 43. ARMv6 ISA 1.Branch instructions 2.Data-processing instructions 3.Multiply instructions 4.Parallel addition and subtraction instructions 5.Extend instructions 6.Miscellaneous arithmetic instructions 7.Other miscellaneous instructions 8.Status register access instructions 9.Load and store instructions 10.Load and Store Multiple instructions 11.Semaphore instructions 12.Exception-generating instructions 13.Coprocessor instructions
- 44. Sensitive Instructions Data-processing instructions S-BIT: MOVS, ... Status register access instructions MRS, MSR, CPS, SETEND Load and store instructions T-BIT: LDRT, STRT, ... Load and Store Multiple instructions LDM(2), LDM(3), STM(2) Exception-generating instructions SWI, BKPT Coprocessor instructions MCR, MRC, MCRR, ...
- 45. Observation from Guest Linux Code • MOVS • (MRS, MSR, CPS) • (LDRBT, LDRT, STRBT, STRT) • (LDM(2), LDM(3), STM(2)) • SWI • (MCR, MRC, MCRR) 15 sensitive instructions used in the guest linux code
- 46. “Trap & emulation” Guest%OS User%space% trap % Kernel%space VMM
- 47. How to “trap”? … mov r0, r0 add sp, sp movs pc, lr …
- 48. How to “trap”? … mov r0, r0 add sp, sp virt_svc_movs “movs pc, lr” …
- 49. How to “trap”? … mov r0, r0 add sp, sp virt_svc_movs “movs pc, lr” …
- 50. How to “trap”? .macro virt_svc_movs, inst SWI 0x190 inst … .endm mov r0, r0 add sp, sp virt_svc_movs “movs pc, lr” …
- 52. oxffff1000 0xffff001c Kernel Vector oxffff0000
- 53. oxffff1000 0xffff001c Kernel Vector oxffff0000
- 54. The KVM trap Interface oxffff1000 0xffff001c Kernel Vector oxffff0000
- 55. UND ABORT SWI IRQ/FIQ KVM Trap Entry KVM/Guest Host Trap Handler Context Switch Unit KVM Trap Dispatcher Instruction MMU Exception/Interrupt QEMU I/O Emulation Emulation Emulation Emulation
- 56. User space Kernel space Guest Mode QEMU KVM Guest OS
- 57. User space Kernel space Guest Mode 1. VM initialization QEMU KVM Guest OS
- 58. User space Kernel space Guest Mode 1. VM initialization 2. Return to QEMU QEMU KVM Guest OS
- 59. User space Kernel space Guest Mode 1. VM initialization 2. Return to QEMU 3. Run VM QEMU KVM Guest OS
- 60. User space Kernel space Guest Mode 1. VM initialization 2. Return to QEMU 3. Run VM 4. Enter Guest QEMU KVM Guest OS
- 61. User space Kernel space Guest Mode 1. VM initialization 2. Return to QEMU 3. Run VM 4. Enter Guest 5. Exit Guest QEMU KVM Guest OS
- 62. User space Kernel space Guest Mode 1. VM initialization 2. Return to QEMU 3. Run VM 4. Enter Guest 5. Exit Guest Lightweight trap QEMU KVM Guest OS
- 63. User space Kernel space Guest Mode 1. VM initialization 2. Return to QEMU 3. Run VM 4. Enter Guest 5. Exit Guest Lightweight trap 6. Enter Guest QEMU KVM Guest OS
- 64. User space Kernel space Guest Mode 1. VM initialization 2. Return to QEMU 3. Run VM 4. Enter Guest 5. Exit Guest Lightweight trap 6. Enter Guest 7. Exit Guest QEMU KVM Guest OS
- 65. User space Kernel space Guest Mode 1. VM initialization 2. Return to QEMU 3. Run VM 4. Enter Guest 5. Exit Guest Lightweight trap 6. Enter Guest 7. Exit Guest 8. Return to QEMU QEMU KVM Guest OS
- 66. User space Kernel space Guest Mode 1. VM initialization 2. Return to QEMU 3. Run VM 4. Enter Guest 5. Exit Guest Lightweight trap 6. Enter Guest Heavyweight trap 7. Exit Guest 8. Return to QEMU QEMU KVM Guest OS
- 67. User space Kernel space Guest Mode 1. VM initialization 2. Return to QEMU 3. Run VM 4. Enter Guest 5. Exit Guest Lightweight trap 6. Enter Guest Heavyweight trap 7. Exit Guest 8. Return to QEMU 9. Run VM QEMU KVM Guest OS
- 68. User space Kernel space Guest Mode 1. VM initialization 2. Return to QEMU 3. Run VM 4. Enter Guest 5. Exit Guest Lightweight trap 6. Enter Guest Heavyweight trap 7. Exit Guest 8. Return to QEMU 9. Run VM 10. Enter Guest QEMU KVM Guest OS
- 69. VCPU oxffff2000 Register Sync File The KVM trap Interface oxffff1000 0xffff001c Kernel Vector oxffff0000
- 70. VCPU oxffff2000 Register Shadow Register File Sync File The KVM trap Interface oxffff1000 0xffff001c Kernel Vector oxffff0000
- 71. mcr cpsr, r1 VCPU oxffff2000 Register Shadow Register File Sync File The KVM trap Interface oxffff1000 0xffff001c Kernel Vector oxffff0000
- 72. mcr cpsr, r1 VCPU oxffff2000 Register Shadow Register File Sync File The KVM trap Interface oxffff1000 0xffff001c Kernel Vector oxffff0000
- 73. mcr cpsr, r1 VCPU oxffff2000 Read/Write Register Shadow Register File Sync Instructions File The KVM trap Interface oxffff1000 0xffff001c Kernel Vector oxffff0000
- 74. mcr cpsr, r1 VCPU oxffff2000 Read/Write Register Shadow Register File Sync Instructions File The KVM trap Interface oxffff1000 0xffff001c Kernel Vector oxffff0000
- 76. PTB Virtual Address Physical Address
- 77. GVA GPA HVA HPA
- 78. Guest PTB GVA GPA HVA HPA
- 79. Guest PTB GVA GPA HVA HPA
- 80. Guest PTB GVA GPA Host PTB HVA HPA
- 81. Guest PTB GVA GPA Host PTB HVA New SPTE !!! HPA
- 83. PABT/DABT trap
- 84. PABT/DABT trap guest page table walker
- 85. PABT/DABT trap guest page table walker True Translation fault
- 86. PABT/DABT trap guest Guest page table permission walker checker True Translation fault
- 87. PABT/DABT trap guest Guest page table permission walker checker True permission fault True Translation fault
- 88. PABT/DABT trap guest Guest MMIO access page table permission checker walker checker True permission fault True Translation fault
- 89. PABT/DABT trap guest Guest MMIO access page table permission checker walker checker True permission fault True Translation fault MMIO emulation
- 90. PABT/DABT trap guest Guest Shadow page MMIO access page table permission table checker walker checker mapping True permission fault True Translation fault MMIO emulation
- 91. Hidden protection fault PABT/DABT trap guest Guest Shadow page MMIO access page table permission table checker walker checker mapping True permission fault True Translation fault MMIO emulation
- 92. Hidden protection fault PABT/DABT trap guest Guest Shadow page MMIO access page table permission table checker walker checker mapping True permission fault Hidden translation fault True Translation fault MMIO emulation
- 93. Hidden protection fault PABT/DABT trap guest Guest Shadow page MMIO access Shadow page page table permission table checker table update walker checker mapping True permission fault Hidden translation fault True Translation fault MMIO emulation
- 95. I/O virtualization • Emulation by QEMU • virtio • IRQ chip in kernel
- 96. Emulate by QEMU
- 97. Guest OS: Linux 2.6.35 QEMU 0.14 Driver Device ARMvisor Driver Host OS: Linux 2.6.38 Hardware: ARM Cortex-A8
- 98. Guest OS: Linux 2.6.35 1 QEMU 0.14 Driver Device ARMvisor Driver Host OS: Linux 2.6.38 Hardware: ARM Cortex-A8
- 99. Guest OS: Linux 2.6.35 1 QEMU 0.14 Driver Device 2 ARMvisor Driver Host OS: Linux 2.6.38 Hardware: ARM Cortex-A8
- 100. Guest OS: Linux 2.6.35 1 QEMU 0.14 Driver Device 3 2 ARMvisor Driver Host OS: Linux 2.6.38 Hardware: ARM Cortex-A8
- 101. Guest OS: Linux 2.6.35 1 QEMU 0.14 Driver Device 3 2 ARMvisor Driver Host OS: Linux 2.6.38 4 Hardware: ARM Cortex-A8
- 102. Guest OS: Linux 2.6.35 QEMU 0.14 Driver Device ARMvisor Driver Host OS: Linux 2.6.38 Hardware: ARM Cortex-A8
- 103. Guest OS: Linux 2.6.35 QEMU 0.14 Driver Device ARMvisor Driver Host OS: Linux 2.6.38 5 Hardware: ARM Cortex-A8
- 104. Guest OS: Linux 2.6.35 QEMU 0.14 Driver Device 6 ARMvisor Driver Host OS: Linux 2.6.38 5 Hardware: ARM Cortex-A8
- 105. Guest OS: Linux 2.6.35 QEMU 0.14 Driver Device 7 6 ARMvisor Driver Host OS: Linux 2.6.38 5 Hardware: ARM Cortex-A8
- 106. Guest OS: Linux 2.6.35 8 QEMU 0.14 Driver Device 7 6 ARMvisor Driver Host OS: Linux 2.6.38 5 Hardware: ARM Cortex-A8
- 107. virtio
- 108. virtio Vir,o Driver Guest Vir,o AMBA Controller Vring Transport Vir,o AMBA Controller QEMU Vir,o Device
- 109. irq_chip in kernel
- 110. Opera,ng System Interrupt Controller Deliver Interrupt Get IRQ number 1 Ack IRQ number Mask IRQ 2 End of IRQ 3 Unmask IRQ 4
- 111. irq_chip in kernel Guest Deliver7 GIC QEMU IRQ7 Control7 GIC7 Device Device Device IOCTL7 ARMvisor
- 112. irq_chip in kernel Guest Deliver$ IRQ$ QEMU Device Device Device Control$ GIC$ GIC$in$Kernel KVM
- 113. Future Works
- 114. Future Works
- 115. Future Works • Support for ARM’s Virtualization extension
- 116. Future Works • Support for ARM’s Virtualization extension • Cortex-A15 and beyond
- 117. Future Works • Support for ARM’s Virtualization extension • Cortex-A15 and beyond • SMP for host and guest
- 118. Future Works • Support for ARM’s Virtualization extension • Cortex-A15 and beyond • SMP for host and guest • AArch64 Support
- 119. Future Works • Support for ARM’s Virtualization extension • Cortex-A15 and beyond • SMP for host and guest • AArch64 Support • ARMv8
- 120. By the way, ...
- 121. OpenSource
- 122. OpenSource • We HAVE opened source in late August, 2012.
- 123. OpenSource • We HAVE opened source in late August, 2012. • GNU GPLv2
- 124. OpenSource • We HAVE opened source in late August, 2012. • GNU GPLv2 • Source code of Host and Guest OS
- 125. OpenSource • We HAVE opened source in late August, 2012. • GNU GPLv2 • Source code of Host and Guest OS • https://github.com/SSLab-NTHU
- 126. Q &A