Virtunoid: Breaking out of KVM

Virtunoid: Breaking out of KVM

Nelson Elhage

DEFCON 19

August 8, 2011

Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 1 / 50

KVM

The new hotness for Virtualization on Linux
Oﬃcial virtualization platform for Ubuntu and RHEL.


Who am I?

Kernel engineer at Ksplice (now Oracle).
Open-source security hacker in my spare time.


Outline

1 KVM: Architecture overview
Attack Surface

2 CVE-2011-1751: The bug

3 virtunoid.c: The exploit
%rip control
Getting to shellcode
Bypassing ASLR

4 Conclusions and further research

5 Demo


KVM: Architecture overview

Attack Surface

2 CVE-2011-1751: The bug

%rip control
Bypassing ASLR


5 Demo



KVM: The components

kvm.ko
kvm-intel.ko / kvm-amd.ko
qemu-kvm



kvm.ko

The core KVM kernel module
Implements the virtual CPU and MMU (with the hardware’s help).
Emulates a few devices in-kernel for eﬃciency.
Provides ioctls for communicating with the kernel module.
Contains an emulator for a subset of x86 used in handling certain
traps (!)




Provides support for Intel’s VMX and AMD’s SVM virtualization
extensions.
Relatively small compared to the rest of KVM (one .c ﬁle each)



qemu-kvm

Provides the most direct user interface to KVM.
Based on the classic qemu emulator.
Implements the bulk of the virtual devices a VM uses.
Implements a wide variety of types of devices.
An order of magnitude more code than the kernel module.
There is work in progress to replace this component, but it’s a ways
out, if ever.


KVM: Architecture overview Attack Surface

kvm.ko

A tempting target – successful exploitation gets ring0 on the host
without further escalation.
Much less code than qemu-kvm, and much of that is dedicated to
interfacing with qemu-kvm, not the guest directly.
The x86 emulator is an interesting target.
A number of bugs have been discovered allowing privesc within the
guest.
A lot of tricky code that is not often exercised.
Not the target of this talk, but I have some ideas for future work.
Also, be on the lookout for privesc within either the host or guest.




Not much direct attack surface.
Largely straight-line code doing lots of low-level bit twiddling with the
hardware structures.
Lots of subtlety, possibly some more complex attacks.



qemu-kvm

A veritable goldmine of targets.
Hundreds of thousands of lines of device emulation code.
Emulated devices communicate directly with the guest via MMIO or
IO ports, lots of attack surface.
Much of the code comes straight from qemu and is ancient.
qemu-kvm is often sandboxed using SELinux or similar, meaning that
successful exploitation will often require a second privesc within the
host.
(Fortunately, Linux never has any of those)


CVE-2011-1751: The bug

Attack Surface

2 CVE-2011-1751: The bug

%rip control
Bypassing ASLR


5 Demo



RHSA-2011:0534-1

“It was found that the PIIX4 Power Management emulation layer in
qemu-kvm did not properly check for hot plug eligibility during device
removals. A privileged guest user could use this ﬂaw to crash the guest or,
possibly, execute arbitrary code on the host. (CVE-2011-1751)”



PIIX4

The PIIX4 was a southbridge chip used in many circa-2000 Intel
chipsets.
The default southbridge emulated by qemu-kvm
Includes ACPI support, a PCI-ISA bridge, an embedded MC146818
RTC, and much more.



Device Hotplug

The PIIX4 supports PCI hotplug, implemented by writing values to IO
port 0xae08.
qemu-kvm emulates this by calling qdev_free(qdev);, which calls a
device’s cleanup function and free()s it.
Many devices weren’t implemented with hotplug in mind!



The PCI-ISA bridge

In particular, it should not be possible to unplug the ISA bridge.
Among other things, the emulated MC146818 RTC hangs oﬀ the ISA
bridge.
KVM’s emulated RTC is not designed to be unplugged; In particular,
it leaves around dangling QEMUTimer objects when unplugged.



QEMUTimer

t y p e d e f v o i d QEMUTimerCB( v o i d * opaque ) ;

s t r u c t QEMUTimer {
...
i n t 6 4 t e x p i r e t i m e ; /* i n n a n o s e c o n d s */
QEMUTimerCB * cb ;
v o i d * opaque ;
s t r u c t QEMUTimer * n e x t ;
};

t y p e d e f s t r u c t RTCState {
...
QEMUTimer * s e c o n d t i m e r ;
...
} RTCState ;


Use-after-free

Unplugging the virtual RTC free()s the RTCState
It doesn’t free() or unregister either of the timers.
So we’re left with dangling pointers from the QEMUTimers
On the next second, we’ll call rtc_update_second(<freed
RTCState> )



Reproducer

#i n c l u d e <s y s / i o . h>

i n t main ( v o i d ) {
iopl (3);
o u t l (2 , 0 xae08 ) ;
return 0;
}


virtunoid.c: The exploit

Attack Surface

2 CVE-2011-1751: The bug

%rip control
Bypassing ASLR


5 Demo


virtunoid.c: The exploit %rip control

High-level TODO

1 Inject a controlled QEMUTimer into qemu-kvm at a known address
2 Eject the emulated ISA bridge
3 Force an allocation into the freed RTCState, with second_timer
pointing at our dummy timer.

When rtc_update_second next runs, our timer will get scheduled.
One second later, boom.



1. Injecting data

The guest’s RAM is backed by a simple mmap()ed region inside the
qemu-kvm process.
So we allocate an object in the guest, and compute
hva = physmem_base + gpa
gpa = (gva_to_gfn(gva) << PAGE_SHIFT)
+ page_offset(gva)
For now, assume we can guess physmem_base (e.g. no ASLR)

hva host virtual address
gva guest virtual address
gpa guest physical address
gfn guest frame (physical page) number



qemu-kvm userspace network stack

qemu-kvm contains a user-mode networking stack.
Implements a DHCP server, DNS server, and a gateway NAT.



Userspace network stack packet delivery

The user-mode stack normally handles packets synchronously.
To prevent recursion, if a second packet is emitted while handling a
ﬁrst packet, the second packet is queued, using malloc().



The virtual network gateway responds synchronously to ICMP ping.



Putting it together

1 Allocate a fake QEMUTimer
Point ->cb at the desired %rip.
2 Calculate its address in the host.
3 Write 2 to IO port 0xae08 to eject the ISA bridge.
4 ping the emulated gateway with ICMP packets containing pointers to
your allocated timer in the host.


virtunoid.c: The exploit Getting to shellcode

We’ve got %rip, now what?




Get EIP = 0x41414141 and declare victory.




Disable NX in my BIOS and call it good enough for a demo.




Do a ROP pivot, ROP to victory.




Do a ROP pivot, ROP to victory.
Do something else clever.



Another look at QEMUTimer

s t r u c t QEMUTimer {
...
i n t 6 4 t e x p i r e t i m e ; /* i n n a n o s e c o n d s */
...
s t r u c t QEMUTimer * n e x t ;
};



qemu_run_timers

s t a t i c v o i d q e m u r u n t i m e r s ( QEMUClock * c l o c k )
{
QEMUTimer ** p t i m e r h e a d , * t s ;
int64 t current time ;

current time = qemu get clock ns ( clock );
p t i m e r h e a d = &a c t i v e t i m e r s [ c l o c k −>t y p e ] ;
for ( ; ; ) {
ts = * ptimer head ;
i f ( ! qemu timer expired ns ( ts , c u r r e n t t i m e ))
break ;
* p t i m e r h e a d = t s −>n e x t ;
t s −>n e x t = NULL ;

t s −>cb ( t s −>opaque ) ;
}
}



Timer chains

second_timer

f1
−>cb
−>opaque X
−>next

f2
−>cb
−>opaque Y
−>next

f3
−>cb
−>opaque Z
−>next
NULL
⇒ f1(X); f2(Y); f3(Z);



More arguments

amd64 calling convention: %rdi, %rsi, %rdx, . . .
Every version of qemu_run_timers I’ve checked leaves %rsi
untouched.



More arguments

set rsi :
movl %r d i , %r s i
ret

Let f1 = set_rsi
f2(Y, X)
Same trick doesn’t work with %rdx.



set_rsi

v o i d c p u o u t l ( p i o a d d r t addr , u i n t 3 2 t v a l )
{
i o p o r t w r i t e ( 2 , addr , v a l ) ;
}



Getting to mprotect

int mprotect(const void *addr, size_t len, int prot);
#define PROT_EXEC 0x4

s t a t i c u i n t 3 2 t i o p o r t r e a d l t h u n k ( v o i d * opaque , u i n t 3 2 t a d d r )
{
IORange * i o p o r t = opaque ;
u i n t 6 4 t data ;

i o p o r t −>ops−>r e a d ( i o p o r t , a d d r − i o p o r t −>b a s e , 4 , &d a t a ) ;
r e t u r n data ;
}



Putting it together

Allocate a fake IORangeOps, with fake_ops->read = mprotect.
Allocate a page-aligned IORange, with ->ops = fake_ops and
->base = -PAGE_SIZE.
Copy shellcode immediately following the IORange.
Construct a timer chain that calls
cpu_outl(0, *)
ioport_readl_thunk(fake_ioport, 0)
fake_ioport + 1



Why not ROP?

Continued execution is dead simple.
Reduced dependence on details of compiled code.
I’m not that good at ROP :)


virtunoid.c: The exploit Bypassing ASLR

Addresses

For a known qemu-kvm binary, we need two addresses.
The base address of the qemu-kvm binary, to ﬁnd code addresses.
physmem_base, the address of the physical memory mapping inside
qemu-kvm.



Option A

Find an information leak.



Option B

Assume non-PIE, and be clever.



fw_cfg

Emulated IO ports 0x510 (address) and 0x511 (data)
Used to communicate various tables to the qemu BIOS (e820 map,
ACPI tables, etc)
Also provides support for exporting writable tables to the BIOS.
However, fw_cfg_write doesn’t check if the target table is supposed
to be writable!



Static data

Several fw_cfg areas are backed by statically-allocated buﬀers.
Net result: nearly 500 writable bytes inside static variables.



read4 your way to victory

mprotect needs a page-aligned address, so these aren’t suitable for
our shellcode.
But, we can construct fake timer chains in this space to build a
read4() primitive.
Follow pointers from static variables to ﬁnd physmem_base
Proceed as before.



Repeated timer chaining

Previously, we ended timer chains with ->next = NULL.
Instead, end them with a timer that calls rtc_update_second.
The timer we control will be scheduled once a second, and we can
change ->cb at any time.
Now we can execute a read4, update structures based on the result,
and then hijack the list again.


Conclusions and further research

Conclusions

VM breakouts aren’t magic.
Hypervisors are just as vulnerable as anything else.
Device drivers are the weak spot.



Comparing with some past breakouts

2008 “Adventures with a certain Xen vulnerability”, Xen, Invisible Things
Lab
2009 “Cloudburst”, Immunity, VMware
2011 “Software attacks against Intel VT-d technology”, Invisible Things
Lab, Xen



Possible hardening directions

Sandbox qemu-kvm (work underway well before this talk).
Build qemu-kvm as PIE.
Lazily mmap/mprotect guest RAM?
XOR-encode key function pointers?
More auditing and fuzzing of qemu-kvm.



Future research directions

Fuzzing/auditing kvm.ko (That x86 emulator sketches me)
Fingerprinting qemu-kvm versions
Searching for infoleaks (Rosenbugs?)


Demo

It’s demo time


Demo

Questions?

nelhage@nelhage.com
http://blog.nelhage.com
@nelhage


Virtunoid: Breaking out of KVM

More Related Content

What's hot (20)

Similar to Virtunoid: Breaking out of KVM (20)

Recently uploaded (20)

Virtunoid: Breaking out of KVM