Slide used at ACM-SAC 2014 by Suzaki
3 likes1,924 views
This document describes a protocol test generator that uses nested virtual machines and rollback mechanisms to perform exhaustive fuzz testing of protocol implementations. It proposes using a virtual test protocol to encapsulate test packets and control the target virtual machine. Special packets allow taking snapshots of the target VM state and rolling back to previous snapshots to repeatedly test protocol states with different fuzzed packets. The current prototype implements this approach with KVM and QEMU virtual machines to find bugs in TLS/SSL protocol implementations through fuzz testing of the handshake process.
![National Institute of Advanced Industrial Science and Technology
Motivation 1/2Motivation 1/2
• Sec re protocols (SSL/TLS SSH etc) are important for• Secure protocols (SSL/TLS, SSH, etc) are important for
Internet commerce.
l bili i f i l i f• However, vulnerabilities of implementation of secure
protocols are discovered repeatedly.
– Incidents of SSL/TLS vulnerabilities
• Insufficient entropy of random numbers [2012]
– It creates same secret-keys for SSL/TLS certificates.It creates same secret keys for SSL/TLS certificates.
– EFF offers a site to check SSL certificates.
» https://www.eff.org/observatory
• CRIME[2012]• CRIME[2012]
– It attacks to the data compression used by the TLS and SPDY.
• Lucky Thirteen[2013]
– It attacks padding oracle using timing side-channel.](https://image.slidesharecdn.com/acm-sac2014-suzaki-140501200238-phpapp02/85/Slide-used-at-ACM-SAC-2014-by-Suzaki-2-320.jpg)
![National Institute of Advanced Industrial Science and Technology
Motivation 2/2Motivation 2/2
• The implementations of secure protocols should be verified.
• Our group increases TLS security with 2 approaches.
F l ifi i– Formal verification
• Verify important parts of protocol hander program.
– “Formal Network Packet Processing with Minimal Fuss: Invertible Syntax Descriptions atFormal Network Packet Processing with Minimal Fuss: Invertible Syntax Descriptions at
Work.” [PLPV’12]
– “Towards Formal Verification of TLS Network Packet Processing Written in C”
[PLPV’13]
– Exhaustive Protocol Fuzz Testing
• Try to check all reaction of protocol handler.
– Protocol Test Generator creates and sends many fuzzed packets to check implementation
of protocol handler.
– (Today’s Topic) Computing environment for protocol fuzz testing](https://image.slidesharecdn.com/acm-sac2014-suzaki-140501200238-phpapp02/85/Slide-used-at-ACM-SAC-2014-by-Suzaki-3-320.jpg)
![National Institute of Advanced Industrial Science and Technology
Sequence of Protocol Test Generator (TLS)Sequence of Protocol Test Generator (TLS)
Normal Fuzz for Client Hello
TargetProtocol Generator TargetProtocol Generator
Client Hello
Take Snapshot
[Fuzz1] Client Hello
Rollback Snapshot1
Client Hello
Server Hello
Server Certificate
Snapshot1
[Fuzz1] Client Hello
????
Good or Bad?
Rollback1
Server Hello Done
Take Snapshot Rollback Snapshot1
Good or Bad?
Take Snapshot
Client Key Exchange
… Snapshot 2
[Fuzz2] Client Hello
????
Rollback 1
Good or Bad?
[FuzzN] Client Hello](https://image.slidesharecdn.com/acm-sac2014-suzaki-140501200238-phpapp02/85/Slide-used-at-ACM-SAC-2014-by-Suzaki-7-320.jpg)
![National Institute of Advanced Industrial Science and Technology
Sequence of Protocol Test Generator (TLS)Sequence of Protocol Test Generator (TLS)
Normal Fuzz for Client Hello Fuzz for Client Key Exchange
TargetProtocol Generator TargetProtocol Generator TargetProtocol Generator
Client Hello
Take Snapshot
[Fuzz1] Client Hello
Rollback Snapshot1
[Fuzz1] Client Key
Rollback Snapshot2
Client Hello
Server Hello
Server Certificate
[Fuzz1] Client Hello
????
Good or Bad?
Snapshot1 Rollback
1
[Fuzz1] Client Key
Exchange
???
Rollback2
Server Hello Done
Take Snapshot Rollback Snapshot1
Good or Bad?
Rollback Snapshot2
Good or Bad?
Take Snapshot
Client Key Exchange
…
[Fuzz2] Client Hello
????
Snapshot 2 Rollback 1 Rollback 2
[Fuzz2] Client Key
Exchange
???
[FuzzN] Client Hello
Good or Bad?
???
Good or Bad?
[FuzzN] Client Key
Exchange](https://image.slidesharecdn.com/acm-sac2014-suzaki-140501200238-phpapp02/85/Slide-used-at-ACM-SAC-2014-by-Suzaki-8-320.jpg)
![National Institute of Advanced Industrial Science and Technology
Design Issues for VM (Computing environment)Design Issues for VM (Computing environment)
• Virtual Machine runs a protocol handler.
• En-capsulated packets by VTP must be de-capsulated on a target VM.
– Original packets must be passed to a target protocol handler on the VM.
• The control packets (take_snapshot and rollback) must be dealt by the VM.
• The VTP packets on the network must be maintained.
– We have to deal with consistent global state defied by [Chandy-Lamport, 1985].
Protocol
Generator
Target Environment
Snapshots of VM
VTP
Protocol
handler
TLS
VTP
VM deals with VTP protocol.
Packets on the network
must be maintained.
VM](https://image.slidesharecdn.com/acm-sac2014-suzaki-140501200238-phpapp02/85/Slide-used-at-ACM-SAC-2014-by-Suzaki-11-320.jpg)
![National Institute of Advanced Industrial Science and Technology
R l t d kRelated works
• Nested VM• Nested VM
– Migration
• Xen-Blanket [EuroSys’12] enables user level migration on CloudXen Blanket [EuroSys 12] enables user level migration on Cloud.
– Security
• CloudVisor [SOSP’11] is inserted on existing VM on multi-tenant
Cloud.
• Process level snapshot
– DMTCP: Distributed Multi Threaded Check Pointing
• http://dmtcp.sourceforge.net/
If d ’t t d t b t h d l l t bilit f• If we don’t not need to care about hardware-level repeatability for
rollback, DMTCP will be a candidate for protocol test generator.](https://image.slidesharecdn.com/acm-sac2014-suzaki-140501200238-phpapp02/85/Slide-used-at-ACM-SAC-2014-by-Suzaki-21-320.jpg)
Slide used at ACM-SAC 2014 by Suzaki
- 1. National Institute of Advanced Industrial Science and Technology Rollback MechanismRollback Mechanism of Nested Virtual Machines for Protocol Fuzz Testing Kuniyasu Suzaki*, Toshiki Yagi*, Akira Tanaka*, Y taka Oi a* Ets a Shiba ama*‡Yutaka Oiwa , Etsuya Shibayama ‡ * National Institute of Advanced Industrial Science and Technology(AIST)National Institute of Advanced Industrial Science and Technology(AIST) ‡The University of Tokyo ACM SAC(Symposium On Applied Computing)2014 at Gyeongju, South Korea March 25
- 2. National Institute of Advanced Industrial Science and Technology Motivation 1/2Motivation 1/2 • Sec re protocols (SSL/TLS SSH etc) are important for• Secure protocols (SSL/TLS, SSH, etc) are important for Internet commerce. l bili i f i l i f• However, vulnerabilities of implementation of secure protocols are discovered repeatedly. – Incidents of SSL/TLS vulnerabilities • Insufficient entropy of random numbers [2012] – It creates same secret-keys for SSL/TLS certificates.It creates same secret keys for SSL/TLS certificates. – EFF offers a site to check SSL certificates. » https://www.eff.org/observatory • CRIME[2012]• CRIME[2012] – It attacks to the data compression used by the TLS and SPDY. • Lucky Thirteen[2013] – It attacks padding oracle using timing side-channel.
- 3. National Institute of Advanced Industrial Science and Technology Motivation 2/2Motivation 2/2 • The implementations of secure protocols should be verified. • Our group increases TLS security with 2 approaches. F l ifi i– Formal verification • Verify important parts of protocol hander program. – “Formal Network Packet Processing with Minimal Fuss: Invertible Syntax Descriptions atFormal Network Packet Processing with Minimal Fuss: Invertible Syntax Descriptions at Work.” [PLPV’12] – “Towards Formal Verification of TLS Network Packet Processing Written in C” [PLPV’13] – Exhaustive Protocol Fuzz Testing • Try to check all reaction of protocol handler. – Protocol Test Generator creates and sends many fuzzed packets to check implementation of protocol handler. – (Today’s Topic) Computing environment for protocol fuzz testing
- 4. National Institute of Advanced Industrial Science and Technology O liOutline M i i• Motivation • Protocol Test Generator: fuzz testing on a protocol handler • Why we use nested VM for protocol fuzz testing? • Design issuesDesign issues • Implementation P f l i• Performance evaluation on current prototype. • Conclusion
- 5. National Institute of Advanced Industrial Science and Technology Protocol Test Generator V if t l h d f h t l t t• Verify a protocol hander for each protocol state – Repeat the following sequence for each protocol state 1. Creates a fuzzed packet 2. Send it to a protocol hander 3. Check the reply • Fuzzed packets are tested on each protocol stateFuzzed packets are tested on each protocol state many times. • Snapshot and rollback mechanism is required.S p q
- 6. National Institute of Advanced Industrial Science and Technology Sequence of Protocol Test Generator (TLS)Sequence of Protocol Test Generator (TLS) Normal TargetProtocol Generator Client Hello Take Snapshot Client Hello Server Hello Server Certificate Snapshot1 Server Hello Done Take SnapshotTake Snapshot Client Key Exchange … Snapshot 2
- 7. National Institute of Advanced Industrial Science and Technology Sequence of Protocol Test Generator (TLS)Sequence of Protocol Test Generator (TLS) Normal Fuzz for Client Hello TargetProtocol Generator TargetProtocol Generator Client Hello Take Snapshot [Fuzz1] Client Hello Rollback Snapshot1 Client Hello Server Hello Server Certificate Snapshot1 [Fuzz1] Client Hello ???? Good or Bad? Rollback1 Server Hello Done Take Snapshot Rollback Snapshot1 Good or Bad? Take Snapshot Client Key Exchange … Snapshot 2 [Fuzz2] Client Hello ???? Rollback 1 Good or Bad? [FuzzN] Client Hello
- 8. National Institute of Advanced Industrial Science and Technology Sequence of Protocol Test Generator (TLS)Sequence of Protocol Test Generator (TLS) Normal Fuzz for Client Hello Fuzz for Client Key Exchange TargetProtocol Generator TargetProtocol Generator TargetProtocol Generator Client Hello Take Snapshot [Fuzz1] Client Hello Rollback Snapshot1 [Fuzz1] Client Key Rollback Snapshot2 Client Hello Server Hello Server Certificate [Fuzz1] Client Hello ???? Good or Bad? Snapshot1 Rollback 1 [Fuzz1] Client Key Exchange ??? Rollback2 Server Hello Done Take Snapshot Rollback Snapshot1 Good or Bad? Rollback Snapshot2 Good or Bad? Take Snapshot Client Key Exchange … [Fuzz2] Client Hello ???? Snapshot 2 Rollback 1 Rollback 2 [Fuzz2] Client Key Exchange ??? [FuzzN] Client Hello Good or Bad? ??? Good or Bad? [FuzzN] Client Key Exchange
- 9. National Institute of Advanced Industrial Science and Technology Requirement for Protocol Test GeneratorRequirement for Protocol Test Generator 1. Packet level granularity control – Generator wants to take a snapshot just before sending a packet. 2. Hardware-level repeatability B l biliti d b it ti i t (– Because some vulnerabilities are caused by its computing environment (e.g., random number) 3. Consistency between generator and target protocol hander3. Consistency between generator and target protocol hander – Most snapshot/rollback mechanism does not care of packets on a wire. 1. We propose a special protocol to control target computing environment. 2. We uses Virtual Machine which offers hardware-level repeatability f h / llb kfor snapshot/rollback. 3. Design issues. (Mention in later slides)
- 10. National Institute of Advanced Industrial Science and Technology Proposed Protocol (VTP: Virtual Test Protocol)Proposed Protocol (VTP: Virtual Test Protocol) • VTP treats packets of test-target protocol as data. T t t t k t l t d t “C l ” k t– Test-target packets are en-capsulated to “Capsule” packets – “Capsule” packets must be de-capsulated on target computing environment. • VTP has special packets to control target computing environment• VTP has special packets to control target computing environment – “Take_Snapshot” and “Rollback” Protocol Action Capsule Packets of secure communication are encapsulated with capsule protocol. The flow of capsulated packets is managed h t k h t llb k k t i i dwhen take_snapshot or rollback packet is issued. Take_ Snapshot Take VM snapshot image. Packets on the wire must be managed by a certain method. Return snapshot ID. R h i C i f Rollback (with ID) Resume snapshot image. Connection of secure communication must be operative, when a previous image is resumed. Target computing environment (VM) has to deal with the VTP.
- 11. National Institute of Advanced Industrial Science and Technology Design Issues for VM (Computing environment)Design Issues for VM (Computing environment) • Virtual Machine runs a protocol handler. • En-capsulated packets by VTP must be de-capsulated on a target VM. – Original packets must be passed to a target protocol handler on the VM. • The control packets (take_snapshot and rollback) must be dealt by the VM. • The VTP packets on the network must be maintained. – We have to deal with consistent global state defied by [Chandy-Lamport, 1985]. Protocol Generator Target Environment Snapshots of VM VTP Protocol handler TLS VTP VM deals with VTP protocol. Packets on the network must be maintained. VM
- 12. National Institute of Advanced Industrial Science and Technology Design choice for handling VTPDesign choice for handling VTP • Customize VM’s network function for VTPCustomize VM s network function for VTP – We planned to customize SLiRP of QEMU • However VM control (snapshot/rollback) from SLiRP is strange, because it( p ) g , is a part of VM. It is not flexible. • Nested VM and Proxies (external & internal) – External proxy manages VTP control packets and takes snapshot of External VM. – A snapshot of External VM keeps connectivity of network between internal proxy and a protocol handler on Internal VM. • Pros: Easy and flexible implementation. » It utilizes existing VM control commands. Don’t need to change VM. • Cons: Overhead? Performance is improved by many previous researchesCons: Overhead? Performance is improved by many previous researches.
- 13. National Institute of Advanced Industrial Science and Technology Design: Nested VM and Proxiesg • External VM (a unit of snapshot) – It is used to take snapshot of Internal VM and Internal Proxy. • Internal VM (hardware repeatability)( p y) – It includes protocol hander and keeps connection with Internal Proxy, when External VM is taken a snapshot. • External Proxy – Connects to Protocol Generator directly, and controls snapshot/rollback of External VM. Real Machine External VMControl the target VM • Internal Proxy – Maintain connectivity of VTP Protocol when a snapshot image is resumed. Real Machine External VM Internal VM Control the target VM (SAVEVM, LOADVM) Decapsulate control commands Internal Ｐｒｏｘｙ openSSL Port forward External Ｐｒｏｘｙ External Ｐｒｏｘｙ Internal ＰｒｏｘｙTLS VTP TLSTLS VTP External VM Internal VM External VM Internal VM Protocol Generator “take_snapshot” packet make a round‐trip between external and internal proxy to confirm no VTP packets on the wire 13 Internal VM SSH openSSL Keep connectionSnapshots no VTP packets on the wire. Rollback TLS
- 14. National Institute of Advanced Industrial Science and Technology Handling VTP “capsule” packetHandling VTP capsule packet • Protocol Generator en-casuplates test-target packets with “capsule” packets. • External Proxy passes capsule packets to Internal Proxy without change. • Internal Proxy de-capsulates VTP packets and passes original packets to protocol hander on Internal VMhander on Internal VM. Real Machine External VMReal Machine External VM Internal VM Decapsulate control commands Internal Ｐｒｏｘｙ openSSL Port forward External Ｐｒｏｘｙ External Ｐｒｏｘｙ Internal ＰｒｏｘｙTLS VTP TLSTLS VTP Protocol Generator
- 15. National Institute of Advanced Industrial Science and Technology Handling VTP “take snapshot” packetHandling VTP take_snapshot packet • When a “take_snapshot” packet is issued, the packet makes a round trip between External and Internal Proxy to confirm that there is no VTP packet on the wire (consistent global state defied by Chandy Lamport)(consistent global state defied by Chandy-Lamport). • After that, External Proxy sends SAVEM command to External VM to take a snapshot. • Connection between Internal Proxy and protocol handler is kept in a snapshot. Real Machine External VMControl the target VMReal Machine External VM Internal VM g (SAVEVM) Decapsulate control commands Internal Ｐｒｏｘｙ openSSL Port forward External Ｐｒｏｘｙ External Ｐｒｏｘｙ Internal Ｐｒｏｘｙ P t l TLS VTP TLSTLS VTP External VM Internal VM External VM Internal VM Protocol Generator “take_snapshot” packet make a round‐trip between external and internal proxy to confirm no VTP packets on the wire. 15 Internal VM SSH openSSL Keep connectionSnapshots no VTP packets on the wire. TLS
- 16. National Institute of Advanced Industrial Science and Technology Handling VTP “rollback” packetHandling VTP rollback packet • When a “rollback” packet is issued, the External Proxy send LOADVM command to External VM It resumes an old imageLOADVM command to External VM. It resumes an old image. • After that, External and Internal proxies re-establish the connection between them, because they loses the network states when rollback., y – Internal proxy and protocol handler do not need to care about network connection. Real Machine External VMControl the target VMReal Machine External VM Internal VM g (LOADVM) Decapsulate control commands Internal Ｐｒｏｘｙ openSSL Port forward External Ｐｒｏｘｙ External Ｐｒｏｘｙ Internal Ｐｒｏｘｙ P t l TLS VTP TLSReestablish External VM Internal VM External VM Internal VM Protocol Generator 16 Internal VM SSH openSSL Keep connection Rollback TLS
- 17. National Institute of Advanced Industrial Science and Technology OptimizationOptimization • Most overhead of VM is caused by video emulation. – Nested VMs must emulate the GUI of internal VM on external VM again.g • Fortunately QEMU and KVM have “curses mode” which emulates text user interface. • We eliminate GUI on both internal and external VMs. N GUI d d k h t i k– No GUI reduces memory usage and make snapshot quick. GUIs of Nested VMs No GUI of Nested VMs OpenSSL on Linux on Internal VM (QEMU) on External VM (KVM) ith d Internal VM (QEMU) OpenSSL on Windows Extenal Proxy Protocol Tester with curses mode Protocol Tester External Proxy External VM (KVM) Internal Proxy on Linux
- 18. National Institute of Advanced Industrial Science and Technology Current ImplementationCurrent Implementation • We implement Nested VM and proxies on normalWe implement Nested VM and proxies on normal Debian GNU Linux x86. – External VM: KVMExternal VM: KVM • The VM is a unit of snapshot and is NOT required hardware repeatability. • KVM is fast with virtualization technology of X86. – Internal VM: QEMU h l h d f bili ( d b• The VM emulates hardware for repeatability (e.g., random number generator). – Proxies are written by PerlProxies are written by Perl. • External Proxy: 430 LOC • Internal Proxy: 132 LOC• Internal Proxy: 132 LOC
- 19. National Institute of Advanced Industrial Science and Technology Performance evaluationPerformance evaluation • We evaluated performance on Protocol Test Generator. – Test for hand shake of TLS (PolarSSL) takes 9 snapshots and 2,311 rollbacks. – Effect of GUI and noGUI. – Effct of Memory size (large 1024MB/512MB small 512MB/256MB)– Effct of Memory size (large 1024MB/512MB, small 512MB/256MB) GUI Mem: NoGUI Mem: NoGUI Mem: 1024/512 (sec) 1024/512 (sec) 512/256 (sec) Setting up nested VMs 266 107 93g p Fuzz packet generator 1,307 1,164 1,080 Nested VMs and Proxies Snapshot (9 times) 57 (Av 6.33) 37 (Av 4.11) 24 (Av 2.67) R llb k 3 135 2 197 1 286 VM overhead isRollback (2,311times) 3,135 (Av 1.36) 2,197 (Av 0.96) 1,286 (Av 0.56) Other 12 13 12 Total 5 043 3 622 2 587 VM overhead is almost same to packet generator. Total 5,043 3,622 2,587 ThinkPad T410 (CPU Intel Core i7-M620 2.67Ghz, Memory 4GB).
- 20. National Institute of Advanced Industrial Science and Technology R lt f P t l F T tiResults of Protocol Fuzz Testing W ifi d th h d h k f 4 TLS• We verified the handshake of 4 TLS servers. – OpenSSL, GnuTLS, CyaSSL, and PolarSSL. • The fuzz testing found 2 bugs in CyaSSL– 2 bugs in CyaSSL – 1 bug in PolarSSL • We reported the results to the mailing list, and the bugs were fixed.g
- 21. National Institute of Advanced Industrial Science and Technology R l t d kRelated works • Nested VM• Nested VM – Migration • Xen-Blanket [EuroSys’12] enables user level migration on CloudXen Blanket [EuroSys 12] enables user level migration on Cloud. – Security • CloudVisor [SOSP’11] is inserted on existing VM on multi-tenant Cloud. • Process level snapshot – DMTCP: Distributed Multi Threaded Check Pointing • http://dmtcp.sourceforge.net/ If d ’t t d t b t h d l l t bilit f• If we don’t not need to care about hardware-level repeatability for rollback, DMTCP will be a candidate for protocol test generator.
- 22. National Institute of Advanced Industrial Science and Technology Conclusion • We propose a combination of nested VM and proxies for protocol fuzz testing environment. – It treats hardware-level repeatability (e.g., random number). – It is easy and flexible to implement. • Our method is not limited to protocol testing. It can be applied on other areas (e.g., I/O fuzz testing, memorypp ( g , g, y fuzz testing). Acknowledgement This work is supported by the National Institute of Information and Communications Technology of Japan.