MCA Daemon: Hybrid Throughput Analysis Beyond Basic Blocks
0 likes489 views
The document discusses the MCA Daemon (MCAD) and its role in hybrid throughput analysis, focusing on assured micro patching and the analysis of binary patches' timing impacts. It presents challenges and methodologies for execution time assessment, leveraging static and dynamic analysis to enhance performance analysis capabilities in embedded systems. The presentation also highlights future plans for improving the MCAD and its integration with tools like LLVM MCA.
![Introduction to MCA
An example
56
vmulps %xmm0, %xmm1, %xmm2
vhaddps %xmm2, %xmm2, %xmm3
vhaddps %xmm3, %xmm3, %xmm4
test/tools/llvm-mca/X86/BtVer2/dot-product.s Timeline
012345
Index 0123456789
[0,0] DeeER. . . vmulps
[0,1] D==eeeER . . vhaddps
[0,2] .D====eeeER . vhaddps
[1,0] .DeeE-----R . vmulps
[1,1] . D=eeeE---R . vhaddps
[1,2] . D====eeeER . vhaddps
[2,0] . DeeE-----R . vmulps
[2,1] . D====eeeER . vhaddps
[2,2] . D======eeeER vhaddps
llvm-mca -mtriple=x86_64 -mcpu=btver2
-iterations=300 dot-products.s](https://image.slidesharecdn.com/eurollvm2022-mcad-220512190346-9dea4672/85/MCA-Daemon-Hybrid-Throughput-Analysis-Beyond-Basic-Blocks-167-320.jpg)
![Introduction to MCA
An example
56
vmulps %xmm0, %xmm1, %xmm2
vhaddps %xmm2, %xmm2, %xmm3
vhaddps %xmm3, %xmm3, %xmm4
test/tools/llvm-mca/X86/BtVer2/dot-product.s Timeline
012345
Index 0123456789
[0,0] DeeER. . . vmulps
[0,1] D==eeeER . . vhaddps
[0,2] .D====eeeER . vhaddps
[1,0] .DeeE-----R . vmulps
[1,1] . D=eeeE---R . vhaddps
[1,2] . D====eeeER . vhaddps
[2,0] . DeeE-----R . vmulps
[2,1] . D====eeeER . vhaddps
[2,2] . D======eeeER vhaddps
llvm-mca -mtriple=x86_64 -mcpu=btver2
-iterations=300 dot-products.s](https://image.slidesharecdn.com/eurollvm2022-mcad-220512190346-9dea4672/85/MCA-Daemon-Hybrid-Throughput-Analysis-Beyond-Basic-Blocks-168-320.jpg)
MCA Daemon: Hybrid Throughput Analysis Beyond Basic Blocks
- 1. Min-Yih “Min” Hsu, David Gens, Michael Franz. University of California, Irvine MCA Daemon Hybrid Throughput Analysis Beyond Basic Blocks Keynote, EuroLLVM 2022
- 2. Outline 2
- 5. Outline Motivation MCA Daemon (MCAD) Future Plans 2
- 6. Outline Motivation MCA Daemon (MCAD) Future Plans Epilogue 2
- 7. Outline Motivation MCA Daemon (MCAD) Future Plans Epilogue 3
- 8. Genesis: Assured Micro Patching (AMP) 4
- 9. Genesis: Assured Micro Patching (AMP) • A research project initiated by United States DARPA to assure the correctness of binary patching with little or no source code 4
- 10. Genesis: Assured Micro Patching (AMP) • A research project initiated by United States DARPA to assure the correctness of binary patching with little or no source code • Including functional and timing aspects 4
- 11. Genesis: Assured Micro Patching (AMP) • A research project initiated by United States DARPA to assure the correctness of binary patching with little or no source code • Including functional and timing aspects • Focuses on small (micro) binary patches 4
- 12. Genesis: Assured Micro Patching (AMP) • A research project initiated by United States DARPA to assure the correctness of binary patching with little or no source code • Including functional and timing aspects • Focuses on small (micro) binary patches • Focuses on embedded systems 4
- 13. Genesis: Assured Micro Patching (AMP) • A research project initiated by United States DARPA to assure the correctness of binary patching with little or no source code • Including functional and timing aspects • Focuses on small (micro) binary patches • Focuses on embedded systems • UCI was studying the timing impacts of binary patches 4
- 14. Genesis: Assured Micro Patching (AMP) • A research project initiated by United States DARPA to assure the correctness of binary patching with little or no source code • Including functional and timing aspects • Focuses on small (micro) binary patches • Focuses on embedded systems • UCI was studying the timing impacts of binary patches • Example: After the fi rmware on a truck is binary-patched to prevent brakes from locking up, we need to make sure latencies do not degrade terribly 4
- 15. Timing impacts of binary patches Problem de fi nition 5 Original Program
- 16. Timing impacts of binary patches Problem de fi nition 5 Original Program Small Binary Patch Patched Program
- 17. Timing impacts of binary patches Problem de fi nition 5 Original Program Small Binary Patch Patched Program Original Program Same set of inputs
- 18. Timing impacts of binary patches Problem de fi nition 5 Original Program Small Binary Patch Patched Program Original Program ΔT? Same set of inputs
- 19. Execution time assessment 6 Original Program Small Binary Patch Patched Program Original Program ΔT? Same set of inputs
- 20. Execution time assessment Interesting use cases 7
- 21. Execution time assessment Interesting use cases • Predicting program run time in remote environments or time-sensitive applications • Examples: fi rmware in cars or satellite (e.g. Kepler space telescope by NASA) 7
- 22. Execution time assessment Interesting use cases • Predicting program run time in remote environments or time-sensitive applications • Examples: fi rmware in cars or satellite (e.g. Kepler space telescope by NASA) • Performance analysis • Insights into performance bottlenecks 7
- 23. Execution time assessment Interesting use cases • Predicting program run time in remote environments or time-sensitive applications • Examples: fi rmware in cars or satellite (e.g. Kepler space telescope by NASA) • Performance analysis • Insights into performance bottlenecks • Examples: Potential CPU pipeline stalling, GPU memory bank con fl icts 7
- 24. Execution time assessment Previous e ff orts 8
- 25. Execution time assessment Previous e ff orts • Static approaches 8
- 26. Execution time assessment Previous e ff orts • Static approaches • Throughput analysis: predicting the cycle counts for linear code (e.g. basic block, loop) statically 8
- 27. Execution time assessment Previous e ff orts • Static approaches • Throughput analysis: predicting the cycle counts for linear code (e.g. basic block, loop) statically • Examples: IACA, OSACA, uiCA, LLVM MCA, Ithemal 8
- 28. Execution time assessment Previous e ff orts • Static approaches • Throughput analysis: predicting the cycle counts for linear code (e.g. basic block, loop) statically • Examples: IACA, OSACA, uiCA, LLVM MCA, Ithemal • Dynamic approaches • Cycle-accurate simulators / emulators 8
- 29. Execution time assessment Previous e ff orts • Static approaches • Throughput analysis: predicting the cycle counts for linear code (e.g. basic block, loop) statically • Examples: IACA, OSACA, uiCA, LLVM MCA, Ithemal • Dynamic approaches • Cycle-accurate simulators / emulators • Examples: gem5, gpgpu-sim 8
- 30. Execution time assessment Challenges 9 Static Dynamic
- 31. Execution time assessment Challenges 9 Static Dynamic High Low Precision
- 32. Execution time assessment Challenges 9 Static Dynamic High Low Precision • Complete execution traces • Higher fi delity on hardware details
- 33. Execution time assessment Challenges 9 Static Dynamic High Low Precision • Poor handling on branches & function calls • Small scope (only few blocks) • Lack of run-time information • Complete execution traces • Higher fi delity on hardware details
- 34. Execution time assessment Challenges 9 Static Dynamic High Low Precision Fast Slow Turnaround • Poor handling on branches & function calls • Small scope (only few blocks) • Lack of run-time information • Complete execution traces • Higher fi delity on hardware details
- 35. Execution time assessment Challenges 9 Static Dynamic High Low Precision Fast Slow Turnaround • Poor handling on branches & function calls • Small scope (only few blocks) • Lack of run-time information • Complete execution traces • Higher fi delity on hardware details • Faster analysis speed (due to coarser granularity) • Easier integration with other tools
- 36. Execution time assessment Challenges 9 Static Dynamic High Low Precision Fast Slow Turnaround • Poor handling on branches & function calls • Small scope (only few blocks) • Lack of run-time information • Complete execution traces • Higher fi delity on hardware details • Faster analysis speed (due to coarser granularity) • Easier integration with other tools • Usually require non-trivial setup • Slow simulation speed
- 37. Execution time assessment Challenges 9 Static Dynamic High Low Precision Fast Slow Turnaround ? • Poor handling on branches & function calls • Small scope (only few blocks) • Lack of run-time information • Complete execution traces • Higher fi delity on hardware details • Faster analysis speed (due to coarser granularity) • Easier integration with other tools • Usually require non-trivial setup • Slow simulation speed
- 38. Outline Motivation MCA Daemon (MCAD) Future Plans Epilogue 10
- 39. MCA Daemon (MCAD) High-level concept 11 Dynamic Runtime Static Throughput Analysis Tool Target Program
- 40. MCA Daemon (MCAD) High-level concept 11 Dynamic Runtime Static Throughput Analysis Tool Target Program Execution Trace
- 41. MCA Daemon (MCAD) High-level concept 11 Dynamic Runtime Static Throughput Analysis Tool Target Program Execution Trace • The instructions that just got executed • Run-time values (e.g. register values)
- 42. MCA Daemon (MCAD) High-level concept 11 Dynamic Runtime Static Throughput Analysis Tool Target Program Execution Trace Online Environment Process 1 Process 2 • The instructions that just got executed • Run-time values (e.g. register values)
- 43. MCA Daemon (MCAD) High-level concept 11 Dynamic Runtime Static Throughput Analysis Tool Target Program Execution Trace Online Environment Process 1 Process 2 Streaming • The instructions that just got executed • Run-time values (e.g. register values)
- 44. MCA Daemon (MCAD) High-level concept 12 QEMU LLVM MCA Libraries Target Program Online Environment Process 1 Process 2 Execution Trace Streaming
- 45. Introduction to LLVM MCA 13
- 46. Introduction to LLVM MCA • A tool (llvm-mca) and library (libLLVMMCA) for predicting cycle counts and potential performance hazards in a sequence of assembly code 13
- 47. Introduction to LLVM MCA • A tool (llvm-mca) and library (libLLVMMCA) for predicting cycle counts and potential performance hazards in a sequence of assembly code • Using instruction scheduling data (e.g. instruction latency) provided by each LLVM target • New ISA (with proper scheduling info) can be supported out of the box 13
- 48. Introduction to LLVM MCA • A tool (llvm-mca) and library (libLLVMMCA) for predicting cycle counts and potential performance hazards in a sequence of assembly code • Using instruction scheduling data (e.g. instruction latency) provided by each LLVM target • New ISA (with proper scheduling info) can be supported out of the box • Accounting for modern processor features: super scalar, out-of-order etc. 13
- 49. Introduction to LLVM MCA • A tool (llvm-mca) and library (libLLVMMCA) for predicting cycle counts and potential performance hazards in a sequence of assembly code • Using instruction scheduling data (e.g. instruction latency) provided by each LLVM target • New ISA (with proper scheduling info) can be supported out of the box • Accounting for modern processor features: super scalar, out-of-order etc. • Implemented via lightweight simulation • Abstract real CPU pipeline stages into a small handful of stages 13
- 50. Introduction to MCA An example 14 vmulps %xmm0, %xmm1, %xmm2 vhaddps %xmm2, %xmm2, %xmm3 vhaddps %xmm3, %xmm3, %xmm4 test/tools/llvm-mca/X86/BtVer2/dot-product.s
- 51. Introduction to MCA An example 14 vmulps %xmm0, %xmm1, %xmm2 vhaddps %xmm2, %xmm2, %xmm3 vhaddps %xmm3, %xmm3, %xmm4 test/tools/llvm-mca/X86/BtVer2/dot-product.s llvm-mca -mtriple=x86_64 -mcpu=btver2 -iterations=300 dot-products.s
- 52. Introduction to MCA An example 14 vmulps %xmm0, %xmm1, %xmm2 vhaddps %xmm2, %xmm2, %xmm3 vhaddps %xmm3, %xmm3, %xmm4 test/tools/llvm-mca/X86/BtVer2/dot-product.s Summary Iterations: 300 Instructions: 900 Total Cycles: 610 Total uOps: 900 Dispatch Width: 2 uOps Per Cycle: 1.48 IPC: 1.48 Block RThroughput: 2.0 llvm-mca -mtriple=x86_64 -mcpu=btver2 -iterations=300 dot-products.s
- 53. 15 llvm-mca MCAD
- 55. 15 QEMU LLVM MCA Libraries Target Program Process 1 Process 2 Execution Trace Streaming llvm-mca Assembly fi le LLVM MCA Libraries llvm-mca MCAD
- 57. MCA Daemon (MCAD) Highlights • Combine the advantages of dynamic & static throughput analysis 16
- 58. MCA Daemon (MCAD) Highlights • Combine the advantages of dynamic & static throughput analysis • Augment the analysis region beyond basic blocks • MCAD is able to analyze the entire program execution trace 16
- 59. MCA Daemon (MCAD) Highlights • Combine the advantages of dynamic & static throughput analysis • Augment the analysis region beyond basic blocks • MCAD is able to analyze the entire program execution trace • Throughput analysis is happening in parallel / on-the-fly with the target program execution 16
- 61. Analyze execution traces using MCA Using unmodi fi ed MCA libraries 18 QEMU Target Program Executed instructions LLVM MCA Libraries Disassembler
- 62. Analyze execution traces using MCA Challenge: Sequential work fl ow 19 QEMU Target Program Blocked until QEMU is fi nished Executed instructions LLVM MCA Libraries Disassembler
- 67. MCA internal 20 MCInst mca::Instruction Stage 0 Stage 1 Stage N …. MCA Simulation Pipeline mca::SourceMgr Assembly fi le
- 68. MCA internal 20 MCInst mca::Instruction Stage 0 Stage 1 Stage N …. MCA Simulation Pipeline Display mca::SourceMgr Assembly fi le
- 69. MCA with execution trace stream as input 21 MCInst mca::Instruction Stage 0 Stage 1 Stage N …. MCA Simulation Pipeline Display mca::SourceMgr Execution Trace Stream
- 70. MCA with execution trace stream as input 21 MCInst mca::Instruction Stage 0 Stage 1 Stage N …. MCA Simulation Pipeline Display mca::SourceMgr Execution Trace Stream Blocking
- 71. Incremental SourceMgr 22 MCInst mca::Instruction Stage 0 Stage 1 Stage N …. MCA Simulation Pipeline Display mca::IncrementalSourceMgr Execution Trace Stream
- 72. Incremental SourceMgr 22 MCInst mca::Instruction Stage 0 Stage 1 Stage N …. MCA Simulation Pipeline Display Making mca::Instruction available to simulation pipeline right away mca::IncrementalSourceMgr Execution Trace Stream
- 73. Incremental SourceMgr 23 MCInst mca::Instruction Stage 0 Stage 1 Stage N …. MCA Simulation Pipeline mca::IncrementalSourceMgr Execution Trace Stream QEMU
- 74. Incremental SourceMgr 23 MCInst mca::Instruction Stage 0 Stage 1 Stage N …. MCA Simulation Pipeline mca::IncrementalSourceMgr Execution Trace Stream QEMU Process 2 Process 1
- 75. Incremental SourceMgr Implement with threads 24 MCInst mca::Instruction Stage 0 Stage 1 Stage N …. MCA Simulation Pipeline mca::IncrementalSourceMgr Execution Trace Stream QEMU Process 1
- 76. Incremental SourceMgr Implement with threads 24 MCInst mca::Instruction Stage 0 Stage 1 Stage N …. MCA Simulation Pipeline mca::IncrementalSourceMgr Execution Trace Stream QEMU Process 1 Simulation Thread
- 77. Incremental SourceMgr Implement with threads 24 MCInst mca::Instruction Stage 0 Stage 1 Stage N …. MCA Simulation Pipeline mca::IncrementalSourceMgr Execution Trace Stream QEMU Process 1 Simulation Thread mca::Instruction fetching loop
- 78. Incremental SourceMgr Implement with threads 24 MCInst mca::Instruction Stage 0 Stage 1 Stage N …. MCA Simulation Pipeline mca::IncrementalSourceMgr Execution Trace Stream QEMU Process 1 Simulation Thread Receiver Thread mca::Instruction fetching loop
- 79. Incremental SourceMgr Implement with threads: Pros & Cons 25 MCInst mca::Instruction Stage 0 Stage 1 Stage N …. MCA Simulation Pipeline mca::IncrementalSourceMgr Execution Trace Stream QEMU Process 1 Simulation Thread Receiver Thread
- 80. Incremental SourceMgr Implement with threads: Pros & Cons 25 MCInst mca::Instruction Stage 0 Stage 1 Stage N …. MCA Simulation Pipeline mca::IncrementalSourceMgr Execution Trace Stream QEMU Process 1 Simulation Thread Receiver Thread Pros: No modi fi cation on the simulation pipeline
- 81. Incremental SourceMgr Implement with threads: Pros & Cons 25 MCInst mca::Instruction Stage 0 Stage 1 Stage N …. MCA Simulation Pipeline mca::IncrementalSourceMgr Execution Trace Stream QEMU Process 1 Simulation Thread Receiver Thread Pros: No modi fi cation on the simulation pipeline Cons: To use IncrementalSourceMgr, you have to use threads
- 82. Incremental SourceMgr Better solution: Resumable simulation pipeline 26 MCInst mca::Instruction Stage 0 Stage 1 Stage N …. Resumable Simulation Pipeline mca::IncrementalSourceMgr Execution Trace Stream QEMU
- 83. Incremental SourceMgr Better solution: Resumable simulation pipeline 26 MCInst mca::Instruction Stage 0 Stage 1 Stage N …. Resumable Simulation Pipeline mca::IncrementalSourceMgr Execution Trace Stream QEMU A subset of trace
- 84. Incremental SourceMgr Better solution: Resumable simulation pipeline 26 MCInst mca::Instruction Stage 0 Stage 1 Stage N …. Resumable Simulation Pipeline mca::IncrementalSourceMgr Execution Trace Stream QEMU A subset of trace A subset of mca::Instruction
- 85. Incremental SourceMgr Better solution: Resumable simulation pipeline 27 MCInst mca::Instruction Stage 0 Stage 1 Stage N …. Resumable Simulation Pipeline mca::IncrementalSourceMgr Execution Trace Stream QEMU A subset of mca::Instruction
- 86. Incremental SourceMgr Better solution: Resumable simulation pipeline 28 MCInst mca::Instruction Stage 0 Stage 1 Stage N …. Resumable Simulation Pipeline mca::IncrementalSourceMgr Execution Trace Stream QEMU Pause
- 87. Incremental SourceMgr Better solution: Resumable simulation pipeline 28 MCInst mca::Instruction Stage 0 Stage 1 Stage N …. Resumable Simulation Pipeline mca::IncrementalSourceMgr Execution Trace Stream QEMU Pause
- 89. Resumable simulation pipeline • Save (and restore) the analysis state from previous subset of instructions 29
- 90. Resumable simulation pipeline • Save (and restore) the analysis state from previous subset of instructions • Threads are not required when using IncrementalSourceMgr + resumable pipeline 29
- 91. Resumable simulation pipeline • Save (and restore) the analysis state from previous subset of instructions • Threads are not required when using IncrementalSourceMgr + resumable pipeline • Much easier to integrate into other uses 29
- 92. Resumable simulation pipeline • Save (and restore) the analysis state from previous subset of instructions • Threads are not required when using IncrementalSourceMgr + resumable pipeline • Much easier to integrate into other uses • You can still wrap resumable pipeline with a thread 29
- 93. Resumable simulation pipeline • Save (and restore) the analysis state from previous subset of instructions • Threads are not required when using IncrementalSourceMgr + resumable pipeline • Much easier to integrate into other uses • You can still wrap resumable pipeline with a thread • Minor downside: Modi fi cations on the simulation pipeline 29
- 94. Incremental SourceMgr + Resumable pipeline Put into real actions 30 MCInst mca::Instruction Stage 0 Stage 1 Stage N …. Resumable Simulation Pipeline mca::IncrementalSourceMgr Execution Trace Stream QEMU
- 95. Incremental SourceMgr + Resumable pipeline Put into real actions 30 MCInst mca::Instruction Stage 0 Stage 1 Stage N …. Resumable Simulation Pipeline mca::IncrementalSourceMgr Execution Trace Stream QEMU • A program that continuously reads from an I/O device
- 96. Incremental SourceMgr + Resumable pipeline Put into real actions 30 MCInst mca::Instruction Stage 0 Stage 1 Stage N …. Resumable Simulation Pipeline mca::IncrementalSourceMgr Execution Trace Stream QEMU • A program that continuously reads from an I/O device • Only record the user space traces
- 97. Incremental SourceMgr + Resumable pipeline Put into real actions 30 MCInst mca::Instruction Stage 0 Stage 1 Stage N …. Resumable Simulation Pipeline mca::IncrementalSourceMgr Execution Trace Stream QEMU • A program that continuously reads from an I/O device • Only record the user space traces • Collected ~1 million instructions
- 98. Challenge 31
- 99. Challenge • Created a signi fi cant amount of memory footprint 31
- 100. Challenge • Created a signi fi cant amount of memory footprint 31 (Unit: MB)
- 101. Challenge • Created a signi fi cant amount of memory footprint • Bottleneck: ~37GB of accumulated (virtual) memory was allocated by mca::InstrBuilder::createInstruction 31 (Unit: MB)
- 102. Challenge • Created a signi fi cant amount of memory footprint • Bottleneck: ~37GB of accumulated (virtual) memory was allocated by mca::InstrBuilder::createInstruction 31 (Unit: MB)
- 103. Challenge • Created a signi fi cant amount of memory footprint • Bottleneck: ~37GB of accumulated (virtual) memory was allocated by mca::InstrBuilder::createInstruction 31 (Unit: MB)
- 104. Challenge • Created a signi fi cant amount of memory footprint • Bottleneck: ~37GB of accumulated (virtual) memory was allocated by mca::InstrBuilder::createInstruction 31 MCInst mca::Instruction mca::InstrBuilder (Unit: MB)
- 105. Large memory footprint Root cause 32
- 106. Large memory footprint Root cause • Most of the translated mca::Instruction objects are never deallocated until the simulation is fi nished 32
- 107. Large memory footprint Root cause • Most of the translated mca::Instruction objects are never deallocated until the simulation is fi nished • mca::Instruction is also used for tracking simulation state, so it’s hard to make it immutable 32
- 108. Large memory footprint Root cause • Most of the translated mca::Instruction objects are never deallocated until the simulation is fi nished • mca::Instruction is also used for tracking simulation state, so it’s hard to make it immutable • Doesn’t scale really well with large input (recall: ~1 million instructions) 32
- 109. Large memory footprint Observation 33 mca::IncrementalSourceMgr mca::Instruction Resumable Simulation Pipeline
- 110. Large memory footprint Observation 33 mca::IncrementalSourceMgr mca::Instruction Resumable Simulation Pipeline Stream direction
- 111. Large memory footprint Observation 33 mca::IncrementalSourceMgr mca::Instruction Resumable Simulation Pipeline Copy Stream direction
- 112. Large memory footprint Solution: Recycling mca::Instruction 34 mca::IncrementalSourceMgr mca::Instruction Resumable Simulation Pipeline Copy Stream direction
- 113. Large memory footprint Solution: Recycling mca::Instruction 34 mca::IncrementalSourceMgr mca::Instruction Resumable Simulation Pipeline Copy Stream direction
- 114. Large memory footprint Solution: Recycling mca::Instruction 34 mca::IncrementalSourceMgr mca::Instruction Resumable Simulation Pipeline Copy Stream direction Recycle mca::InstrBuilder
- 115. 67% improvement on accumulated memory consumption 35
- 116. ~70% of the mca::Instruction objects are recycled 36
- 117. Collecting execution traces via QEMU 37 user mode qemu Target Program QEMU MCAD
- 118. Collecting execution traces via QEMU 37 user mode qemu Target Program Custom TCG Plugin Instrument QEMU MCAD
- 119. Broker Collecting execution traces via QEMU 37 user mode qemu Target Program Custom TCG Plugin Instrument Receiver TCP Socket QEMU MCAD
- 120. Broker Collecting execution traces via QEMU 37 user mode qemu Target Program Custom TCG Plugin Instrument Disassembler Receiver TCP Socket MCInst QEMU MCAD
- 121. Custom QEMU TCG plugin 38
- 122. Custom QEMU TCG plugin • The plugin interface allows us to tap into various TCG events to collect executed instructions • Example: When a TCG block is translated / executed 38
- 123. Custom QEMU TCG plugin • The plugin interface allows us to tap into various TCG events to collect executed instructions • Example: When a TCG block is translated / executed • We also added a few plugin interfaces (not upstreamed yet) • Example: Retrieving CPU register values 38
- 124. Custom QEMU TCG plugin • The plugin interface allows us to tap into various TCG events to collect executed instructions • Example: When a TCG block is translated / executed • We also added a few plugin interfaces (not upstreamed yet) • Example: Retrieving CPU register values • Sending raw binary instructions* through TCP sockets 38
- 125. QEMU Complete structure of MCAD 39 TCG Plugin MCAD Core IncrementalSourceMgr Resumable Simulation Pipeline Display (QEMU) Broker Disassembler Receiver Target Program TCP Sockets Recycling InstrBuilder
- 126. QEMU Complete structure of MCAD 39 TCG Plugin MCAD Core IncrementalSourceMgr Resumable Simulation Pipeline Display (QEMU) Broker Disassembler Receiver Target Program TCP Sockets ☹ Recycling InstrBuilder
- 127. QEMU Complete structure of MCAD A modular design 40 TCG Plugin Display (QEMU) Broker Disassembler Receiver Target Program TCP Sockets MCAD Core IncrementalSourceMgr Resumable Simulation Pipeline Recycling InstrBuilder
- 128. QEMU Complete structure of MCAD A modular design 40 TCG Plugin Display (QEMU) Broker Disassembler Receiver Target Program TCP Sockets Loadable Plugin MCAD Core IncrementalSourceMgr Resumable Simulation Pipeline Recycling InstrBuilder
- 129. Loadable Plugin Complete structure of MCAD Example: Assembly broker plugin 41 Display Assembly Broker AsmParser Assembly File MCAD Core IncrementalSourceMgr Resumable Simulation Pipeline Recycling InstrBuilder
- 130. Evaluation Scalability compared against llvm-mca 42
- 131. Evaluation Scalability compared against llvm-mca • Compare MCAD against llvm-mca (i.e. baseline) on analysis speed and memory consumption 42
- 132. Evaluation Scalability compared against llvm-mca • Compare MCAD against llvm-mca (i.e. baseline) on analysis speed and memory consumption • Using execution trace collected from running x86_64 FFmpeg 4.2 to decode a 14KB MPEG-4 video fi le • Command: ffmpeg -i input.mp4 -f null - • Size of the trace: ~27 million x86_64 instructions 42
- 133. Evaluation Scalability compared against llvm-mca • Compare MCAD against llvm-mca (i.e. baseline) on analysis speed and memory consumption • Using execution trace collected from running x86_64 FFmpeg 4.2 to decode a 14KB MPEG-4 video fi le • Command: ffmpeg -i input.mp4 -f null - • Size of the trace: ~27 million x86_64 instructions • For baseline, we dump the execution trace (assembly instructions) to a fi le before feeding into llvm-mca • Time measurement on baseline only accounts for llvm-mca’s run time. Excluding the trace collection time. 42
- 134. Evaluation Scalability compared against llvm-mca 43
- 135. Evaluation Scalability compared against llvm-mca 43 Analysis time seconds 0 44 88 132 176 220 llvm-mca MCAD
- 136. Evaluation Scalability compared against llvm-mca 43 Analysis time seconds 0 44 88 132 176 220 llvm-mca MCAD 4x Faster
- 137. Evaluation Scalability compared against llvm-mca 43 Analysis time seconds 0 44 88 132 176 220 llvm-mca MCAD Max resident memory Gigabytes 0 5 10 15 20 25 30 4x Faster
- 138. Evaluation Scalability compared against llvm-mca 43 Analysis time seconds 0 44 88 132 176 220 llvm-mca MCAD Max resident memory Gigabytes 0 5 10 15 20 25 30 4x Faster 13x Less
- 139. Evaluation Scalability compared against other static throughput analysis tools 44 Analysis Time Max Resident Memory uiCA Timeout after 48h 113 GB OSACA Exit w/ error after 24h N/A Ithemal Exit w/ error after 2m N/A MCAD 52.69s 2.16 GB
- 140. Outline Motivation MCA Daemon (MCAD) Future Plans Epilogue 45
- 141. // TODO 46
- 142. // TODO • More e ffi cient ways to collect traces without QEMU 46
- 143. // TODO • More e ffi cient ways to collect traces without QEMU • Analyzing traces from multi-thread programs 46
- 144. // TODO • More e ffi cient ways to collect traces without QEMU • Analyzing traces from multi-thread programs • Improve MCA’s precision via dynamic information (e.g. memory accesses) 46
- 145. // TODO • More e ffi cient ways to collect traces without QEMU • Analyzing traces from multi-thread programs • Improve MCA’s precision via dynamic information (e.g. memory accesses) • Visualizing analysis results. Or: Improve MCA’s result display 46
- 146. // TODO • More e ffi cient ways to collect traces without QEMU • Analyzing traces from multi-thread programs • Improve MCA’s precision via dynamic information (e.g. memory accesses) • Visualizing analysis results. Or: Improve MCA’s result display • Example: Loadable plugins for custom display of the result 46
- 147. // TODO • More e ffi cient ways to collect traces without QEMU • Analyzing traces from multi-thread programs • Improve MCA’s precision via dynamic information (e.g. memory accesses) • Visualizing analysis results. Or: Improve MCA’s result display • Example: Loadable plugins for custom display of the result • Going upstream: QEMU & LLVM 46
- 148. Going upstream: LLVM The plan 47
- 149. Going upstream: LLVM The plan • We would like to upstream components that are bene fi cial to the core MCA libraries fi rst 47
- 150. Going upstream: LLVM The plan: components to upstream 48 QEMU TCG Plugin MCAD Core IncrementalSourceMgr Resumable Simulation Pipeline (QEMU) Broker Disassembler Receiver Target Program TCP Sockets Recycling InstrBuilder Display* Not intended to upstream for now Intended to upstream Unrelated / no change
- 151. Going upstream: LLVM The plan • We would like to upstream components that are bene fi cial to the core MCA libraries fi rst 49
- 152. Going upstream: LLVM The plan • We would like to upstream components that are bene fi cial to the core MCA libraries fi rst • QEMU broker plugin & our TCG plugin will be maintained out-of-tree 49
- 153. Going upstream: LLVM The plan • We would like to upstream components that are bene fi cial to the core MCA libraries fi rst • QEMU broker plugin & our TCG plugin will be maintained out-of-tree • We’re not sure about upstreaming rest of the tool right now 49
- 154. Going upstream: LLVM The plan • We would like to upstream components that are bene fi cial to the core MCA libraries fi rst • QEMU broker plugin & our TCG plugin will be maintained out-of-tree • We’re not sure about upstreaming rest of the tool right now • With the assembly broker, MCAD can be a drop-in replacement for llvm- mca…with even more features (e.g. the broker plugin infrastructure) 49
- 155. Going upstream: LLVM The plan • We would like to upstream components that are bene fi cial to the core MCA libraries fi rst • QEMU broker plugin & our TCG plugin will be maintained out-of-tree • We’re not sure about upstreaming rest of the tool right now • With the assembly broker, MCAD can be a drop-in replacement for llvm- mca…with even more features (e.g. the broker plugin infrastructure) • Some of the (advanced) interfaces in broker plugin are only used by QEMU broker. So, without the latter, it’s not well tested. 49
- 156. Outline Motivation MCA Daemon (MCAD) Future Plans Epilogue 50
- 157. Summary 51
- 158. Summary • MCA Daemon (MCAD) is a high-performance hybrid throughput analysis tool built on top of LLVM MCA libraries 51
- 159. Summary • MCA Daemon (MCAD) is a high-performance hybrid throughput analysis tool built on top of LLVM MCA libraries • Online, whole-program analysis on real-world applications 51
- 160. Summary • MCA Daemon (MCAD) is a high-performance hybrid throughput analysis tool built on top of LLVM MCA libraries • Online, whole-program analysis on real-world applications • Scale up with large-scale programs with tens of millions of instructions 51
- 161. Summary • MCA Daemon (MCAD) is a high-performance hybrid throughput analysis tool built on top of LLVM MCA libraries • Online, whole-program analysis on real-world applications • Scale up with large-scale programs with tens of millions of instructions • We improved the performance & fl exibility of core MCA libraries 51
- 162. Summary • MCA Daemon (MCAD) is a high-performance hybrid throughput analysis tool built on top of LLVM MCA libraries • Online, whole-program analysis on real-world applications • Scale up with large-scale programs with tens of millions of instructions • We improved the performance & fl exibility of core MCA libraries • We would like to merge these changes upstream to bene fi t the wider community 51
- 164. Acknowledgements 53 This material is based upon work partially supported by the Defense Advanced Research Projects Agency (DARPA) under contract N66001-20-C-4027 Special Thanks: Galois Inc. (https://galois.com) Immunant Inc. (https://immunant.com)
- 165. Thank You! 54 Q&A Aldrich Park @ UC Irvine, California Me
- 166. Appendix 55
- 167. Introduction to MCA An example 56 vmulps %xmm0, %xmm1, %xmm2 vhaddps %xmm2, %xmm2, %xmm3 vhaddps %xmm3, %xmm3, %xmm4 test/tools/llvm-mca/X86/BtVer2/dot-product.s Timeline 012345 Index 0123456789 [0,0] DeeER. . . vmulps [0,1] D==eeeER . . vhaddps [0,2] .D====eeeER . vhaddps [1,0] .DeeE-----R . vmulps [1,1] . D=eeeE---R . vhaddps [1,2] . D====eeeER . vhaddps [2,0] . DeeE-----R . vmulps [2,1] . D====eeeER . vhaddps [2,2] . D======eeeER vhaddps llvm-mca -mtriple=x86_64 -mcpu=btver2 -iterations=300 dot-products.s
- 168. Introduction to MCA An example 56 vmulps %xmm0, %xmm1, %xmm2 vhaddps %xmm2, %xmm2, %xmm3 vhaddps %xmm3, %xmm3, %xmm4 test/tools/llvm-mca/X86/BtVer2/dot-product.s Timeline 012345 Index 0123456789 [0,0] DeeER. . . vmulps [0,1] D==eeeER . . vhaddps [0,2] .D====eeeER . vhaddps [1,0] .DeeE-----R . vmulps [1,1] . D=eeeE---R . vhaddps [1,2] . D====eeeER . vhaddps [2,0] . DeeE-----R . vmulps [2,1] . D====eeeER . vhaddps [2,2] . D======eeeER vhaddps llvm-mca -mtriple=x86_64 -mcpu=btver2 -iterations=300 dot-products.s