Instruction Level Parallelism (ILP) Limitations
5 likes9,461 views
This document discusses instruction-level parallelism (ILP) limitations. It covers ILP background using a MIPS example, hardware models that were studied including register renaming and branch/jump prediction assumptions. A study of ILP limitations found diminishing returns with larger window sizes and realizable processors are limited by complexity and power constraints. Simultaneous multithreading was explored as a technique to improve ILP but has its own design challenges. Today, x86 and ARM processors employ various ILP optimizations within pipeline constraints.
Instruction Level Parallelism (ILP) Limitations
- 1. Instruction-Level Parallelism Limitations EECE528: Parallel and Reconfigurable Computing Jose P. Pinilla
- 2. CONTENT I. ILP Background II. Hardware Model III. Study of Limitations IV. Simultaneous Multithreading V. ILP today
- 3. CONTENT I. ILP Background II. Hardware Model III. Study of Limitations IV. Simultaneous Multithreading V. ILP today
- 4. I. ILP • MIPS Example – Hazards • Structural • Data • Control • Power5 • ILP Optimizations – Register Renaming – Branch/Jump Prediction – Alias Analysis
- 5. Time (clock cycles) I$ Load Instr 1 Instr 2 Instr 3 Instr 4 I$ Reg D$ Reg ALU I$ Reg D$ Reg ALU I$ Reg D$ Reg ALU Reg D$ Reg ALU I$ Reg D$ Reg ALU I. ILP: MIPS
- 6. I. ILP: Hazards 1. Structural 2. Data 3. Control
- 7. I. ILP: Structural Hazards Conflict over the use of resources Time (clock cycles) I$ Load Instr 1 Instr 2 Instr 3 Instr 4 I$ Reg D$ Reg ALU I$ Reg D$ Reg ALU I$ Reg D$ Reg ALU Reg D$ Reg ALU I$ Reg D$ Reg ALU
- 8. I. ILP: Structural Hazards Conflict over the use of resources Time (clock cycles) I$ Load Instr 1 Instr 2 Instr 3 Instr 4 I$ Reg D$ Reg ALU I$ Reg D$ Reg ALU I$ Reg D$ Reg ALU Reg D$ Reg ALU I$ Reg D$ Reg ALU
- 9. I. ILP: Structural Hazards Time (clock cycles) I$ Load Instr 1 Instr 2 Instr 3 Instr 4 I$ Reg D$ Reg ALU I$ Reg D$ Reg ALU I$ Reg D$ Reg ALU Reg D$ Reg ALU I$ Reg D$ Reg ALU Solutions R/W: *On same clock cycle On different R/W ports
- 10. I. ILP: Data Hazards Time (clock cycles) I$ Instr 1 Instr 2 Instr 3 Instr 4 Instr 5 I$ Reg D$ Reg ALU I$ Reg D$ Reg ALU I$ Reg D$ Reg ALU Reg D$ Reg ALU I$ Reg D$ Reg ALU
- 11. I. ILP: Data Hazards add $t0, $t1, $t2 sub $t4, $t0 ,$t3 and $t5, $t0 ,$t6 or $t7, $t0 ,$t8 xor $t9, $t0 ,$t10
- 12. I. ILP: Data Hazards
- 13. I. ILP: Data Hazards Forwarding
- 14. I. ILP: Data Hazards Forwarding
- 15. I. ILP: Data Hazards Hardware Interlock Allows Forwarding
- 16. I. ILP: Data Hazards Stalling by compiler
- 17. I. ILP: Data Hazards Stalling by compiler The compiler could schedule a better use of that cycle. Hardware can also do it.
- 18. I. ILP: Data Hazards Avoid Stalling
- 19. I. ILP: Control Hazards
- 20. I. ILP: Control Hazards Solutions: Add HW to be able to compute branch on stage 2 (DECODE) Predict Branch: To simplify hardware, predict branch as NOT TAKEN most of the times. End of the loop will always be wrong, but then is just once Insert instruction after branch, always gets executed. Compiler. MIPS
- 21. I. ILP: Power5 Architecture 16 different stages
- 22. I. ILP: Optimizations Instruction window: Trace of incoming instructions to analyze for execution. Register Renaming: On false data dependences, hardware can rename the register. Compilers should optimize this false dependences: R2R memory model. Branch Prediction: Static: Always not taken, always taken. Forward/Backward taken. Branch delay slot. Dynamic: One-level (1bit, 2bit...), Two-level and Multiple Component Jump Prediction: Static profiling. Dynamic: Last taken, 2bit tables, return stack. Alias Analysis: Indirect memory references. Instruction Inspection.
- 23. I. ILP: Branch Prediction Saturated counter: Increment on branch taken, decrement on not taken. No Over or Under flow. Branch correlation: Inter/Intra Two-level: Remembers the history of the last n occurrences of the branch and uses one saturating counter for each of the possible 2n history patterns. Many more...
- 24. CONTENT I. ILP Background II. Hardware Model III. Study of Limitations IV. Simultaneous Multithreading V. ILP today
- 25. II. HARDWARE MODEL • Profiling Framework – Assumptions – Window Size
- 26. II. HW MODEL: Profiling Framework A set of assumptions and a methodology to, experimentally, extract a parallelism profile out of a set of benchmarks. Program is executed completely, resulting in a trace of instructions. Trace includes data addresses referenced, and the results of branches and jumps. (D. Wall's 1993 study) Divides the trace in cycles of 64 instructions in flight. The only limits on ILP in such a processor are those imposed by the actual data flows through either registers or memory.
- 27. II. HW MODEL: Assumptions • No limits on replicated functional units or ports to registers or memory. • Register Renaming: Perfect, Infinite, Finite, None • Branch Prediction: Perfect, Infinite, Finite, None • Jump Prediction: Perfect, Infinite, Finite, None • Memory Address Alias Analysis: • Perfect Caches • Unit cycle • 2k Window size
- 28. II. HW MODEL: Register Renaming • Perfect: Infinite number of registers to avoid false register dependencies. • Finite: Normally 256 integer registers and 256 floating point registers used in LRU (Least Recently Used) fashion. • No renaming: Number of registers used in the code.
- 29. II. HW MODEL: Branch Prediction • Perfect: All branches are correctly predicted. • 2bit predictor with infinite tables: Dynamic. A 2bit counter per branch option (2). Indexed by low-order bits of branch's address. Incremented on branch taken. Does not overflow. Branch is taken if table entry is 2 or 3. Up to 512 2bit entries. • 2bit predictor with infinite tables: Infinite number of counters. • Tournament-based branch predictor: 2 2bit counters competing. A 2bit selector that is decremented/incremented according to the correct prediction of the table entries. • Profile based: Static predictions. • No prediction: Every branch is predicted wrong. Not in order of performance
- 30. II. HW MODEL: Jump Prediction • Direct Jumps are known. • Indirect jumps – Perfect: Always performed correctly. – Finite prediction: A table with destination addresses. The address of a jump provides the index of the table. Whenever a jump is executed, we put its address in the table. Next jump should be to address in the table. – Infinite prediction: Infinite table entries. • No prediction: Every jump is predicted wrong.
- 31. II. HW MODEL: Alias Analysis • If two memory references do not refer to the same address, then they may be safely interchanged. • Indirect memory references are previous to the instruction execution. • No need to predict the actual values, only whether those values conflict. • Perfect: All global and stack reference predictions are perfect, heap • Inspection: Examine base and offset • None: All indirect memory references conflict.
- 32. II. HW MODEL: Window Size • The set of instructions which is examined for simultaneous execution. • The cycle width limits the number of instructions which can be scheduled. • A window size of 2k will look at 2048 instructions. • Cycle width: Assume we have found 111 instructions which can be parallelized. A cycle width of 64 would limit actual parallelism to 64 in flight instructions.
- 33. II. HW MODEL ctr: counter gsh: gshared (global history)
- 34. CONTENT I. ILP Background II. Hardware Model III. Study of Limitations IV. Simultaneous Multithreading V. ILP today
- 35. III. STUDY OF LIMITATIONS • Effects of... – Register Renaming – Branch/Jump Prediction – Alias Analysis – Realizable processor • Window Size (Discrete/Continuous) • Results
- 37. III. LIMITATIONS: Register Renaming
- 38. III. LIMITATIONS: Branch Prediction
- 39. III. LIMITATIONS: Branch Prediction
- 40. III. LIMITATIONS: Alias Analysis
- 42. III. LIMITATIONS: Realizable Processor • Up to 64 instruction issues per clock with no issue restrictions, or roughly 10 times the total issue width of the widest processor in 2011 • A tournament predictor with 1K entries and a 16-entry return predictor. This predictor is comparable to the best predictors in 2011; the predictor is not a primary bottleneck • Perfect disambiguation of memory references done dynamically—this is ambitious but perhaps attainable for small window sizes (and hence small issue rates and load-store buffers) or through address aliasing prediction • Register renaming with 64 additional integer and 64 additional FP registers, which is slightly less than the most aggressive processor in 2011 • No issue restrictions, no cache misses, unit latencies • Variable Window Size (Power5 200, Intel Core i7 ~128)
- 43. III. LIMITATIONS: Realizable Processor
- 44. III. LIMITATIONS: Conclusions • Plateau behavior • Window size effect on integer programs (3 top) is not as severe. Due to loop-level parallelism. • Designers are faced with the challenge: – Simpler processors with larger caches and higher clock rates Vs – ILP with slower clock and smaller caches • Persistent limitations: – WAW and WAR hazards through memory – Unnecessary dependences – Data flow limit
- 45. CONTENT I. ILP Background II. Hardware Model III. Study of Limitations IV. Simultaneous Multithreading V. ILP today
- 46. IV. SIMULTANEOUS MULTITHREADING • TLP Background – TLP approaches – Design Challenges • Limits of Multiple-Issue Processors – Power – Complexity
- 47. IV. SMT: TLP Background • Largely independent – Separate copy of regFile, PC and page table • Thread could represent – A process that is part of a parallel program consisting of multiple processes – An independent program on its own • Thread level parallelism occurs naturally • It can be used to employ the functional units idle when ILP is insufficient
- 48. IV. SMT: TLP Approaches
- 49. IV. SMT: Changes • Increasing the associativity of the L1 instruction cache and the instruction address translation buffers • Adding per-thread load and store queues • Increasing the size of the L2 and L3 caches • Adding separate instruction prefetch and buffering • Increasing the number of virtual registers from 152 to 240 • Increasing the size of several issue queues
- 50. IV. SMT: Results
- 51. IV. SMT: Results • SMT reduces energy by 7% • “Because of the costs and diminishing returns in performance, however, rather than implement wider superscalars and more aggressive versions of SMT, many designers are opting to implement multiple CPU cores on a single die with slightly less aggressive support for multiple issue and multithreading; we return to this topic in the next chapter.” - Hennessy et al.
- 52. CONTENT I. ILP Background II. Hardware Model III. Study of Limitations IV. Simultaneous Multithreading V. ILP today
- 53. V. ILP TODAY: x86 • Instruction fetch—The processor uses a multilevel branch target buffer to achieve a balance between speed and prediction accuracy. There is also a return address stack to speed up function return. Mispredictions cause a penalty of about 17 cycles. Using the predicted address, the instruction fetch unit fetches 16 bytes from the instruction cache. • Micro-code and Macro-code • Total pipeline depth is 14 stages • 128 reorder (renaming) buffer size
- 54. V. ILP TODAY: x86 • Hyper-Threading: – SMT – The processor may stall due to a cache miss, branch misprediction, or data dependency. – Branch misprediction costs 17 cycles
- 55. V. ILP TODAY: x86
- 56. V. ILP TODAY: ARM - The average CPI for the ARM7 family is about 1.9 cycles per instruction. - The average CPI for the ARM9 family is about 1.5 cycles per instruction. - The average CPI for the ARM11 family is about 1.39 cycles per instruction.
- 57. SOURCES Computer Architecture: A Quantitative Approach. Hennessy, J.L., Patterson, D.A., Asanović, K.. 5th Ed. 2012. Morgan Kaufmann/Elsevier. Limits of instruction-level parallelism. D. W. Wall. IV international conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), pages 176–188, 1991. Computer Science 61C - Lecture 31: Instruction Level Parallelism. Mike Franklin, Dan Garcia. UC Berkeley. Fall. 2011 ILP and TLP in Shared Memory Applications: A Limit Study. E. Fatehi, P. V. Gratz, Proceedings of the 23rd international conference on Parallel architectures and compilation, pages 113-126, 2014. MIPS Multicycle Model: Pipelining. Michael Langer. Introduction to Computer Systems. McGill University. 2012. IBM Power5 Chip: A Dual-Core Multithreaded Processor. R. Kalla, B. Sinharoy, J. M. Tendler. IBM. IEEE CS. 2004.