module2.ppt

1
Pipeline Hazards and
Their Resolution
Mechanisms
Dr. SUBHASIS DASH
School of Computer Engineering
KIIT UNIVERSITY
BHUBANESWAR
High Performance
Computing

2
Drags on Pipeline Performance
 Factors that can degrade pipeline
performance:
 Unbalanced stages
 Pipeline overheads
 Clock skew
 Hazards
 Hazards cause the worst drag on
the performance of a pipeline.

3
LOAD R3, 9R6)
DIV R1, R3, R4
ADD R5, R1, R3
STORE R2, 2(R5)
SUB R2, R1, R8
MUL R9, R2, R10
XOR R2, R9, R7
LOAD R3, 9(R6)
DIV R1, R3, R4
ADD R5, R1, R3
STORE R2, 2(R5)
SUB R2, R1, R8
MUL R9, R2, R10
XOR R2, R9, R7

5
Pipeline Hazards
 What is a pipeline hazard?
 A situation that prevents an instruction
from executing during its designated
clock cycles.
 There are 3 classes of hazards:
 Structural Hazards
 Data Hazards
 Control Hazards

6
Structural Hazards
 Arise from resource conflicts among
instructions executing concurrently:
 Same resource is required by two (or more)
concurrently executing instructions at the same
time.
 Easy way to avoid structural hazards:
 Duplicate resources (sometimes not practical)
 Memory interleaving ( lower & higher order )
 Examples of Resolution of Structural Hazard:
 An ALU to perform an arithmetic operation and an
adder to increment PC.
 Separate data cache and instruction cache
accessed simultaneously in the same cycle.

7
Structural Hazard: Example
IF ID EXE MEM WB
IF ID EXE MEM WB
IF ID EXE MEM WB
IF ID EXE MEM WB

8
An Example of a Structural
Hazard
ALU
Reg
Mem DM Reg
ALU
Reg
Mem DM Reg
ALU
Reg
Mem DM Reg
ALU
Reg
Mem DM Reg
Time
ALU
Reg
Mem DM Reg
Load
Instruction 1
Instruction 2
Instruction 3
Instruction 4
Would there be a hazard here?

9
How is it Resolved?
ALU
Reg
Mem DM Reg
ALU
Reg
Mem DM Reg
ALU
Reg
Mem DM Reg
Time
ALU
Reg
Mem DM Reg
Load
Instruction 1
Instruction 2
Stall
Instruction 3
Bubble Bubble Bubble Bubble Bubble
A Pipeline can be stalled by
inserting a “bubble” or NOP

10
Performance with Stalls
 Stalls degrade performance of a
pipeline:
 Result in deviation from 1
instruction executing/clock cycle.
 Let’s examine by how much stalls can
impact CPI…

11
Stalls and Performance
• CPI pipelined =
=Ideal CPI + Pipeline stall cycles per instruction
=1 + Pipeline stall cycles per instruction
• Ignoring overhead and assuming stages are
balanced:
n
instructio
per
cycles
stall
pipeline
d
unpipeline
CPI
Speedup


1

12
Alternate Speedup Expression
depth
Pipeline
d
unpipeline
cycle
Clock
pipelined
cycle
Clock 
pipelined
cycle
Clock
d
unpipeline
cycle
Clock
depth
Pipeline 
pipelined
cycle
Clock
d
unpipeline
cycle
Clock
n
instructio
per
cycles
stall
Pipeline
pipelining
from
Speedup 


1
1
If pipe stages are perfectly balanced & there is no
overhead, the clock cycle on the pipelined processor is
smaller than the clock cycle of the un-pipelined processor by
a factor equal to the pipelined depth.

13
An Example of Performance
Impact of Structural Hazard
 Assume:
 Pipelined processor.
 Data references constitute 40% of an
instruction mix.
 Ideal CPI of the pipelined machine is 1.
 Consider two cases:
 Unified data and instruction cache vs. separate data
and instruction cache.
 What is the impact on performance?

14
An Example
cont…
 Avg. Inst. Time = CPI x Clock Cycle Time
(i) For Separate cache: Avg. Instr. Time=1*1=1
(ii) For Unified cache case:
= (1 + 0.4 x 1) x (Clock cycle timeideal)
= 1.4 x Clock cycle timeideal=1.4
 Speedup= 1/1.4
= 0.7
 30% degradation in performance

15
Data Hazards
 Occur when an instruction under execution
depends on:
 Data from an instruction ahead in pipeline.
 Example:
 Dependent instruction uses old data:
 Results in wrong computations
IF ID EXE MEM WB
IF ID EXE MEM WB
A=B+C; D=A+E;
A=B+C;
D=A+E;

16
Types of Data Hazards
 Data hazards are of three types:
 Read After Write (RAW)
 Write After Read (WAR)
 Write After Write (WAW)
 With an in-order execution machine:
 WAW, WAR hazards can not occur.
 Assume instruction i is issued before
j.

17
Read after Write (RAW) Hazards
 Hazard between two instructions I & J
may occur when j attempts to read some
data object that has been modified by I.
 instruction j tries to read its operand before
instruction i writes it.
 j would incorrectly receive an old or incorrect
value.
 Example:
… j i …
Instruction j is a
read instruction
issued after i
Instruction i is a
write instruction
issued before j
i: ADD R1, R2, R3
j: SUB R4, R1, R6

18
Read after Write (RAW)
Hazards
D (I)
Instn . I
Write
R (I)
D (J)
R (J)
Instn . J
Read
RAW
R (I) ∩ D (J) ≠ Ø for RAW

19
RAW Dependency: More
Examples
 Example program (a):
 i1: load r1, addr;
 i2: add r2, r1,r1;
 Program (b):
 i1: mul r1, r4, r5;
 i2: add r2, r1, r1;
 Both cases, i2 does not get operand until i1
has completed writing the result
 In (a) this is due to load-use dependency
 In (b) this is due to define-use dependency

20
Write after Read (WAR) Hazards
 Hazard may occur when j attempts to
modify (write) some data object that is
going to read by I.
 Instruction J tries to write its operand at
destination before instruction I read it.
 I would incorrectly receive a new or incorrect
value.
 Example:
… j i …
Instruction j is a
write instruction
issued after i
Instruction i is a
read instruction
issued before j
i: ADD R1, R2, R3
j: SUB R2, R4, R6

21
Write after Read (WAR)
Hazards
D (J)
Instn . J
Write
R (J)
D (I) R (I)
Instn . I
Read
WAR
D (I) ∩ R (J) ≠ Ø for WAR

22
Write After Write (WAW) Hazards
 WAW hazard:
 Both I & J wants to modify a same data object.
 instruction j tries to write an operand before
instruction i writes it.
 Writes are performed in wrong order.
 Example:
… j i …
Instruction j is a
write instruction
issued after i
Instruction i is a
write instruction
issued before j
i: DIV F1, F2, F3
j: SUB F1, F4, F6
(How can this happen???)

23
Write After Write (WAW)
Hazards
D (I)
Instn . I
Write
R (I)
R (J) D (J)
Instn . J
Write
WAW
R (I) ∩ R (J) ≠ Ø for WAW

24
WAR and WAW Dependency:
More Examples
 Example program (a):
 i1: mul r1, r2, r3;
 i2: add r2, r4, r5;
 Example program (b):
 i1: mul r1, r2, r3;
 i2: add r1, r4, r5;
 Both cases have dependence between i1 and i2
 in (a) r2 must be read before it is written into
 in (b) r1 must be written by i2 after it has been
written into by i1

25
Inter-Instruction Dependences
 Data dependence
r3  r1 op r2 Read-after-Write
r5  r3 op r4 (RAW)
 Anti-dependence
r3  r1 op r2 Write-after-Read
r1  r4 op r5 (WAR)
 Output dependence
r3  r1 op r2 Write-after-Write
r5  r3 op r4 (WAW)
r3  r6 op r7
Control dependence
False
Dependency

26
Data Dependencies : Summary
Data dependencies
in straight-line code
RAW
Read After Write
dependency
Load-Use
dependency
Define-Use
dependency
WAR
Write After Read
dependency
WAW
Write After Write
dependency
( Flow dependency ) ( Anti dependency ) ( Output dependency )
True dependency
Cannot be overcome
False dependency
Can be eliminated by register renaming

28
Solutions to Data Hazard
 Operand forwarding
 By S/W (NOP)
 Reordering the instruction

29
Recollect Data Hazards
What causes them?
 Pipelining changes the order of read/write
accesses to operands.
 Order differs from that of an unpipelined
machine.
 Example:
 ADD R1, R2, R3
 SUB R4, R1, R5
For MIPS, ADD writes the
register in WB but SUB
needs it in ID.
This is a data hazard

30
Illustration of a Data Hazard
ALU
Reg
Mem DM Reg
ALU
Reg
Mem DM Reg
ALU
Reg
Mem DM
Reg
Mem
Time
ADD R1, R2, R3
SUB R4, R1, R5
AND R6, R1, R7
OR R8, R1, R9
XOR R10, R1, R11
ALU
Reg
Mem
ADD instruction causes a hazard in next 3 instructions
because register not written until after those 3 read it.

31
ADD R1, R2, R3
SUB R4, R1, R5
AND R6, R1, R7
OR R8, R1, R9
XOR R10, R1, R11
1 + 1 = 2 STALLS
Time
Inst.n 1 2 3 4 5 6 7 8 9 10 11 12
IF ID EXE MEM WB
IF STALL STALL ID EXE MEM WB
STALL STALL IF ID EXE MEM WB
IF ID EXE MEM WB
IF ID EXE MEM WB
Total time to execute = 11 clock cycle.

32
ADD R1, R2, R3
SUB R4, R1, R5
AND R6, R4, R7
OR R8, R6, R9
MUL R10, R8, R11
1 + 1 1 + 1 1 + 1 1 + 1= 8 STALLS
Time
Inst.n
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
IF ID EXE MEM WB
IF STALL STALL ID EXE MEM WB
STALL STALL IF STALL STALL ID EXE MEM WB

33
Forwarding
 Simplest solution to data hazard:
 forwarding
 Result of the ADD instruction not really
needed:
 until after ADD actually produces it.
 Can we move the result from EX/MEM
register to the beginning of ALU (where
SUB needs it)?
 Yes!

34
Forwarding
cont…
 Generally speaking:
 Forwarding occurs when a result is
passed directly to the functional
unit that requires it.
 Result goes from output of one
pipeline stage to input of another.

35
Forwarding Technique
EXECUTE
ALU
WRITE
RESULT
Latch Latch
Forwarding
Path

36
When Can We Forward?
ALU
Reg
Mem DM Reg
ALU
Reg
Mem DM Reg
ALU
Reg
Mem DM
Reg
Mem
Time
ADD R1, R2, R3
SUB R4, R1, R5
AND R6, R1, R7
OR R8, R1, R9
XOR R10, R1, R11
ALU
Reg
Mem
SUB gets info.
from EX/MEM
pipe register
AND gets info.
from MEM/WB
pipe register
OR gets info. by
forwarding from
register file
If line goes “forward” you can do forwarding.
If its drawn backward, it’s physically impossible.

37
Illustration of a Data Hazard Using
Operand Forwarding
ADD R1, R2, R3
SUB R4, R1, R5
AND R6, R1, R7
OR R8, R1, R9
XOR R10, R1, R11
Time
Inst.n 1 2 3 4 5 6 7 8 9 10 11 12
IF ID EXE MEM WB
IF ID EXE MEM WB
IF ID EXE MEM WB
IF ID EXE MEM WB
IF ID EXE MEM WB
So, here in this illustration due to operand forwarding pipeline
execution takes 2 number of less clock cycles in comparison to
the pipeline execution without operand forwarding.

38
Using Operand Forwarding
 A five stage pipeline processor has IF, ID, EXE, MEM,
WB. The IF, ID, MEM, WB stages takes 1 clock cycles
each for any instruction. The EXE stage takes 1 clock
cycle for ADD & SUB instructions, 3 clock cycles for
MUL instructions and 6 clock cycles for DIV
instructions respectively. Operand forwarding is used
in the pipeline. Then calculate the number of clock
cycles needed to execute the following sequence of
instruction.
 MUL R2, R0, R1
 DIV R5, R3, R4
 ADD R2, R5, R2
 SUB R5, R2, R6

39
Illustration of a Data Hazard Using
Operand Forwarding
MUL R2, R0, R1
DIV R5, R3, R4
ADD R2, R5, R2
SUB R5, R2, R6
Time
Inst.n 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
IF ID EXE EXE EXE MEM WB
IF ID STALL STALL EXE EXE EXE EXE EXE EXE MEM WB
IF STALL STALL ID STALL STALL STALL STALL STALL EXE MEM WB
IF STALL STALL STALL STALL STALL ID EXE MEM WB
1 + 1 + 1 + 1 + 1 + 1 + 1 = 7 STALL

40
Handling data hazard by S/W
 Compiler introduce NOP in between two
instructions
 NOP = a piece of code which keeps a gap
between two instruction
 Detection of the dependency is left entirely
on the S/W
 Advantage :- We find the easy technique
called as instruction reordering.

42
Instruction Reordering
 ADD R1 , R2 , R3
 SUB R4 , R1 , R5
 XOR R8 , R6 , R7
 AND R9 , R10 , R11
 ADD R1 , R2 , R3
 XOR R8 , R6 , R7
 AND R9 , R10 , R11
 SUB R4 , R1 , R5
Before
After

43
Data Hazard Requiring Stall
LOAD R1, 10(R2)
DSUB R4, R1,R5
AND R6, R1, R7
OR R8, R1,R9
IF ID EXE MEM WB
IF ID EXE MEM WB
IF ID EXE MEM WB
IF ID EXE MEM WB
IF ID EXE MEM WB
IF ID EXE MEM WB
IF ID EXE MEM WB
IF ID EXE MEM WB
STALL
LOAD R1, 10(R2)
DSUB R4, R1,R5
AND R6, R1, R7
OR R8, R1,R9
LOAD instruction has a latency that can not be eliminated by operand
forwarding . This is called as pipeline inter-lock to preserve the correct
execution.
Pipeline Inter-lock

48
Control Hazards
 Result from branch and other instructions that
change the flow of a program (i.e. change PC).
 Example:
 Statement in line 2 is control dependent on
statement at line 1.
 Until condition evaluation completes:
 It is not known whether s1 or s2 will execute next.
1: If(cond){
2: s1}
3: s2

49
Can You Identify Control
Dependencies?
1: if(cond1){
2: s1;
3: if(cond2){
4: s2;}
5: }
If a branch instruction
changes the PC to its target
address then it is a taken
branch. Otherwise, if it falls
through then it is untaken.

50
Control Hazard
 Result of branch instruction not known until
end of MEM/EXE stage (Why)
 Solution: Stall until result of branch
instruction is known
 That an instruction is a branch is known at the
end of its ID cycle
 Note: “IF” may have to be repeated

51
Reducing Branch Penalty
 Two approaches:
1) Move condition comparator to ID
stage:
 Decide branch outcome and target
address in the ID stage itself:
2) Branch prediction

52
An Example of Impact of
Branch Penalty
 Assume for a MIPS pipeline:
 16% of all instructions are
branches:
 4% unconditional branches: 3 cycle penalty
 12% conditional: 50% taken: 3 cycle penalty
 Calculate Overall CPI ???

53
Impact of Branch Penalty
 For a sequence of N instructions:
 N cycles to initiate each
 3 * 0.04 * N = 0.12N delays due to unconditional
branches
 3 * 0.5 * 0.12 * N = 0.18N delays due to
conditional taken
 Overall CPI= (1 + sum of delays due to unconditional
& conditional branches) * N
= 1 + (0.12+ 0.18)*N = 1.3*N
 (or 1.3 cycles/instruction)

54
Branch Hazard Solutions
#1: Stall
- until branch direction is clear – flushing pipe
 Three simplest methods of dealing
with branches:
 Flush Pipeline:
 Redo the instructions following a branch,
once an instruction is detected to be
branch taken during the ID stage.
 Very simple for design
 Branch penalty is fixed & can not be
reduced by S/W.

55
Branch Hazard Solutions
#2: Predict Branch Not Taken
 Execute successor instructions in sequence as if
there is no branch
 undo instructions in pipeline if branch actually
taken
 NOTE: If branch is taken, we need to turn the
fetch of instruction into NOP & restart the fetch at
target address.

56
Predict Untaken Scheme
When branch is untaken, determine during ID phase, we have
fetched the fall through and just continue.

57
Branch Hazard Solutions cont…
#3: Predict Branch Taken
• Treat every branch as taken
• Decode (ID) branch instn . then IF from branch target
 But branch target address not available after IF in
MIPS
 MIPS still incurs 1 cycle branch penalty even with predict
taken
 Other machines: branch target known before branch
outcome computed, significant benefits can accrue

58
Predict Taken Scheme
When branch is taken during ID phase, we have to restart the
fetched at branch target & all instructions following the branch to be
STALL by 1 clock cycle.
STALL

59
Branch Hazard Solutions Cont…
#4: Delayed Branch
 Instruction(s) after branch are executed anyway
 Sequential successors are called branch-delay-slots
 Insert unrelated successor in the branch delay slot
branch instruction
sequential successor1
sequential successor2
........
sequential successorn
branch target if taken
 1 slot delay required in 5 stage pipeline
Branch delay of length n

60
Delayed Branch
 Simple idea: Put an instruction that would be executed
anyway right after a branch.
 Question: What instruction do we put in the delay slot?
 Answer: one that can safely be executed no matter
what the branch does.
 The compiler decides this.
IF ID EX MEM WB
IF
IF ID EX MEM WB
ID EX MEM WB
Branch
Delayed slot instruction
Branch target OR successor
delay slot

61
Delayed Branch
 One possibility: Fill the slot with an instruction from
before the branch instruction
 Example:
 Restriction: branch must not depend on result of the filled
instruction
 The DADD instruction is executed no matter what happens in
the branch:
 Because it is executed before the branch!
 Therefore, it can be moved
 Don’t have dependency with the branch instruction.
DADD R1, R2, R3
if R2 == 0 then
delay slot
DADD R1, R2, R3
if R2 == 0 then
DADD R1, R2, R3

62
Delayed Branch
IF ID EX MEM WB
IF ID EX MEM WB
IF ID EX MEM WB
branch
add instruction
branch target/successor
By this time, we know whether
to take the branch or whether not
to take it
 We get to execute the “DADD” execution
“for free”

63
Delayed Branch
 Another possibility: Fill the slot with an independent
instruction from much before or an independent
instruction from the target of the branch instruction
 Example:
 Restriction: Should be OK to execute instruction
even if not taken
 The DSUB instruction can be replicated into the delay slot
DSUB R4, R5, R6
...
DADD R1, R2, R3
if R1 == 0 then
delay slot

64
Delayed Branch
 Example:
 The DSUB instruction can be replicated into the
delay slot
 Improves performance: when branch is taken
DSUB R4, R5, R6
...
DADD R1, R2, R3
if R1 == 0 then
DSUB R4, R5, R6

65
Delayed Branch
 Yet another possibility: Fill the slot with an
independent instruction from fall through (any 1
independent instruction from branch successor) of
branch
 Example:
 Restriction: Should be OK to execute instruction
even if taken
 The OR instruction can be moved into the delay slot
ONLY IF its execution doesn’t disrupt the program
execution
DADD R1, R2, R3
if R1 == 0 then
OR R7, R8, R9
DSUB R4, R5, R6
delay slot

66
Delayed Branch
 Example:
 The OR instruction can be moved into the delay
slot ONLY IF its execution doesn’t disrupt the
program execution
 Improves performance: when branch is not
taken
DADD R1, R2, R3
if R1 == 0 then
OR R7, R8, R9
DSUB R4, R5, R6
OR R7, R8, R9

67
LD
DSUBU
BEQZ
R1,0(R2)
R1,R1,R3
R1,L
OR
DADDU
R4,R5,R6
R10,R4,R3
DADDU R7,R8,R9
R1 == 0
R1 != 0
LD
DSUBU
BEQZ
OR
DADDU
DADDU
L :
R1,0(R2)
R1,R1,R3
R1,L
R4,R5,R6
R10,R4,R3
R7,R8,R9
1.) BEQZ is dependent on DSUBU and DSUBU on LD,
2.) If we knew that the branch was taken with a high probability, then
DADDU could be moved into block B1, since it doesn’t have any dependencies
with block B2,
3.) Conversely, knowing the branch was not taken, then OR could be moved
into block B1, since it doesn’t affect anything in B3,
B1
B2
B3
Delayed Branch Example
Taken
Un-taken

68
Delayed Branch
 Where to get instructions to fill
branch delay slots?
 Before branch instruction
 From the target address: Useful only if
branch taken.
 From fall through: Useful only if branch
not taken.

69
Performance of branch with Stalls
 Stalls degrade performance of a
pipeline:
 Result in deviation from 1
instruction executing/clock cycle.
 Let’s examine by how much stalls can
impact CPI…

70
Stalls and Performance with
branch
• CPI pipelined =
=Ideal CPI + Pipeline stall cycles per instruction
=1 + Pipeline stall cycles per instruction

71
Performance of branch instruction
 Pipeline speed up
 Pipeline stall cycle from branches = Branch
frequency * branch penalty
 Pipeline speed up =
Pipeline depth
1+ pipeline stall cycle from branch
Pipeline depth
1+ Branch frequency * Branch Penalty

72
Program Dependences Can
Cause Hazards!
 Hazards can be caused by dependences
within a program.
 There are three main types of
dependences in a program:
 Data dependence
 Name dependence(False Dependancy)
 Control dependence

73
Types of Name Dependences
 Two types of name dependences:
 Anti-dependence
 Output dependence

74
Anti-Dependence or (WAR)
 Anti-dependence occurs between two
instructions i and j, iff:
 j writes to a register or memory location that i
reads.
 Original ordering must be preserved to ensure
that i reads the correct value.
 Example:
 ADD F0,F6,F8
 SUB F8,F4,F5

75
Output Dependence or (WAW)
 Output dependence occurs between
two instructions i and j, iff:
 The two instructions write to the same
memory location.
 Ordering of the instructions must be
preserved to ensure:
 Finally written value corresponds to j.
 Example:- ADD f6,f0,f8
 Mul f6,f10,f8


76
Exercise
 Identify all the dependences in
the following C code:
1. a=b+c;
2. b=c+d;
3. a=a+c;
4. c=b+a;

77
A Solution to WAR and WAW
Hazards
 Rename Registers
 i1: mul r1, r2, r3;
 i2: add r1, r4, r5;
 Register renaming can get rid of most false
dependencies:
 Compiler can do register renaming in the register
allocation process (i.e., the process that assigns
registers to variables).

78
Out-of-order Pipelining
• • •
• • •
• • •
• • •
IF
ID
RD
WB
INT Fadd1 Fmult1 LD/ST
Fadd2 Fmult2
Fmult3
EX
Program Order
Ia: F1  F2 x F3
. . . . .
Ib: F1  F4 + F5
Out-of-order WB
Ib: F1  “F4 + F5”
. . . . . .
Ia: F1  “F2 x F3”

79
Branch Prediction
Mr. SUBHASIS DASH
KIIT UNIVERSITY
BHUBANESWAR
High Performance
Computing

80
Dynamic Branch Prediction
• KEY IDEA: Hope that branch assumption is
correct.
• If yes, then we’ve gained a performance
improvement.
• Otherwise, discard instructions
• program is still correct, all we’ve done is “waste” a
clock cycle.
Two approaches
Direction Based Prediction History /Profile Based Prediction

81
Direction Based Prediction
• Simple to implement
• However, often branch behavior is
variable (dynamic).
• Can’t capture such behavior at compile
time with simple direction based
prediction!
• Need history ( profile)-based
prediction.

82
History-based Branch
Prediction
 An important example is State-based
branch prediction:
 Needs 2 parts:
 “Predictor” to guess where/if
instruction will branch (and to where)
 “Recovery Mechanism”: i.e. a way to fix
mistakes

83
History-based Branch
Prediction cont…
 One bit predictor:
 Use result from last time this
instruction executed.
 Problem:
 Even if branch is almost always taken,
we will be wrong at least twice
 if branch alternates between taken, not
taken
 We get 0% accuracy

84
1-Bit Prediction Example
All Most All are taken
 Let initial value = NT, actual outcome
of branches is- T, T, T, T, T, NT
 Predictions are:
NT, T,T,T,T,T
 2 wrong (in red), 4 correct = 66% accuracy
 2-bit predictors can do even better
 In general, can have k-bit predictors.

85
All Most All are untaken
 Let initial value = T, actual outcome of
branches is- NT, NT, NT, NT, NT, T
T, NT, NT, NT, NT, NT

86
A Mix Type
branches is- NT, NT,NT,T,T,T
T, NT,NT,NT,T,T

87
Alternate Taken Untaken
branches is- NT, T, NT, T
T, NT, T, NT

88
1-bit Predictor
 Set bit to 1 or 0:
Depending on whether branch Taken
(T) or Not-taken (NT)
 Pipeline checks bit value and predicts
 If incorrect then need to discard
speculatively executed instruction
 Actual outcome used to set the bit
value.

89
• Change prediction only if twice mispredicted:
• Adds hysteresis to decision making process
2-bit Dynamic Branch
Prediction Scheme
T
T
NT
Predict Taken
Predict Not
Taken
Predict Taken
Predict Not
Taken
11 10
01 00
T
NT
T
NT
NT

90
branches is- NT, NT,NT,T,T,T
T, T,NT,NT,NT,T

91
of branches is- NT, NT,NT,T,T,T
NT, NT,NT,NT,NT,T
 2 wrong (in red), 4correct = 66% accuracy

92
branches is- NT, T, NT, T
T, T, T, T

93
of branches is- NT, T, NT, T
NT, NT, NT, NT

94
An Example of Computing
Performance
 Program assumptions:
 23% loads and in ½ of cases, next
instruction uses load value
 13% stores
 19% conditional branches
 2% unconditional branches
 43% other

95
Example
cont…
 Machine Assumptions:
 5 stage pipe
 Penalty of 1 cycle on use of load value
immediately after a load.
 Jumps are resolved in ID stage for a 1
cycle branch penalty.
 75% branch prediction accuracy.
 1 cycle delay on misprediction.
Calculate CPI???

96
Example
cont…
 CPI penalty calculation:
 Loads:
 50% of 23% of loads have 1 cycle penalty:0.5*0.23=0.115
 Jumps:
 All of the 2% of jumps have 1 cycle penalty: 0.02*1 = 0.02
 Conditional Branches:
 25% of the 19% are mispredicted, have a 1 cycle penalty:
0.25*0.19*1 = 0.0475
 Total Penalty: 0.115 + 0.02 + 0.0475 = 0.1825
 Average CPI: 1 + 0.1825 = 1.1825

97
Loop-level Parallelism
 It may be possible to execute
different iterations of a loop in
parallel.
 Example:
 For(i=0;i<1000;i++){
 a[i]=a[i]+b[i];
 b[i]=b[i]*2;
 }

98
Problems in Exploiting Loop-
level Parallelism
• Loop Carried Dependences:
• A dependence across different
iterations of a loop.
 Loop Independent Dependences:
 A dependence within the body of the
loop itself (i.e. within one iteration).

99
Loop-level Dependence
 Example:
 For(i=0;i<1000;i++){
 a[i+1]=b[i]+c[i]
 b[i+1]=a[i+1]+d[i];
 }
 Loop-carried dependence from one iteration to
the preceding iteration.
 Also, loop-independent dependence on account of
a[i+1]

100
Eliminating Loop-level
Dependences Through Code
Transformations
 We shall examine 2 techniques:
 Static loop unrolling
 Software pipelining

101
Static Loop Unrolling
- A high proportion of loop instructions are
loop management instructions.
- Eliminating this overhead can significantly
increase the performance of the loop.
- for(i=1000;i>0;i--)
- {
- a[i]=a[i]+c;
- }

102
L.D
ADD.D
S.D
DADDUI
BNE
Loop : F0,0(R1)
F4,F0,F2
F4,0(R1)
R1,R1,#-8
R1,R2,Loop
; F0 = array elem.
; add scalar in F2
; store result
; decrement ptr
; branch if R1 !=R2
for(i=1000;i>0;i--)
- {
- a[i]=a[i]+c;
- }

103
Instruction
Producing Result
Instruction Using
Result
Latency
LOAD FP ALU 1
FP ALU FP ALU 3
FP ALU STORE 2
INT. ALU Branch Instruction 1
LOAD STORE / Int. ALU 0
STORE Any / FP ALU
/ INT. ALU
0
Dependency Table

104
LOOP : LOAD F0, 0(R1)
ADD F4, F0, F2
STORE F4, 0(R1)
ADD R1, R1, # -8
BNE R1, R2, LOOP
Introducing Latency

105
STALL
ADD F4, F0, F2
STALL
STALL
STORE F4, 0(R1)
ADD R1, R1, # -8
STALL
BNE R1, R2, LOOP
NOP
After Introducing Latency

106
ADD R1, R1, # -8
ADD F4, F0, F2
STALL
BNE R1, R2, LOOP
STORE F4, 8(R1)
After Re-ordering Instruction

107
L.D
ADD.D
S.D
L.D
ADD.D
S.D
L.D
ADD.D
S.D
L.D
ADD.D
S.D
DADDUI
BNE
Loop : F0,0(R1)
F4,F0,F2
F4,0(R1)
F6,-8(R1)
F8,F6,F2
F8,-8(R1)
F10,-16(R1)
F12,F10,F2
F12,-16(R1)
F14,-24(R1)
F16,F14,F2
F16,-24(R1)
R1,R1,#-32
R1,R2,Loop
cont…

108
cont…
Loop : F0,0(R1)
F4,F0,F2
F4,0(R1)
F6,-8(R1)
F8,F6,F2
F8,-8(R1)
F10,-16(R1)
F12,F10,F2
F12,-16(R1)
F14,-24(R1)
F16,F14,F2
F16,-24(R1)
R1,R1,#-32
R1,R2,Loop
n loop
Bodies for
n = 4
Adjusted loop
overhead
instructions
Note the adjustments
for store and load
offsets (only store
highlighted red)!
Note the renamed
registers. This eliminates
dependencies between
each of n loop bodies of
different iterations.
L.D
ADD.D
S.D
L.D
ADD.D
S.D
L.D
ADD.D
S.D
L.D
ADD.D
S.D
DADDUI
BNE

109
LOOP :- LOAD F0, 0(R1)
LOAD F6, -8(R1)
LOAD F10, -16(R1)
LOAD F10, -24(R1)
ADD F4, F0, F2
ADD F8, F6, F2
ADD F12, F10, F2
ADD F16, F14, F2
STORE F4, 0(R1)
STORE F8, -8(R1)
STORE F4, -16(R1)
STORE F4, -24(R1)
ADD R1, R1, # -32
BNE R1, R2, LOOP
L.D
ADD.D
S.D
L.D
ADD.D
S.D
L.D
ADD.D
S.D
L.D
ADD.D
S.D
DADDUI
BNE
Loop : F0,0(R1)
F4,F0,F2
F4,0(R1)
F6,-8(R1)
F8,F6,F2
F8,-8(R1)
F10,-16(R1)
F12,F10,F2
F12,-16(R1)
F14,-24(R1)
F16,F14,F2
F16,-24(R1)
R1,R1,#-32
R1,R2,Loop

110
LOOP:- LOAD F0, 0(R1)
LOAD F6, -8(R1)
LOAD F10, -16(R1)
LOAD F10, -24(R1)
ADD F4, F0, F2
ADD F8, F6, F2
ADD F12, F10, F2
ADD F16, F14, F2
STORE F4, 0(R1)
STORE F8, -8(R1)
ADD R1, R1, # -32
STORE F4, 24(R1)
BNE R1, R2, LOOP
STORE F4, 8(R1)

111
for( i=0; i > 1000; i++ )
- {
- a [i + 1] = b[i] + c [i] ;
- b [i + 1] = a [i + 1] + d [i]
- }
ADD F4, F0, F2
ADD F6, F4, F3
STORE F6, 0(R1)
ADD R1, R1, # +8
BNE R1, R2, LOOP

112
Instruction
Producing Result
Instruction Using
Result
Latency
LOAD FP ALU 1
FP ALU FP ALU 3
FP ALU STORE 2
INT. ALU Branch Instruction 1
LOAD STORE / Int. ALU 0
STORE Any / FP ALU
/ INT. ALU
0
Dependency Table

113
Introducing Latency
ADD F4, F0, F2
ADD F6, F4, F3
STORE F6, 0(R1)
ADD R1, R1, # +8
BNE R1, R2, LOOP

114
STALL
ADD F4, F0, F2
STALL
STALL
STALL
ADD F6, F4, F3
STALL
STALL
STORE F6, 0(R1)
ADD R1, R1, # +8
STALL
BNE R1, R2, LOOP
STALL

115
ADD R1, R1, # +8
ADD F4, F0, F2
STALL
STALL
STALL
ADD F6, F4, F3
STALL
BNE R1, R2, LOOP
STORE F6, -8(R1)

116
ADD F4, F0, F2
ADD F6, F4, F3
STORE F6, 0(R1)
LOAD F8, 8(R1)
ADD F7, F8, F2
ADD F9, F7, F3
STORE F9, 8(R1)
LOAD F10, 16(R1)
ADD F11, F10, F2
ADD F12, F11, F3
STORE F12, 16(R1)
LOAD F13, 24(R1)
ADD F14, F13, F2
ADD F15, F14, F3
STORE F15, 24(R1)
ADD R1, R1, # +32
BNE R1, R2, LOOP
Static Loop
Unrolling

117
LOAD F8, 8(R1)
LOAD F10, 16(R1)
LOAD F13, 24(R1)
ADD F4, F0, F2
ADD F7, F8, F2
ADD F11, F10, F2
ADD F14, F13, F2
ADD F6, F4, F3
ADD F9, F7, F3
ADD F12, F11, F3
ADD F15, F14, F3
STORE F6, 0(R1)
STORE F9, 8(R1)
STORE F12, 16(R1)
STORE F15, 24(R1)
ADD R1, R1, # +32
BNE R1, R2, LOOP
Static Loop
Unrolling

118
LOAD F8, 8(R1)
LOAD F10, 16(R1)
LOAD F13, 24(R1)
ADD F4, F0, F2
ADD F7, F8, F2
ADD F11, F10, F2
ADD F14, F13, F2
ADD F6, F4, F3
ADD F9, F7, F3
ADD F12, F11, F3
ADD F15, F14, F3
STORE F6, 0(R1)
STORE F9, 8(R1)
ADD R1, R1, # +32
STORE F12, -16(R1)
BNE R1, R2, LOOP
STORE F15, -8(R1)
Static Loop
Unrolling

119
The latencies of Floating point operations are given
below
Instruction Producing
Result
Instruction Using
Result
Latency
LOAD FP ALU 1
FP ALU FP ALU 3
FP ALU STORE 2
INT. ALU BRANCH 1
LOAD STORE / INT. ALU 0
STORE Any 0

120
Consider the following MIPS code
Loop: L.D F0, 0(R1)
L.D F2,0(R2)
ADD.D F4,F0,F2
S.D F4, 0(R1)
DADDUI R1,R1,#-8
DADDUI R2,R2,#-8
BNE R1,R2,Loop
Unroll the loop two times. Find out the number of
clock cycles needed per iteration for execution
with and without unrolling.

121
Assuming R1 – R2 (that is, the size of the array) is initially a
multiple of 32.
Show how the loop would look both unscheduled by the
compiler and after compiler scheduling for both floating
point ALU operation and branch delays, including any
stalls or idle clock cycles.
Unroll the loop as many times it is necessary to schedule it
without any stalls, collapsing the loop overhead
instructions.
Calculate the number of clock cycles required to execute
each iteration for the above given loop after unrolling is
used.

122
Loop: L.D F0, 0(R1)
L.D F2,0(R2)
STALL
ADD.D F4,F0,F2
STALL
STALL
S.D F4, 0(R1)
DADDUI R1,R1,#-8
DADDUI R2,R2,#-8
STALL
BNE R1,R2,Loop

123
Loop: L.D F0, 0(R1)
L.D F2,0(R2)
STALL DADDUI R1,R1,#-8
ADD.D F4,F0,F2
STALL  DADDUI R2,R2,#-8
STALL 
BNE R1,R2,Loop
S.D F4, 0(R1)

124
Static Loop
Unrolling
L.D F0, 0(R1)
L.D F2,0(R2)
ADD.D F4,F0,F2
S.D F4, 0(R3)
L.D F3, -8(R1)
L.D F5,-8(R2)
ADD.D F6,F3,F5
S.D F6, -8(R3)
L.D F7, -16(R1)
L.D F8,-16(R2)
ADD.D F9,F7,F8
S.D F9, -16(R3)
L.D F10, -24(R1)
L.D F11,-24(R2)
ADD.D F12,F10,F11
S.D F12, -24(R3)

128
Dependency
A
B
C
Unrolling
A
B
C
A
B
C
A
B
C

129
 Unrolling Benefits
 It improves the performance by eliminating
overhead instructions.
 Unrolling Limitations
 Decrease in the amount of overhead amortized
with each unroll.
 Growth in cache size due to loop unrolling may
increase cache miss rates.
 Shortfall of registers created by aggressive
unrolling register renaming and scheduling.
 Significant increase in complexity of
compilers.

130
Software Pipelining
 Eliminates loop-independent
dependence through code restructuring.
 Reduces stalls
 Helps achieve better performance in
pipelined execution.
 As compared to simple loop unrolling:
 Consumes less code space

131
Software Pipelining
cont…
 Central idea: reorganize loops
 Each iteration is made from instructions
chosen from different iterations of the
original loop.
i4
i3
i2
i1
i0
Software
Pipeline
Iteration
i5

132
Software Pipelining
cont…
 Observation: if iterations from loops
are independent, then can get more
ILP by taking instructions from
different iterations
 Exactly just as it happens in a
hardware pipeline:
 In each iteration of a software pipelined
code, some instruction of some iteration
of the original loop is executed.

133
Static Loop Unrolling Example
L.D
ADD.D
S.D
DADDUI
BNE
Loop : F0,0(R1)
F4,F0,F2
F4,0(R1)
R1,R1,#-8
R1,R2,Loop
; F0 = array elem.
; add scalar in F2
; store result
; decrement ptr
; branch if R1 !=R2

134
Software Pipelining
cont…
- How is this done?
1  unroll loop body with an unroll
factor of n. (we have taken n = 3 for our
example)
2  select order of instructions from
different iterations to pipeline
3  “paste” instructions from different
iterations into the new pipelined loop body

135
Software Pipelining: Step 1
Iteration i:
Iteration i + 1:
Iteration i + 2:
Note:
1.) We are unrolling the loop
Hence no loop overhead
Instructions are needed!
2.) A single loop body of
restructured loop would
contain instructions from
different iterations of the
original loop body.
Unrolled 3 times
1 L.D F0,0(R1)
2 ADD F4,F0,F2
3 S.D F4,0(R1)
4 L.D F0,0(R1)
5 ADD F4,F0,F2
6 S.D F4,0(R1)
7 L.D F0,0(R1)
8 ADD F4,F0,F2
9 S.D F4,0(R1)
10 DSUBUI R1,R1,#24
11 BNEZ R1,LOOP

136
Iteration i:
Iteration i + 1:
Iteration i + 2:
before:-Unrolled 3 times
1 L.D F0,0(R1)
2 ADD F4,F0,F2
3 S.D F4,0(R1)
4 L.D F6,-8(R1)
5 ADD F8,F6,F2
6 S.D F8,-8(R1)
7 L.D F10,-16(R1)
8 ADD F12,F10,F2
9 S.D F12,-16(R1)
10 DSUBUI R1,R1,#24
11BNEZ R1,LOOP
1.)
2.)
3.)
Notes:
1.) We’ll select the following
order in our pipelined loop:
2.) Each instruction (L.D
ADD.D S.D) must be
selected at least once to
make sure that we don’t
leave out any instructions of
the original loop in the
pipelined loop.

137
Iteration i:
Iteration i + 1:
Iteration i + 2:
Before:-Unrolled 3 times
1 L.D F0,0(R1)
2 ADD F4,F0,F2
3 S.D F4,0(R1)
4 L.D F6,-8(R1)
5 ADD F8,F6,F2
6 S.D F8,-8(R1)
7 L.D F10,-16(R1)
8 ADD F12,F10,F2
9 S.D F12,-16(R1)
10 DSUBUI R1,R1,#24
11 BNEZ R1,LOOP
After: Software Pipelined
before:-Unrolled 3 times
1 L.D F0,0(R1)
2 ADD F4,F0,F2
3 S.D F4,0(R1)
4 L.D F6,-8(R1)
5 ADD F8,F6,F2
6 S.D F8,-8(R1)
7 L.D F10,-16(R1)
8 ADD F12,F10,F2
9 S.D F12,-16(R1)
10 DSUBUI R1,R1,#24
11 BNEZ R1,LOOP

138
L.D
ADD.D
S.D
Iteration i: F0,0(R1)
F4,F0,F2
F4,0(R1)
L.D
ADD.D
S.D
L.D
ADD.D
S.D
F6,-8(R1)
F8,F6,F2
F8,-8(R1)
F10,-16(R1)
F12,F10,F2
F12,-16(R1)
Iteration i + 1:
Iteration i + 2:
1.)
2.)
3.)
S.D
ADD.D
L.D
DADDU
BNE
Loop : F4,0(R1)
F4,F0,F2
F0,-16(R1)
R1,R1,#-8
R1,R2,Loop
The Pipelined Loop
• Symbolic Loop Unrolling
– Maximize result-use distance
– Less code space than unrolling
– Fill & drain pipe only once per loop
vs. once per each unrolled iteration in loop unrolling

139
S.D
ADD.D
L.D
DADDUI
BNE
Loop : F4,16(R1)
F4,F0,F2
F0,0(R1)
R1,R1,#-8
R1,R2,Loop
; M[ i ]
; M[ i – 1 ]
; M[ i – 2 ]
Instructions to fill “software pipeline”
Pipelined Loop Body
Preheader
Postheader
Instructions to drain “software pipeline”

140
Software Pipelining Issues
 Register management can be tricky.
 In more complex examples, we may need to
increase the iterations between when data
is read and when the results are used.
 Optimal software pipelining has been
shown to be an NP-complete problem:
 Present solutions are based on heuristics.

141
Software Pipelining versus Loop
Unrolling
 Software pipelining takes less code space.
 Software pipelining and loop unrolling
reduce different types of inefficiencies:
 Loop unrolling reduces loop management
overheads.
 Software pipelining allows a pipeline to run
at full efficiency by eliminating loop-
independent dependencies.

142
Advantages of
Dynamic Scheduling
 Can handle dependences unknown at
compile time:
 E.g. dependences involving memory references.
 Simplifies the compiler.
 Allows code compiled for one pipeline to
run efficiently on a different pipeline.
 Hardware speculation can be used:
 Can lead to further performance advantages,
builds on dynamic scheduling.

143
Overview of Dynamic
Instruction Scheduling
 We shall discuss two schemes for
implementing dynamic scheduling:
 Scoreboarding: First used in the 1964 CDC
6600 computer.
 Tomasulo’s Algorithm: Implemented for
the FP unit of the IBM 360/91 in 1966.
 Since scoreboarding is a little closer to
in-order execution, we’ll look at it first.

144
A Point to Note About
Dynamic Scheduling
 WAR and WAW hazards that did
not exist in an in-order pipeline:
 Can arise in a dynamically scheduled
processor.

145
Scoreboarding
cont…
 Scoreboarding allows
instructions to execute out of
order:
When there are sufficient
resources.
 Named after the scoreboard:
Originally developed for CDC
6600.

146
Scoreboarding
The 5 Stage MIPS Pipeline
 Split the ID pipe stage of simple
5-stage pipeline into 2 stages:
Issue: Decode instructions, check
for structural hazards.
Read operands: Wait until no data
hazards, then read operands.

147
Scoreboarding
cont…
 Instructions pass through the
issue stage in order.
 Instructions can bypass each
other in the read operands
stage:
 Then enter execution out of
order.

148
Scoreboarding Concepts
 We had observed that WAR and
WAW hazards can occur in out-of-
order execution:
Instructions involved in a dependence
are stalled,
But, instructions having no dependence
are allowed to continue.
Different units are kept as busy as
possible.

149
 Essence of scoreboarding:
 Execute instructions as early as
possible.
 When an instruction is stalled,
 Later instructions are issued and
executed if they do not depend on
any active or stalled instruction.

150
A Few More Basic
 Every instruction goes through the
scoreboard:
 Scoreboard constructs the data
dependences of the instruction.
 Scoreboard decides when an instruction
can execute.
 Scoreboard also controls when an
instruction can write its results into the
destination register.

151
Scoreboarding
 Out-of-order execution requires multiple
instructions to be in the EX stage
simultaneously:
 Achieved with multiple functional units,
along with pipelined functional units.
 All instructions go through the scoreboard:
 Centralized control of issue, operand
reading, execution and writeback.
 All hazard resolution is centralized in the
scoreboard as well.

152
A Scoreboard for MIPS
FP Mult
FP Mult
FP Divide
FP Add
Integer Unit
Scoreboard
R
e
g
i
s
t
e
r
s
Data buses – source of structural hazard
Control/
status
Control/
status

153
4 Steps of Execution with
Scoreboarding
1. Issue: when a functional unit for an
instruction is free and no other active
instruction has the same destination
register:
• Avoids structural and WAW hazards.
2. Read operands: when all source operands
are available:
• Note: forwarding not used.
• A source operand is available if no earlier
issued active instruction is going to write it.
• Thus resolves RAW hazards dynamically.

154
Steps in Execution with
Scoreboarding
1. Execution: begins when the functional
unit. receives its operands; scoreboard
notified when execution completes.
2. Write Result: after WAR hazards have
been resolved.

155
Different parts of Scoreboard
 Instruction Status Unit :-
 Indicates which of 4 phases the instruction is in
 Functional Status Unit :-
 Indicates the states of functional unit & each
functional unit has 9 different fields.
1. BUSY:- Indicates weather the functional unit is busy or free.
2. OP :- Indicates the type of operation to be perform in the unit
3. Fi :- Indicates the destination register
4. Fj, Fk :- Indicates the source registers
5. Qj, Qk :- Indicates the functional units producing source
registers value
6. Rj, Rk :- Flags indicating when are ready & not yet read
1. IF operand is ready & not yet read then  YES
2. IF operand is not ready then  NO
3. IF operand is already read then  NO

156
Different parts of Scoreboard
 Register Result Status Unit :-
 Indicates which functional unit will write each
register, if an active instruction has the register as
its destination.
 The field is set to blank whenever there are no
pending instruction that will write that register.

157
Steps in Execution with Scoreboard Approach
• Write the steps of execution with scoreboard approach.
Assume we have 2 multipliers, 1 adders, 1 dividers & 1 integer
unit. The following set of MIPS instruction is going to be
executed in a pipelined system.
• Example:
• LOAD F6, 34(R2)
• LOAD F2, 45(R3)
• MUL FO, F2, F4
• SUB F8, F6, F2
• DIV F10, F0, F6
• ADD F6, F8, F2
• Consider following execution status for above instruction set:-
• 1st LOAD instruction is completed its write result phase.
• 2nd LOAD instruction is completed its execute phase.
• Other remaining instruction are in their instruction issue phase.
• Show the status of instruction, functional unit, & register
result status using dynamic scheduling with scoreboard
approach.

158
Scoreboard Example
Instruction status Read Execution
Write
Instruction j k Issue operands
complete
Result
LD F6 34+ R2
LD F2 45+ R3
MULTD
F0 F2 F4
SUBD
F8 F6 F2
DIV F10 F0 F6
ADDD
F6 F8 F2
Functional unit status dest S1 S2 FU for j
FU for k
Fj? Fk?
Time Name Busy Op Fi Fj Fk Qj Qk Rj Rk
Integer No
Mult1 No
Mult2 No
Add No
Divide No
Register result status
Clock F0 F2 F4 F6 F8 F10 F12 ... F30
FU

159
 Write the steps of execution with scoreboard approach.
 Example:
• LOAD F6, 34(R2)
• LOAD F2, 45(R3)
• MUL FO, F2, F4
• SUB F8, F6, F2
• DIV F10, F0, F6
• ADD F6, F8, F2
 Consider following execution status for above instruction set:-
• 1st two LOAD & SUB instructions are completed their write result phase.
• MUL & DIV instructions are ready to go for their write result phase.
• ADD instruction is in its execution phase.
 Show the status of instruction, functional unit, & register result
status using dynamic scheduling with scoreboard approach.

160
Scoreboard Example
Write
complete
Result
LD F6 34+ R2
LD F2 45+ R3
MULTD
F0 F2 F4
SUBD
F8 F6 F2
DIV F10 F0 F6
ADDD
F6 F8 F2
FU for k
Fj? Fk?
Integer No
Mult1 No
Mult2 No
Add No
Divide No
Clock F0 F2 F4 F6 F8 F10 F12 ... F30
FU

161
 Write the steps of execution with scoreboard approach.
 Example:
• LOAD F6, 34(R2)
• LOAD F2, 45(R3)
• MUL FO, F2, F4
• SUB F8, F6, F2
• DIV F10, F0, F6
• ADD F6, F8, F2
• 1st two LOAD, MUL, SUB & ADD instructions are completed their write
result phase.
• DIV instructions is ready to write its result .
 Show the status of instruction, functional unit, & register result
status using dynamic scheduling with scoreboard approach.

162
Scoreboard Example
Write
complete
Result
LD F6 34+ R2
LD F2 45+ R3
MULTD
F0 F2 F4
SUBD
F8 F6 F2
DIV F10 F0 F6
ADDD
F6 F8 F2
FU for k
Fj? Fk?
Integer No
Mult1 No
Mult2 No
Add No
Divide No
Clock F0 F2 F4 F6 F8 F10 F12 ... F30
FU

163
• Functional units:-
1 integer, 1 adder, 2 multipliers, 1 divider
• Instruction sequence:-
1. L.D F6, 34(R2)
2. L.D F2, 45(R3)
3. MUL F0, F2, F4
4. SUB F8, F6, F2
5. DIV F10, F0, F6
6. ADD F6, F8, F2
• Latencies:
– Integer → 1 cycle
– FP add → 2 cycles
– FP multiply → 10 cycles
– FP divide → 40 cycles
Scoreboard Example

164
Scoreboard Example
Write
complete
Result
LD F6 34+ R2
LD F2 45+ R3
MULTD
F0 F2 F4
SUBD
F8 F6 F2
DIV F10 F0 F6
ADDD
F6 F8 F2
FU for k
Fj? Fk?
Integer No
Mult1 No
Mult2 No
Add No
Divide No
Clock F0 F2 F4 F6 F8 F10 F12 ... F30
FU

165
Scoreboard Example Cycle 1
Write
complete
Result
LD F6 34+ R2 1
LD F2 45+ R3
MUL F0 F2 F4
SUBD
F8 F6 F2
DIV F10 F0 F6
ADDD
F6 F8 F2
FU for k
Fj? Fk?
Time
Name Busy Op Fi Fj Fk Qj Qk Rj Rk
Integer Yes Load F6 R2 Yes
Mult1 No
Mult2 No
Add No
Divide No
Clock F0 F2 F4 F6 F8 F10 F12 ... F30
1 FU Integer
Issue
LOAD

166
Instruction status Read Execution Write
complete Result
LD F6 34+ R2 1 2
LD F2 45+ R3
MULT F0 F2 F4
SUB F8 F6 F2
DIVD F10 F0 F6
ADD F6 F8 F2
FU for k
Fj? Fk?
Time
Integer Yes Load F6 R2 No
Mult1 No
Mult2 No
Add No
Divide No
Clock F0 F2 F4 F6 F8 F10 F12 ... F30
2 FU Integer
Note: Can’t issue I2
because Integer unit
is busy. Can’t issue
next instruction
MULT due to in-order
issue

167
complete Result
LD F6 34+ R2 1 2 3
LD F2 45+ R3
MULT F0 F2 F4
SUB F8 F6 F2
DIV F10 F0 F6
ADD F6 F8 F2
FU for k
Fj? Fk?
Time
Mult1 No
Mult2 No
Add No
Divide No
Clock F0 F2 F4 F6 F8 F10 F12 ... F30
3 FU Integer
Executing LOAD
Instruction

168
complete Result
LD F6 34+ R2 1 2 3 4
LD F2 45+ R3
MULT F0 F2 F4
SUB F8 F6 F2
DIV F10 F0 F6
ADD F6 F8 F2
FU for k
Fj? Fk?
Time
Mult1 No
Mult2 No
Add No
Divide No
Clock F0 F2 F4 F6 F8 F10 F12 ... F30
4 FU
Writing Result to F6
( LOAD Instruction )

169
complete Result
LD F6 34+ R2 1 2 3 4
LD F2 45+ R3 5
MULT F0 F2 F4
SUB F8 F6 F2
DIV F10 F0 F6
ADD F6 F8 F2
FU for k
Fj? Fk?
TimeName Busy Op Fi Fj Fk Qj Qk Rj Rk
Integer Yes Load F2 R3 Yes
Mult1 No
Mult2 No
Add No
Divide No
Clock F0 F2 F4 F6 F8 F10 F12 ... F30
5 FU Integer
Issue LD #2 since
integer unit is now
free.

170
complete Result
LD F6 34+ R2 1 2 3 4
LD F2 45+ R3 5 6
MULT F0 F2 F4 6
SUB F8 F6 F2
DIV F10 F0 F6
ADD F6 F8 F2
Functional unit status dest S1 S2 FU for j FU for k Fj? Fk?
Mult1 Yes Mult F0 F2 F4 Integer No Yes
Mult2 No
Add No
Divide No
Clock F0 F2 F4 F6 F8 F10 F12 ... F30
6 FU Mult Integer
Issue MULT.
Instruction

171
complete Result
LD F6 34+ R2 1 2 3 4
LD F2 45+ R3 5 6 7
MULT F0 F2 F4 6
SUB F8 F6 F2 7
DIV F10 F0 F6
ADD F6 F8 F2
Mult1 Yes Mult F0 F2 F4 Integer No Yes
Mult2 No
Add Yes Subd F8 F6 F2 Integer Yes No
Divide No
Clock F0 F2 F4 F6 F8 F10 F12 ... F30
7 FU Mult Integer Add
MULT can’t read its
operands (F2) because
LD #2 hasn’t finished.

172
Instruction status Read EX Write
Instruction j k Issue Op compl. Result
LD F6 34+ R2 1 2 3 4
LD F2 45+ R3 5 6 7 8
MULT F0 F2 F4 6
SUB F8 F6 F2 7
DIV F10 F0 F6 8
ADD F6 F8 F2
FU for k
Fj? Fk?
Time
Integer No
Mult1 Yes Mult F0 F2 F4 Yes Yes
Mult2 No
Add Yes Sub F8 F6 F2 Yes Yes
Divide Yes Div F10 F0 F6 Mult1 No Yes
Clock F0 F2 F4 F6 F8 F10 F12 ... F30
8 FU Mult1 Add Divide
DIV issues.
MULT and SUB
both waiting for F2.
LD #2 writes F2.

173
Instruction status Read EX Write
Instruction j k IssueOp complete
Result
LD F6 34+ R2 1 2 3 4
LD F2 45+ R3 5 6 7 8
MULT F0 F2 F4 6 9
SUB F8 F6 F2 7 9
DIV F10 F0 F6 8
ADD F6 F8 F2
dest S1 S2 FU for j FU for k Fj? Fk?
Time Name BusyOp Fi Fj Fk Qj Qk Rj Rk
Integer No
10 Mult1 Yes Mult F0 F2 F4 No No
Mult2 No
2 Add Yes Sub F8 F6 F2 No No
Clock F0 F2 F4 F6 F8 F10 F12 ... F30
Now MULT and
SUBD can both read
F2 as it is now available. ADD
can’t be issued because SUB uses
the adder

174
Write
Instruction j k Issueoperands
complete
Result
LD F6 34+ R2 1 2 3 4
LD F2 45+ R3 5 6 7 8
MULT
F0 F2 F4 6 9
SUB F8 F6 F2 7 9 11
DIV F10 F0 F6 8
ADD F6 F8 F2
Time
Name BusyOp Fi Fj Fk Qj Qk Rj Rk
Integer No
Mult2 No
0 Add Yes Sub F8 F6 F2 No No
Clock F0 F2 F4 F6 F8 F10 F12 ... F30
Add takes 2 cycles, so
nothing happens in cycle 10.
MUL & SUB continues.

175
Write
complete
Result
LD F6 34+ R2 1 2 3 4
LD F2 45+ R3 5 6 7 8
MULT F0 F2 F4 6 9
SUB F8 F6 F2 7 9 11 12
DIV F10 F0 F6 8
ADD F6 F8 F2
FU for k
Fj? Fk?
Time
Integer No
Mult2 No
Add No
Clock F0 F2 F4 F6 F8 F10 F12 ... F30
12 FU Mult1 Divide
SUB finishes.
DIV waiting for F0.

176
Write
complete
Result
LD F6 34+ R2 1 2 3 4
LD F2 45+ R3 5 6 7 8
MULT F0 F2 F4 6 9
SUB F8 F6 F2 7 9 11 12
DIV F10 F0 F6 8
ADD F6 F8 F2 13
Functional unit status dest S1 S2 FU for jFU for kFj? Fk?
Time
Integer No
Mult2 No
Add Yes Add F6 F8 F2 Yes Yes
Clock F0 F2 F4 F6 F8 F10 F12 ... F30
Now ADD is issued because
SUB has completed

177
Write
complete
Result
LD F6 34+ R2 1 2 3 4
LD F2 45+ R3 5 6 7 8
MULT F0 F2 F4 6 9
SUB F8 F6 F2 7 9 11 12
DIV F10 F0 F6 8
ADD F6 F8 F2 13 14
FU for k
Fj? Fk?
Time
Integer No
Mult2 No
2 Add Yes Add F6 F8 F2 No No
Clock F0 F2 F4 F6 F8 F10 F12 ... F30

178
Write
complete
Result
LD F6 34+ R2 1 2 3 4
LD F2 45+ R3 5 6 7 8
MULT F0 F2 F4 6 9
SUB F8 F6 F2 7 9 11 12
DIV F10 F0 F6 8
ADD F6 F8 F2 13 14
Time
Integer No
Mult2 No
Clock F0 F2 F4 F6 F8 F10 F12 ... F30
ADD takes 2 cycles, so no
change

179
Write
complete
Result
LD F6 34+ R2 1 2 3 4
LD F2 45+ R3 5 6 7 8
MULT F0 F2 F4 6 9
SUB F8 F6 F2 7 9 11 12
DIV F10 F0 F6 8
ADD F6 F8 F2 13 14 16
Time
Integer No
Mult2 No
Clock F0 F2 F4 F6 F8 F10 F12 ... F30
ADD completes
execution, but MULT and
DIV go on

180
Write
complete
Result
LD F6 34+ R2 1 2 3 4
LD F2 45+ R3 5 6 7 8
MULT F0 F2 F4 6 9
SUB F8 F6 F2 7 9 11 12
DIV F10 F0 F6 8
ADD F6 F8 F2 13 14 16
FU for k
Fj? Fk?
Time
Integer No
Mult2 No
Add Yes Add F6 F8 F2 No No
Clock F0 F2 F4 F6 F8 F10 F12 ... F30
ADD stalls, can’t write
back due to WAR with
DIV. MULT and DIV
continue

181
Write
complete
Result
LD F6 34+ R2 1 2 3 4
LD F2 45+ R3 5 6 7 8
MULT F0 F2 F4 6 9
SUB F8 F6 F2 7 9 11 12
DIV F10 F0 F6 8
ADD F6 F8 F2 13 14 16
FU for k
Fj? Fk?
Time
Integer No
Mult2 No
Clock F0 F2 F4 F6 F8 F10 F12 ... F30
MULT and
DIV continue

182
Write
complete
Result
LD F6 34+ R2 1 2 3 4
LD F2 45+ R3 5 6 7 8
MULTD F0 F2 F4 6 9
SUBD F8 F6 F2 7 9 11 12
DIVD F10 F0 F6 8
ADDD F6 F8 F2 13 14 16
FU for k
Fj? Fk?
Time
Integer No
Mult2 No
Clock F0 F2 F4 F6 F8 F10 F12 ... F30
19
MULT completes
after 10 cycles

183
complete
Result
LD F6 34+ R2 1 2 3 4
LD F2 45+ R3 5 6 7 8
MULTD F0 F2 F4 6 9 19 20
SUBD F8 F6 F2 7 9 11 12
DIVD F10 F0 F6 8
ADDD F6 F8 F2 13 14 16
FU for k
Fj? Fk?
Time
Integer No
Mult1 No
Mult2 No
Divide Yes Div F10 F0 F6 Yes Yes
Clock F0 F2 F4 F6 F8 F10 F12 ... F30
20 FU Add Divide
MULT completes and
writes to F0

184
complete
Result
LD F6 34+ R2 1 2 3 4
LD F2 45+ R3 5 6 7 8
MULTD F0 F2 F4 6 9 19 20
SUBD F8 F6 F2 7 9 11 12
DIVD F10 F0 F6 8 21
ADDD F6 F8 F2 13 14 16
FU for k
Fj? Fk?
Time
Integer No
Mult1 No
Mult2 No
Divide Yes Div F10 F0 F6 No No
Clock F0 F2 F4 F6 F8 F10 F12 ... F30
21 FU Add Divide
Now DIV reads because
F0 is available

185
complete
Result
LD F6 34+ R2 1 2 3 4
LD F2 45+ R3 5 6 7 8
MULTD F0 F2 F4 6 9 19 20
SUBD F8 F6 F2 7 9 11 12
DIVD F10 F0 F6 8 21
ADDD F6 F8 F2 13 14 16 22
FU for k
Fj? Fk?
Time
Integer No
Mult1 No
Mult2 No
Add No
Clock F0 F2 F4 F6 F8 F10 F12 ... F30
21 FU Divide
ADD writes result because
WAR is removed.

186
complete
Result
LD F6 34+ R2 1 2 3 4
LD F2 45+ R3 5 6 7 8
MULTD F0 F2 F4 6 9 19 20
SUBD F8 F6 F2 7 9 11 12
DIVD F10 F0 F6 8 21 61
ADDD F6 F8 F2 13 14 16 22
FU for k
Fj? Fk?
Time
Integer No
Mult1 No
Mult2 No
Add No
Clock F0 F2 F4 F6 F8 F10 F12 ... F30
61 FU Divide
DIVD completes execution
(40 cycles)

187
Write
complete
Result
LD F6 34+ R2 1 2 3 4
LD F2 45+ R3 5 6 7 8
MULTD
F0 F2 F4 6 9 19 20
SUBD F8 F6 F2 7 9 11 12
DIVD F10 F0 F6 8 21 61 62
ADDD F6 F8 F2 13 14 16 22
Integer No
Mult1 No
Mult2 No
Add No
0 Divide No
Clock F0 F2 F4 F6 F8 F10 F12 ... F30
62 FU
Execution is finished

188
An Assessment of
Scoreboarding
 Pro: A scoreboard effectively handles
true data dependencies:
 Minimizes the number of stalls due to true
data dependencies.
 Con: Anti dependences and output
dependences (WAR and WAW hazards)
are also handled using stalls:
 Could have been better handled.

189
Tomasulo’s Algorithm
Mr. SUBHASIS DASH
KIIT UNIVERSITY
BHUBANESWAR
High Performance
Computer Architecture

190
A More Sophisticated
Approach: Tomasulo’s
Algorithm
 Developed for IBM 360/91:
 Goal: To keep the floating point pipeline as
busy as possible.
 This led Tomasulo to try to figure out how to
achieve renaming in hardware!
 The descendants of this have flourished!
 Alpha 21264, HP 8000, MIPS 10000, Pentium
III, PowerPC 604, Pentium 4…

191
Key Innovations in Dynamic
Instruction Scheduling
 Reservation stations:
 Single entry buffer at the head of each
functional unit has been replaced by a multiple
entry buffer.
 Common Data Bus (CDB):
 Connects the output of the functional units to
the reservation stations as well as registers.
 Register Tags:
 Tag corresponds to the reservation station entry
number for the instruction producing the result.

192
Reservation Stations
 The basic idea:
 An instruction waits in the reservation
station, until its operands become
available.
 Helps overcome RAW hazards.
 A reservation station fetches and
buffers an operand as soon as it is
available:
 Eliminates the need to get operands from
registers.

193
 Control & buffers distributed with Function Units (FU)
 In the form of “reservation stations” associated
with every function unit.
 Store operands for issued but pending instructions.
 Registers in instructions replaced by values and
others with pointers to reservation stations (RS):
 Achieves register renaming.
 Avoids WAR, WAW hazards without stalling.
 Many more reservation stations than registers
(why?), so can do optimizations that compilers can’t.

194
cont…
 Results passed to FUs from RSs,
 Not through registers, therefore similar to
forwarding.
 Over Common Data Bus (CDB) that
broadcasts results to all FUs.
 Load and Stores:
 Treated as FUs with RSs as well.
 Integer instructions can go past branches:
 Allows FP ops beyond basic block in FP queue.

195
Tomasulo’s Scheme
Reservation
Stations
Adder Multiplier
Address Unit
Memory Unit
Registers
Load
Buffer
Instruction
Queue
From Instruction Unit
Store
Buffer
CDB

196
Three Stages of Tomasulo
Algorithm
1. Issue: Get instruction from Instruction Queue
 Issue instruction only if a matching reservation
station is free (no structural hazard).
 Send registers or the functional unit that would
produce the result (achieves renaming).
 If there is a matching reservation station that is
empty, issue the instruction to the station (otherwise
structural hazard) with the operand values, if they
are currently in the registers (otherwise keep track
of FU that will produce the operand & WAR, WAW
hazard).

197
Three Stages of Tomasulo
Algorithm
2. Execute: Operate on operands (EX)
 If one or more of the operands is not yet available,
monitor the common data bus while waiting for it to
be computed.
 When both operands ready then execute;
if not ready, watch Common Data Bus for result
 RAW hazard are avoided.
3. Write result: Finish execution (WB)
 When result is available, write it on the CDB and
from there into the registers and into the
reservation stations waiting for this result.
 Write on CDB to all awaiting units; mark
reservation station available.

198
Different parts of Tomasulo
 Instruction Status Unit :-
 Indicates which of 3 phases the instruction is in
 Reservation station Status Unit :-
 Indicates the states of reservation station & each
reservation station has 7 different fields.
1. BUSY:- Indicates weather the reservation station and its
accompanying functional unit is busy or free.
2. OP :- Indicates the type of operation to be perform in the
reservation station.
3. Vj, Vk :- Indicates the value of source operands (hold the
offset field).
4. Qj, Qk :- Indicates the reservation station that will produce
the source operand.
5. A:- Use the hold information for the memory address
calculation for the load and store.

199
Different parts of Tomasulo
 Register Result Status Unit :-
 Indicates which reservation station will write each
register, if an active instruction has the register as
its destination.
 The field Qi is set to blank whenever there are no
pending instruction that will write that register.
 The no. of reservation station contains the operation
whose result should be stored into the register.

200
Steps in Execution with Tomasulo Approach
• Write the steps of execution with tomasulo approach. Assume
we have 2 multipliers, 3 adders & 2 loader unit. The following
set of MIPS instruction is going to be executed in a pipelined
system.
• Example:
• LOAD F6, 34(R2)
• LOAD F2, 45(R3)
• MUL FO, F2, F4
• SUB F8, F6, F2
• DIV F10, F0, F6
• ADD F6, F8, F2
• Consider following execution status for above instruction set:-
• 1st LOAD instruction is completed its write result phase.
• 2nd LOAD instruction is completed its execute phase.
• Other remaining instruction are in their instruction issue phase.
• Show the status of instruction, reservation station, & register
result status using dynamic scheduling with tomasulo approach.

201
 Write the steps of execution with tomasulo approach. Assume
we have 2 multipliers, 3 adders & 2 load unit. The following set
of MIPS instruction is going to be executed in a pipelined
system.
 Example:
• LOAD F6, 34(R2)
• LOAD F2, 45(R3)
• MUL FO, F2, F4
• SUB F8, F6, F2
• DIV F10, F0, F6
• ADD F6, F8, F2
• 1st two LOAD & SUB instructions are completed their write result phase.
• MUL & DIV instructions are ready to go for their write result phase.
• ADD instruction is in its execution phase.
 Show the status of instruction, reservation station, & register
result status using dynamic scheduling with Tomasulo approach.

202
 Write the steps of execution with tomasulo approach. Assume
we have 2 multipliers, 3 adders & 2 load unit. The following set
of MIPS instruction is going to be executed in a pipelined
system.
 Example:
• LOAD F6, 34(R2)
• LOAD F2, 45(R3)
• MUL FO, F2, F4
• SUB F8, F6, F2
• DIV F10, F0, F6
• ADD F6, F8, F2
• 1st two LOAD, MUL, SUB & ADD instructions are completed their write
result phase.
• DIV instructions is ready to write its result .
 Show the status of instruction, reservation station & register
result status using dynamic scheduling with tomasulo approach.

203
• Reservation station units:-
3 LOAD, 3 ADDer, 2 MULtiplier.
• Instruction sequence:-
1. L.D F6, 34(R2)
2. L.D F2, 45(R3)
3. MUL F0, F2, F4
4. SUB F8, F6, F2
5. DIV F10, F0, F6
6. ADD F6, F8, F2
• Latencies:
– LOAD → 2 cycle
– FP add → 2 cycles
– FP multiply → 10 cycles
– FP divide → 40 cycles
Tomasulo Example

204
Tomasulo Example Cycle 0
Instruction status Execution Write
Instruction j k Issue complete Result Busy
LD F6 34+ R2 Load1 No
MULTD F0 F2 F4 Load3 No
SUBD F8 F6 F2
DIVD F10 F0 F6
ADDD F6 F8 F2
Reservation Stations S1 S2 RS for j RS for k
Time Name Busy Op Vj Vk Qj Qk
0 Add1 No
0 Add2 No
Add3 No
0 Mult1 No
0 Mult2 No
Clock F0 F2 F4 F6 F8 F10 F12 ... F30
0 FU
Address

205
LD F6 34+ R2 1 Load1 Yes
SUBD F8 F6 F2
DIVD F10 F0 F6
ADDD F6 F8 F2
0 Add1 No
0 Add2 No
Add3 No
0 Mult1 No
0 Mult2 No
Clock F0 F2 F4 F6 F8 F10 F12 ... F30
1 FU Load1
Address
34+R2

206
LD F6 34+ R2 1 2- Load1 Yes
LD F2 45+ R3 2 Load2 Yes
SUBD F8 F6 F2 Assume Load takes 2 cycles
DIVD F10 F0 F6
ADDD F6 F8 F2
0 Add1 No
0 Add2 No
Add3 No
0 Mult1 No
0 Mult2 No
Clock F0 F2 F4 F6 F8 F10 F12 ... F30
2 FU Load2 Load1
Address
34+R2
45+R3

207
LD F6 34+ R2 1 2--3 Load1 Yes
LD F2 45+ R3 2 3- Load2 Yes
MULTD F0 F2 F4 3 Load3 No
SUBD F8 F6 F2
DIVD F10 F0 F6
ADDD F6 F8 F2
0 Add1 No
0 Add2 No
Add3 No
0 Mult1 Yes Mult R(F4) Load2
0 Mult2 No
Clock F0 F2 F4 F6 F8 F10 F12 ... F30
3 FU Mult1 Load2 Load1
read value
Address
34+R2
45+R3

208
LD F6 34+ R2 1 2--3 4 Load1 No
LD F2 45+ R3 2 3--4 Load2 Yes
SUBD F8 F6 F2 4
DIVD F10 F0 F6
ADDD F6 F8 F2
0 Add1 Yes Sub M(A1) Load2
0 Add2 No
Add3 No
0 Mult1 Yes Mult R(F4) Load2
0 Mult2 No
Clock F0 F2 F4 F6 F8 F10 F12 ... F30
4 FU Mult1 Load2 M(A1) Add1
Address
45+R3

209
LD F6 34+ R2 1 2--3 4 Load1 No
LD F2 45+ R3 2 3--4 5 Load2 No
SUBD F8 F6 F2 4
DIVD F10 F0 F6 5
ADDD F6 F8 F2
2 Add1 Yes Sub M(A1) M(A2)
0 Add2 No
Add3 No
10 Mult1 Yes Mult M(A2) R(F4)
0 Mult2 Yes Div M(A1) Mult1
Clock F0 F2 F4 F6 F8 F10 F12 ... F30
5 FU Mult1 M(A2) M(A1) Add1 Mult2
Address

210
LD F6 34+ R2 1 2--3 4 Load1 No
LD F2 45+ R3 2 3--4 5 Load2 No
MULTD F0 F2 F4 3 6 -- Load3 No
SUBD F8 F6 F2 4 6 --
DIVD F10 F0 F6 5
ADDD F6 F8 F2 6
0 Add2 Yes Add M(A2) Add1
Add3 No
Clock F0 F2 F4 F6 F8 F10 F12 ... F30
6 FU Mult1 M(A2) Add2 Add1 Mult2
Address

211
LD F6 34+ R2 1 2--3 4 Load1 No
LD F2 45+ R3 2 3--4 5 Load2 No
SUBD F8 F6 F2 4 6 -- 7
DIVD F10 F0 F6 5
ADDD F6 F8 F2 6
0 Add2 Yes Add M(A2) Add1
Add3 No
Clock F0 F2 F4 F6 F8 F10 F12 ... F30
7 FU Mult1 M(A2) Add2 Add1 Mult2
Address

212
LD F6 34+ R2 1 2--3 4 Load1 No
LD F2 45+ R3 2 3--4 5 Load2 No
SUBD F8 F6 F2 4 6 -- 7 8
DIVD F10 F0 F6 5
ADDD F6 F8 F2 6
0 Add1 No
2 Add2 Yes Add M1-M2 M(A2)
Add3 No
Clock F0 F2 F4 F6 F8 F10 F12 ... F30
8 FU Mult1 M(A2) Add2 M1-M2 Mult2
Address

213
LD F6 34+ R2 1 2--3 4 Load1 No
LD F2 45+ R3 2 3--4 5 Load2 No
SUBD F8 F6 F2 4 6 -- 7 8
DIVD F10 F0 F6 5
ADDD F6 F8 F2 6 9 --
0 Add1 No
Add3 No
Clock F0 F2 F4 F6 F8 F10 F12 ... F30
Address

214
LD F6 34+ R2 1 2--3 4 Load1 No
LD F2 45+ R3 2 3--4 5 Load2 No
SUBD F8 F6 F2 4 6 -- 7 8
DIVD F10 F0 F6 5
ADDD F6 F8 F2 6 9 -- 10
0 Add1 No
Add3 No
Clock F0 F2 F4 F6 F8 F10 F12 ... F30
Address

215
LD F6 34+ R2 1 2--3 4 Load1 No
LD F2 45+ R3 2 3--4 5 Load2 No
SUBD F8 F6 F2 4 6 -- 7 8
DIVD F10 F0 F6 5
ADDD F6 F8 F2 6 9 -- 10 11
0 Add1 No
Add2 No
Add3 No
Clock F0 F2 F4 F6 F8 F10 F12 ... F30
11 FU Mult1 M(A2) M1-M2+M(A2)
M1-M2 Mult2
Address

216
LD F6 34+ R2 1 2--3 4 Load1 No
LD F2 45+ R3 2 3--4 5 Load2 No
SUBD F8 F6 F2 4 6 -- 7 8
DIVD F10 F0 F6 5
ADDD F6 F8 F2 6 9 -- 10 11
0 Add1 No
Add2 No
Add3 No
Clock F0 F2 F4 F6 F8 F10 F12 ... F30
M1-M2 Mult2
Address

217
LD F6 34+ R2 1 2--3 4 Load1 No
LD F2 45+ R3 2 3--4 5 Load2 No
MULTD F0 F2 F4 3 6 -- 15 Load3 No
SUBD F8 F6 F2 4 6 -- 7 8
DIVD F10 F0 F6 5
ADDD F6 F8 F2 6 9 -- 10 11
0 Add1 No
Add2 No
Add3 No
Clock F0 F2 F4 F6 F8 F10 F12 ... F30
M1-M2 Mult2
Address

218
LD F6 34+ R2 1 2--3 4 Load1 No
LD F2 45+ R3 2 3--4 5 Load2 No
MULTD F0 F2 F4 3 6 -- 15 16 Load3 No
SUBD F8 F6 F2 4 6 -- 7 8
DIVD F10 F0 F6 5
ADDD F6 F8 F2 6 9 -- 10 11
0 Add1 No
Add2 No
Add3 No
Mult1 No
40 Mult2 Yes Div M2*F4 M(A1)
Clock F0 F2 F4 F6 F8 F10 F12 ... F30
16 FU M*F4 M(A2) M1-M2+M(A2) M1-M2 Mult2
Address

219
LD F6 34+ R2 1 2--3 4 Load1 No
LD F2 45+ R3 2 3--4 5 Load2 No
MULTD F0 F2 F4 3 6 -- 15 16 Load3 No
SUBD F8 F6 F2 4 6 -- 7 8
DIVD F10 F0 F6 5 17 -- 56
ADDD F6 F8 F2 6 9 -- 10 11
0 Add1 No
Add2 No
Add3 No
Mult1 No
0 Mult2 Yes Div M*F4 M(A1)
Clock F0 F2 F4 F6 F8 F10 F12 ... F30
56 FU M*F4 M(A2) M1-M2+M(A2)
M1-M2 Mult2
Address

220
LD F6 34+ R2 1 2--3 4 Load1 No
LD F2 45+ R3 2 3--4 5 Load2 No
MULTD F0 F2 F4 3 6 -- 15 16 Load3 No
SUBD F8 F6 F2 4 6 -- 7 8
DIVD F10 F0 F6 5 17 -- 56 57
ADDD F6 F8 F2 6 9 -- 10 11
0 Add1 No
Add2 No
Add3 No
Mult1 No
0 Mult2 No
Clock F0 F2 F4 F6 F8 F10 F12 ... F30
57 FU M*F4 M(A2) M1-M2+M(A2)
M1-M2 result
Address

221
Tomasulo’s Scheme- Loop
Example
Loop: LD F0 0 R1
MULTD F4 F0 F2
SD F4 0 R1
SUBI R1 R1 #8
BNEZ R1 Loop

222
LD F0 0 R1
MULTD F4 F0 F2
SD F4 0 R1
LD F0 0 R1
MULTD F4 F0 F2
SD F4 0 R1

224
Tomasulo’s Scheme:
Drawbacks
 Performance is limited by CDB:
 CDB connects to multiple functional units
high capacitance, high wiring density
 Number of functional units that can
complete per cycle limited to one!
 Multiple CDBs  more FU logic for parallel
stores.
 Imprecise exceptions!
 Effective handling is a major performance
bottleneck.

225
Why Can Tomasulo’s Scheme
Overlap Iterations of Loops?
 Register renaming using reservation
stations:
Avoids the WAR stall that occurred in
the scoreboard.
Also, multiple iterations use different
physical destinations facilitating
dynamic loop unrolling.

226
Tomasulo’s Scheme Offers
Three Major Advantages
• 1. Distribution of hazard detection logic:
 Distributed reservation stations.
 If multiple instructions wait on a single result,
 Instructions can be passed simultaneously by broadcast
on CDB.
 If a centralized register file were used,
 Units would have to read their results from registers .
 2. Elimination of stalls for WAW and WAR
hazards.
 3. Possible to have superscalar execution:
 Because results directly available to FUs, rather
than from registers.

227
Superscalar and VLIW
Processors
Mr. Subhasis Dash
School Of Computer Engineering
Kiit University
Bhubaneswar
High Performance
Computer Architecture

228
Two Paths to Higher ILP
 Superscalar processors:
 Multiple issue, dynamically scheduled,
speculative execution, branch prediction
 More hardware functionalities and
complexities.
 VLIW:
 Let complier take the complexity.
 Simple hardware, smart compiler.

229
VLIW Processors
 The compiler has complete
responsibility of selecting a set of
instructions:
 These can be concurrently be executed.
 VLIW processors have static
instruction issue capability:
 As compared, superscalar processors
have dynamic issue capability.

230
The Basic VLIW
Approach
 VLIW processors deploy multiple
independent functional units.
 Early VLIW processors operated
lock step:
 There was no hazard detection in
hardware at all.
 A stall in any functional unit causes
the entire pipeline to stall.

231
VLIW Processors
 Assume a 4-issue static superscalar
processor:
 During fetch stage, 1 to 4 instructions would
be fetched.
 The group of instructions that could be issued
in a single cycle are called:
 An issue packet or a Bundle.
 If an instruction could cause a structural or
data hazard:
 It is not issued.

232
VLIW (Very Long Instruction
Word)
 One single VLIW instruction:
 separately targets differently functional units.
 MultiFlow TRACE, TI C6X, IA-64
add r1,r2,r3
FU FU FU FU
load r4,r5+4 mov r6,r2 mul r7,r8,r9
VLIW Instruction
Schematic Explanation for a
Bundle

233
Example of Scheduling by the Compiler
How these operations would be scheduled by
compiler?
1. ADD r1, r2, r3
2. SUB r16, r14, r7
3. LD r2, (r4)
4. LD r14, (r15)
5. MUL r5, r1, r9
6. ADD r9, r10, r11
7. SUB r12, r2, r14
Assume:-
•Number of execution units = 3
•All operations has latency = 2 cycles
•Any execution unit can execute any operation

235
VLIW Processors: Some
Considerations
 Issue hardware is simpler.
 Compiler has a bigger context from which to
select co-scheduled instructions.
 Compilers, however, do not have runtime
information such as cache misses.
 Scheduling is, therefore, inherently conservative.
 Branch and memory prediction is more difficult.
 Typical VLIW processors are limited to 4-way
to 8-way parallelism.

236
VLIW Summary
 Each “instruction” is very large
 Bundles multiple operations that are independent.
 Complier detects hazard, and determines
scheduling.
 There is no (or only partial) hardware hazard
detection:
 No dependence check logic for instructions issued
at the same cycle.
 Tradeoff instruction space for simple decoding
 The long instruction word has room for many
operations.
 But have to fill with NOP if enough operations
cannot be found.

237
VLIW vs Superscalar
 VLIW - Compiler finds parallelism:
 Superscalar – hardware finds parallelism
 VLIW – Simpler hardware:
 Superscalar – More complex hardware
 VLIW – less parallelism can be
exploited for a typical program:
 Superscalar – Better performance

238
Superscalar Execution
 Scheduling of instructions is determined by a
number of factors:
 True Data Dependency: The result of one
operation is an input to the next.
 Resource constraints: Two operations require the
same resource.
 Branch Dependency: Scheduling instructions
across conditional branch statements cannot be
done deterministically a-priori.
 An appropriate number of instructions issued.
 Superscalar processor of degree m.

239
Superscalar Execution With
Dynamic Scheduling
 Multiple instruction issue:
 Very well accommodated with dynamic
instruction scheduling approach.
 The issue stage can be:
 Replicated, pipelined, or both.

240
A Superscalar MIPS Processor
 Assume two instructions can be
issued per clock cycle:
 One of the instructions can be load,
store, or integer ALU operations.
 The other can be a floating point
operation.

241
Limitations of Scalar Pipelines:
A Reflection
 Maximum throughput bounded by one
instruction per cycle.
 Inefficient unification of instructions
into one pipeline:
 ALU, MEM stages very diverse eg: FP
 Rigid nature of in-order pipeline:
 If a leading instruction is stalled every
subsequent instruction is stalled

242
A Rigid Pipeline
Stalled Instruction
Backward
Propagation
of stalling
Bypassing
of stalled
instruction
not allowed

243
Solving Problems of Scalar
Pipelines: Modern Processors
 Maximum throughput bounded by one
instruction per cycle:
 parallel pipelines (superscalar)
 Inefficient unification into a single
pipeline:
 diversified pipelines.
 Rigid nature of in order pipeline
 Allow out of ordering or dynamic
instruction scheduling.

244
Difference Between Superscalar
and Super-Pipelined
• Super-Pipelined
– Many pipeline stages need less than half a
clock cycle
– Double internal clock speed gets two tasks per
external clock cycle
• Superscalar
– Allows for parallel execution of independent
instructions

245
Difference
between
Superscalar
and Super-
pipelined

246
Machine Parallelism
(a) No Parallelism (Nonpipelined)
(b) Temporal Parallelism
(Pipelined)
(c) Spatial Parallelism (Multiple
units)
(d) Combined Temporal and
Spatial Parallelism

247
Scalar and Parallel Pipeline
(a) The five-stage i486
scalar pipeline
(b) The five-stage Pentium
Parallel Pipeline of
width=2

module2.ppt

More Related Content

Similar to module2.ppt (20)

Recently uploaded (20)

module2.ppt