ECE 4750 Computer Architecture Topic 3: Pipelining Structural & Data Hazards

Transcription

1 ECE 4750 Computer Architecture Topic 3: Pipelining Structural & Data Hazards Christopher Batten School of Electrical and Computer Engineering Cornell University slide revision:

2 Pipelining Basics Structural Hazards Data Hazards Iron Law of Processor Performance Time Program = Instructions Program Cycles Instruction Time Cycles Microarchitecture CPI Cycle Time topic 2 Microcoded >1 short topic 3 Single-Cycle Unpipelined 1 long topic 3 Multi-Cycle Unpipelined >1 short topics 4 6 Pipelined 1 short ECE 4750 T03: Pipelining Structural & Data Hazards 2 / 35

3 Pipelining Basics Structural Hazards Data Hazards CPI Of Various Microarchitectures ECE 4750 T03: Pipelining Structural & Data Hazards 3 / 35

4 Pipelining Basics Structural Hazards Data Hazards Agenda Pipelining Basics Structural Hazards Data Hazards ECE 4750 T03: Pipelining Structural & Data Hazards 4 / 35

5 Pipelining Basics Structural Hazards Data Hazards An Ideal Pipeline stage 1 stage 2 stage 3 stage 4 All objects go through the same stages No sharing of resources between any two stages Propagation delay through all pipeline stages is equal Scheduling of a transaction entering pipeline is not affected by transactions in other stages These conditions generally hold for industry assembly lines, but instructions depend on each other causing various hazards ECE 4750 T03: Pipelining Structural & Data Hazards 5 / 35

6 Pipelining Basics Structural Hazards Data Hazards Pipelined MIPS Processor: Start With Multi-Cycle Unpipelined Processor Clock period is reduced by dividing the execution of an instruction into multiple cycles t c < max(t ifetch, t rf, t ALU, t dmem, t rfwr ) ECE 4750 T03: Pipelining Structural & Data Hazards 6 / 35

7 Pipelining Basics Structural Hazards Data Hazards Pipelined MIPS Processor: Add Pipeline Registers to Control Unit As instruction goes down the pipeline, fewer control bits are needed ECE 4750 T03: Pipelining Structural & Data Hazards 7 / 35

8 Pipelining Basics Structural Hazards Data Hazards Pipelining Technology Assumptions Small amount of very fast memory (caches), backed by large, slower memory Fast ALU (at least for integers) Multiported register files (slower!) Thus, the following timing assumption is reasonable t IM t RF t ALU t DM t RW A 5-stage pipeline will be the focus of our detailed design, although real commercial designs usually have many more stages ECE 4750 T03: Pipelining Structural & Data Hazards 8 / 35

9 Pipelining Basics Structural Hazards Data Hazards Pipeline Diagrams: Block Diagram We need to be able to show multiple simultaneous transactions in both space and time ECE 4750 T03: Pipelining Structural & Data Hazards 9 / 35

10 Pipelining Basics Structural Hazards Data Hazards Pipeline Diagrams: Transactions vs. Time time t0 t1 t2 t3 t4 t5 t6 t7.... instruction1 IF 1 ID 1 EX 1 MA 1 WB 1 instruction2 IF 2 ID 2 EX 2 MA 2 WB 2 instruction3 IF 3 ID 3 EX 3 MA 3 WB 3 instruction4 IF 4 ID 4 EX 4 MA 4 WB 4 instruction5 IF 5 ID 5 EX 5 MA 5 WB 5 ECE 4750 T03: Pipelining Structural & Data Hazards 10 / 35

11 Pipelining Basics Structural Hazards Data Hazards Pipeline Diagrams: Space vs. Time Resources time t0 t1 t2 t3 t4 t5 t6 t7.... IF I 1 I 2 I 3 I 4 I 5 ID I 1 I 2 I 3 I 4 I 5 EX I 1 I 2 I 3 I 4 I 5 MA I 1 I 2 I 3 I 4 I 5 WB I 1 I 2 I 3 I 4 I 5 ECE 4750 T03: Pipelining Structural & Data Hazards 11 / 35

12 Pipelining Basics Structural Hazards Data Hazards Instructions Interact With Each Other in Pipeline Structural Hazard An instruction in the pipeline needs a resource being used by another instruction in the pipeline Data Hazard An instruction depends on a data value produced by an earlier instruction Control Hazard Whether or not an instruction should be executed depends on a control decision made by an earlier instruction ECE 4750 T03: Pipelining Structural & Data Hazards 12 / 35

13 Pipelining Basics Structural Hazards Data Hazards Agenda Pipelining Basics Structural Hazards Data Hazards ECE 4750 T03: Pipelining Structural & Data Hazards 13 / 35

14 Pipelining Basics Structural Hazards Data Hazards Overview Structural Hazards Structural hazards occur when two instructions need the same hardware resource at the same time Approaches to resolving structural hazards Schedule Programmer explicitly avoids scheduling instructions that would create structural hazards Stall Hardware includes control logic that stalls until earlier instruction is no longer using contended resource Duplicate Add more hardware to design so that each instruction can access to independent resources at the same time Simple 5-stage MIPS pipeline has no structural hazards specifically because ISA was designed that way ECE 4750 T03: Pipelining Structural & Data Hazards 14 / 35

15 Pipelining Basics Structural Hazards Data Hazards Example Structural Hazard: Unified Memory Pipeline diagram on board ECE 4750 T03: Pipelining Structural & Data Hazards 15 / 35

16 Pipelining Basics Structural Hazards Data Hazards Example Structural Hazard: 2-Cycle Data Memory Pipeline diagram on board ECE 4750 T03: Pipelining Structural & Data Hazards 16 / 35

17 Agenda Pipelining Basics Structural Hazards Data Hazards ECE 4750 T03: Pipelining Structural & Data Hazards 17 / 35

18 Overview of Data Hazards Data hazards occur when one instruction depends on a data value produced by an preceding instruction still in the pipeline Approaches to resolving data hazards Schedule Programmer explicitly avoids scheduling instructions that would create data hazards Stall Hardware includes control logic that freezes earlier stages until preceding instruction has finished producing data value Bypass Hardware datapath allows values to be sent to an earlier stage before preceding instruction has left the pipeline Speculate Guess that there is not a problem, if incorrect kill speculative instruction and restart ECE 4750 T03: Pipelining Structural & Data Hazards 18 / 35

19 Example Data Hazard r4 r1 r1 0x4 Add 31 PC addr inst Inst Memory we rs1 rs2 rd1 ws wd rd2 GPRs Imm Ext A B ALU Y we addr rdata Data Memory wdata wdata R MD1 MD2... r1 r r4 r Pipeline diagram on board ECE 4750 T03: Pipelining Structural & Data Hazards 19 / 35

20 Feedback to Resolve Hazards FB 1 FB 2 FB 3 FB 4 stage 1 stage 2 stage 3 stage 4 Later stages provide dependence information to earlier stages which can stall (or kill) following instructions Controlling a pipeline in this manner works provided the instruction at stage i+1 can complete without any interaction from instructions in stages 1 to i (otherwise deadlock) ECE 4750 T03: Pipelining Structural & Data Hazards 20 / 35

21 Resolving Data Hazards with Stalls Stall Condition 0x4 Add nop 31 PC addr inst Inst Memory... r1 r r4 r we rs1 rs2 rd1 ws wd rd2 GPRs Imm Ext A B MD1 ALU Y MD2 we addr rdata Data Memory wdata wdata Pipeline diagram on board R ECE 4750 T03: Pipelining Structural & Data Hazards 21 / 35

22 Stalled Stages and Pipeline Bubbles time t0 t1 t2 t3 t4 t5 t6 t7.... (I 1 ) r1 (r0) + 10 IF 1 ID 1 EX 1 MA 1 WB 1 (I 2 ) r4 (r1) + 17 IF 2 ID 2 ID 2 ID 2 ID 2 EX 2 MA 2 WB 2 (I 3 ) IF 3 IF 3 IF 3 IF 3 ID 3 EX 3 MA 3 WB 3 (I 4 ) stalled stages IF 4 ID 4 EX 4 MA 4 WB 4 (I 5 ) IF 5 ID 5 EX 5 MA 5 WB 5 Resource Usage time t0 t1 t2 t3 t4 t5 t6 t7.... IF I 1 I 2 I 3 I 3 I 3 I 3 I 4 I 5 ID I 1 I 2 I 2 I 2 I 2 I 3 I 4 I 5 EX I 1 nop nop nop I 2 I 3 I 4 I 5 MA I 1 nop nop nop I 2 I 3 I 4 I 5 WB I 1 nop nop nop I 2 I 3 I 4 I 5 nop pipeline bubble ECE 4750 T03: Pipelining Structural & Data Hazards 22 / 35

23 Stall Control logic stall C stall ws rs rt? 0x4 Add nop 31 PC addr inst Inst Memory we rs1 rs2 rd1 ws wd rd2 GPRs Imm Ext A B ALU Y we addr rdata Data Memory wdata wdata R MD1 MD2 Compare the source registers of the instruction in the decode stage with the destination register of the uncommitted instructions ECE 4750 T03: Pipelining Structural & Data Hazards 23 / 35

24 Stall Control logic (ignoring jumps/branches) stall ws we C stall rs rt? re1 re2 we C dest ws we C dest ws 0x4 Add C re nop 31 PC addr inst Inst Memory we rs1 rs2 rd1 ws wd rd2 GPRs Imm Ext A B ALU Y we addr rdata Data Memory wdata wdata C dest R MD1 MD2 Always stall if rs field matches some rd? Not every instruction writes or reads a register! ECE 4750 T03: Pipelining Structural & Data Hazards 24 / 35

25 Source and Destination Registers srcs dest ALU R[rd] R[rs] func R[rt] rs, rt rd ALUI R[rt] R[rs] op immediate rs rt LW R[rt] M[ R[rs] + offset ] rs rt SW M[ R[rs] + offset ] R[rt] rs, rt BEQZ if ( R[rs] == 0 ) PC PC+4 + offset*4 rs J PC jtarg(pc,imm) JAL R[31] PC; PC jtarg(pc,imm) 31 JR PC R[rs] rs JALR R[31] PC, PC R[rs] rs 31 ECE 4750 T03: Pipelining Structural & Data Hazards 25 / 35

26 Deriving the Stall Signal C dest ws = Case opcode ALU rd ALUi, LW rt JAL, JALR R31 we = Case opcode ALU, ALUi, LW (ws 0) JAL, JALR on... off C re re1 = Case opcode ALU, ALUi, LW, SW, BZ, JR, JALR on J, JAL off re2 = Case opcode ALU, SW on... off C stall stall = ((rs D =ws E ).we E + (rs D =ws M ).we M + (rs D =ws W ).we W ). re1 D + ((rt D =ws E ).we E + (rt D =ws M ).we M + (rt D =ws W ).we W ). re2 D This is not the full story! ECE 4750 T03: Pipelining Structural & Data Hazards 26 / 35

27 Data Hazards Due to Loads and Stores Stall Condition 0x4 Add nop 31 PC addr inst Inst Memory we rs1 rs2 rd1 ws wd rd2 GPRs Imm Ext A B ALU Y we addr rdata Data Memory wdata R... M[(r1)+7] (r2) r4 M[(r3)+5]... MD1 MD2 Is there any possible data hazard in this instruction sequence? ECE 4750 T03: Pipelining Structural & Data Hazards 27 / 35

28 Data Hazards Due to Loads and Stores Example instruction sequence M[ R[r1] + 7 ] R[r2] R[r4] M[ R[r3] + 5 ] What if R[r1]+7 == R[r3]+5? Writing and reading from the same address Hazard is avoided because our memory system completes writes in a single cycle More realistic memory system will require more careful handling of data hazards due to loads and stores Pipeline diagram on board ECE 4750 T03: Pipelining Structural & Data Hazards 28 / 35

29 Adding a Bypass to the Datapath 0x4 Add stall r4 r1... nop r1... E M W 31 PC addr inst Inst Memory D we rs1 rs2 rd1 ws wd rd2 GPRs Imm Ext ASrc A B ALU Y we addr rdata Data Memory wdata R MD1 MD2... (I 1 ) r1 r (I 2 ) r4 r When does this bypass help? r1 M[r0 + 10] r4 r JAL 500 r4 r Pipeline diagram on board ECE 4750 T03: Pipelining Structural & Data Hazards 29 / 35

30 Deriving the Bypass Signal time t0 t1 t2 t3 t4 t5 t6 t7.... (I 1 ) r1 r IF 1 ID 1 EX 1 MA 1 WB 1 (I 2 ) r4 r IF 2 ID 2 ID 2 ID 2 ID 2 EX 2 MA 2 WB 2 (I 3 ) IF 3 IF 3 IF 3 IF 3 ID 3 EX 3 MA 3 (I 4 ) stalled stages IF 4 ID 4 EX 4 (I 5 ) IF 5 ID 5 Each stall or kill introduces a bubble in the pipeline CPI > 1 A new datapath, i.e., a bypass, can get the data from the output of the ALU to its input time t0 t1 t2 t3 t4 t5 t6 t7.... (I 1 ) r1 r IF 1 ID 1 EX 1 MA 1 WB 1 (I 2 ) r4 r IF 2 ID 2 EX 2 MA 2 WB 2 (I 3 ) IF 3 ID 3 EX 3 MA 3 WB 3 (I 4 ) IF 4 ID 4 EX 4 MA 4 WB 4 (I 5 ) IF 5 ID 5 EX 5 MA 5 WB 5 ECE 4750 T03: Pipelining Structural & Data Hazards 30 / 35

31 Deriving the Bypass Signal stall = ( ((rs D =ws E ).we E + (rs D =ws M ).we M + (rs D =ws W ).we W ).re1 D +((rt D =ws E ).we E + (rt D =ws M ).we M + (rt D =ws W ).we W ).re2 D ) ws = Case opcode ALU rd ALUi, LW rt JAL, JALR R31 we = Case opcode ALU, ALUi, LW (ws 0) JAL, JALR on... off ASrc = (rs D =ws E ).we E.re1 D Is this correct? No, because only ALU and ALUI instruction can benefit from this bypass Split we E into two components: we-bypass and we-stall ECE 4750 T03: Pipelining Structural & Data Hazards 31 / 35

32 Bypass and Stall Signals Split we E into two components: we-bypass and we-stall we-bypass E = Case opcode E ALU, ALUi (ws 0)... off we-stall E = Case opcode E LW (ws 0) JAL, JALR on... off ASrc = (rs D =ws E ).we-bypass E. re1 D stall = ((rs D =ws E ).we-stall E + (rs D =ws M ).we M + (rs D =ws W ).we W ). re1 D +((rt D = ws E ).we E + (rt D = ws M ).we M + (rt D = ws W ).we W ). re2 D ECE 4750 T03: Pipelining Structural & Data Hazards 32 / 35

33 Fully Bypassed Datapath stall PC for JAL,... 0x4 Add nop ASrc E M W 31 PC addr inst Inst Memory Is there still a need for the stall signal? D we rs1 rs2 rd1 ws wd rd2 GPRs Imm Ext BSrc A B MD1 ALU Y MD2 we addr rdata Data Memory wdata stall = (rs D =ws E ). (opcode E =LW E ).(ws E 0 ).re1 D + (rt D =ws E ). (opcode E =LW E ).(ws E 0 ).re2 D R ECE 4750 T03: Pipelining Structural & Data Hazards 33 / 35

34 Summary Structural Hazard An instruction in the pipeline needs a resource being used by another instruction in the pipeline (this topic) Data Hazard An instruction depends on a data value produced by an earlier instruction (this topic) Control Hazard Whether or not an instruction should be executed depends on a control decision made by an earlier instruction (next topic) ECE 4750 T03: Pipelining Structural & Data Hazards 34 / 35

35 Acknowledgements Some of these slides contain material developed and copyrighted by: Arvind (MIT), Krste Asanović (MIT/UCB), Joel Emer (Intel/MIT) James Hoe (CMU), John Kubiatowicz (UCB), David Patterson (UCB) MIT material derived from course UCB material derived from courses CS152 and CS252 ECE 4750 T03: Pipelining Structural & Data Hazards 35 / 35