OpenSoC Fabric: On-Chip Network Generator

Size: px

Start display at page:

Download "OpenSoC Fabric: On-Chip Network Generator"

Ruby McDaniel
8 years ago
Views:

1 OpenSoC Fabric: On-Chip Network Generator Using Chisel to Generate a Parameterizable On-Chip Interconnect Fabric Farzad Fatollahi-Fard, David Donofrio, George Michelogiannakis, John Shalf MODSIM 2014 Presentation 2014 Aug 13

Fabric Farzad Fatollahi-Fard, David Donofrio, George

2 Abstract Machine Model For Emerging Node Architectures The number of cores on a chip will be on the order of 1000s Throughput Optimized Cores (Thin Cores) Massively Parallel,Simple Core Latency Optimized Core (Fat Cores) Coherence Domain Maintaining cache coherence is NOT scalable Expect coherence domains Flat and infinitely fast on-chip interconnect is NO longer practical Expect complex NOCs Processing elements within a node are NOT equidistant. Expect non-uniformity Download the CAL AMM doc from

coherence is NOT scalable Expect coherence domains Flat and infinitely fast on-chip interconnect is NO longer practical Expect

3 Motivation Straw-man Exascale Processor Shekhar Borkar, IPDPS 2013 Simplest Core First level of hierarchy RF C C C C * Shared Cache Logic C C C C 600K Transistors Logic Processor Next level of hierarchy Interconnect Next level cache Next level of hierarchy Interconnect Next level cache Next level of hierarchy Interconnect Next level cache Next level of hierarchy Interconnect Last level cache Interconnect Next level cache Next level of hierarchy Interconnect Next level cache Technology 7nm, 2018 Die area 500 mm2 Cores 2048 Frequency 4.2 GHz TFLOPs 17.2 Power 600 Watts E Efficiency 34 pj/flop Computations alone consume 34 MW for Exascale 41

Interconnect Next level cache Next level of hierarchy Interconnect Last level cache Interconnect Next level cache Next level of hierarchy Interconnect Next level

4 Impact of NoCs Application Performance Dual FPMA Cs 34% Power Router s and links 26% Clock distrib ution 10% An analysis of on-chip interconnection networks for large-scale chip multiprocessors ACM Transactions on computer architecture and code optimization (TACO), April 2010 IMEM and DMEM 20% 10-port RF 10% A 5-GHz Mesh Interconnect for a Teraflops Processor. IEEE Micro. 2007

multiprocessors ACM Transactions on computer architecture and code optimization (TACO), April

5 Evaluation Infrastructure Software simulation slow for large-scale chips and systems Also does not evaluate cycle time Hardware emulation requires a large development effort Offers limited internal visibility for statistics 5 A complexity-effective architecture for accelerating full-system multiprocessor simulations using FPGAs. FPGA 2008

effort Offers limited internal visibility for statistics 5 A complexity-effective

6 Building an SoC from IP Logic Blocks It s Legos with a some extra integration and verification cost Processor Core (ARM, Tensilica, MIPS deriv) With extra options like DP FPU, ECC OpenSoC Fabric (on-chip network) (currently proprietary ARM or Arteris) DDR memory controller (Denali/Cadence, SiCreations) + Phy & Programmable PLL PCIe Gen3 Root complex Memory DRAM Memory DRAM memctl memctl PCIe FLASH ctl IB or GigE IB or GigE Integrated FLASH Controller 6 10GigE or IB DDR 4x Channel

or Arteris) DDR memory controller (Denali/Cadence, SiCreations) + Phy & Programmable PLL PCIe Gen3 Root complex Memory

7 Design Priorities Primary goal is re-configurability Provide a powerful collection of parameters Make creating new modules and replacing existing ones easy Fast verification of hardware and software models Provide standardized connection interface Invite community to participate and contribute 7

ones easy Fast verification of hardware and software models Provide

8 Abstract Block Diagram CPU(s) CPU(s) CPU(s) AXI AXI AXI CPU(s) AXI OpenSoC Fabric AXI CPU(s) AXI AXI AXI PCIe 10GbE HMC 8

9 Example Instantiation class MyOpenSOC extends Module {! val mycpus = Vec.fill(5) {Module(new CPU())}! val myhmc = Module(new HMC())! val mytengbe = Module(new TenGbE())! val mypcie = Module(new PCIe())! val NumNodes = 8!! val OpenSoCFabric = Module(new MyOpenSoCFabric(NumNodes))!! for (i <- 0 until 4) {! mycpus(i).io.axiport <> MyOpenSoCFabric.io.AXI(i)! }! myhmc.io.axiport <> MyOpenSoCFabric.io.AXI(5)! mytengbe.io.axiport <> MyOpenSoCFabric.io.AXI(6)! mypcie.io.axiport <> MyOpenSoCFabric.io.AXI(7)! }! 9

! val OpenSoCFabric = Module(new MyOpenSoCFabric(NumNodes))!! for (i <- 0 until 4) {! mycpus(i).io.axiport <> MyOpenSoCFabric.

10 What is Chisel? Constructing Hardware In a Scala Embedded Language An open-source hardware construction language developed at the ASPIRE Lab at UC Berkeley Hierarchical + Object Oriented + Functional Construction Generates both software and hardware Easy functional verification with software model (C++) Use hardware (Verilog) in FPGAs or ASICs import Chisel._!! class GCD extends Module {! val io = new Bundle {! val a = UInt(INPUT, 16)! val b = UInt(INPUT, 16)! val e = Bool(INPUT)! val z = UInt(OUTPUT, 16)! val v = Bool(OUTPUT)! }! val x = Reg(UInt())! val y = Reg(UInt())! when (x > y) { x := x - y }! unless (x > y) { y := y - x }! when (io.e) { x := io.a; y := io.b }! io.z := x! io.v := y === UInt(0)! }!! object Example {! def main(args: Array[String]): Unit = {! chiselmain(args,! () => Module(new GCD()))! }! }!

$Generates both software and hardware Easy functional verification with software model (C++) Use hardware (Verilog) in FPGAs or ASICs import Chisel._!! class GCD extends Module {!$

11 ASICs in Chisel Raven 28nm Processor Site Clock test DCDC site test site SRAM test site

12 Why Chisel? For thousand-core chips, neither software or hardware models suffice alone Software models too slow and do not regard hardware complexity Hardware RTL too labor intensive and collecting internal statistics can be difficult Software Compilation Chisel Scala Hardware Compilation Chisel provides both models from the same codebase C++ Simulation SystemC Simulation Verilog Provides hierarchical design for easy parameterization and module replacement SST FPGA ASIC 12

hardware complexity Hardware RTL too labor intensive and collecting internal statistics can be difficult Software

13 Fabric Internal Block Diagram Top-Level Network Interface Parameters Router Topology Generator Routing Function Allocator Arbiter Arbiter Channel Channel Router Switch Router Channel Channel AXI AXI Network Interface Network Interface... Channel InjectionQ Channel... Channel EjectionQ Channel Network Interface Network Interface AXI AXI InjectionQ EjectionQ 13

Channel Channel AXI AXI Network Interface Network Interface... Channel InjectionQ...... Channel... Channel EjectionQ.

14 Class Hierarchy Diagram Flow of Instantiation Top-Level Network Network Interface Injection/ Ejection Queues Topology Other PIF Routing Function Router AXI Torus Flatten Butterfly Mesh Allocator Switch Arbiter Cyclic 14 Priority Round Robin

Topology Other PIF Routing Function Router AXI Torus

15 Existing NoC Generators A few examples such as: Arteris (FlexNoC) ARM (CoreLink and CoreSight) Orchestra (SoC generator) Academic tools (Verilog for FPGA) Our project will be open source Functionality can be extended by users Both software and hardware models 15

tools (Verilog for FPGA) Our project will be open source

16 Current Status All major architectural components are complete and tested Each with individual testers Initial network is simple mesh network Dimension ordered routing Wormhole buffered flow control Currently debugging network functionality and adding features Implementing and testing Virtual Channels Implementing and testing different topologies (Flattened Butterfly, Torus, etc.) 16 Developing more sophisticated testbenches to ensure robustness

network functionality and adding features Implementing and testing Virtual Channels Implementing and testing

17 Future Work Continue to add network features Adaptive routing Additional topologies And so much more! Expand to support circuit-switched networks Enables optical networking studies Validate against Booksim Booksim has been validated against RTL implementations Complete AXI Interface Enable integration with ARM-based cores Integrate into multi-core SystemC based software models Include HMC, DRAM, and NVRAM memory endpoints Use ASIC flow to incorporate power and area calculations into software model

validated against RTL implementations Complete AXI Interface Enable integration with ARM-based cores Integrate into

18 Questions? For more information, please visit:

Build an Energy Efficient Supercomputer from Items You can Find in Your Home (Sort of)!

Build an Energy Efficient Supercomputer from Items You can Find in Your Home (Sort of)! Marty Deneroff Chief Technology Officer Green Wave Systems, Inc. deneroff@grnwv.com 1 Using COTS Intellectual Property,