OpenSPARC T1 on Xilinx FPGAs Updates. RAMP Retreat August 2008, Stanford

Transcription

1 OpenSPARC T1 on Xilinx FPGAs Updates Thomas Thatcher OpenSPARC Engineering Paul Hartke Xilinx University Program RAMP Retreat August 2008, Stanford

2 Agenda Quick OpenSPARC Overview Progress timeline Current Status > OpenSPARC T1 1.6 Release > OpenSPARC Book > OpenSPARC FPGA Board > Multi-core T1 design > T1 core on BEE3 Roadmap Q & A 2

3 What is OpenSPARC? Open-Sourced versions of Sun's Microprocessor Products > RTL, Verification Env,documentation, system software > Available for download at Two Processors Available > OpenSPARC T1 > 8 cores, 4 hardware threads per core > 1 floating-point unit external to core, shared by all cores > 4 banks of L2 cache > OpenSPARC T2 > 8 cores, 8 hardware threads per core > Floating-point internal to core, one per core > 8 banks of L2 cache 3

4 OpenSPARC T1 SPARC V9 implementation Eight cores, four threads each 32 simultaneous threads All cores connect through a GB/s crossbar switch High BW 12-way associative 3 MB on-chip L2 cache 4 DDR2 channels (23 GB/s) 70W power ~300M transistors 4

5 Sun/Xilinx Partnership: Big Goals Proliferation of OpenSPARC technology Proliferation of Xilinx FPGA technology Make OpenSPARC FPGA friendly > Create reference design with complete system functionality > Boot Solaris/Linux on the reference design > Open it up > Seed ideas in the community Enable multi-core research 5

6 Timeline July 06 Jan 07 June 07 Jan 08 Aug 08 OpenSPARC T1 Sun/Xilinx OpenSpARC Stand-alone OpenSolaris on Collaboration T1 on ML411 program under ML411 board Begins board hypervisor First ML505 Support Today 6

7 New Developments OpenSPARC T1 1.6 Release OpenSPARC Book New OpenSPARC Development Kit > ML505 board with XC5VLX110T FPGA Multi-core Design OpenSPARC T1 core running on BEE3 Board 7

8 OpenSPARC T1 1.6 Release Released May, 2008 Implementation of 4-thread T1 core on Virtex 5 FPGAs > ML505-V5LX110T board > EDK Project files (for EDK 9.2) > Scripts to run complete RTL regression on hardware Complete setup to boot Solaris > Networking support, including telnet and ftp Quick start ace files included > Creates an out-of-the-box experience > T1 core boots OpenSolaris in 30 minutes 8

9 Hardware Block Diagram FPGA Boundary MultiPort Memory Controller External DDR2 Dimm Xilinx Embedded Developer s (EDK) Design Cache-processor interface (CPX) MCH-OPB MemCon SPARC T1 Core CCX-FSL Interface Microblaze Proc Microblaze Debug UART SPARC T1 UART processorcache interface (PCX) Fast Simplex Links interface (FSL) IBM Coreconnect OPB Bus 10/100 Ethernet Developed and Working 9

10 Software Setup OpenSolaris is booted from a RAM disk Image Memory Allocation: > 1 MB used by Microblaze firmware > 1 MB used for OpenSPARC Boot PROM image > 80 MB for RAM disk image > Leaving 174 MB for OpenSPARC RAM Microblaze firmware does address translation to map SPARC addresses to board addresses. 10

11 OpenSPARC Development Kit A kit for OpenSPARC development now available > Board based on the ML505, but with an XC5VLX110T FPGA > Includes USB interface for FPGA programming > Tested with OpenSPARC T1 release 1.6 release design > Eliminates the need to buy a board and then upgrade the FPGA Shipping now! Kit Includes: > Board, with power supply and 256 MB DRAM > Platform USB download cable > Host to host SATA crossover cable > Compact flash card with OpenSPARC T1 1.6 ace files 11

12 Kit Contents 12

13 OpenSPARC Kit Donation Program Sun will donate OpenSPARC Development Kits to qualified universities > Web address below: See the web page for more details Also available from directly from Digilent >

14 OpenSPARC Internals Book Covers both OpenSPARC T1 and T2 Includes > Architectural Overview > Development environments for OpenSPARC > Source (RTL) code overview > Configuring, extending, and verifying OpenSPARC > Porting operating systems to OpenSPARC 350 Pages Available in both hardcopy (Amazon.com) and PDF format Sign-up sheet for early release PDF copy (by poster) 14

15 Implementing a Multi-core design We have created a multi-core system by interconnecting two boards. > Opens the door to multi-core designs on BEE3 board Uses Xilinx Aurora link-layer protocol running over RocketIO GTP serial tranceivers > Connected through the SATA connectors on the board Each GTP channel is 16 bits at 75 Mhz > Connected to Microblaze through an FSL FIFO 15

16 Multi-core System Block Diagram FPGA Boundary Xilinx CacheLink (XCL) External DDR2 Dimm Xilinx Embedded Developer s (EDK) Design Fast Simplex Links (FSL) MemCon SPARC T1 Core CCX-FSL Interface Microblaze Proc Microblaze Debug UART Aurora over GTP SPARC T1 UART Ethernet FSL connected Aurora-over-GTP module to connect to other board Developed and Working New 16

17 Dual-core system implementation SATA Cable Master Node 17

18 Four-core system implementation SATA Cable SMA cables SATA Cable Master Node 18

19 Initial Configuration Master FPGA hosts entire OpenSPARC Address space. > However, Each client MicroBlaze will run firmware code out of its own memory Both boards have the same bit file > Avoids need to develop and implement separate bit files > CPU ID set by DIP switches on the board However, software will be different for each board > Master software: services all memory requests > Slave software: only routes memory requests to the other board 19

20 OpenSPARC on the BEE3 BEE3 board uses Virtex 5 FPGAs (same family as ML505) Re-implemented Release 1.6 design on BEE3 > Very easy to re-target design. > Updated design to EDK 10.1 > Re-implemented on both XC5VLX110T and XC5VLX155T > Generated ace files for BEE3 board > Verified OpenSolaris Boot Seamless and trouble-free porting experience Validated BEE3 board infrastructure Stop by to see our demo! 20

21 OpenSPARC on the Bee3 Details Single 4-thread OpenSPARC core 62.5MHz OpenSPARC; 125MHz Microblaze 6-LUTs: 59,350 / 97,280 61% 36kbit BRAM: 147 / % Ethernet in design but not tested 1.5 hour implementation time MPMC/MIG Microblaze EDK project based on Bee3 EDK reference design Uses EDK MPMC4 and MIG 4-thread OpenSPARC core 21

22 Bee3 OpenSPARC Four-Core System Diagram (Master-node Configuration) QSH cross-over cable QSH-DP- 040 Ring Wiring CCX link QSH-DP x2 40x2 Master Node DDR2 DIMM0 DDR2 DIMM1 DDR2 DIMM2 DDR2 DIMM User1 72* PCI-E 8X PCI-E 8X User3 72* DDR2 DIMM0 DDR2 DIMM1 DDR2 DIMM2 DDR2 DIMM3 User2 72* PCI-E 8X Ring Wiring CCX link DDR2 DIMM0 DDR2 DIMM1 DDR2 DIMM2 DDR2 DIMM3 72* 133 PCI-E 8X User4 133 DDR2 DIMM0 DDR2 DIMM1 DDR2 DIMM2 DDR2 DIMM3 40x2 40x2 QSH-DP- 040 QSH-DP RJ45 Ring Wiring CCX link V Master Node Fujitsu 2x2 FF1136 FF V V PCI-Express 8x QSH-DP-040 QSH-DP-040 PCI-Express 8x V PCI-Express 8x QSH-DP-040 QSH-DP-040 PCI-Express 8x V Fujitsu 2x2 1.8V TXL 5V F631 F 1 TXL 5V F631 F V QSH cross-over cable RJ JTAG 50 pin 2mm Header 12V 4-pin 12V 8-pin 24 pin ATX PWR Ring Wiring CCX link

23 Bee3 Multi-core Node Block Diagram FPGA Boundary Xilinx CacheLink (XCL) External DDR2 Dimm Xilinx Embedded Developer s (EDK) Design Fast Simplex Links (FSL) MemCon SPARC T1 Core CCX-FSL Interface Microblaze Proc Microblaze Debug UART Ring Ring Wiring Ring Wiring Wiring SPARC T1 UART Ethernet FSL interconnect to other FPGA(s) Maintain compatability between ml505_v5lx110t and Bee3 designs 23 Developed and Working New

24 Bee3 OpenSPARC Four-Core System Diagram (Full Mesh interconnect) QSH cross-over board QSH-DP- 040 Ring Wiring CCX link QSH-DP x2 40x2 DDR2 DIMM0 DDR2 DIMM1 DDR2 DIMM2 DDR2 DIMM User1 72* PCI-E 8X PCI-E 8X User3 72* DDR2 DIMM0 DDR2 DIMM1 DDR2 DIMM2 DDR2 DIMM3 User2 72* PCI-E 8X DDR2 DIMM0 DDR2 DIMM1 DDR2 DIMM2 DDR2 DIMM3 72* 133 PCI-E 8X User4 133 DDR2 DIMM0 DDR2 DIMM1 DDR2 DIMM2 DDR2 DIMM3 40x2 40x2 QSH-DP- 040 QSH-DP RJ45 Ring Wiring CCX link V Fujitsu 2x2 FF1136 FF V V PCI-Express 8x QSH-DP-040 QSH-DP-040 PCI-Express 8x V PCI-Express 8x QSH-DP-040 QSH-DP-040 PCI-Express 8x V Fujitsu 2x2 1.8V TXL 5V F631 F 1 TXL 5V F631 F V RJ JTAG 50 pin 2mm Header 12V 4-pin 12V 8-pin 24 pin ATX PWR Ring Wiring CCX link Ring Wiring CCX link 24 QSH cross-over board

25 Q S H -D P Q S H -D P Q S H -D P TXL5V F6F311 4 G B D D R D R A M 4 G B D D R D R A M T XL5V F6F311 Q S H -D P V 1.8 V 1.0 V 1.8 V 1.8 V 1.0 V 1.8 V 1.8 V 1.0 V T XL5V F6F311 4 G B D D R D R A M 4 G B D D R D R A M TXL5V F6F311 Q S H -D P Q S H -D P Q S H -D P p in A T X P W R 1.8 V 1 2V 8 -p in 1.8 V 12 V 4 -p in 2.5 V 2 4 p in A T X P W R Q S H -D P V 8 -p i n 2.5 V p i n 2 m m H e a de r 1 2V 4 -p in 5 0 p in 2 m m H e a d e r JTA G P C I- E x p r e s s 8 x P C I- E x p r e s s 8 x P C I- E x p r e s s 8 x P C I- E x p r e s s 8 x JT A G P C I-E x p re s s 8 x P C I-E x p re s s 8 x P C I-E x p re s s 8 x P C I-E x p re s s 8 x cables FF FF FF RJ45 RJ RJ45 Fujitsu 2x Fujitsu 2x Fujitsu 2x2 Fujitsu 2x2 RJ FF V Bee3 OpenSPARC Eight-Core, Two-Board System Diagram Use Aurora over cables to connect extended fourcore Bee3 board to base four-core Bee3 board. > Maintain native OpenSPARC T1 4-way L2 architecture > [Almost] full OpenSPARC T1 32-thread system!

26 Roadmap OpenSPARC T1 release 1.7 > Setup to boot Ubuntu Linux > Update of EDK project to EDK 10.1 > Improvements to memory controller > Improvements to place and route > Multi-core design Future Work (possible projects) > Connect T1 core directly to system > Increase size of L1 caches > Current size doesn't efficiently utilize the Block RAMs > Should be able to quadruple size without increasing logic 26

27 Summary OpenSPARC: The tools you need to do multi-core research! > Complete EDK project to implement a system > Implemented on both BEE3 board and OpenSPARC Kit > Complete verification environment > Complete software stack > OpenSolaris Boot demonstrated > Ubuntu Linux boot underway 27

28 OpenSPARC momentum Innovation will happen everywhere Innovation Happens Everywhere > 8400 downloads FCRC-RAMP-2007-San Diego 28

29 Team Ismet Bayraktaroglu Thomas thatcher Durgam Vahia Not Pictured: Gopal Reddy Paul Hartke (Xilinx) 29