Sun Architekturen für HPTC Roland Rambau

Transcription

1 Sun Architekturen für HPTC Roland Rambau Technical Systems Ambassador Plattform Technologie Team Sun Microsystems GmbH HPTC Archtiektur 10/2005 Seite 1

2 Agenda: Sun's HPTC Strategy Leverage AMD Opteron for best price performance in x86 cluster market > Partner with AMD on 64-bit OS design, compilers, tools and HyperTransport interface Push CMT = Chip Multi-Threading Innovation with SPARC New SPARC64 and APL partnership with Fujitsu Continue extreme performance and security innovation with Solaris > Solaris Containers, ZFS File System, Dtrace > Sun Studio Development Tools > HPC ClusterTools Extend our leading disk and tape brand StorageTek Roland.Rambau@Sun.Com HPTC Archtiektur 10/2005 Seite 2

3 Sun's HPTC Strategy (cont) Back to our roots with education and research > Global Science and Technology Network Sell and Support Linux: RedHat and SUSE Partner with all interconnect players > Infiniband, Myrinet, 10 Gig Ethernet Partner with Open Source Community > Lustre parallel file system > Sun N1 Grid Engine > Open Solaris Sun is the #1 Contributor to the Open Source Community NFS TCP/IP Java.net HPTC Archtiektur 10/2005 Seite 3

4 2003 An industry defining moment. Joining 20 years of enterprise expertise with industry-standard economics, targeting the delivery of systems with extreme performance at compelling prices. HPTC Archtiektur 10/2005 Seite 4

5 Announced Collaboration Areas Applications Operating System Maximization of performance of Java Platform and tools Support of Java Enterprise System Offer customers a choice of Solaris or Linux on Sun Fire with Opteron Optimize Solaris on Opteron performance Define&promote 64-bit Unix-Linux API Architecture Undertake joint technology initiatives such as HyperTransport Roland.Rambau@Sun.Com HPTC Archtiektur 10/2005 Seite 5

6 Why CPU Architecture Matters IA32, IA64, & EM64T AMD64 CPU CPU CPU CPU CPU CPU Northbridge CPU CPU Southbridge PCI/E Bridge PCI/E Bridge 4 CPU architectures 11 Chipset architectures No CPU has integrated memory controller All CPUs share the FSB performance bottleneck 1 CPU architecture 1 Chipset architecture CPU has integrated memory controller Glue-less multiprocessor without FSB bottleneck Roland.Rambau@Sun.Com HPTC Archtiektur 10/2005 Seite 6

7 Future AMD Product Line Galaxy Systems x2100: 1U, 1S, 4 DIMM, 4GB, 1 PCI-E slots, 2disks x4100: 1U, 2S, 8 DIMM, 16GB, 2 PCI-X/E slots, 2disks x4200: 2U, 2S, 8 DIMM, 16GB, 5 PCI-X/E slots, 4 disks Galaxy 3 Galaxy 4 100% Sun Designed Roland.Rambau@Sun.Com HPTC Archtiektur 10/2005 Seite 7

8 Trends in High Performance Computing? HPTC Archtiektur 10/2005 Seite 8

9 X64 Compute Node Size Evolution Old Architecture P Building Block 32-bit architecture 4 GByte memory New Architecture P Building Block 64-bit architecture 32 GByte memory Historical Choice due to 32-bit architecture limitations Next generation node taking full advantage of new 64-bit architecture Larger compute nodes with much larger memory enable much more efficient models of physical world Roland.Rambau@Sun.Com HPTC Archtiektur 10/2005 Seite 9

10 Sun TM Grid Rack System Grids Made Easy Reduce Time and Risk for Deployments Sun Expertise Sun SM Customer Ready Systems Products, Function (Compute, Data, Access) Optimized Sample Configuration Optimized Customer Configuration Build it Deploy it Sample Config Small Medium Large X-Large Requirements Size Interconnect Power Cooling others Factory Built and Tested Ready to Run Available: Sun Architecture, Implementation and Migration Services HPTC Archtiektur 10/2005 Seite 10

11 Sun TM N1 TM Grid Computing Pay-Per-Use Cycles Service A virtual compute ranch that gives customers access to a secure, pay-per-use N1 Grid service, optimized to solve their non-transactional workloads at an introductory price of $1/hour/ Off-load computational workloads that require 10s or 100s or 1,000s of CPUs. Use compute power on an hourly basis, without ownership or onerous outsourcing contracts. Leverage economies of scale achieved by mapping your workload to a singular service provider with a common infrastructure. Roland.Rambau@Sun.Com HPTC Archtiektur 10/2005 Seite 11

12 CMT Throughput Computing ***Chip R&D Matters - 64 Bit Moves Ahead USIV+ Panther Niagara APL Rock The SPARC Architecture Manual Roland.Rambau@Sun.Com HPTC Archtiektur 10/2005 Seite 12

13 Memory Bottleneck Relative Performance CPU Frequency DRAM Speeds x Every 2 Years Gap x Every 6 Years Roland.Rambau@Sun.Com HPTC Archtiektur 10/2005 Seite 13

14 Typical Complex High Frequency Processor Time Saved Thread C M C M C M Memory Latency Compute Time Thread HURRY UP AND WAIT! C M C M C Memory Latency Compute Note: Up to 7 5 % Cycles Waiting for Mem ory M Time Roland.Rambau@Sun.Com HPTC Archtiektur 10/2005 Seite 14

15 Chip Multithreading (CMT) Thread 4 C M C M C M Thread 3 C M C M C M Thread 2 C M C M C M Thread 1 C M C M C M Time Memory Latency Compute Roland.Rambau@Sun.Com HPTC Archtiektur 10/2005 Seite 15

16 CMT Multiple Multithreaded Cores Core 8 Thread 4 Thread 3 Thread 2 Thread 1 Core 7 Thread 4 Thread 3 Thread 2 Thread 1 Core 6 Thread 4 Thread 3 Thread 2 Thread 1 Core 5 Thread 4 Thread 3 Thread 2 Core 4 Core 3 Thread 4 Thread 3 Thread 2 Thread 1 Core 2 Thread 4 Thread 3 Thread 2 Core 1 Thread 1 Thread 4 Thread 3 Thread 2 Thread 1 Thread 1 Thread 4 Thread 3 Thread 2 Thread 1 Memory Latency Compute Time Roland.Rambau@Sun.Com HPTC Archtiektur 10/2005 Seite 16

17 Sun and Fujitsu: Expanding SPARC's reach in the Market UltraSPARC IV Sun Fire UltraSPARC IV+ Throughput Computing Design Excellence Fujitsu PRIMEPOWER Thousands of Applications SPARC64 V SPARC64 V+ Mission-critical Computing Heritage APL Roland.Rambau@Sun.Com HPTC Archtiektur 10/2005 Seite 17

18 High Productivity Computing Systems 6 5 % Providing a new generation of economically viable high productivity computing systems for the national security and industrial user community ( ) Roland.Rambau@Sun.Com HPTC Archtiektur 10/2005 Seite 18

19 Awards For Each Phase yr I: Industry Concept Study ~ 3M 3 yrs II: Research ~50M (July 2003) yrs III: Product Development 100M?? 1 or 2 to be selected Roland.Rambau@Sun.Com HPTC Archtiektur 10/2005 Seite 19

20 Hero Program Overview Sun's HPCS High End Research Opportunity HPTC Archtiektur 10/2005 Seite 20

21 Hero Technology #2 Proximity Communication I/O Density Comparison I/O Density per mm^ Year Proximity I/O Optical VCSEL Area Ball Bonding Chip-to-chip communication without wires! HPTC Archtiektur 10/2005 Seite 21

22 SPARC Roadmap Data- Intensive UltraSPARC III and UltraSPARC IV Systems Olympus Systems (OPL) APL Rock Systems Network- Intensive UltraSPARC IIIi Systems Niagara Systems APL Opteron Systems HPTC Archtiektur 10/2005 Seite 22

23 Solaris TM 10 OS Great Product, Great Price, Open Extreme Performance Dynamic Tracing Optimal Utilization Solaris Containers Virtually Unparalleled Security Process Rights Management Solaris Cryptographic Infrastructure Relentless Availability Predictive Self Healing Platform Choice Optimized for SPARC, Opteron and Intel Momentum 2m licenses 500 applications 400 systems Source: Sun Microsystems, Inc., June 2005 HPTC Archtiektur 10/2005 Seite 23

24 Solaris in HPTC Re-Designed with AMD Engineers CMT Ready today Sun Studio 10 > C/C++/Fortran Compilers MPAVICH MPI support on x64 > Ohio State University Collaboration > Next Release Better x64 support and SPARC support HPC ClusterTools is Back!! InfiniBand, Myrinet, and 10 Gigabit Ethernet support HPTC Archtiektur 10/2005 Seite 24

25 Interconnects - Ethernet Theoretical and Bi-Directional Gigabit Ethernet > GB/sec > 60 us Latency > 20us Latency (On-Chip TCP Offload Engine) 10 Gigabit Ethernet > 1.28 GB/sec > 40us Latency > 10us Latency (On-Chip TCP Offload Engine) Roland.Rambau@Sun.Com HPTC Archtiektur 10/2005 Seite 25

26 Interconnects - Myrinet Theoretical and Bi-Directional Myrinet 2 > Single Port Card GB/sec (PCI-X) > Dual Port Card GB/sec (PCI-X ) > 2.6 us Latency Myrinet 10G > 2.4 GB/sec (PCI-E 8x) > 2.0 us Latency Roland.Rambau@Sun.Com HPTC Archtiektur 10/2005 Seite 26

27 Interconnects - InfiniBand Theoretical and Bi-Directional InfiniBand 4x Single Data Rate (SDR) 80% calculated > GB/sec (1x 4+4Gbps) with PCI-X > 2.0 GB/sec (1x 8+8Gbps) with PCI-E 4x or 8x > 3.5 us Latency InfiniBand 4x Double Data Rate (DDR) 80% calculated > 4.0 GB/sec (1x 16+16Gbps) with PCI-E 8x > 2.x us Latency > Known as InfiniBand 8x InfiniBand 12x Single Data Rate 80% calculated > 6.0 GB/sec (1x 24+24Gbps) > 1.x - 2.x us Latency InfiniBand 12x Double Data Rate 80% calculated > 12.0 GB/sec (1x 48+48Gbps) > 1.x 2.x us Latency Roland.Rambau@Sun.Com HPTC Archtiektur 10/2005 Seite 27

28 OpenIB: open implementation of industry standard upper level protocols HPTC Archtiektur 10/2005 Seite 28

29 Why is OpenIB so Important? Because open implementations of Networking Protocols are a prerequisite to become a standard Roland.Rambau@Sun.Com HPTC Archtiektur 10/2005 Seite 29

30 Flashback to 1985: The Big Networking Protocol Debate TCP/IP IPX/SPX DECNET SNA TCP/IP Appletalk OSI HPTC Archtiektur 10/2005 Seite 30

31 The deciding factors behind the wide-spread adoption of TCP/IP 1. An unencumbered open protocol specification designed to keep things simple and working 2. Availability of quality open implementations that enabled broad multi vendor support 3. Market acceptance driven by many benefits of an open multi-vendor solution Lesson learned: proprietary networking protocols don t survive Roland.Rambau@Sun.Com HPTC Archtiektur 10/2005 Seite 31

32 Success factors for OpenIB 1. Unencumbered open protocol specification 2. Availability of quality open implementations 3. System vendors providing and customers accepting these implementations We are about half-way there HPTC Archtiektur 10/2005 Seite 32

33 Infiniband vs Ethernet HPTC Archtiektur 10/2005 Seite 33

34 The History of Ether and EtherNot There is a widespread belief in the networking world that there is Ethernet and EtherNot and if you are EtherNot you are not going to make it This is based on the history of Ethernet gobbling up all competing networking technology, including Tokenring, FDDI, and ATM However Ethernet and Infiniband have very different attributes that make Infiniband a very good choice for clustering applications Roland.Rambau@Sun.Com HPTC Archtiektur 10/2005 Seite 34

35 Fabric Market Update Infiniband is on a roll industry expects to have shipped 1M ports by the end of the year, more than 10X the number of 10 GigE ports 2. There are both technical and business reasons why Infiniband is likely to continue to be the Interconnect of choice for clusters 3. As a result, IBA combined with OpenIB is likely to dominate the open standard cluster interconnect landscape going forward Roland.Rambau@Sun.Com HPTC Archtiektur 10/2005 Seite 35

36 Ethernet vs Infiniband Similarities 1. Both are packet oriented transports 2. Infiniband was designed with an RDMA model, WARP/RDMA is just being added to Ethernet 3. Of course WARP can be implemented in SW however performance and latency is not great But let s assume that in a few years all Ethernet NICs will have RDMA support. What will happen then? Roland.Rambau@Sun.Com HPTC Archtiektur 10/2005 Seite 36

37 Difference #1: Flow control 1. Infiniband uses credit based flow control which means that Infiniband switch chips can be built with much smaller on-chip buffers 2. Ethernet switches rely on explicit packet drops for flow control, which requires larger buffers because the cost of retransmission is very high This technical difference enables to build larger and lower cost switch chips for Infiniband vs Ethernet HPTC Archtiektur 10/2005 Seite 37

38 Difference #2: Reliable Transport 1. Infiniband uses hardware based retransmission 2. Ethernet relies on the TCP/IP protocol which is typically implemented in SW for retransmission 3. The effort to implement TCP/IP in hardware has been proven much more challenging than what people imagined Same functionality, but very different performance HPTC Archtiektur 10/2005 Seite 38

39 Difference #3: Switch Performance 1. Infiniband fabrics do cut-through switching for low latency non-blocking performance 2. Virtually all Ethernet switches are designed for L3/L4 switching, which requires packet rewrite, which requires a store-forward architecture The latency impact of store forward is quite significant: for a 512Byte packet at 1 Gbps it is 4.1 usec, for a 1536 Byte packet at 1 Gbps it is 12.3 usec Roland.Rambau@Sun.Com HPTC Archtiektur 10/2005 Seite 39

40 Ethernet Business Reality 1. Ethernet is all about stability to minimize customer transitions: 1 GigE was 1997, wide spread deployment of 10 GigE is now Competition has been about switch features: whoever has the most features wins. This has not helped to make switches go faster GigE has been used only for backbone aggregation, not for Server NICs As a result, even though 1 GigE is everywhere, 10 GigE is basically nowhere in HPC fabrics Roland.Rambau@Sun.Com HPTC Archtiektur 10/2005 Seite 40

41 Infiniband Business Reality 1. Infiniband is all about going faster: SDR -> DDR -> QDR, 4X -> 12X Rapid Innovation over a few years 2. Infiniband switches are being designed to minimize latency and cost 3. There is an open system environment for Infiniband leading to lower costs Infiniband from a business standpoints addresses the need of the user and system communitiy to deliver the highest performance cluster interconnections Roland.Rambau@Sun.Com HPTC Archtiektur 10/2005 Seite 41

42 Challenges for Ethernet 1. Add RDMA to motherboard NICs 2. Reduce Datacenter latencies 3. Deliver more scalable, cut-through switches 4. Reduce the cost per 10 GigE Switch port 5. Support large non-blocking network fabrics Ethernet needs renewed focus on low latency, high bandwidth, cost-effective data center fabric switches This is not how the market has been evolving, but it is not too late to fix it! HPTC Archtiektur 10/2005 Seite 42

43 Challenges for Infiniband 1. Deliver the OpenIB stack 2. Further reduce end-to-end MPI latency 3. Further increase Switch bandwidth and ports 4. Maintain the Cost-performance advantage 5. Deliver system level cluster integration Infiniband has been on a roll, but need to maintain focus on cost-performance to maintain market momentum OpenIB is a prerequisite for the future success of Infiniband We cannot win with proprietary software implementations Roland.Rambau@Sun.Com HPTC Archtiektur 10/2005 Seite 43

44 HPTC Outlook for Sun Microsystems > Build Opteron market share > With Linux where necessary, with Solaris wherever possible > SPARC begins offensive for larger SMPs with APL and SPARC64 VI > Apart from integer-intensive where Niagara plays > Opteron for up to 16 core 2008 and beyond > Rock takes SPARC on offensive across the board > Opteron for up to 32 core and much more... HPTC Archtiektur 10/2005 Seite 44

45 Conclusion HPTC is a Growth Target with Sun Corporate Support AMD Opteron for low cost, high performance distributed computing SPARC Innovation with CMT for high performance large shared memory computing Solaris OE Single OS across all hardware providing high performance Compilers and MPI stack > InfiniBand, Myrinet and 10 Gigabit Ethernet Support Sun Microsystems, Inc. is ready to take over the HPTC Market. Roland.Rambau@Sun.Com HPTC Archtiektur 10/2005 Seite 45

46 High Performance and Technical Computing If there is an insatiable appetite for computing then it s probably HPTC. HPTC is a primary demand function for mainstream computing technologies Greg Papadopoulos CTO, Sun Microsystems Roland.Rambau@Sun.Com ComputaCenter 07/2005 Seite 46

47 Danke für Ihre Aufmerksamkeit Roland Rambau HPTC Archtiektur 10/2005 Seite 47