Integrated Cluster Environment. SGI Ultraviolet. Reiner Vogelsang Senior Systems Engineer

Transcription

1 Integrated Cluster Environment SGI Ultraviolet Reiner Vogelsang Senior Systems Engineer SGI SGI

2 ICE DLR Braunschweig Processors: 1120 Intel E v2 2.4 GHz (13440 cores) Memory: 71.8 TB Interconnect: FDR Infiniband Operating System: Suse Enterprise Linux Rpeak: 258,048 GFLOPS Rmax: 245,876 GFLOPS Nmax: 1,885,596 Power Consumption: kwh Warm water cooling: 30 Celsius PUE:

3 E-Cell Designing for 60cm Tiles Closed Loop Air Flow Isolates Cell From Data Center Environment E-Cell (2 x E-Racks) 72 Blades (up to 4 nodes each) Up to 288 Logical Nodes 2015 SGI 2

4 E-Cell Piping Coolant Distribution Unit (CDU) E-Rack Supply & Return Water Facility supplied water temperature range 6 C to 32 C Facilities water temperature rise 6 C nominal Roadmap to 45 C planned Facilities Supply & Return Water Cooling-Racks Do Not Connect to CDU 2015 SGI 3

5 ICE XA Blade Enclosure Select Components (9) Expansion Volumes Signal & Power Midplane CMC Coolant Manifolds (9) Blade Slots (4) FDR IB Switch Blades 10.5U (8+1) 3000 W 12VDC PSUs connect directly to Midplane 2015 SGI 4

6 IP-125CS (Hawkins) Compute Centric Blade Four Logical 2-Socket Nodes with Intel Broadwell Four DIMMs per socket: Up to 64GB (512GB/ node) Four HCA Mezzanine Card Slots Mellanox Connect-IB (FDR- at MR) Mellanox ConnectX-4 (EDR- when available) No stuff option to cable Intel True Scale/ Omni Scale in extension space Extension Space Options (per node) Up to (2) 2.5 SATA HDD/SSD Up to (2) low profile PCIe 3.0 slots Up to (1) HDD/SSD and (1) PCIe slot 2015 SGI 5

7 Interconnects Interconnects planned Infiniband - integrated FDR: ConnectIB (vs ConnectX3 in M-Cell) with 1 st ship Higher throughput, lower latency compared with ConnectX3 Offloading of data movement, Mellanox HPC-X Suite EDR: ConnectX4 late Q3/early Q4 CY15 - Omni-Path late 2015/early Either expansion space using standard place Q4 - Eventually integrated with Knights Landing 2015 SGI 6

8 Common InfiniBand Topologies All-to-All Fat Tree (CLOS Networks) Hypercube Enhanced Hypercube All supported on SGI ICE XA as single and dual rail network SlimFly topology under investigation 2015 SGI 7

9 SGI ICE XA Standard IB Switch Blade 1SW18x18 Single IB Switch ASIC Single port FDR IB SwitchX-2 ASIC 18 ports connect to backplane 18 ports to external bulkhead QSFP connectors (18) Optimal for building fat tree and all-toall topologies - Can also build smaller hypercube or enhanced hypercube systems 36-Port Front View Switch Blade Plan View Right Justified = Signal Integrity, Cable Lengths Side View 2015 SGI 8

10 ICE XA- Next Gen ICE SGI ICE XA Scale-Out Server Roadmap (CY15+) 15CYQ1 15CYQ2 15CYQ3 15CYQ4 16CYQ1 16CYQ2 SGI ICE X Enclosure/ Rack E-Cell w/ E-Rack 3/15 Closed Loop & Warm Water Cell Support Enclosure Switch Blade/Mezz Card FDR 36 Port Switch 3/15 Connect-IB Mezz Card EDR 36 Port Switch 10/15 ConnectX-4 Mezz Card Intel StormLake 1 Compute Blade/ Socket Haswell-EP Broadwell-EP (Date TBD) Hawkins 3/15 Compute Blade Hixton IO 6/15 Compute Blade Kendall- KNL 3/16 Accelerator Blade 2015 SGI 9

11 SGI Ultraviolet 3000/ SGI SGI Company 10

12 20 years of Extreme Scale-up and In-Memory Computing Leadership 2014 : UV Gen3, Intel Xeon, Linux 2012 : UV Gen2, Intel Xeon, Linux 2010 : UV Gen1, Intel Xeon, Linux 2003 : Altix3000, Intel Itanium, Linux 1996 : Origin2000, MIPS R10K, IRIX 2015 SGI 12

13 Globally Shared Memory System Cache coherent shared memory in hardware Lenowski et al., The Standford DASH Multiprocessor, IEEE Computer Vol. 25 No. 3, 1992, To 64TB Cache-Coherent Shared Memory 2015 SGI 13

14 SGI UV Single-node to manage, use far less complex than scale-out systems with many nodes. Commodity Scale-out/Cluster InfiniBand or Gigabit Ethernet SGI UV Shared Memory SGI NUMAlink Interconnect Mem ~64GB system + OS mem system + OS mem system + OS mem system + OS... mem system + OS Global shared memory to 64TB System to 4096 thread + OS one instance Off-node data access requires internode messaging scheme All data transparently accessible to all processors. Ideal environment for complex multidisciplinary workflows 2015 SGI 14

15 UV 3000 Blade 4 NL6 Low profile PCI-E card Connector for top motherboard HARP SandyBridge 12 NL6+ Single Motherboard Shown Here 2015 SGI 15

16 From UV2000 to UV3000 UV2000 Availability now Intel Ivybridge EP 4S DDR3 Memory NUMAlink6 Topology: Cross-bar connected hypercubes Market: CAE,CFD,Life Sciences UV3000 Availability August 2015 Intel Haswell EP 4S DDR4 Memory NUMAlink6+ Increased linkspeed by 12% Reduced latency in some arenas of the HUB ASIC More resiliency features Topology: Cross-bar connected hypercubes Market: CAE, CFD, Life Sciences 2015 SGI 16

17 Why UV300? Development driven by demands of the enterprise market Data analytics requires overall low latency Data intensiv workloads (huge working low core counts) I/O rich system architecture SAP Hana is certified on Ivybridge EX (E7) Intel EX processor series provide highest RAS level 2015 SGI 17

18 UV 300 Topology with HARP2 NL7 Topology UV 2000/UV 3000 UV 300 All-to-all up to 16 sockets Average hop count ~1.4 All-to-all up to 32 sockets Max hop count = SGI 18

19 4 socket UV 300 Overview UV 300 is the 7 th generation cachecoherent, distributed (NUMA), shared memory system based on Intel s Brickland (E7) processor family The scalable unit is a 5U rack-mount chassis build with 4 processor sockets, memory, I/O and the SGI chipset (HARP2) Scaling is effected using SGI s NumaLink7 interconnect up to a maximum architectural limit of 48TB and 1280 CPUs (cores or threads w/ hyperthreading enabled) within a Single System Image (SSI) I/O per 5U Chassis 1 Base IO card (boot devices) 8 to 12 PCIe slot IO options 4 PCI-E x16, 8 PCI-E x8 x16 slots can accommodate double width, aux power for GPU/accelerator 2015 SGI 19

20 UV 300H UV300H 32 Sockets 24TB Main Memory 2015 SGI 20

21 Q & A 2015 SGI 21

22 2015 SGI