PACE Predictive Analytics Center of San Diego Supercomputer Center, UCSD. Natasha Balac, Ph.D.

Size: px

Start display at page:

Download "PACE Predictive Analytics Center of Excellence @ San Diego Supercomputer Center, UCSD. Natasha Balac, Ph.D."

Erik Barber
8 years ago
Views:

1 PACE Predictive Analytics Center of San Diego Supercomputer Center, UCSD Natasha Balac, Ph.D.

Brief History of SDSC 1985-1997: NSF national supercomputer center; managed by General Atomics 1997-2007: NSF PACI program leadership

diversified research computing still NSF national resource provider 2009-future: Multi-constituency cyberinfrastructure (CI) center 2

2 Brief History of SDSC : NSF national supercomputer center; managed by General Atomics : NSF PACI program leadership center; managed by UCSD PACI: Partnerships for Advanced Computational Infrastructure : Internal transition to support more diversified research computing still NSF national resource provider 2009-future: Multi-constituency cyberinfrastructure (CI) center 2 provide data-intensive CI resources, services, and expertise for campus, state, and nation Approaching $1B in lifetime contract and grant activity!

3 Mission: Transforming Science and Society Through Cyberinfrastructure The comprehensive infrastructure needed to capitalize on dramatic advances in information technology has been termed cyberinfrastructure. 3 D. Atkins, NSF Office of Cyberinfrastructure

capitalize on dramatic advances in information technology has

4 Today s Information Technologies Drive 21 st Century Solutions Means to an end: Cyberinfrastructure is the foundation for modern research and education Cyberinfrastructure components: Digital data Computers Wireless and wireline networks Personal digital devices Scientific instruments Storage Software Sensors People What therapies can be used to cure or control cancer? What plants work best for biofuels? Will California levees hold in a large-scale earthquake? 4 Can financial data be used to accurately predict market outcomes?

devices Scientific instruments Storage Software Sensors People What therapies can be used to cure or control cancer?

5 5 Simulation of SoCal Earthquake

Power ~20 compute systems >2 PF aggregate Most flops in TACC s Ranger & NICS Kraken New systems

6 The TeraGrid: Providing High-End Resources to Academic Researchers 11 Resource Providers (RPs) Grid Infrastructure Group (GIG) Led by UC/ANL Annual Budget RPs: ~$50M total GIG: ~$15M Compute Power ~20 compute systems >2 PF aggregate Most flops in TACC s Ranger & NICS Kraken New systems coming Additional HW Resources >4 PB disk storage Wide-area file systems >60 PB tape storage 6

7 7

8 A data-intensive supercomputer based on SSD flash memory and virtual shared memory Emphasizes MEM and IO over FLOPS A custom-designed system based on COTS to accelerate access to massive databases in all fields of science, engineering, medicine, and social science The NSF s most recent Track 2 award to SDSC 8

COTS to accelerate access to massive databases in all fields of science,

9 Gordon is designed specifically for data-intensive HPC applications Such applications involve very large data-sets or very large input-output requirements Two data-intensive application classes are important and growing Data Mining the process of extracting hidden patterns from data with the amount of data doubling every three years, data mining is becoming an increasingly important tool to transform this data into information. Wikipedia Data-Intensive Predictive Science solution of scientific problems via simulations that generate large amounts of data

patterns from data with the amount of data doubling every three years, data mining is becoming an increasingly important tool to transform

10 Why Gordon? EXPONENTIAL DATA GROWTH Growth of digital data is exponential Driven by advances in sensors, networking, and storage technologies 10,000, ,000, HDD Capacity vs. Random IOPS vs. Access Density Trends 100, , , RED SHIFT While processor speeds have been increasing at Moore s Law DRAM and disk speeds stay almost constant System utilization is cut in half every 18 months? It takes 2x time to read your disk every 18 months? Capacity (MB) Max Random IOPS Rate Ballpark Access Density 10

Random IOPS vs. Access Density Trends 100,000.00 10,000.00 1,000.00 100.00 10.00 1.00 0.10 0.

11 Modern HPC systems have a latency gap 11

12 Gordon architecture fills that gap Flash O(TB) 1,000 cycles 12

13 High Performance Computing (HPC) vs High Performance Data (HPD) Attribute HPC HPD Key HW metric Peak FLOPS Peak IOPS Architectural features Many small-memory multicore nodes Fewer large-memory SMP nodes Typical application Numerical simulation Database query Data mining Concurrency High concurrency Low concurrency or serial Data structures Data easily partitioned e.g. grid Data not easily partitioned e.g. graph Typical disk I/O patterns Large block sequential Small block random Typical usage mode Batch process Interactive 13

mining Concurrency High concurrency Low concurrency or serial Data structures Data easily partitioned e.g. grid Data not easily partitioned e.

14 SDSC Wins SC09 Data Challenge With Dash Prototype System 5.2 Tflops prototype of Gordon with 4 TB of flash New Teragrid resource that has already sped up some scientific calculations by 2 orders-of-magnitude 14

15 15 Exponential Growth of Research Data Protein Data SDSC

SDSC s Triton Resource computational and data resource for UC researchers Repository for valuable research databases UCSD/UC Research Labs

Data Analysis Facility) Designed for analysis of very large data sets x 3 2 Triton Compute Cluster Condo-style cluster starter to create

16 SDSC s Triton Resource computational and data resource for UC researchers Repository for valuable research databases UCSD/UC Research Labs and Sensor Nets Data Oasis 2 4 Petabytes Large scale Storage for research data sets Administered with the UCSD Libraries PDAF (Petascale Data Analysis Facility) Designed for analysis of very large data sets x 3 2 Triton Compute Cluster Condo-style cluster starter to create expandable UCSD computing resources. Unique computer for extreme-scale data analysis Seed for UCSD s home grown supercomputer Triton Resource 16

17 Questions? For further information, contact Natasha Balac

Hadoop on the Gordon Data Intensive Cluster

Hadoop on the Gordon Data Intensive Cluster Amit Majumdar, Scientific Computing Applications Mahidhar Tatineni, HPC User Services San Diego Supercomputer Center University of California San Diego Dec 18,