urika! Unlocking the Power of Big Data at PSC

Transcription

1 urika! Unlocking the Power of Big Data at PSC Nick Nystrom Director, Strategic Applications Pittsburgh Supercomputing Center February 1, Pittsburgh Supercomputing Center

2 Big Data is Everywhere 2013 Pittsburgh Supercomputing Center 2

3 Some Important Types of Big Data Pan-STARRS telescope NOAA climate modeling Web graph of major Internet routers, Ross Richardson and Fan Chung Graham Genome sequencers (Wikipedia Commons) Cosmological simulations of black hole formation, Tiziana Di Matteo Human interactome (Wikipedia Commons) structured, regular, mostly independent 2013 Pittsburgh Supercomputing Center structured, regular, coupled or independent 3 unstructured, irregular, nonpartitionable

4 Graphs Represent Sets of Complex Relationships A graph is a set of vertices (or nodes) and edges Edges can be directed a e b f d c 2013 Pittsburgh Supercomputing Center 4

5 An Example: the WMATA Map Metro stops are the nodes, and the routes between them are the edges. There are 6 lines (red, orange, yellow, blue, green, silver), which can also be expressed as 6 graphs having some nodes in common. Distances between stops, if expressed, could make this a weighted graph Pittsburgh Supercomputing Center 5

6 Another Example: from NELL The Never Ending Language Learner (William Cohen and Tom Mitchell, CMU), graphs are semantic networks of beliefs, with nodes representing entities and edges representing their relationships Concept: Person type Concept: Politician type type Concept: Bill_Clinton Concept: George_Bush knows 2013 Pittsburgh Supercomputing Center 6

7 Graph Analytics Graph analytics is the science of solving problems using data expressed as graphs. Examples: Revealing patterns and relationships Community detection Anomaly detection Classification (e.g. via structural characterization) Inferencing 2013 Pittsburgh Supercomputing Center 7

8 Graph Nonpartitionability A cut is a partition of nodes into two subsets The cut size is the number of edges joining those subsets of nodes V = 8, E = 8, ξ = 2 V = 8, E = 28, ξ = 18 If the cut size is not substantially smaller than the number of edges, the graph is nonpartitionable. Many important graph problems are nonpartitionable Pittsburgh Supercomputing Center 8

9 Nonpartitionable = Challenging! Graph algorithms typically require following edges from vertex to vertex, which is irregular and unpredictable. For a large, nonpartitionable graph, data distribution onto a distributed system requires very frequent communications and remote memory accesses. The order of access is irregular and unpredictable, and the amount of data at each vertex is typically small. Messaging imposes high overhead (but PGAS can help). Caching is ineffective or counterproductive. Hardware prefetching from memory is counterproductive. Large graph problems require a specialized architecture Pittsburgh Supercomputing Center 9

10 Going Back to Our Examples Web graph of major Internet routers, Ross Richardson and Fan Chung Graham Human interactome (Wikipedia Commons) 2013 Pittsburgh Supercomputing Center 10

11 Sherlock: a YarcData urika Data Appliance with PSC Enhancements Graph Analytics Platform urika application architecture Universal RDF Integration Knowledge Appliance 32 Graph Analytics Platform nodes, each containing: 2 Cray Threadstorm 4.0 processors, 128 threads/proc SeaStar 2 ASIC 1 TB globally shared memory can accommodate graphs of up to ~10 billion edges General-purpose XT5 (AMD Opteron) nodes Massive multithreading and sophisticated memory handling for latency hiding Next-generation Cray XMT Remote Memory Access block with Extended Memory Semantics, providing a single, shared address space Enable additional, heterogeneous applications 2013 Pittsburgh Supercomputing Center 11

12 The Sherlock Project Funded by the National Science Foundation through the Strategic Technologies for Cyberinfrastructure (STCI) program An experimental system, intended for the community to gain experience with urika, develop new kinds of applications, and pursue cutting-edge research For additional information, see Pittsburgh Supercomputing Center 12

13 urika: Enabling Productive Graph Analytics Leverage emerging Web 3.0 standards The Semantic Web provides a common framework that allows data to be shared and reused across application, enterprise, and community boundaries. 1 Resource Description Framework (RDF) SPARQL Protocol and RDF Query Language Application-specific GUIs for user interaction This is general framework that s useful for much more than the Semantic Web. 1. W3C Semantic Web Activity. World Wide Web Consortium (W3C). November 7, Pittsburgh Supercomputing Center 13

14 RDF: Resource Description Framework Represents data as directed, labeled graphs Unifies data and metadata Although originally developed for the Web, the format is general Format: Triple = subject, predicate, object (or resource, property, value) Subject : URI or blank node Predicate : URI Object : URI, blank node, or literal urika Quad = triple + named graph This very general format allows fusing data from multiple sources There are tools for converting data to RDF from RDBMS, Hadoop, etc. But ontologies must either be the same or mapped onto each other There are tools for this, too Pittsburgh Supercomputing Center 14

15 SPARQL SPARQL: SPARQL Protocol and RDF Query Language contains capabilities for querying required and optional graph patterns along with their conjunctions and disjunctions supports extensible value testing and constraining queries by source RDF graph query results can be RDF graphs or simpler result sets is a convenient resource for experimenting with SPARQL queries Pittsburgh Supercomputing Center 15

16 SPARQL Example 1 Query: PREFIX rdfs: < PREFIX type: < PREFIX prop: < SELECT?country_name?population WHERE {?country a type:landlockedcountries ; rdfs:label?country_name ; prop:populationestimate?population. FILTER (?population > && langmatches(lang(?country_name),"en") ). } 2013 Pittsburgh Supercomputing Center 16

17 SPARQL Example 1 (2) Result: country_name "Kazakhstan"@en "Uzbekistan"@en "Afghanistan"@en "Ethiopia"@en "Burkina Faso"@en "Nepal"@en "Niger"@en population " "^^< " "^^< " "^^< " "^^< " "^^< " "^^< " "^^< Pittsburgh Supercomputing Center 17

18 A Semantic Web Approach to Graph Analytics Unstructured data: sparse, irregular, no need to define schema a priori Express relationships as subject predicate object Analyzing data with relationships allows powerful inferencing to discover information that s buried in the data Applications: Medicine, biology, finance, sociology, economics, machine learning, epidemiology, security, business, networking, Graphical representation of part of the human metabolic network, focusing on genes in the skeletal muscles of diabetes patients. From Kiran Raosaheb Patil, Systems biology: Looking beyond the genome, Pittsburgh Supercomputing Center 18

19 Opening New Directions in Research = Complex analytics with urika + Structural characterization, dynamics, New communities Reaching (again) the long tail of research New scales of complex analysis Real-world problems at scale New types of applications Focus on actionable insights from data 2013 Pittsburgh Supercomputing Center 19

20 Using Sherlock: 3 Paradigms New communities, New scales of analysis, New types of applications As a urika data appliance RDF, SPARQL, SNORQL, GUIs Extremely powerful for complex queries Support for named graphs provides additional capability Using the Graph Analytics Platform for standalone applications Heterogeneous XT5 / Graph Analytics Platform applications Compute nodes: C or C++, with XMT extensions for memory, synchronization, etc. System: Java, Fortran, etc. on general-purpose XT5 nodes Homogeneous (NG-XMT) and heterogeneous programming (XT5 + NG-XMT) to support a wide range of applications and to ease porting Pittsburgh Supercomputing Center 20

21 Anticipated Research (Examples 1) Identification of genes and pathways leading to tumors Real-time social network analysis: recognizing misinformation and improving disaster response Never-Ending Language Learning (NELL) Agent-based epidemiological modeling Anomaly detection, feature extraction, and analysis of large, time-evolving graphs Large-scale, high-dimension analysis Detecting anomalous subgraphs, e.g. disease outbreaks 2013 Pittsburgh Supercomputing Center 21

22 Anticipated Research (Examples 2) Cluster finding in astrophysics Understanding community dynamics across different types of social networks Data stream management systems and annotation management Generating sparse signal processing and graph kernels Extracting subpopulations from large social networks Genome sequence assembly 2013 Pittsburgh Supercomputing Center 22

23 Enabling Breakthrough Research with Big Data Innovative hardware and software architectures allows researchers to tackle graph datasets at full scale. NSF users will increasingly exploit graph analytics as part of their data-intensive research. This is complementary to other aspects of HPC and big data. PSC and XSEDE can help researchers in several ways: Disseminating information about resources Establishing best practices for XSEDE and other NSF platforms Assisting with data modeling Helping to develop workflows for exploratory and routine analytics Scaling and optimizing graph applications and algorithms Extending graph analytic capabilities, e.g. probabilistic predicates, reification 2013 Pittsburgh Supercomputing Center 23

24 Thank You 2013 Pittsburgh Supercomputing Center 24