Yogesh Simmhan. Computer Engineering & Center for Energy Informatics

Transcription

1 Yogesh Simmhan Computer Engineering & Center for Energy Informatics

2 Big Data 3 V s of Big Data (Gartner) Volume Variety Velocity Is Big Data a hype or a new realization? Next Frontiers in Big Data, Y. Simmhan 4-Sep-13 2

3 Academia was ahead of the curve HPC, Grid, SC Center, Computational Science, MPI, TeraFLOPs, Extreme Computing Compute Intensive Data Intensive Data Grid, Clouds, Data Centers, Informatics, 4 th Paradigm, Hadoop, Petabyte, Distributed Data Everywhere Next Frontiers in Big Data, Y. Simmhan 4-Sep-13 3

4 And they keep growing Enterprise Data Warehouses, Web Logs High Thru put Instruments (NGS) Cyber Physical Systems (Smart Infrastructure) Large Instruments (LHC, LSST) Social Networks Quantified Self, Personal Informatics Internet of Things Next Frontiers in Big Data, Y. Simmhan 4-Sep-13 4

5 Vast & Fast Data Complexity & Dynamism have been less examined! Next Frontiers in Big Data, Y. Simmhan 4-Sep-13 5

6 Evolving Nature of Cyber-Infra Traditional HPC and Accelerated computing Clouds, GPGPU, FPGA, XMT, ARM/SoC, Phi, NTV Democratized, Massively parallel, Faster, Power Efficient Some are here, some are coming soon Both research & usability questions Which ones work well? For what apps? Benchmarks, user support, tutorials, APIs Next Frontiers in Big Data, Y. Simmhan 4-Sep-13 6

7 and Big Data Platforms Hadoop (MapReduce) Tuple/NoSQL Giraph (Pregel) Graph Processing using BSP Storm, InfoSphere Streams Stream Processing Esper, Siddhi Complex Event Processing Next Frontiers in Big Data, Y. Simmhan 4-Sep-13 7

8 THE APPLICATION Next Frontiers in Big Data, Y. Simmhan 4-Sep-13 8

9 Smart Grids Distribution Transmission Generation A Cyber-Physical- Social System Residential Utility Commercial Microgrids Buildings Control Center Solar Cogeneration Rooms HVAC & Lighting Electric Vehicles Next Frontiers in Big Data, Y. Simmhan 4-Sep-13 9

10 Energy Technology Shift US EIA s Annual Energy Outlook 2011 Next Frontiers in Big Data, Y. Simmhan 4-Sep-13 10

11 Technology Shift Big Data Ed. i-lab.usc.edu Next Frontiers in Big Data, Y. Simmhan 4-Sep-13 11

12 Demand Response Optimization Demand curtailment by consumers thru shedding, shifting & shaping load in response to utility request LADWP s Renewable Portfolio Standard: Challenges and Implementation, DWP When to curtail? Forecast when demand will outstrip supply based on realtime information Whom to target? Predict customers & buildings to request based on current conditions Next Frontiers in Big Data, Y. Simmhan 4-Sep-13 12

13 Temperature 'F Consumers Load Shedding in Lindley Hall 1 0 Next Frontiers in Big Data, Y. Simmhan 4-Sep-13 13

14 Consumers smartgrid.usc.edu ladwp.com Next Frontiers in Big Data, Y. Simmhan 4-Sep-13 14

15 USC Campus Microgrid Testbed City within a city Largest private institutional customer of LA DWP Electric load of 28 MW Diversity Dorms, Classrooms, Offices, Hospitals, Restaurants 33k students, 13k staff 301 acres A living, learning laboratory for controlled & calibrated validation of systems, operational scenarios, & behavior Next Frontiers in Big Data, Y. Simmhan 4-Sep-13 15

16 Big Data in Campus Microgrid Real-time streaming data ~50,000 1/15min intervals Dynamic Data 5 years of historical data ~170 1/15 min intervals Big Data Integration with social & infra. Data Customer surveys, building & organization details Complex Data Next Frontiers in Big Data, Y. Simmhan 4-Sep-13 16

17 From Microgrid to City Grid LA Smart Grid Demonstration Project DOE & LA Dept of Water and Power De-risk novel analytics for smart grids Scalable software platform & algorithms LA DWP: Largest municipal utility in US 4M residents, 1.4M customers 7GW load, ~1% of US electricity use 50,000 customers getting Smart Meters Big Data software platform for demand-side energy management Next Frontiers in Big Data, Y. Simmhan 4-Sep-13 17

18 Scalable Software Architecture for DR in the USC Microgrid Environment, Events Customers Analysts Customers, Facilities Engineers Visualization Researchers Generation Capacity Data Equipment, Sensors Monitor Ingest Data Store & Share Data Forecast Demand Decide D 2 R Strategy Curtailment Notification Voluntary and Direct Load Control Article in CiSE Cloud Special Issue, 2013 Next Frontiers in Big Data, Y. Simmhan 4-Sep-13 18

19 Consumption (in kwh) Consumption (in kwh) Consumption Data Variability office building 15-min time Interval of the day dorm building 15-min time Interval of the day Next Frontiers in Big Data, Y. Simmhan 4-Sep-13 19

20 Consumption Data Variability Next Frontiers in Big Data, Y. Simmhan 4-Sep-13 20

21 Customer Predictive Analytics on Big Data Historical KWH Time Series Data Set Historical Feature Data Set ARIMA Model Training Current Feature Data Set Regression Tree ML Model Training Campus-scale 15-min Prediction Errors ARIMA Forecasting Forecasted KWH Time Series Data Set Regression Tree Forecasting Actual KWH Time Series Data Set Calculate Error Measures Applications to Outage management, Energy markets, sustainable plant construction, Reliable renewable integration Next Frontiers in Big Data, Y. Simmhan 4-Sep-13 21

22 Analytics over Dynamic Data THE PLATFORM Next Frontiers in Big Data, Y. Simmhan 4-Sep-13 22

23 Floe: Adaptive Stream Processing Provision Cloud Service Provider Cloud Infrastructure IEEE SCALE Challenge winner, 2012 SC & escience, 2013 Papers on Dynamic Dataflows, Online Dataflow Updates Next Frontiers in Big Data, Y. Simmhan 4-Sep-13 23

24 SCEPter: Complex Event Processing with Semantics and operating across offline & online data Next Frontiers in Big Data, Y. Simmhan Big Data, 2013 paper on High Availability SCEP 4-Sep-13 24

25 GoFFish: Temporal Graph Analytics Scalable platform for graphoriented event analytics Highly performant Big Data framework for clusters & clouds Data analytics over heterogeneous inter-connected event sources Novel data mapping to time-series graphs, with efficient layout on distributed storage Compose and efficiently execute dataflow applications over timeseries graphs Next Frontiers in Big Data, Y. Simmhan 4-Sep-13 25

26 Analysis over Time-series Graphs Fixed graph of event sources Known relationships between sources E.g. pathway E.g. red car event from a camera Event Streams form graph time-series Track path of the red car Mine for interesting trajectory Inferring track from micro-paths Other analysis Wide Area outage management Event clustering and aggregation Topic propagation in social networks Next Frontiers in Big Data, Y. Simmhan 4-Sep-13 26

27 GoFFish Software Platform Platform to store, compose & execute analytics on time-series graph datasets At scale, on distributed systems GoFS: Distributed Graphoriented File System Gopher Compose sub-graph centric complex analytics Executed on Floe streaming dataflow engine Data & Compute collocated Next Frontiers in Big Data, Y. Simmhan 4-Sep-13 27

28 When Had ** p is just not good enough Gap in frameworks for dynamic graphs Tuple/row/column-oriented frameworks E.g. Hadoop, Hive, Impala Graph Databases & In-Memory E.g. Flock DB, Neo4J, MSR Trinity, Giraph Focus on large simple graphs, complex queries, distributed in-memory Parallel Graph Frameworks MPI, parallel computing, steep curve Specialized HPC/shared-memory hardware Storage & compute are (loosely)coupled Vertical Scaling Horizontal Scaling Next Frontiers in Big Data, Y. Simmhan 4-Sep-13 28

29 Design Insights Mitigate Weak Links 1) Disk I/O is the weakest link for big data Do more with less disk reads, parallel disk I/O Graph layout on distributed disks is key! 2) Network I/O Limit network communicate. Transfer in chunks. Graph partitioning on distributed hosts is key! 3) Memory Capacity Not all data fits in distributed memory Incremental loading & computation 4) CPU Think Hadoop Elastic Cloud execution, Leverage many-core Think Hive Think Pregel USP! Next Frontiers in Big Data, Y. Simmhan 4-Sep-13 29

30 Data Model Designed for sub-graph centric distributed computing Graphs» Partitions Distributed evenly across machines Sub-graphs Logical unit of operation Vertices Host A Host B Sub-graph is unit of distributed data access & operation Extends Google Pregel/Apache Giraph s vertex-centric BSP model no global view Next Frontiers in Big Data, Y. Simmhan 4-Sep-13 30

31 Data Model Designed for sub-graph distributed centric computing over time-series graphs Graph template Common features Graph instances Time-variant features Instance vertex follows the template vertex s partition Host A t 1 t 2 t 3 t4 t 5 Host B Next Frontiers in Big Data, Y. Simmhan 4-Sep-13 31

32 Logical (User) API Graph instances have topology, timestamp, name-value attributes for V & ESubgraph { } timestamp, V[], E_local[], E_remote[], V_attr[][], E_attr[][] PTId GetLocalPartition(GTId) SGTId[] GetSubgraphTemplates(PTId) Iterator<Subgraph> GetSubgraphInstances(SGTId, Start_time, End_time, Vertex_Attrs[], Edge_Attrs[]) Next Frontiers in Big Data, Y. Simmhan 4-Sep-13 32

33 Data Layout on local & DFS We operate on a distributed file system Commodity disks, hosts similar to HDFS Distributed Graph Layout (Across hosts) Local Partition Layout (Within a host) How do sub-graph templates, instances & attributes, over time, map to files on disk? Slice is a unit of disk access Split sub-graph model (by time, attributes) Group similar items (topology, attribute types, time-ranges) Compactly packed... Kryo, protobuf, custom Next Frontiers in Big Data, Y. Simmhan 4-Sep-13 33

34 GoFS Architecture Gopher Host Z Head Node Sub-Graph Task 0 Host A Data Node Host B Data Node Host X Data Node Task 1 Task 2 Task N Sub-Graph Task 1 Sub-Graph Task 2 Sub-Graph Task N Time-series Graph Data Model API [User] GoFS Slice (File System) Layout API [Developer] Graph Template Slices Partition Metadata Slice SubGraph Template Slices Attribute Instances Attribute Instances Partition Metadata Slice SubGraph Template Slices Attribute Instances Attribute Instances Partition Metadata Slice Graph Template Metadata Slice SubGraph Template Slices Attribute Instances Attribute Instances Network Next Frontiers in Big Data, Y. Simmhan 4-Sep-13 34

35 Gopher Design N 1 Vertex-centric graph model Google Pregel (Apache Giraph) N Message overhead, dynamic graphs Sub-graph centric streaming dataflows Sub-graphs reduce messaging, more local ops Streaming allows incremental execution Dataflow composition more flexible than BSP v i, M N 2 3 Optimizations and Analysis of BSP Graph Processing Models on Public Clouds, Redekopp, Simmhan & Prasanna, IPDPS 2013 Next Frontiers in Big Data, Y. Simmhan 4-Sep-13 35

36 Gopher SubGraph Prog. Model Sub-graph initiator [SG Id] compute(iterator t <SubGraph>) {SGId,t,M[]} Sub-graph processor [SG Id] compute(t, M[]) {SGId,t,M[]} Initiator/processor runs on GoFS data node Streaming/BSP messaging to remote edges Process subgraph before expanding to neighbors Allows # of supersteps to be reduced by O(subgraph), relative to Giraph/Pregel! Next Frontiers in Big Data, Y. Simmhan 4-Sep-13 36

37 IP # Vantage Point Visual Analytics Abstract rendering with Continuum/IU Hop Distance IP # Next Frontiers in Big Data, Y. Simmhan 4-Sep-13 37

38 To Conclude There is more to Big Data than 3-V s Interesting problems at the Intersection Dynamic Data & Realtime analytics are essential for emerging Cyber-Physical apps Industry vs Academia Gap in scalable analytics platforms for dynamic graphs GoFFish s focus on temporal graphs Look towards leveraging accelerated CI Next Frontiers in Big Data, Y. Simmhan 4-Sep-13 38

39 Acknowledgement ceng.usc.edu/~simmhan THANK YOU! Next Frontiers in Big Data, Y. Simmhan 4-Sep-13 39