Big Data and Advanced Analytics Technologies for the Smart Grid

Transcription

1 1 Big Data and Advanced Analytics Technologies for the Smart Grid Arnie de Castro, PhD SAS Institute IEEE PES 2014 General Meeting July 27-31, 2014 Panel Session: Using Smart Grid Data to Improve Planning, Analytics, and Operation of the US Capital region T&D Systems

2 BIG DATA Big Data is Relative, not Absolute When volume, velocity and variety of data exceeds an organization s storage or compute capacity for accurate and timely decision-making Meter Traditional AMI Meter PMU Reads/month 1 2,880 77,760,000

3

4 Analytics Across the Energy Value Chain

5 5 Technologies for the Smart Grid Enterprise Analytics Situational Awareness, Descriptive to Predictive, Visualization Grid Operations Analytics Predictive Asset Maintenance, Outage Management, PMU Monitoring and Analytics, Smart Meter Analytics, Distribution Optimization Consumer Analytics Energy Forecasting, Consumption Analysis, Revenue Protection

6 ENTERPRISE ANALYTICS

7 Innovative Strategies for Big Data Analytics A flexible enterprise architecture that supports many data types and usage patterns Upstream use of analytics to optimize data relevance Real-time visualization and advanced analytics to accelerate understanding and action Common analytical framework across the enterprise

8 GRID OPTIMIZATION ANALYTICS

9 Predictive Asset Maintenance Identify equipment that is likely to fail and/or determine its remaining lifetime Prioritize problems based on business impact Determine root cause more quickly Provide automated reporting and alerting Provide a collaborative environment

10 The Problem: humidity sensor failure Pi graph: Indicates key parameters during the humidity sensor failure. NOTE: Humidity Sensor failed High NOx limit is 20ppm (we are seeing ~30 ppm below). Magenta line is instantaneous NOx ppm. Copyr i g ht 2012, SAS Ins titut e Inc. All rights res er ve d.

11 Root Cause Analysis Copyr i g ht 2012, SAS Ins titut e Inc. All rights res er ve d.

12 Diagnostic Sequence Diagram:

13

14 OUTAGE OPTIMIZATION

15 Outage Optimization- Balancing Customer Satisfaction and Reliability: SAIDI, SAIFI Hospital High Business High Residential High Residential Low Residential Medium Police High Emergency Services High School Medium Outages Outage Outage > 90

16 16 Traditional and Advanced Analytical Impact Methods

17 Outages by City during Hurricane Irene

18 18 Predicting Outages Narrowing Down the Variables Affecting Asset Failure Survival Analyses Modeling Asset Potential Storm Failure Outage Prediction Model Scoring

19 Outage Prediction Model 19

20 20 After the Storm Analytics Travel time calculation Modify MILP Framework for Customer Restoration Constraints MILP Solver to Create Optimal Solutions versus Standard Utility Routing

21 Constrained Customer Restoration Problem 21

22 PMU MONITORING AND ANALYTICS

23 Phasor Measurement Units (PMUs) Issue: A REAL WORLD EXAMPLE FROM THE POWER GRID Latency; a delay of 3 seconds or more may be too late to take action to control system stability, leading to a blackout. Background: With Phasor Measurement Units (PMUs), measurements taken are precisely time-synchronized and taken many times a second (i.e. 30 to 60 samples/second) offering dynamic visibility into the power system. Approach: Develop analytics to: Understand Steady State operation Detect events on the network Categorize the event on the network Direct appropriate action based on the event Capture data for post event analysis

24 WHAT ARE PHASOR MEASUREMENT UNITS? Phasor Measurement Units Next-Gen measurement devices for power grid Collect measurements (frequency, voltage, current, phase angle) at 30 meas/second Synchronized across locations by GPS clock Courtesy: US Dept of Energy

25 PMU Analytics Process Pi Server Data Quality/ Transform Event Detection Event Identification Event Quantification Notifications

26 DETAIL CHARTS FOR EVENT PMU Event Analysis Current oscillates after event, but then dampens down to normal

27 SIMILARITY ANALYSIS Event Identification Reference time series for various events Incoming data stream is compared to reference time series

28 SIMILARITY ANALYSIS Event Identification Similarity between incoming stream and reference time series is measured and quantified

29 SMART METER ANALYTICS

30 Smart Meter Analytics 30

31 Customer Analysis 31

32 Load Analysis 32

33 DISTRIBUTION OPTIMIZATION

34 Distribution Optimization GIS, OMS SCADA/DMS, Meter Data, Sensor Data Distribution Network Model Tap Changing Transformers Capacitors Regulators Distributed Generation Energy Storage Network Operations Model Load Forecasts Load Models Load Analytics Measurement and Verification Distribution Optimization Conservation Voltage Reduction Loss Minimization Direct Load Control Cost Optimization Distributed Intelligence Connectivity (Static) Data Operational (Dynamic) Data Optimization Software

35 ENERGY FORECASTING

36 Energy Forecasting Spatial load forecasting Outlier detection Demand response forecasting Weather forecasting Hydro/wind/solar generation forecasting Price forecasting

37 CONSUMPTION ANALYSIS

38 Load Profile Comparisons via Segmentation

39 ENABLING TECHNOLOGIES

40 HIGH PERFORMANCE ANALYTICS

41 HIGH- PERFORMANCE ANALYTICS

42 Analytics Server Architecture Massively Parallel Processing ( MPP ) in the context of SAS Visual Analytics SAS VA Server LASR Cluster LASR Cluster LASR Cluster Workspace Server SAS LASR Analytic Server SAS LASR Analytic Server SAS LASR Analytic Server MEMORY Mid-Tier Co-Located Data Storage Co-Located Data Storage Co-Located Data Storage STORAGE Metadata PROCESSING RDBMS Nonrelational ERP Hadoop unstructured PC Files DATA SOURCES

43 HIGH PERFORMANCE ANALYTICS TECHNIQUES

44 ANALYTICS FORECASTING Leveraging historical data to drive better insight into decision-making for the future TEXT ANALYTICS Finding treasures in unstructured data like social media or survey tools that could uncover insights about consumer sentiment DATA MINING Mine transaction databases for usage patterns that indicate abnormalities INFORMATION MANAGEMENT STATISTICS OPTIMIZATION Analyze massive amounts of data in order to accurately identify areas likely to produce the most profitable results Copyright 2012, SAS Institute Inc. All rights reserved.

45 EVENT STREAM PROCESSING

46 Event Stream Processing (ESP) ESP is a subcategory of Complex Event Processing (CEP) focused on analyzing/processing events in motion called Event Streams.* The SAS ESP is an embeddable engine that can be integrated into or front-end SAS solutions. * This is the definition provided by the Event Processing Technical Society Copyr i g ht 2012, SAS Ins titut e Inc. All rights res er ve d.

47 TYPICAL CHARACTERISTICS OF EVENT STREAM PROCESSING APPLICATIONS: Continuous queries on data in motion (with incremental results) Moves analytics from centralized data warehouse to edge analytics (closer to the occurrence of the events) Very low (max) event processing latencies (i.e., µsecs-msecs) High volumes (>100k events/sec) Derived event windows with retention policies Memory constrained for performance (i.e., Bounded state) Predetermined data mining, decision making, alerting, position management, scoring, profiling, Copyr i g ht 2012, SAS Ins titut e Inc. All rights res er ve d.

48 Hybrid (Multi-staged) Analytics: Streaming Analytics front-ending historical/predictive analytics ESP RDBMS ESPs store the queries and continuously stream data through the queries Databases store the data and periodically run queries against the stored data EVENTS INCREMENTAL RESULTS QUERIES RESULTS Copyr i g ht 2012, SAS Ins titut e Inc. All rights res er ve d.

49 ESP Utilities Power Grid Management Monitor the Power Grid for Concerning Meter Reading Patterns Suggestive of Less Than Optimal Health Event Stream Processing Server Data Flow Model: Substations Meter Readings Slot1: vmin>vmax Readings (source) vmingtvmax (copy) volchk1 (filter) volchk2 (filter) sigma1 (filter & pattern) sigma2 (filter & pattern) Meter readings are continuously published into Readings source window Readings window uses output slot feature with exvolion vmin>vmax to send bad readings to vmingtvmax window & good readings to cleanse readings window DataFlux data quality functions are used to cleanse the readings. Null fields are fixed via procedural window using prior state. Aggregate window adds meter stats to readings: count, ave ave vmax, ave vmin, stdev, stdev vmax, stdev vmin volchk1: vmin<minthresh1 or vmax>maxthresh1 Cleanse readings (compute& procedural) Slot 0: vmin<=vmax readingswstats (aggregate) volchk3 (filter) integral (pattern) volchk2: vmin<miinthresh2or vmax>maxthresh2 volchk3: vmin<=0 or vmax<=0 volchk4: vmin>avevmin+2*stdvmin or vmin<avevmin-2*stdvmin or : vmax>avevmax+2*stdvmax or vmax<avevmax- 2*stdVMAX sigma1: 4 out of 5 consecutive points fall beyond 1σ, on the same side of the centerline (mean) Grid Management Console volchk4 (filter) downtrend (pattern) Sigma2: 2 out of 3 consecutive points beyond 2σ, on the same side of the centerline (mean) Integral: 9 consecutive points either above or below the centerline (mean) Downtrend: trend of 6 points in a row either increasing or decreasing

50 Connected Device PROCESS REFERENCE ARCHITECTURE SOURCE DATA EVENT STREAM PROCESSING Batch Processing Data MODEL DEVELOPMENT / BATCH ANALYSIS / ALERT / REPORT / ROOT CAUSE / ADJUDICATE Trade/ Financial Feeds Sensor Data/ Smart Device Threshold Models Patterns Queries Model Deployment ACCESS SERVER Maintenance/ Quality Dashboard/Alerts Telemetry Network Traffic Databases Low Latency ACCESS ENGINES Streaming Data Access/Cleanse Data In-Memory Extreme Parallelism Distribution of Analytics Processes Customer Seg/ Next Best Offer Network Security/ Management Fraud & Compliance Mobile Dashboard/ Alerts Data Visualization Routers, switches Data Management Batch Processing Workflows/ Case Management

51 Analytics Solutions Across the Energy Value Chain