The Rise of Industrial Big Data Brian Courtney General Manager Industrial Data Intelligence
Agenda Introduction Big Data for the industrial sector Case in point: Big data saves millions at GE Energy Seeking an industrial big data solution The power of advanced historians Investing in the future of Industrial Big Data Conclusion
Introduction
Introduction Industrial businesses have entered the age of big data manufacturing stores more data than any other sector close to 2 exabytes of new data stored in 2010. - McKinsey & Company Raw Industry Distribution Customer Proliferation of systems & devices Massive amounts of operational data Time-Series characteristics 4
Big Data for the industrial sector
Big Data for the industrial sector What is Time Series Data? 1 Definition Time-Series Control devices 2 Example data sources Pressure High-velocity Temperature Speed 3 Sampled and captured 4 Stored and analyzed Sampled down to the microsecond Millions streamed into storage Available for modeling, statistics and analytics 6
Big Data for the industrial sector How is Time Series Data Managed? Historians are the foundation of time series data management Collects, archives and distributes production information, securely, at extremely high speed Empowers everyone across the organization 7
Big Data for the industrial sector How is Time Series Data Contextualized and delivered? Asset Model Navigation Trending KPIs Alarms, Events & Advisories Near Real-Time Information 8
Big Data for the industrial sector How is Time Series Data Processed Batch processing Stream processing Offline analysis of massive data repositories for patterns and insights Real-time analysis of webscale data to identify trends and anomalies in data streams, as they occur Time series data stream 9
Big Data for the industrial sector How does data get Big? Customer example These figures reflect the data generated from just one of many machines that produce a particular personal care product, underscoring the sheer volume of data created by industrial companies. (174 Terabytes)
Big Data for the industrial sector GE has many Big Data Challenges Across the company GE handles over 5TB of data per day 11
Case in point Leveraging big data saves millions at GE Energy
Case Study GE Energy M&D Center 13
The problem turbine trips 14
Turbines monitored ~1,550 units monitored by M&D today 24x7x365 coverage Major Equipment Saves 51 58 61 37 08 09 10 11 *Not actual Turbine locations
Interface System deployed Where we were Data in 3 days Inefficient storage Unreliable & custom 80 application servers Legacy system Where we are Data in 30 seconds 10x speed 10X data density 4x lower storage & app costs Highly reliable & scalable Reduced IT footprint Proficy What we did 10 month project $4MM investment Agile approach ~30 software changes New system Qualify Collect Qualify P1 Analytics Analytics Data Loader File Storage Oracle DB P2 Oracle DB P3 Oracle DB LIM Database File Storage Visualization Visualization Workflow Visualization Workflow Workflow Reporting Vending Web Portal Portal Reporting Vending Analytics Workflow Visualization SOA Interface Time Series Historian 16
Allowed us to do Real Time Remote Performance Diagnostics Condition-Based Maintenance Trip limit Exhaust temp Prevention Detection Predictive Technology 17
Results Reduction of RDBs to Historian Reduced infrastructure maintenance cost Years of data on-line vs months Faster response early detection Fewer trips (unplanned outages) $75MM cost avoidance 2X increased customer service levels 18
Seeking an industrial data solution
BUT! Can we answer this question? Have I seen this start up sequence before in the last 10 years over my global fleet of assets? 0 500 1000 1500 2000 2500 20
GE Global Research 6 global sites >7,000 technologists Two Nobel Prize winners First U.S. industrial lab $600 million annual funding Founding principle improve businesses through technology
Investing in the future of Industrial Big Data What caused the quality issue? Ahh, an unknown correlation existed I was unaware of (100MB) 174 Terabytes down to Terabytes 22
Investing in the future of Industrial Big Data How do I optimize my plant of 50 assets by comparing performance over the last 12 months? Answer: 0.5 Terabytes X 50 machines crashed my analysis client! X 50 = 1305 Terabyte s or 1.3 Petabyte s 23
Can Industrial Big Data be solved? Questions MapReduce Distributed file storage Analytics at the data Don t move data Answers???? 24
Benchmarking environment 1 2 Tech vendor Tech vendor HP DL180s G6: Intel X5650 Processors, 48GB ram, 24 x 500GB 6G SAS 7.2k SFF disk 3 Tech vendor 4 5 Tech vendor Tech vendor 10 Gb 10 Gb Ethernet switch (Cisco Nexus 5010) 10 Gb 10 Gb 5 DL180s (1 master, 4 workers) 5 DL180s (1 master, 4 workers) 6 DL180 (1 used for benchmarking) 10 Gb 10 Gb 1 heavy storage node With 12 TB Storage 2 DL180s Read/Write load generation 25
Benchmark results One of the fastest write to disk Fastest read from 17.5x to 75,000x faster Only one optimized to read and write at same time Smallest Data footprint
Investing in the future of Industrial Big Data How do I optimize my plant of 50 assets by comparing performance over the last months? Return answer back to user. Distributed file storage Analytics at the data Don t move data 27
BUT! Can we answer this question? Have I seen this start up sequence before in the last 10 years over my global fleet of assets? 0 500 1000 1500 2000 2500 28
Yes We Can! In 4 TB of data with a 5 node Hadoop Cluster, 14 signals were found in 2 min 48 sec. 0 500 1000 1500 2000 2500 29
Investing in the future of Industrial Big Data
Conclusion Value exists in Big Data Industrial applications contain Big Data Advanced historians work for operational stores Combined with Big Data technology offers new answers GE has a solution today and investing for tomorrow Raw Industry Distribution Customer 31
The Rise of Industrial Big Data Brian Courtney General Manager Industrial Data Intelligence