Optimized for the Industrial Internet: GE s Industrial Data Lake Platform

Size: px
Start display at page:

Download "Optimized for the Industrial Internet: GE s Industrial Data Lake Platform"

Transcription

1 Optimized for the Industrial Internet: GE s Industrial Lake Platform

2 Agenda The Opportunity The Solution The Challenges The Results Solutions for Industrial Internet, deep domain expertise 2 #IndustrialInternet

3 Big opportunities with Industrial Big The power of 1% Driving outcomes that matter Increasing freight utilization rail Predictive maintenance healthcare Predictive diagnostics power $27B Industry value by reducing system inefficiency $63B Industry value by reducing process inefficiency $66B Industry value with efficiency improvements in gas-fired power plant fleets Note: Illustrative examples based on potential one percent savings applied across specific global industry sectors over 15 years. Source: GE estimates 3 #IndustrialInternet

4 Industrial Big fast and vast 50B Machines will be connected on the internet by X Industrial growth within next 10 years Sensor Historian CRM, ERP, etc. Geo-location Content (images, videos, manuals, etc.) Machine Logs Social network 35GB per day from each Smart Meter 50X growth in healthcare ( ) 1TB per flight In practice only 3% of potentially useful is tagged and even less is analyzed* 9MM points per hour for each locomotive 500GB per blade by gas turbines *Sources: IDC, IDC, Ericsson, Wikibon, Fast Company, ComputerWeekly 4 #IndustrialInternet

5 Today s approaches are not prepared for onslaught of Industrial Big Too slow Too expensive Too rigid 80% of an analytics project typically involves gathering and then preparing the for analysis* *Source: IDC 5 #IndustrialInternet

6 Yesterday s warehouse architecture What is it telling me? How is it doing? How does it look? 1 2 All over the place across multiple locations Limited types Mostly structured and semi-structured types scientist Field operations Business analyst ONE STATIC DATA MODEL 3 Snapshot Limited to narrow snapshots and time CRM, ERP, etc. Logs Social network Geo-location TRADITIONAL DATA WAREHOUSE 6 #IndustrialInternet

7 Industrial Lake architecture Underpinned by governance appropriate to Business and Location How long will it last without failures or maintenance? Is my asset performing optimally? How to configure for best operational results? Is my asset ready when there is market opportunity? One place Access to all in one place to quickly respond to the speed of business change Any Handing of all types including documents, images machine, sensor All Access to real-time and historical and not limited to snapshot of Sensor scientist Field operations Business analyst Content (images, videos, manuals, etc.) FLEXIBLE DATA MODELS INDUSTRIAL DATA LAKE Machine Historian CRM, ERP, etc. Logs, click streams Social network Geolocation Rapid access to all for analytics 7 #IndustrialInternet

8 A day in the life management scientist Current situation Field operations Business analyst Analytics and operations scientist New way Field operations Business analyst Add semantic meta Replica of source governance INDUSTRIAL DATA LAKE Add semantic meta Replica of source loading ingestion Real-time ingestion CRM, ERP, etc. Logs Social network Geo-location collection CRM, ERP, etc. Logs, click streams Social network Geolocation Sensor Content (images, videos, manuals, etc.) Machine Historian Time to analyze Cost scientist Field operations Business analyst Agility scientist Field operations Business analyst Time Cost INDUSTRIAL DATA LAKE collection ingestion Analytics and governance operations collection ingestion Analytics and governance operations Rigid Agile 8 #IndustrialInternet

9 Industrial Lake Customer focus Industrial Lake Appliance Pre-integrated with management, compute, and storage monetization and outcomes Consume Analyze Predictive / prescriptive analytics and visualization Security Management of all, any in one place Manage Process High performance computing 9 #IndustrialInternet

10 Industrial Lake Optimized for industrial workloads Optimized for missioncritical workloads for addressing key SLAs such as Security, resiliency etc. for Industrial Internet applications Fast ingestion, storage and compute including machine to support multiple schema and types Highperformance analysis using massively parallel processing architecture supporting Apache Hadoop governance and federation, with geographicallydispersed deployment options 10 #IndustrialInternet

11 Big without Governance Dumping into Big lake without repeatable processes and governance will create messy, uncontrollable environment Insights harvested from ungoverned lake, is not reliable and trustworthy If the insights can not be fully trusted, it s difficult to make business decisions confidently. Solutions for Industrial Internet, deep domain expertise 11 #IndustrialInternet

12 GE as a Custodian of Customer Owned & Services Custodian Infrastructure a person who has responsibility for or looks after something Synonyms: keeper, guardian, steward, protector "the custodian of the relic" Management Custodian Roles Enforcement & Measurement Protection Customer Owned Access Controls Visibility Metrics Privacy 12 #IndustrialInternet

13 Governance Disciplines Protect, Manage and Improve Information Quality Accuracy Completeness Consistency Lifecycle Provenance Lineage Retention Complianc e Regulatory Corporate Meta Dictionary Directory of all assets Classification and Tagging Auditing Monitoring Logging Log Analysis 13 #IndustrialInternet

14 Evolving Hadoop Governance Cluster Apache Falcon Uses Oozie and Ambari Set Process Define pipelines Monitor pipelines Trace pipelines for dependency, lineage 14 #IndustrialInternet

15 Industrial Lake Supports SLAs for industrial workload KPIs Availability Optimized for missioncritical workloads for Industrial Internet applications Capacity Elastic On-demand >99.99% 99.95% Planned downtime active disaster recovery Continuous operations, active-active Resiliency <30ms 30-40ms Medium/High High Performance / latency Industrial solutions OT focus (ex: M&D, CBM, ALM, etc.) Enterprise solutions IT focus (ex: CRM, SCM, ERP, etc.) Security 15 #IndustrialInternet

16 Security Risk for Big More implies higher risk of exposure New types may give rise to new security breach scenarios Evolving and experimental analysis implies security policies are less likely to be in place Linkage to other already under compliance may create scenarios where compliance could be violated. 16 #IndustrialInternet

17 Security Requirements Perimeter security Access control protection Visibility Challenge: Complete security solution does not exist for any of the popular big products 17 #IndustrialInternet

18 Top Opportunity Areas for Security Perimeter: Infrastructure Protection: Encryption Access Control: Privacy Visibility: Management Communication protocols Access policy based encryption Secure dissemination integrity/proven ance Key management Searching / filtering encrypted Secure collection / aggregation Proof of storage Secure outsourcing of computation Secure collaboration 18 #IndustrialInternet

19 Lake Security Solutions Physical Security Network Security Authentication Protecting the cluster(s) at rest and motion security obfuscation File Permissions Group Authorizations RBAC Configuration Management Provenance Lineage Change management Center Deployments Kerberos Authentication LDAP integration Segregation of duties Encryption and masking solutions FileSystem Groups LDAP Groups Identity Mgmt Tagging ETL Tools Map Reduce 19 #IndustrialInternet

20 Evolving Hadoop Security Apache Knox: Perimeter / Network security Apache Ranger : Authorization protection Audit tracking Apache Sentry: Authorization 20 #IndustrialInternet

21 Availability Excellence Framework H A f o r N I C, I S P, S e r v e r s, D i s k D a t a B a c k u p D R S t r a t e g y M o n i t o r i n g / A l e r t i n g N a m e N o d e HA C C B P r o c e s s M o n i t o r i n g / A l e r t i n g C o n t i n u o u s t o o l i m p r o v e m e n t s Q u i c k r e s p o n s e t o A l e r t s J V M i n s t r u m e n t a t i o n A u d i t i n g c h a n g e s C o n f i g f i l e s c o m m i t t e d t o Git R e s t r i c t e d a c c e s s t o P R O D P r e - t e s t e d, p r e - a p p r o v e d c h a n g e s t o b e d e p l o y e d o n l y #IndustrialInternet

22 Target Availability SLA Cost comparison SLA Cost associated Typical industry Use Case Feature list required <=99% $ Batch update systems, Retail Web Sites, Social Media sites, Big clusters 99.9% $$ Retail Web Sites, Social Media sites, Relational bases 99.99% $$$ Hi-Frequency Trading, Medical support systems % $$$$$ Hi-Frequency Trading, Medical support systems, Stock Exchanges ex. Nasdaq, NYSE, Air-traffic controllers NameNode HA, Higher Replication than 3, Hardware redundancy, Monitoring and Alerting, Centre Redundancy, 2X Projected Capacity implementation All of the above + Full Centre Redundancy, Automatic Failover, 3X Projected Capacity implementation All of the above + Full Centre Redundancy including near real time replication, 4X Projected Capacity implementation All of the above + Auto-recovering components, 5X Projected Capacity implementation 100% $$$$$$ Real-time Trading systems, Stock Exchanges ex. Nasdaq, NYSE, On-board flight computer, Air-traffic controllers All of the above + 10X Projected Capacity implementation 22 #IndustrialInternet

23 Case study GE Aviation Asset productivity, minimize disruptions, improved forecasting 25 Airlines 3.4M Flights 340TB 2000X Performance improvement 10X Cost reduction 7 days Time-to-market for new analytic app Note: Illustrative Aviation example based on Predix solution currently in development. Estimates based on exploration, simulation and asset utilization models. Isolate root causes Identify sub-optimal performance parts Minimize disruptions 23 #IndustrialInternet

24 Thank you General Electric reserves the right to make changes in specifications and features, or discontinue the product or service described at any time, without notice or obligation. These materials do not constitute a representation, warranty or documentation regarding the product or service featured. Illustrations are provided for informational purposes, and your configuration may differ. This information does not constitute legal, financial, coding, or regulatory advice in connection with your use of the product or service. Please consult your professional advisors for any such advice. GE, the GE Monogram, Predix, Predictivity are trademarks of General Electric Company General Electric Company All rights reserved.

Optimized for the Industrial Internet: GE s Industrial Data Lake Platform

Optimized for the Industrial Internet: GE s Industrial Data Lake Platform Optimized for the Industrial Internet: GE s Industrial Lake Platform Agenda Opportunity Solution Challenges Result GE Lake 2 GESoftware.com @GESoftware #IndustrialInternet Big opportunities with Industrial

More information

Industrial Internet @GE. Dr. Stefan Bungart

Industrial Internet @GE. Dr. Stefan Bungart Industrial Internet @GE Dr. Stefan Bungart The vision is clear The real opportunity for change surpassing the magnitude of the consumer Internet is the Industrial Internet, an open, global network that

More information

Big Data Storage Challenges for the Industrial Internet of Things

Big Data Storage Challenges for the Industrial Internet of Things Big Data Storage Challenges for the Industrial Internet of Things Shyam V Nath Diwakar Kasibhotla SDC September, 2014 Agenda Introduction to IoT and Industrial Internet Industrial & Sensor Data Big Data

More information

The IoT Inc Business Meetup Silicon Valley

The IoT Inc Business Meetup Silicon Valley The IoT Inc Business Meetup Silicon Valley Meeting 6 February 2015 Bruce Sinclair (Organizer): bruce@iot-inc.com Target of Meetup For business people selling products and services into IoT but of course

More information

The Future of Data Management

The Future of Data Management The Future of Data Management with Hadoop and the Enterprise Data Hub Amr Awadallah (@awadallah) Cofounder and CTO Cloudera Snapshot Founded 2008, by former employees of Employees Today ~ 800 World Class

More information

Predictive Analytics & Business Insights 2015, Chicago. Mudit Mangal Project Lead, Data Analytics, Supply Chain Sears Holdings Corporation 06/11/2015

Predictive Analytics & Business Insights 2015, Chicago. Mudit Mangal Project Lead, Data Analytics, Supply Chain Sears Holdings Corporation 06/11/2015 Predictive Analytics & Business Insights 2015, Chicago Mudit Mangal Project Lead, Data Analytics, Supply Chain Sears Holdings Corporation 06/11/2015 Agenda WHAT IS HAPPENING WHAT ARE BENEFITS AND CHALLENGES

More information

Datenverwaltung im Wandel - Building an Enterprise Data Hub with

Datenverwaltung im Wandel - Building an Enterprise Data Hub with Datenverwaltung im Wandel - Building an Enterprise Data Hub with Cloudera Bernard Doering Regional Director, Central EMEA, Cloudera Cloudera Your Hadoop Experts Founded 2008, by former employees of Employees

More information

HDP Hadoop From concept to deployment.

HDP Hadoop From concept to deployment. HDP Hadoop From concept to deployment. Ankur Gupta Senior Solutions Engineer Rackspace: Page 41 27 th Jan 2015 Where are you in your Hadoop Journey? A. Researching our options B. Currently evaluating some

More information

HADOOP SOLUTION USING EMC ISILON AND CLOUDERA ENTERPRISE Efficient, Flexible In-Place Hadoop Analytics

HADOOP SOLUTION USING EMC ISILON AND CLOUDERA ENTERPRISE Efficient, Flexible In-Place Hadoop Analytics HADOOP SOLUTION USING EMC ISILON AND CLOUDERA ENTERPRISE Efficient, Flexible In-Place Hadoop Analytics ESSENTIALS EMC ISILON Use the industry's first and only scale-out NAS solution with native Hadoop

More information

Data Refinery with Big Data Aspects

Data Refinery with Big Data Aspects International Journal of Information and Computation Technology. ISSN 0974-2239 Volume 3, Number 7 (2013), pp. 655-662 International Research Publications House http://www. irphouse.com /ijict.htm Data

More information

More Data in Less Time

More Data in Less Time More Data in Less Time Leveraging Cloudera CDH as an Operational Data Store Daniel Tydecks, Systems Engineering DACH & CE Goals of an Operational Data Store Load Data Sources Traditional Architecture Operational

More information

The Enterprise Data Hub and The Modern Information Architecture

The Enterprise Data Hub and The Modern Information Architecture The Enterprise Data Hub and The Modern Information Architecture Dr. Amr Awadallah CTO & Co-Founder, Cloudera Twitter: @awadallah 1 2013 Cloudera, Inc. All rights reserved. Cloudera Overview The Leader

More information

Integrating a Big Data Platform into Government:

Integrating a Big Data Platform into Government: Integrating a Big Data Platform into Government: Drive Better Decisions for Policy and Program Outcomes John Haddad, Senior Director Product Marketing, Informatica Digital Government Institute s Government

More information

The Promise of Industrial Big Data

The Promise of Industrial Big Data The Promise of Industrial Big Data Big Data Real Time Analytics Katherine Butler 1 st Annual Digital Economy Congress San Diego, CA Nov 14 th 15 th, 2013 Individual vs. Ecosystem What Happened When 1B

More information

Exploiting Data at Rest and Data in Motion with a Big Data Platform

Exploiting Data at Rest and Data in Motion with a Big Data Platform Exploiting Data at Rest and Data in Motion with a Big Data Platform Sarah Brader, sarah_brader@uk.ibm.com What is Big Data? Where does it come from? 12+ TBs of tweet data every day 30 billion RFID tags

More information

The Future of Data Management with Hadoop and the Enterprise Data Hub

The Future of Data Management with Hadoop and the Enterprise Data Hub The Future of Data Management with Hadoop and the Enterprise Data Hub Amr Awadallah Cofounder & CTO, Cloudera, Inc. Twitter: @awadallah 1 2 Cloudera Snapshot Founded 2008, by former employees of Employees

More information

Upcoming Announcements

Upcoming Announcements Enterprise Hadoop Enterprise Hadoop Jeff Markham Technical Director, APAC jmarkham@hortonworks.com Page 1 Upcoming Announcements April 2 Hortonworks Platform 2.1 A continued focus on innovation within

More information

Big Data, Physics, and the Industrial Internet! How Modeling & Analytics are Making the World Work Better."

Big Data, Physics, and the Industrial Internet! How Modeling & Analytics are Making the World Work Better. Big Data, Physics, and the Industrial Internet! How Modeling & Analytics are Making the World Work Better." Matt Denesuk! Chief Data Science Officer! GE Software! October 2014! Imagination at work. Contact:

More information

Hortonworks CISC Innovation day

Hortonworks CISC Innovation day Hortonworks CISC Innovation day Simon gregory sgregory@hortonworks.com Here was the ask Hortonworks' data reposition - how this works and the types of data you work with. 1: Data Types & Value. What have

More information

End to End Solution to Accelerate Data Warehouse Optimization. Franco Flore Alliance Sales Director - APJ

End to End Solution to Accelerate Data Warehouse Optimization. Franco Flore Alliance Sales Director - APJ End to End Solution to Accelerate Data Warehouse Optimization Franco Flore Alliance Sales Director - APJ Big Data Is Driving Key Business Initiatives Increase profitability, innovation, customer satisfaction,

More information

The role of technology in optimizing operations & improving productivity Anup Sharma, Global CIO, GE Oil & Gas

The role of technology in optimizing operations & improving productivity Anup Sharma, Global CIO, GE Oil & Gas Keynote: The role of technology in optimizing operations & improving productivity Anup Sharma, Global CIO, GE Oil & Gas Imagination at work GE today Power & Water Energy Management Oil & Gas GE Capital

More information

How a global bank is overcoming technical, business and regulatory barriers to use Hadoop for mission-critical applications

How a global bank is overcoming technical, business and regulatory barriers to use Hadoop for mission-critical applications Case study: How a global bank is overcoming technical, business and regulatory barriers to use Hadoop for mission-critical applications Background The bank operates on a global scale, with widely distributed

More information

A New Era Of Analytic

A New Era Of Analytic Penang egovernment Seminar 2014 A New Era Of Analytic Megat Anuar Idris Head, Project Delivery, Business Analytics & Big Data Agenda Overview of Big Data Case Studies on Big Data Big Data Technology Readiness

More information

SAP and Hortonworks Reference Architecture

SAP and Hortonworks Reference Architecture SAP and Hortonworks Reference Architecture Hortonworks. We Do Hadoop. June Page 1 2014 Hortonworks Inc. 2011 2014. All Rights Reserved A Modern Data Architecture With SAP DATA SYSTEMS APPLICATIO NS Statistical

More information

and NoSQL Data Governance for Regulated Industries Using Hadoop Justin Makeig, Director Product Management, MarkLogic October 2013

and NoSQL Data Governance for Regulated Industries Using Hadoop Justin Makeig, Director Product Management, MarkLogic October 2013 Data Governance for Regulated Industries Using Hadoop and NoSQL Justin Makeig, Director Product Management, MarkLogic October 2013 Who am I? Product Manager for 6 years at MarkLogic Background in FinServ

More information

Comprehensive Analytics on the Hortonworks Data Platform

Comprehensive Analytics on the Hortonworks Data Platform Comprehensive Analytics on the Hortonworks Data Platform We do Hadoop. Page 1 Page 2 Back to 2005 Page 3 Vertical Scaling Page 4 Vertical Scaling Page 5 Vertical Scaling Page 6 Horizontal Scaling Page

More information

Enterprise-grade Hadoop: The Building Blocks

Enterprise-grade Hadoop: The Building Blocks Enterprise-grade Hadoop: The Building Blocks An Ovum white paper for MapR Publication Date: 24 Sep 2014 Author name Summary Catalyst Hadoop was initially developed for trusted environments that did not

More information

Igniting the Next Industrial Revolution

Igniting the Next Industrial Revolution Igniting the Next Industrial Revolution Defining an M2M Technology Platform for the Industrial Internet M2M Evolution Conference, 30 Jan 2014 Nikhil Chauhan Director Product Marketing, GE Software Sufficiently

More information

HP Adaptive Backup and Recovery

HP Adaptive Backup and Recovery HP Adaptive Backup and Recovery Addressing Your BURA Strategy Today Reflects Your Business Purposes Tomorrow Scott Baker - Director, Enterprise Data Protection Andrew Dickerson Senior Manager, Backup,

More information

Session 0202: Big Data in action with SAP HANA and Hadoop Platforms Prasad Illapani Product Management & Strategy (SAP HANA & Big Data) SAP Labs LLC,

Session 0202: Big Data in action with SAP HANA and Hadoop Platforms Prasad Illapani Product Management & Strategy (SAP HANA & Big Data) SAP Labs LLC, Session 0202: Big Data in action with SAP HANA and Hadoop Platforms Prasad Illapani Product Management & Strategy (SAP HANA & Big Data) SAP Labs LLC, Bellevue, WA Legal disclaimer The information in this

More information

Big Data Management and Security

Big Data Management and Security Big Data Management and Security Audit Concerns and Business Risks Tami Frankenfield Sr. Director, Analytics and Enterprise Data Mercury Insurance What is Big Data? Velocity + Volume + Variety = Value

More information

Data Governance for Regulated Industries

Data Governance for Regulated Industries Data Governance for Regulated Industries Amir Halfon CTO, Worldwide Financial Service Agenda Components of Data Governance Challenges Solutions and Case Studies Q&A SLIDE: 2 Data Governance Considerations

More information

Microsoft SQL Server 2008 R2 Enterprise Edition and Microsoft SharePoint Server 2010

Microsoft SQL Server 2008 R2 Enterprise Edition and Microsoft SharePoint Server 2010 Microsoft SQL Server 2008 R2 Enterprise Edition and Microsoft SharePoint Server 2010 Better Together Writer: Bill Baer, Technical Product Manager, SharePoint Product Group Technical Reviewers: Steve Peschka,

More information

Internet of Things. Opportunity Challenges Solutions

Internet of Things. Opportunity Challenges Solutions Internet of Things Opportunity Challenges Solutions Copyright 2014 Boeing. All rights reserved. GPDIS_2015.ppt 1 ANALYZING INTERNET OF THINGS USING BIG DATA ECOSYSTEM Internet of Things matter for... Industrial

More information

White paper. The Big Data Security Gap: Protecting the Hadoop Cluster

White paper. The Big Data Security Gap: Protecting the Hadoop Cluster The Big Data Security Gap: Protecting the Hadoop Cluster Introduction While the open source framework has enabled the footprint of Hadoop to logically expand, enterprise organizations face deployment and

More information

Data Security in Hadoop

Data Security in Hadoop Data Security in Hadoop Eric Mizell Director, Solution Engineering Page 1 What is Data Security? Data Security for Hadoop allows you to administer a singular policy for authentication of users, authorize

More information

INDUSTRY BRIEF DATA CONSOLIDATION AND MULTI-TENANCY IN FINANCIAL SERVICES

INDUSTRY BRIEF DATA CONSOLIDATION AND MULTI-TENANCY IN FINANCIAL SERVICES INDUSTRY BRIEF DATA CONSOLIDATION AND MULTI-TENANCY IN FINANCIAL SERVICES Data Consolidation and Multi-Tenancy in Financial Services CLOUDERA INDUSTRY BRIEF 2 Table of Contents Introduction 3 Security

More information

Secure Cloud Computing Concepts Supporting Big Data in Healthcare. Ryan D. Pehrson Director, Solutions & Architecture Integrated Data Storage, LLC

Secure Cloud Computing Concepts Supporting Big Data in Healthcare. Ryan D. Pehrson Director, Solutions & Architecture Integrated Data Storage, LLC Secure Cloud Computing Concepts Supporting Big Data in Healthcare Ryan D. Pehrson Director, Solutions & Architecture Integrated Data Storage, LLC Learning Objectives After this session, the learner should

More information

HDP Enabling the Modern Data Architecture

HDP Enabling the Modern Data Architecture HDP Enabling the Modern Data Architecture Herb Cunitz President, Hortonworks Page 1 Hortonworks enables adoption of Apache Hadoop through HDP (Hortonworks Data Platform) Founded in 2011 Original 24 architects,

More information

The Impact of PaaS on Business Transformation

The Impact of PaaS on Business Transformation The Impact of PaaS on Business Transformation September 2014 Chris McCarthy Sr. Vice President Information Technology 1 Legacy Technology Silos Opportunities Business units Infrastructure Provisioning

More information

Ganzheitliches Datenmanagement

Ganzheitliches Datenmanagement Ganzheitliches Datenmanagement für Hadoop Michael Kohs, Senior Sales Consultant @mikchaos The Problem with Big Data Projects in 2016 Relational, Mainframe Documents and Emails Data Modeler Data Scientist

More information

THE JOURNEY TO A DATA LAKE

THE JOURNEY TO A DATA LAKE THE JOURNEY TO A DATA LAKE 1 THE JOURNEY TO A DATA LAKE 85% OF DATA GROWTH BY 2020 WILL COME FROM NEW TYPES OF DATA ACCORDING TO IDC, AS MUCH AS 85% OF DATA GROWTH BY 2020 WILL COME FROM NEW TYPES OF DATA,

More information

Non-Stop Hadoop Paul Scott-Murphy VP Field Techincal Service, APJ. Cloudera World Japan November 2014

Non-Stop Hadoop Paul Scott-Murphy VP Field Techincal Service, APJ. Cloudera World Japan November 2014 Non-Stop Hadoop Paul Scott-Murphy VP Field Techincal Service, APJ Cloudera World Japan November 2014 WANdisco Background WANdisco: Wide Area Network Distributed Computing Enterprise ready, high availability

More information

How to avoid building a data swamp

How to avoid building a data swamp How to avoid building a data swamp Case studies in Hadoop data management and governance Mark Donsky, Product Management, Cloudera Naren Korenu, Engineering, Cloudera 1 Abstract DELETE How can you make

More information

Quickly Deploy Microsoft Private Cloud and SQL Server 2012 Data Warehouse on Hitachi Converged Solutions. September 25, 2013

Quickly Deploy Microsoft Private Cloud and SQL Server 2012 Data Warehouse on Hitachi Converged Solutions. September 25, 2013 Quickly Deploy Microsoft Private Cloud and SQL Server 2012 Data Warehouse on Hitachi Converged Solutions September 25, 2013 1 WEBTECH EDUCATIONAL SERIES QUICKLY DEPLOY MICROSOFT PRIVATE CLOUD AND SQL SERVER

More information

Building Confidence in Big Data Innovations in Information Integration & Governance for Big Data

Building Confidence in Big Data Innovations in Information Integration & Governance for Big Data Building Confidence in Big Data Innovations in Information Integration & Governance for Big Data IBM Software Group Important Disclaimer THE INFORMATION CONTAINED IN THIS PRESENTATION IS PROVIDED FOR INFORMATIONAL

More information

Attunity Better Data Movement For The Internet Of Things

Attunity Better Data Movement For The Internet Of Things Attunity Better Data Movement For The Internet Of Things Internet of Things North America Chicago 4-15-2015 Kevin Petrie Senior Director Attunity 4/15/15 Telemetry Has Come A Long Way Weather Sensors on

More information

Enable your Modern Data Architecture by delivering Enterprise Apache Hadoop

Enable your Modern Data Architecture by delivering Enterprise Apache Hadoop Modern Data Architecture with Enterprise Apache Hadoop Hortonworks. We do Hadoop. Jeff Markham Technical Director, APAC jmarkham@hortonworks.com Page 1 Our Mission: Enable your Modern Data Architecture

More information

Big Data overview. Livio Ventura. SICS Software week, Sept 23-25 Cloud and Big Data Day

Big Data overview. Livio Ventura. SICS Software week, Sept 23-25 Cloud and Big Data Day Big Data overview SICS Software week, Sept 23-25 Cloud and Big Data Day Livio Ventura Big Data European Industry Leader for Telco, Energy and Utilities and Digital Media Agenda some data on Data Big Data

More information

Unisys ClearPath Forward Fabric Based Platform to Power the Weather Enterprise

Unisys ClearPath Forward Fabric Based Platform to Power the Weather Enterprise Unisys ClearPath Forward Fabric Based Platform to Power the Weather Enterprise Introducing Unisys All in One software based weather platform designed to reduce server space, streamline operations, consolidate

More information

IBM Software Hadoop in the cloud

IBM Software Hadoop in the cloud IBM Software Hadoop in the cloud Leverage big data analytics easily and cost-effectively with IBM InfoSphere 1 2 3 4 5 Introduction Cloud and analytics: The new growth engine Enhancing Hadoop in the cloud

More information

2015 Analyst and Advisor Summit. Advanced Data Analytics Dr. Rod Fontecilla Vice President, Application Services, Chief Data Scientist

2015 Analyst and Advisor Summit. Advanced Data Analytics Dr. Rod Fontecilla Vice President, Application Services, Chief Data Scientist 2015 Analyst and Advisor Summit Advanced Data Analytics Dr. Rod Fontecilla Vice President, Application Services, Chief Data Scientist Agenda Key Facts Offerings and Capabilities Case Studies When to Engage

More information

Hadoop in the Hybrid Cloud

Hadoop in the Hybrid Cloud Presented by Hortonworks and Microsoft Introduction An increasing number of enterprises are either currently using or are planning to use cloud deployment models to expand their IT infrastructure. Big

More information

Securing Hadoop in an Enterprise Context

Securing Hadoop in an Enterprise Context Securing Hadoop in an Enterprise Context Hellmar Becker, Senior IT Specialist Apache: Big Data conference Budapest, September 29, 2015 Who am I? 2 Securing Hadoop in an Enterprise Context 1. The Challenge

More information

Data Services Advisory

Data Services Advisory Data Services Advisory Modern Datastores An Introduction Created by: Strategy and Transformation Services Modified Date: 8/27/2014 Classification: DRAFT SAFE HARBOR STATEMENT This presentation contains

More information

Building Data-Driven Internet of Things (IoT) Applications

Building Data-Driven Internet of Things (IoT) Applications Building Data-Driven Internet of Things (IoT) Applications A four-step primer IOT DEMANDS NEW APPLICATIONS Automated homes. Connected cars. Smart cities. The Internet of Things (IoT) will forever change

More information

Journey to the Private Cloud. Key Enabling Technologies

Journey to the Private Cloud. Key Enabling Technologies Journey to the Private Cloud Key Enabling Technologies Jeffrey Nick Chief Technology Officer Senior Vice President EMC Corporation June 2010 1 The current I/T state: Infrastructure sprawl Information explosion

More information

Protecting Big Data Data Protection Solutions for the Business Data Lake

Protecting Big Data Data Protection Solutions for the Business Data Lake White Paper Protecting Big Data Data Protection Solutions for the Business Data Lake Abstract Big Data use cases are maturing and customers are using Big Data to improve top and bottom line revenues. With

More information

The Big Data Revolution: welcome to the Cognitive Era.

The Big Data Revolution: welcome to the Cognitive Era. The Big Data Revolution: welcome to the Cognitive Era. Yves Eychenne, Cloud Advisor, IBM Email: yves.eychenne@fr.ibm.com @yeychenne 2015 INTERNATIONAL BUSINESS MACHINES CORPORATION Agenda Big Data and

More information

The Advantages of Enterprise Historians vs. Relational Databases

The Advantages of Enterprise Historians vs. Relational Databases GE Intelligent Platforms The Advantages of Enterprise Historians vs. Relational Databases Comparing Two Approaches for Data Collection and Optimized Process Operations The Advantages of Enterprise Historians

More information

IBM Software Delivering trusted information for the modern data warehouse

IBM Software Delivering trusted information for the modern data warehouse Delivering trusted information for the modern data warehouse Make information integration and governance a best practice in the big data era Contents 2 Introduction In ever-changing business environments,

More information

Increased Security, Greater Agility, Lower Costs for AWS DELPHIX FOR AMAZON WEB SERVICES WHITE PAPER

Increased Security, Greater Agility, Lower Costs for AWS DELPHIX FOR AMAZON WEB SERVICES WHITE PAPER Increased Security, Greater Agility, Lower Costs for AWS DELPHIX FOR AMAZON WEB SERVICES TABLE OF CONTENTS Introduction... 3 Overview: Delphix Virtual Data Platform... 4 Delphix for AWS... 5 Decrease the

More information

Big Data Analytics Roadmap Energy Industry

Big Data Analytics Roadmap Energy Industry Douglas Moore, Principal Consultant, Architect June 2013 Big Data Analytics Energy Industry Agenda Why Big Data in Energy? Imagine Overview - Use Cases - Readiness Analysis - Architecture - Development

More information

Oracle Big Data Strategy Simplified Infrastrcuture

Oracle Big Data Strategy Simplified Infrastrcuture Big Data Oracle Big Data Strategy Simplified Infrastrcuture Selim Burduroğlu Global Innovation Evangelist & Architect Education & Research Industry Business Unit Oracle Confidential Internal/Restricted/Highly

More information

Compliance & Data Protection in the Big Data Age - MongoDB Security Architecture

Compliance & Data Protection in the Big Data Age - MongoDB Security Architecture Compliance & Data Protection in the Big Data Age - MongoDB Security Architecture Mat Keep MongoDB Product Management & Marketing mat.keep@mongodb.com @matkeep Agenda Data Security Landscape and Challenges

More information

HP Vertica OnDemand. Vertica OnDemand. Enterprise-class Big Data analytics in the cloud. Enterprise-class Big Data analytics for any size organization

HP Vertica OnDemand. Vertica OnDemand. Enterprise-class Big Data analytics in the cloud. Enterprise-class Big Data analytics for any size organization Data sheet HP Vertica OnDemand Enterprise-class Big Data analytics in the cloud Enterprise-class Big Data analytics for any size organization Vertica OnDemand Organizations today are experiencing a greater

More information

Luncheon Webinar Series May 13, 2013

Luncheon Webinar Series May 13, 2013 Luncheon Webinar Series May 13, 2013 InfoSphere DataStage is Big Data Integration Sponsored By: Presented by : Tony Curcio, InfoSphere Product Management 0 InfoSphere DataStage is Big Data Integration

More information

Data Lake In Action: Real-time, Closed Looped Analytics On Hadoop

Data Lake In Action: Real-time, Closed Looped Analytics On Hadoop 1 Data Lake In Action: Real-time, Closed Looped Analytics On Hadoop 2 Pivotal s Full Approach It s More Than Just Hadoop Pivotal Data Labs 3 Why Pivotal Exists First Movers Solve the Big Data Utility Gap

More information

Intel HPC Distribution for Apache Hadoop* Software including Intel Enterprise Edition for Lustre* Software. SC13, November, 2013

Intel HPC Distribution for Apache Hadoop* Software including Intel Enterprise Edition for Lustre* Software. SC13, November, 2013 Intel HPC Distribution for Apache Hadoop* Software including Intel Enterprise Edition for Lustre* Software SC13, November, 2013 Agenda Abstract Opportunity: HPC Adoption of Big Data Analytics on Apache

More information

Big Data Analytics: Today's Gold Rush November 20, 2013

Big Data Analytics: Today's Gold Rush November 20, 2013 Copyright 2013 Vivit Worldwide Big Data Analytics: Today's Gold Rush November 20, 2013 Brought to you by Copyright 2013 Vivit Worldwide Hosted by Bernard Szymczak Vivit Leader Ohio Chapter TQA SIG Copyright

More information

Becoming a Cloud Services Broker. Neelam Chakrabarty Sr. Product Marketing Manager, HP SW Cloud Products, HP April 17, 2013

Becoming a Cloud Services Broker. Neelam Chakrabarty Sr. Product Marketing Manager, HP SW Cloud Products, HP April 17, 2013 Becoming a Cloud Services Broker Neelam Chakrabarty Sr. Product Marketing Manager, HP SW Cloud Products, HP April 17, 2013 Hybrid delivery for the future Traditional IT Evolving current state Future Information

More information

Simple. Extensible. Open.

Simple. Extensible. Open. White Paper Simple. Extensible. Open. Unleash the Value of Data with EMC ViPR Global Data Services Abstract The following paper opens with the evolution of enterprise storage infrastructure in the era

More information

Addressing Risk Data Aggregation and Risk Reporting Ben Sharma, CEO. Big Data Everywhere Conference, NYC November 2015

Addressing Risk Data Aggregation and Risk Reporting Ben Sharma, CEO. Big Data Everywhere Conference, NYC November 2015 Addressing Risk Data Aggregation and Risk Reporting Ben Sharma, CEO Big Data Everywhere Conference, NYC November 2015 Agenda 1. Challenges with Risk Data Aggregation and Risk Reporting (RDARR) 2. How a

More information

Apache Hadoop in the Enterprise. Dr. Amr Awadallah, CTO/Founder @awadallah, aaa@cloudera.com

Apache Hadoop in the Enterprise. Dr. Amr Awadallah, CTO/Founder @awadallah, aaa@cloudera.com Apache Hadoop in the Enterprise Dr. Amr Awadallah, CTO/Founder @awadallah, aaa@cloudera.com Cloudera The Leader in Big Data Management Powered by Apache Hadoop The Leading Open Source Distribution of Apache

More information

Performance Management for Enterprise Applications

Performance Management for Enterprise Applications performance MANAGEMENT a white paper Performance Management for Enterprise Applications Improving Performance, Compliance and Cost Savings Teleran Technologies, Inc. 333A Route 46 West Fairfield, NJ 07004

More information

The Principles of the Business Data Lake

The Principles of the Business Data Lake The Principles of the Business Data Lake The Business Data Lake Culture eats Strategy for Breakfast, so said Peter Drucker, elegantly making the point that the hardest thing to change in any organization

More information

An Oracle White Paper October 2011. Oracle: Big Data for the Enterprise

An Oracle White Paper October 2011. Oracle: Big Data for the Enterprise An Oracle White Paper October 2011 Oracle: Big Data for the Enterprise Executive Summary... 2 Introduction... 3 Defining Big Data... 3 The Importance of Big Data... 4 Building a Big Data Platform... 5

More information

WHITEPAPER. A Technical Perspective on the Talena Data Availability Management Solution

WHITEPAPER. A Technical Perspective on the Talena Data Availability Management Solution WHITEPAPER A Technical Perspective on the Talena Data Availability Management Solution BIG DATA TECHNOLOGY LANDSCAPE Over the past decade, the emergence of social media, mobile, and cloud technologies

More information

What is a Petabyte? Gain Big or Lose Big; Measuring the Operational Risks of Big Data. Agenda

What is a Petabyte? Gain Big or Lose Big; Measuring the Operational Risks of Big Data. Agenda April - April - Gain Big or Lose Big; Measuring the Operational Risks of Big Data YouTube video here http://www.youtube.com/watch?v=o7uzbcwstu April, 0 Steve Woolley, Sr. Manager Business Continuity Dennis

More information

Agenda. Big Data & Hadoop ViPR HDFS Pivotal Big Data Suite & ViPR HDFS ViON Customer Feedback #EMCVIPR

Agenda. Big Data & Hadoop ViPR HDFS Pivotal Big Data Suite & ViPR HDFS ViON Customer Feedback #EMCVIPR 1 Agenda Big Data & Hadoop ViPR HDFS Pivotal Big Data Suite & ViPR HDFS ViON Customer Feedback 2 A World of Connected Devices Need a new data management architecture for Internet of Things 21% the % of

More information

Data Governance in the Hadoop Data Lake. Michael Lang May 2015

Data Governance in the Hadoop Data Lake. Michael Lang May 2015 Data Governance in the Hadoop Data Lake Michael Lang May 2015 Introduction Product Manager for Teradata Loom Joined Teradata as part of acquisition of Revelytix, original developer of Loom VP of Sales

More information

Internet of Things. Point of View. Turn your data into accessible, actionable insights for maximum business value.

Internet of Things. Point of View. Turn your data into accessible, actionable insights for maximum business value. Internet of Things Turn your data into accessible, actionable insights for maximum business value Executive Summary Use a connected ecosystem to create new levels of business value The Internet of Things

More information

Executive Summary... 2 Introduction... 3. Defining Big Data... 3. The Importance of Big Data... 4 Building a Big Data Platform...

Executive Summary... 2 Introduction... 3. Defining Big Data... 3. The Importance of Big Data... 4 Building a Big Data Platform... Executive Summary... 2 Introduction... 3 Defining Big Data... 3 The Importance of Big Data... 4 Building a Big Data Platform... 5 Infrastructure Requirements... 5 Solution Spectrum... 6 Oracle s Big Data

More information

WhitePaper. Private Cloud Computing Essentials

WhitePaper. Private Cloud Computing Essentials Private Cloud Computing Essentials The 2X Private Cloud Computing Essentials This white paper contains a brief guide to Private Cloud Computing. Contents Introduction.... 3 About Private Cloud Computing....

More information

BMC Control-M Workload Automation

BMC Control-M Workload Automation solution overview BMC Control-M Workload Automation Accelerating Delivery of Digital Services with Workload Management Table of Contents 1 SUMMARY 2 FASTER AND CHEAPER DYNAMIC WORKLOAD MANAGEMENT Minimize

More information

Deploying an Operational Data Store Designed for Big Data

Deploying an Operational Data Store Designed for Big Data Deploying an Operational Data Store Designed for Big Data A fast, secure, and scalable data staging environment with no data volume or variety constraints Sponsored by: Version: 102 Table of Contents Introduction

More information

Building your Big Data Architecture on Amazon Web Services

Building your Big Data Architecture on Amazon Web Services Building your Big Data Architecture on Amazon Web Services Abhishek Sinha @abysinha sinhaar@amazon.com AWS Services Deployment & Administration Application Services Compute Storage Database Networking

More information

ENABLING GLOBAL HADOOP WITH EMC ELASTIC CLOUD STORAGE

ENABLING GLOBAL HADOOP WITH EMC ELASTIC CLOUD STORAGE ENABLING GLOBAL HADOOP WITH EMC ELASTIC CLOUD STORAGE Hadoop Storage-as-a-Service ABSTRACT This White Paper illustrates how EMC Elastic Cloud Storage (ECS ) can be used to streamline the Hadoop data analytics

More information

SAP HANA Vora : Gain Contextual Awareness for a Smarter Digital Enterprise

SAP HANA Vora : Gain Contextual Awareness for a Smarter Digital Enterprise Frequently Asked Questions SAP HANA Vora SAP HANA Vora : Gain Contextual Awareness for a Smarter Digital Enterprise SAP HANA Vora software enables digital businesses to innovate and compete through in-the-moment

More information

IBM System x reference architecture solutions for big data

IBM System x reference architecture solutions for big data IBM System x reference architecture solutions for big data Easy-to-implement hardware, software and services for analyzing data at rest and data in motion Highlights Accelerates time-to-value with scalable,

More information

Secure Enterprise Mobility Management. Cloud-Based Enterprise Mobility Management. White Paper: soti.net

Secure Enterprise Mobility Management. Cloud-Based Enterprise Mobility Management. White Paper: soti.net Secure Enterprise Mobility Management White Paper: Cloud-Based Enterprise Mobility Management soti.net Background Facing a business environment of constant change and increasing complexity, enterprises

More information

Oracle Database - Engineered for Innovation. Sedat Zencirci Teknoloji Satış Danışmanlığı Direktörü Türkiye ve Orta Asya

Oracle Database - Engineered for Innovation. Sedat Zencirci Teknoloji Satış Danışmanlığı Direktörü Türkiye ve Orta Asya Oracle Database - Engineered for Innovation Sedat Zencirci Teknoloji Satış Danışmanlığı Direktörü Türkiye ve Orta Asya Oracle Database 11g Release 2 Shipping since September 2009 11.2.0.3 Patch Set now

More information

QLIKVIEW DEPLOYMENT FOR BIG DATA ANALYTICS AT KING.COM

QLIKVIEW DEPLOYMENT FOR BIG DATA ANALYTICS AT KING.COM QLIKVIEW DEPLOYMENT FOR BIG DATA ANALYTICS AT KING.COM QlikView Technical Case Study Series Big Data June 2012 qlikview.com Introduction This QlikView technical case study focuses on the QlikView deployment

More information

Machina Research. Where is the value in IoT? IoT data and analytics may have an answer. Emil Berthelsen, Principal Analyst April 28, 2016

Machina Research. Where is the value in IoT? IoT data and analytics may have an answer. Emil Berthelsen, Principal Analyst April 28, 2016 Machina Research Where is the value in IoT? IoT data and analytics may have an answer Emil Berthelsen, Principal Analyst April 28, 2016 About Machina Research Machina Research is the world s leading provider

More information

Big Data Services From Hitachi Data Systems

Big Data Services From Hitachi Data Systems SOLUTION PROFILE Big Data Services From Hitachi Data Systems Create Strategy, Implement and Manage a Solution for Big Data for Your Organization Big Data Consulting Services and Big Data Transition Services

More information

IBM Software Integrating and governing big data

IBM Software Integrating and governing big data IBM Software big data Does big data spell big trouble for integration? Not if you follow these best practices 1 2 3 4 5 Introduction Integration and governance requirements Best practices: Integrating

More information

Securing Hadoop. Sudheesh Narayanan. Chapter No.1 "Hadoop Security Overview"

Securing Hadoop. Sudheesh Narayanan. Chapter No.1 Hadoop Security Overview Securing Hadoop Sudheesh Narayanan Chapter No.1 "Hadoop Security Overview" In this package, you will find: A Biography of the author of the book A preview chapter from the book, Chapter NO.1 "Hadoop Security

More information

Data movement for globally deployed Big Data Hadoop architectures

Data movement for globally deployed Big Data Hadoop architectures Data movement for globally deployed Big Data Hadoop architectures Scott Rudenstein VP Technical Services November 2015 WANdisco Background WANdisco: Wide Area Network Distributed Computing " Enterprise

More information

How In-Memory Data Grids Can Analyze Fast-Changing Data in Real Time

How In-Memory Data Grids Can Analyze Fast-Changing Data in Real Time SCALEOUT SOFTWARE How In-Memory Data Grids Can Analyze Fast-Changing Data in Real Time by Dr. William Bain and Dr. Mikhail Sobolev, ScaleOut Software, Inc. 2012 ScaleOut Software, Inc. 12/27/2012 T wenty-first

More information

BIG DATA ANALYTICS REFERENCE ARCHITECTURES AND CASE STUDIES

BIG DATA ANALYTICS REFERENCE ARCHITECTURES AND CASE STUDIES BIG DATA ANALYTICS REFERENCE ARCHITECTURES AND CASE STUDIES Relational vs. Non-Relational Architecture Relational Non-Relational Rational Predictable Traditional Agile Flexible Modern 2 Agenda Big Data

More information