4e INDUSTRIELE REVOLUTIE

Transcription

1 4e INDUSTRIELE REVOLUTIE MOUWEN OPSTROPEN & KANSEN BENUTTEN 1

2 4e INDUSTRIELE REVOLUTIE Sander van Lingen, Pathfinder - EMC ARCHITECTUUR Kenny Pool, Advisory System Engineer - EMC DIGITALE TRANSFORMATIE Edwin Leinse, Global Director Business Data Lake as a Service - Capgemini 2

3 3

4 MOVEMENTS SHIFT DIY GREEN ACCESS CONNECTED 4

5 5

6 6

7 OPPORTUNITIES EFFICIENCY GROW NEW SELL 7

8 CHALLENGES CULTURE INNOVATION DATA 8

9 9

10 Let s grap this opportunity Joe Biden Vice President of the USA 10

11 EMC BUSINESS DATA LAKE SOLUTIONS THE PLATFORM 11

12 CONSOLIDATED PLATFORM MANAGEMENT AUTOMATED DATA MANAGEMENT WITHOUT THE IT ADMIN PLATFORM MANAGER INGEST MANAGER ANALYTICS MANAGER DATA MANAGER Provision data storage environment Provision analytical processing environment Provision ingest environment Content indexing for all data types vagrant up for data 12

13 AVOIDING A DATA SWAMP HOW TO STAY OUT OF THE MUD AND KEEP INSIGHTS FLOWING Policy-driven data storage and retention Keep data relevant and reliable Control and audit data usage Demonstrate regulatory compliance DATA GOVERNOR Master data and metadata management 13

14 DELIVERING ANALYTICS FOR EVERYONE DATA SCIENCE FOR EVERYONE WITHOUT THE DATA SCIENTIST DATA AND ANALYTICS CATALOG Provide data services to applications But also analytics services Build views and applications users really need Rapidly move from tinkering to monetization 14

15 THE EMC BUSINESS DATA LAKE DATA AND ANALYTICS CATALOG OPEN PLATFORM MANAGER INGEST MANAGER ANALYTICS MANAGER DATA MANAGER ANALYTICS TOOLBOX PIVOTAL BIG DATA SUITE APPLICATIONS AT SCALE DATA PROCESSING HADOOP SPRING XD GREENPLUM DATABASE SPARK HAWQ PIVOTAL HD VMWARE VCLOUD SUITE EMC II STORAGE DATA LAKE FOUNDATION: ISILON VCE VBLOCK XTREMIO DATA DOMAIN ADVANCED ANALYTICS REDIS RABBITMQ GEMFIRE BDS ON PIVOTAL CLOUD FOUNDRY DATA GOVERNOR 15

16 FEDERATION BUSINESS DATA LAKE LOGICAL VIEW WORKSPACE USERS DATA SCIENTIST BUSINESS ANALYST IT ADMINISTRATOR ENTERPRISE ESI DISCOVERY AND INGEST Structured Data Unstructured Data Public records APPLICATION DEVELOPMENT Social Networks, UGC ENTERPRISE GOVERNANCE MANAGEMENT CATALOG Dark Data ANALYTICS HADOOP FABRIC Internet of things Location Data POLICIES LINEAGE DATA INFRASTRUCTURE DATA LAKE QUALITY VIRTUALIZATION SECURITY INGEST NETWORKING COMPUTE STORAGE POLICY-DRIVEN DATA LAKE DATA MANAGEMENT LAKE MANAGEMENT ENVIRONMENT ANALYTICS DATA SETS 16

17 A UNIFIED SOLUTION APPS SPEED ANALYTICS DATA 17

18 Capgemini Business Data Lake As A Service (BDLaaS)

19 Case Study: European Retail Major Retail Business Operations cover 2 countries, 220 product categories, 929 sub-categories, segments and products. Huge Challenges in timely reporting and analytics impacting agile business decisions for e.g., each category manager requires a custom report, which took more than an hour to run. Problem Areas Approach Benefits Delivered Performance Bottlenecks Defined Use Case and Test Scenarios 8bn records down to ticket line item level Ticket Header Analytics 60x faster Business Reports Delayed SKU Item Detail Analytics SKU Item Availability 100x faster 360x faster Inability to Use Predictive Analytics YoY Comparison SKU Item Detail YoY Comparison Ticket Header 45x faster 40x faster 19

20 What does this mean? Business driven Operations Marketing campaign SMAC Distill Keep the history Load everything HDFS Transactional systems Web Sensor Network ERP CRM PLM Supplier Social Media Market 20

21 Business Data Lake as a Service Platform as a Service Data Lake Analytical Tools Pivotal/Hortonwor ks/cloudera/haw Q Hadoop Access Layer 1 Access Layer 2 Access Layer 3 Cloud Foundry Layer Service Orchestration Layer Data Lake 1 Data Lake 2 Data Lake 3 Data Lake X Infrastructure Broker Layer Cloud On premise Compliance and Process Monitoring Monitoring 21

22 Architecture of Traditional Hadoop and Our solution DATA Node Name Node Traditional Hadoop cluster If you need more Analytical capacity you need too add more Nodes If you need more Data spaces you need too add more Nodes If you need more Parallelization you need too add more Nodes Our Solution as a Service App A Compute power Parallelization or IO/ups Storage Capacity If you need more Analytical capacity, You add more CPU & Memory If needed more Parallelization, You add More IO/up s If you need more storage space, You add More Disk Distil Store 22

23 Current offerings BDLaaS (PAAS) options Each offering scales up and out to do more in parallel and increase performance Scale Out Sandpit/POC Scale Out Business service Scale Out Enterprise Service Data Lake Primary Storage Primary Primary Storage Storage Analytical Tools Pivotal/ Hortonworks/Cloudera / Cloud Foundry Hortonworks Hadoop Data Lake Data Processing And Analytics Data Lake Data Processing And Analytics Scale Up Basic Data Basic Lack Data Lake Data Processing Data Processing And Analytics And Analytics Scale Up Availability Best Endeavor Availability of 99,5% Availability of 99,9% Better Availability and Service Guarantees 23

24 References Business objective of customers Data Big Data & Analytics Case Management as a Service High performance, assured application environment High reliability with availability options for the enterprise Reduced dependencies between analytics tools Enhanced availability and performance monitoring Capgemini BDLaaS platform solution Consumer Products & Retail All-Channel Experience (CP&R) Demand Driven Supply Chain Utilities Digital Utilities Transformation Smart Energy Services Automotive Connected Customer Connected Vehicle Connected Insights Platform is offered via Capgemini private EU cloud and delivers a secure, flexible and scalable Environment Capacity is delivered on a elastic usage principle to deploy BI and Analytics programs then execute information strategy in an agile and efficient manner to remain compliant with legislation This service is the foundation on which customers can perform their Big Data Analytics and comes with all required Big Data Suite installed. The Data Lake service also includes a customer specific Data Transport Layer, a technical helpdesk and technical application support. Results BDLaaS (POC and Production) platform available for customers since March 2015 and is located at the datacenter in Amsterdam Several POC and production customers are using the platform 24

25 Solution Advantages Hadoop allows to store and analyze all Structured and unstructured data Fully integrated solution A fully governance and compliance SLA driven service Provisioning in days or max week Fully automated and monitored offering in a Hybrid provisioning model A full Opex model possible Release management streamlined with limited dependency Specific tools for specific needs-hadoop, MPP, in Memory A fully managed Service Technical Application Management included Local Business fully empowered according global Business rules Agility Flexibility Cost Business Satisfaction 25

26

27 Use cases increase competitive advantage Virtually every industry has invested in big data to help solve specific challenges those industries face. Healthcare, for example, uses big data to improve patient outcomes, and agriculture uses data to boost crop yields. Financial firms use big data to help them identify sophisticated fraud schemes by combining multiple points of data Big data analytics can track trends in security breaches and allow companies to proactively go after threats Fraud Detection Through Big Data (from multiple sources) retailers can better optimize their product selection and pricing, as well as decide where to target ads Optimize Funnel Conversion Big Data Use Cases Market Basket Analysis and Pricing Optimization Big data analytics allows companies to track leads through the entire sales conversion process, from a click on an ad-word ad to the final transaction Customer Segmentati on Companies can learn what prompts a customer loyalty through targeted insights on their customer s characteristics and purchasing habits in order to maximize marketing & profits Assessing Big data about the consumer from multiple sources, such as social media data and transaction history, companies can better segment and target make personalized offers Through sensors and other machinegenerated data, companies can identify when a malfunction is likely to occur. Companies can preemptively order parts and make repairs in order to avoid downtime and lost profits. 27

28 Data in Transit issues or Collecting data AN Transactional systems ERP CRM 3 rd Party Web Social Media Supplier Sensor Data Transport Service Data lake Repor t Data Processing And Analytics 28