ROME, BIG DATA ANALYTICS

Transcription

1 ROME, BIG DATA ANALYTICS

2 BIG DATA FOUNDATIONS Big Data is #1 on the 2012 and the 2013 list of most ambiguous terms - Global language monitor 2

3 BIG DATA FOUNDATIONS Big Data refers to data sets whose size is beyond the ability of commonly used software tools to capture, manage, and process the data within a tolerable elapsed time. VOLUME VARIETY VELOCITY Volume: Big Data Analytics process a very large amount of records The size of Big Data is constantly moving : From TeraByte to PetaByte data From 100 Million to Billion row (and growing..) Variety: public, social media, comercial, operational, enteprise dark data Velocity: Not real-time (nor nearreal-time) processing, typically batch processing 3

4 BIG DATA FOUNDATIONS 4

5 BIG DATA KILLED DWH? 5

6 BIG DATA TECHNOLOGY LANDSCAPE Hadoop Appliances NoSQL Column Oriented In memory 6

7 BIG DATA FOUNDATIONS The general solution to big data problem is massive parallelism processing on distributed hardware (for both storage and processing), eventually using commodity hardware In Database In Memory MapReduce 7

8 BIG DATA FOUNDATIONS In Database In Memory MapReduce Performing Big Data Analytics involves manipulating very large datasets, we explained above dataset need to be partitioned, distributed, stored with caution Moving such large dataset to central memory in order to be elaborated would be difficult, so the idea is to move a part of the computation in the database itself. Dimensions are first reduced and then data can be moved to memory for further elaboration This is a challenge both for Business Analytics/BI, which needs mainly to aggregate those data, but also for Advanced Analytics which need to perform more complicated algorithms such as regression, clustering, time series algorithms Moving computation to the database + nodes model = algorithms performed on a subset 8

9 BIG DATA FOUNDATIONS In Database In Memory MapReduce In-memory processing speeds up BI by reducing or removing the need for disk input/output (I/O) The key benefit is fast response times the ability to return queries and deliver analysis to BI users much more rapidly For most organizations, in-memory processing reduces, but does not eliminate, the need to create aggregates or summaries in advance In-memory BI's "sweet spot" lies in powering interactive visualizations of large multidimensional datasets. It is less important for reporting use cases where interactivity is less intensive and traditional performance improvement techniques are applicable NEED FOR SPEED POWERS IN-MEMORY BUSINESS INTELLIGENCE JAMES RICHARDSON 9

10 BIG DATA FOUNDATIONS In Database In Memory MapReduce One of the bottlenecks towards processing large datasets is the need to store all data in memory. Therefore, users are limited to datasets that fit in memory limit. To avoid this, the natural approach is to split statistical algorithms in two steps In the first step the data processing is performed in database or on flat text file resulting in pre-computed data aggregates. In the second step these aggregates are imported into analytical engine where the rest of the analysis is performed. Such data aggregates are called sufficient statistics, because they contain all information necessary to compute parameter estimates, test statistics, confidence intervals and model summaries while are much smaller than the original dataset MASSIVELY PARALLEL ANALYTICS FOR LARGE DATASETS PRZEMYSLAW BIECEK, PAWEL CHUDZIAN, CEZARY DENDEK, JUSTIN LINDSEY 10

11 I4C BIG DATA PROPOSITION Advanced Analytics 11

12 I4C BIG DATA PROPOSITION BUSINESS NEED CAPABILITY TECHNOLOGY Semistructured, unstructured data NoSQL Hadoop, Cassandra, Mongo Big relational Appliance Netezza, SAP Hana Real time analytics In memory Kognition, ParDB,VoltDB 12

13 ACE LOGICAL SCHEMA 13

14 ACE LOGICAL SCHEMA 14

15 POLYGLOT PERSISTENCE One size does NOT fit all: because the diversity of the functional and technical requirements in enterprise applications. The factors that are driving the innovation in the data persistence space are: Data Volume Scalability High availability Fault tolerance Distributability Flexibility (i.e. "schemaless" databases) Polyglot Persistence is all about choosing the right persistence option for each analytical task POLYGLOT PERSISTENCE MARTIN FOWLER, SCOTT LEBERKNIGHT 15

16 CONNECTORS i4c proposition about polyglot persistence on Big Data databases is based on connectors Connectors decline ACE general-purpose analytic features on specific persistence stores, leveraging their own technical capabilities ACE Connectors: RDBMS (In-database analytics): Oracle Database IBM DB2 Appliances (parallel In-database and In-memory analytics): IBM PureData (formerly Netezza) SAP Hana DB Hadoop (Massively parallel Mapreduce) An ACE internal NoSQL In memory storage (High scalability and In-memory analytics) 16

17 ACE LOGICAL SCHEMA 17

18 BIG DATA ADVANCED ANALYTICS PARADIGM BFSI BUSINESS QUESTION Total Amount of Money in Accounts?

19 BIG DATA ADVANCED ANALYTICS PARADIGM BFSI Regression Money / customer AGE Linear Model Library of distributed Exact Algorythms Approximations LIN REG REG REG REG REG 19

20 BIG DATA ADVANCED ANALYTICS PARADIGM ADVANCED ANALYTICS BIG DATA ADVANCED ANALYTICS 20

21 ACE LOGICAL SCHEMA 21

22 BIG DATA ANALYTICS 22

23 BIG DATA ANALYTICS 23

24 BIG DATA ANALYTICS USE CASES MARKETING OPERATIONS FEEDBACK RISK TARGETING DEMAND FORECAST CUSTOMER SATIFACTION SOLVENCY PRICING CC FORECASTING COMPLAINTS MGMT CREDIT COLLECTION PROPENSITY IT PLANNING SENTIMENT ANALYSIS FRAUD DETECTION NEXT BEST PREDICTIVE MAINT. CUSTOMER VALUE REVENUE FORECASTING LOYALTY 24

25 AUTOGRILL CUSTOMER PROFILE Autogrill is the world s leading provider of food & beverage and retail services for travellers. Autogrill serves people on the move and operates primarily under concession agreements. The Group operates mainly in airports and motorways, followed by railway stations and a selective presence in high street, shopping centres, trade fairs, museums, and other cultural facilities. CHALLENGES Increasing shrink rate hard to manage Lack of an efficient loss prevention process Need to promptly detect frauds and take actions Understand new fraud patterns Managing frauds in a multi-country group 25

26 AUTOGRILL GOAL Leverage on business knowledge and find new anomalous behaviours Being able to detect frauds starting from a 360 view on stores and Cashiers Introducing analytics in the loss prevention process to gain insights i4c APPLICATION i4c APP Fraud Detection for Retail KEY SUCCESS FACTORS Ability to work at single transaction row from any point of view: store - cashier Ability to link actions to hi-risk transactions and user driven - automatic alerting Using both business knowledge and predictive analytics to evaluate every single transaction Easy to read indicators including analytical insights 26

27 AMADORI CUSTOMER PROFILE Amadori is one of European leaders in the production and trade of meat products. In particular, it has a market share of 30% of the total poultry meat. Italian innovative food company and a reference point for dishes based on meat. The turnover in 2011was over 1.2 billion Euros. CHALLENGES Listen and analyze consumer conversations about the Amadori brand, products and market, in order to define marketing strategies and communications. 27

28 AMADORI GOAL Detect news, hot topics, viral phenomena that cause evolution of tastes, behavior and consumption habits Profile consumers of food vertical Analyze Reputation & Brand Awareness of the brand, products and people within the Company i4c APPLICATION Analysis of conversations related to food Segmentation based on consumer sentiment towards the hot topics, interests, shopping habits Root-Cause Analysis of the brand strengths and weaknesses perceived by the market KEY SUCCESS FACTORS Launch of initiatives aimed at food bloggers and young target Review and improve the Corporate Social Responsibility Improve online presence to respond to consumer habits Development and launch of new products based on hot topics and desires of consumers 28

29 CREDEM CUSTOMER PROFILE Credem is a major private Italian bank, but behind its modern appearance lies a century-long history. Founded in 1910, it took on its present name in 1983, and today CREDEM can be found in 16 regions of Italy, offering a coverage achieved through a mixture of acquisitions and new branches. The banks focuses considerable attention on innovative channels, offering advanced remote banking systems to meet the needs for transaction speed and security. Credem combines technological innovation with a completely customer-centric approach to banking. CHALLENGES Having accurate customer information allows modern banking to anticipate customers needs. 29

30 credem GOAL carrying out prospecting that brings in high-potential customers developing the existing customer base stopping valued customers going elsewhere i4c APPLICATION Simone Parrotto, CRM at CREDEM: We have built the logic and models we use to offer our customers products (complementary to their existing portfolios) based on their history and contact channel. With i4c Analytics, we have found a partner with both knowledge of the finance market and in-depth advanced analytics expertise. The application is based on customer segmentation and propensity models which enhance the information stored in the DWH. KEY SUCCESS FACTORS This activity has already resulted in tangible benefits, namely increased revenue per campaign alongside lower costs. Mr Parrotto summed up: i4c Analytics offers an analytic journey, starting with the adoption of innovative tools, moving along a pathway of gradually increased automation, and ending with a highly effective embedding of these tools into the bank s processes. 30