Big Data Unlock the mystery and see what the future holds. Philip Sow SE Manager, SEA

Transcription

1 Big Data Unlock the mystery and see what the future holds Philip Sow SE Manager, SEA

2 THE ERA OF BIG DATA Big Data Market: Reach $32.1 Billion in 2015 & to $54.4 billion by 2017 The Vs Structure/Semi/Unstructured Business Operations Consumer Behaviour Demographic Sentiments, etc

3 MACHINE DATA

4 MACHINE DATA HAS VALUE Timestamp Phone Number Mar 01 19:18:50:000 aaa2 radiusd[12548]:[id local1.info] INFO RADOP(13) acct start for from recorded OK. IP Address :18:50: GET /sync/addtolibrary/ Track ID "Mozilla/5.0 (iphone; CPU iphone OS 5_0_1 like Mac OS X) AppleWebKit/ (KHTML, like Gecko) Version/5.1 Mobile/9A405 Safari/ " Mar 01 19:18:50:163 aaa2 radiusd[12548]:[id local1.info] INFO RADOP(13) acct stop for @splunktel.com from recorded OK. Error Code

5 Deliver Powerful Real Business Insights User Activity Top Tracks Click to investigate Customer experience Download Errors by device

6 COMPANIES WHO COMPETE WITH DATA WILL WIN

7 Addressing Business Operation Through Analytics App Dev & App Mgmt. IT Operations Security & Security & Complianc Compliance e Digital Intelligence Business Analytics Industrial Industrial Data Data and and Internet of Internet Things of Things Reduced MTTR 70% 90% Increased Uptime Drive innovation with analytics Reduced Escalations Tools consolidation Same data serves your business users Data Correlation Integration Predictive Analytics

8 The Big Data Challenge Multiple Siloes Existing system not design for big data Information does not keep up with time Not equipped with the right skills

9 Challenges With Alternative Approaches OPTION 1 Do it yourself Hadoop / Pig OPTION 2 Hive or SQL on Hadoop OPTION 3 Extract to in-memory store Problems Problems Problems Need to know MapReduce Wait for slow jobs to finish No interactive exploration Pre-defined fixed schema Need knowledge of data Miss data that doesn t fit Data too big to move Limited drill down to raw data Another data mart 9

10 THE BIG DATA REVOLUTIONS SCHEMA AT READ NOT AT WRITE TIME

11 ANALYTICS ECOSYSTEM SCHEMA AT WRITE 30 YEARS Ecosystem ~$100Bn ERP, CRM, SFA, etc SQL RDBMS APPS QUERY LANGUAGE DATA FABRIC SCHEMA AT READ NEXT 15 YEARS+ Next Gen Analytics Ecosystem Security? Personalization??? Search?? BIG DATA?? PLATFORM

12 THE BIG DATA REVOLUTIONS SEARCH WILL BE THE DE FACTO DATA QUERY LANGUAGE

13 Search, Investigate and Explore Your Data Actionable Visibility, Insight and Intelligence Web Services Shipping RFID Desktops Developers Databases/ DWH Online Shopping Carts Web Services GPS/Cellular Energy Manufacturing Servers Security Telecoms Networking Storage Messaging App Support Clickstream Social Media 13

14 Steps to Unlock the Big Data Mystery Existing data store External Intelligence Gathering of data Ad hoc search Monitor and alert Report and analyze Business Insight

15 Overview of a Typical Banking Environment Multi-channel Customer Service Multi-channel Digital Intelligence (Customer dept.) Mobile, App Internet Banking / Web ATM Branch Call Center Wealth Mgmt. Sales Big Data Business Analytics (Sales & Marketing Dept.) Product & Services Personal Financial Corporate Financial Credit Card Sales Mgmt. Marketing Mgmt. Operation Mgmt. Customer Service Wealth Mgmt. Sales Analysis Opportunity Mgmt. Customer Analysis Loyalty Analysis Core Banking Corporate Process Interact Mgmt. Wealth Mgmt. Insurance Agent Profile Mgmt. Analysis Mgmt. Service Mgmt. Mgmt. Cross Selling Investment Up & Cross Sell Market Campaign Product Financial Service Active Recommend Other Services Application Real-time Monitor &Contribution Use Analysis (Service Mgmt.) Behavior Branch Relationship Performance Customer Relationship Mgmt. Recommend Forecast IT Operational Intelligence (ITNetwork dept.)analysis Security Compliance (Security Infrastructure Security && Compliance dept.) IT & Application Infrastructure Real-time Risk Management & Fraud Detection (Risk Mgmt.) 15

16 Multi-Channel Digital Intelligence In-depth analysis in digital media to make better decision-making in the digital media era to drive better business operations 16

17 Use Case: Content Delivery and Web Analysis Better analytics using Splunk. Analyzes data across mobile and web delivery platforms Insight into: Mobile portal visits Most popular content Website volume by time Ad click analysis optimize delivery of ads based on value Individual ad performance metrics Mobile content / portal usage by subscribers 17

18 Strategic Focus Areas for Financial Services Diverse Business Processes Support End Customer Offerings Representative Business Processes Product & Services Customer Centricity Trade Processing FIX Order Management Online Banking Agile IT Infrastructure Regulatory & Compliance Customer Communication Settlement Systems Mortgage Origination Payment Processing Core Banking Processes Branch / ATM Management Trading Credit Cards Loans Insurance Cost-optimization Security Compliance (SOX/Basel) Business Services 18

19 Application Environment Complex Transaction Flows Sample Trade Processing Flow Book Trade Distributed Environment Validate Trade Multiple Applications Process Trade Match Trade Calculate Fees Calculate Risk Calculate Profit/Loss Middleware Compute Grid 1000s of Servers Distributed Cache Aggregation Engine LAN/WAN View Results Desktops 19

20 Visibility Across Systems is Challenging Highly Distributed SOA Environment Multiple Applications 1000s of Servers Middleware Distributed Processing Networking Desktops Example Issues Latency in trade processing or valuation Stale or incorrect market data Incorrect calculations for key fields Data not displayed on trader desktop Too much time spent between developers & support staff to diagnose 20

21 Transaction Profiling Across Processes How data across environment is used Troubleshoot performance issues Transaction tracing insight Root-cause analysis 21

22 Production Issue Investigation More intelligence Alerts on application errors Quick Root Cause Analysis Correlate to system utilization Correlate to Corporate Asset DB 22

23 Real Time Operational Intelligence Monitor timeout event Monitor average request time Number of ATM errors Drilldown capabilities 23

24 BIG DATA FUTURE

25 Thank You