1 Big Analytics: A Next Generation Roadmap Cloud Developers Summit & Expo: October 1, 2014 Neil Fox, CTO: SoftServe, Inc SoftServe, Inc.
2 Remember Life Before The Web? 1994
3 Even Revolutions Take Time 3
4 Big Analytics?
5 Big Analytics also describes the technology solutions used by giants such as Google, Amazon, and Facebook to process enormous data to provide you the best in internet, real-time services.
6 Value Proposition Public Safety: Durham, NC Police use Predictive Analytics to reduce crime rate by 50% Fraud Prevention: Insurance company uses Predictive Analytics to save $12 m annually Supply Chain Optimization Retailers using Predictive Analytics to forecast product demand, price, promotion and inventory management Healthcare Impacting every aspect of the healthcare system giving more personalized data to patients, providers and payers. Areas of focus include lifestyle, diet, exercise, research, and clinical trials Retail: Macy s boosts same store sales by 10% using 10s of millions of data points from twitter, social media, in-store and on-line.
7 And just how powerful is this stuff? Marketing (e.g. Target): "... computers crawled through the data, he was able to identify about 25 products that, when analyzed together, allowed him to assign each shopper a pregnancy prediction score. More important, he could also estimate her due date to within a small window, so Target could send coupons timed to very specific stages of her pregnancy."  Government (via SAP survey): "87% of federal and 75% of state IT officials believe that real-time Big Data has the potential to save a significant number of lives. [2}  
8 A Success Story The Durkheim Project Developed linguistics-driven prediction models to estimate the risk of suicide. & Reached 70% accuracy in cohort distinction  Deployed a real-time big data framework to capture opt-in mobile and social media data from veterans Fast Company said This may be the most vital use of big data we ve ever seen.   
9 So What is Slowing Adoption Down? 1. Resistance to change there's the drag exerted by relational database administrators who badly want to stick to what they know.  2. Volume, Variety, and Velocity big data problems have just as much to do with changing how you do data querying and processing as they do with handling the oft-cited "three V's" -- the big data parameters of volume, variety, and velocity. Volume (of data under management) - Data is growing from the Terabytes to the Petabytes for everyone Velocity (of transactions)- NoSQL simply lets you access your data differently Variety (of data) Unstructured?, structured?, semi-structured? 
10 Big Analytics Engineering Challenges Real Time Intelligence How to achieve Low Latency for personalized customer experience in real-time? Consumers Intelligent Agents How to improve System Performance for Data Science/ Analytics team? Data Discovery Business Reporting How to implement Self-Service with high Data Quality over terabytes and petabytes? Data Scientists/ Analysts Business Users
11 Complex Environment
12 Sample Technologies
13 But now Hadoop
14 though the adoption is still behind
15 Big Analytics Reference Architectures and Practices ROI-driven Big Analytics systems design based on proven Architectures and Technologies Maximum efficiency with the lowest Cost per Terabyte
16 So What to Do? To Properly Frame a Big Data + Analytics Project 1) What is the business goal? 2) What data can we get in support of this? 3) How can we display business intelligence intuitively? 4) Iterate, iterate, iterate Start with a focused, business-driven project, make sure the data is consistent with your vision and then apply advanced analytics without moving beyond human-understandable decisions.  
17 SoftServe, Inc. is a unique software development partner, offering specialized outsource technical staffing and integrative, and professional services to some of the world s premier technology companies.
18 Our Unique Integrated Approach Abiliton
19 Abiliton Big Analytics SoftServe s adaptive best practice framework for Big Data/Business Analytics transformation and optimization People Process Technology Competence Development Knowledge Model Performance Management Practices Training Catalog Organization Structure Optimal Team Structure Roles and Balance SDLC Optimization Governance Project Management Business Analysis Software Engineering Quality Control DevOps Metrics for Continuous Improvement Project Status Dashboard Data Science & Analysis Predictive Modeling Statistical Analysis Standard and Ad-Hoc Reports Data Engineering Solution Architecture System Modernization System Optimization Operations Automation
20 Data Integration Big Analytics: Technology Framework A modern integrated approach for solving Big Data/Business Analytics needs across multiple verticals and domains All Data Data Acquisition and Storing Analytics Applications Application data Media data: images, video, etc Social data Enterprise content data Machine, sensor, log data Docs and archives data Data Lake (Landing, Exploration and Archiving) Enterprise Data Warehousing Real-time Data Processing Complex Event Processing Real-time Query and Search Data Management (Governance, Security, Quality, MDM) Reporting and Analysis Predictive Modeling Data Mining UX and Visualization Customer Analytics Marketing Analytics Web/Mobile /Social Analytics IT Operational Analytics Fraud and Risk Analytics
21 Big Analytics Case Study: Network Security Business Goals: 1. Provide reporting platform in the cloud for services & applications usage analysis Charge customers based on the platform they are using, number of consumers applications etc. Technical Specs: Machine generated data Big Data: 7.5BLN log records per day Near real-time reporting Reports which touch billions of rows Solution: ETL - Talend DW HP Vertica/ InfoBright EE OLAP Pentaho BI JasperServer Pro
22 Big Analytics Case Study: Online Analytics Business Goals: Insights and optimization of all web, mobile, and social channels Optimization of recommendations for each visitor High return on online marketing investments Technical specs: Big Data > 1PB 10+ GB per customer/day 10+ Hadoop Clusters 15+ Aster Data Clusters Solution: Hadoop/HBase/Hive Aster Data Oracle Java/Flex
23 Takeaway: Metrics for Success Qualitative Performance: For example, compelling visualizations that make tasks easier (i.e. less complex) Quantitative Performance: For example, maximizing systems ROI, reducing TCO, or even saving lives Compliance and Data Governance: For example, privacy concerns & jurisdictional issues
24 Remember Life Before Big Analytics? 2025? Applications were generic Doctors (and patients) had one data point per year Doctors had to rely on their own ability to research Shoppers had to search for deals Companies had exabytes of data they were not using 24
25 Thank you! US OFFICES Austin, TX Fort Myers, FL Boston, MA Newport Beach, CA Salt Lake City, UT EUROPE OFFICES United Kingdom Germany Netherlands Ukraine Bulgaria WEBSITE: USA TELEPHONE Toll-Free: Office:
BUY BIG DATA IN RETAIL Table of contents What is Big Data?... How Data Science creates value in Retail... Best practices for Retail. Case studies... 3 7 11 1. Social listening... 2. Cross-selling... 3.
Benchmarking Large-Scale Data Management Insights from Presentation Confidentiality Statement The materials in this presentation are protected under the confidential agreement and/or are copyrighted materials
Proceedings of the 2015 Industrial and Systems Engineering Research Conference S. Cetinkaya and J. K. Ryan, eds. ERP adoption in enterprises with emerging Big Data Mehdi Khazaeli 1, Leili Javadpour 1 and
Big Data and Analytics The New Core of Financial Services Cloud Standards Customer Council March 18, 2013 Michael Versace, Global Research Director, Global Risk, IDC Financial Insights IDC Snap Shot Market
An Oracle White Paper March 2013 Big Data Analytics Advanced Analytics in Oracle Database Advanced Analytics in Oracle Database Disclaimer The following is intended to outline our general product direction.
White paper Proactive Planning for.. Big Data.. In government, Big Data presents both a challenge and an opportunity that will grow over time. Executive Summary Consider this list of government-adopted
A Forrester Consulting Thought Leadership Paper Commissioned By SAP Real-Time Data Management Delivers Faster Insights, Extreme Transaction Processing, And Competitive Advantage June 2013 Table Of Contents
Big Data at Cloud Scale Pushing the limits of flexible & powerful analytics Copyright 2015 Pentaho Corporation. Redistribution permitted. All trademarks are the property of their respective owners. For
BIG DATA ANALYTICS REFERENCE ARCHITECTURES AND CASE STUDIES Relational vs. Non-Relational Architecture Relational Non-Relational Rational Predictable Traditional Agile Flexible Modern 2 Agenda Big Data
QLIKVIEW AND BIG DATA A QlikView Technology White Paper July 2012 victa.nl firstname.lastname@example.org +31 74 2915208 Introduction There is an incredible amount of interest in the topic of Big Data at present: for many
Hurwitz ViCtOrY index Advanced Analytics: The Hurwitz Victory Index Report SAP Hurwitz Index d o u b l e v i c t o r Marcia Kaufman COO and Principal Analyst Daniel Kirsch Senior Analyst Table of Contents
How to embrace Big Data A methodology to look at the new technology Contents 2 Big Data in a nutshell 3 Big data in Italy 3 Data volume is not an issue 4 Italian firms embrace Big Data 4 Big Data strategies
IBM Software Thought Leadership White Paper June 2013 The top five ways to get started with big data 2 The top five ways to get started with big data Big data: A high-stakes opportunity Remember what life
For Big Data Analytics There s No Such Thing as Too Big The Compelling Economics and Technology of Big Data Computing March 2012 By: 4syth.com Emerging big data thought leaders Forsyth Communications 2012.
Why is BIG Data Important? March 2012 1 Why is BIG Data Important? A Navint Partners White Paper May 2012 Why is BIG Data Important? March 2012 2 What is Big Data? Big data is a term that refers to data
OPEN DATA CENTER ALLIANCE : sm Big Data Consumer Guide SM Table of Contents Legal Notice...3 Executive Summary...4 Introduction...5 Objective...5 Big Data 101...5 Defining Big Data...5 Big Data Evolution...7
Balancing Opportunity and Risk in Big Data A Survey of Enterprise Priorities and Strategies for Harnessing Big Data WHITE PAPER This document contains Confidential, Proprietary and Trade Secret Information
COULD VS. SHOULD: BALANCING BIG DATA AND ANALYTICS TECHNOLOGY The business world is abuzz with the potential of data. In fact, most businesses have so much data that it is difficult for them to process
what can businesses do to capture the full potential of big data? helping companies observe and assess their data sets, identify potential revenues and mitigate challenges contents introduction 3 identify
Industry Point of View: The Future of Retail Analytics: Pervasive Customer Insights Retail is a data-intensive industry, and has always been so. Retailers serve millions of customers via multiple formats
CIO Roundtable - Big March 13, 2013 Big and its Dimensions Big refers to internal and external data that is multi-structured, generated from diverse sources in near real-time and in large volumes making
SAP BusinessObjects Business Intelligence SAP BusinessObjects Business Intelligence 4.0 Solutions Empowering the Real-Time, Mobile, Social, and Global Enterprise SAP BusinessObjects Business Intelligence
THE BIGGER THE BETTER? Big Data in Financial Services T H E B I G G E R T H E B E T T E R? BIG DATA IN FINANCIAL SERVICES 4 CONTENTS ABOUT THIS REPORT 5 GENERAL VIEWPOINT 6 BIG DATA PILLARS 8 Pillar 1:
CHAPTER 1.7 Harnessing the Power of Big Data in Real Time through In-Memory Technology and Analytics SAP AG Companies today have more data on hand than they have ever had before. It is estimated that an
IBM Software Big Data & Analytics Thought Leadership White Paper Better business outcomes with IBM Big Data & Analytics The insights to transform your business with speed and conviction 2 Better business
Big Data in Big Companies Date: May 2013 Authored by: Thomas H. Davenport Jill Dyché Copyright Thomas H. Davenport and SAS Institute Inc. All Rights Reserved. Used with permission Introduction Big data
EXECUTIVE SUMMARY Big Data is not an uncommon term in the technology industry anymore. It s of big interest to many leading IT providers and archiving companies. But what is Big Data? While many have formed