1 Big Data for Big Intel Moty Fania, PE Big data Analytics Assaf Araki, Sr. Arch. Big data Analytics
2 Advanced Analytics Intel IT Corporate ownership of advanced analytics Team charter Solve strategic high value business units problems Leverage analytics to grow Intel s revenue Specialized in Big data and Machine Learning Skills: Software Engineering, Decision Science and Business Acumen
3 Harnessing Analytics Transform Data into actionable knowledge Actionable Insights We are drowning in data, but starving for knowledge
4 4 Copyright 2013, Intel Corporation. All rights reserved. The Challenge Datasets that are unmanageable using traditional technologies Capture Visualization Storage Big Data requires a new approach Analytics Search Sharing Adapted from Forrester Growing need to derive meaning from previously unexplored data
5 Intel s IT Strategy for Big Data Priority Embrace Big Data - Form an enterprise Big Data Analytics Competency Center Build Implement an internal, cost-effective big data platform and inparallel build the necessary skill- set Approach Systematically apply big data analytics across Intel to solves high value problems -> Business Value The value of our Big- Data efforts was about USD $100M in We expect that figure to grow 10x by 2014.
6 Proving the Value of Big Data Analytics Manufacturing Decrease manufacturing costs by Personalizing unit testing using its historical data Test time reduction Yield improvements Chip Validation Optimize the chip validation process to cut product time-to-market Coverage Bug handling Content optimization $100M Cut TTM by 25%
7 Proving the Value of Big Data Other Examples Advanced Threats & Malware Detection Uses big data technologies and statistical models to detect anomalous patterns of malicious activity. Sales & Marketing Drive customer engagements based upon analytics leveraging internal & external info Prioritize new customers engagements (Who?) Optimize offering (What?) Improve triggering (When?) Context Aware Recommendation Engine Generic, context aware recommendation system developed for Telmap and now leveraged by other use cases
8 Big Data Analytics Challenges
9 Big Data Challenges Analytic Platform Limitations Not all platforms support code execution (e.g. R, Java, C etc.) Most platform are specialized for specific purpose Storage structure (key value, document, relational etc.) Mix processing loads (batch vs. real time) Data load into the DBMS (batch vs. streaming) Solutions are immature ( lack of features, security, HA & multi tenancy) Big Data Analytics Platform Off lin e Operation Source Prediction Model Builder Prediction Model Query
10 Analytics Algorithms Challenges Task characteristics - State dependency, Distributed Learning, CPU & IO intensive, possibly real time processing Algorithm Limitations The Distribution Curse Most algorithms are written sequential A change in Data Scientist mindset is needed No cross platforms code Can t leverage most of R packages (~4000)
11 Solution A two layer Hybrid architecture Crunch raw data into meaningful patterns which do not tend to change dramatically Offline Raw data algorithm Underline patterns Run on a scalable platform (Hadoop), Gain scalability Use latest user data and underline patterns to compute user prediction on demand Online Compute prediction using computed model and latest data Prediction Use latest feedback for real time prediction DB
12 Noticeable trends Hadoop 2.x - YARN Copyright 2013, Intel Corporation. All rights reserved.
13 Noticeable trends In Memory GB Named - Berkeley Data Analytic Stack ( BDAS ) Distributed RAM processing 40-60GB/s Batch, Interactive & Stream in one Stack 16 cores
14 Intel BGU Hadoop Lab Joint effort of Intel & Information System Engineering department The cluster has ~200TB of storage Installed with Hadoop 2.x & Spark Focused on development of new distributed algorithms for ML Impact: Research - Allows researchers to mine larger datasets than before and develop more complex, distributed algorithms Curriculum - Run a masters course for mining massive datasets which focus on implementing distributed machine-learning algorithms
15 Summary We, at Intel, leverage Big Data analytics to systematically solve high-value business problems across Intel that couldn t be addressed effectively in the past Big Data analytics offers high value but has its own challenges Notable trends - Hadoop 2.0 and in-memory technologies The new Intel-BGU Hadoop Lab will support research and enable new curriculum
16 Q&A Copyright 2013, Intel Corporation. All rights reserved.
white paper Boosting Retail Revenue and Efficiency with Big Data Analytics A Simplified, Automated Approach to Big Data Applications: StackIQ Enterprise Data Management and Monitoring Abstract Contents
Hur hanterar vi utmaningar inom området - Big Data Jan Östling Enterprise Technologies Intel Corporation, NER Legal Disclaimers All products, computer systems, dates, and figures specified are preliminary
E-PAPER March 2014 Big Data & the Cloud: The Sum Is Greater Than the Parts Learn how to accelerate your move to the cloud and use big data to discover new hidden value for your business and your users.
An Oracle White Paper June 2013 Oracle: Big Data for the Enterprise Executive Summary... 2 Introduction... 3 Defining Big Data... 3 The Importance of Big Data... 4 Building a Big Data Platform... 5 Infrastructure
Big Data at Cloud Scale Pushing the limits of flexible & powerful analytics Copyright 2015 Pentaho Corporation. Redistribution permitted. All trademarks are the property of their respective owners. For
CIO Roundtable - Big March 13, 2013 Big and its Dimensions Big refers to internal and external data that is multi-structured, generated from diverse sources in near real-time and in large volumes making
ORACLE REAL-TIME DECISIONS KEY BUSINESS BENEFITS Improve business responsiveness. Optimize customer experiences with cross-channel real-time decisions at the point of interaction. Maximize the value of
PARC and SAP Co-innovation: High-performance Graph Analytics for Big Data Powered by SAP HANA Harnessing the combined power of SAP HANA and PARC s HiperGraph graph analytics technology for real-time insights
BUY BIG DATA IN RETAIL Table of contents What is Big Data?... How Data Science creates value in Retail... Best practices for Retail. Case studies... 3 7 11 1. Social listening... 2. Cross-selling... 3.
COULD VS. SHOULD: BALANCING BIG DATA AND ANALYTICS TECHNOLOGY The business world is abuzz with the potential of data. In fact, most businesses have so much data that it is difficult for them to process
IBM Software Big Data Retail Capitalizing on the power of big data for retail Adopt new approaches to keep customers engaged, maintain a competitive edge and maximize profitability 2 Capitalizing on the
How to embrace Big Data A methodology to look at the new technology Contents 2 Big Data in a nutshell 3 Big data in Italy 3 Data volume is not an issue 4 Italian firms embrace Big Data 4 Big Data strategies
Solution Brief Big Data in the Cloud: Converging Technologies How to Create Competitive Advantage Using Cloud-Based Big Data Analytics Why You Should Read This Document This paper describes how cloud and
Big Data Getting Value from Big Data: Focus on the Opportunities, Not the Obstacles Table of Contents 2 Embark on Your Big Data Journey with Confidence Getting Started, Keeping Moving 3 Big Data Hype Versus
White Paper BIG DATA-AS-A-SERVICE What Big Data is about What service providers can do with Big Data What EMC can do to help EMC Solutions Group Abstract This white paper looks at what service providers
IBM Software Thought Leadership White Paper June 2013 The top five ways to get started with big data 2 The top five ways to get started with big data Big data: A high-stakes opportunity Remember what life
32 Big Data: present and future Big Data: present and future Mircea Răducu TRIFU, Mihaela Laura IVAN University of Economic Studies, Bucharest, Romania email@example.com, firstname.lastname@example.org
Technology Insight Paper Big Data Maximizing the Flow By John Webster August 15, 2012 Enabling you to make the best technology decisions Big Data Maximizing the Flow 1 Big Data Maximizing the Flow 2 The
IBM Software Big Data & Analytics Thought Leadership White Paper Better business outcomes with IBM Big Data & Analytics The insights to transform your business with speed and conviction 2 Better business
Plug Into The Cloud with Oracle Database 12c ORACLE WHITE PAPER DECEMBER 2014 Disclaimer The following is intended to outline our general product direction. It is intended for information purposes only,
April 2013 Operational Intelligence: What It Is and Why You Need It Now Sponsored by Splunk Contents Introduction 1 What Is Operational Intelligence? 1 Trends Driving the Need for Operational Intelligence
INTELLIGENT BUSINESS STRATEGIES W H I T E P A P E R Architecting A Big Data Platform for Analytics By Mike Ferguson Intelligent Business Strategies October 2012 Prepared for: Table of Contents Introduction...
American Journal of Engineering Research (AJER) e-issn : 2320-0847 p-issn : 2320-0936 Volume-03, Issue-05, pp-266-270 www.ajer.org Research Paper Open Access Convergence of Big Data and Cloud Sreevani.Y.V.
Trends in Cloud Computing and Big Data Nikita Bhagat, Ginni Bansal, Dr.Bikrampal Kaur email@example.com, firstname.lastname@example.org, email@example.com Abstract - BIG data refers to the
BIG DATA IN THE CLOUD : CHALLENGES AND OPPORTUNITIES MARY- JANE SULE & PROF. MAOZHEN LI BRUNEL UNIVERSITY, LONDON Overview * Introduction * Multiple faces of Big Data * Challenges of Big Data * Cloud Computing
Digitizing Manufacturing: Ready, Set, Go! Manufacturing at the verge of a new industrial era 2 Content Executive Summary 04 The Need for Digitization 06 The Industry s Digital Maturity 08 Digital business
IT@Intel Achieving Intel Transformation through IT Innovation 2014 2015 Intel IT Business Review Annual Edition The Transformative Power of Innovation Kim Stevenson Intel Chief Information Officer Contents