Why big data? Lessons from a Decade+ Experiment in Big Data

Size: px
Start display at page:

Download "Why big data? Lessons from a Decade+ Experiment in Big Data"

Transcription

1 Why big data? Lessons from a Decade+ Experiment in Big Data David Belanger PhD Senior Research Fellow Stevens Institute of Technology 1

2 What Does Big Look Like? 7 Image Source Page: 1,000 Image Source Page: ~C(10^5) DGB 5/2013 2

3 Challenge: Getting More Value from Data 9,570,000,000,000,000,000,000 Bytes** Managing Risk: Security Fraud Compliance Customer* Experience Data Analysis: Industry leading Information Mining Technology including RT Decision Support Data In Flight Efficient, Reliable, Secure Data Transportb Hundreds of Data Sources Volume Velocity Variability Information Visualization: The most effective ways to deliver Information & Alerts to decision makers? Operations: Market Network Services Process Control: Provisioning Forecasting & Planning ** * Customer 3 Specific Data is anonymized or aggregated. CS & Data Management: Storage and processing architectures That operate at scale, and in real time Vertical Services: Medical Smart Grid Hospitality Managed Services

4 Where Does It Come From? Transactional Data Internet + WWW + BB Access Mobility, Anywhere, Anytime Sensors, Ad Hoc Networks, IoT Integration Communications Entertainment 4

5 Applications of Big Data Application Type Service Oriented Retrieval, Individual Precision, Sparse Data Paths, Graphs, Relationships Diversity of Sources Real Time, Predictive, Data in Flight Driving Technology

6 Sparse Targets & Individual Precision Then Precision for many measurements, and most targets, are aggregate. Sampling is often used. Accessing a dataset which is a small subset of a huge dataset is difficult, especially for unstructured data. Surveys used for customer experience. Now Analysis is of individual events, and measured against individual metrics. Map/Reduce useful for finding relatively small subsets of very large datasets. Customer behavior/results can be used for customer experience. Search Misuse Fraud Customer Experience photoblog.nbcnews.com At&t Labs - Research DGB 5/2013 6

7 Paths, Graphs, & Relationships Then Most analysis and visualization done on graphs is relatively small scale, and seldom interactive. Now Networks, including explicit, implicit, and inferred, are analyzed and visualized at very large scale. DGB 5/2013 7

8 Diversity of Sources - Crowd Then Sources of large databases are largely from transactional systems such as credit card swipe, ATM, Financial Markets, or engineered systems such as communication networks, as well as science. Now Sources range from individual contributors, to sensors, to networks. The WWW and Mobile Smartphones have turned consumption and generation of data into an always on, anywhere activity. Including video. Health Vital Signs Personal Environment Sensors Ad Hoc Sensor Networks Image Source Page: 2011/09/07/follow-us-on-twitter-and-like-us-on- Image Source Page:

9 Consumer Items & Sensors The next crowd Devices That Can Be Networked & IP Addressable How can we best exploit the billions of devices, many mobile, as computing, sensing, and communications platforms? Pallets and Cases Machinery Home Appliances Vehicles & Handheld Devices Crowd Computers b Invisible Computing 10 Consumer Items Pallets and Cases Home Appliances Machinery Vehicles and Handheld Devices Sensors Machine to Machine Internet Will Far outnumber current IT Devices Page 9 DGB 5/2013 9

10 Convergence of Communication and Entertainment Can I find what I want and watch, when I want? Base Issue How do we differentiate our media services in a world of commodity media services? Vision Access to all media, from all time, from anywhere Challenges User interface, especially search Indexing Storage Distribution 10 Source: AT&T Labs

11 Diversity of Data Types - Variety Then Most large data sets are either combinations of alphanumeric fields, or text. Now Data types range from the traditional, structured alpha-numic fields, to semi-structured (e.g. Web), to unstructured (text, speech, video, image). All of these, and mixtures of them, are analyzed at scale. Vodeo/Image Mining Personal Environment Sensors Speech Mining Customer Experience Customer Interaction Record

12 Then Real time systems are custom engineered and controlled, typically with relatively small data in flight. Data communications expensive at scale. Data in Flight Now Analysis of individual events, and measured against individual metrics, and at very large scale is becoming relatively common. Internet of Things starting to drive another spike in growth rate. Health - Smart Slippers Safety, Gaming Location Based Services DGB 5/

13 Data Analysis Lifecycle: Process Control Monitor Analyze Instrument Decide Control 13

14 Then Data Analysis For large datasets, it is usually the case that relatively small samples must be used. Customer studies are often based on surveys. Study results are frequently on aggregate data. Data numeric or text. Now Characterized by analytics on the population of data, though some datasets are still so big that sampling must be used. Customer studies based on behavior, and at extreme detail. Wide scale use of relationships e.g. social networks. Data numeric, text, speech, image. Graphs, networks, and paths Relationships Visualized Recommender Systems Visual Pattern Recognition Machine Learning openclassroom.stanford.edu 14

15 Information Visualization Human in the loop Then Largely descriptive and embedded in reports or dashboards. Aggregate measures most common, and created from a fairly restricted set of models characterized by statistical system. Now Characterized by scale, interactivity, and integration. Usually real time with immediate drill down facilities. Often with powerful new models to express detail Sometimes derived from gaming systems. Graphs, networks, and paths Word Clouds Relationships Visualized Recommender Systems VizGems Transparency, Integration, Control Through Visualization DGB 5/

16 Then Driving Technology Innovation within Data Management over a period of 30 years largely controlled by Relational Model, with some exceptions such as OODB Data Manaaement Systems Interactive Visualization & Data Integration Now Dynamic research in new tools embodying technologies such as NoSQL, Map/Reduce, DSMS, DSW, R, et al.. Much driven by large web companies. Some basic technologies, e.g. compression, become much moer important. DSWS Data Stream Warehouse Systems DS1 Graph Analysis at Large Scale DS2 DS3 DSMS DSW Image Source Page: DGB 5/

17 Then Meta Challenges Significant systems containing sensitive data are not easily accessed. Complex semantics and poor integrity often exist, but impact is hidden because data is relatively closed. Integration, outside of joins, uncommon at scale. Now Protection of SPI data a constant problem. Transparency of use, integrity of data a concern. Open data provides much more opportunity for interesting new apps from integration, and semantic confusion. Integration complex. Data Governance Security Privacy Integrity Semantics Integration DGB 5/

18 Organizing for Innovation Then Classical research or exploratory development teams create new products, often in large teams with significant timelines. Careful attention is paid to decision gates to prevent runaway costs. Due to costs, decisions often top down. Now Small, elite teams create prototypes of potential products quickly, trial the prototypes, and, when successful, present for funding to go to market. Go to trial, very quick. Classical research provides technology base to prototyping teams, and partners with them. Crowd Sourced Funnel Innovation Laboratory InfoLab DGB 5/

19 It s About the Data: Opportunity An Environment for Discovery InfoLab Sandbox Data Production Research InfoLab Clients Transfer Customer Applications Platform Incubation Informa1on & Data Product/Marketing IT InfoLab Cycle Engineering Operations Source: 19 AT&T Labs

20 Some Lessons Learned Strategy A defendable niche provides time to mature scale It s all about the data The Goal is to Enable Fundamental Process and Product Changes Ask Big Questions: e.g. Can an IP Network Run Itself? DGB 5/2013 Page 20

21 Some Lessons Learned Technology Multidimensional technical expertise is essential: Network Computing Data Analysis Visualization - Domains The Nature of analytics has changed: parallel, streams, predictive, geospatial Data Feed management can scale linearly. That is really bad Tradeoffs: Optimize Speed vs. Accuracy Depth & Volatility: Rules vs. Differences DGB 5/

22 Some Lessons Learned Ecosystem Few corporations can ignore the broader technology world, and none should. Sometimes the most effective way to impress management is to go outside e.g. Netflix, Idol Customer Focus - Choose your partners well, and make them heroes Components of an ecosystemthe above image of an ecosystem includes the DGB 5/2013 following components some of...www.westone.wa.gov.au Page 22

23 A More Complete Picture Data Analysis Data Management Visualization Applications Sandbox Privacy Data Governance Policy, Process Data Security OA&M Software Sustainability Integrity Semantics Framing Questions Distribution & Ownership of Results DGB 5/2013 Page 23

24 Where Information Services Are Going Pervasive Monitoring/Control Internet of Things OPEN DATA COSM Xively 1 Traditional Services TP, DW, Analytic Reports 3 Next- Generation Value from Data Immersive, Augmented Reality Interfaces s2 s1 s3 Multiple RT Streams 2 Info in Flight Real Time Stream Mining Mining Unstructured Data Mobility Next Gen Analytics, Prediction Data Stream Mining Speech/Text Mining DGB 5/2013 Anywhere, AnyDevice 24

Big Data The Next Phase Lessons from a Decade+ Experiment in Big Data

Big Data The Next Phase Lessons from a Decade+ Experiment in Big Data Big Data The Next Phase Lessons from a Decade+ Experiment in Big Data David Belanger PhD Senior Research Fellow Stevens Institute of Technology dbelange@stevens.edu 1 Outline Big Data Overview Thinking

More information

Galaxy BI Consulting Services. Listening to Business, Applying Technology

Galaxy BI Consulting Services. Listening to Business, Applying Technology Galaxy BI Consulting Services Listening to Business, Applying Technology Who we are Incorporated in 1987. An ISO 9000:2008 organization. Amongst the most respected Information Technology Integrators. Leading

More information

The future: Big Data, IoT, VR, AR. Leif Granholm Tekla / Trimble buildings Senior Vice President / BIM Ambassador

The future: Big Data, IoT, VR, AR. Leif Granholm Tekla / Trimble buildings Senior Vice President / BIM Ambassador The future: Big Data, IoT, VR, AR Leif Granholm Tekla / Trimble buildings Senior Vice President / BIM Ambassador What is Big Data? 2 Big Data is when the amount of data becomes part of the problem 3 Big

More information

The Next Wave of Data Management. Is Big Data The New Normal?

The Next Wave of Data Management. Is Big Data The New Normal? The Next Wave of Data Management Is Big Data The New Normal? Table of Contents Introduction 3 Separating Reality and Hype 3 Why Are Firms Making IT Investments In Big Data? 4 Trends In Data Management

More information

An Integrated Big Data & Analytics Infrastructure June 14, 2012 Robert Stackowiak, VP Oracle ESG Data Systems Architecture

An Integrated Big Data & Analytics Infrastructure June 14, 2012 Robert Stackowiak, VP Oracle ESG Data Systems Architecture An Integrated Big Data & Analytics Infrastructure June 14, 2012 Robert Stackowiak, VP ESG Data Systems Architecture Big Data & Analytics as a Service Components Unstructured Data / Sparse Data of Value

More information

VIEWPOINT. High Performance Analytics. Industry Context and Trends

VIEWPOINT. High Performance Analytics. Industry Context and Trends VIEWPOINT High Performance Analytics Industry Context and Trends In the digital age of social media and connected devices, enterprises have a plethora of data that they can mine, to discover hidden correlations

More information

Architecting for the Internet of Things & Big Data

Architecting for the Internet of Things & Big Data Architecting for the Internet of Things & Big Data Robert Stackowiak, Oracle North America, VP Information Architecture & Big Data September 29, 2014 Safe Harbor Statement The following is intended to

More information

Understanding traffic flow

Understanding traffic flow White Paper A Real-time Data Hub For Smarter City Applications Intelligent Transportation Innovation for Real-time Traffic Flow Analytics with Dynamic Congestion Management 2 Understanding traffic flow

More information

Big Data Executive Survey

Big Data Executive Survey Big Data Executive Full Questionnaire Big Date Executive Full Questionnaire Appendix B Questionnaire Welcome The survey has been designed to provide a benchmark for enterprises seeking to understand the

More information

Data Refinery with Big Data Aspects

Data Refinery with Big Data Aspects International Journal of Information and Computation Technology. ISSN 0974-2239 Volume 3, Number 7 (2013), pp. 655-662 International Research Publications House http://www. irphouse.com /ijict.htm Data

More information

5 Keys to Unlocking the Big Data Analytics Puzzle. Anurag Tandon Director, Product Marketing March 26, 2014

5 Keys to Unlocking the Big Data Analytics Puzzle. Anurag Tandon Director, Product Marketing March 26, 2014 5 Keys to Unlocking the Big Data Analytics Puzzle Anurag Tandon Director, Product Marketing March 26, 2014 1 A Little About Us A global footprint. A proven innovator. A leader in enterprise analytics for

More information

BEYOND BI: Big Data Analytic Use Cases

BEYOND BI: Big Data Analytic Use Cases BEYOND BI: Big Data Analytic Use Cases Big Data Analytics Use Cases This white paper discusses the types and characteristics of big data analytics use cases, how they differ from traditional business intelligence

More information

ICT Perspectives on Big Data: Well Sorted Materials

ICT Perspectives on Big Data: Well Sorted Materials ICT Perspectives on Big Data: Well Sorted Materials 3 March 2015 Contents Introduction 1 Dendrogram 2 Tree Map 3 Heat Map 4 Raw Group Data 5 For an online, interactive version of the visualisations in

More information

Big Data and Your Data Warehouse Philip Russom

Big Data and Your Data Warehouse Philip Russom Big Data and Your Data Warehouse Philip Russom TDWI Research Director for Data Management April 5, 2012 Sponsor Speakers Philip Russom Research Director, Data Management, TDWI Peter Jeffcock Director,

More information

Addressing Open Source Big Data, Hadoop, and MapReduce limitations

Addressing Open Source Big Data, Hadoop, and MapReduce limitations Addressing Open Source Big Data, Hadoop, and MapReduce limitations 1 Agenda What is Big Data / Hadoop? Limitations of the existing hadoop distributions Going enterprise with Hadoop 2 How Big are Data?

More information

ANALYTICS STRATEGY: creating a roadmap for success

ANALYTICS STRATEGY: creating a roadmap for success ANALYTICS STRATEGY: creating a roadmap for success Companies in the capital and commodity markets are looking at analytics for opportunities to improve revenue and cost savings. Yet, many firms are struggling

More information

Big Data: Key Concepts The three Vs

Big Data: Key Concepts The three Vs Big Data: Key Concepts The three Vs Big data in general has context in three Vs: Sheer quantity of data Speed with which data is produced, processed, and digested Diversity of sources inside and outside.

More information

This Symposium brought to you by www.ttcus.com

This Symposium brought to you by www.ttcus.com This Symposium brought to you by www.ttcus.com Linkedin/Group: Technology Training Corporation @Techtrain Technology Training Corporation www.ttcus.com Big Data Analytics as a Service (BDAaaS) Big Data

More information

www.objectivity.com Choosing The Right Big Data Tools For The Job A Polyglot Approach

www.objectivity.com Choosing The Right Big Data Tools For The Job A Polyglot Approach www.objectivity.com Choosing The Right Big Data Tools For The Job A Polyglot Approach Nic Caine NoSQL Matters, April 2013 Overview The Problem Current Big Data Analytics Relationship Analytics Leveraging

More information

Augmented Search for IT Data Analytics. New frontier in big log data analysis and application intelligence

Augmented Search for IT Data Analytics. New frontier in big log data analysis and application intelligence Augmented Search for IT Data Analytics New frontier in big log data analysis and application intelligence Business white paper May 2015 IT data is a general name to log data, IT metrics, application data,

More information

Turn your information into a competitive advantage

Turn your information into a competitive advantage INDLÆG 03 Data Driven Business Value Turn your information into a competitive advantage Jonas Linders 04.10.2015 (dato) CGI Group Inc. 2015 Jonas Linders Education Role Industries M.Sc Informatics Experience

More information

Better Decision Making

Better Decision Making Better Decision Making Big Data Analytics Webinar, November 2013 Dr. Wolfgang Martin Analyst and Member of the Boulder BI Brain Trust Better Decision Making Process Oriented Businesses. Decision Making:

More information

BIG DATA & ANALYTICS. Transforming the business and driving revenue through big data and analytics

BIG DATA & ANALYTICS. Transforming the business and driving revenue through big data and analytics BIG DATA & ANALYTICS Transforming the business and driving revenue through big data and analytics Collection, storage and extraction of business value from data generated from a variety of sources are

More information

A Hurwitz white paper. Inventing the Future. Judith Hurwitz President and CEO. Sponsored by Hitachi

A Hurwitz white paper. Inventing the Future. Judith Hurwitz President and CEO. Sponsored by Hitachi Judith Hurwitz President and CEO Sponsored by Hitachi Introduction Only a few years ago, the greatest concern for businesses was being able to link traditional IT with the requirements of business units.

More information

Luncheon Webinar Series May 13, 2013

Luncheon Webinar Series May 13, 2013 Luncheon Webinar Series May 13, 2013 InfoSphere DataStage is Big Data Integration Sponsored By: Presented by : Tony Curcio, InfoSphere Product Management 0 InfoSphere DataStage is Big Data Integration

More information

Big Data Challenges and Success Factors. Deloitte Analytics Your data, inside out

Big Data Challenges and Success Factors. Deloitte Analytics Your data, inside out Big Data Challenges and Success Factors Deloitte Analytics Your data, inside out Big Data refers to the set of problems and subsequent technologies developed to solve them that are hard or expensive to

More information

Social Data Science for Intelligent Cities

Social Data Science for Intelligent Cities Social Data Science for Intelligent Cities The Role of Social Media for Sensing Crowds Prof.dr.ir. Geert-Jan Houben TU Delft Web Information Systems & Delft Data Science WIS - Web Information Systems Why

More information

Hadoop Beyond Hype: Complex Adaptive Systems Conference Nov 16, 2012. Viswa Sharma Solutions Architect Tata Consultancy Services

Hadoop Beyond Hype: Complex Adaptive Systems Conference Nov 16, 2012. Viswa Sharma Solutions Architect Tata Consultancy Services Hadoop Beyond Hype: Complex Adaptive Systems Conference Nov 16, 2012 Viswa Sharma Solutions Architect Tata Consultancy Services 1 Agenda What is Hadoop Why Hadoop? The Net Generation is here Sizing the

More information

Enterprise Application Enablement for the Internet of Things

Enterprise Application Enablement for the Internet of Things Enterprise Application Enablement for the Internet of Things Prof. Dr. Uwe Kubach VP Internet of Things Platform, P&I Technology, SAP SE Public Internet of Things (IoT) Trends 12 50 bn 40 50 % Devices

More information

Big Data better business benefits

Big Data better business benefits Big Data better business benefits Paul Edwards, HouseMark 2 December 2014 What I ll cover.. Explain what big data is Uses for Big Data and the potential for social housing What Big Data means for HouseMark

More information

Big Data and Healthcare Payers WHITE PAPER

Big Data and Healthcare Payers WHITE PAPER Knowledgent White Paper Series Big Data and Healthcare Payers WHITE PAPER Summary With the implementation of the Affordable Care Act, the transition to a more member-centric relationship model, and other

More information

Achieving Business Value through Big Data Analytics Philip Russom

Achieving Business Value through Big Data Analytics Philip Russom Achieving Business Value through Big Data Analytics Philip Russom TDWI Research Director for Data Management October 3, 2012 Sponsor 2 Speakers Philip Russom Research Director, Data Management, TDWI Brian

More information

Managing Big Data with Hadoop & Vertica. A look at integration between the Cloudera distribution for Hadoop and the Vertica Analytic Database

Managing Big Data with Hadoop & Vertica. A look at integration between the Cloudera distribution for Hadoop and the Vertica Analytic Database Managing Big Data with Hadoop & Vertica A look at integration between the Cloudera distribution for Hadoop and the Vertica Analytic Database Copyright Vertica Systems, Inc. October 2009 Cloudera and Vertica

More information

REAL-TIME OPERATIONAL INTELLIGENCE. Competitive advantage from unstructured, high-velocity log and machine Big Data

REAL-TIME OPERATIONAL INTELLIGENCE. Competitive advantage from unstructured, high-velocity log and machine Big Data REAL-TIME OPERATIONAL INTELLIGENCE Competitive advantage from unstructured, high-velocity log and machine Big Data 2 SQLstream: Our s-streaming products unlock the value of high-velocity unstructured log

More information

2015 Analyst and Advisor Summit. Advanced Data Analytics Dr. Rod Fontecilla Vice President, Application Services, Chief Data Scientist

2015 Analyst and Advisor Summit. Advanced Data Analytics Dr. Rod Fontecilla Vice President, Application Services, Chief Data Scientist 2015 Analyst and Advisor Summit Advanced Data Analytics Dr. Rod Fontecilla Vice President, Application Services, Chief Data Scientist Agenda Key Facts Offerings and Capabilities Case Studies When to Engage

More information

End to End Solution to Accelerate Data Warehouse Optimization. Franco Flore Alliance Sales Director - APJ

End to End Solution to Accelerate Data Warehouse Optimization. Franco Flore Alliance Sales Director - APJ End to End Solution to Accelerate Data Warehouse Optimization Franco Flore Alliance Sales Director - APJ Big Data Is Driving Key Business Initiatives Increase profitability, innovation, customer satisfaction,

More information

Reference Architecture, Requirements, Gaps, Roles

Reference Architecture, Requirements, Gaps, Roles Reference Architecture, Requirements, Gaps, Roles The contents of this document are an excerpt from the brainstorming document M0014. The purpose is to show how a detailed Big Data Reference Architecture

More information

Machina Research. Where is the value in IoT? IoT data and analytics may have an answer. Emil Berthelsen, Principal Analyst April 28, 2016

Machina Research. Where is the value in IoT? IoT data and analytics may have an answer. Emil Berthelsen, Principal Analyst April 28, 2016 Machina Research Where is the value in IoT? IoT data and analytics may have an answer Emil Berthelsen, Principal Analyst April 28, 2016 About Machina Research Machina Research is the world s leading provider

More information

Big Data Defined Introducing DataStack 3.0

Big Data Defined Introducing DataStack 3.0 Big Data Big Data Defined Introducing DataStack 3.0 Inside: Executive Summary... 1 Introduction... 2 Emergence of DataStack 3.0... 3 DataStack 1.0 to 2.0... 4 DataStack 2.0 Refined for Large Data & Analytics...

More information

Enhance Collaboration and Data Sharing for Faster Decisions and Improved Mission Outcome

Enhance Collaboration and Data Sharing for Faster Decisions and Improved Mission Outcome Enhance Collaboration and Data Sharing for Faster Decisions and Improved Mission Outcome Richard Breakiron Senior Director, Cyber Solutions Rbreakiron@vion.com Office: 571-353-6127 / Cell: 803-443-8002

More information

DATAMEER WHITE PAPER. Beyond BI. Big Data Analytic Use Cases

DATAMEER WHITE PAPER. Beyond BI. Big Data Analytic Use Cases DATAMEER WHITE PAPER Beyond BI Big Data Analytic Use Cases This white paper discusses the types and characteristics of big data analytics use cases, how they differ from traditional business intelligence

More information

Collaborations between Official Statistics and Academia in the Era of Big Data

Collaborations between Official Statistics and Academia in the Era of Big Data Collaborations between Official Statistics and Academia in the Era of Big Data World Statistics Day October 20-21, 2015 Budapest Vijay Nair University of Michigan Past-President of ISI vnn@umich.edu What

More information

Architecting for Big Data Analytics and Beyond: A New Framework for Business Intelligence and Data Warehousing

Architecting for Big Data Analytics and Beyond: A New Framework for Business Intelligence and Data Warehousing Architecting for Big Data Analytics and Beyond: A New Framework for Business Intelligence and Data Warehousing Wayne W. Eckerson Director of Research, TechTarget Founder, BI Leadership Forum Business Analytics

More information

BIG DATA AND THE ENTERPRISE DATA WAREHOUSE WORKSHOP

BIG DATA AND THE ENTERPRISE DATA WAREHOUSE WORKSHOP BIG DATA AND THE ENTERPRISE DATA WAREHOUSE WORKSHOP Business Analytics for All Amsterdam - 2015 Value of Big Data is Being Recognized Executives beginning to see the path from data insights to revenue

More information

USE CASES BROADBAND EXPERIENCE EVERYWHERE, ANYTIME SMART VEHICLES, TRANSPORT & INFRASTRUCTURE MEDIA EVERYWHERE CRITICAL CONTROL OF REMOTE DEVICES

USE CASES BROADBAND EXPERIENCE EVERYWHERE, ANYTIME SMART VEHICLES, TRANSPORT & INFRASTRUCTURE MEDIA EVERYWHERE CRITICAL CONTROL OF REMOTE DEVICES 5g Use Cases BROADBAND EXPERIENCE EVERYWHERE, ANYTIME 5g USE CASES SMART VEHICLES, TRANSPORT & INFRASTRUCTURE MEDIA EVERYWHERE CRITICAL CONTROL OF REMOTE DEVICES INTERACTION HUMAN-IOT Ericsson Internal

More information

Innovative Security for an Accelerating World New Approaches for Chief Security Officers

Innovative Security for an Accelerating World New Approaches for Chief Security Officers Information Systems Security Association Innovative Security for an Accelerating World New Approaches for Chief Security Officers John N. Stewart Senior Vice President Chief Security and Trust Officer

More information

Industry Impact of Big Data in the Cloud: An IBM Perspective

Industry Impact of Big Data in the Cloud: An IBM Perspective Industry Impact of Big Data in the Cloud: An IBM Perspective Inhi Cho Suh IBM Software Group, Information Management Vice President, Product Management and Strategy email: inhicho@us.ibm.com twitter: @inhicho

More information

DATA MANAGEMENT FOR THE INTERNET OF THINGS

DATA MANAGEMENT FOR THE INTERNET OF THINGS DATA MANAGEMENT FOR THE INTERNET OF THINGS February, 2015 Peter Krensky, Research Analyst, Analytics & Business Intelligence Report Highlights p2 p4 p6 p7 Data challenges Managing data at the edge Time

More information

Industry 4.0 and Big Data

Industry 4.0 and Big Data Industry 4.0 and Big Data Marek Obitko, mobitko@ra.rockwell.com Senior Research Engineer 03/25/2015 PUBLIC PUBLIC - 5058-CO900H 2 Background Joint work with Czech Institute of Informatics, Robotics and

More information

SURVEY REPORT DATA SCIENCE SOCIETY 2014

SURVEY REPORT DATA SCIENCE SOCIETY 2014 SURVEY REPORT DATA SCIENCE SOCIETY 2014 TABLE OF CONTENTS Contents About the Initiative 1 Report Summary 2 Participants Info 3 Participants Expertise 6 Suggested Discussion Topics 7 Selected Responses

More information

Survey of Big Data Architecture and Framework from the Industry

Survey of Big Data Architecture and Framework from the Industry Survey of Big Data Architecture and Framework from the Industry NIST Big Data Public Working Group Sanjay Mishra May13, 2014 3/19/2014 NIST Big Data Public Working Group 1 NIST BD PWG Survey of Big Data

More information

The Canadian Realities of Big Data and Business Analytics. Utsav Arora February 12, 2014

The Canadian Realities of Big Data and Business Analytics. Utsav Arora February 12, 2014 The Canadian Realities of Big Data and Business Analytics Utsav Arora February 12, 2014 Things to think about for today How Important is Big Data for me? Why do I need to implement Big Data and Analytics

More information

Demystifying Big Data Government Agencies & The Big Data Phenomenon

Demystifying Big Data Government Agencies & The Big Data Phenomenon Demystifying Big Data Government Agencies & The Big Data Phenomenon Today s Discussion If you only remember four things 1 Intensifying business challenges coupled with an explosion in data have pushed

More information

Professional Organization Checklist for the Computer Science Curriculum Updates. Association of Computing Machinery Computing Curricula 2008

Professional Organization Checklist for the Computer Science Curriculum Updates. Association of Computing Machinery Computing Curricula 2008 Professional Organization Checklist for the Computer Science Curriculum Updates Association of Computing Machinery Computing Curricula 2008 The curriculum guidelines can be found in Appendix C of the report

More information

Integrating a Big Data Platform into Government:

Integrating a Big Data Platform into Government: Integrating a Big Data Platform into Government: Drive Better Decisions for Policy and Program Outcomes John Haddad, Senior Director Product Marketing, Informatica Digital Government Institute s Government

More information

The 4 Pillars of Technosoft s Big Data Practice

The 4 Pillars of Technosoft s Big Data Practice beyond possible Big Use End-user applications Big Analytics Visualisation tools Big Analytical tools Big management systems The 4 Pillars of Technosoft s Big Practice Overview Businesses have long managed

More information

Research trends relevant to data warehousing and OLAP include [Cuzzocrea et al.]: Combining the benefits of RDBMS and NoSQL database systems

Research trends relevant to data warehousing and OLAP include [Cuzzocrea et al.]: Combining the benefits of RDBMS and NoSQL database systems DATA WAREHOUSING RESEARCH TRENDS Research trends relevant to data warehousing and OLAP include [Cuzzocrea et al.]: Data source heterogeneity and incongruence Filtering out uncorrelated data Strongly unstructured

More information

BIG DATA IN THE CLOUD : CHALLENGES AND OPPORTUNITIES MARY- JANE SULE & PROF. MAOZHEN LI BRUNEL UNIVERSITY, LONDON

BIG DATA IN THE CLOUD : CHALLENGES AND OPPORTUNITIES MARY- JANE SULE & PROF. MAOZHEN LI BRUNEL UNIVERSITY, LONDON BIG DATA IN THE CLOUD : CHALLENGES AND OPPORTUNITIES MARY- JANE SULE & PROF. MAOZHEN LI BRUNEL UNIVERSITY, LONDON Overview * Introduction * Multiple faces of Big Data * Challenges of Big Data * Cloud Computing

More information

Technology Megatrends & The Role for Geospatial

Technology Megatrends & The Role for Geospatial Technology Megatrends & The Role for Geospatial Candace Kinser Chief Executive Officer New Zealand Technology Industry Association New Zealand ESRI User Conference 21 August 2013 Auckland What does the

More information

Big Data Analytics. with EMC Greenplum and Hadoop. Big Data Analytics. Ofir Manor Pre Sales Technical Architect EMC Greenplum

Big Data Analytics. with EMC Greenplum and Hadoop. Big Data Analytics. Ofir Manor Pre Sales Technical Architect EMC Greenplum Big Data Analytics with EMC Greenplum and Hadoop Big Data Analytics with EMC Greenplum and Hadoop Ofir Manor Pre Sales Technical Architect EMC Greenplum 1 Big Data and the Data Warehouse Potential All

More information

The Future of Data Management

The Future of Data Management The Future of Data Management with Hadoop and the Enterprise Data Hub Amr Awadallah (@awadallah) Cofounder and CTO Cloudera Snapshot Founded 2008, by former employees of Employees Today ~ 800 World Class

More information

White Paper. How Streaming Data Analytics Enables Real-Time Decisions

White Paper. How Streaming Data Analytics Enables Real-Time Decisions White Paper How Streaming Data Analytics Enables Real-Time Decisions Contents Introduction... 1 What Is Streaming Analytics?... 1 How Does SAS Event Stream Processing Work?... 2 Overview...2 Event Stream

More information

CS144R/244R Network Design Project on Software Defined Networking for Computing

CS144R/244R Network Design Project on Software Defined Networking for Computing CS144R/244R Network Design Project on Software Defined Networking for Computing (introduction and course overview) 9/2/2015 Instructor: Professor HT Kung Harvard Paulson School of Engineering and Applied

More information

BIG DATA What it is and how to use?

BIG DATA What it is and how to use? BIG DATA What it is and how to use? Lauri Ilison, PhD Data Scientist 21.11.2014 Big Data definition? There is no clear definition for BIG DATA BIG DATA is more of a concept than precise term 1 21.11.14

More information

BIG DATA & DATA SCIENCE

BIG DATA & DATA SCIENCE BIG DATA & DATA SCIENCE ACADEMY PROGRAMS IN-COMPANY TRAINING PORTFOLIO 2 TRAINING PORTFOLIO 2016 Synergic Academy Solutions BIG DATA FOR LEADING BUSINESS Big data promises a significant shift in the way

More information

UNIVERSITY OF INFINITE AMBITIONS. MASTER OF SCIENCE COMPUTER SCIENCE DATA SCIENCE AND SMART SERVICES

UNIVERSITY OF INFINITE AMBITIONS. MASTER OF SCIENCE COMPUTER SCIENCE DATA SCIENCE AND SMART SERVICES UNIVERSITY OF INFINITE AMBITIONS. MASTER OF SCIENCE COMPUTER SCIENCE DATA SCIENCE AND SMART SERVICES MASTER S PROGRAMME COMPUTER SCIENCE - DATA SCIENCE AND SMART SERVICES (DS3) This is a specialization

More information

How to Leverage Big Data in the Cloud to Gain Competitive Advantage

How to Leverage Big Data in the Cloud to Gain Competitive Advantage How to Leverage Big Data in the Cloud to Gain Competitive Advantage James Kobielus, IBM Big Data Evangelist Editor-in-Chief, IBM Data Magazine Senior Program Director, Product Marketing, Big Data Analytics

More information

Big Data, Cloud Computing, Spatial Databases Steven Hagan Vice President Server Technologies

Big Data, Cloud Computing, Spatial Databases Steven Hagan Vice President Server Technologies Big Data, Cloud Computing, Spatial Databases Steven Hagan Vice President Server Technologies Big Data: Global Digital Data Growth Growing leaps and bounds by 40+% Year over Year! 2009 =.8 Zetabytes =.08

More information

Big Analytics: A Next Generation Roadmap

Big Analytics: A Next Generation Roadmap Big Analytics: A Next Generation Roadmap Cloud Developers Summit & Expo: October 1, 2014 Neil Fox, CTO: SoftServe, Inc. 2014 SoftServe, Inc. Remember Life Before The Web? 1994 Even Revolutions Take Time

More information

Introducing Oracle Exalytics In-Memory Machine

Introducing Oracle Exalytics In-Memory Machine Introducing Oracle Exalytics In-Memory Machine Jon Ainsworth Director of Business Development Oracle EMEA Business Analytics 1 Copyright 2011, Oracle and/or its affiliates. All rights Agenda Topics Oracle

More information

Deploying Big Data to the Cloud: Roadmap for Success

Deploying Big Data to the Cloud: Roadmap for Success Deploying Big Data to the Cloud: Roadmap for Success James Kobielus Chair, CSCC Big Data in the Cloud Working Group IBM Big Data Evangelist. IBM Data Magazine, Editor-in- Chief. IBM Senior Program Director,

More information

IBM AND NEXT GENERATION ARCHITECTURE FOR BIG DATA & ANALYTICS!

IBM AND NEXT GENERATION ARCHITECTURE FOR BIG DATA & ANALYTICS! The Bloor Group IBM AND NEXT GENERATION ARCHITECTURE FOR BIG DATA & ANALYTICS VENDOR PROFILE The IBM Big Data Landscape IBM can legitimately claim to have been involved in Big Data and to have a much broader

More information

International Journal of Advanced Engineering Research and Applications (IJAERA) ISSN: 2454-2377 Vol. 1, Issue 6, October 2015. Big Data and Hadoop

International Journal of Advanced Engineering Research and Applications (IJAERA) ISSN: 2454-2377 Vol. 1, Issue 6, October 2015. Big Data and Hadoop ISSN: 2454-2377, October 2015 Big Data and Hadoop Simmi Bagga 1 Satinder Kaur 2 1 Assistant Professor, Sant Hira Dass Kanya MahaVidyalaya, Kala Sanghian, Distt Kpt. INDIA E-mail: simmibagga12@gmail.com

More information

Enabling Manufacturing Transformation in a Connected World. John Shewchuk Technical Fellow DX

Enabling Manufacturing Transformation in a Connected World. John Shewchuk Technical Fellow DX Enabling Manufacturing Transformation in a Connected World John Shewchuk Technical Fellow DX Internet of Things What is the Internet of Things? The network of physical objects that contain embedded technology

More information

Massive Cloud Auditing using Data Mining on Hadoop

Massive Cloud Auditing using Data Mining on Hadoop Massive Cloud Auditing using Data Mining on Hadoop Prof. Sachin Shetty CyberBAT Team, AFRL/RIGD AFRL VFRP Tennessee State University Outline Massive Cloud Auditing Traffic Characterization Distributed

More information

III Big Data Technologies

III Big Data Technologies III Big Data Technologies Today, new technologies make it possible to realize value from Big Data. Big data technologies can replace highly customized, expensive legacy systems with a standard solution

More information

Addressing government challenges with big data analytics

Addressing government challenges with big data analytics IBM Software White Paper Government Addressing government challenges with big data analytics 2 Addressing government challenges with big data analytics Contents 2 Introduction 4 How big data analytics

More information

Danny Wang, Ph.D. Vice President of Business Strategy and Risk Management Republic Bank

Danny Wang, Ph.D. Vice President of Business Strategy and Risk Management Republic Bank Danny Wang, Ph.D. Vice President of Business Strategy and Risk Management Republic Bank Agenda» Overview» What is Big Data?» Accelerates advances in computer & technologies» Revolutionizes data measurement»

More information

Life Insurance & Big Data Analytics: Enterprise Architecture

Life Insurance & Big Data Analytics: Enterprise Architecture Life Insurance & Big Data Analytics: Enterprise Architecture Author: Sudhir Patavardhan Vice President Engineering Feb 2013 Saxon Global Inc. 1320 Greenway Drive, Irving, TX 75038 Contents Contents...1

More information

I D C A N A L Y S T C O N N E C T I O N. C o g n i t i ve C o m m e r c e i n B2B M a rketing a n d S a l e s

I D C A N A L Y S T C O N N E C T I O N. C o g n i t i ve C o m m e r c e i n B2B M a rketing a n d S a l e s I D C A N A L Y S T C O N N E C T I O N Dave Schubmehl Research Director, Cognitive Systems and Content Analytics Greg Girard Program Director, Omni-Channel Retail Analytics Strategies C o g n i t i ve

More information

Busin i ess I n I t n e t ll l i l g i e g nce c T r T e r nds For 2013

Busin i ess I n I t n e t ll l i l g i e g nce c T r T e r nds For 2013 Business Intelligence Trends For 2013 10 Trends The last few years the change in Business Intelligence seems to accelerate under the pressure of increased business demand and technology innovations. Here

More information

Ganzheitliches Datenmanagement

Ganzheitliches Datenmanagement Ganzheitliches Datenmanagement für Hadoop Michael Kohs, Senior Sales Consultant @mikchaos The Problem with Big Data Projects in 2016 Relational, Mainframe Documents and Emails Data Modeler Data Scientist

More information

Modern Data Warehouse

Modern Data Warehouse 1 Modern Data Warehouse Are you ready for Big Data? Does your DWH / BI roadmap contain all the necessary components? IDG: Big data technologies describe a new generation of technologies and architectures,

More information

HOW TO DO A SMART DATA PROJECT

HOW TO DO A SMART DATA PROJECT April 2014 Smart Data Strategies HOW TO DO A SMART DATA PROJECT Guideline www.altiliagroup.com Summary ALTILIA s approach to Smart Data PROJECTS 3 1. BUSINESS USE CASE DEFINITION 4 2. PROJECT PLANNING

More information

Alexander Nikov. 5. Database Systems and Managing Data Resources. Learning Objectives. RR Donnelley Tries to Master Its Data

Alexander Nikov. 5. Database Systems and Managing Data Resources. Learning Objectives. RR Donnelley Tries to Master Its Data INFO 1500 Introduction to IT Fundamentals 5. Database Systems and Managing Data Resources Learning Objectives 1. Describe how the problems of managing data resources in a traditional file environment are

More information

Big Data (Adv. Analytics) in 15 Mins. Peter LePine Managing Director Sales Support IM & BI Practice

Big Data (Adv. Analytics) in 15 Mins. Peter LePine Managing Director Sales Support IM & BI Practice Big Data (Adv. Analytics) in 15 Mins. Peter LePine Managing Director Sales Support IM & BI Practice Agenda Big Data in 15 Mins. Goal: Provide a basic understanding of; What is Big Data; Why it s important

More information

CONNECTING DATA WITH BUSINESS

CONNECTING DATA WITH BUSINESS CONNECTING DATA WITH BUSINESS Big Data and Data Science consulting Business Value through Data Knowledge Synergic Partners is a specialized Big Data, Data Science and Data Engineering consultancy firm

More information

Advanced Big Data Analytics with R and Hadoop

Advanced Big Data Analytics with R and Hadoop REVOLUTION ANALYTICS WHITE PAPER Advanced Big Data Analytics with R and Hadoop 'Big Data' Analytics as a Competitive Advantage Big Analytics delivers competitive advantage in two ways compared to the traditional

More information

Safe Harbor Statement

Safe Harbor Statement Defining a Roadmap to Big Data Success Robert Stackowiak, Oracle Vice President, Big Data 17 November 2015 Safe Harbor Statement The following is intended to outline our general product direction. It is

More information

Where is... How do I get to...

Where is... How do I get to... Big Data, Fast Data, Spatial Data Making Sense of Location Data in a Smart City Hans Viehmann Product Manager EMEA ORACLE Corporation August 19, 2015 Copyright 2014, Oracle and/or its affiliates. All rights

More information

The Lab and The Factory

The Lab and The Factory The Lab and The Factory Architecting for Big Data Management April Reeve DAMA Wisconsin March 11 2014 1 A good speech should be like a woman's skirt: long enough to cover the subject and short enough to

More information

Amplify Serviceability and Productivity by integrating machine /sensor data with Data Science

Amplify Serviceability and Productivity by integrating machine /sensor data with Data Science Data Science & Big Data Practice INSIGHTS ANALYTICS INNOVATIONS Manufacturing IoT Amplify Serviceability and Productivity by integrating machine /sensor data with Data Science What is Internet of Things

More information

The Data Lifecycle: Managing Data through Business. Ewan Willars Friday 27 February

The Data Lifecycle: Managing Data through Business. Ewan Willars Friday 27 February The Lifecycle: Managing through Business Ewan Willars Friday 27 February ACCA s unrivalled global network 4,000 members & students in Canada The changing role of finance and the CFO Traditional control

More information

Big Data Analytics Platform @ Nokia

Big Data Analytics Platform @ Nokia Big Data Analytics Platform @ Nokia 1 Selecting the Right Tool for the Right Workload Yekesa Kosuru Nokia Location & Commerce Strata + Hadoop World NY - Oct 25, 2012 Agenda Big Data Analytics Platform

More information

Top 5 Analytics Applications in Financial Services

Top 5 Analytics Applications in Financial Services Top 5 Analytics Applications in Financial Services Learn how you can boost your bottom line, manage risk, and take action on your insights with the world s most comprehensive analytics platform. 5 game-changing

More information

Big Data Are You Ready? Jorge Plascencia Solution Architect Manager

Big Data Are You Ready? Jorge Plascencia Solution Architect Manager Big Data Are You Ready? Jorge Plascencia Solution Architect Manager Big Data: The Datafication Of Everything Thoughts Devices Processes Thoughts Things Processes Run the Business Organize data to do something

More information

Big Data, Physics, and the Industrial Internet! How Modeling & Analytics are Making the World Work Better."

Big Data, Physics, and the Industrial Internet! How Modeling & Analytics are Making the World Work Better. Big Data, Physics, and the Industrial Internet! How Modeling & Analytics are Making the World Work Better." Matt Denesuk! Chief Data Science Officer! GE Software! October 2014! Imagination at work. Contact:

More information

Eric Ledu, The Createch Group, a BELL company

Eric Ledu, The Createch Group, a BELL company Eric Ledu, The Createch Group, a BELL company Intelligence Analytics maturity Past Present Future Predictive Modeling Optimization What is the best that could happen? Raw Data Cleaned Data Standard Reports

More information

www.pwc.de Mobile Business 1 How Internet of Things enables innovative business models 27 th of May 2015

www.pwc.de Mobile Business 1 How Internet of Things enables innovative business models 27 th of May 2015 www.pwc.de Mobile Business 1 How Internet of Things enables innovative business models 27 th of Introduction 2 Today s speaker Digital Transformation Digital Business Model Michael Pachmajer Director Friedrich-Ebert-Anlage

More information