Taming the Internet of Things: The Lord of the Things

Size: px
Start display at page:

Download "Taming the Internet of Things: The Lord of the Things"

Transcription

1 Taming the Internet of Things: The Lord of the Things Kirk School of Physics, Astronomy, & Computational Sciences College of Science, George Mason University, Fairfax, VA

2 Taming the Internet of Things: The Lord of the Things Kirk moving next week from George Mason University to Booz-Allen Hamilton Strategic Innovations Group

3 1) Fellowship of Things Big Data = Everything is Quantified and Tracked Big Data collected everywhere Many diverse sources (combined = the 360 o view) 2) Twin Towers Deep Data = High-Volume Data Wide Data = High-Variety (Complex) Data 3) Return of the Things The Lord of the Things the emerging IoT = Internet of Things Fast Data = time series from IoT, IoE = Everything is Measured and Tracked (temporally, and spatially) Everything is a sensor

4 Data Science = 4 Types of Discovery 1) Correlation (Predictive Power!) Discovery Finding patterns and dependencies, which reveal new governing principles or behavioral patterns (the DNA ) 2) Novelty (Surprise!) Discovery Finding new, rare, one-in-a-[million / billion / trillion] objects and events 3) Class Discovery Finding new classes of objects and behaviors Learning the rules that constrain class boundaries 4) Association Discovery Learning from Data in the IoT Finding unusual (improbable) co-occurring associations

5 Goal of Data Science: Take Data to Information to Knowledge to Insights (and Action!) ü From Sensors (Measurement & Data Collection) ü to Sentinels (Monitoring & Alerts) ü to Sense-making (Data Science) ü to Cents-making (ROI) Productizing / Monetizing your Big Data 5

6 Creating Value from Big Data in the IoT : The 3 D2D s o Knowledge Discovery Data-to-Discovery (D2D) o Data-driven Decision Support Data-to-Decisions (D2D) o Big ROI (Return On Innovation) Data-to-Dollars (D2D) or Data-to-Dividends 6

7 Creating Value from Big Data in the IoT : The 3 D2D s ü Knowledge Discovery LSST (discovery) o Data-driven Decision Support Mars Rovers (decisions) o Big ROI (Return On Innovation) Innovative Applications of Big Data everywhere 7

8 Astronomy Big Data Example The LSST (Large Synoptic Survey Telescope) 8

9 LSST = Large Synoptic Survey Telescope h"p://www.lsst.org/ (mirror funded by private donors) 8.4-meter diameter primary mirror = 10 square degrees! Hello! 9

10 LSST = Large Synoptic Survey Telescope h"p://www.lsst.org/ (mirror funded by private donors) 8.4-meter diameter primary mirror = 10 square degrees! Construction began August 2014 (funded by NSF and DOE) Hello! 10

11 LSST = Large Synoptic Survey Telescope h"p://www.lsst.org/ (mirror funded by private donors) 8.4-meter diameter primary mirror = 10 square degrees! Petabyte image archive Petabyte database catalog Hello! 11

12 LSST Key Science Drivers: Mapping the Dynamic Universe Complete inventory of the Solar System (Near- Earth Objects; killer asteroids???) Nature of Dark Energy (Cosmology; Supernovae at edge of the known Universe) Op.cal transients (10 million daily event no.fica.ons sent within 60 seconds) Digital Milky Way (Dark MaYer; Loca.ons and veloci.es of 20 billion stars!) LSST in.me and space: When? ~ Where? Cerro Pachon, Chile Architect s design of LSST Observatory 12

13 LSST Summary 3-Gigapixel camera One 6-Gigabyte image every 20 seconds 30 Terabytes every night for 10 years Repeat images of the entire night sky every 3 nights: Celestial Cinematography 100-Petabyte final image data archive anticipated all data are public!!! 20-Petabyte final database catalog anticipated Real-Time Event Mining: ~10 million events per night, every night, for 10 years! Follow-up observations required to classify these Which ones should we follow up? TRIAGE! Decisions! Decisions! ( = D2D!) 13

14 Creating Value from Big Data in the IoT : The 3 D2D s o Knowledge Discovery LSST (discovery) ü Data-driven Decision Support Mars Rovers (decisions) o Big ROI (Return On Innovation) Innovative Applications of Big Data everywhere 14

15 The Mars Rover Metaphor 15

16 Mars Rover: intelligent data-gatherer, mobile data mining agent, and autonomous decision-support system Rove around the surface of Mars and take samples of rocks (experimental data type: mass spectroscopy = data histogram = feature vector) Intelligent Data Operations in Action: Classification (assign rocks to known classes) Supervised Learning (search for rocks with known compositions) Unsupervised Learning (discover what types of rocks are present, without preconceived biases) Clustering (find the set of unique classes of rocks) Association Mining (find unusual associations) Deviation/Outlier Detection (one-of-kind; interesting?) On-board Intelligent Data Understanding & Decision Support Systems (Fuzzy Logic & Decision Trees & Cased-Based Reasoning ) = = Science Goal Monitoring : stay here and do more ; or else follow trend to most interesting location send results to Earth immediately ; or send results later 16

17 Smart Sensors & Sentinels for Data-Driven Sense-Making and Decision Support From Sensors to to Sense New knowledge and insights are acquired by mining actionable data from all digital inputs (Sensors!) Sensors acts autonomously, without human intervention, in Deep Space environment, applying machine learning rules for targeted object recognition and classification. (Sentinels!) Smart Sensors (powered by Machine Learning-enabled sentinels) deliver actionable intelligence (Sense!)

18 Creating Value from Big Data in the IoT : The 3 D2D s o Knowledge Discovery LSST (discovery) o Data-driven Decision Support Mars Rovers (decisions) ü Big ROI (Return On Innovation)!!! Innovative Applications of sense-making from IoT sensors and sentinels everywhere 18

19 Enter... Advanced Analy.cs Automa.on! Learning from Data (Data Science): Outlier / Anomaly / Novelty / Surprise detection Clustering (= New Class discovery, Segmentation) Correlation & Association discovery Classification, Diagnosis, Prediction for D2D in the IoT: Data-to-Discoveries Data-to-Decisions Data-to-Dividends (big ROI = Return on Innovation) 19

20 The BIG Big Data Challenge in the IoT General example of streaming data analytics: v Real-Time Event Mining for Actionable Intelligence: q Identifying, characterizing, & responding to millions of events in real-time streaming data q Deciding which events (out of millions) need investigation and/or response (triage!) Web Analytics example: v Web Behavior Modeling and Automated System Response (from online interactions & web browse patterns, behavioral analytics, user segmentation, data-driven discovery, ) Many other examples: v Health alerts (from EHRs and national health systems) v Tsunami alerts (from geo sensors everywhere) v Cybersecurity alerts (from network logs) v Social event alerts or early warnings (from social media) v Preventive Fraud alerts (from financial applications) v Predictive Maintenance alerts (from machine / engine sensors) v Infrastructure Monitoring alerts (from ubiquitous sensors) Risk Mi.ga.on

21 The MIPS Architecture Design for Dynamic Data- Driven Systems (DDDAS) hyp://dddas.org MIPS = Measurement Inference Prediction Steering This applies to any Network of Sensors: Web user interactions & actions (web analytics data), Cyber network usage logs, Social network sentiment, Machine logs (of any kind), Manufacturing sensors, Health & Epidemic monitoring systems, Financial transactions, National Security, Utilities and Energy, Remote Sensing, Tsunami warnings, Weather/Climate events, Astronomical sky events, Machine Learning enables the IP part of MIPS: Autonomous (or semi-autonomous) Classification Intelligent Data Understanding Rule-based Model-based Neural Networks Markov Models Bayes Inference Engines Alert & Response systems: LSST 10million events Mars Rover anywhere Automation of any datadriven operational system 21

22 The MIPS Architecture Design for Dynamic Data- Driven Systems (DDDAS) hyp://dddas.org MIPS = Measurement Inference Prediction Steering This applies to any Network of Sensors: Web user interactions & actions (web analytics data), Cyber network usage logs, Social network sentiment, Machine logs (of any kind), Manufacturing sensors, Health & Epidemic monitoring systems, Financial transactions, National Security, Utilities and Energy, Remote Sensing, Tsunami warnings, Weather/Climate events, Astronomical sky events, Machine Learning enables the IP part of MIPS: Autonomous (or semi-autonomous) Classification Intelligent Data Understanding Rule-based Model-based Neural Networks Markov Models Bayes Inference Engines Alert & Response systems: LSST 10million events Mars Rover anywhere Automation of any datadriven operational system 22

23 Data Perspec.ve The Lord of the Things 1) Fellowship of Things Big Data = Everything is Quantified and Tracked Big Data collected everywhere Many diverse sources (combined = the 360 o view) 2) Twin Towers Deep Data = High-Volume Data Wide Data = High-Variety (Complex) Data 3) Return of the Things Fast Data = time series from IoT, IoE = Everything is Measured and Tracked (temporally, and spatially) Everything is a sensor

24 Analy.cs Perspec.ve Taming the IoT 1) Fellowship of Things Data Lakes of diverse data (structured, unstructured, ) Diverse data access and processing in Hadoop / HDFS Multi-data query integration with Apache Drill 2) Twin Powers Hadoop : Flexible Batch on semi-/un-/structured data Spark : Interactive (in-memory, real-time) processing and analytics on streaming data from many sensors 3) The Rise of the Things IoT sensors everywhere, explored & exploited via deep and fast analytics processing platforms

Analytics-as-a-Service: From Science to Marketing

Analytics-as-a-Service: From Science to Marketing Analytics-as-a-Service: From Science to Marketing Data Information Knowledge Insights (Discovery & Decisions) Kirk Borne George Mason University, Fairfax, VA www.kirkborne.net @KirkDBorne Big Data: What

More information

Data Literacy For All: Astrophysics and Beyond (Astronomy is evidence-based forensic science, thus it is a data & information science)

Data Literacy For All: Astrophysics and Beyond (Astronomy is evidence-based forensic science, thus it is a data & information science) Data Literacy For All: Astrophysics and Beyond (Astronomy is evidence-based forensic science, thus it is a data & information science) Kirk Borne George Mason University, Fairfax, VA www.kirkborne.net

More information

The Tonnabytes Big Data Challenge: Transforming Science and Education. Kirk Borne George Mason University

The Tonnabytes Big Data Challenge: Transforming Science and Education. Kirk Borne George Mason University The Tonnabytes Big Data Challenge: Transforming Science and Education Kirk Borne George Mason University Ever since we first began to explore our world humans have asked questions and have collected evidence

More information

Learning from Big Data in

Learning from Big Data in Learning from Big Data in Astronomy an overview Kirk Borne George Mason University School of Physics, Astronomy, & Computational Sciences http://spacs.gmu.edu/ From traditional astronomy 2 to Big Data

More information

The Past, Present, and Future of Data Science Education

The Past, Present, and Future of Data Science Education The Past, Present, and Future of Data Science Education Kirk Borne @KirkDBorne http://kirkborne.net George Mason University School of Physics, Astronomy, & Computational Sciences Outline Research and Application

More information

Conquering the Astronomical Data Flood through Machine

Conquering the Astronomical Data Flood through Machine Conquering the Astronomical Data Flood through Machine Learning and Citizen Science Kirk Borne George Mason University School of Physics, Astronomy, & Computational Sciences http://spacs.gmu.edu/ The Problem:

More information

LSST and the Cloud: Astro Collaboration in 2016 Tim Axelrod LSST Data Management Scientist

LSST and the Cloud: Astro Collaboration in 2016 Tim Axelrod LSST Data Management Scientist LSST and the Cloud: Astro Collaboration in 2016 Tim Axelrod LSST Data Management Scientist DERCAP Sydney, Australia, 2009 Overview of Presentation LSST - a large-scale Southern hemisphere optical survey

More information

curation, analyses and interpretation of massive datasets opportunities are varied across disciplines

curation, analyses and interpretation of massive datasets opportunities are varied across disciplines ! Efficiency in scientific discovery through curation, analyses and interpretation of massive datasets! Uptake level and concentration on Big Data opportunities are varied across disciplines The nature

More information

Exploiting Data at Rest and Data in Motion with a Big Data Platform

Exploiting Data at Rest and Data in Motion with a Big Data Platform Exploiting Data at Rest and Data in Motion with a Big Data Platform Sarah Brader, sarah_brader@uk.ibm.com What is Big Data? Where does it come from? 12+ TBs of tweet data every day 30 billion RFID tags

More information

How to use Big Data in Industry 4.0 implementations. LAURI ILISON, PhD Head of Big Data and Machine Learning

How to use Big Data in Industry 4.0 implementations. LAURI ILISON, PhD Head of Big Data and Machine Learning How to use Big Data in Industry 4.0 implementations LAURI ILISON, PhD Head of Big Data and Machine Learning Big Data definition? Big Data is about structured vs unstructured data Big Data is about Volume

More information

Big Data Analytics. An Introduction. Oliver Fuchsberger University of Paderborn 2014

Big Data Analytics. An Introduction. Oliver Fuchsberger University of Paderborn 2014 Big Data Analytics An Introduction Oliver Fuchsberger University of Paderborn 2014 Table of Contents I. Introduction & Motivation What is Big Data Analytics? Why is it so important? II. Techniques & Solutions

More information

AGENDA. What is BIG DATA? What is Hadoop? Why Microsoft? The Microsoft BIG DATA story. Our BIG DATA Roadmap. Hadoop PDW

AGENDA. What is BIG DATA? What is Hadoop? Why Microsoft? The Microsoft BIG DATA story. Our BIG DATA Roadmap. Hadoop PDW AGENDA What is BIG DATA? What is Hadoop? Why Microsoft? The Microsoft BIG DATA story Hadoop PDW Our BIG DATA Roadmap BIG DATA? Volume 59% growth in annual WW information 1.2M Zetabytes (10 21 bytes) this

More information

Introduction to Data Mining

Introduction to Data Mining Introduction to Data Mining 1 Why Data Mining? Explosive Growth of Data Data collection and data availability Automated data collection tools, Internet, smartphones, Major sources of abundant data Business:

More information

LSST Data Management. Tim Axelrod Project Scientist - LSST Data Management. Thursday, 28 Oct 2010

LSST Data Management. Tim Axelrod Project Scientist - LSST Data Management. Thursday, 28 Oct 2010 LSST Data Management Tim Axelrod Project Scientist - LSST Data Management Thursday, 28 Oct 2010 Outline of the Presentation LSST telescope and survey Functions and architecture of the LSST data management

More information

Data-Driven Discovery through e-science Technologies

Data-Driven Discovery through e-science Technologies Data-Driven Discovery through e-science Technologies Kirk D. Borne George Mason University, QSS Group Inc., and NASA-GSFC Kirk.borne@gsfc.nasa.gov Abstract Future space missions and science programs will

More information

The State of Real-Time Big Data Analytics & the Internet of Things (IoT) January 2015 Survey Report

The State of Real-Time Big Data Analytics & the Internet of Things (IoT) January 2015 Survey Report The State of Real-Time Big Data Analytics & the Internet of Things (IoT) January 2015 Survey Report Executive Summary Much of the value from the Internet of Things (IoT) will come from data, making Big

More information

Example application (1) Telecommunication. Lecture 1: Data Mining Overview and Process. Example application (2) Health

Example application (1) Telecommunication. Lecture 1: Data Mining Overview and Process. Example application (2) Health Lecture 1: Data Mining Overview and Process What is data mining? Example applications Definitions Multi disciplinary Techniques Major challenges The data mining process History of data mining Data mining

More information

Are You Ready for Big Data?

Are You Ready for Big Data? Are You Ready for Big Data? Jim Gallo National Director, Business Analytics April 10, 2013 Agenda What is Big Data? How do you leverage Big Data in your company? How do you prepare for a Big Data initiative?

More information

Advanced In-Database Analytics

Advanced In-Database Analytics Advanced In-Database Analytics Tallinn, Sept. 25th, 2012 Mikko-Pekka Bertling, BDM Greenplum EMEA 1 That sounds complicated? 2 Who can tell me how best to solve this 3 What are the main mathematical functions??

More information

The Canadian Realities of Big Data and Business Analytics. Utsav Arora February 12, 2014

The Canadian Realities of Big Data and Business Analytics. Utsav Arora February 12, 2014 The Canadian Realities of Big Data and Business Analytics Utsav Arora February 12, 2014 Things to think about for today How Important is Big Data for me? Why do I need to implement Big Data and Analytics

More information

2015 Analyst and Advisor Summit. Advanced Data Analytics Dr. Rod Fontecilla Vice President, Application Services, Chief Data Scientist

2015 Analyst and Advisor Summit. Advanced Data Analytics Dr. Rod Fontecilla Vice President, Application Services, Chief Data Scientist 2015 Analyst and Advisor Summit Advanced Data Analytics Dr. Rod Fontecilla Vice President, Application Services, Chief Data Scientist Agenda Key Facts Offerings and Capabilities Case Studies When to Engage

More information

Big Data in Subsea Solutions

Big Data in Subsea Solutions Big Data in Subsea Solutions Subsea Valley Conference 2014 Telenor Arena, Fornebu, April 2-3 Roar Fjellheim, Computas AS Computas AS - Brief company profile Norwegian IT consulting company providing services

More information

Impact of Big Data in Oil & Gas Industry. Pranaya Sangvai Reliance Industries Limited 04 Feb 15, DEJ, Mumbai, India.

Impact of Big Data in Oil & Gas Industry. Pranaya Sangvai Reliance Industries Limited 04 Feb 15, DEJ, Mumbai, India. Impact of Big Data in Oil & Gas Industry Pranaya Sangvai Reliance Industries Limited 04 Feb 15, DEJ, Mumbai, India. New Age Information 2.92 billions Internet Users in 2014 Twitter processes 7 terabytes

More information

The Next Big Thing in the Internet of Things: Real-time Big Data Analytics

The Next Big Thing in the Internet of Things: Real-time Big Data Analytics The Next Big Thing in the Internet of Things: Real-time Big Data Analytics Dale Skeen CTO and Co-Founder 2014. VITRIA TECHNOLOGY, INC. All rights reserved. Internet of Things (IoT) Devices > People In

More information

Big Data Use Cases Update

Big Data Use Cases Update Big Data Use Cases Update Sanat Joshi Industry Solutions Manufacturing Industries Business Unit 1 Data Explosion Web & social networks experienced it first Infographic by Go-gulf.com 2 Number Of Connected

More information

What do Big Data & HAVEn mean? Robert Lejnert HP Autonomy

What do Big Data & HAVEn mean? Robert Lejnert HP Autonomy What do Big Data & HAVEn mean? Robert Lejnert HP Autonomy Much higher Volumes. Processed with more Velocity. With much more Variety. Is Big Data so big? Big Data Smart Data Project HAVEn: Adaptive Intelligence

More information

Data Isn't Everything

Data Isn't Everything June 17, 2015 Innovate Forward Data Isn't Everything The Challenges of Big Data, Advanced Analytics, and Advance Computation Devices for Transportation Agencies. Using Data to Support Mission, Administration,

More information

Information Management course

Information Management course Università degli Studi di Milano Master Degree in Computer Science Information Management course Teacher: Alberto Ceselli Lecture 01 : 06/10/2015 Practical informations: Teacher: Alberto Ceselli (alberto.ceselli@unimi.it)

More information

EVERYTHING THAT MATTERS IN ADVANCED ANALYTICS

EVERYTHING THAT MATTERS IN ADVANCED ANALYTICS EVERYTHING THAT MATTERS IN ADVANCED ANALYTICS Marcia Kaufman, Principal Analyst, Hurwitz & Associates Dan Kirsch, Senior Analyst, Hurwitz & Associates Steve Stover, Sr. Director, Product Management, Predixion

More information

Real-Time Big Data Analytics + Internet of Things (IoT) = Value Creation

Real-Time Big Data Analytics + Internet of Things (IoT) = Value Creation Real-Time Big Data Analytics + Internet of Things (IoT) = Value Creation January 2015 Market Insights Report Executive Summary According to a recent customer survey by Vitria, executives across the consumer,

More information

Welcome to Services Discovery Channel. Host: Jean Wong, Head of Service Marketing, Asia Pacific, Japan and Greater China

Welcome to Services Discovery Channel. Host: Jean Wong, Head of Service Marketing, Asia Pacific, Japan and Greater China Welcome to Services Discovery Channel Host: Jean Wong, Head of Service Marketing, Asia Pacific, Japan and Greater China Connecting Analytics to Insight Are You Ready? Keynote Speakers: Mike Riegel, VP,

More information

HDP Hadoop From concept to deployment.

HDP Hadoop From concept to deployment. HDP Hadoop From concept to deployment. Ankur Gupta Senior Solutions Engineer Rackspace: Page 41 27 th Jan 2015 Where are you in your Hadoop Journey? A. Researching our options B. Currently evaluating some

More information

Simplifying Big Data Analytics: Unifying Batch and Stream Processing. John Fanelli,! VP Product! In-Memory Compute Summit! June 30, 2015!!

Simplifying Big Data Analytics: Unifying Batch and Stream Processing. John Fanelli,! VP Product! In-Memory Compute Summit! June 30, 2015!! Simplifying Big Data Analytics: Unifying Batch and Stream Processing John Fanelli,! VP Product! In-Memory Compute Summit! June 30, 2015!! Streaming Analy.cs S S S Scale- up Database Data And Compute Grid

More information

The Challenge of Handling Large Data Sets within your Measurement System

The Challenge of Handling Large Data Sets within your Measurement System The Challenge of Handling Large Data Sets within your Measurement System The Often Overlooked Big Data Aaron Edgcumbe Marketing Engineer Northern Europe, Automated Test National Instruments Introduction

More information

Introduction to Data Mining and Machine Learning Techniques. Iza Moise, Evangelos Pournaras, Dirk Helbing

Introduction to Data Mining and Machine Learning Techniques. Iza Moise, Evangelos Pournaras, Dirk Helbing Introduction to Data Mining and Machine Learning Techniques Iza Moise, Evangelos Pournaras, Dirk Helbing Iza Moise, Evangelos Pournaras, Dirk Helbing 1 Overview Main principles of data mining Definition

More information

Are You Ready for Big Data?

Are You Ready for Big Data? Are You Ready for Big Data? Jim Gallo National Director, Business Analytics February 11, 2013 Agenda What is Big Data? How do you leverage Big Data in your company? How do you prepare for a Big Data initiative?

More information

Big Data and Complex Networks Analytics. Timos Sellis, CSIT Kathy Horadam, MGS

Big Data and Complex Networks Analytics. Timos Sellis, CSIT Kathy Horadam, MGS Big Data and Complex Networks Analytics Timos Sellis, CSIT Kathy Horadam, MGS Big Data What is it? Most commonly accepted definition, by Gartner (the 3 Vs) Big data is high-volume, high-velocity and high-variety

More information

BIG DATA & SOCIAL INNOVATION KENNETH THOMAS, CLIENT MANAGER

BIG DATA & SOCIAL INNOVATION KENNETH THOMAS, CLIENT MANAGER BIG DATA & SOCIAL INNOVATION KENNETH THOMAS, CLIENT MANAGER 1 MAKING THE RIGHT DECISSION AT THE RIGHT PLACE AT THE RIGHT TIME 2 THE DATA MULTIPLIER EFFECT AT WORK BUSINESS DRIVEN HUMAN DRIVEN MACHINE DRIVEN

More information

Data Science & Big Data Practice

Data Science & Big Data Practice INSIGHTS ANALYTICS INNOVATIONS Data Science & Big Data Practice Manufacturing Internet of Things (IoT) Amplify Serviceability and Productivity by integrating machine /sensor data with Data Science What

More information

International Journal of Advanced Engineering Research and Applications (IJAERA) ISSN: 2454-2377 Vol. 1, Issue 6, October 2015. Big Data and Hadoop

International Journal of Advanced Engineering Research and Applications (IJAERA) ISSN: 2454-2377 Vol. 1, Issue 6, October 2015. Big Data and Hadoop ISSN: 2454-2377, October 2015 Big Data and Hadoop Simmi Bagga 1 Satinder Kaur 2 1 Assistant Professor, Sant Hira Dass Kanya MahaVidyalaya, Kala Sanghian, Distt Kpt. INDIA E-mail: simmibagga12@gmail.com

More information

NITRD and Big Data. George O. Strawn NITRD

NITRD and Big Data. George O. Strawn NITRD NITRD and Big Data George O. Strawn NITRD Caveat auditor The opinions expressed in this talk are those of the speaker, not the U.S. government Outline What is Big Data? Who is NITRD? NITRD's Big Data Research

More information

Big Data. Fast Forward. Putting data to productive use

Big Data. Fast Forward. Putting data to productive use Big Data Putting data to productive use Fast Forward What is big data, and why should you care? Get familiar with big data terminology, technologies, and techniques. Getting started with big data to realize

More information

Astrophysics with Terabyte Datasets. Alex Szalay, JHU and Jim Gray, Microsoft Research

Astrophysics with Terabyte Datasets. Alex Szalay, JHU and Jim Gray, Microsoft Research Astrophysics with Terabyte Datasets Alex Szalay, JHU and Jim Gray, Microsoft Research Living in an Exponential World Astronomers have a few hundred TB now 1 pixel (byte) / sq arc second ~ 4TB Multi-spectral,

More information

Data Driven Initiatives in Astronomy and Biology

Data Driven Initiatives in Astronomy and Biology Data Driven Initiatives in Astronomy and Biology An NKN Project By Kaustubh Vaghmare (IUCAA, Pune) 22nd January, 2016 th [4 NKN Workshop, Hyderabad] Partnership with NKN and... IUCAA (Inter-University

More information

Promises and Pitfalls of Big-Data-Predictive Analytics: Best Practices and Trends

Promises and Pitfalls of Big-Data-Predictive Analytics: Best Practices and Trends Promises and Pitfalls of Big-Data-Predictive Analytics: Best Practices and Trends Spring 2015 Thomas Hill, Ph.D. VP Analytic Solutions Dell Statistica Overview and Agenda Dell Software overview Dell in

More information

Data Lake In Action: Real-time, Closed Looped Analytics On Hadoop

Data Lake In Action: Real-time, Closed Looped Analytics On Hadoop 1 Data Lake In Action: Real-time, Closed Looped Analytics On Hadoop 2 Pivotal s Full Approach It s More Than Just Hadoop Pivotal Data Labs 3 Why Pivotal Exists First Movers Solve the Big Data Utility Gap

More information

Integrating a Big Data Platform into Government:

Integrating a Big Data Platform into Government: Integrating a Big Data Platform into Government: Drive Better Decisions for Policy and Program Outcomes John Haddad, Senior Director Product Marketing, Informatica Digital Government Institute s Government

More information

End to End Solution to Accelerate Data Warehouse Optimization. Franco Flore Alliance Sales Director - APJ

End to End Solution to Accelerate Data Warehouse Optimization. Franco Flore Alliance Sales Director - APJ End to End Solution to Accelerate Data Warehouse Optimization Franco Flore Alliance Sales Director - APJ Big Data Is Driving Key Business Initiatives Increase profitability, innovation, customer satisfaction,

More information

Amplify Serviceability and Productivity by integrating machine /sensor data with Data Science

Amplify Serviceability and Productivity by integrating machine /sensor data with Data Science Data Science & Big Data Practice INSIGHTS ANALYTICS INNOVATIONS Manufacturing IoT Amplify Serviceability and Productivity by integrating machine /sensor data with Data Science What is Internet of Things

More information

The Large Synoptic Survey Telescope: Status Update

The Large Synoptic Survey Telescope: Status Update The Large Synoptic Survey Telescope: Status Update Steven M. Kahn LSST Director Mid-Decadal Review Committee December 13, 2015 LSST in a Nutshell The LSST is an integrated survey system designed to conduct

More information

The 4 Pillars of Technosoft s Big Data Practice

The 4 Pillars of Technosoft s Big Data Practice beyond possible Big Use End-user applications Big Analytics Visualisation tools Big Analytical tools Big management systems The 4 Pillars of Technosoft s Big Practice Overview Businesses have long managed

More information

Hadoop s Advantages for! Machine! Learning and. Predictive! Analytics. Webinar will begin shortly. Presented by Hortonworks & Zementis

Hadoop s Advantages for! Machine! Learning and. Predictive! Analytics. Webinar will begin shortly. Presented by Hortonworks & Zementis Webinar will begin shortly Hadoop s Advantages for Machine Learning and Predictive Analytics Presented by Hortonworks & Zementis September 10, 2014 Copyright 2014 Zementis, Inc. All rights reserved. 2

More information

locuz.com Big Data Services

locuz.com Big Data Services locuz.com Big Data Services Big Data At Locuz, we help the enterprise move from being a data-limited to a data-driven one, thereby enabling smarter, faster decisions that result in better business outcome.

More information

Big Data and Your Data Warehouse Philip Russom

Big Data and Your Data Warehouse Philip Russom Big Data and Your Data Warehouse Philip Russom TDWI Research Director for Data Management April 5, 2012 Sponsor Speakers Philip Russom Research Director, Data Management, TDWI Peter Jeffcock Director,

More information

Embedded inside the database. No need for Hadoop or customcode. True real-time analytics done per transaction and in aggregate. On-the-fly linking IP

Embedded inside the database. No need for Hadoop or customcode. True real-time analytics done per transaction and in aggregate. On-the-fly linking IP Operates more like a search engine than a database Scoring and ranking IP allows for fuzzy searching Best-result candidate sets returned Contextual analytics to correctly disambiguate entities Embedded

More information

Mastering Big Data. Steve Hoskin, VP and Chief Architect INFORMATICA MDM. October 2015

Mastering Big Data. Steve Hoskin, VP and Chief Architect INFORMATICA MDM. October 2015 Mastering Big Data Steve Hoskin, VP and Chief Architect INFORMATICA MDM October 2015 Agenda About Big Data MDM and Big Data The Importance of Relationships Big Data Use Cases About Big Data Big Data is

More information

BIG Big Data Public Private Forum

BIG Big Data Public Private Forum DATA STORAGE Martin Strohbach, AGT International (R&D) THE DATA VALUE CHAIN Value Chain Data Acquisition Data Analysis Data Curation Data Storage Data Usage Structured data Unstructured data Event processing

More information

internet of things Patrick Pax Business Development Manager Chris Geary Innovation Manager

internet of things Patrick Pax Business Development Manager Chris Geary Innovation Manager internet of things Patrick Pax Business Development Manager Chris Geary Innovation Manager agenda 1 2 3 4 internet of things moving into main stream uncover new opportunities in your business call for

More information

Hortonworks & SAS. Analytics everywhere. Page 1. Hortonworks Inc. 2011 2014. All Rights Reserved

Hortonworks & SAS. Analytics everywhere. Page 1. Hortonworks Inc. 2011 2014. All Rights Reserved Hortonworks & SAS Analytics everywhere. Page 1 A change in focus. A shift in Advertising From mass branding A shift in Financial Services From Educated Investing A shift in Healthcare From mass treatment

More information

Reimagining Business with SAP HANA Cloud Platform for the Internet of Things

Reimagining Business with SAP HANA Cloud Platform for the Internet of Things SAP Brief SAP HANA SAP HANA Cloud Platform for the Internet of Things Objectives Reimagining Business with SAP HANA Cloud Platform for the Internet of Things Connect, transform, and reimagine Connect,

More information

Big Data Analytics. Lucas Rego Drumond

Big Data Analytics. Lucas Rego Drumond Big Data Analytics Lucas Rego Drumond Information Systems and Machine Learning Lab (ISMLL) Institute of Computer Science University of Hildesheim, Germany Big Data Analytics Big Data Analytics 1 / 36 Outline

More information

The Future of Data Management

The Future of Data Management The Future of Data Management with Hadoop and the Enterprise Data Hub Amr Awadallah (@awadallah) Cofounder and CTO Cloudera Snapshot Founded 2008, by former employees of Employees Today ~ 800 World Class

More information

YOU VS THE SENSORS. Six Requirements for Visualizing the Internet of Things. Dan Potter Chief Marketing Officer, Datawatch Corporation

YOU VS THE SENSORS. Six Requirements for Visualizing the Internet of Things. Dan Potter Chief Marketing Officer, Datawatch Corporation YOU VS THE SENSORS Six Requirements for Visualizing the Internet of Things Dan Potter Chief Marketing Officer, Datawatch Corporation About Datawatch NASDAQ: DWCH Pioneer in real-time visual data discovery

More information

Enhance Collaboration and Data Sharing for Faster Decisions and Improved Mission Outcome

Enhance Collaboration and Data Sharing for Faster Decisions and Improved Mission Outcome Enhance Collaboration and Data Sharing for Faster Decisions and Improved Mission Outcome Richard Breakiron Senior Director, Cyber Solutions Rbreakiron@vion.com Office: 571-353-6127 / Cell: 803-443-8002

More information

Predictive Analytics Techniques: What to Use For Your Big Data. March 26, 2014 Fern Halper, PhD

Predictive Analytics Techniques: What to Use For Your Big Data. March 26, 2014 Fern Halper, PhD Predictive Analytics Techniques: What to Use For Your Big Data March 26, 2014 Fern Halper, PhD Presenter Proven Performance Since 1995 TDWI helps business and IT professionals gain insight about data warehousing,

More information

Danny Wang, Ph.D. Vice President of Business Strategy and Risk Management Republic Bank

Danny Wang, Ph.D. Vice President of Business Strategy and Risk Management Republic Bank Danny Wang, Ph.D. Vice President of Business Strategy and Risk Management Republic Bank Agenda» Overview» What is Big Data?» Accelerates advances in computer & technologies» Revolutionizes data measurement»

More information

5 Keys to Unlocking the Big Data Analytics Puzzle. Anurag Tandon Director, Product Marketing March 26, 2014

5 Keys to Unlocking the Big Data Analytics Puzzle. Anurag Tandon Director, Product Marketing March 26, 2014 5 Keys to Unlocking the Big Data Analytics Puzzle Anurag Tandon Director, Product Marketing March 26, 2014 1 A Little About Us A global footprint. A proven innovator. A leader in enterprise analytics for

More information

Data Mining and Machine Learning in Bioinformatics

Data Mining and Machine Learning in Bioinformatics Data Mining and Machine Learning in Bioinformatics PRINCIPAL METHODS AND SUCCESSFUL APPLICATIONS Ruben Armañanzas http://mason.gmu.edu/~rarmanan Adapted from Iñaki Inza slides http://www.sc.ehu.es/isg

More information

Surfing the Data Tsunami: A New Paradigm for Big Data Processing and Analytics

Surfing the Data Tsunami: A New Paradigm for Big Data Processing and Analytics Surfing the Data Tsunami: A New Paradigm for Big Data Processing and Analytics Dr. Liangxiu Han Future Networks and Distributed Systems Group (FUNDS) School of Computing, Mathematics and Digital Technology,

More information

ANALYTICS IN BIG DATA ERA

ANALYTICS IN BIG DATA ERA ANALYTICS IN BIG DATA ERA ANALYTICS TECHNOLOGY AND ARCHITECTURE TO MANAGE VELOCITY AND VARIETY, DISCOVER RELATIONSHIPS AND CLASSIFY HUGE AMOUNT OF DATA MAURIZIO SALUSTI SAS Copyr i g ht 2012, SAS Ins titut

More information

Doing Multidisciplinary Research in Data Science

Doing Multidisciplinary Research in Data Science Doing Multidisciplinary Research in Data Science Assoc.Prof. Abzetdin ADAMOV CeDAWI - Center for Data Analytics and Web Insights Qafqaz University aadamov@qu.edu.az http://ce.qu.edu.az/~aadamov 16 May

More information

Tutorial: Big Data Algorithms and Applications Under Hadoop KUNPENG ZHANG SIDDHARTHA BHATTACHARYYA

Tutorial: Big Data Algorithms and Applications Under Hadoop KUNPENG ZHANG SIDDHARTHA BHATTACHARYYA Tutorial: Big Data Algorithms and Applications Under Hadoop KUNPENG ZHANG SIDDHARTHA BHATTACHARYYA http://kzhang6.people.uic.edu/tutorial/amcis2014.html August 7, 2014 Schedule I. Introduction to big data

More information

Industry Impact of Big Data in the Cloud: An IBM Perspective

Industry Impact of Big Data in the Cloud: An IBM Perspective Industry Impact of Big Data in the Cloud: An IBM Perspective Inhi Cho Suh IBM Software Group, Information Management Vice President, Product Management and Strategy email: inhicho@us.ibm.com twitter: @inhicho

More information

www.pwc.com/oracle Next presentation starting soon Business Analytics using Big Data to gain competitive advantage

www.pwc.com/oracle Next presentation starting soon Business Analytics using Big Data to gain competitive advantage www.pwc.com/oracle Next presentation starting soon Business Analytics using Big Data to gain competitive advantage If every image made and every word written from the earliest stirring of civilization

More information

Big Data Are You Ready? Jorge Plascencia Solution Architect Manager

Big Data Are You Ready? Jorge Plascencia Solution Architect Manager Big Data Are You Ready? Jorge Plascencia Solution Architect Manager Big Data: The Datafication Of Everything Thoughts Devices Processes Thoughts Things Processes Run the Business Organize data to do something

More information

The Analytics Value Chain Key to Delivering Value in IoT

The Analytics Value Chain Key to Delivering Value in IoT Vitria Operational Intelligence The Value Chain Key to Delivering Value in IoT Dr. Dale Skeen CTO and Co-Founder Internet of Things Value Potential $20 Trillion by 2025 40% 2015 Vitria Technology, Inc.

More information

Gain Contextual Awareness for a Smarter Digital Enterprise with SAP HANA Vora

Gain Contextual Awareness for a Smarter Digital Enterprise with SAP HANA Vora SAP Brief SAP Technology SAP HANA Vora Objectives Gain Contextual Awareness for a Smarter Digital Enterprise with SAP HANA Vora Bridge the divide between enterprise data and Big Data Bridge the divide

More information

Internet of Things. Opportunity Challenges Solutions

Internet of Things. Opportunity Challenges Solutions Internet of Things Opportunity Challenges Solutions Copyright 2014 Boeing. All rights reserved. GPDIS_2015.ppt 1 ANALYZING INTERNET OF THINGS USING BIG DATA ECOSYSTEM Internet of Things matter for... Industrial

More information

A New Era Of Analytic

A New Era Of Analytic Penang egovernment Seminar 2014 A New Era Of Analytic Megat Anuar Idris Head, Project Delivery, Business Analytics & Big Data Agenda Overview of Big Data Case Studies on Big Data Big Data Technology Readiness

More information

Introduction. A. Bellaachia Page: 1

Introduction. A. Bellaachia Page: 1 Introduction 1. Objectives... 3 2. What is Data Mining?... 4 3. Knowledge Discovery Process... 5 4. KD Process Example... 7 5. Typical Data Mining Architecture... 8 6. Database vs. Data Mining... 9 7.

More information

Big Data & Analytics for Semiconductor Manufacturing

Big Data & Analytics for Semiconductor Manufacturing Big Data & Analytics for Semiconductor Manufacturing 半 導 体 生 産 におけるビッグデータ 活 用 Ryuichiro Hattori 服 部 隆 一 郎 Intelligent SCM and MFG solution Leader Global CoC (Center of Competence) Electronics team General

More information

Comprehensive Analytics on the Hortonworks Data Platform

Comprehensive Analytics on the Hortonworks Data Platform Comprehensive Analytics on the Hortonworks Data Platform We do Hadoop. Page 1 Page 2 Back to 2005 Page 3 Vertical Scaling Page 4 Vertical Scaling Page 5 Vertical Scaling Page 6 Horizontal Scaling Page

More information

Kai Wähner. The Next-Generation BPM for a Big Data World: Intelligent Business Process Management Suites (ibpms)

Kai Wähner. The Next-Generation BPM for a Big Data World: Intelligent Business Process Management Suites (ibpms) The Next-Generation BPM for a Big Data World: Intelligent Business Process Management Suites (ibpms) Kai Wähner kontakt@kai-waehner.de @KaiWaehner www.kai-waehner.de Xing / LinkedIn Please connect! Kai

More information

Streamdrill: Analyzing Big Data Streams in Realtime

Streamdrill: Analyzing Big Data Streams in Realtime Streamdrill: Analyzing Big Data Streams in Realtime Mikio L. Braun mikio@streamdrill.com @mikiobraun th 6 Realtime Big Data: Sources Finance Gaming Monitoring Advertisment Sensor Networks Social Media

More information

Managing Big Data with Hadoop & Vertica. A look at integration between the Cloudera distribution for Hadoop and the Vertica Analytic Database

Managing Big Data with Hadoop & Vertica. A look at integration between the Cloudera distribution for Hadoop and the Vertica Analytic Database Managing Big Data with Hadoop & Vertica A look at integration between the Cloudera distribution for Hadoop and the Vertica Analytic Database Copyright Vertica Systems, Inc. October 2009 Cloudera and Vertica

More information

Integrating Hadoop. Into Business Intelligence & Data Warehousing. Philip Russom TDWI Research Director for Data Management, April 9 2013

Integrating Hadoop. Into Business Intelligence & Data Warehousing. Philip Russom TDWI Research Director for Data Management, April 9 2013 Integrating Hadoop Into Business Intelligence & Data Warehousing Philip Russom TDWI Research Director for Data Management, April 9 2013 TDWI would like to thank the following companies for sponsoring the

More information

The future of Big Data A United Hitachi View

The future of Big Data A United Hitachi View The future of Big Data A United Hitachi View Alex van Die Pre-Sales Consultant 1 Oktober 2014 1 Agenda Evolutie van Data en Analytics Internet of Things Hitachi Social Innovation Vision and Solutions 2

More information

Transforming the Telecoms Business using Big Data and Analytics

Transforming the Telecoms Business using Big Data and Analytics Transforming the Telecoms Business using Big Data and Analytics Event: ICT Forum for HR Professionals Venue: Meikles Hotel, Harare, Zimbabwe Date: 19 th 21 st August 2015 AFRALTI 1 Objectives Describe

More information

The New Normal: Get Ready for the Era of Extreme Information Management. John Mancini President, AIIM @jmancini77 DigitalLandfill.

The New Normal: Get Ready for the Era of Extreme Information Management. John Mancini President, AIIM @jmancini77 DigitalLandfill. The New Normal: Get Ready for the Era of Extreme Information Management John Mancini President, AIIM @jmancini77 DigitalLandfill.org Giving Credit Where Credit is Due I didn t make up the term Extreme

More information

Extend your analytic capabilities with SAP Predictive Analysis

Extend your analytic capabilities with SAP Predictive Analysis September 9 11, 2013 Anaheim, California Extend your analytic capabilities with SAP Predictive Analysis Charles Gadalla Learning Points Advanced analytics strategy at SAP Simplifying predictive analytics

More information

Big Data. George O. Strawn NITRD

Big Data. George O. Strawn NITRD Big Data George O. Strawn NITRD Caveat auditor The opinions expressed in this talk are those of the speaker, not the U.S. government Outline What is Big Data? NITRD's Big Data Research Initiative Big Data

More information

PALANTIR CYBER An End-to-End Cyber Intelligence Platform for Analysis & Knowledge Management

PALANTIR CYBER An End-to-End Cyber Intelligence Platform for Analysis & Knowledge Management PALANTIR CYBER An End-to-End Cyber Intelligence Platform for Analysis & Knowledge Management INTRODUCTION Traditional perimeter defense solutions fail against sophisticated adversaries who target their

More information

Big Data: Image & Video Analytics

Big Data: Image & Video Analytics Big Data: Image & Video Analytics How it could support Archiving & Indexing & Searching Dieter Haas, IBM Deutschland GmbH The Big Data Wave 60% of internet traffic is multimedia content (images and videos)

More information

Data Centric Computing Revisited

Data Centric Computing Revisited Piyush Chaudhary Technical Computing Solutions Data Centric Computing Revisited SPXXL/SCICOMP Summer 2013 Bottom line: It is a time of Powerful Information Data volume is on the rise Dimensions of data

More information

The Future of Data Management with Hadoop and the Enterprise Data Hub

The Future of Data Management with Hadoop and the Enterprise Data Hub The Future of Data Management with Hadoop and the Enterprise Data Hub Amr Awadallah Cofounder & CTO, Cloudera, Inc. Twitter: @awadallah 1 2 Cloudera Snapshot Founded 2008, by former employees of Employees

More information

Big data and its transformational effects

Big data and its transformational effects Big data and its transformational effects Professor Fai Cheng Head of Research & Technology September 2015 Working together for a safer world Topics Lloyd s Register Big Data Data driven world Data driven

More information

Unleashing the Potential of your Social Media and CRM Data. Markus Hirsch Sales Manager

Unleashing the Potential of your Social Media and CRM Data. Markus Hirsch Sales Manager Unleashing the Potential of your Social Media and CRM Data Markus Hirsch Sales Manager To get the RIGHT INFORMATION to the RIGHT PLACE, at the RIGHT TIME and put it in the RIGHT CONTEXT to make the WORLD

More information

G-Cloud Big Data Suite Powered by Pivotal. December 2014. G-Cloud. service definitions

G-Cloud Big Data Suite Powered by Pivotal. December 2014. G-Cloud. service definitions G-Cloud Big Data Suite Powered by Pivotal December 2014 G-Cloud service definitions TABLE OF CONTENTS Service Overview... 3 Business Need... 6 Our Approach... 7 Service Management... 7 Vendor Accreditations/Awards...

More information

Big Data Decision Making

Big Data Decision Making Big Data Decision Making Use Cases Daniel J. Power University of Northern Iowa and DSSResources.com, Cedar Falls, Iowa USA Daniel.Power@UNI.edu 1 About me Daniel J. Power Technology generalist Decision

More information

IoT Analytics: Four Key Essentials and Four Target Industries

IoT Analytics: Four Key Essentials and Four Target Industries IoT Analytics: Four Key Essentials and Four Target Industries 1 Introduction Analysts and IT personnel across all industries seek technology to better engage and manage data generated by the Internet of

More information