Industry 4.0 and Big Data

Size: px
Start display at page:

Download "Industry 4.0 and Big Data"

Transcription

1 Industry 4.0 and Big Data Marek Obitko, Senior Research Engineer 03/25/2015 PUBLIC PUBLIC CO900H

2 2 Background Joint work with Czech Institute of Informatics, Robotics and Cybernetics Big Data related topics investigated in RA-DIC laboratory within CIIRC Goal of the effort: Semantic Big Data Historian

3 3 Agenda Overview of related trends Industry 4.0 Big Data Semantics Semantic Big Data Historian Architecture Use Case Outlook Conclusion

4 4 Agenda Overview of related trends Industry 4.0 Big Data Semantics Semantic Big Data Historian Architecture Use Case Outlook Conclusion

5 5 Industry 4.0 Fourth Industrial Revolution Predicted a-priori, not observed ex-post Economic impact predicted to be huge Operational effectiveness, new business models, services and products Clear definition not provided Usually: vision, basic technologies, selected scenarios Design principles Interoperability, virtualization, decentralization, real-time capability, service orientation, and modularity

6 6 Industry 4.0 Components Primary components Cyber-physical systems Fusion of physical and virtual world integration of computation and physical processes Features: unique identification RFID tags, centralized storage and analytics, multiple sensors and actuators, network compatible Example: virtual battery a battery in electric car has its virtual counterpart updated in real time, which allows diagnostics, simulation, prediction etc. for better customer experience Internet of Things Network of physical systems that are uniquely identified and can interact to reach common goals Example: Smart Homes connected devices (temperature sensor, heating, mobile phone) Internet of Services Offering services via Internet so that they can be offered and combined into value-added services by various suppliers Example: forming virtual production technologies and capabilities Smart Factory often mentioned as a key feature of Industry 4.0 Information coming from physical and virtual world used to provide context and assistance for people and machines to execute their tasks in a better way Example: demand driven production, intelligent work piece carriers Other also related components: Smart product, Machine to machine (M2M), Big Data, Cloud

7 7 Agenda Overview of related trends Industry 4.0 Big Data Semantics Semantic Big Data Historian Architecture Use Case Outlook Conclusion

8 8 Big Data Motivation A CPG (consumer packaged goods) company generates 5,000 data samples every 33 milliseconds This corresponds to 70TB per year Can we meaningfully use such amount of data? Big Data dataset that is growing so that it becomes difficult to manage it using existing database management concepts and tools 3Vs Volume, Velocity, Variety

9 9 Big Data Volume data will grow 50 times by 2020 FB 50PB Velocity storing and getting data fraud detection Variety unstructured, 90% of new data videos Applications Online marketing targeting products based on user clickstream (Google, Amazon, Netflix ) Medicine, biology, chemistry data analysis Technologies Map-Reduce framework, introduced by Google Running on cheap machines in parallel in clusters (splitting data) implemented in e.g. Apache Hadoop It s about variety, not volume The Big is not the main problem, focus on heterogeneous data integration new analytic applications based on data that were not tracked so far

10 10 Agenda Overview of related trends Industry 4.0 Big Data Semantics Semantic Big Data Historian Architecture Use Case Outlook Conclusion

11 11 Semantics Linked Data / Semantic Web (machine processable data) Tens of RDF Gtriples on web Resource Description Framework Resources uniquely identified by URI Triples subject property object In fact relations between objects, values of properties Together forming RDF graph(s) Web Ontology Language Ontology specifies the conceptualization In fact description of vocabulary, constraints, attaches meaning to identifiers Designed for internet and web And so also usable for Internet of Things, Internet of Services etc. Inherently distributed approach, integration of data from heterogeneous and unreliable data sources

12 12 Agenda Overview of related trends Industry 4.0 Big Data Semantics Semantic Big Data Historian Architecture Use Case Outlook Conclusion

13 13 Plant Data Processing Traditional Historian Time series data collection, focus on fast scan rate Analyzing data What the ph was at 2:34:56 PM March 15, 2015 Not a problem, single retrieval, unless there is a problem with volume What the ph trend was from 1 to 7 PM of March 15, 2013, plus compare it to previous similar weekdays, holidays, after it rained, when different suppliers were used etc. Not easily possible in historians available today, especially for large scale data Samples of needed data processing Pattern recognition, pattern matching Predictive maintenance Benchmarking of KPIs Clustering similar machines Real time statistics / analytics / reporting

14 14 Semantic Big Data Historian Vision, currently being implemented to verify the technologies Collecting data from sensors Architecture based on OPC UA Sensors semantically described All data processed using Semantic Web languages and technologies allows linking data together Data stored in Hadoop Analyzing data Querying using SPARQL (RDF querying language) More complex queries implemented directly in Map-Reduce framework

15 15 Description of sensors and data Ontology building on top of SSN Semantic Sensor Network Ontology (W3C effort) Ontology describes Sensors Observations, including physical units, time, data quality etc. Data expressed using the ontology Particular observations All data linked together Directly stored as RDF triples

16 16 Agenda Overview of related trends Industry 4.0 Big Data Semantics Semantic Big Data Historian Architecture Use Case Outlook Conclusion

17 17 Case study data from passive house Our goal: evaluate the suitability of proposed technologies, scalability etc. Data focus: indoor air quality Environmental parameters: Temperature, Carbon dioxide concentration, Relative humidity, Air pressure Sample analysis tasks Relaxation time of the house Impact of sunlight on indoor temperature Detection of people inside

18 18 Case study data from passive house Raw data conversion to RDF to be stored to triple store

19 19 Case study data from passive house Sample task detection of people inside Time series processing of CO 2 data Values in sliding window, comparing with threshold Verified the results by comparing with people occupancy list Main result Data not really very big, however, reaching the limits of MATLAB package Map-Reduce implementation in Hadoop (both pre-processing and detection) much faster than in MATLAB The task proved the advantage of Hadoop implementation scalability

20 20 Agenda Overview of related trends Industry 4.0 Big Data Semantics Semantic Big Data Historian Architecture Use Case Outlook Conclusion

21 21 Outlook Semantic Big Data Historian overall goal: Semantic: connect data together Provide semantic description in the endpoints, connect to OPC UA and let the Historian to connect the data appropriately Big Data: be able to work with larger volume of data Historian Using Map-Reduce and similar frameworks to store, retrieve and analyze larger volume of heterogeneous data Focus on time-series data, however be able to also include other types of data E.g., information about suppliers, orders, shifts, various annotations etc. Achieve analytics that was not possible without current technologies Also connect to actions in physical world, not only ad-hoc analysis

22 22 Agenda Overview of related trends Industry 4.0 Big Data Semantics Semantic Big Data Historian Architecture Use Case Outlook Conclusion

23 23 Conclusion Industry 4.0 fusion of physical and virtual world, network of physical systems that interact to reach common goals, integration of services, smart devices, homes, factories, Big Data and Semantics prerequisite for processing large volume of heterogeneous data Semantic Big Data Historian The goal is to provide advanced analytics on plant heterogeneous data, in the scale that was not possible until now Demonstrated the Hadoop scalability Demonstrated Semantic Web suitability for data integration Next steps include advanced data analysis Industry 4.0 both distributed and centralized approaches needed Small scale (M2M) versus large scale (cloud) data processing

24 Thank you! Questions? Contact: PUBLIC PUBLIC CO900H

Standards for Big Data in the Cloud

Standards for Big Data in the Cloud Standards for Big Data in the Cloud International Cloud Symposium 15/10/2013 Carola Carstens (Project Officer) DG CONNECT, Unit G3 Data Value Chain European Commission Outline 1) Data Value Chain Unit

More information

COMP9321 Web Application Engineering

COMP9321 Web Application Engineering COMP9321 Web Application Engineering Semester 2, 2015 Dr. Amin Beheshti Service Oriented Computing Group, CSE, UNSW Australia Week 11 (Part II) http://webapps.cse.unsw.edu.au/webcms2/course/index.php?cid=2411

More information

Big Data and Analytics: Challenges and Opportunities

Big Data and Analytics: Challenges and Opportunities Big Data and Analytics: Challenges and Opportunities Dr. Amin Beheshti Lecturer and Senior Research Associate University of New South Wales, Australia (Service Oriented Computing Group, CSE) Talk: Sharif

More information

Volume 3, Issue 6, June 2015 International Journal of Advance Research in Computer Science and Management Studies

Volume 3, Issue 6, June 2015 International Journal of Advance Research in Computer Science and Management Studies Volume 3, Issue 6, June 2015 International Journal of Advance Research in Computer Science and Management Studies Research Article / Survey Paper / Case Study Available online at: www.ijarcsms.com Image

More information

INTERSEC BENCHMARK. High Performance for Fast Data & Real-Time Analytics Part I: Vs Hadoop

INTERSEC BENCHMARK. High Performance for Fast Data & Real-Time Analytics Part I: Vs Hadoop INTERSEC BENCHMARK High Performance for Fast Data & Real-Time Analytics Part I: Vs Hadoop BENCHMARK VS HADOOP (STAND ALONE OR COMBINED) Intersec solution in a Redhat Openstack NFV framework complements

More information

Analytics in the Cloud. Peter Sirota, GM Elastic MapReduce

Analytics in the Cloud. Peter Sirota, GM Elastic MapReduce Analytics in the Cloud Peter Sirota, GM Elastic MapReduce Data-Driven Decision Making Data is the new raw material for any business on par with capital, people, and labor. What is Big Data? Terabytes of

More information

Secure and Semantic Web of Automation

Secure and Semantic Web of Automation Secure and Semantic Web of Automation Wolfgang Kastner 1, Andreas Fernbach 1, Wolfgang Granzer 2 1 Technische Universität Wien 2 NETxAutomation Software GmbH Automation Systems Group Computer Engineering/Software

More information

Analyzing Big Data with AWS

Analyzing Big Data with AWS Analyzing Big Data with AWS Peter Sirota, General Manager, Amazon Elastic MapReduce @petersirota What is Big Data? Computer generated data Application server logs (web sites, games) Sensor data (weather,

More information

Big Data and Industrial Internet

Big Data and Industrial Internet Big Data and Industrial Internet Keijo Heljanko Department of Computer Science and Helsinki Institute for Information Technology HIIT School of Science, Aalto University keijo.heljanko@aalto.fi 16.6-2015

More information

Big Data Explained. An introduction to Big Data Science.

Big Data Explained. An introduction to Big Data Science. Big Data Explained An introduction to Big Data Science. 1 Presentation Agenda What is Big Data Why learn Big Data Who is it for How to start learning Big Data When to learn it Objective and Benefits of

More information

BIG. Big Data Analysis John Domingue (STI International and The Open University) Big Data Public Private Forum

BIG. Big Data Analysis John Domingue (STI International and The Open University) Big Data Public Private Forum Big Data Analysis John Domingue (STI International and The Open University) Project co-funded by the European Commission within the 7th Framework Program (Grant Agreement No. 257943) 1 The Data landscape

More information

The Rise of Industrial Big Data

The Rise of Industrial Big Data GE Intelligent Platforms The Rise of Industrial Big Data Leveraging large time-series data sets to drive innovation, competitiveness and growth capitalizing on the big data opportunity The Rise of Industrial

More information

BIG Big Data Public Private Forum

BIG Big Data Public Private Forum DATA STORAGE Martin Strohbach, AGT International (R&D) THE DATA VALUE CHAIN Value Chain Data Acquisition Data Analysis Data Curation Data Storage Data Usage Structured data Unstructured data Event processing

More information

Big Data a threat or a chance?

Big Data a threat or a chance? Big Data a threat or a chance? Helwig Hauser University of Bergen, Dept. of Informatics Big Data What is Big Data? well, lots of data, right? we come back to this in a moment. certainly, a buzz-word but

More information

Chapter 7. Using Hadoop Cluster and MapReduce

Chapter 7. Using Hadoop Cluster and MapReduce Chapter 7 Using Hadoop Cluster and MapReduce Modeling and Prototyping of RMS for QoS Oriented Grid Page 152 7. Using Hadoop Cluster and MapReduce for Big Data Problems The size of the databases used in

More information

Big Data on Microsoft Platform

Big Data on Microsoft Platform Big Data on Microsoft Platform Prepared by GJ Srinivas Corporate TEG - Microsoft Page 1 Contents 1. What is Big Data?...3 2. Characteristics of Big Data...3 3. Enter Hadoop...3 4. Microsoft Big Data Solutions...4

More information

Big Data. Fast Forward. Putting data to productive use

Big Data. Fast Forward. Putting data to productive use Big Data Putting data to productive use Fast Forward What is big data, and why should you care? Get familiar with big data terminology, technologies, and techniques. Getting started with big data to realize

More information

BIG DATA ANALYTICS REFERENCE ARCHITECTURES AND CASE STUDIES

BIG DATA ANALYTICS REFERENCE ARCHITECTURES AND CASE STUDIES BIG DATA ANALYTICS REFERENCE ARCHITECTURES AND CASE STUDIES Relational vs. Non-Relational Architecture Relational Non-Relational Rational Predictable Traditional Agile Flexible Modern 2 Agenda Big Data

More information

Semantic Heterogeneity Reduction for Big Data in Industrial Automation

Semantic Heterogeneity Reduction for Big Data in Industrial Automation Semantic Heterogeneity Reduction for Big Data in Industrial Automation Václav Jirkovský a,b and Marek Obitko b a Czech Technical University in Prague, Prague, Czech Republic b Rockwell Automation Research

More information

Addressing Open Source Big Data, Hadoop, and MapReduce limitations

Addressing Open Source Big Data, Hadoop, and MapReduce limitations Addressing Open Source Big Data, Hadoop, and MapReduce limitations 1 Agenda What is Big Data / Hadoop? Limitations of the existing hadoop distributions Going enterprise with Hadoop 2 How Big are Data?

More information

The Rise of Industrial Big Data. Brian Courtney General Manager Industrial Data Intelligence

The Rise of Industrial Big Data. Brian Courtney General Manager Industrial Data Intelligence The Rise of Industrial Big Data Brian Courtney General Manager Industrial Data Intelligence Agenda Introduction Big Data for the industrial sector Case in point: Big data saves millions at GE Energy Seeking

More information

Associate Professor, Department of CSE, Shri Vishnu Engineering College for Women, Andhra Pradesh, India 2

Associate Professor, Department of CSE, Shri Vishnu Engineering College for Women, Andhra Pradesh, India 2 Volume 6, Issue 3, March 2016 ISSN: 2277 128X International Journal of Advanced Research in Computer Science and Software Engineering Research Paper Available online at: www.ijarcsse.com Special Issue

More information

bigdata Managing Scale in Ontological Systems

bigdata Managing Scale in Ontological Systems Managing Scale in Ontological Systems 1 This presentation offers a brief look scale in ontological (semantic) systems, tradeoffs in expressivity and data scale, and both information and systems architectural

More information

How to use Big Data in Industry 4.0 implementations. LAURI ILISON, PhD Head of Big Data and Machine Learning

How to use Big Data in Industry 4.0 implementations. LAURI ILISON, PhD Head of Big Data and Machine Learning How to use Big Data in Industry 4.0 implementations LAURI ILISON, PhD Head of Big Data and Machine Learning Big Data definition? Big Data is about structured vs unstructured data Big Data is about Volume

More information

Jenny Woodruff Innovation & Low Carbon Networks Engineer Steve Burns Innovation & Low Carbon Networks Engineer LCNF2013 Thursday 14 th November 2013

Jenny Woodruff Innovation & Low Carbon Networks Engineer Steve Burns Innovation & Low Carbon Networks Engineer LCNF2013 Thursday 14 th November 2013 NETWORK MONITORING DATA Using and manipulating data to predict network behaviour. Jenny Woodruff Innovation & Low Carbon Networks Engineer Steve Burns Innovation & Low Carbon Networks Engineer Super Conducting

More information

Cloud Search Based Applications for Big Data - Challenges and Methodologies for Acceleration

Cloud Search Based Applications for Big Data - Challenges and Methodologies for Acceleration Cloud Search Based Applications for Big Data - Challenges and Methodologies for Acceleration George Suciu, Ana Maria Sticlan, Cristina Butca, Alexandru Vulpe, Alexandru Stancu and Simona Halunga R&D Department,

More information

Transforming the Telecoms Business using Big Data and Analytics

Transforming the Telecoms Business using Big Data and Analytics Transforming the Telecoms Business using Big Data and Analytics Event: ICT Forum for HR Professionals Venue: Meikles Hotel, Harare, Zimbabwe Date: 19 th 21 st August 2015 AFRALTI 1 Objectives Describe

More information

Where is... How do I get to...

Where is... How do I get to... Big Data, Fast Data, Spatial Data Making Sense of Location Data in a Smart City Hans Viehmann Product Manager EMEA ORACLE Corporation August 19, 2015 Copyright 2014, Oracle and/or its affiliates. All rights

More information

BIG DATA TOOLS. Top 10 open source technologies for Big Data

BIG DATA TOOLS. Top 10 open source technologies for Big Data BIG DATA TOOLS Top 10 open source technologies for Big Data We are in an ever expanding marketplace!!! With shorter product lifecycles, evolving customer behavior and an economy that travels at the speed

More information

Open source Google-style large scale data analysis with Hadoop

Open source Google-style large scale data analysis with Hadoop Open source Google-style large scale data analysis with Hadoop Ioannis Konstantinou Email: ikons@cslab.ece.ntua.gr Web: http://www.cslab.ntua.gr/~ikons Computing Systems Laboratory School of Electrical

More information

HOW TO MAKE SENSE OF BIG DATA TO BETTER DRIVE BUSINESS PROCESSES, IMPROVE DECISION-MAKING, AND SUCCESSFULLY COMPETE IN TODAY S MARKETS.

HOW TO MAKE SENSE OF BIG DATA TO BETTER DRIVE BUSINESS PROCESSES, IMPROVE DECISION-MAKING, AND SUCCESSFULLY COMPETE IN TODAY S MARKETS. HOW TO MAKE SENSE OF BIG DATA TO BETTER DRIVE BUSINESS PROCESSES, IMPROVE DECISION-MAKING, AND SUCCESSFULLY COMPETE IN TODAY S MARKETS. ALTILIA turns Big Data into Smart Data and enables businesses to

More information

Big Data & QlikView. Democratizing Big Data Analytics. David Freriks Principal Solution Architect

Big Data & QlikView. Democratizing Big Data Analytics. David Freriks Principal Solution Architect Big Data & QlikView Democratizing Big Data Analytics David Freriks Principal Solution Architect TDWI Vancouver Agenda What really is Big Data? How do we separate hype from reality? How does that relate

More information

Machine Learning and Cloud Computing. trends, issues, solutions. EGI-InSPIRE RI-261323

Machine Learning and Cloud Computing. trends, issues, solutions. EGI-InSPIRE RI-261323 Machine Learning and Cloud Computing trends, issues, solutions Daniel Pop HOST Workshop 2012 Future plans // Tools and methods Develop software package(s)/libraries for scalable, intelligent algorithms

More information

Zero-in on business decisions through innovation solutions for smart big data management. How to turn volume, variety and velocity into value

Zero-in on business decisions through innovation solutions for smart big data management. How to turn volume, variety and velocity into value Zero-in on business decisions through innovation solutions for smart big data management How to turn volume, variety and velocity into value ON THE LOOKOUT FOR NEW SOURCES OF VALUE CREATION WHAT WILL DRIVE

More information

Big Data. A general approach to process external multimedia datasets. David Mera

Big Data. A general approach to process external multimedia datasets. David Mera Big Data A general approach to process external multimedia datasets David Mera Laboratory of Data Intensive Systems and Applications (DISA) Masaryk University Brno, Czech Republic 7/10/2014 Table of Contents

More information

CIS492 Special Topics: Cloud Computing د. منذر الطزاونة

CIS492 Special Topics: Cloud Computing د. منذر الطزاونة CIS492 Special Topics: Cloud Computing د. منذر الطزاونة Big Data Definition No single standard definition Big Data is data whose scale, diversity, and complexity require new architecture, techniques, algorithms,

More information

Enabling Self Organising Logistics on the Web of Things

Enabling Self Organising Logistics on the Web of Things Enabling Self Organising Logistics on the Web of Things Monika Solanki, Laura Daniele, Christopher Brewster Aston Business School, Aston University, Birmingham, UK TNO Netherlands Organization for Applied

More information

Big Data Framework for u-healthcare System. Tae-Woong Kim 1, Jai-Hyun Seu 2. jaiseu@inje.ac.kr

Big Data Framework for u-healthcare System. Tae-Woong Kim 1, Jai-Hyun Seu 2. jaiseu@inje.ac.kr Big Data Framework for u-healthcare System Tae-Woong Kim 1, Jai-Hyun Seu 2 1. Department of Computer Education, Silla University, Sasang-Gu, Busan, Korea 2. School of Computer Engineering, Inje University,

More information

Leveraging Big Data Technologies to Support Research in Unstructured Data Analytics

Leveraging Big Data Technologies to Support Research in Unstructured Data Analytics Leveraging Big Data Technologies to Support Research in Unstructured Data Analytics BY FRANÇOYS LABONTÉ GENERAL MANAGER JUNE 16, 2015 Principal partenaire financier WWW.CRIM.CA ABOUT CRIM Applied research

More information

A Professional Big Data Master s Program to train Computational Specialists

A Professional Big Data Master s Program to train Computational Specialists A Professional Big Data Master s Program to train Computational Specialists Anoop Sarkar, Fred Popowich, Alexandra Fedorova! School of Computing Science! Education for Employable Graduates: Critical Questions

More information

SIMATIC IT Historian. Increase your efficiency. SIMATIC IT Historian. Answers for industry.

SIMATIC IT Historian. Increase your efficiency. SIMATIC IT Historian. Answers for industry. SIMATIC IT Historian Increase your efficiency SIMATIC IT Historian Answers for industry. SIMATIC IT Historian: Clear Information at every level Supporting Decisions and Monitoring Efficiency Today s business

More information

E6895 Advanced Big Data Analytics Lecture 4:! Data Store

E6895 Advanced Big Data Analytics Lecture 4:! Data Store E6895 Advanced Big Data Analytics Lecture 4:! Data Store Ching-Yung Lin, Ph.D. Adjunct Professor, Dept. of Electrical Engineering and Computer Science Mgr., Dept. of Network Science and Big Data Analytics,

More information

Peer Research Big Data Analytics

Peer Research Big Data Analytics AUGUST 2012 Peer Research Big Data Analytics Intel s IT Manager Survey on How Organizations Are Using Big Data Why You Should Read This Document This report describes key findings from a survey of 200

More information

Mag. Vikash Kumar, Dr. Anna Fensel kumar@ftw.at, fensel@ftw.at SEMANTIC DATA ANALYTICS AS A BASIS FOR ENERGY EFFICIENCY SERVICES

Mag. Vikash Kumar, Dr. Anna Fensel kumar@ftw.at, fensel@ftw.at SEMANTIC DATA ANALYTICS AS A BASIS FOR ENERGY EFFICIENCY SERVICES Mag. Vikash Kumar, Dr. Anna Fensel kumar@ftw.at, fensel@ftw.at SEMANTIC DATA ANALYTICS AS A BASIS FOR ENERGY EFFICIENCY SERVICES Outline Big data trends changing the ways energy infrastructures operate

More information

Data Virtualization A Potential Antidote for Big Data Growing Pains

Data Virtualization A Potential Antidote for Big Data Growing Pains perspective Data Virtualization A Potential Antidote for Big Data Growing Pains Atul Shrivastava Abstract Enterprises are already facing challenges around data consolidation, heterogeneity, quality, and

More information

Machina Research. Where is the value in IoT? IoT data and analytics may have an answer. Emil Berthelsen, Principal Analyst April 28, 2016

Machina Research. Where is the value in IoT? IoT data and analytics may have an answer. Emil Berthelsen, Principal Analyst April 28, 2016 Machina Research Where is the value in IoT? IoT data and analytics may have an answer Emil Berthelsen, Principal Analyst April 28, 2016 About Machina Research Machina Research is the world s leading provider

More information

So What s the Big Deal?

So What s the Big Deal? So What s the Big Deal? Presentation Agenda Introduction What is Big Data? So What is the Big Deal? Big Data Technologies Identifying Big Data Opportunities Conducting a Big Data Proof of Concept Big Data

More information

Real Time Big Data Processing

Real Time Big Data Processing Real Time Big Data Processing Cloud Expo 2014 Ian Meyers Amazon Web Services Global Infrastructure Deployment & Administration App Services Analytics Compute Storage Database Networking AWS Global Infrastructure

More information

Copyright 2013 Splunk Inc. Introducing Splunk 6

Copyright 2013 Splunk Inc. Introducing Splunk 6 Copyright 2013 Splunk Inc. Introducing Splunk 6 Safe Harbor Statement During the course of this presentation, we may make forward looking statements regarding future events or the expected performance

More information

III Big Data Technologies

III Big Data Technologies III Big Data Technologies Today, new technologies make it possible to realize value from Big Data. Big data technologies can replace highly customized, expensive legacy systems with a standard solution

More information

Storage and Retrieval of Large RDF Graph Using Hadoop and MapReduce

Storage and Retrieval of Large RDF Graph Using Hadoop and MapReduce Storage and Retrieval of Large RDF Graph Using Hadoop and MapReduce Mohammad Farhan Husain, Pankil Doshi, Latifur Khan, and Bhavani Thuraisingham University of Texas at Dallas, Dallas TX 75080, USA Abstract.

More information

Design and Implementation of a Semantic Web Solution for Real-time Reservoir Management

Design and Implementation of a Semantic Web Solution for Real-time Reservoir Management Design and Implementation of a Semantic Web Solution for Real-time Reservoir Management Ram Soma 2, Amol Bakshi 1, Kanwal Gupta 3, Will Da Sie 2, Viktor Prasanna 1 1 University of Southern California,

More information

Big Data, Cloud Computing, Spatial Databases Steven Hagan Vice President Server Technologies

Big Data, Cloud Computing, Spatial Databases Steven Hagan Vice President Server Technologies Big Data, Cloud Computing, Spatial Databases Steven Hagan Vice President Server Technologies Big Data: Global Digital Data Growth Growing leaps and bounds by 40+% Year over Year! 2009 =.8 Zetabytes =.08

More information

Big Data and Your Data Warehouse Philip Russom

Big Data and Your Data Warehouse Philip Russom Big Data and Your Data Warehouse Philip Russom TDWI Research Director for Data Management April 5, 2012 Sponsor Speakers Philip Russom Research Director, Data Management, TDWI Peter Jeffcock Director,

More information

Big Data lisää älyä tiedosta

Big Data lisää älyä tiedosta 2011 Tieto Corporation Big Data lisää älyä tiedosta ebusiness Forum 21.5.2013 Ilkka Korkiakoski VP Financial Services Agenda Megatrends and needs for Big Data What is the value of Big Data? Use scenarios

More information

The 3 questions to ask yourself about BIG DATA

The 3 questions to ask yourself about BIG DATA The 3 questions to ask yourself about BIG DATA Do you have a big data problem? Companies looking to tackle big data problems are embarking on a journey that is full of hype, buzz, confusion, and misinformation.

More information

Eric Ledu, The Createch Group, a BELL company

Eric Ledu, The Createch Group, a BELL company Eric Ledu, The Createch Group, a BELL company Intelligence Analytics maturity Past Present Future Predictive Modeling Optimization What is the best that could happen? Raw Data Cleaned Data Standard Reports

More information

Intel HPC Distribution for Apache Hadoop* Software including Intel Enterprise Edition for Lustre* Software. SC13, November, 2013

Intel HPC Distribution for Apache Hadoop* Software including Intel Enterprise Edition for Lustre* Software. SC13, November, 2013 Intel HPC Distribution for Apache Hadoop* Software including Intel Enterprise Edition for Lustre* Software SC13, November, 2013 Agenda Abstract Opportunity: HPC Adoption of Big Data Analytics on Apache

More information

Scaling-out Semantic Data Management and Processing

Scaling-out Semantic Data Management and Processing Scaling-out Semantic Data Management and Processing Tomasz Wiktor Wlodarczyk, Norway CIPSI Director: prof. Chunming Rong Areas of interest: Reasoning, analytics and simulation Distributed systems Dependability

More information

Publishing Linked Data Requires More than Just Using a Tool

Publishing Linked Data Requires More than Just Using a Tool Publishing Linked Data Requires More than Just Using a Tool G. Atemezing 1, F. Gandon 2, G. Kepeklian 3, F. Scharffe 4, R. Troncy 1, B. Vatant 5, S. Villata 2 1 EURECOM, 2 Inria, 3 Atos Origin, 4 LIRMM,

More information

Improving Data Processing Speed in Big Data Analytics Using. HDFS Method

Improving Data Processing Speed in Big Data Analytics Using. HDFS Method Improving Data Processing Speed in Big Data Analytics Using HDFS Method M.R.Sundarakumar Assistant Professor, Department Of Computer Science and Engineering, R.V College of Engineering, Bangalore, India

More information

Big Data Challenges and Success Factors. Deloitte Analytics Your data, inside out

Big Data Challenges and Success Factors. Deloitte Analytics Your data, inside out Big Data Challenges and Success Factors Deloitte Analytics Your data, inside out Big Data refers to the set of problems and subsequent technologies developed to solve them that are hard or expensive to

More information

Big Data. White Paper. Big Data Executive Overview WP-BD-10312014-01. Jafar Shunnar & Dan Raver. Page 1 Last Updated 11-10-2014

Big Data. White Paper. Big Data Executive Overview WP-BD-10312014-01. Jafar Shunnar & Dan Raver. Page 1 Last Updated 11-10-2014 White Paper Big Data Executive Overview WP-BD-10312014-01 By Jafar Shunnar & Dan Raver Page 1 Last Updated 11-10-2014 Table of Contents Section 01 Big Data Facts Page 3-4 Section 02 What is Big Data? Page

More information

Unlocking the Intelligence in. Big Data. Ron Kasabian General Manager Big Data Solutions Intel Corporation

Unlocking the Intelligence in. Big Data. Ron Kasabian General Manager Big Data Solutions Intel Corporation Unlocking the Intelligence in Big Data Ron Kasabian General Manager Big Data Solutions Intel Corporation Volume & Type of Data What s Driving Big Data? 10X Data growth by 2016 90% unstructured 1 Lower

More information

Outils pour l'analyse prédictive parallèle de multiples sources de données non structurées

Outils pour l'analyse prédictive parallèle de multiples sources de données non structurées Outils pour l'analyse prédictive parallèle de multiples sources de données non structurées Forum Ter@tec Mercredi 25 juin 2015 Marc Wolff Application Engineer HPC & Big Data 2015 The MathWorks, Inc. 1

More information

Big Data & Analytics: Your concise guide (note the irony) Wednesday 27th November 2013

Big Data & Analytics: Your concise guide (note the irony) Wednesday 27th November 2013 Big Data & Analytics: Your concise guide (note the irony) Wednesday 27th November 2013 Housekeeping 1. Any questions coming out of today s presentation can be discussed in the bar this evening 2. OCF is

More information

Big Data Technology ดร.ช ชาต หฤไชยะศ กด. Choochart Haruechaiyasak, Ph.D.

Big Data Technology ดร.ช ชาต หฤไชยะศ กด. Choochart Haruechaiyasak, Ph.D. Big Data Technology ดร.ช ชาต หฤไชยะศ กด Choochart Haruechaiyasak, Ph.D. Speech and Audio Technology Laboratory (SPT) National Electronics and Computer Technology Center (NECTEC) National Science and Technology

More information

Danny Wang, Ph.D. Vice President of Business Strategy and Risk Management Republic Bank

Danny Wang, Ph.D. Vice President of Business Strategy and Risk Management Republic Bank Danny Wang, Ph.D. Vice President of Business Strategy and Risk Management Republic Bank Agenda» Overview» What is Big Data?» Accelerates advances in computer & technologies» Revolutionizes data measurement»

More information

Data collection architecture for Big Data

Data collection architecture for Big Data Data collection architecture for Big Data a framework for a research agenda (Research in progress - ERP Sense Making of Big Data) Wout Hofman, May 2015, BDEI workshop 2 Big Data succes stories bias our

More information

SEAIP 2009 Presentation

SEAIP 2009 Presentation SEAIP 2009 Presentation By David Tan Chair of Yahoo! Hadoop SIG, 2008-2009,Singapore EXCO Member of SGF SIG Imperial College (UK), Institute of Fluid Science (Japan) & Chicago BOOTH GSB (USA) Alumni Email:

More information

Challenges. Department of Informatics University of Oslo. Presenter. kashifd@ifi.uio.no. October 25, 2011

Challenges. Department of Informatics University of Oslo. Presenter. kashifd@ifi.uio.no. October 25, 2011 Internet t of Things: Applications and Challenges Presenter Kashif Dar kashifd@ifi.uio.no INF9910: Cyber Physical Systems Department of Informatics University of Oslo October 25, 2011 Overview Internet

More information

Towards a Thriving Data Economy: Open Data, Big Data, and Data Ecosystems

Towards a Thriving Data Economy: Open Data, Big Data, and Data Ecosystems Towards a Thriving Data Economy: Open Data, Big Data, and Data Ecosystems Volker Markl volker.markl@tu-berlin.de dima.tu-berlin.de dfki.de/web/research/iam/ bbdc.berlin Based on my 2014 Vision Paper On

More information

Big Data: Opportunities & Challenges, Myths & Truths 資 料 來 源 : 台 大 廖 世 偉 教 授 課 程 資 料

Big Data: Opportunities & Challenges, Myths & Truths 資 料 來 源 : 台 大 廖 世 偉 教 授 課 程 資 料 Big Data: Opportunities & Challenges, Myths & Truths 資 料 來 源 : 台 大 廖 世 偉 教 授 課 程 資 料 美 國 13 歲 學 生 用 Big Data 找 出 霸 淩 熱 點 Puri 架 設 網 站 Bullyvention, 藉 由 分 析 Twitter 上 找 出 提 到 跟 霸 凌 相 關 的 詞, 搭 配 地 理 位 置

More information

UNIVERSITY OF INFINITE AMBITIONS. MASTER OF SCIENCE COMPUTER SCIENCE DATA SCIENCE AND SMART SERVICES

UNIVERSITY OF INFINITE AMBITIONS. MASTER OF SCIENCE COMPUTER SCIENCE DATA SCIENCE AND SMART SERVICES UNIVERSITY OF INFINITE AMBITIONS. MASTER OF SCIENCE COMPUTER SCIENCE DATA SCIENCE AND SMART SERVICES MASTER S PROGRAMME COMPUTER SCIENCE - DATA SCIENCE AND SMART SERVICES (DS3) This is a specialization

More information

Challenges for Data Driven Systems

Challenges for Data Driven Systems Challenges for Data Driven Systems Eiko Yoneki University of Cambridge Computer Laboratory Quick History of Data Management 4000 B C Manual recording From tablets to papyrus to paper A. Payberah 2014 2

More information

Big Data - Infrastructure Considerations

Big Data - Infrastructure Considerations April 2014, HAPPIEST MINDS TECHNOLOGIES Big Data - Infrastructure Considerations Author Anand Veeramani / Deepak Shivamurthy SHARING. MINDFUL. INTEGRITY. LEARNING. EXCELLENCE. SOCIAL RESPONSIBILITY. Copyright

More information

Copyright 2014, Neudesic. All rights reserved.

Copyright 2014, Neudesic. All rights reserved. 2 Accelerating Modernization Across the Enterprise Cloud Computing User Experience Enterprise Mobility Customer Relationship Management Business Analysis Managed Services Custom Application Development

More information

Datameer Cloud. End-to-End Big Data Analytics in the Cloud

Datameer Cloud. End-to-End Big Data Analytics in the Cloud Cloud End-to-End Big Data Analytics in the Cloud Datameer Cloud unites the economics of the cloud with big data analytics to deliver extremely fast time to insight. With Datameer Cloud, empowered line

More information

Datenverwaltung im Wandel - Building an Enterprise Data Hub with

Datenverwaltung im Wandel - Building an Enterprise Data Hub with Datenverwaltung im Wandel - Building an Enterprise Data Hub with Cloudera Bernard Doering Regional Director, Central EMEA, Cloudera Cloudera Your Hadoop Experts Founded 2008, by former employees of Employees

More information

Ubuntu and Hadoop: the perfect match

Ubuntu and Hadoop: the perfect match WHITE PAPER Ubuntu and Hadoop: the perfect match February 2012 Copyright Canonical 2012 www.canonical.com Executive introduction In many fields of IT, there are always stand-out technologies. This is definitely

More information

SURVEY REPORT DATA SCIENCE SOCIETY 2014

SURVEY REPORT DATA SCIENCE SOCIETY 2014 SURVEY REPORT DATA SCIENCE SOCIETY 2014 TABLE OF CONTENTS Contents About the Initiative 1 Report Summary 2 Participants Info 3 Participants Expertise 6 Suggested Discussion Topics 7 Selected Responses

More information

Outline. What is Big data and where they come from? How we deal with Big data?

Outline. What is Big data and where they come from? How we deal with Big data? What is Big Data Outline What is Big data and where they come from? How we deal with Big data? Big Data Everywhere! As a human, we generate a lot of data during our everyday activity. When you buy something,

More information

Shared Infrastructure: What and Where is Collaboration Needed to Build the SM Platform?

Shared Infrastructure: What and Where is Collaboration Needed to Build the SM Platform? Smart Manufacturing Forum Shared Infrastructure: What and Where is Collaboration Needed to Build the SM Platform? 10:45-11:45am panel discussion Moderator: John Bernaden, Vice Chair, Smart Manufacturing

More information

De la Business Intelligence aux Big Data. Marie- Aude AUFAURE Head of the Business Intelligence team Ecole Centrale Paris. 22/01/14 Séminaire Big Data

De la Business Intelligence aux Big Data. Marie- Aude AUFAURE Head of the Business Intelligence team Ecole Centrale Paris. 22/01/14 Séminaire Big Data De la Business Intelligence aux Big Data Marie- Aude AUFAURE Head of the Business Intelligence team Ecole Centrale Paris 22/01/14 Séminaire Big Data 1 Agenda EvoluHon of Business Intelligence SemanHc Technologies

More information

HPC technology and future architecture

HPC technology and future architecture HPC technology and future architecture Visual Analysis for Extremely Large-Scale Scientific Computing KGT2 Internal Meeting INRIA France Benoit Lange benoit.lange@inria.fr Toàn Nguyên toan.nguyen@inria.fr

More information

BIG DATA: BIG CHALLENGE FOR SOFTWARE TESTERS

BIG DATA: BIG CHALLENGE FOR SOFTWARE TESTERS BIG DATA: BIG CHALLENGE FOR SOFTWARE TESTERS Megha Joshi Assistant Professor, ASM s Institute of Computer Studies, Pune, India Abstract: Industry is struggling to handle voluminous, complex, unstructured

More information

2 Linked Data, Non-relational Databases and Cloud Computing

2 Linked Data, Non-relational Databases and Cloud Computing Distributed RDF Graph Keyword Search 15 2 Linked Data, Non-relational Databases and Cloud Computing 2.1.Linked Data The World Wide Web has allowed an unprecedented amount of information to be published

More information

Log Mining Based on Hadoop s Map and Reduce Technique

Log Mining Based on Hadoop s Map and Reduce Technique Log Mining Based on Hadoop s Map and Reduce Technique ABSTRACT: Anuja Pandit Department of Computer Science, anujapandit25@gmail.com Amruta Deshpande Department of Computer Science, amrutadeshpande1991@gmail.com

More information

Research trends relevant to data warehousing and OLAP include [Cuzzocrea et al.]: Combining the benefits of RDBMS and NoSQL database systems

Research trends relevant to data warehousing and OLAP include [Cuzzocrea et al.]: Combining the benefits of RDBMS and NoSQL database systems DATA WAREHOUSING RESEARCH TRENDS Research trends relevant to data warehousing and OLAP include [Cuzzocrea et al.]: Data source heterogeneity and incongruence Filtering out uncorrelated data Strongly unstructured

More information

Big Data Analytics. Prof. Dr. Lars Schmidt-Thieme

Big Data Analytics. Prof. Dr. Lars Schmidt-Thieme Big Data Analytics Prof. Dr. Lars Schmidt-Thieme Information Systems and Machine Learning Lab (ISMLL) Institute of Computer Science University of Hildesheim, Germany 33. Sitzung des Arbeitskreises Informationstechnologie,

More information

AGENDA. What is BIG DATA? What is Hadoop? Why Microsoft? The Microsoft BIG DATA story. Our BIG DATA Roadmap. Hadoop PDW

AGENDA. What is BIG DATA? What is Hadoop? Why Microsoft? The Microsoft BIG DATA story. Our BIG DATA Roadmap. Hadoop PDW AGENDA What is BIG DATA? What is Hadoop? Why Microsoft? The Microsoft BIG DATA story Hadoop PDW Our BIG DATA Roadmap BIG DATA? Volume 59% growth in annual WW information 1.2M Zetabytes (10 21 bytes) this

More information

LDIF - Linked Data Integration Framework

LDIF - Linked Data Integration Framework LDIF - Linked Data Integration Framework Andreas Schultz 1, Andrea Matteini 2, Robert Isele 1, Christian Bizer 1, and Christian Becker 2 1. Web-based Systems Group, Freie Universität Berlin, Germany a.schultz@fu-berlin.de,

More information

HadoopRDF : A Scalable RDF Data Analysis System

HadoopRDF : A Scalable RDF Data Analysis System HadoopRDF : A Scalable RDF Data Analysis System Yuan Tian 1, Jinhang DU 1, Haofen Wang 1, Yuan Ni 2, and Yong Yu 1 1 Shanghai Jiao Tong University, Shanghai, China {tian,dujh,whfcarter}@apex.sjtu.edu.cn

More information

Session 1: IT Infrastructure Security Vertica / Hadoop Integration and Analytic Capabilities for Federal Big Data Challenges

Session 1: IT Infrastructure Security Vertica / Hadoop Integration and Analytic Capabilities for Federal Big Data Challenges Session 1: IT Infrastructure Security Vertica / Hadoop Integration and Analytic Capabilities for Federal Big Data Challenges James Campbell Corporate Systems Engineer HP Vertica jcampbell@vertica.com Big

More information

IO-Link an integral part in the next industrial revolution known as Industry 4.0

IO-Link an integral part in the next industrial revolution known as Industry 4.0 IO-Link an integral part in the next industrial revolution known as Industry 4.0 IO-Link an integral part in the next industrial revolution known as Industry 4.0 The manufacturing industry is on the verge

More information

Big Data and Transactional Databases Exploding Data Volume is Creating New Stresses on Traditional Transactional Databases

Big Data and Transactional Databases Exploding Data Volume is Creating New Stresses on Traditional Transactional Databases Big Data and Transactional Databases Exploding Data Volume is Creating New Stresses on Traditional Transactional Databases Introduction The world is awash in data and turning that data into actionable

More information

BIG DATA TRENDS AND TECHNOLOGIES

BIG DATA TRENDS AND TECHNOLOGIES BIG DATA TRENDS AND TECHNOLOGIES THE WORLD OF DATA IS CHANGING Cloud WHAT IS BIG DATA? Big data are datasets that grow so large that they become awkward to work with using onhand database management tools.

More information

Big Data Technologies Compared June 2014

Big Data Technologies Compared June 2014 Big Data Technologies Compared June 2014 Agenda What is Big Data Big Data Technology Comparison Summary Other Big Data Technologies Questions 2 What is Big Data by Example The SKA Telescope is a new development

More information

Sensor Information Representation for the Internet of Things

Sensor Information Representation for the Internet of Things Sensor Information Representation for the Internet of Things Jiehan Zhou Jiehan.zhou@ee.oulu.fi University of Oulu, Finland Carleton University, Canada Agenda Internet of Things and Challenges Application

More information

A Novel Cloud Based Elastic Framework for Big Data Preprocessing

A Novel Cloud Based Elastic Framework for Big Data Preprocessing School of Systems Engineering A Novel Cloud Based Elastic Framework for Big Data Preprocessing Omer Dawelbeit and Rachel McCrindle October 21, 2014 University of Reading 2008 www.reading.ac.uk Overview

More information