Mastering the Velocity Dimension of Big Data

Size: px
Start display at page:

Download "Mastering the Velocity Dimension of Big Data"

Transcription

1 Mastering the Velocity Dimension of Big Data Emanuele Della Valle DEIB - Politecnico di Milano

2 It's a streaming world Agenda Mastering the velocity dimension with informaeon flow processing Hands- on on esper and the Event Processing Language Stream Reasoning: adding the variety dimension What about Veracity? 2

3 It's a streaming World! 1/2 Off- shore oil operaeons Smart CiEes Global Contact Center Social networks Generate data streams! 3

4 It's a streaming World! 2/2 What is the expected Eme to failure when that turbine's barring starts to vibrate as detected in the last 10 minutes? Is public transportaeon where the people are? Who are the best available agents to route all these unexpected contacts about the tariff plan launched yesterday? Who is driving the discussion about the top 10 emerging topics? E. Della Valle, S. Ceri, F. van Harmelen, D. Fensel It's a Streaming World! Reasoning upon Rapidly Changing InformaEon. IEEE Intelligent Systems 24(6): (2009) 4

5 Requirements 1/8 A system able to answer those queries must be able to handle massive datasets A typical oil produceon plaborm is equipped with about sensors Telecom data is the most pervasive data source in urban are, in Milano there are 1.8 million mobile users A global contact centre of a Telecom operator counts 500 millions of clients Facebook alone has 1.1 billion of aceve users 5

6 Requirements 2/8 A system able to answer those queries must be able to process data streams on the fly The sensors on typical oil produceon plaborm generates 10,000 observaeons per minute with peaks of 100,000 o/m The mobile users in Milano generates 20,000 call/sms/data conneceons per minute with peaks of 80,000 c/m A global contact centre receives 10,000 contacts per minute with peaks of 30,000 c/m Facebook, as of May 2013, observes 3 millions "I like" per minute 6

7 Requirements 3/8 A system able to answer those queries must be able to cope with heterogeneous dataset The sensors on typical oil produceon have been deployed over 10 years by 10s of different producers Tens of data sources are normally needed to make sense of an urban phenomena A global contact centre consists in 100s of offices owned by different subsidiary companies engaged yearly Each social network has its own data model, APIs, 7

8 Requirements 4/8 A system able to answer those queries must be able to cope with incomplete data 10s of sensors and networking links broke down daily Coverage is incomplete Only standard cases are covered by fully machine processable data records 100s of contacts per minute are manage ad- hoc ConversaKons happen outside the social networks, too! 8

9 Requirements 5/8 A system able to answer those queries must be able to cope with uncertain data Sensor out- of- operakng range Faulty sensors Agents misunderstand, get Kred, Irony, sarcasm, 9

10 Requirements 6/8 A system able to answer those queries must be able to provide reackve answers deteceon of dangerous situaeons must occur within minutes recommendaeons to ciezens must be performed in few seconds roueng a contact through each step of the decision tree must take less than a second Search autocompleeng may need to be updated every few minutes 10

11 Requirements 7/8 A system able to answer those queries must be able to support fine- grained informakon access IdenEfy a turbine among thousands Locate a bus among thousands Contact an agent among thousands IdenEfy an opinion maker among thousands of influencers for a topic 11

12 Requirements 8/8 A system able to answer those queries must be able to integrate complex domain models of operakonal and control process various city aspects contact management, contract types, agent skills, contactor profiles, topics, user profiles, 12

13 Requirements (wrap up) A system able to answer those queries must be able to handle massive datasets x process data streams on the fly cope with heterogeneous dataset cope with incomplete data cope with uncertain data provide reackve answers support fine- grained informakon access integrate complex domain models 13 x Volume' x x x Velocity' x x x x Variety' x Veracity'

14 Other domains Intrusion deteceon Fraud DetecEon Emergency Response Services TransportaEon and LogisEcs Supply Chain OpEmizaEon System monitoring Click inspeceon... 14

15 Mastering the Velocity dimension with InformaEon Flow Processing (IFP) solueons 15

16 Velocity: Say goodbye to DBMS The DBMS way query The IFP way data data reply 16 query reply

17 IFP - Gartner The Gartner hype cycle 17

18 IFP - Forrester Forrester s top 15 emerging tech to watch: Now to

19 Is there a market beyond hype? Complex Event Processing Market Worth $3,322M by 2018 (2014 Report by MarketsandMarkets) Major players include: Microsom IBM Oracle SAP Tibco... 19

20 InformaEon Flow Processing The IFP engine processes incoming flows of informa1on according to a set of processing rules Processing is on line Sources produce the incoming informaeon flows, sinks consume the results of processing, rule managers add or remove rules InformaEon flows are composed of informa1on items Items part of the same flow are neither necessarily ordered nor of the same kind Processing involve filtering, combining, and aggregaeng flows, item by item as they enter the engine Sources Informa1on Flows 20 IFP Engine Informa1on Flows Rules Rule managers Sinks

21 IFP: A bit of history of two approaches Traditional DBMS Eventbased Systems Active DBMS DSMS 21 CEP

22 From Passive to AcEve DBMSs Standard DBMSs Purely passive: Human- ac1ve database- passive (HADP) ExecuEon happens only when asked by clients (through queries) AcEve DBMSs The reaceve behavior moves (in part) from the applicaeon to the DB layer which executes Event CondiEon AcEon (ECA) rules 22

23 As a DBMS extension AcEve DBMSs Rules may only refer to the internal state of the DB Closed DB applicaeons Rules may support the semanecs of the applicaeon, but external sources of events are not allowed But events may come from external sources 23

24 Data Stream Management Systems (DSMS) Data streams are (unbounded) sequences of Eme- varying data elements Represent: an (almost) conenuous flow of informaeon Eme with the recent informaeon being more relevant as it describes the current state of a dynamic system 24

25 Data Stream Management Systems (DSMS) The nature of streams requires a paradigmaec change* from persistent data one Eme semanecs to transient data conenuous * This paradigmaec change first arose in DB community in the late '90s 25

26 ConEnuous SemanEcs ConEnuous queries registered over streams that are observed trough windows window Dynamic System 26 input streams Registered ConEnuous Query streams of answer

27 Event- based systems Components collaborate by exchanging informaeon about occurrent events. In parecular: Components publish noeficaeons about the events they observe, or they subscribe to the events they are interested to be noefied about CommunicaEon is: Purely message based Asynchronous MulEcast Implicit Anonymous topic=fire* & place=* topic=* & place=1 st floor topic=fire alarm & place=* 27 fire training at 1 st floor fire alarm at 1 st fire floor training at 1 st floor fire alarm at 1 st fire floor training at 1 st floor fire alarm at 1 st floor fire alarm at 1 st floor

28 Complex Event Processing (CEP) CEP systems adds the ability to deploy rules that describe how composite events can be generated from primieve (or composite) ones Typical CEP rules search for sequences of events Raise C if A B Time is a key aspect in CEP Rules

29 Transforming vs. DetecEng Systems Transforming rules Define an execueon plan combining primieve operators Each operator transforms one or more input flows into one or more output flows The execueon plan can be defined: explicitly (e.g., through graphical notaeon) Usually, deteceon is based on a implicitly (using a high level language) Omen used with homogeneous informaeon flows To take advantage of the predefined structure of input and output 29 DetecKng rules Present an explicit disenceon between a detec1on and produc1on phase Detect a situaeon of interest Produce a complex event to noefy that the situaeon has happened logical predicate that captures paferns of interest in the history of received items

30 DeclaraKve Transforming languages Specify the expected result rather than the desired execueon flow Usually derive from relaeonal languages ImperaKve Specify the desired execueon flow StarEng from primieve operators Can be user- defined RelaEonal algebra / SQL CQL/Stream: Select IStream(*) From F1[Rows 5], F2[Rows 10] Where F1.A = F2.A 30 Usually adopt a graphical notaeon Arvind Arasu, Shivnath Babu, Jennifer Widom: The CQL conenuous query language: semanec foundaeons and query execueon. VLDB J. 15(2): (2006)

31 ImperaEve Languages Aurora (Boxes & Arrows Model) 31 Daniel J. Abadi, Donald Carney, Ugur ÇeEntemel, Mitch Cherniack, ChrisEan Convey, Sangdon Lee, Michael Stonebraker, Nesime Tatbul, Stanley B. Zdonik: Aurora: a new model and architecture for data stream management. VLDB J. 12(2): (2003)

32 DeclaraEve languages Specify a firing condi1on as a pa?ern Select a poreon of incoming flows through Logic operators Content / Eming constraints The ac1on part uses the selected items to produce new knowledge TESLA / T-Rex Define Fire(area: string, measuredtemp: double) From Smoke(area=$a) and last Temp(area=$a and value>45) within 5 min. from Smoke Where area=smoke.area and measuredtemp=temp.value Gianpaolo Cugola, Alessandro Margara: TESLA: a formally defined event specificaeon language. DEBS 2010: Gianpaolo Cugola, Alessandro Margara: Complex event processing with T- REX. Journal of Systems and Somware 85(8): (2012) 32

33 Hands- on Esper and EPL See dedicated slides 33

34 Adding Variety to Velocity 34

35 Variety: Syntax changes, semanecs not Data comes from muleple sources in muleple formats The SemanEc Web is teaching us how to represent informaeon (knowledge) in a standard way RDF, OWL, Ontologies, SPARQL,... But the world in not staec... 35

36 DSMS/CEP + SemanEc Web = Stream Reasoning Requirement massive datasets data streams heterogeneous dataset DSMS CEP SemanKc Web incomplete data noisy data reaceve answers fine- grained informaeon access complex domain models The Stream Reasoning Era is coming 36

37 IFP: A bit of history (ConEnued) Traditional DBMS Eventbased Systems Active DBMS DSMS Semantic Web Stream Reasoning 37 CEP

38 Research QuesEon is it possible to make sense in real Kme of mulkple, heterogeneous, gigankc and inevitably noisy and incomplete data streams in order to support the decision process of extremely large numbers of concurrent user? E. Della Valle, S. Ceri, F. van Harmelen, D. Fensel It's a Streaming World! Reasoning upon Rapidly Changing InformaEon. IEEE Intelligent Systems 24(6): (2009) 38

39 Stream Reasoning feasibility (intuieon) Many relevant reasoning methods are not able to deal with high frequency data streams However, trade- off exists between the complexity of the reasoning method and the frequency of the data stream NEXPTIME AbstracEon DL Reasoning 1 Hz Complexity PTIME AC 0 SelecEon InterpretaEon DL- Lite SemanEc Streams Raw Stream Processing Complexity vs. Dynamics Querying Re- wrieng 10 4 Hz Change Frequency Heiner Stuckenschmidt, Stefano Ceri, Emanuele Della Valle, Frank van Harmelen: Towards Expressive Stream Reasoning. Proceedings of the Dagstuhl Seminar on SemanEc Aspects of Sensor Networks,

40 PracEcal cases 10+ deployments in Sensor Networks, and Social media analyecs BOTTARI City Data Fusion Winner of SemanKc Web Challenge 2011 Winner of IBM faculty award 2013 M. Balduini, I. Celino, D. Dell Aglio, E. Della Valle, Y. Huang, T. Lee, S.- H. Kim, V. Tresp: BOTTARI: An augmented reality mobile applicaeon to deliver personalized and locaeon- based recommendaeons by conenuous analysis of social media streams. J. Web Sem. 16: (2012) 40

41 Where to learn more about stream General Info reasoning h?p://streamreasoning.org W3C Community h?p://www.w3.org/community/rsp/ Papers h?p://scholar.google.com/scholar?hl=en&q=stream +reasoning 41

42 What about Veracity? 42

43 Support for Uncertainty Many applicakons deal with imprecise (or even incorrect) data from sources E.g., sensors, RFID, The ability to associate a degree of uncertainty to informaeon items increases expressiveness To the content of items Imprecise temperature reading To the presence of an item (occurrence of an event) Spurious RFID reading 43 Two orthogonal aspects Support for uncertain input Allows rules to deal with/ reason about uncertain input data Support for uncertain output Allows rules to associate a degree of uncertainty to the output produced Uncertain rules

44 Where to learn more on InformaEon Flow Processing PhD course on Stream And Complex Event Processing Lecturers Emanuele Della Valle Gianpaolo Cugola Alessandro Margara More info h?p://bit.ly/phd- course- IFP 44

45 Thank you! Any QuesEon? Emanuele Della Valle DEIB - Politecnico di Milano

Processing Flows of Information: From Data Stream to Complex Event Processing

Processing Flows of Information: From Data Stream to Complex Event Processing Processing Flows of Information: From Data Stream to Complex Event Processing GIANPAOLO CUGOLA and ALESSANDRO MARGARA Dip. di Elettronica e Informazione Politecnico di Milano, Italy A large number of distributed

More information

BOTTARI: Location based Social Media Analysis with Semantic Web

BOTTARI: Location based Social Media Analysis with Semantic Web BOTTARI: Location based Social Media Analysis with Semantic Web Irene Celino 1, Daniele Dell Aglio 1, Emanuele Della Valle 2,1, Marco Balduini 2, Yi Huang 3, Tony Lee 4, Seon-Ho Kim 4, and Volker Tresp

More information

Aurora: a new model and architecture for data stream management

Aurora: a new model and architecture for data stream management Aurora: a new model and architecture for data stream management Daniel J. Abadi 1, Don Carney 2, Ugur Cetintemel 2, Mitch Cherniack 1, Christian Convey 2, Sangdon Lee 2, Michael Stonebraker 3, Nesime Tatbul

More information

Processing Flows of Information: From Data Stream to Complex Event Processing

Processing Flows of Information: From Data Stream to Complex Event Processing Processing Flows of Information: From Data Stream to Complex Event Processing GIANPAOLO CUGOLA and ALESSANDRO MARGARA, Politecnico di Milano A large number of distributed applications requires continuous

More information

Deployment Strategies for Distributed Complex Event Processing

Deployment Strategies for Distributed Complex Event Processing Noname manuscript No. (will be inserted by the editor) Deployment Strategies for Distributed Complex Event Processing Gianpaolo Cugola Alessandro Margara Received: July 19th, 212 / Accepted: date Abstract

More information

Solving big data problems in real-time with CEP and Dashboards - patterns and tips

Solving big data problems in real-time with CEP and Dashboards - patterns and tips September 10-13, 2012 Orlando, Florida Solving big data problems in real-time with CEP and Dashboards - patterns and tips Kevin Wilson Karl Kwong Learning Points Big data is a reality and organizations

More information

Mastering the variety dimension

Mastering the variety dimension Mastering the variety dimension Emanuele Della Valle DEIB - Politecnico di Milano emanuele.dellavalle@polimi.it h>p://emanueledellavalle.org Agenda In a changing world variety is unavoidable SemanEc Technologies

More information

Complex Information Management Using a Framework Supported by ECA Rules in XML

Complex Information Management Using a Framework Supported by ECA Rules in XML Complex Information Management Using a Framework Supported by ECA Rules in XML Bing Wu, Essam Mansour and Kudakwashe Dube School of Computing, Dublin Institute of Technology Kevin Street, Dublin 8, Ireland

More information

Industry 4.0 and Big Data

Industry 4.0 and Big Data Industry 4.0 and Big Data Marek Obitko, mobitko@ra.rockwell.com Senior Research Engineer 03/25/2015 PUBLIC PUBLIC - 5058-CO900H 2 Background Joint work with Czech Institute of Informatics, Robotics and

More information

Social Data Science for Intelligent Cities

Social Data Science for Intelligent Cities Social Data Science for Intelligent Cities The Role of Social Media for Sensing Crowds Prof.dr.ir. Geert-Jan Houben TU Delft Web Information Systems & Delft Data Science WIS - Web Information Systems Why

More information

Social Media Analy.cs (SMA)

Social Media Analy.cs (SMA) Social Media Analy.cs (SMA) Emanuele Della Valle DEIB - Politecnico di Milano emanuele.dellavalle@polimi.it hap://emanueledellavalle.org What's social media? haps://www.youtube.com/watch?v=sgniiud_oqg

More information

Combining Sequence Databases and Data Stream Management Systems Technical Report Philipp Bichsel ETH Zurich, 2-12-2007

Combining Sequence Databases and Data Stream Management Systems Technical Report Philipp Bichsel ETH Zurich, 2-12-2007 Combining Sequence Databases and Data Stream Management Systems Technical Report Philipp Bichsel ETH Zurich, 2-12-2007 Abstract This technical report explains the differences and similarities between the

More information

Real Time Business Performance Monitoring and Analysis Using Metric Network

Real Time Business Performance Monitoring and Analysis Using Metric Network Real Time Business Performance Monitoring and Analysis Using Metric Network Pu Huang, Hui Lei, Lipyeow Lim IBM T. J. Watson Research Center Yorktown Heights, NY, 10598 Abstract-Monitoring and analyzing

More information

Achieving Semantic Interoperability By UsingComplex Event Processing Technology

Achieving Semantic Interoperability By UsingComplex Event Processing Technology Achieving Semantic Interoperability By UsingComplex Event Processing Technology Margarete Donovang-Kuhlisch Geschäftsbereich Verteidigung IBMDeutschlandGmbH Gorch-Fock-Str. 4 D-53229Bonn mdk@de.ibm.com

More information

Management Consulting Systems Integration Managed Services WHITE PAPER DATA DISCOVERY VS ENTERPRISE BUSINESS INTELLIGENCE

Management Consulting Systems Integration Managed Services WHITE PAPER DATA DISCOVERY VS ENTERPRISE BUSINESS INTELLIGENCE Management Consulting Systems Integration Managed Services WHITE PAPER DATA DISCOVERY VS ENTERPRISE BUSINESS INTELLIGENCE INTRODUCTION Over the past several years a new category of Business Intelligence

More information

Low Latency Complex Event Processing on Parallel Hardware

Low Latency Complex Event Processing on Parallel Hardware Low Latency Complex Event Processing on Parallel Hardware Gianpaolo Cugola a, Alessandro Margara a a Dip. di Elettronica e Informazione Politecnico di Milano, Italy surname@elet.polimi.it Abstract Most

More information

COMPUTING SCIENCE. Scalable and Responsive Event Processing in the Cloud. Visalakshmi Suresh, Paul Ezhilchelvan and Paul Watson

COMPUTING SCIENCE. Scalable and Responsive Event Processing in the Cloud. Visalakshmi Suresh, Paul Ezhilchelvan and Paul Watson COMPUTING SCIENCE Scalable and Responsive Event Processing in the Cloud Visalakshmi Suresh, Paul Ezhilchelvan and Paul Watson TECHNICAL REPORT SERIES No CS-TR-1251 June 2011 TECHNICAL REPORT SERIES No

More information

Dynamic M2M Event Processing Complex Event Processing and OSGi on Java Embedded

Dynamic M2M Event Processing Complex Event Processing and OSGi on Java Embedded Dynamic M2M Event Processing Complex Event Processing and OSGi on Java Embedded Oleg Kostukovsky - Master Principal Sales Consultant Walt Bowers - Hitachi CTA Chief Architect 1 2 1. The Vs of Big Data

More information

Data Stream Management System for Moving Sensor Object Data

Data Stream Management System for Moving Sensor Object Data SERBIAN JOURNAL OF ELECTRICAL ENGINEERING Vol. 12, No. 1, February 2015, 117-127 UDC: 004.422.635.5 DOI: 10.2298/SJEE1501117J Data Stream Management System for Moving Sensor Object Data Željko Jovanović

More information

See the wood for the trees

See the wood for the trees See the wood for the trees Dr. Harald Schöning Head of Research The world is becoming digital socienty government economy Digital Society Digital Government Digital Enterprise 2 Data is Getting Bigger

More information

Supporting Views in Data Stream Management Systems

Supporting Views in Data Stream Management Systems 1 Supporting Views in Data Stream Management Systems THANAA M. GHANEM University of St. Thomas AHMED K. ELMAGARMID Purdue University PER-ÅKE LARSON Microsoft Research and WALID G. AREF Purdue University

More information

An XML Framework for Integrating Continuous Queries, Composite Event Detection, and Database Condition Monitoring for Multiple Data Streams

An XML Framework for Integrating Continuous Queries, Composite Event Detection, and Database Condition Monitoring for Multiple Data Streams An XML Framework for Integrating Continuous Queries, Composite Event Detection, and Database Condition Monitoring for Multiple Data Streams Susan D. Urban 1, Suzanne W. Dietrich 1, 2, and Yi Chen 1 Arizona

More information

The Synergy of SOA, Event-Driven Architecture (EDA), and Complex Event Processing (CEP)

The Synergy of SOA, Event-Driven Architecture (EDA), and Complex Event Processing (CEP) The Synergy of SOA, Event-Driven Architecture (EDA), and Complex Event Processing (CEP) Gerhard Bayer Senior Consultant International Systems Group, Inc. gbayer@isg-inc.com http://www.isg-inc.com Table

More information

Addressing Self-Management in Cloud Platforms: a Semantic Sensor Web Approach

Addressing Self-Management in Cloud Platforms: a Semantic Sensor Web Approach Addressing Self-Management in Cloud Platforms: a Semantic Sensor Web Approach Rustem Dautov Iraklis Paraskakis Dimitrios Kourtesis South-East European Research Centre International Faculty, The University

More information

Toward Effective Big Data Analysis in Continuous Auditing. By Juan Zhang, Xiongsheng Yang, and Deniz Appelbaum

Toward Effective Big Data Analysis in Continuous Auditing. By Juan Zhang, Xiongsheng Yang, and Deniz Appelbaum Toward Effective Big Data Analysis in Continuous Auditing By Juan Zhang, Xiongsheng Yang, and Deniz Appelbaum Introduction New sources: emails, phone calls, click stream traffic, social media, news media,

More information

Kai Wähner. The Next-Generation BPM for a Big Data World: Intelligent Business Process Management Suites (ibpms)

Kai Wähner. The Next-Generation BPM for a Big Data World: Intelligent Business Process Management Suites (ibpms) The Next-Generation BPM for a Big Data World: Intelligent Business Process Management Suites (ibpms) Kai Wähner kontakt@kai-waehner.de @KaiWaehner www.kai-waehner.de Xing / LinkedIn Please connect! Kai

More information

ECS 165A: Introduction to Database Systems

ECS 165A: Introduction to Database Systems ECS 165A: Introduction to Database Systems Todd J. Green based on material and slides by Michael Gertz and Bertram Ludäscher Winter 2011 Dept. of Computer Science UC Davis ECS-165A WQ 11 1 1. Introduction

More information

Inferring Fine-Grained Data Provenance in Stream Data Processing: Reduced Storage Cost, High Accuracy

Inferring Fine-Grained Data Provenance in Stream Data Processing: Reduced Storage Cost, High Accuracy Inferring Fine-Grained Data Provenance in Stream Data Processing: Reduced Storage Cost, High Accuracy Mohammad Rezwanul Huq, Andreas Wombacher, and Peter M.G. Apers University of Twente, 7500 AE Enschede,

More information

Big Data Driven Knowledge Discovery for Autonomic Future Internet

Big Data Driven Knowledge Discovery for Autonomic Future Internet Big Data Driven Knowledge Discovery for Autonomic Future Internet Professor Geyong Min Chair in High Performance Computing and Networking Department of Mathematics and Computer Science College of Engineering,

More information

Data Integration and Exchange. L. Libkin 1 Data Integration and Exchange

Data Integration and Exchange. L. Libkin 1 Data Integration and Exchange Data Integration and Exchange L. Libkin 1 Data Integration and Exchange Traditional approach to databases A single large repository of data. Database administrator in charge of access to data. Users interact

More information

INTERNATIONAL MASTER IN BUSINESS ANALYTICS AND BIG DATA

INTERNATIONAL MASTER IN BUSINESS ANALYTICS AND BIG DATA POLITECNICO DI MILANO GRADUATE SCHOOL OF BUSINESS BABD INTERNATIONAL MASTER IN BUSINESS ANALYTICS AND BIG DATA Courses Description A JOINT PROGRAM WITH POLITECNICO DI MILANO SCHOOL OF MANAGEMENT PRE-COURSES

More information

Better Decision Making

Better Decision Making Better Decision Making Big Data Analytics Webinar, November 2013 Dr. Wolfgang Martin Analyst and Member of the Boulder BI Brain Trust Better Decision Making Process Oriented Businesses. Decision Making:

More information

Data Quality in Information Integration and Business Intelligence

Data Quality in Information Integration and Business Intelligence Data Quality in Information Integration and Business Intelligence Leopoldo Bertossi Carleton University School of Computer Science Ottawa, Canada : Faculty Fellow of the IBM Center for Advanced Studies

More information

Data Stream Management System

Data Stream Management System Case Study of CSG712 Data Stream Management System Jian Wen Spring 2008 Northeastern University Outline Traditional DBMS v.s. Data Stream Management System First-generation: Aurora Run-time architecture

More information

THE USE OF STRONG AUTHENTICATION IN CONJUNCTION WITH IDAAS, IDENTITY AS A SERVICE FOR IDENTITY MANAGEMENT. Donald Malloy May 14 th, 2014

THE USE OF STRONG AUTHENTICATION IN CONJUNCTION WITH IDAAS, IDENTITY AS A SERVICE FOR IDENTITY MANAGEMENT. Donald Malloy May 14 th, 2014 THE USE OF STRONG AUTHENTICATION IN CONJUNCTION WITH IDAAS, IDENTITY AS A SERVICE FOR IDENTITY MANAGEMENT Donald Malloy May 14 th, 2014 Agenda What are some Technology trends? How do you define IDaaS?

More information

Chapter 7, System Design Architecture Organization. Construction. Software

Chapter 7, System Design Architecture Organization. Construction. Software Chapter 7, System Design Architecture Organization Object-Oriented Software Construction Armin B. Cremers, Tobias Rho, Daniel Speicher & Holger Mügge (based on Bruegge & Dutoit) Overview Where are we right

More information

Complexity and Scalability in Semantic Graph Analysis Semantic Days 2013

Complexity and Scalability in Semantic Graph Analysis Semantic Days 2013 Complexity and Scalability in Semantic Graph Analysis Semantic Days 2013 James Maltby, Ph.D 1 Outline of Presentation Semantic Graph Analytics Database Architectures In-memory Semantic Database Formulation

More information

OBDA for Temporal Querying and Streams

OBDA for Temporal Querying and Streams OBDA for Temporal Querying and Streams Christian Neuenstadt, Ralf Möller, and Özgür L. Özçep Institute of Information Systems (Ifis) University of Lübeck Lübeck, Germany {moeller,neuenstadt,oezcep}@ifis.uni-luebeck.de

More information

Event-based middleware services

Event-based middleware services 3 Event-based middleware services The term event service has different definitions. In general, an event service connects producers of information and interested consumers. The service acquires events

More information

2 Types of Complex Event Processing

2 Types of Complex Event Processing Complex Event Processing (CEP) Michael Eckert and François Bry Institut für Informatik, Ludwig-Maximilians-Universität München michael.eckert@pms.ifi.lmu.de, http://www.pms.ifi.lmu.de This is an English

More information

Big Data Management Assessed Coursework Two Big Data vs Semantic Web F21BD

Big Data Management Assessed Coursework Two Big Data vs Semantic Web F21BD Big Data Management Assessed Coursework Two Big Data vs Semantic Web F21BD Boris Mocialov (H00180016) MSc Software Engineering Heriot-Watt University, Edinburgh April 5, 2015 1 1 Introduction The purpose

More information

A Practical Approach to Process Streaming Data using Graph Database

A Practical Approach to Process Streaming Data using Graph Database A Practical Approach to Process Streaming Data using Graph Database Mukul Sharma Research Scholar Department of Computer Science & Engineering SBCET, Jaipur, Rajasthan, India ABSTRACT In today s information

More information

An Approach for Knowledge-Based IT Management of Air Traffic Control Systems

An Approach for Knowledge-Based IT Management of Air Traffic Control Systems An Approach for Knowledge-Based IT Management of Air Traffic Control Systems Fabian Meyer, Reinhold Kroeger RheinMain University of Applied Sciences D-65195 Wiesbaden, Germany {firstname.lastname}@hs-rm.de

More information

A Framework for Ontology-based Context Base Management System

A Framework for Ontology-based Context Base Management System Association for Information Systems AIS Electronic Library (AISeL) PACIS 2005 Proceedings Pacific Asia Conference on Information Systems (PACIS) 12-31-2005 A Framework for Ontology-based Context Base Management

More information

Are You Ready for Big Data?

Are You Ready for Big Data? Are You Ready for Big Data? Jim Gallo National Director, Business Analytics February 11, 2013 Agenda What is Big Data? How do you leverage Big Data in your company? How do you prepare for a Big Data initiative?

More information

Why big data? Lessons from a Decade+ Experiment in Big Data

Why big data? Lessons from a Decade+ Experiment in Big Data Why big data? Lessons from a Decade+ Experiment in Big Data David Belanger PhD Senior Research Fellow Stevens Institute of Technology dbelange@stevens.edu 1 What Does Big Look Like? 7 Image Source Page:

More information

CSE 544 Principles of Database Management Systems. Magdalena Balazinska (magda) Winter 2009 Lecture 1 - Class Introduction

CSE 544 Principles of Database Management Systems. Magdalena Balazinska (magda) Winter 2009 Lecture 1 - Class Introduction CSE 544 Principles of Database Management Systems Magdalena Balazinska (magda) Winter 2009 Lecture 1 - Class Introduction Outline Introductions Class overview What is the point of a db management system

More information

Big Data: Opportunities and Challenges. Raja Chiky raja.chiky@isep.fr

Big Data: Opportunities and Challenges. Raja Chiky raja.chiky@isep.fr Big Data: Opportunities and Challenges Raja Chiky raja.chiky@isep.fr OUTLINE 3 About me What is Big Data? Evolution of Business Intelligence Big Data Opportunities Big Data challenges Conclusion About

More information

Enabling Self Organising Logistics on the Web of Things

Enabling Self Organising Logistics on the Web of Things Enabling Self Organising Logistics on the Web of Things Monika Solanki, Laura Daniele, Christopher Brewster Aston Business School, Aston University, Birmingham, UK TNO Netherlands Organization for Applied

More information

Data collection architecture for Big Data

Data collection architecture for Big Data Data collection architecture for Big Data a framework for a research agenda (Research in progress - ERP Sense Making of Big Data) Wout Hofman, May 2015, BDEI workshop 2 Big Data succes stories bias our

More information

Converged, Real-time Analytics Enabling Faster Decision Making and New Business Opportunities

Converged, Real-time Analytics Enabling Faster Decision Making and New Business Opportunities Technology Insight Paper Converged, Real-time Analytics Enabling Faster Decision Making and New Business Opportunities By John Webster February 2015 Enabling you to make the best technology decisions Enabling

More information

Requirements for Context-dependent Mobile Access to Information Services

Requirements for Context-dependent Mobile Access to Information Services Requirements for Context-dependent Mobile Access to Information Services Augusto Celentano Università Ca Foscari di Venezia Fabio Schreiber, Letizia Tanca Politecnico di Milano MIS 2004, College Park,

More information

Big Data and Semantic Web in Manufacturing. Nitesh Khilwani, PhD Chief Engineer, Samsung Research Institute Noida, India

Big Data and Semantic Web in Manufacturing. Nitesh Khilwani, PhD Chief Engineer, Samsung Research Institute Noida, India Big Data and Semantic Web in Manufacturing Nitesh Khilwani, PhD Chief Engineer, Samsung Research Institute Noida, India Outline Big data in Manufacturing Big data Analytics Semantic web technologies Case

More information

Integrating and Processing Events from Heterogeneous Data Sources

Integrating and Processing Events from Heterogeneous Data Sources Integrating and Processing Events from Heterogeneous Data Sources Ivo Koga 1, Claudia Bauzer Medeiros 1 1 Institute of Computing State University of Campinas (UNICAMP) 13083-852 Campinas SP Brazil {koga,cmbm}@ic.unicamp.br

More information

Sensor Information Representation for the Internet of Things

Sensor Information Representation for the Internet of Things Sensor Information Representation for the Internet of Things Jiehan Zhou Jiehan.zhou@ee.oulu.fi University of Oulu, Finland Carleton University, Canada Agenda Internet of Things and Challenges Application

More information

Programming Without a Call Stack: Event-driven Architectures

Programming Without a Call Stack: Event-driven Architectures Gregor Hohpe Google Programming Without a Call Stack: -driven Architectures www.eaipatterns.com Who's Gregor? Distributed systems, enterprise integration, service-oriented architectures MQ, MSMQ, JMS,

More information

Klarna Tech Talk: Mind the Data! Jeff Pollock InfoSphere Information Integration & Governance

Klarna Tech Talk: Mind the Data! Jeff Pollock InfoSphere Information Integration & Governance Klarna Tech Talk: Mind the Data! Jeff Pollock InfoSphere Information Integration & Governance IBM s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice

More information

De la Business Intelligence aux Big Data. Marie- Aude AUFAURE Head of the Business Intelligence team Ecole Centrale Paris. 22/01/14 Séminaire Big Data

De la Business Intelligence aux Big Data. Marie- Aude AUFAURE Head of the Business Intelligence team Ecole Centrale Paris. 22/01/14 Séminaire Big Data De la Business Intelligence aux Big Data Marie- Aude AUFAURE Head of the Business Intelligence team Ecole Centrale Paris 22/01/14 Séminaire Big Data 1 Agenda EvoluHon of Business Intelligence SemanHc Technologies

More information

Scalable Processing and mining of Complex Events for Security-analytics

Scalable Processing and mining of Complex Events for Security-analytics Scalable Processing and mining of Complex Events for Security-analytics Secur IT kickoff 8 sept 2015 Security Information and Event Management Existing tools for security monitoring SIM/SEM (Near) real-time

More information

M2M Communications and Internet of Things for Smart Cities. Soumya Kanti Datta Mobile Communications Dept. Email: Soumya-Kanti.Datta@eurecom.

M2M Communications and Internet of Things for Smart Cities. Soumya Kanti Datta Mobile Communications Dept. Email: Soumya-Kanti.Datta@eurecom. M2M Communications and Internet of Things for Smart Cities Soumya Kanti Datta Mobile Communications Dept. Email: Soumya-Kanti.Datta@eurecom.fr WHAT IS EURECOM A graduate school & research centre in communication

More information

Hemant Kulkarni MOBILE BI HYPE OR REAL NEED?

Hemant Kulkarni MOBILE BI HYPE OR REAL NEED? Hemant Kulkarni MOBILE BI HYPE OR REAL NEED? AGENDA WHY PEOPLE STILL FILL ITS HYPE? WHAT GARTNER HYPE CYCLE FOR EMERGING TRENDS SAYS? WHAT IS THE REALITY? MOBILE BI EVOLUTION MOBILE BI IS NOT JUST BI ON

More information

HOW TO DO A SMART DATA PROJECT

HOW TO DO A SMART DATA PROJECT April 2014 Smart Data Strategies HOW TO DO A SMART DATA PROJECT Guideline www.altiliagroup.com Summary ALTILIA s approach to Smart Data PROJECTS 3 1. BUSINESS USE CASE DEFINITION 4 2. PROJECT PLANNING

More information

Deductive and Inductive Stream Reasoning for Semantic Social Media Analytics

Deductive and Inductive Stream Reasoning for Semantic Social Media Analytics DeductiveandInductiveStreamReasoning forsemanticsocialmediaanalytics DavideBarbieri 1,DanieleBraga 1,StefanoCeri 1,EmanueleDellaValle 1,YiHuang 2,VolkerTresp 2, AchimRettinger 3,andHendrikWermser 4 1 Dip.diElettronicaeInformazione,PolitecnicodiMilano,Milano,Italy

More information

Oracle SOA Suite: The Evaluation from 10g to 11g

Oracle SOA Suite: The Evaluation from 10g to 11g KATTA Durga Reddy TATA Consultancy Services. Oracle SOA Suite: The Evaluation from 10g to 11g Introduction Oracle SOA Suite is an essential middleware layer of Oracle Fusion Middleware. It provides a complete

More information

Transforming the Telecoms Business using Big Data and Analytics

Transforming the Telecoms Business using Big Data and Analytics Transforming the Telecoms Business using Big Data and Analytics Event: ICT Forum for HR Professionals Venue: Meikles Hotel, Harare, Zimbabwe Date: 19 th 21 st August 2015 AFRALTI 1 Objectives Describe

More information

TopBraid Insight for Life Sciences

TopBraid Insight for Life Sciences TopBraid Insight for Life Sciences In the Life Sciences industries, making critical business decisions depends on having relevant information. However, queries often have to span multiple sources of information.

More information

Data Stream Management and Complex Event Processing in Esper. INF5100, Autumn 2010 Jarle Søberg

Data Stream Management and Complex Event Processing in Esper. INF5100, Autumn 2010 Jarle Søberg Data Stream Management and Complex Event Processing in Esper INF5100, Autumn 2010 Jarle Søberg Outline Overview of Esper DSMS and CEP concepts in Esper Examples taken from the documentation A lot of possibilities

More information

K@ A collaborative platform for knowledge management

K@ A collaborative platform for knowledge management White Paper K@ A collaborative platform for knowledge management Quinary SpA www.quinary.com via Pietrasanta 14 20141 Milano Italia t +39 02 3090 1500 f +39 02 3090 1501 Copyright 2004 Quinary SpA Index

More information

Ricerca e classificazione documentale su basi dati per gli studi professionali

Ricerca e classificazione documentale su basi dati per gli studi professionali Ricerca e classificazione documentale su basi dati per gli studi professionali The business case: Scarsi & Co. fabio.scarsi@scarsieco.it Alberto.Ciaramella@intellisemantic.com 1 Business Scenario: fierce

More information

Management of Human Resource Information Using Streaming Model

Management of Human Resource Information Using Streaming Model , pp.75-80 http://dx.doi.org/10.14257/astl.2014.45.15 Management of Human Resource Information Using Streaming Model Chen Wei Chongqing University of Posts and Telecommunications, Chongqing 400065, China

More information

Uncertain Data Management for Sensor Networks

Uncertain Data Management for Sensor Networks Uncertain Data Management for Sensor Networks Amol Deshpande, University of Maryland (joint work w/ Bhargav Kanagal, Prithviraj Sen, Lise Getoor, Sam Madden) Motivation: Sensor Networks Unprecedented,

More information

Querying RDF Streams with C-SPARQL

Querying RDF Streams with C-SPARQL Querying RDF Streams with C-SPARQL Davide Francesco Barbieri Daniele Braga Stefano Ceri Emanuele Della Valle Michael Grossniklaus Politecnico di Milano, Dipartimento di Elettronica e Informazione Piazza

More information

CSE 544 Principles of Database Management Systems. Magdalena Balazinska (magda) Fall 2007 Lecture 1 - Class Introduction

CSE 544 Principles of Database Management Systems. Magdalena Balazinska (magda) Fall 2007 Lecture 1 - Class Introduction CSE 544 Principles of Database Management Systems Magdalena Balazinska (magda) Fall 2007 Lecture 1 - Class Introduction Outline Introductions Class overview What is the point of a db management system

More information

Sensor Event Processing on Grid

Sensor Event Processing on Grid Sensor Event Processing on Grid Eui-Nam Huh Dept. of Computer Engineering Kyung Hee University #1 Seochon Kiheung, Yoingin, Kyunggi-Do, Korea johnhuh@khu.ac.kr Abstract. Wireless sensor networks are increasingly

More information

Combining SAWSDL, OWL DL and UDDI for Semantically Enhanced Web Service Discovery

Combining SAWSDL, OWL DL and UDDI for Semantically Enhanced Web Service Discovery Combining SAWSDL, OWL DL and UDDI for Semantically Enhanced Web Service Discovery Dimitrios Kourtesis, Iraklis Paraskakis SEERC South East European Research Centre, Greece Research centre of the University

More information

Real-Time Integration of Building Energy Data

Real-Time Integration of Building Energy Data 2014 IEEE International Congress on Big Data Real-Time Integration of Building Energy Data Diogo Anjos INESC-ID Lisboa Instituto Superior Técnico (IST) Lisboa, Portugal diogo.silva.anjos@tecnico.ulisboa.pt

More information

Iotivity Programmer s Guide Soft Sensor Manager for Android

Iotivity Programmer s Guide Soft Sensor Manager for Android Iotivity Programmer s Guide Soft Sensor Manager for Android 1 CONTENTS 2 Introduction... 3 3 Terminology... 3 3.1 Physical Sensor Application... 3 3.2 Soft Sensor (= Logical Sensor, Virtual Sensor)...

More information

Smart Grid - Big Data visualized in GIS

Smart Grid - Big Data visualized in GIS Smart Grid - Big Data visualized in GIS Maximizing Data Value with GIS and CIM in a Smart Grid World ESRI UC, San Diego, July 15 th 2014 Presenters from DONG Energy Signe Bramming Andersen, sigba@dongenergy.dk

More information

Semantic Business Analytics in Industrial Facilities a Case Study

Semantic Business Analytics in Industrial Facilities a Case Study Semantic Business Analytics in Industrial Facilities a Case Study Jürgen Angele, Eddie Mönch ontoprise GmbH An der RaumFabrik 29 76227 Karlsruhe angele@ontoprise.de eddie.moench@ontoprise.de Abstract:

More information

A Comparison of Database Query Languages: SQL, SPARQL, CQL, DMX

A Comparison of Database Query Languages: SQL, SPARQL, CQL, DMX ISSN: 2393-8528 Contents lists available at www.ijicse.in International Journal of Innovative Computer Science & Engineering Volume 3 Issue 2; March-April-2016; Page No. 09-13 A Comparison of Database

More information

CARRIOTS TECHNICAL PRESENTATION

CARRIOTS TECHNICAL PRESENTATION CARRIOTS TECHNICAL PRESENTATION Alvaro Everlet, CTO alvaro.everlet@carriots.com @aeverlet Oct 2013 CARRIOTS TECHNICAL PRESENTATION 1. WHAT IS CARRIOTS 2. BUILDING AN IOT PROJECT 3. DEVICES 4. PLATFORM

More information

Find the Information That Matters. Visualize Your Data, Your Way. Scalable, Flexible, Global Enterprise Ready

Find the Information That Matters. Visualize Your Data, Your Way. Scalable, Flexible, Global Enterprise Ready Real-Time IoT Platform Solutions for Wireless Sensor Networks Find the Information That Matters ViZix is a scalable, secure, high-capacity platform for Internet of Things (IoT) business solutions that

More information

Complex Event Processing with T-REX

Complex Event Processing with T-REX Complex Event Processing with T-REX Gianpaolo Cugola a, Alessandro Margara a a Dip. di Elettronica e Informazione Politecnico di Milano, Italy Abstract Several application domains involve detecting complex

More information

Oracle Data Integrator: Administration and Development

Oracle Data Integrator: Administration and Development Oracle Data Integrator: Administration and Development What you will learn: In this course you will get an overview of the Active Integration Platform Architecture, and a complete-walk through of the steps

More information

Taming Big Data Variety with Semantic Graph Databases. Evren Sirin CTO Complexible

Taming Big Data Variety with Semantic Graph Databases. Evren Sirin CTO Complexible Taming Big Data Variety with Semantic Graph Databases Evren Sirin CTO Complexible About Complexible Semantic Tech leader since 2006 (née Clark & Parsia) software, consulting W3C leadership Offices in DC

More information

Introduction. Background

Introduction. Background Predictive Operational Analytics (POA): Customized Solutions for Improving Efficiency and Productivity for Manufacturers using a Predictive Analytics Approach Introduction Preserving assets and improving

More information

Survey of Distributed Stream Processing for Large Stream Sources

Survey of Distributed Stream Processing for Large Stream Sources Survey of Distributed Stream Processing for Large Stream Sources Supun Kamburugamuve For the PhD Qualifying Exam 12-14- 2013 Advisory Committee Prof. Geoffrey Fox Prof. David Leake Prof. Judy Qiu Table

More information

Secure Semantic Web Service Using SAML

Secure Semantic Web Service Using SAML Secure Semantic Web Service Using SAML JOO-YOUNG LEE and KI-YOUNG MOON Information Security Department Electronics and Telecommunications Research Institute 161 Gajeong-dong, Yuseong-gu, Daejeon KOREA

More information

DBMS / Business Intelligence, SQL Server

DBMS / Business Intelligence, SQL Server DBMS / Business Intelligence, SQL Server Orsys, with 30 years of experience, is providing high quality, independant State of the Art seminars and hands-on courses corresponding to the needs of IT professionals.

More information

Business-Driven Software Engineering Lecture 3 Foundations of Processes

Business-Driven Software Engineering Lecture 3 Foundations of Processes Business-Driven Software Engineering Lecture 3 Foundations of Processes Jochen Küster jku@zurich.ibm.com Agenda Introduction and Background Process Modeling Foundations Activities and Process Models Summary

More information

Big Data & Analytics: Your concise guide (note the irony) Wednesday 27th November 2013

Big Data & Analytics: Your concise guide (note the irony) Wednesday 27th November 2013 Big Data & Analytics: Your concise guide (note the irony) Wednesday 27th November 2013 Housekeeping 1. Any questions coming out of today s presentation can be discussed in the bar this evening 2. OCF is

More information

Evolution of Taxonomies and A Supply Chain of Things. Daniel E. O Leary University of Southern California 2012

Evolution of Taxonomies and A Supply Chain of Things. Daniel E. O Leary University of Southern California 2012 Evolution of Taxonomies and A Supply Chain of Things Daniel E. O Leary University of Southern California 2012 0. Research about Research Before I drill down on taxonomies and ontologies I would like to

More information

Data-intensive HPC: opportunities and challenges. Patrick Valduriez

Data-intensive HPC: opportunities and challenges. Patrick Valduriez Data-intensive HPC: opportunities and challenges Patrick Valduriez Big Data Landscape Multi-$billion market! Big data = Hadoop = MapReduce? No one-size-fits-all solution: SQL, NoSQL, MapReduce, No standard,

More information

Report on the Dagstuhl Seminar Data Quality on the Web

Report on the Dagstuhl Seminar Data Quality on the Web Report on the Dagstuhl Seminar Data Quality on the Web Michael Gertz M. Tamer Özsu Gunter Saake Kai-Uwe Sattler U of California at Davis, U.S.A. U of Waterloo, Canada U of Magdeburg, Germany TU Ilmenau,

More information

Hadoop Beyond Hype: Complex Adaptive Systems Conference Nov 16, 2012. Viswa Sharma Solutions Architect Tata Consultancy Services

Hadoop Beyond Hype: Complex Adaptive Systems Conference Nov 16, 2012. Viswa Sharma Solutions Architect Tata Consultancy Services Hadoop Beyond Hype: Complex Adaptive Systems Conference Nov 16, 2012 Viswa Sharma Solutions Architect Tata Consultancy Services 1 Agenda What is Hadoop Why Hadoop? The Net Generation is here Sizing the

More information

Capital Market Day 2015

Capital Market Day 2015 Capital Market Day 2015 Digital Business Platform & Product Roadmap Dr. Wolfram Jost Chief Technology Officer February 4, 2015 1 For Internal use only. Market Application infrastructure and middleware

More information

JOURNAL OF OBJECT TECHNOLOGY

JOURNAL OF OBJECT TECHNOLOGY JOURNAL OF OBJECT TECHNOLOGY Online at www.jot.fm. Published by ETH Zurich, Chair of Software Engineering JOT, 2008 Vol. 7, No. 8, November-December 2008 What s Your Information Agenda? Mahesh H. Dodani,

More information

IT outsourcing and cloud computing: definitions, models, and emerging trends. Index. From IT outsourcing to cloud computing: a

IT outsourcing and cloud computing: definitions, models, and emerging trends. Index. From IT outsourcing to cloud computing: a Politecnico di Torino IT outsourcing and cloud computing: definitions, models, and emerging trends Paolo Neirotti, Ph.D Politecnico di Torino DISPEA I.A.E. Grenoble Visiting Professor Paolo.neirotti@polito.it

More information

Oracle Database 12c and the Future of Data Warehousing in the Era of Big Data

Oracle Database 12c and the Future of Data Warehousing in the Era of Big Data Oracle Database 12c and the Future of Data Warehousing in the Era of Big Data George Lumpkin Data Warehousing Neil Mendelson Big Data & Advanced AnalyEcs Vice Presidents Server Technologies September 29,

More information

Using reporting and data mining techniques to improve knowledge of subscribers; applications to customer profiling and fraud management

Using reporting and data mining techniques to improve knowledge of subscribers; applications to customer profiling and fraud management Using reporting and data mining techniques to improve knowledge of subscribers; applications to customer profiling and fraud management Paper Jean-Louis Amat Abstract One of the main issues of operators

More information