Managing Data in Motion

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "Managing Data in Motion"

Transcription

1 Managing Data in Motion Data Integration Best Practice Techniques and Technologies April Reeve ELSEVIER AMSTERDAM BOSTON HEIDELBERG LONDON NEW YORK OXFORD PARIS SAN DIEGO SAN FRANCISCO SINGAPORE SYDNEY TOKYO Morgan Kaufmann is an imprint of Elsevier M<

2 Contents Foreword Acknowledgements Biography Introduction xv xvii xix xxi PART 1 INTRODUCTION TO DATA INTEGRATION Chapter 1 The Importance of Data Integration з The natural complexity of data interfaces 3 The rise of purchased vendor packages 4 Key enablement of big data and virtualization 5 Chapter 2 What Is Data Integration? 7 Data in motion 7 Integrating into a common format transforming data 7 Migrating data from one system to another 8 Moving data around the organization 9 Pulling information from unstructured data 11 Moving process to data 12 Chapter 3 Types and Complexity of Data Integration 15 The differences and similarities in managing data in motion and persistent data 15 Batch data integration 16 Real-time data integration 16 Big data integration 17 Data virtualization 17 Chapter 4 The Process of Data Integration Development 19 The data integration development life cycle 19 Inclusion of business knowledge and expertise 20 PART 2 BATCH DATA INTEGRATION Chapter 5 Introduction to Batch Data Integration 25 What is batch data integration? 25 Batch data integration life cycle 26

3 viii Contents Chapter 6 Extract, Transform, and Load 29 WhatisETL? 29 Profiling 30 Extract 30 Staging 31 Access layers 32 Transform 33 Simple mapping 33 Lookups 33 Aggregation and normalization 33 Calculation 34 Load 34 Chapter 7 Data Warehousing 37 What is data warehousing? 37 Layers in an enterprise data warehouse architecture 38 Operational application layer 38 External data 38 Data staging areas coming into a data warehouse 39 Data warehouse data structure 40 Staging from data warehouse to data mart or business intelligence 40 Business Intelligence Layer 40 Types of data to load in a data warehouse 41 Master data in a data warehouse 41 Balance and snapshot data in a data warehouse 42 Transactional data in a data warehouse 43 Events 43 Reconciliation 43 Interview with an expert: Krish Krishnan on data warehousing and data integration 44 Chapter 8 Data Conversion 51 What is data conversion? 51 Data conversion life cycle 51 Data conversion analysis 52 Best practice data loading 52 Improving source data quality 53

4 Contents ix Mapping to target 53 Configuration data 54 Testing and dependencies 55 Private data 55 Proving 56 Environments 56 Chapter 9 Data Archiving 59 What is data archiving? 59 Selecting data to archive 60 Can the archived data be retrieved? 60 Conforming data structures in the archiving environment 61 Flexible data structures 61 Interview with an expert: John Anderson on data archiving and data integration 62 Chapter 10 Batch Data Integration Architecture and Metadata 67 What is batch data integration architecture? 67 Profiling tool 67 Modeling tool 68 Metadata repository 69 Data movement 69 Transformation 70 Scheduling 71 Interview with an expert: Adrienne Tannenbaum on metadata and data integration 73 PART 3 REAL TIME DATA INTEGRATION Chapter 11 Introduction to Real-Time Data Integration 77 Why real-time data integration? 77 Why two sets of technologies? 78 Chapter 12 Data Integration Patterns 79 Interaction patterns 79 Loose coupling 79 Hub and spoke 80 Synchronous and asynchronous interaction 83

5 x Contents Request and reply 83 Publish and subscribe 84 Two-phase commit 84 Integrating interaction types 85 Chapter 13 Core Real-Time Data Integration Technologies 87 Confusing terminology 87 Enterprise service bus (ESB) 88 Interview with an expert: David S. Linthicum on ESB and data integration 89 Service-oriented architecture (SOA) 90 Extensible markup language (XML) 92 Interview with an expert: M. David Allen on XML and data integration 92 Data replication and change data capture 95 Enterprise application integration (EAI) 97 Enterprise information integration (Ell) 97 Chapter 14 Data Integration Modeling 99 Canonical modeling 99 Interview with an expert: Dagna Gaythorpe on canonical modeling and data integration 100 Message modeling 103 Chapter 15 Master Data Management 105 Introduction to master data management 105 Reasons for a master data management solution 105 Purchased packages and master data 106 Reference data 107 Masters and slaves 107 External data 110 Master data management functionality 110 Types of master data management solutions registry and data hub Ill Chapter 16 Data Warehousing with Real-Time Updates 113 Corporate information factory 113 Operational data store 113

6 Contents xi Master data moving to the data warehouse 116 Interview with an expert: Krish Krishnan on real-time data warehousing updates 116 Chapter 17 Real-Time Data Integration Architecture and Metadata 119 What is real-time data integration metadata? 119 Modeling 120 Profiling 120 Metadata repository 120 Enterprise service bus data transformation and orchestration 121 Technical mediation 122 Business content 122 Data movement and middleware 123 External interaction 123 PART 4 BIG, CLOUD, VIRTUAL DATA Chapter 18 Introduction to Big Data Integration 127 Data integration and unstructured data 127 Big data, cloud data, and data virtualization 127 Chapter 19 Cloud Architecture and Data Integration 129 Why is data integration important in the cloud? 129 Public cloud 129 Cloud security 130 Cloud latency 131 Cloud redundancy 132 Chapter 20 Data Virtualization 135 A technology whose time has come 135 Business uses of data virtualization 137 Business intelligence solutions 137 Integrating different types of data 137 Quickly add or prototype adding data to a data warehouse 137 Present physically disparate data together 138 Leverage various data and models triggering transactions 138

7 xii Contents Data virtualization architecture 138 Sources and adapters 138 Mappings and models and views 138 Transformation and presentation 139 Chapter 21 Big Data Integration 141 What is big data? 142 Big data dimension volume 142 Massive parallel processing moving process to data 142 Hadoop and MapReduce 143 Integrating with external data 144 Visualization 144 Big data dimension variety 145 Types of data 145 Integrating different types of data 145 Interview with an expert: William McKnight on Hadoop and data integration 145 Big data dimension velocity 146 Streaming data 147 Sensor and GPS data 147 Social media data 147 Traditional big data use cases 147 More big data use cases 148 Health care 148 Logistics 148 National security 149 Leveraging the power of big data real-time decision support 149 Triggering action 149 Speed of data retrieval from memory versus disk 150 From data analytics to models, from streaming data to decisions 150 Big data architecture 151 Operational systems and data sources 151 Intermediate data hubs 151 Business intelligence tools 152 Data virtualization server 153

8 Contents xiii Batch and real-time data integration tools 153 Analytic sandbox 153 Risk response systems/recommendation engines 153 Interview with an expert: John Haddad on Big Data and data integration 154 Chapter 22 Conclusion to Managing Data in Motion 157 Data integration architecture 157 Why data integration architecture? 157 Data integration life cycle and expertise 158 Security and privacy 158 Data integration engines 160 Operational continuity 160 ETL engine 160 Enterprise service bus 161 Data virtualization server 161 Data movement 162 Data integration hubs 162 Master data 163 Data warehouse and operational data store 164 Enterprise content management 164 Data archive 164 Metadata management 164 Data discovery 165 Data profiling 165 Data modeling 165 Data flow modeling 165 Metadata repository 166 The end 166 References 167 Index 169

Managing Data in Motion

Managing Data in Motion Managing Data in Motion This page intentionally left blank Managing Data in Motion Data Integration Best Practice Techniques and Technologies April Reeve AMSTERDAM BOSTON HEIDELBERG LONDON NEW YORK OXFORD

More information

Data Warehousing in the Age of Big Data

Data Warehousing in the Age of Big Data Data Warehousing in the Age of Big Data Krish Krishnan AMSTERDAM BOSTON HEIDELBERG LONDON NEW YORK OXFORD * PARIS SAN DIEGO SAN FRANCISCO SINGAPORE SYDNEY TOKYO Morgan Kaufmann is an imprint of Elsevier

More information

Master Data Management

Master Data Management Master Data Management David Loshin AMSTERDAM BOSTON HEIDELBERG LONDON NEW YORK OXFORD PARIS SAN DIEGO Ик^И V^ SAN FRANCISCO SINGAPORE SYDNEY TOKYO W*m k^ MORGAN KAUFMANN PUBLISHERS IS AN IMPRINT OF ELSEVIER

More information

AMSTERDAM BOSTON HEIDELBERG LONDON NEW YORK OXFORD PARIS SAN DIEGO SAN FRANCISCO SINGAPORE SYDNEY TOKYO

AMSTERDAM BOSTON HEIDELBERG LONDON NEW YORK OXFORD PARIS SAN DIEGO SAN FRANCISCO SINGAPORE SYDNEY TOKYO DW2.0 The Architecture for the Next Generation of Data Warehousing W. H. Inmon Forest Rim Technology Derek Strauss Gavroshe Genia Neushloss Gavroshe AMSTERDAM BOSTON HEIDELBERG LONDON NEW YORK OXFORD PARIS

More information

Data Model ing Essentials

Data Model ing Essentials Data Model ing Essentials Third Edition Graeme C. Simsion and Graham C. Witt MORGAN KAUFMANN PUBLISHERS AN IMPRINT OF ELSEVIER AMSTERDAM BOSTON LONDON NEW YORK OXFORD PARIS SAN DIEGO SAN FRANCISCO SINGAPORE

More information

Securing the Cloud. Cloud Computer Security Techniques and Tactics. Vic (J.R.) Winkler. Technical Editor Bill Meine ELSEVIER

Securing the Cloud. Cloud Computer Security Techniques and Tactics. Vic (J.R.) Winkler. Technical Editor Bill Meine ELSEVIER Securing the Cloud Cloud Computer Security Techniques and Tactics Vic (J.R.) Winkler Technical Editor Bill Meine ELSEVIER AMSTERDAM BOSTON HEIDELBERG LONDON NEW YORK OXFORD PARIS SAN DIEGO SAN FRANCISCO

More information

Big Data Analytics From Strategie Planning to Enterprise Integration with Tools, Techniques, NoSQL, and Graph

Big Data Analytics From Strategie Planning to Enterprise Integration with Tools, Techniques, NoSQL, and Graph Big Data Analytics From Strategie Planning to Enterprise Integration with Tools, Techniques, NoSQL, and Graph David Loshin ELSEVIER AMSTERDAM BOSTON HEIDELBERG LONDON NEW YORK OXFORD PARIS SAN DIEGO SAN

More information

IMPROVEMENT THE PRACTITIONER'S GUIDE TO DATA QUALITY DAVID LOSHIN

IMPROVEMENT THE PRACTITIONER'S GUIDE TO DATA QUALITY DAVID LOSHIN i I I I THE PRACTITIONER'S GUIDE TO DATA QUALITY IMPROVEMENT DAVID LOSHIN ELSEVIER AMSTERDAM BOSTON HEIDELBERG LONDON NEW YORK OXFORD PARIS SAN DIEGO SAN FRANCISCO SINGAPORE SYDNEY TOKYO Morgan Kaufmann

More information

Architectures, and. Service-Oriented. Cloud Computing. Web Services, The Savvy Manager's Guide. Second Edition. Douglas K. Barry. with.

Architectures, and. Service-Oriented. Cloud Computing. Web Services, The Savvy Manager's Guide. Second Edition. Douglas K. Barry. with. Web Services, Service-Oriented Architectures, and Cloud Computing The Savvy Manager's Guide Second Edition Douglas K. Barry with David Dick ELSEVIER AMSTERDAM BOSTON HEIDELBERG LONDON NEW YORK OXFORD PARIS

More information

Measuring Data Quality for Ongoing Improvement

Measuring Data Quality for Ongoing Improvement Measuring Data Quality for Ongoing Improvement A Data Quality Assessment Framework Laura Sebastian-Coleman ELSEVIER AMSTERDAM BOSTON HEIDELBERG LONDON NEW YORK OXFORD PARIS SAN DIEGO SAN FRANCISCO SINGAPORE

More information

JOURNAL OF OBJECT TECHNOLOGY

JOURNAL OF OBJECT TECHNOLOGY JOURNAL OF OBJECT TECHNOLOGY Online at www.jot.fm. Published by ETH Zurich, Chair of Software Engineering JOT, 2008 Vol. 7, No. 8, November-December 2008 What s Your Information Agenda? Mahesh H. Dodani,

More information

Computing. Federal Cloud. Service Providers. The Definitive Guide for Cloud. Matthew Metheny ELSEVIER. Syngress is NEWYORK OXFORD PARIS SAN DIEGO

Computing. Federal Cloud. Service Providers. The Definitive Guide for Cloud. Matthew Metheny ELSEVIER. Syngress is NEWYORK OXFORD PARIS SAN DIEGO Federal Cloud Computing The Definitive Guide for Cloud Service Providers Matthew Metheny ELSEVIER AMSTERDAM BOSTON HEIDELBERG LONDON NEWYORK OXFORD PARIS SAN DIEGO SAN FRANCISCO SINGAPORE SYDNEY TOKYO

More information

Customer Relationship Management

Customer Relationship Management Customer Relationship Management Concepts and Technologies Second edition Francis Buttle xlloillvlcjx. AMSTERDAM BOSTON HEIDELBERG LONDON NEW YORK OXFORD PARIS SAN DIEGO SAN FRANCISCO SINGAPORE SYDNEY

More information

Cloud Computing. Theory and Practice. Dan C. Marinescu. Morgan Kaufmann is an imprint of Elsevier HEIDELBERG LONDON AMSTERDAM BOSTON

Cloud Computing. Theory and Practice. Dan C. Marinescu. Morgan Kaufmann is an imprint of Elsevier HEIDELBERG LONDON AMSTERDAM BOSTON Cloud Computing Theory and Practice Dan C. Marinescu AMSTERDAM BOSTON HEIDELBERG LONDON NEW YORK OXFORD PARIS SAN DIEGO SAN FRANCISCO SINGAPORE SYDNEY TOKYO M< Morgan Kaufmann is an imprint of Elsevier

More information

Data Integration Checklist

Data Integration Checklist The need for data integration tools exists in every company, small to large. Whether it is extracting data that exists in spreadsheets, packaged applications, databases, sensor networks or social media

More information

Master Data Management. Zahra Mansoori

Master Data Management. Zahra Mansoori Master Data Management Zahra Mansoori 1 1. Preference 2 A critical question arises How do you get from a thousand points of data entry to a single view of the business? We are going to answer this question

More information

Klarna Tech Talk: Mind the Data! Jeff Pollock InfoSphere Information Integration & Governance

Klarna Tech Talk: Mind the Data! Jeff Pollock InfoSphere Information Integration & Governance Klarna Tech Talk: Mind the Data! Jeff Pollock InfoSphere Information Integration & Governance IBM s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice

More information

MDM and Data Warehousing Complement Each Other

MDM and Data Warehousing Complement Each Other Master Management MDM and Warehousing Complement Each Other Greater business value from both 2011 IBM Corporation Executive Summary Master Management (MDM) and Warehousing (DW) complement each other There

More information

Enterprise Information Integration (EII) A Technical Ally of EAI and ETL Author Bipin Chandra Joshi Integration Architect Infosys Technologies Ltd

Enterprise Information Integration (EII) A Technical Ally of EAI and ETL Author Bipin Chandra Joshi Integration Architect Infosys Technologies Ltd Enterprise Information Integration (EII) A Technical Ally of EAI and ETL Author Bipin Chandra Joshi Integration Architect Infosys Technologies Ltd Page 1 of 8 TU1UT TUENTERPRISE TU2UT TUREFERENCESUT TABLE

More information

The Data Access Handbook

The Data Access Handbook The Data Access Handbook Achieving Optimal Database Application Performance and Scalability John Goodson and Robert A. Steward PRENTICE HALL Upper Saddle River, NJ Boston Indianapolis San Francisco New

More information

Emerging Technologies Shaping the Future of Data Warehouses & Business Intelligence

Emerging Technologies Shaping the Future of Data Warehouses & Business Intelligence Emerging Technologies Shaping the Future of Data Warehouses & Business Intelligence Service Oriented Architecture SOA and Web Services John O Brien President and Executive Architect Zukeran Technologies

More information

EII - ETL - EAI What, Why, and How!

EII - ETL - EAI What, Why, and How! IBM Software Group EII - ETL - EAI What, Why, and How! Tom Wu 巫 介 唐, wuct@tw.ibm.com Information Integrator Advocate Software Group IBM Taiwan 2005 IBM Corporation Agenda Data Integration Challenges and

More information

Open Source Toolkit. Penetration Tester's. Jeremy Faircloth. Third Edition. Fryer, Neil. Technical Editor SYNGRESS. Syngrcss is an imprint of Elsevier

Open Source Toolkit. Penetration Tester's. Jeremy Faircloth. Third Edition. Fryer, Neil. Technical Editor SYNGRESS. Syngrcss is an imprint of Elsevier Penetration Tester's Open Source Toolkit Third Edition Jeremy Faircloth Neil Fryer, Technical Editor AMSTERDAM BOSTON HEIDELBERG LONDON NEW YORK OXFORD PARIS. SAN DIEGO SAN FRANCISCO. SINGAPORE SYDNEY

More information

Next-Generation Data Virtualization Fast and Direct Data Access, More Reuse, and Better Agility and Data Governance for BI, MDM, and SOA

Next-Generation Data Virtualization Fast and Direct Data Access, More Reuse, and Better Agility and Data Governance for BI, MDM, and SOA white paper Next-Generation Data Virtualization Fast and Direct Data Access, More Reuse, and Better Agility and Data Governance for BI, MDM, and SOA Executive Summary It s 9:00 a.m. and the CEO of a leading

More information

Data Integration: Using ETL, EAI, and EII Tools to Create an Integrated Enterprise. Colin White Founder, BI Research TDWI Webcast October 2005

Data Integration: Using ETL, EAI, and EII Tools to Create an Integrated Enterprise. Colin White Founder, BI Research TDWI Webcast October 2005 Data Integration: Using ETL, EAI, and EII Tools to Create an Integrated Enterprise Colin White Founder, BI Research TDWI Webcast October 2005 TDWI Data Integration Study Copyright BI Research 2005 2 Data

More information

Private Cloud Computing

Private Cloud Computing Private Cloud Computing Consolidation, Virilization, and Service-Oriented Infrastructure Stephen R. Smoot Nam K. Tan ELSEVIER AMSTERDAM BOSTON HEIDELBERG LONDON NEW YORK OXFORD PARIS SAN DIEGO M< SAN FRANCISCO

More information

Network Security. Windows 2012 Server. Securing Your Windows. Infrastructure. Network Systems and. Derrick Rountree. Richard Hicks, Technical Editor

Network Security. Windows 2012 Server. Securing Your Windows. Infrastructure. Network Systems and. Derrick Rountree. Richard Hicks, Technical Editor Windows 2012 Server Network Security Securing Your Windows Network Systems and Infrastructure Derrick Rountree Richard Hicks, Technical Editor AMSTERDAM BOSTON HEIDELBERG LONDON NEW YORK OXFORD PARIS SAN

More information

Knowledgent White Paper Series. Developing an MDM Strategy WHITE PAPER. Key Components for Success

Knowledgent White Paper Series. Developing an MDM Strategy WHITE PAPER. Key Components for Success Developing an MDM Strategy Key Components for Success WHITE PAPER Table of Contents Introduction... 2 Process Considerations... 3 Architecture Considerations... 5 Conclusion... 9 About Knowledgent... 10

More information

Enterprise Service Bus Defined. Wikipedia says (07/19/06)

Enterprise Service Bus Defined. Wikipedia says (07/19/06) Enterprise Service Bus Defined CIS Department Professor Duane Truex III Wikipedia says (07/19/06) In computing, an enterprise service bus refers to a software architecture construct, implemented by technologies

More information

Risk Analysis and the Security Survey

Risk Analysis and the Security Survey Risk Analysis and the Security Survey Fourth Edition James F. Broder Eugene Tucker ELSEVIER AMSTERDAM BOSTON HEIDELBERG LONDON NEWYORK OXFORD PARIS SAN DIEGO SAN FRANCISCO SINGAPORE SYDNEY TOKYO Butterworth-Heinemann

More information

Virtualization and Forensics

Virtualization and Forensics Virtualization and Forensics A Digital Forensic Investigator's Guide to Virtual Environments Diane Barrett Gregory Kipper Technical Editor Samuel Liles ELSEVIER AMSTERDAM BOSTON HEIDELBERG LONDON NEWYORK

More information

Obj ect-oriented Construction Handbook

Obj ect-oriented Construction Handbook Obj ect-oriented Construction Handbook Developing Application-Oriented Software with the Tools & Materials Approach Heinz Züllighoven IT'Workplace Solutions, Inc., and LJniversity of Hamburg, Germany as

More information

Traditional BI vs. Business Data Lake A comparison

Traditional BI vs. Business Data Lake A comparison Traditional BI vs. Business Data Lake A comparison The need for new thinking around data storage and analysis Traditional Business Intelligence (BI) systems provide various levels and kinds of analyses

More information

BIG DATA AND THE ENTERPRISE DATA WAREHOUSE WORKSHOP

BIG DATA AND THE ENTERPRISE DATA WAREHOUSE WORKSHOP BIG DATA AND THE ENTERPRISE DATA WAREHOUSE WORKSHOP Business Analytics for All Amsterdam - 2015 Value of Big Data is Being Recognized Executives beginning to see the path from data insights to revenue

More information

Management. Oracle Fusion Middleware. 11 g Architecture and. Oracle Press ORACLE. Stephen Lee Gangadhar Konduri. Mc Grauu Hill.

Management. Oracle Fusion Middleware. 11 g Architecture and. Oracle Press ORACLE. Stephen Lee Gangadhar Konduri. Mc Grauu Hill. ORACLE Oracle Press Oracle Fusion Middleware 11 g Architecture and Management Reza Shafii Stephen Lee Gangadhar Konduri Mc Grauu Hill New York Chicago San Francisco Lisbon London Madrid Mexico City Milan

More information

Cyber Attacks. Protecting National Infrastructure Student Edition. Edward G. Amoroso

Cyber Attacks. Protecting National Infrastructure Student Edition. Edward G. Amoroso Cyber Attacks Protecting National Infrastructure Student Edition Edward G. Amoroso ELSEVIER. AMSTERDAM BOSTON HEIDELBERG LONDON NEW YORK OXFORD PARIS SAN DIEGO SAN FRANCISCO SINGAPORE SYDNEY TOKYO Butterworth-Heinemann

More information

Master Data Management and Data Warehousing. Zahra Mansoori

Master Data Management and Data Warehousing. Zahra Mansoori Master Data Management and Data Warehousing Zahra Mansoori 1 1. Preference 2 IT landscape growth IT landscapes have grown into complex arrays of different systems, applications, and technologies over the

More information

Offload Historical Data to Big Data Lake. Ample White Paper

Offload Historical Data to Big Data Lake. Ample White Paper Offload Historical Data to Big Data Lake The Need to Offload Historical Data for Compliance Queries How often have heard that the legal or compliance department group needs to have access to your company

More information

Supply Chain Strategies

Supply Chain Strategies Supply Chain Strategies Customer-driven and customer-focused Tony Hines ELSEVIER BUTTERWORTH HEINEMANN AMSTERDAM BOSTON HEIDELBERG LONDON NEW YORK OXFORD PARIS SAN DIEGO SAN FRANCISCO SINGAPORE SYDNEY

More information

Service Oriented Architecture (SOA) Architecture, Governance, Standards and Technologies

Service Oriented Architecture (SOA) Architecture, Governance, Standards and Technologies Service Oriented Architecture (SOA) Architecture, Governance, Standards and Technologies 3-day seminar Give Your Business the Competitive Edge SOA has rapidly seized the momentum and center stage because

More information

The Lab and The Factory

The Lab and The Factory The Lab and The Factory Architecting for Big Data Management April Reeve DAMA Wisconsin March 11 2014 1 A good speech should be like a woman's skirt: long enough to cover the subject and short enough to

More information

Building a Data Warehouse

Building a Data Warehouse Building a Data Warehouse With Examples in SQL Server EiD Vincent Rainardi BROCHSCHULE LIECHTENSTEIN Bibliothek Apress Contents About the Author. ; xiij Preface xv ^CHAPTER 1 Introduction to Data Warehousing

More information

Data Virtualization for Agile Business Intelligence Systems and Virtual MDM. To View This Presentation as a Video Click Here

Data Virtualization for Agile Business Intelligence Systems and Virtual MDM. To View This Presentation as a Video Click Here Data Virtualization for Agile Business Intelligence Systems and Virtual MDM To View This Presentation as a Video Click Here Agenda Data Virtualization New Capabilities New Challenges in Data Integration

More information

Rapid System Prototyping with FPGAs

Rapid System Prototyping with FPGAs Rapid System Prototyping with FPGAs By R.C. Coferand Benjamin F. Harding AMSTERDAM BOSTON HEIDELBERG LONDON NEW YORK OXFORD PARIS SAN DIEGO SAN FRANCISCO SINGAPORE SYDNEY TOKYO Newnes is an imprint of

More information

Data Ownership and Enterprise Data Management: Implementing a Data Management Strategy (Part 3)

Data Ownership and Enterprise Data Management: Implementing a Data Management Strategy (Part 3) A DataFlux White Paper Prepared by: Mike Ferguson Data Ownership and Enterprise Data Management: Implementing a Data Management Strategy (Part 3) Leader in Data Quality and Data Integration www.flux.com

More information

Data Warehouse Overview. Srini Rengarajan

Data Warehouse Overview. Srini Rengarajan Data Warehouse Overview Srini Rengarajan Please mute Your cell! Agenda Data Warehouse Architecture Approaches to build a Data Warehouse Top Down Approach Bottom Up Approach Best Practices Case Example

More information

BUSINESS INTELLIGENCE

BUSINESS INTELLIGENCE SECOND EDITION BUSINESS INTELLIGENCE A MANAGERIAL APPROACH INTERNATIONAL EDITION Efraim Turban University of Hawaii Ramesh Sharda Oklahoma State University Dursun Deleii Oklahoma State University David

More information

Oracle Big Data Handbook

Oracle Big Data Handbook ORACLG Oracle Press Oracle Big Data Handbook Tom Plunkett Brian Macdonald Bruce Nelson Helen Sun Khader Mohiuddin Debra L. Harding David Segleau Gokula Mishra Mark F. Hornick Robert Stackowiak Keith Laker

More information

Digital Forensics with Open Source Tools

Digital Forensics with Open Source Tools Digital Forensics with Open Source Tools Cory Altheide Harlan Carvey Technical Editor Ray Davidson AMSTERDAM BOSTON HEIDELBERG LONDON NEW YORK OXFORD PARIS SAN DIEGO SAN FRANCISCO SINGAPORE SYDNEY TOKYO

More information

Data Virtualization Usage Patterns for Business Intelligence/ Data Warehouse Architectures

Data Virtualization Usage Patterns for Business Intelligence/ Data Warehouse Architectures DATA VIRTUALIZATION Whitepaper Data Virtualization Usage Patterns for / Data Warehouse Architectures www.denodo.com Incidences Address Customer Name Inc_ID Specific_Field Time New Jersey Chevron Corporation

More information

Configuration. Management for. Senior Managers. Essential Product Configuration. and Lifecycle Management

Configuration. Management for. Senior Managers. Essential Product Configuration. and Lifecycle Management Configuration Management for Senior Managers Essential Product Configuration and Lifecycle Management for Manufacturing Frank B. Watts ELSEVIER AMSTERDAM BOSTON HEIDELBERG LONDON NEW YORK OXFORD PARIS

More information

Best Practices in Leveraging a Staging Area for SaaS-to-Enterprise Integration

Best Practices in Leveraging a Staging Area for SaaS-to-Enterprise Integration white paper Best Practices in Leveraging a Staging Area for SaaS-to-Enterprise Integration David S. Linthicum Introduction SaaS-to-enterprise integration requires that a number of architectural calls are

More information

Service Oriented Architecture (SOA) Architecture, Governance, Standards and Technologies

Service Oriented Architecture (SOA) Architecture, Governance, Standards and Technologies Service Oriented Architecture (SOA) Architecture, Governance, Standards and Technologies 3-day seminar Give Your Business the Competitive Edge SOA has rapidly seized the momentum and center stage because

More information

SharePoint 2010. Overview, Governance, and Planning. (^Rll^^fc^ i ip?"^biifiis:'iissiipi. Scott Jamison. Susan Hanley Mauro Cardarelli.

SharePoint 2010. Overview, Governance, and Planning. (^Rll^^fc^ i ip?^biifiis:'iissiipi. Scott Jamison. Susan Hanley Mauro Cardarelli. Ec,V$%fMM SharePoint 2010 i ip?"^biifiis:'iissiipi Overview, Governance, (^Rll^^fc^ and Planning Ipft^'" Scott Jamison Susan Hanley Mauro Cardarelli Upper Saddle River, NJ Boston Indianapolis San Francisco

More information

Enterprise Data Integration for Microsoft Dynamics CRM

Enterprise Data Integration for Microsoft Dynamics CRM Enterprise Data Integration for Microsoft Dynamics CRM Daniel Cai http://danielcai.blogspot.com About me Daniel Cai Developer @KingswaySoft a software company offering integration software and solutions

More information

Securing SQL Server. Protecting Your Database from. Second Edition. Attackers. Denny Cherry. Michael Cross. Technical Editor ELSEVIER

Securing SQL Server. Protecting Your Database from. Second Edition. Attackers. Denny Cherry. Michael Cross. Technical Editor ELSEVIER Securing SQL Server Second Edition Protecting Your Database from Attackers Denny Cherry Technical Editor Michael Cross AMSTERDAM BOSTON HEIDELBERG LONDON ELSEVIER NEW YORK OXFORD PARIS SAN DIEGO SAN FRANCISCO

More information

IBM AND NEXT GENERATION ARCHITECTURE FOR BIG DATA & ANALYTICS!

IBM AND NEXT GENERATION ARCHITECTURE FOR BIG DATA & ANALYTICS! The Bloor Group IBM AND NEXT GENERATION ARCHITECTURE FOR BIG DATA & ANALYTICS VENDOR PROFILE The IBM Big Data Landscape IBM can legitimately claim to have been involved in Big Data and to have a much broader

More information

Oracle Data Integrator for OWB Developers. Mark Rittman, Rittman Mead Consulting

Oracle Data Integrator for OWB Developers. Mark Rittman, Rittman Mead Consulting Oracle Data Integrator for OWB Developers Mark Rittman, Rittman Mead Consulting http://www.rittmanmead.com Who Am I? Oracle BI&W Architecture & Development Specialist The Rittman of Rittman Mead Consulting

More information

FIFTH EDITION. Oracle Essentials. Rick Greenwald, Robert Stackowiak, and. Jonathan Stern O'REILLY" Tokyo. Koln Sebastopol. Cambridge Farnham.

FIFTH EDITION. Oracle Essentials. Rick Greenwald, Robert Stackowiak, and. Jonathan Stern O'REILLY Tokyo. Koln Sebastopol. Cambridge Farnham. FIFTH EDITION Oracle Essentials Rick Greenwald, Robert Stackowiak, and Jonathan Stern O'REILLY" Beijing Cambridge Farnham Koln Sebastopol Tokyo _ Table of Contents Preface xiii 1. Introducing Oracle 1

More information

Data Virtualization A Potential Antidote for Big Data Growing Pains

Data Virtualization A Potential Antidote for Big Data Growing Pains perspective Data Virtualization A Potential Antidote for Big Data Growing Pains Atul Shrivastava Abstract Enterprises are already facing challenges around data consolidation, heterogeneity, quality, and

More information

ORACLE DATA INTEGRATOR ENTERPRISE EDITION

ORACLE DATA INTEGRATOR ENTERPRISE EDITION ORACLE DATA INTEGRATOR ENTERPRISE EDITION Oracle Data Integrator Enterprise Edition 12c delivers high-performance data movement and transformation among enterprise platforms with its open and integrated

More information

IBM Software Delivering trusted information for the modern data warehouse

IBM Software Delivering trusted information for the modern data warehouse Delivering trusted information for the modern data warehouse Make information integration and governance a best practice in the big data era Contents 2 Introduction In ever-changing business environments,

More information

SAS Enterprise Data Integration Server - A Complete Solution Designed To Meet the Full Spectrum of Enterprise Data Integration Needs

SAS Enterprise Data Integration Server - A Complete Solution Designed To Meet the Full Spectrum of Enterprise Data Integration Needs Database Systems Journal vol. III, no. 1/2012 41 SAS Enterprise Data Integration Server - A Complete Solution Designed To Meet the Full Spectrum of Enterprise Data Integration Needs 1 Silvia BOLOHAN, 2

More information

A Next-Generation Analytics Ecosystem for Big Data. Colin White, BI Research September 2012 Sponsored by ParAccel

A Next-Generation Analytics Ecosystem for Big Data. Colin White, BI Research September 2012 Sponsored by ParAccel A Next-Generation Analytics Ecosystem for Big Data Colin White, BI Research September 2012 Sponsored by ParAccel BIG DATA IS BIG NEWS The value of big data lies in the business analytics that can be generated

More information

SERVICE ORIENTED ARCHITECTURE

SERVICE ORIENTED ARCHITECTURE SERVICE ORIENTED ARCHITECTURE Introduction SOA provides an enterprise architecture that supports building connected enterprise applications to provide solutions to business problems. SOA facilitates the

More information

SOA REFERENCE ARCHITECTURE: SERVICE TIER

SOA REFERENCE ARCHITECTURE: SERVICE TIER SOA REFERENCE ARCHITECTURE: SERVICE TIER SOA Blueprint A structured blog by Yogish Pai Service Tier The service tier is the primary enabler of the SOA and includes the components described in this section.

More information

Data Warehouse Design

Data Warehouse Design Data Warehouse Design Modern Principles and Methodologies Matteo Golfarelli Stefano Rizzi Translated by Claudio Pagliarani Mc Grauu Hill New York Chicago San Francisco Lisbon London Madrid Mexico City

More information

Decoding the Big Data Deluge a Virtual Approach. Dan Luongo, Global Lead, Field Solution Engineering Data Virtualization Business Unit, Cisco

Decoding the Big Data Deluge a Virtual Approach. Dan Luongo, Global Lead, Field Solution Engineering Data Virtualization Business Unit, Cisco Decoding the Big Data Deluge a Virtual Approach Dan Luongo, Global Lead, Field Solution Engineering Data Virtualization Business Unit, Cisco High-volume, velocity and variety information assets that demand

More information

Knowledge-Based Systems IS430. Mostafa Z. Ali

Knowledge-Based Systems IS430. Mostafa Z. Ali Winter 2009 Knowledge-Based Systems IS430 Data Warehousing Lesson 6 Mostafa Z. Ali mzali@just.edu.jo Lecture 2: Slide 1 Learning Objectives Understand the basic definitions and concepts of data warehouses

More information

Information Architecture

Information Architecture The Bloor Group Actian and The Big Data Information Architecture WHITE PAPER The Actian Big Data Information Architecture Actian and The Big Data Information Architecture Originally founded in 2005 to

More information

Chapter 5. Learning Objectives. DW Development and ETL

Chapter 5. Learning Objectives. DW Development and ETL Chapter 5 DW Development and ETL Learning Objectives Explain data integration and the extraction, transformation, and load (ETL) processes Basic DW development methodologies Describe real-time (active)

More information

Filtering the Web to Feed Data Warehouses

Filtering the Web to Feed Data Warehouses Witold Abramowicz, Pawel Kalczynski and Krzysztof We^cel Filtering the Web to Feed Data Warehouses Springer Table of Contents CHAPTER 1 INTRODUCTION 1 1.1 Information Systems 1 1.2 Information Filtering

More information

Data Virtualization. Paul Moxon Denodo Technologies. Alberta Data Architecture Community January 22 nd, 2014. 2014 Denodo Technologies

Data Virtualization. Paul Moxon Denodo Technologies. Alberta Data Architecture Community January 22 nd, 2014. 2014 Denodo Technologies Data Virtualization Paul Moxon Denodo Technologies Alberta Data Architecture Community January 22 nd, 2014 The Changing Speed of Business 100 25 35 45 55 65 75 85 95 Gartner The Nexus of Forces Today s

More information

USING BIG DATA FOR INTELLIGENT BUSINESSES

USING BIG DATA FOR INTELLIGENT BUSINESSES HENRI COANDA AIR FORCE ACADEMY ROMANIA INTERNATIONAL CONFERENCE of SCIENTIFIC PAPER AFASES 2015 Brasov, 28-30 May 2015 GENERAL M.R. STEFANIK ARMED FORCES ACADEMY SLOVAK REPUBLIC USING BIG DATA FOR INTELLIGENT

More information

REAL-TIME OPERATIONAL INTELLIGENCE. Competitive advantage from unstructured, high-velocity log and machine Big Data

REAL-TIME OPERATIONAL INTELLIGENCE. Competitive advantage from unstructured, high-velocity log and machine Big Data REAL-TIME OPERATIONAL INTELLIGENCE Competitive advantage from unstructured, high-velocity log and machine Big Data 2 SQLstream: Our s-streaming products unlock the value of high-velocity unstructured log

More information

Web Development with TIBCO General Interface

Web Development with TIBCO General Interface Web Development with TIBCO General Interface Building AJAX Clients for Enterprise SOA Anil Gurnani /TAddison-Wesley Upper Saddle River, NJ Boston Indianapolis San Francisco New York Toronto Montreal London

More information

... Foreword... 17. ... Preface... 19

... Foreword... 17. ... Preface... 19 ... Foreword... 17... Preface... 19 PART I... SAP's Enterprise Information Management Strategy and Portfolio... 25 1... Introducing Enterprise Information Management... 27 1.1... Defining Enterprise Information

More information

High-Volume Data Warehousing in Centerprise. Product Datasheet

High-Volume Data Warehousing in Centerprise. Product Datasheet High-Volume Data Warehousing in Centerprise Product Datasheet Table of Contents Overview 3 Data Complexity 3 Data Quality 3 Speed and Scalability 3 Centerprise Data Warehouse Features 4 ETL in a Unified

More information

LIMS Integration Framework Model

LIMS Integration Framework Model May 2010 LIMS Integration Framework Model Dr. Partha Mukherjee Contents Abstract 2 Market Trend 3 Target Audience 3 Problem statement 3 Solution 4 Conclusion 8 References 8 About the Author 9 ABOUT HCL

More information

TABLE OF CONTENTS CHAPTER NO. TITLE PAGE NO. ABSTRACT

TABLE OF CONTENTS CHAPTER NO. TITLE PAGE NO. ABSTRACT vi TABLE OF CONTENTS CHAPTER NO. TITLE PAGE NO. ABSTRACT ii LIST OF TABLES ix LIST OF FIGURES x LIST OF ABBREVIATIONS xiii 1 INTRODUCTION 1 2 ARCHITECTURES FOR SYSTEM INTEGRATION 8 2.1 Enterprise Application

More information

Enterprise Data Integration

Enterprise Data Integration Enterprise Data Integration Access, Integrate, and Deliver Data Efficiently Throughout the Enterprise brochure How Can Your IT Organization Deliver a Return on Data? The High Price of Data Fragmentation

More information

The four (five) Sensors

The four (five) Sensors The four (five) Sensors SWE based sensor integration in the German Indonesian Tsunami Early Warning and Mitigation System project (GITEWS) Rainer Häner, GeoForschungsZentrum Potsdam Content GITEWS: A short

More information

TECHNOLOGY TRANSFER PRESENTS MAX. From EAI to SOA ACHIEVING BUSINESS AGILITY THROUGH INTEGRATION

TECHNOLOGY TRANSFER PRESENTS MAX. From EAI to SOA ACHIEVING BUSINESS AGILITY THROUGH INTEGRATION TECHNOLOGY TRANSFER PRESENTS MAX DOLGICER From EAI to SOA to Cloud Integration ACHIEVING BUSINESS AGILITY THROUGH INTEGRATION DECEMBER 12-14, 2011 RESIDENZA DI RIPETTA - VIA DI RIPETTA, 231 ROME (ITALY)

More information

Real Time Big Data Processing

Real Time Big Data Processing Real Time Big Data Processing Cloud Expo 2014 Ian Meyers Amazon Web Services Global Infrastructure Deployment & Administration App Services Analytics Compute Storage Database Networking AWS Global Infrastructure

More information

Fixed/Mobile Convergence and Beyond AMSTERDAM BOSTON. HEIDELBERG LONDON

Fixed/Mobile Convergence and Beyond AMSTERDAM BOSTON. HEIDELBERG LONDON Fixed/Mobile Convergence and Beyond Unbounded Mobile Communications Richard Watson AMSTERDAM BOSTON. HEIDELBERG LONDON NEW YORK. OXFORD PARIS SAN DIEGO SAN FRANCISCO SINGAPORE SYDNEY. TOKYO ELSEVIER

More information

Solving Your Big Data Problems with Fast Data (Better Decisions and Instant Action)

Solving Your Big Data Problems with Fast Data (Better Decisions and Instant Action) Solving Your Big Data Problems with Fast Data (Better Decisions and Instant Action) Does your company s integration strategy support your mobility, big data, and loyalty projects today and are you prepared

More information

Manifest for Big Data Pig, Hive & Jaql

Manifest for Big Data Pig, Hive & Jaql Manifest for Big Data Pig, Hive & Jaql Ajay Chotrani, Priyanka Punjabi, Prachi Ratnani, Rupali Hande Final Year Student, Dept. of Computer Engineering, V.E.S.I.T, Mumbai, India Faculty, Computer Engineering,

More information

Integrate and Deliver Trusted Data and Enable Deep Insights

Integrate and Deliver Trusted Data and Enable Deep Insights SAP Technical Brief SAP s for Enterprise Information Management SAP Data Services Objectives Integrate and Deliver Trusted Data and Enable Deep Insights Provide a wide-ranging view of enterprise information

More information

The Data Warehouse ETL Toolkit

The Data Warehouse ETL Toolkit 2008 AGI-Information Management Consultants May be used for personal purporses only or by libraries associated to dandelon.com network. The Data Warehouse ETL Toolkit Practical Techniques for Extracting,

More information

Embracing the Cloud, Mobile, Social & Big Data

Embracing the Cloud, Mobile, Social & Big Data Embracing the Cloud, Mobile, Social & Big Data Introducing ARIS 9.0 and webmethods 9.0 Dr. Wolfram Jost CTO Software AG 2010-2013 2 Positioning 3 2013 Software 2013 AG. Software All rights AG. reserved.

More information

Contents. Foreword. Acknowledgments Introduction

Contents. Foreword. Acknowledgments Introduction The Manager's Handbook for Corporate Security Establishing and Managing a Successful Assets Protection Program Dr. Gerald L Kovacich Edward P. Halibozek ilu TTERWORTH I N E M A N N An imprint of Elsevier

More information

SOLUTION BRIEF. JUST THE FAQs: Moving Big Data with Bulk Load. www.datadirect.com

SOLUTION BRIEF. JUST THE FAQs: Moving Big Data with Bulk Load. www.datadirect.com SOLUTION BRIEF JUST THE FAQs: Moving Big Data with Bulk Load 2 INTRODUCTION As the data and information used by businesses grow exponentially, IT organizations face a daunting challenge moving what is

More information

Implementation & Administration

Implementation & Administration Microsoft SQL Server 2008 R2 Master Data Services: Implementation & Administration Tyler Graham Suzanne Selhorn Mc Grauu Hill New York Chicago San Francisco Lisbon London Madrid Mexico City Milan New Delhi

More information

for the Entire Organization

for the Entire Organization Enterprise Risk Management A Common Framework for the Entire Organization Philip E. J. Green ELSEVIER AMSTERDAM. BOSTON. HEIDELBERG. LONDON NEW YORK OXFORD. PARIS. SAN DIEGO SAN FRANCISCO. SINGAPORE. SYDNEY.

More information

A Service-oriented Architecture for Business Intelligence

A Service-oriented Architecture for Business Intelligence A Service-oriented Architecture for Business Intelligence Liya Wu 1, Gilad Barash 1, Claudio Bartolini 2 1 HP Software 2 HP Laboratories {name.surname@hp.com} Abstract Business intelligence is a business

More information

Offload Enterprise Data Warehouse (EDW) to Big Data Lake. Ample White Paper

Offload Enterprise Data Warehouse (EDW) to Big Data Lake. Ample White Paper Offload Enterprise Data Warehouse (EDW) to Big Data Lake Oracle Exadata, Teradata, Netezza and SQL Server Ample White Paper EDW (Enterprise Data Warehouse) Offloads The EDW (Enterprise Data Warehouse)

More information

Independent process platform

Independent process platform Independent process platform Megatrend in infrastructure software Dr. Wolfram Jost CTO February 22, 2012 2 Agenda Positioning BPE Strategy Cloud Strategy Data Management Strategy ETS goes Mobile Each layer

More information

THE DATA WAREHOUSE ETL TOOLKIT CDT803 Three Days

THE DATA WAREHOUSE ETL TOOLKIT CDT803 Three Days Three Days Prerequisites Students should have at least some experience with any relational database management system. Who Should Attend This course is targeted at technical staff, team leaders and project

More information

Using Master Data in Business Intelligence

Using Master Data in Business Intelligence helping build the smart business Using Master Data in Business Intelligence Colin White BI Research March 2007 Sponsored by SAP TABLE OF CONTENTS THE IMPORTANCE OF MASTER DATA MANAGEMENT 1 What is Master

More information

Understanding and Selecting Integration Approaches

Understanding and Selecting Integration Approaches Understanding and Selecting Integration Approaches David McGoveran Alternative Technologies 6221A Graham Hill Road, Suite 8001 Felton, California, 95018 Website: Email: mcgoveran@alternativetech.com Telephone:

More information