M6P. TDWI Data Warehouse Automation: Better, Faster, Cheaper You Can Have It All. Mark Peco
|
|
- Abigayle McKenzie
- 7 years ago
- Views:
Transcription
1 M6P European TDWI Conference with June 22 24, 2015 MOC Munich / Germany TDWI Data Warehouse Automation: Better, Faster, Cheaper You Can Have It All Mark Peco TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY
2 Better, Faster, Cheaper... You Can Have It All TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY
3 COURSE OBJECTIVES You will learn about: Concepts, principles, and practices of Data Warehouse Automation The current state of Data Warehouse Automation technology Automation opportunities and benefits when building or managing a data warehouse How to get started with Data Warehouse Automation Best practices and mistakes to avoid with Data Warehouse Automation TDWI takes pride in the educational soundness and technical accuracy of all of our courses. Please send us your comments we d like to hear from you. Address your feedback to: info@tdwi.org Publication Date: February 2015 Copyright 2015 by TDWI. All rights reserved. No part of this document may be reproduced in any form, or by any means, without written permission from The Data Warehousing Institute. ii TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY
4 TABLE OF CONTENTS Module 1 Data Warehouse Automation Concepts and Principles Module 2 Building and Managing the Data Warehouse Module 3 Using Data Warehouse Automation Module 4 Data Warehouse Automation in Action Module 5 Getting Started with Data Warehouse Automation Appx A Bibliography and References... A-1 Appx B Data Warehouse Automation Platforms... B-1 TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY iii
5 iv TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY
6 Data Warehouse Automation Concepts and Principles Module 1 Data Warehouse Automation Concepts and Principles Topic Page Data Warehouse Automation Basics 1-2 Why Data Warehouse Automation? 1-10 The Foundation 1-16 Activities and Deliverables 1-22 The Technology Landscape 1-26 TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY 1-1
7 Data Warehouse Automation Concepts and Principles TDWI Data Warehouse Automation Data Warehouse Automation Basics Data Warehouse Automation Defined 1-2 TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY
8 Data Warehouse Automation Concepts and Principles Data Warehouse Automation Basics Data Warehouse Automation Defined COVERING THE WAREHOUSING LIFECYCLE CHANGING BEST PRACTICES Data warehouse automation is more than simply automation of ETL development, or even the entire development process. It encompasses the entire data warehousing lifecycle from planning, analysis, and design through development and extending into operations, maintenance, and change management. Adoption of data warehouse automation changes the way that we think about building data warehouses. The widely accepted best practice of extensive up-front analysis, design, and modeling can be left behind as the mindset changes from get it right the first time to develop fast and develop frequently an approach that is aligned with today s agile development practices. TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY 1-3
9 Data Warehouse Automation Concepts and Principles TDWI Data Warehouse Automation Data Warehouse Automation Basics Automated vs. Hand-Crafted 1-4 TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY
10 Data Warehouse Automation Concepts and Principles Data Warehouse Automation Basics Automated vs. Hand-Crafted BUILD IT BETTER, FASTER, AND AT LOWER COST CHANGE WHEN AND AS NEEDED WHY HANDCRAFT? Automation in data warehousing has many of the same benefits as in manufacturing: Increased productivity and speed of production Reduction of manual effort Improved quality and consistency Better controls and process optimization opportunities The manufacturing parallel hold when building a data warehouse; we can think of it as an information factory. But data warehousing is more complex than product manufacturing. Manufactured products are typically delivered to a consumer and the job is done. Data warehouses must be sustained through a long life cycle where changes in source data, business requirements, and underlying technologies are ongoing considerations. Automation helps to implement the right changes in the right ways and as quickly as they are needed. There are, of course, popular arguments in favor of handcrafting. In consumer goods, for example, advocates of hand made use terms such as unique, personal, and human touch. These are all valid reasons to buy a hand made scarf, but are they qualities that are needed in a data warehouse? TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY 1-5
11 Data Warehouse Automation Concepts and Principles TDWI Data Warehouse Automation Data Warehouse Automation Basics Automation vs. Customization 1-6 TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY
12 Data Warehouse Automation Concepts and Principles Data Warehouse Automation Basics Automation vs. Customization NOT MUTUALLY EXCLUSIVE Automation does not preclude customization, nor does it imply mass production. Neither is practical in data warehousing as most data warehouses need to have some level of customization. The purpose of customizing is to meet unique and specific needs that are likely to exist for every data warehouse. Strategic and business-critical uses typically have greater need for customizing than routine and repetitive uses. TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY 1-7
13 Data Warehouse Automation Concepts and Principles TDWI Data Warehouse Automation Data Warehouse Automation Basics Data Warehouse Automation vs. CASE 1-8 TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY
14 Data Warehouse Automation Concepts and Principles Data Warehouse Automation Basics Data Warehouse Automation vs. CASE SMART AUTOMATION Some may recall the popularity of Computer Aided Systems Engineering (CASE) in the 1980 s. The purpose of CASE was to automatically generate program code and database schema from models and specifications collected and stored in a repository. The objective was to fully automate the process of systems development. The labor-intensive activities shifted from programming to analysis/design/specification with the goal of populating the repository. Methodology was a predominant factor and systems quality almost a singular goal. Data warehouse automation is distinctly different from CASE. The goal is to automate that which is smart to automate not necessarily to automate everything. Analysis can reasonably be accelerated with requirements discovery and design optimized by use of design patterns. Methodology is applied to support the processes, not to define and dominate them. And the goals include speed, quality, efficiency, effectiveness, and adaptability. TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY 1-9
15 Data Warehouse Automation Concepts and Principles TDWI Data Warehouse Automation Why Data Warehouse Automation? Business Benefits 1-10 TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY
16 Data Warehouse Automation Concepts and Principles Why Data Warehouse Automation? Business Benefits QUALITY AND EFFECTIVENESS AGILE BUSINESS SPEED COST SAVINGS Data warehouse automation delivers quality and effectiveness through ability to build better solutions. Better solutions are those that best meet real business requirements, and it is especially difficult to get complete and correct requirements when limited to an early phase of a linear development process. With data warehouse automation the business can make changes much later in the development process and change can occur more frequently with less disruption, waste, and rework. Ability to change fast and frequently extends beyond the warehouse development process. Changes that occur in business requirements can be met with quick response. Responding to change in real time and without the delay of lengthy projects is the essence of business agility. Speed is the critical factor that enables agility both for agile business and for agile development. Ability to generate quickly and to regenerate equally fast when change occurs are fundamental automation capabilities. Ultimately building better, building faster, and changing quickly when needed bring substantial cost savings to data warehouse development, operation, maintenance, and evolution. TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY 1-11
17 Data Warehouse Automation Concepts and Principles TDWI Data Warehouse Automation Why Data Warehouse Automation? Technical Benefits 1-12 TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY
18 Data Warehouse Automation Concepts and Principles Why Data Warehouse Automation? Technical Benefits THE VALUE FOR IT ORGANIZATIONS The benefits of automation aren t solely for business. IT organizations also derive real value from data warehouse automation in all of the ways that are illustrated on the facing page. The first four items agility, time savings, adaptability and cost savings are important and valuable for developers. The remaining part of the list documentation, impact analysis, testing, and maintainability arguably have greater impact because they apply beyond the development project and throughout the life of the data warehouse. TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY 1-13
19 Data Warehouse Automation Concepts and Principles TDWI Data Warehouse Automation Why Data Warehouse Automation? Barriers and Resistance 1-14 TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY
20 Data Warehouse Automation Concepts and Principles Why Data Warehouse Automation? Barriers and Resistance REASONS FOR RESISTANCE PROCESS BARRIERS AND CONSTRAINTS HUMAN BARRIERS AND CONSTRAINTS Despite the obvious benefits, adoption of data warehouse automation is not a given. You may encounter resistance from two perspectives processes that aren t automation ready and people who are reluctant to change. Be prepared to adjust and adapt your data warehousing processes development, operation, and maintenance when implementing data warehouse automation. Processes that are designed to minimize risk in non-automated environments create conflict and limit the benefits of automation. We ll discuss the process considerations of change management in Module 5: Getting Started with Data Warehouse Automation. People will resist automation for a variety of reasons that are generally associated with fear that they will lose something loss of job, loss of status, loss of control, etc. We ll discuss the human aspects of change management in Module 5: Getting Started with Data Warehouse Automation. TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY 1-15
21 Data Warehouse Automation Concepts and Principles TDWI Data Warehouse Automation The Foundation Components of Data Warehousing 1-16 TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY
22 Data Warehouse Automation Concepts and Principles The Foundation Components of Data Warehousing DATA FLOW FROM DISPARATE DATA TO INTEGRATED INFORMATION The facing page illustrates the core elements of data warehousing beginning with disparate data sources at the bottom of the diagram and leading to integrated information resources at the top. The information resources aren t necessarily the end of the line they are simply the end of the data integration processes. Business value is created when they are used for reporting, business intelligence, decision-making, analytics, etc. The center of the diagram shows the processing steps to get from data sources to integrated information using two methods data consolidation through extract-transform-load processing and data virtualization through a series of abstract data views. Consolidation and virtualization are often combined to optimize the flow of data. Data warehouse automation can implement both approaches independently or in a mix-and-match form. ALL OF THE PARTS Data warehousing encompasses many different techniques and produces many components to enable the data-to-information flow. Among these are architectural standards, data models, mappings, data transformations, database load procedures, tests and controls, events and errors, and metadata. Data warehouse automation includes capabilities to create, connect, manage, and apply all of these components. TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY 1-17
23 Data Warehouse Automation Concepts and Principles TDWI Data Warehouse Automation The Foundation Data Warehousing Projects 1-18 TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY
24 Data Warehouse Automation Concepts and Principles The Foundation Data Warehousing Projects PROGRAM VS. PROJECT Data warehousing is more than just a project or a series of projects. It involves many projects within a program context that also includes evolving architecture and ongoing operations. Projects are iterative, timebounded activities to produce specific data warehousing capabilities. The program is a continuous investment of resources that encompasses projects, operations, and value creation. TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY 1-19
25 Data Warehouse Automation Concepts and Principles TDWI Data Warehouse Automation The Foundation Design Patterns 1-20 TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY
26 Data Warehouse Automation Concepts and Principles The Foundation Design Patterns ENCAPSULATING STANDARDS AND CONVENTIONS As described earlier, one of the benefits of automation is accelerated through use of design patterns. Design patterns can encapsulate architectural standards as well as best practices for data design, data management, data integration, and data usage. The facing page identifies common design patterns in all of these categories. Creating and applying patterns in a data warehouse automation tool supports the goal of accelerated design, but more importantly it drives compliance with standards and consistency of data warehousing results. TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY 1-21
27 Data Warehouse Automation Concepts and Principles TDWI Data Warehouse Automation Activities and Deliverables Data Warehousing Processes 1-22 TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY
28 Data Warehouse Automation Concepts and Principles Activities and Deliverables Data Warehousing Processes LABOR INTENSITY, COMPLEXITY, AND VOLATILITY A typical data warehouse development process is burdened by laborintensive activities, a high degree of complexity and dependency among deliverables, and volatility of data and requirements. Some of the data warehousing pain that can be relieved by automation includes: Slow, laborious, and difficult to get it right requirement gathering Interdependency between source data analysis and warehouse data modeling Manually mapping sources to targets Detailed specification of data transformation logic Translating data models into schema and building databases Hand coding and manual testing of ETL processing Preparing and loading initial start-up data Acceptance testing and deployment to production These activities would be difficult enough in a linear process, but in reality data warehousing projects are never linear. We re always going back because: Data modeling may change requirements Source-target mapping may affect data modeling Data transformation design may change mappings Integration process design may affect database design Incremental testing may cause changes throughout Acceptance testing may discover incorrect requirements and so on... When we survive all of these things and deploy the data warehouse it doesn t remain stable. Business needs, source data, and technology will all change (at different times and speeds) driving continuous change in the warehouse. And through it all we need to manage projects, track changes, manage code and system versions, isolate production from development and testing environments, sustain daily operations, and try to keep metadata and documentation synchronized with what is actually implemented. TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY 1-23
29 Data Warehouse Automation Concepts and Principles TDWI Data Warehouse Automation Activities and Deliverables What Can You Automate? 1-24 TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY
30 Data Warehouse Automation Concepts and Principles Activities and Deliverables What Can You Automate? USING DATA WAREHOUSE AUTOMATION Data warehouse automation is capable of relieving much of the pain discussed on the previous pages. Nearly everything in the diagram on the facing page can be automated. All automation tools support core activities such as data modeling and data integration. The most robust automation products support all of the activities. TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY 1-25
31 Data Warehouse Automation Concepts and Principles TDWI Data Warehouse Automation The Technology Landscape Automation Tools and Vendors 1-26 TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY
32 Data Warehouse Automation Concepts and Principles The Technology Landscape Automation Tools and Vendors COMMON TOOLS The facing page illustrates the four most common tools for data warehouse automation Attunity Compose, Magnitude Software Kalido, timextender, and WhereScape. These are the pure DWA product vendors. Others, such as Birst, include some automation capabilities in a broader product suite. Later in the course you ll see more about the approaches and capabilities of Attunity Compose, Magnitude Software Kalido, and WhereScape each with different perspectives and capabilities. The intent is not to compare the tools or to perform product evaluation, but to show a representative sample of mainstream DWA technology. TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY 1-27
33 Data Warehouse Automation Concepts and Principles TDWI Data Warehouse Automation 1-28 TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY
34 Building and Managing the Data Warehouse Module 2 Building and Managing the Data Warehouse Topic Page The Data Warehousing Lifecycle 2-2 Building the Data Warehouse 2-22 Managing the Data Warehouse 2-30 TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY 2-1
35 Building and Managing the Data Warehouse TDWI Data Warehouse Automation The Data Warehousing Lifecycle A Closed Loop 2-2 TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY
36 Building and Managing the Data Warehouse The Data Warehousing Lifecycle A Closed Loop CONTINUOUS WITHOUT AN EXIT POINT The diagram on the facing page revisits the data warehousing lifecycle with emphasis on the blending of program and project work. The important introductory concept is that the lifecycle is a closed-loop system. There is no exit point in the diagram and no end to continuous evolution, change, and redevelopment. This is the fundamental reason that conventional design-and-build cycles don t work well for data warehousing. TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY 2-3
37 Building and Managing the Data Warehouse TDWI Data Warehouse Automation The Data Warehousing Lifecycle Architecture and Planning 2-4 TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY
38 Building and Managing the Data Warehouse The Data Warehousing Lifecycle Architecture and Planning ARCHITECTURE DELIVERABLES The facing page expands the architecture phase of the data warehousing lifecycle to illustrate common deliverables of architecture and planning including: Program Charter describing goals, purpose, resources, governance, sponsorship, etc. Readiness Assessment identifying strengths, weaknesses, risks for the program Business Architecture identifying stakeholders and their interests, business applications, and business processes to be affected Organization Architecture describing the people, purpose, processes, and structure of warehouse development, deployment, operation, and application Data Architecture with standards for structure, storage, distribution, access, etc. Integration Architecture describing the standards and patterns for flow of data from disparate sources, to integrated information, and access by data consumers Process Architecture with standards, conventions, and patterns for data sources, mappings, and integration procedures TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY 2-5
39 Building and Managing the Data Warehouse TDWI Data Warehouse Automation The Data Warehousing Lifecycle Architecture and Planning - Preparation 2-6 TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY
40 Building and Managing the Data Warehouse The Data Warehousing Lifecycle Architecture and Planning - Preparation AUTOMATION BENEFITS Preparation activities of charter and readiness assessment can get value from automation in several ways. Automation helps to: Maximize finances Achieve goals Meet expectations Optimize people Automate processes Increase capabilities TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY 2-7
41 Building and Managing the Data Warehouse TDWI Data Warehouse Automation The Data Warehousing Lifecycle Architecture and Planning - Architecture 2-8 TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY
42 Building and Managing the Data Warehouse The Data Warehousing Lifecycle Architecture and Planning - Architecture AUTOMATION BENEFITS Architectural activities can use automation to: Engage stakeholders Capture business rules Focus on purpose Encapsulate data standards Encapsulate integration standards Automate DW methodology TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY 2-9
43 Building and Managing the Data Warehouse TDWI Data Warehouse Automation The Data Warehousing Lifecycle Implementation 2-10 TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY
44 Building and Managing the Data Warehouse The Data Warehousing Lifecycle Implementation IMPLEMENTATION DELIVERABLES The facing page illustrates high-level categories of deliverables from data warehouse implementation projects including: Source Data Understanding describing content, meaning, and quality of data sources Target Data Models describing the organization and structure of warehouse data Source-Target Mappings associating warehouse data fields with the sources from which they are populated Data Acquisition Processes procedures to obtain data from sources Data Transform Processes procedures to change the data to achieve data warehousing characteristics (integrated, subject oriented, time variant, non-volatile, and sometimes dimensional) Database Load Processes procedures to load data into warehouse databases Warehousing Databases the physical databases in which warehouse data is stored TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY 2-11
45 Building and Managing the Data Warehouse TDWI Data Warehouse Automation The Data Warehousing Lifecycle Implementation Analysis and Design 2-12 TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY
46 Building and Managing the Data Warehouse The Data Warehousing Lifecycle Implementation Analysis and Design AUTOMATION BENEFITS The analysis and design activities of implementation projects source data understanding, target data modeling, and source-target mapping use automation for: Profiling and exploration of data sources Stakeholder driven requirements Applied data standards & templates Data modeling & schema generation Encapsulation of integration standards Metadata driven source-target mapping TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY 2-13
47 Building and Managing the Data Warehouse TDWI Data Warehouse Automation The Data Warehousing Lifecycle Implementation Design and Build 2-14 TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY
48 Building and Managing the Data Warehouse The Data Warehousing Lifecycle Implementation Design and Build AUTOMATION BENEFITS The design and build activities of implementation projects building processes and databases use automation to: Generate processes & procedures Sequence & link processes & procedures Test processes & procedures Generate database schema TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY 2-15
49 Building and Managing the Data Warehouse TDWI Data Warehouse Automation The Data Warehousing Lifecycle Deployment and Operations 2-16 TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY
50 Building and Managing the Data Warehouse The Data Warehousing Lifecycle Deployment and Operations DEPLOYMENT AND OPERATIONS DELIVERABLES The facing page illustrates deliverables and activities of data warehouse deployment and operations including: Managed Environments separating development, testing, and production environments Acceptance Testing confirming that the implementation meets business requirements and formalizing acceptance as a production system Transfer to Operations transfer of knowledge, responsibility, and accountability from development to operations team User Training and Support enabling effective use with the right knowledge, skills, and support structure Scheduling and Execution ensuring that data warehousing processes run when and as they should with attention to dependencies on source systems and among warehousing processes Maintenance and Versioning making small changes and repairs with complete change history and version management Change and Evolution identifying and initiating architectural changes and new projects due to changed data sources, business requirements, or technology capabilities TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY 2-17
51 Building and Managing the Data Warehouse TDWI Data Warehouse Automation The Data Warehousing Lifecycle Deployment and Operations Production Implementation 2-18 TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY
52 Building and Managing the Data Warehouse The Data Warehousing Lifecycle Deployment and Operations Production Implementation AUTOMATION BENEFITS The production implementation activities use automation for: Incremental testing Data warehouse versioning Environment management and migration Documentation and metadata especially as applied for initial training of operations staff and users TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY 2-19
53 Building and Managing the Data Warehouse TDWI Data Warehouse Automation The Data Warehousing Lifecycle Deployment and Operations Execution and Maintenance 2-20 TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY
54 Building and Managing the Data Warehouse The Data Warehousing Lifecycle Deployment and Operations Execution and Maintenance AUTOMATION BENEFITS The execution and maintenance activities use automation for: Incremental testing Source & version management Scheduling Validation testing Documentation and metadata especially as applied for troubleshooting, investigation, and ongoing training Impact analysis TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY 2-21
55 Building and Managing the Data Warehouse TDWI Data Warehouse Automation Building the Data Warehouse Requirements Driven Development 2-22 TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY
56 Building and Managing the Data Warehouse Building the Data Warehouse Requirements Driven Development A TRADITIONAL APPROACH Requirements driven development is the traditional process that begins with requirements gathering and proceeds in a linear fashion through analysis, modeling and design, specification and coding, and deployment. This approach is a legacy of conventional software development processes that doesn t work well for data warehousing. It is characterized by: Intensive and detailed planning Long development timelines Limited business participation early in the process Substantial staffing and skills demand Difficulty in requirements gathering and analysis Much cycling back to previous steps to correct errors and oversights Much waste and rework Post-deployment discovery of incorrect and missed requirements TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY 2-23
57 Building and Managing the Data Warehouse TDWI Data Warehouse Automation Building the Data Warehouse Modeling and Metadata Driven Development 2-24 TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY
58 Building and Managing the Data Warehouse Building the Data Warehouse Modeling and Metadata Driven Development A DESIGN APPROACH Model and metadata driven development focuses on capturing business requirements and translating them to a design that is recorded as models and metadata. The models and metadata are then used in conjunction with design patterns to generate data warehouse deliverables. This approach is an incremental improvement over requirements-driven development in ability to build models and collect metadata that are useful for many related projects. Model and metadata driven development naturally implies some degree of automation. It is characterized by: Need for stated business requirements and well defined scope Development timeline that correlates with project scope Business participation early and late in the process Specialized staffing and skills to perform critical modeling tasks Evolving granularity of requirements through model development Some cycling back to previous steps as a result of model refinement Iterative development with many related and dependent projects TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY 2-25
59 Building and Managing the Data Warehouse TDWI Data Warehouse Automation Building the Data Warehouse Discovery Driven Development 2-26 TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY
60 Building and Managing the Data Warehouse Building the Data Warehouse Discovery Driven Development A LEARNING APPROACH Discovery driven development acknowledges the problems of traditional requirements approaches in data warehousing, replacing them with the concept of requirements through discovery. Fuzzy requirements become clear through cycles of prototyping and refinement of requirements. This approach is a substantial shift away from requirements first approaches. It is highly dependent on automation to enable rapid cycles of prototyping, and is characterized by: Projects of small to modest scope with fuzzy requirements Accelerated development timelines achieved through automation Extensive business participation throughout the prototyping activities and again at acceptance and deployment Demand for data warehouse automation skills and business subject expertise Discovery of requirements through prototyping and evaluation Some rework when inevitable source data surprises occur Multiple projects and iterative development TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY 2-27
61 Building and Managing the Data Warehouse TDWI Data Warehouse Automation Building the Data Warehouse Data Driven Development 2-28 TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY
62 Building and Managing the Data Warehouse Building the Data Warehouse Data Driven Development AN AGILE APPROACH Data driven development adapts the discovery approach to recognize a reality of data warehousing that source data is as challenging as requirements gathering and is often filled with surprises. Combining data exploration with prototyping and requirements discovery creates a truly agile process for data warehousing. Rework caused by unexpected conditions in source data is eliminated by early data exploration. Automation is a must for this approach which is characterized by: Many projects of small scope appropriate for agile development Accelerated development achieved through automation Extensive business participation throughout the entire development process Demand for data warehouse automation skills, agile development mindset, and business subject expertise Discovery of business requirements through prototyping Discovery of data requirements through data exploration Elimination of rework with agile development practices Many small projects and iterative development cycles TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY 2-29
63 Building and Managing the Data Warehouse TDWI Data Warehouse Automation Managing the Data Warehouse Operations 2-30 TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY
64 Building and Managing the Data Warehouse Managing the Data Warehouse Operation AUTOMATION FOR OPERATIONS Operations comprises a core set of activities in managing a data warehouse, with attention to: Sequencing Dependencies Scheduling Execution Verification Validation Error Handling Automation aids data warehouse operations with features and functions for: Scheduling Documentation & Metadata Managed Environments Validation Testing TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY 2-31
65 Building and Managing the Data Warehouse TDWI Data Warehouse Automation Managing the Data Warehouse Monitoring and Tuning 2-32 TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY
66 Building and Managing the Data Warehouse Managing the Data Warehouse Monitoring and Tuning AUTOMATION FOR MONITORING AND TUNING Monitoring and tuning are essential activities of managing a data warehouse, with attention to: Usage Performance Availability Access Security Growth Capacity Automation aids data warehouse monitoring and tuning with features and functions for: Applied Standards Documentation & Metadata Managed Environments Iterative Testing TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY 2-33
67 Building and Managing the Data Warehouse TDWI Data Warehouse Automation Managing the Data Warehouse Maintenance and Change 2-34 TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY
68 Building and Managing the Data Warehouse Managing the Data Warehouse Maintenance and Change AUTOMATION FOR WAREHOUSE MAINTENANCE AND CHANGE The need for maintenance and change is inevitable when managing a data warehouse. Specific activities include: Tracing And Troubleshooting Bug Fixes Data Correction Error Recovery Upgrades Enhancements Automation aids data warehouse maintenance and change management with features and functions for: Applied Standards Impact Analysis Documentation & Metadata Managed Environments Iterative Testing Versioning & Source Control TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY 2-35
69 Building and Managing the Data Warehouse TDWI Data Warehouse Automation Managing the Data Warehouse Evolution and Change 2-36 TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY
70 Building and Managing the Data Warehouse Managing the Data Warehouse Evolution and Change AUTOMATION FOR WAREHOUSE EVOLUTION Evolution is a natural part of the data warehouse lifecycle driven by ongoing change and needs for: Extension Expansion Migration Modernization Re-Architecting Re-Sourcing Refactoring Automation offers many features and functions to support data warehouse evolution including: Applied Standards Impact Analysis Documentation & Metadata Data Source Profiling Managed Environments Data Warehouse Versioning Encapsulated Standards Schema Generation Iterative Testing Versioning & Source Control TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY 2-37
71 Building and Managing the Data Warehouse TDWI Data Warehouse Automation Managing the Data Warehouse Support 2-38 TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY
72 Building and Managing the Data Warehouse Managing the Data Warehouse Support AUTOMATION FOR WAREHOUSE SUPPORG Support is critical to success and sustainability of every data warehouse. Automation aids data warehouse support with features and functions for: Applied Standards Documentation & Metadata Managed Environments Versioning & Change Control TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY 2-39
73 Building and Managing the Data Warehouse TDWI Data Warehouse Automation 2-40 TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY
74 Using Data Warehouse Automation Module 3 Using Data Warehouse Automation Topic Page Automation Use Cases 3-2 Case Studies 3-26 TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY 3-1
75 Using Data Warehouse Automation TDWI Data Warehouse Automation Automation Use Cases Building a New Data Warehouse 3-2 TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY
76 Using Data Warehouse Automation Automation Use Cases Building a New Data Warehouse SCENARIO CHALLENGES OPPORTUNITIES You need to build a new data warehouse either where none exists or as a complete replacement of an existing and dysfunctional warehouse. Without automation, all of the normal data warehousing challenges exist. Source data is messy, warehouses are hard to build, they take too long to build, and they re obsolete before they are deployed. Automation opportunities are abundant in this scenario. You can automate everything from planning to deployment and operation using any of model-driven, discovery-driven, or data-driven approaches. TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY 3-3
77 Using Data Warehouse Automation TDWI Data Warehouse Automation Automation Use Cases Building Dependent Data Marts 3-4 TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY
78 Using Data Warehouse Automation Automation Use Cases Building Dependent Data Marts SCENARIO CHALLENGES OPPORTUNITIES You have a data warehouse but you need to build dependent data marts sourced from that warehouse. Sometimes the data marts are star-schema or cubes and sometimes non-dimensional reporting databases. Data marts are user facing so requirements are critical, but requirements are as difficult for data marts as for data warehouses. Handcrafting of schema and ETL is time-consuming, labor-intensive, and likely to require rework due to unclear requirements. Automation is a natural fit for this scenario. Overcome the requirements challenges with discovery-driven or data-driven development and accelerate the process with automated schema and process generation. TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY 3-5
79 Using Data Warehouse Automation TDWI Data Warehouse Automation Automation Use Cases Building Dimensional Data Marts 3-6 TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY
80 Using Data Warehouse Automation Automation Use Cases Building Dimensional Data Marts SCENARIO CHALLENGES OPPORTUNITIES You need to build dimensional data marts that are sourced directly from operational systems without using a hub data warehouse. This scenario often involves high demand with every business function and many work groups wanting data marts quickly. Matching marts to business needs and source data with mart requirements is difficult. Handcrafted star-schema and ETL processing is too slow to meet demand when the business wants more and faster. This is another natural fit for automation. Any of model-driven, discovery-driven, or data-driven approaches step up to the challenges of business alignment, and automated schema and ETL generation help to meet the demand to deliver more and faster. TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY 3-7
81 Using Data Warehouse Automation TDWI Data Warehouse Automation Automation Use Cases Conforming Independent Data Marts 3-8 TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY
82 Using Data Warehouse Automation Automation Use Cases Conforming Independent Data Marts SCENARIO CHALLENGES OPPORTUNITIES You have many independent data marts that were built without considering the need to conform the dimensions. Now the business wants the ability to drill across and perform analysis that involves more than one data mart. This scenario is challenged to analyze and understand the differences among variations of each dimension, to establish conformity standards for each dimension, to analyze the impact of dimension conformity on each existing data mart, and to rebuild the data marts with dimension standards applied. Automatically documenting the data marts is a good first step to analyze and understand the differences. Standards for conformed dimensions can be encapsulated as design patterns in an automation tool. When the tool is populated with metadata about the existing data marts (a necessary step in automating documentation) then analyzing the impact of change can also be aided by automation. Ultimately data-driven or model-driven techniques can be used to rebuild the data marts with conformed dimensions. TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY 3-9
83 Using Data Warehouse Automation TDWI Data Warehouse Automation Automation Use Cases Data Warehouse Prototyping 3-10 TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY
84 Using Data Warehouse Automation Automation Use Cases Data Warehouse Prototyping SCENARIO CHALLENGES OPPORTUNITIES You need to engage the business and understand requirements before committing time and resources to data warehouse development. And you re working with business users whose perspective on requirements is I ll know what I need when I see it! Prototyping doesn t work without business participation, and it doesn t work with long lag time between prototype building, prototype evaluation, and the next cycle. Automation and prototyping fit together quite naturally. Both discoverydriven and data-driven approaches make it possible to produce visible and tangible results quickly, and to apply the findings and feedback of evaluation in the spirit of rapid prototyping. TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY 3-11
85 Using Data Warehouse Automation TDWI Data Warehouse Automation Automation Use Cases Agile Data Warehousing 3-12 TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY
86 Using Data Warehouse Automation Automation Use Cases Agile Data Warehousing SCENARIO CHALLENGES OPPORTUNITIES You need all of the benefits of agile development fast, focused, and business driven when building the data warehouse. But agile data seems to be much harder to achieve than agile processing. Not only is agile data harder than agile process, but agile techniques are challenged by constraints of the existing data warehouse and of the source data. You need engaged business subject experts, very fast development cycles, and the ability to make the data tangible as requirements come to the surface. Handcrafted schema and ETL is too slow for agile. The agile value of working software over comprehensive documentation is a concern. You need to be agile without sacrificing the documentation and metadata that are essential to warehouse operations, maintenance, usage, and evolution. Automation enables agile data warehousing by supporting very fast development and redevelopment cycles while capturing essential metadata as part of the process. Existing warehouse constraints are easily incorporated, and with a data-driven approach source data constraints are also easily addressed. Generated schema and ETL, quick changes, and versioning all help when applying agile methods to data warehouse development. TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY 3-13
87 Using Data Warehouse Automation TDWI Data Warehouse Automation Automation Use Cases Data Warehouse Migration 3-14 TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY
88 Using Data Warehouse Automation Automation Use Cases Data Warehouse Migration SCENARIO CHALLENGES OPPORTUNITIES You have an existing data warehouse that is functionally sound but needs to be moved to a new technology platform. Business and logical views of the data warehouse are sound, but the physical differences between databases and data integration platforms are a consideration. Only the physical metadata is reliable because conceptual and logical models and designs haven t kept pace with changes to the existing warehouse. Having the warehouse offline for more than a very brief period is not acceptable. Manually rebuilding schema and data integration processes will take too long and is likely to be error-prone. In addition to moving the schema and processes to a new platform, you also need to move the data. Overcome the inadequacies of documentation by populating the automation tool with physical metadata from the old warehouse and then extracting the design. Rebuild schema and processes for the new platform by generating them with the automation tool. Map old schema to new schema, then build and execute data migration processes to move the data. A quick turnaround from old platform to new, with automated testing and validation, minimizes downtime that the warehouse is not available for business use. TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY 3-15
89 Using Data Warehouse Automation TDWI Data Warehouse Automation Automation Use Cases Re-Architecting the Data Warehouse 3-16 TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY
90 Using Data Warehouse Automation Automation Use Cases Re-Architecting the Data Warehouse SCENARIO Your data warehouse needs a comprehensive architectural update perhaps to overcome long-standing architectural deficiencies, to align with today s best practices, to meet real time data demands, to incorporate unstructured data, or for a variety of other reasons. CHALLENGES OPPORTUNITIES The challenges are similar to those for data warehouse migration. The significant difference is that the architecture not the platform is changing. Obsolete documentation, downtime constraints, and the time and cost to rebuild manually are real challenges. But these challenges are compounded by the probability that re-architecting will bring structural and standards changes that have greater impact than technology platform changes. Migration opportunities to automate apply here too. Extracting the design, generating schema and process, automating data migration, and automating testing and validation are all valuable. New opportunities also exist to embed architectural structures as design patterns and to encapsulate architectural standards as part of automation. TDWI. All rights reserved. Reproductions in whole or in part are prohibited except by written permission. DO NOT COPY 3-17
TDWI strives to provide course books that are content-rich and that serve as useful reference documents after a class has ended.
Previews of TDWI course books offer an opportunity to see the quality of our material and help you to select the courses that best fit your needs. The previews cannot be printed. TDWI strives to provide
More informationData Warehouse Automation A Decision Guide
Data Warehouse Automation A Decision Guide A White Paper by Dave Wells Infocentric LLC Table of Contents Seven Myths of Data Warehouse Automation 1 Why Automate Data Warehousing? 2 The Basis of Data Warehouse
More informationWHITE PAPER. SAS IT Intelligence. Balancing enterprise strategy, business objectives, IT enablement and costs
WHITE PAPER SAS IT Intelligence Balancing enterprise strategy, business objectives, IT enablement and costs Table of Contents Executive summary... 1 SAS IT Intelligence leaping tactical pitfalls... 2 Resource
More informationData warehouse and Business Intelligence Collateral
Data warehouse and Business Intelligence Collateral Page 1 of 12 DATA WAREHOUSE AND BUSINESS INTELLIGENCE COLLATERAL Brains for the corporate brawn: In the current scenario of the business world, the competition
More informationTDWI strives to provide course books that are content-rich and that serve as useful reference documents after a class has ended.
Previews of TDWI course books are provided as an opportunity to see the quality of our material and help you to select the courses that best fit your needs. The previews can not be printed. TDWI strives
More informationImplementing Oracle BI Applications during an ERP Upgrade
Implementing Oracle BI Applications during an ERP Upgrade Summary Jamal Syed BI Practice Lead Emerging solutions 20 N. Wacker Drive Suite 1870 Chicago, IL 60606 Emerging Solutions, a professional services
More informationHYPERION MASTER DATA MANAGEMENT SOLUTIONS FOR IT
HYPERION MASTER DATA MANAGEMENT SOLUTIONS FOR IT POINT-AND-SYNC MASTER DATA MANAGEMENT 04.2005 Hyperion s new master data management solution provides a centralized, transparent process for managing critical
More informationThe following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into
The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material,
More informationAutomated Business Intelligence
Automated Business Intelligence Delivering real business value,quickly, easily, and affordably 2 Executive Summary For years now, the greatest weakness of the Business Intelligence (BI) industry has been
More informationImplementing Oracle BI Applications during an ERP Upgrade
1 Implementing Oracle BI Applications during an ERP Upgrade Jamal Syed Table of Contents TABLE OF CONTENTS... 2 Executive Summary... 3 Planning an ERP Upgrade?... 4 A Need for Speed... 6 Impact of data
More information(Refer Slide Time: 01:52)
Software Engineering Prof. N. L. Sarda Computer Science & Engineering Indian Institute of Technology, Bombay Lecture - 2 Introduction to Software Engineering Challenges, Process Models etc (Part 2) This
More informationAgile Business Intelligence Data Lake Architecture
Agile Business Intelligence Data Lake Architecture TABLE OF CONTENTS Introduction... 2 Data Lake Architecture... 2 Step 1 Extract From Source Data... 5 Step 2 Register And Catalogue Data Sets... 5 Step
More informationCOURSE OUTLINE. Track 1 Advanced Data Modeling, Analysis and Design
COURSE OUTLINE Track 1 Advanced Data Modeling, Analysis and Design TDWI Advanced Data Modeling Techniques Module One Data Modeling Concepts Data Models in Context Zachman Framework Overview Levels of Data
More informationKnowledge Base Data Warehouse Methodology
Knowledge Base Data Warehouse Methodology Knowledge Base's data warehousing services can help the client with all phases of understanding, designing, implementing, and maintaining a data warehouse. This
More informationTDWI strives to provide course books that are content-rich and that serve as useful reference documents after a class has ended.
Previews of TDWI course books are provided as an opportunity to see the quality of our material and help you to select the courses that best fit your needs. The previews can not be printed. TDWI strives
More informationDriving Your Business Forward with Application Life-cycle Management (ALM)
Driving Your Business Forward with Application Life-cycle Management (ALM) Published: August 2007 Executive Summary Business and technology executives, including CTOs, CIOs, and IT managers, are being
More informationBusiness Intelligence
Transforming Information into Business Intelligence Solutions Business Intelligence Client Challenges The ability to make fast, reliable decisions based on accurate and usable information is essential
More informationAgile Manufacturing for ALUMINIUM SMELTERS
Agile Manufacturing for ALUMINIUM SMELTERS White Paper This White Paper describes how Advanced Information Management and Planning & Scheduling solutions for Aluminium Smelters can transform production
More informationData Discovery, Analytics, and the Enterprise Data Hub
Data Discovery, Analytics, and the Enterprise Data Hub Version: 101 Table of Contents Summary 3 Used Data and Limitations of Legacy Analytic Architecture 3 The Meaning of Data Discovery & Analytics 4 Machine
More informationOPTIMUS SBR. Optimizing Results with Business Intelligence Governance CHOICE TOOLS. PRECISION AIM. BOLD ATTITUDE.
OPTIMUS SBR CHOICE TOOLS. PRECISION AIM. BOLD ATTITUDE. Optimizing Results with Business Intelligence Governance This paper investigates the importance of establishing a robust Business Intelligence (BI)
More informationNCOE whitepaper Master Data Deployment and Management in a Global ERP Implementation
NCOE whitepaper Master Data Deployment and Management in a Global ERP Implementation Market Offering: Package(s): Oracle Authors: Rick Olson, Luke Tay Date: January 13, 2012 Contents Executive summary
More informationBuying vs. Building Business Analytics. A decision resource for technology and product teams
Buying vs. Building Business Analytics A decision resource for technology and product teams Introduction Providing analytics functionality to your end users can create a number of benefits. Actionable
More informationQlik UKI Consulting Services Catalogue
Qlik UKI Consulting Services Catalogue The key to a successful Qlik project lies in the right people, the right skills, and the right activities in the right order www.qlik.co.uk Table of Contents Introduction
More informationUS Department of Education Federal Student Aid Integration Leadership Support Contractor January 25, 2007
US Department of Education Federal Student Aid Integration Leadership Support Contractor January 25, 2007 Task 18 - Enterprise Data Management 18.002 Enterprise Data Management Concept of Operations i
More informationBuilding Secure Software at Enterprise Scale
Building Secure Software at Enterprise Scale EXECUTIVE SUMMARY There are innovative methods for performing static analysis of application code that results in secure, higher-quality software at a significantly
More informationBI Dashboards the Agile Way
BI Dashboards the Agile Way Paul DeSarra Paul DeSarra is Inergex practice director for business intelligence and data warehousing. He has 15 years of BI strategy, development, and management experience
More informationManaging TM1 Projects
White Paper Managing TM1 Projects What You ll Learn in This White Paper: Traditional approaches to project management A more agile approach Prototyping Achieving the ideal outcome Assessing project teams
More informationData Warehouse Overview. Srini Rengarajan
Data Warehouse Overview Srini Rengarajan Please mute Your cell! Agenda Data Warehouse Architecture Approaches to build a Data Warehouse Top Down Approach Bottom Up Approach Best Practices Case Example
More informationWhite Paper. An Overview of the Kalido Data Governance Director Operationalizing Data Governance Programs Through Data Policy Management
White Paper An Overview of the Kalido Data Governance Director Operationalizing Data Governance Programs Through Data Policy Management Managing Data as an Enterprise Asset By setting up a structure of
More informationRO-Why: The business value of a modern intranet
RO-Why: The business value of a modern intranet 1 Introduction In the simplest terms, companies don t build products, do deals, or make service calls people do. But most companies struggle with isolated
More informationSoftware Development Process
Software Development Process A software development process, also known as software development lifecycle, is a structure imposed on the development of a software product. Similar terms include software
More informationThree Fundamental Techniques To Maximize the Value of Your Enterprise Data
Three Fundamental Techniques To Maximize the Value of Your Enterprise Data Prepared for Talend by: David Loshin Knowledge Integrity, Inc. October, 2010 2010 Knowledge Integrity, Inc. 1 Introduction Organizations
More informationA discussion of information integration solutions November 2005. Deploying a Center of Excellence for data integration.
A discussion of information integration solutions November 2005 Deploying a Center of Excellence for data integration. Page 1 Contents Summary This paper describes: 1 Summary 1 Introduction 2 Mastering
More informationChapter 6 Basics of Data Integration. Fundamentals of Business Analytics RN Prasad and Seema Acharya
Chapter 6 Basics of Data Integration Fundamentals of Business Analytics Learning Objectives and Learning Outcomes Learning Objectives 1. Concepts of data integration 2. Needs and advantages of using data
More informationBuilding Software in an Agile Manner
Building Software in an Agile Manner Abstract The technology industry continues to evolve with new products and category innovations defining and then redefining this sector's shifting landscape. Over
More informationFoundations for Systems Development
Foundations for Systems Development ASSIGNMENT 1 Read this assignment introduction. Then, read Chapter 1, The Systems Development Environment, on pages 2 25 in your textbook. What Is Systems Analysis and
More informationScalable Enterprise Data Integration Your business agility depends on how fast you can access your complex data
Transforming Data into Intelligence Scalable Enterprise Data Integration Your business agility depends on how fast you can access your complex data Big Data Data Warehousing Data Governance and Quality
More informationCHAPTER 1: INTRODUCTION TO RAPID APPLICATION DEVELOPMENT (RAD)
CHAPTER 1: INTRODUCTION TO RAPID APPLICATION DEVELOPMENT (RAD) 1. INTRODUCTIONS RAD refers to a development life cycle designed Compare to traditional life cycle it is Faster development with higher quality
More informationADVANTAGES OF IMPLEMENTING A DATA WAREHOUSE DURING AN ERP UPGRADE
ADVANTAGES OF IMPLEMENTING A DATA WAREHOUSE DURING AN ERP UPGRADE Advantages of Implementing a Data Warehouse During an ERP Upgrade Upgrading an ERP system presents a number of challenges to many organizations.
More informationIT SERVICE MANAGEMENT: HOW THE SAAS APPROACH DELIVERS MORE VALUE
1 IT Service Management: How the SaaS Approach Delivers More Value IT SERVICE MANAGEMENT: HOW THE SAAS APPROACH DELIVERS MORE VALUE EXECUTIVE SUMMARY Today s companies are very reliant on their technology
More informationIntegrating SAP and non-sap data for comprehensive Business Intelligence
WHITE PAPER Integrating SAP and non-sap data for comprehensive Business Intelligence www.barc.de/en Business Application Research Center 2 Integrating SAP and non-sap data Authors Timm Grosser Senior Analyst
More informationEnabling Data Quality
Enabling Data Quality Establishing Master Data Management (MDM) using Business Architecture supported by Information Architecture & Application Architecture (SOA) to enable Data Quality. 1 Background &
More informationDatamaker for Skytap. Provide full-sized environments filled with up-to-date test data in minutes
Datamaker for Skytap Provide full-sized environments filled with up-to-date test data in minutes Is your testing constrained by environments and data? As applications have become more complex, provisioning
More informationArchitected Blended Big Data with Pentaho
Architected Blended Big Data with Pentaho A Solution Brief Copyright 2013 Pentaho Corporation. Redistribution permitted. All trademarks are the property of their respective owners. For the latest information,
More informationMOF MSF. Unitek. Microsoft Operations Framework. Microsoft Solutions Framework. Train. Certify. Succeed.
Unitek MOF MSF Train. Certify. Succeed. Unitek Fremont 39465 Paseo Padre Pkwy #2900 Fremont CA, 94538 Tel: 510-249-1060 Fax: 510-249-9125 Unitek Santa Clara 1700 Wyatt Dr. Suite 15 Santa Clara, CA 95054
More informationCHAPTER SIX DATA. Business Intelligence. 2011 The McGraw-Hill Companies, All Rights Reserved
CHAPTER SIX DATA Business Intelligence 2011 The McGraw-Hill Companies, All Rights Reserved 2 CHAPTER OVERVIEW SECTION 6.1 Data, Information, Databases The Business Benefits of High-Quality Information
More informationData Virtualization A Potential Antidote for Big Data Growing Pains
perspective Data Virtualization A Potential Antidote for Big Data Growing Pains Atul Shrivastava Abstract Enterprises are already facing challenges around data consolidation, heterogeneity, quality, and
More informationMETA DATA QUALITY CONTROL ARCHITECTURE IN DATA WAREHOUSING
META DATA QUALITY CONTROL ARCHITECTURE IN DATA WAREHOUSING Ramesh Babu Palepu 1, Dr K V Sambasiva Rao 2 Dept of IT, Amrita Sai Institute of Science & Technology 1 MVR College of Engineering 2 asistithod@gmail.com
More informationAdvantages of Implementing a Data Warehouse During an ERP Upgrade
Advantages of Implementing a Data Warehouse During an ERP Upgrade Advantages of Implementing a Data Warehouse During an ERP Upgrade Introduction Upgrading an ERP system represents a number of challenges
More informationBusiness Intelligence
1 3 Business Intelligence Support Services Service Definition BUSINESS INTELLIGENCE SUPPORT SERVICES Service Description The Business Intelligence Support Services are part of the Cognizant Information
More informationHow CFOs and their teams are supercharging financial reporting
How CFOs and their teams are supercharging financial reporting Are your finance operations running smoothly? Today s Chief Finance Officers have an opportunity to take a more visible role in strategic
More informationAgile Enterprise Data Warehousing Radical idea or practical concept?
Agile Enterprise Warehousing Radical idea or practical concept? Larissa T. Moss Method Focus Inc. methodfocus@earthlink.net TDWI South Florida Chapter March 11, 2011 Copyright 2011, Larissa T. Moss, Method
More informationBusiness Agility SURVIVAL GUIDE
Business Agility SURVIVAL GUIDE 1 Every industry is subject to disruption. Only a truly agile business is equipped to respond.* Agile firms grow revenue 37% faster. Agile firms generate 30% higher profits.**
More informationMoving Service Management to SaaS Key Challenges and How Nimsoft Service Desk Helps Address Them
Moving Service Management to SaaS Key Challenges and How Nimsoft Service Desk Helps Address Them Table of Contents Executive Summary... 3 Introduction: Opportunities of SaaS... 3 Introducing Nimsoft Service
More informationData center transformation: an application focus that breeds success
White Paper Data center transformation: an application focus that breeds success Introduction Behind any significant data center transformation is often the act of migrating, relocating, upgrading, or
More informationEnterprise Content Management (ECM)
Business Assessment: A Quick-Reference Summary Intro to MIKE2 methodology and phase 1 The methodology that will be used throughout the specialist track is based on the MIKE2 methodology. MIKE stands for
More informationOutline Business Intelligence Roadmap: The Complete Project Lifecycle for Decision-Support Applications
Outline Business Intelligence Roadmap: The Complete Project Lifecycle for Decision-Support Applications Introduction to the BI Roadmap Business Intelligence Framework DW role in BI From Chaos to Architecture
More informationTEST MANAGEMENT SOLUTION Buyer s Guide WHITEPAPER. Real-Time Test Management
TEST MANAGEMENT SOLUTION Buyer s Guide WHITEPAPER Real-Time Test Management How to Select the Best Test Management Vendor? The implementation of a Test Management system to automate business processes
More informationIs ETL Becoming Obsolete?
Is ETL Becoming Obsolete? Why a Business-Rules-Driven E-LT Architecture is Better Sunopsis. All rights reserved. The information contained in this document does not constitute a contractual agreement with
More informationCONCEPTUALIZING BUSINESS INTELLIGENCE ARCHITECTURE MOHAMMAD SHARIAT, Florida A&M University ROSCOE HIGHTOWER, JR., Florida A&M University
CONCEPTUALIZING BUSINESS INTELLIGENCE ARCHITECTURE MOHAMMAD SHARIAT, Florida A&M University ROSCOE HIGHTOWER, JR., Florida A&M University Given today s business environment, at times a corporate executive
More informationCisco Unified Communications and Collaboration technology is changing the way we go about the business of the University.
Data Sheet Cisco Optimization s Optimize Your Solution using Cisco Expertise and Leading Practices Optimizing Your Business Architecture Today, enabling business innovation and agility is about being able
More informationWhite Paper www.wherescape.com
What s your story? White Paper Agile Requirements Epics and Themes help get you Started The Task List The Story Basic Story Structure One More Chapter to the Story Use the Story Structure to Define Tasks
More informationAPPLICATION LIFECYCLE MANAGEMENT AS A BUSINESS PROCESS
APPLICATION LIFECYCLE MANAGEMENT AS A BUSINESS PROCESS DAVID CHAPPELL SPONSORED BY MICROSOFT CORPORATION COPYRIGHT 2010 CHAPPELL & ASSOCIATES Not too long ago, the bond rating agency Moody s disclosed
More informationIT Operations Management: A Service Delivery Primer
IT Operations Management: A Service Delivery Primer Agile Service Delivery Creates Business Value Today, IT has to innovate at an ever- increasing pace to meet accelerating business demands. Rapid service
More informationWho Doesn t Want to be Agile? By: Steve Dine President, Datasource Consulting, LLC 7/10/2008
Who Doesn t Want to be Agile? By: Steve Dine President, Datasource Consulting, LLC 7/10/2008 Who wants to be involved in a BI project or program that is labeled slow or inflexible? While I don t believe
More informationITIL V3: Making Business Services Serve the Business
ITIL V3: Making Business Services Serve the Business An ENTERPRISE MANAGEMENT ASSOCIATES (EMA ) White Paper Prepared for ASG October 2008 IT Management Research, Industry Analysis, and Consulting Table
More informationKey organizational factors in data warehouse architecture selection
Key organizational factors in data warehouse architecture selection Ravi Kumar Choudhary ABSTRACT Deciding the most suitable architecture is the most crucial activity in the Data warehouse life cycle.
More informationTen steps to better requirements management.
White paper June 2009 Ten steps to better requirements management. Dominic Tavassoli, IBM Actionable enterprise architecture management Page 2 Contents 2 Introduction 2 Defining a good requirement 3 Ten
More informationUsers: The Missing Link in BI Delivery
Users: The Missing Link in BI Delivery George Labelle, Chief Information Officer Mark Henschel, Manager, BI & DW Independent Purchasing Cooperative A Subway Franchisee Owned Organization Sponsored by:
More informationDigital Business Platform for SAP
BUSINESS WHITE PAPER Digital Business Platform for SAP SAP ERP is the foundation on which the enterprise runs. Software AG adds the missing agility component with a digital business platform. CONTENT 1
More informationAbstract. White Paper on Application Modernization 1
Abstract The present day market is loaded with extreme competition that invites only the most innovative and smart organizations to compete together and stay above the water level. Added to this are the
More informationOverview. The Knowledge Refinery Provides Multiple Benefits:
Overview Hatha Systems Knowledge Refinery (KR) represents an advanced technology providing comprehensive analytical and decision support capabilities for the large-scale, complex, mission-critical applications
More informationCRM for Real Estate Part 2: Realizing the Vision
CRM for Real Estate Anne Taylor Contents Introduction... 1 Meet the Challenges... 2 Implementation Approach... 3 Demystifying CRM... 5 Conclusion... 7 Introduction Once the decision to implement a CRM
More informationCHAPTER - 5 CONCLUSIONS / IMP. FINDINGS
CHAPTER - 5 CONCLUSIONS / IMP. FINDINGS In today's scenario data warehouse plays a crucial role in order to perform important operations. Different indexing techniques has been used and analyzed using
More informationSelect the right configuration management database to establish a platform for effective service management.
Service management solutions Buyer s guide: purchasing criteria Select the right configuration management database to establish a platform for effective service management. All business activities rely
More informationData Vault and The Truth about the Enterprise Data Warehouse
Data Vault and The Truth about the Enterprise Data Warehouse Roelant Vos 04-05-2012 Brisbane, Australia Introduction More often than not, when discussion about data modeling and information architecture
More informationWHITE PAPER Get Your Business Intelligence in a "Box": Start Making Better Decisions Faster with the New HP Business Decision Appliance
WHITE PAPER Get Your Business Intelligence in a "Box": Start Making Better Decisions Faster with the New HP Business Decision Appliance Sponsored by: HP and Microsoft Dan Vesset February 2011 Brian McDonough
More informationIncrease Business Intelligence Infrastructure Responsiveness and Reliability Using IT Automation
White Paper Increase Business Intelligence Infrastructure Responsiveness and Reliability Using IT Automation What You Will Learn That business intelligence (BI) is at a critical crossroads and attentive
More informationProcess Methodology. Wegmans Deli Kiosk. for. Version 1.0. Prepared by DELI-cious Developers. Rochester Institute of Technology
Process Methodology for Wegmans Deli Kiosk Version 1.0 Prepared by DELI-cious Developers Rochester Institute of Technology September 15, 2013 1 Table of Contents 1. Process... 3 1.1 Choice... 3 1.2 Description...
More informationThe Information Management Center of Excellence: A Pragmatic Approach
1 The Information Management Center of Excellence: A Pragmatic Approach Peter LePine & Tom Lovell Table of Contents TABLE OF CONTENTS... 2 Executive Summary... 3 Business case for an information management
More informationPresented By: Leah R. Smith, PMP. Ju ly, 2 011
Presented By: Leah R. Smith, PMP Ju ly, 2 011 Business Intelligence is commonly defined as "the process of analyzing large amounts of corporate data, usually stored in large scale databases (such as a
More informationOctober 16, 2009 Florida Chapter Presented by Raphael Klebanov, WhereScape USA Best Practices Building a Data Warehouse Quickly
October 16, 2009 Florida Chapter Presented by Raphael Klebanov, WhereScape USA Best Practices Building a Data Warehouse Quickly Copyright 2009 by WhereScape Software Abstract Key factors that influence
More informationImplementing a Data Warehouse with Microsoft SQL Server 2014
Implementing a Data Warehouse with Microsoft SQL Server 2014 MOC 20463 Duración: 25 horas Introducción This course describes how to implement a data warehouse platform to support a BI solution. Students
More informationETPL Extract, Transform, Predict and Load
ETPL Extract, Transform, Predict and Load An Oracle White Paper March 2006 ETPL Extract, Transform, Predict and Load. Executive summary... 2 Why Extract, transform, predict and load?... 4 Basic requirements
More informationAn Agile Project Management Model
Agile Project Management Jim Highsmith Chapter 5 An Agile Project Management Model We improve effectiveness and reliability through situationally specific strategies, processes, and practices. One of the
More informationAGILE METHODOLOGY IN SOFTWARE DEVELOPMENT
AGILE METHODOLOGY IN SOFTWARE DEVELOPMENT Shivangi Shandilya, Surekha Sangwan, Ritu Yadav Dept. of Computer Science Engineering Dronacharya College Of Engineering, Gurgaon Abstract- Looking at the software
More informationBusiness Process Validation: What it is, how to do it, and how to automate it
Business Process Validation: What it is, how to do it, and how to automate it Automated business process validation is the best way to ensure that your company s business processes continue to work as
More informationTapping the benefits of business analytics and optimization
IBM Sales and Distribution Chemicals and Petroleum White Paper Tapping the benefits of business analytics and optimization A rich source of intelligence for the chemicals and petroleum industries 2 Tapping
More informationTableau Metadata Model
Tableau Metadata Model Author: Marc Reuter Senior Director, Strategic Solutions, Tableau Software March 2012 p2 Most Business Intelligence platforms fall into one of two metadata camps: either model the
More informationBecome A Paperless Company In Less Than 90 Days
Become A Paperless Company In Less Than 90 Days www.docuware.com Become A Paperless Company...... In Less Than 90 Days Organizations around the world feel the pressure to accomplish more and more with
More informationI D C T E C H N O L O G Y S P O T L I G H T
I D C T E C H N O L O G Y S P O T L I G H T Capitalizing on the Future with Data Solutions December 2015 Adapted from IDC PeerScape: Practices for Ensuring a Successful Big Data and Analytics Project,
More informationhow can I deliver better services to my customers and grow revenue?
SOLUTION BRIEF CA Wily Application Performance Management May 2010 how can I deliver better services to my customers and grow revenue? we can With the right solution, you can be certain that you are providing
More informationApplied Business Intelligence. Iakovos Motakis, Ph.D. Director, DW & Decision Support Systems Intrasoft SA
Applied Business Intelligence Iakovos Motakis, Ph.D. Director, DW & Decision Support Systems Intrasoft SA Agenda Business Drivers and Perspectives Technology & Analytical Applications Trends Challenges
More informationJagir Singh, Greeshma, P Singh University of Northern Virginia. Abstract
224 Business Intelligence Journal July DATA WAREHOUSING Ofori Boateng, PhD Professor, University of Northern Virginia BMGT531 1900- SU 2011 Business Intelligence Project Jagir Singh, Greeshma, P Singh
More informationData virtualization: Delivering on-demand access to information throughout the enterprise
IBM Software Thought Leadership White Paper April 2013 Data virtualization: Delivering on-demand access to information throughout the enterprise 2 Data virtualization: Delivering on-demand access to information
More informationBig Data for the Rest of Us Technical White Paper
Big Data for the Rest of Us Technical White Paper Treasure Data - Big Data for the Rest of Us 1 Introduction The importance of data warehousing and analytics has increased as companies seek to gain competitive
More informationProcess-Centric Back Office Transformation
Industry Insights Banking Process-Centric Back Office Transformation Executive Summary By driving back-office efficiency, banks and other financial institutions seek to lower expenses and reduce business
More informationSOA + BPM = Agile Integrated Tax Systems. Hemant Sharma CTO, State and Local Government
SOA + BPM = Agile Integrated Tax Systems Hemant Sharma CTO, State and Local Government Nothing Endures But Change 2 Defining Agility It is the ability of an organization to recognize change and respond
More informationCOLUMN. Planning your SharePoint intranet project. Intranet projects on SharePoint need a clear direction APRIL 2011. Challenges and opportunities
KM COLUMN APRIL 2011 Planning your SharePoint intranet project Starting a SharePoint intranet project, whether creating a new intranet or redeveloping an existing one, can be daunting. Alongside strategy
More information