Chapter 6: Data Warehouse Development Methodology. The risks in developing a warehouse can be broken into three broad categories:

Transcription

1 Data Warehousing Risks Chapter 6: Data Warehouse Development Methodology The risks in developing a warehouse can be broken into three broad categories: Technology risk, Project Management risk, and Business risk. Technology risk Technology risk is the risk that the project team just can t get the technologies to work correctly. Perhaps they can t get the data movement tools to work. Perhaps they can t get their query tools to speak to the database. Perhaps they can t get the database to load or deliver data quickly enough. These, and similar issues, represent technology risk. Tools to minimize the level of technology risk are: Experience It is important that at least some of the project team s members have experience in data warehousing and, especially, with tools the project team has selected or, at least, with tools in the same category as the tools to be used. Training Time must be allocated to train individuals in the technologies that they are unfamiliar with. Avoiding untried technologies Frequently, new technologies are fraught with bugs and inconsistencies. If your project intends to use a new technology, make sure that you ve planned for two things: i. A checkpoint, at which you review the performance of the tool and determine whether or not fulfilling the project team s need. ii. A backup plan, should the new technology not perform adequately. Proof of concept It s frequently useful to build quick tests of key pieces of the system to ensure that they will meet expectations. Architecture reviews Don t work in a vacuum. It s a good idea to have an outside party review the warehouse s technical architecture design and express their concerns with it.

2 Project Management Risk Project management risk is the risk that, while you may be able to get the technologies to work, you just can t bring the project in on time or on budget. Tools to overcome project management risk include: Experience Knowledge of data warehousing and the tasks it entails is important to successfully, estimating and managing a data warehousing effort. Methodology A strong methodology should serve as a roadmap to project management and the project team, showing them how to proceed and what guideposts to look for along the way. Dedicated project managers The dedicated project manager s primary job should be to develop and monitor the project s work plan. Tasks expected of a project manager should include: i. Developing initial work plan, ii. Notifying team members of project management expectations including target dates of each task, iii. Monitoring progress on individual tasks, and iv. Allocating resources to address needs as the project progresses. Scope control policies Data warehouse projects should have policies for recording and evaluating changes in scope. Business risk Business risk is the risk that the system, once finished, does not get used. Tools for addressing business risks include: Constant user involvement Interfacing with users is critical not only during requirement gathering but, in fact, throughout the Discovery and Construction phases. During the discovery phase, users should be asked to validate the report and analysis designs to be constructed by the report developers. During Construction, users should review complete reports and provide opinions on their sufficiency. Focus on business process implications

3 When a company implements a new system, the employees have no choice but to use it. Data warehouses are frequently treated quite differently. In many ways, warehouse use is optional. As an optional system, there is a risk that it won t get used. To address this dire possibility, it is recommended that warehouse analysts and designers attempt to develop a clear understanding of how the warehouse will be used and how it will fit into the user s job. Methodology Overview One of the best tools for addressing the risks of warehousing is methodology. A methodology can be viewed as a cookbook for developing warehouse. It should layout the steps you need to perform and provide information to help you plan and budget those steps. The methodology here to develop a data warehouse consists of six phases. Phase Vision Discovery Architecture Construction Implementation Audit and Iteration Goal Determine if the warehouse is justified and define a data warehouse strategy. This step is typically performed just once. It will result in a list of warehouse releases. The remaining steps in the methodology are performed for each of these releases. Get detailed custom/user requirements for one functional area. Design the technologies that solve the needs. Build the system. Roll the system to users. Gather feedback for enhancements. Phase One Vision Vision is an early planning phase. Of all the phases, it is the most flexible: the results of Vision for one type of effort might make no sense at all for another. The general idea behind Vision is to construct a long-term plan for the company s decision support efforts and for its enterprise data warehouse (EDW). Because it delivers a list of releases planned for development, Vision is generally performed once. Vision Goals The goals of Vision are: To determine whether or not an investment in warehousing and related technologies is warranted To determine where warehousing and related technologies can best serve the organization. To create a long-term plan for the enterprise data warehouse (EDW).

4 Key Vision Deliverables Key deliverables of the Vision phase include A vision report, A long-term work plan for the enterprise warehouse effort, and An entity-level data model of the enterprise data warehouse Vision Report The primary deliverable of the Vision phase is a report that documents and summarizes its findings. It s found very useful to create an outline of the final report before the start of the phase. This outline should be shared with the project s key sponsors and senior managers. Together this group can hone the outline. Proposed Vision Findings Outline 1. Executive summary of findings 2. Introduction and target audience 3. IT Infrastructure a. Existing transaction processing systems i. Roles ii. Technologies iii. Issues b. Existing decision support systems iv. Roles v. Technologies vi. Issues c. Planned changes to the system environment 4. Decision support requirements a. Overall Goals b. Sales and Marketing users i. Description of the group and its subgroups ii. Needs iii. Hot buttons c. Finance i. Description of the group and its subgroups ii. Needs iii. Hot buttons d. Distribution users i. Description of the group and its subgroups ii. Needs iii. Hot buttons e. Key entities necessary in an enterprise data model 5. Decision support vision a. Architecture/approach (including discussion of the merits of centralized versus a decentralized approaches)

5 b. Required marts c. Key data sources d. Prioritization long-term implementation plan 6. Approaches and cost estimates a. Discussion of build versus buy issues b. Cost estimates c. First cut of an approach ranking matrix 7. Benefits analysis d. Quantifiable benefits e. Non-quantifiable benefits 8. Next steps 9. Issues Long-term Work plan The long-term plan detail the major releases and projects that must be pursued over the life of that effort. It is generally a high-level document that covers multiple years. One job that this plan performs is to let all sponsors and interested parties plan for the warehouse. A sample Long-term Work plan for a Vision Study Task Name Duration Q1 01 Q1 02 Q1 03 Q1 04 Q1 01 Q1 02 Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Vision 40 days Sales & 120 Marketing release days Finance release 120 days HR release 80 days Enterprise Data Model In this model we try to build at the entity level. In other words we try to capture the key entities and the relationships between them. We don t try to capture every field that will appear in the complete, detailed warehouse. Vision Flow We start by gathering input from the company s manager. This input allows us to develop a list of the areas that must be covered in the warehouse or in a release list. A basic estimate of the time required for each release, provides the information needed to create a long-term work plan. At the same time the project team meets with key figures in the IT and data architecture groups to identify which entities are key to the organization. These entities are linked together into a summary level entity-relationship diagram.

6 Vision phase thus, gives a list of the releases to be covered by the warehouse. The remaining phases, starting with Discovery, are repeated for each release. Senior management s business vision List of initiatives/ releases Long-term work plan Discovery phase (one initiative) List of key entities across enterprise Entity-level EDW data model