MICHAEL SCHMITZ NOVEMBER 20-22, 2006 NOVEMBER 23-24, 2006 RESIDENZA DI RIPETTA - VIA DI RIPETTA, 231 ROME (ITALY)

Transcription

1 TECHNOLOGY TRANSFER PRESENTS MICHAEL SCHMITZ DATA WAREHOUSING Advanced Design and Implementation Issues ETL FOR THE DATA WAREHOUSE A Template-Driven Approach NOVEMBER 20-22, 2006 NOVEMBER 23-24, 2006 RESIDENZA DI RIPETTA - VIA DI RIPETTA, 231 ROME (ITALY) info@technologytransfer.it

2 DATA WAREHOUSING: ADVANCED DESIGN AND IMPLEMENTATION ISSUES ABOUT THIS SEMINAR This 3 day workshop addresses advanced issues involved in designing and implementing an Enterprise Data Warehouse and introduces a new Data Architecture, Dimensional Normal Form. Dimensional Normal Form combines the strengths of both the normalized and the Dimensional Design methodologies to provide usable, flexible, scalable, and high performing Designs for the Enterprise Data Warehouse. Dimensional Normal Form is used for all levels of the Data Warehouse, ETL, atomic, and analytic levels. Workshop participants will learn the new Data Architecture and study advanced database design techniques. Participants work in teams as they develop solutions for difficult Design problems. Their solutions are then analyzed and simulated for both usability and performance, providing important feedback to the students on their design choices. Attendees will leave with an understanding of the new Data Architecture and appreciation of how different Design choices affect both the usability and performance of both their database and ETL design. The workshop is taught by an experienced designer and implementer of Data Warehouses. WHAT YOU WILL LEARN The Dimensional Normal Form Data Architecture In Depth Enterprise Data Warehouse Design Concepts Advanced Database Design Techniques for Data Warehousing Physical Implementation Tips and Guidelines Pre-requisites: General understanding of Data Warehousing design concepts WHO SHOULD ATTEND Data Modelers Data Warehouse DBAs Data Warehouse Architects and Developers Managers seeking to understand the intricacies of Data Warehouse design Business Intelligence and OLAP designers and developers

3 OUTLINE Introduction Course Objectives Course Agenda 1. Data Warehousing Back- Ground Historical Approaches to Meeting Corporate Analytical Requirements Data Warehouse Architectures Data Warehousing Yesterday, Today, and Tomorrow 2. Data Warehouse Design Basics Modeling Normalization Review The Data Warehouse Design Challenge Dimensional Normal Form Sample Models, Data, Queries, and Reports Fact Tables - Fact Table Traits, Types, and Examples - Fact Table Granularity, Dimensionality, Cubes - Fact Table Content - Fact Table Measures and Additivity Dimension Tables - Dimensional Table Traits, Types, and Examples - Dimension Table Families 3. Business Dimensions in Depth The Date Dimension Family The Time Dimension Indicator attributes Separate Business Dimensions with the Same Entity Base Why not to Snowflake the Query Schema Data Warehouse Dimension Keys (Surrogates Always) Data Quality Enforcement and Reporting 4. Enterprise Warehouse Modeling Example Case Study 5. Tracking Dimension Attribute History Methodology Considerations Dimension Attribute History Worksheet Methods Decision Matrix Case Study 6. Advanced Design Issues Avoiding Semi-Additive Count Measures Using Embedded Dimensions for Performance and Usability Heterogeneous Dimensions and Fact Tables/Vertical Partitioning Hierarchically Related Fact Tables - Consolidation - Surrogate Fact Table Keys Degenerate Dimensions - Transaction Detail Fact Tables - Transaction Summary Fact Tables Handling Large Dimensions - Analytical (Profile) Dimensions - Correlation Dimensions - Dimensions acting as Fact Tables Multi-valued Dimension to Fact Table Relationships - Mix Dimensions - Bridge Tables - Associated Fact Tables - Concatenated String Dimensions Value Range Dimensions Value Band Table Bracketing Attributes Variable-Depth Hierarchies Supporting One to Many Dimension to Dimension Member Relationships Supporting Custom Groups Enabling Percentage Calculations Nulls in the Data Warehouse Multiple Units of Measures Promotion Detail Case Study Distributed Dimensions 7. Multi-National Support Situations Issues Multi-National Calendars Time Zones Multiple Currencies 8. Summary Strategies and Implementation Why Considerations Selection Sort Order Impact Automatic Summary Table Navigation and Maintenance Summary Selection Exercise 9. Physical Design and Performance Tuning Storing Derived Measures Implementing Count Columns Partitioned Fact Tables Partitioning Transaction Detail Dimension Tables Index Types - Btree Indexes - Bitmap Indexes Dimensional Query Optimization - Pair-Wise Joins - Composite Index - Index Intersection Fact Table Indexing Dimension Table Indexing Storage Considerations Performance Tuning 10. Design Exercises

4 ETL FOR THE DATA WAREHOUSE: A TEMPLATE-DRIVEN APPROACH ABOUT THIS SEMINAR Extract, transformation, and load process development (ETL) typically accounts for more than half of the work on a Data Warehouse project. Although complex and challenging a rigorous ETL process ensures data quality and currency thus ensuring Data Warehouse credibility and usefulness. The good news is that using a standardized approach along with proven techniques and templates can exponentially lessen the amount of effort required and can ensure data quality, scalability and performance. This class gives a broad overview of ETL processing for the Data Warehouse and delves into the in-depth issues and considerations involved. The class looks at the increasing need for Real-Time data feeds to the Warehouse and discusses the various methods to meet these needs. It specifically presents and teaches a template-driven approach which quickens development speed and provides completeness. These templates are demonstrated with working Informatica/Oracle code, but can and have been adapted for other ETL tools and database platforms. They are also applicable for hand-coded efforts. CD s containing the code will be distributed to all class attendees. Topics include: Basic ETL Overview Current ETL Architectures ETL Tools and Hand-coding Considerations In-Depth ETL Processing Techniques Near Real-Time Methods Standardized ETL Maintenance Templates BENEFITS OF ATTENDING Gain a thorough understanding of the critical ETL development issues Understand current main-stream ETL architectural approaches Learn in-depth techniques for addressing common development issues including how to develop near Real-Time data feeds Be introduced to using standardized maintenance templates and learn how to apply them to your particular environment Take back working code to jump start your ETL development efforts WHAT YOU WILL LEARN Data Warehouse Architects, Designers and Developers ETL Architects, Designers and Developers Data Warehouse DBAs Managers seeking to understand the intricacies of ETL Development Business Intelligence and OLAP designers and developers working with ETL

5 OUTLINE INFORMATION 1. Introduction 2. The ETL Process Overview 3. ETL Process Detail Data Sourcing and Profiling Target Schema Design Data Mapping Data Extraction Data Validation and Cleaning Data Transformation Data Loading 4. ETL Architectures Hub and Spoke Hardware and Software Components 5. ETL Development Frameworks Build vs. Buy Custom Code Hybrid with selected 3rd party tools ETL Tool based - Code Generators - Engine Based - Database Based 6. Processing Options Custom Code Performance Issues - Cursor-based vs. Set Processing - Piping - Caching - Load Methods - Database Updates ETL Tool Performance Issues - Database stored procedure usage - Database vs. Tool choices A Near Real-Time Case Study 7. The BK-PRO Template-Driven Approach - Intermediate Table Usage - Dimension Processing - Fact Table Processing PARTICIPATION FEE Data Warehousing: Advanced Design and Implementation Issues 1400 ETL for the Data Warehouse: A Template-Driven Approach 1200 Special price for the delegates who attend both seminars: 2400 The fee includes all seminar documentation, luncheon and coffee breaks. HOW TO REGISTER You must send the registration form with the receipt of the payment to: TECHNOLOGY TRANSFER S.r.l. Piazza Cavour, Rome (Italy) Fax within November 6, 2006 PAYMENT Wire transfer to: Technology Transfer S.r.l. - Banca Intesa S.p.A. - Agenzia 3 - Rome Iban Code: IT-34-Y GENERAL CONDITIONS If anyone registered to participate is unable to attend, a substitute may participate in their place. A full refund is given for any cancellation received more than 15 days before the seminar starts. Cancellations less than 15 days prior the event are liable for 50% of the fee. Cancellations less than one week prior to the event are liable for the full fees as invoiced. In case of cancellation of the seminar, Technology Transfer s responsibility only applies to the refund of the participation fees which have already been forwarded. MICHAEL SCHMITZ DATA WAREHOUSING: ADVANCED DESIGN AND IMPLEMENTATION ISSUES ETL FOR THE DATA WAREHOUSE: A TEMPLATE-DRIVEN APPROACH BOTH SEMINARS VENUE Residenza di Ripetta - Via di Ripetta, 231 Rome (Italy) SEMINAR TIMETABLE 9.30 am pm 2.00 pm pm first name... surname... job title... organisation... address... postcode... city... Rome November 20-22, 2006 Residenza di Ripetta - Via di Ripetta, 231 Registration fee: 1400 Rome November 23-24, 2006 Residenza di Ripetta - Via di Ripetta, 231 Registration fee: 1200 Special price for the delegates who attend both seminars: 2400 If anyone registered is unable to attend, or in case of cancellation of the seminar, the general conditions mentioned before are applicable. Stamp and signature " 8. Template Case Study country... telephone... fax Send your registration form with the receipt of the payment to: Technology Transfer S.r.l. Piazza Cavour, Rome (Italy) Tel Fax info@technologytransfer.it

6 SPEAKER Michael Schmitz is an industry recognized expert in Data Warehouse/Data Mart Design and Implementation. He has an in depth understanding of efficient and scalable techniques whether dealing with large-scale Data Warehouse or small-scale, platform constrained Data Mart Implementations. Mr. Schmitz has helped implement Data Warehouses using Informix, Redbrick, Oracle, Teradata, Sybase, DB2, and SQL Server on Mainframe, Massively Parallel, UNIX, and NT platforms. He has worked with small and large Businesses across a variety of industries including customers such as Hewlett Packard, American Express, General Mills, AT&T, Bell South, MCI, Mobil Oil, JD Power & Associates, Oracle Slovakia, and the French Railroad SNCF. He most recently served as Director of Data Warehousing for Oracles Mid-Market Solutions Group and currently is the founder and principal of High Performance Data Warehousing. Mr. Schmitz specializes in high performance Data Warehousing solutions. He speaks at industry conferences and delivers workshops on Data Warehouse Design and Implementation techniques worldwide.