TDWI Data Integration Techniques: ETL & Alternatives for Data Consolidation

Transcription

1 TDWI Data Integration Techniques: ETL & Alternatives for Data Consolidation Format : C3 Education Course Course Length : 9am to 5pm, 2 consecutive days Date : Sydney Nov 2011, Melbourne Nov 2011 Venue : Syd / Melb - TBC Cost : Early bird rate $1,998 excluding GST per participant (valid until 10 Oct, 2011) : Regular rate $2,200 excluding GST per participant : Discounts available for team attendance Inclusions : Morning tea, lunch & afternoon tea both days : Course workbook & presentation notes Overview Data integration is becoming increasingly complex as new expectations and technologies change the face of data warehousing and business intelligence. Design of data integration systems was comparatively straightforward when extract, transform, and load (ETL) was the only option. In today's world, the demand for real-time and right-time data increases expectations, while scorecards and dashboards increase visibility. Simultaneously, enterprise information integration (EII), enterprise application integration (EAI), master data management (MDM), and customer data integration (CDI) technologies expand the range of possibilities. This course teaches techniques and skills to build data integration systems that can meet today s needs and evolve to meet demands of the future. Starting with the right requirements, using the right technologies, and designing for adaptability are central themes throughout the course. Learn > Analysis techniques to capture data integration requirements, including those for source data, data consolidation, data quality, data granularity, data currency, and historical data > How the alphabet soup of integration technologies - ETL, EII, EAI, MDM, and CDI - fits into overall data integration architecture > Design techniques for the mainstream of data integration, including source-to-target mapping, source data capture, data transformation and cleansing, and database loading > Techniques to enrich the data integration design with processes for automated scheduling, execution monitoring, metadata capture, restart and recovery, and more > Tips to design for the complex issues of data integration, including detecting data changes, identifying data quality defects, managing complex schedule dependencies, meeting real-time data demands, and more.

2 Ideal for > Business intelligence and data warehousing architects > Data integration process designers and developers > Business intelligence and data warehousing program and project managers. Presenter: Michael Gonzales Michael L. Gonzales, CBIP, has been a chief architecture and solutions strategist for over a decade. Michael specialises in the formulation of BI strategy for competitive advantage, risk management and valuation for BI, and conducts research into industry best practices and product assessment. He is an independent consultant, Ph.D. candidate at the University of Texas, and a successful author. His most recent paper is "Risk and IT Factors that Contribute to Competitive Advantage and Corporate Performance." Registration Please register your interest on the Education page to secure your place and receive date confirmation notifications. About TDWI TDWI, a division of 1105 Media, is the premier provider of in-depth, high-quality education and research in the business intelligence and data warehousing industry. Starting in 1995 with a single conference, TDWI is now a comprehensive resource for industry information and professional development opportunities. TDWI sponsors and promotes quarterly World Conferences, topical seminars, onsite education, a worldwide Membership program, business intelligence certification, resourceful publications, industry news, an in-depth research program, and a comprehensive website, 2

3 Course Detail: TDWI Data Integration Techniques: ETL & Alternatives for Data Consolidation Module One - Data Integration Concepts The Need for Data Integration Why We Integrate Data A Projects Perspective The Challenges of Data Integration Understanding Data Sources Choosing the Right Data Sources Data Quality Data Availability Data Integration Architectures Integration Hub Integration Bus Integration Services Data Integration Projects Kinds of Projects Project Activities Data Integration Technologies Extract-Transform-Load (ETL) Enterprise Information Integration (EII) Enterprise Application Integration (EAI) Master Data Management (MDM) and More Module Two - Requirements Analysis for Data Integration Integration Requirements Concepts Source Data Requirements An Overview Kinds of Data Sources Evaluating Data Sources Source Data Analysis and Profiling Choosing Data Sources Data Unification Requirements Subject Orientation Entity Consolidation Identity Consolidation Relationship Consolidation Attributes and Values Consolidation Data Aggregation and Summary Requirements Levels of Detail Data Quality Requirements Data Correctness Timeliness Data Integrity Data Capture Requirements Frequency of Data Capture Collecting Historical Data Level of Detail 3

4 Audit, Balance and Control Requirements ABC s of Data Integration Metadata Capture Requirements Data About Integration Processes Service Level Requirements Meeting Expectations Module Three - Data Integration Functional Design Functional Design Concepts Source/Target Mapping Mapping Techniques Entity Mapping Data Store Mapping Data Element Mapping The Full Set of Data Elements Data Capture Design and Specification An Overview Kinds of Data Push vs. Pull All Data vs. Changed Data Changed Data Detection Data Extraction Data Replication Transaction Logging Messaging Storing Captured Data Data Transformation Design and Specification Kinds of Transformations Data Selection and Filtering Conversion and Translation Derivation and Summarization Identifying Transformations Specifying Transformation Logic Data Cleansing Design and Specification Detecting Data Quality Defects Repairing Data Quality Defects Quality Metadata and the ABCs of Cleansing Identity and Key Management De-Duplication Surrogate Key Assignment Design for Integrated Data Delivery Choosing the Right Delivery System Data Integration Process Design Requirements Driven Processing Module Four - Data Integration Technical Design Technical Design Concepts Comprehensive Processing Design Data Flow Design 4

5 Moving Data through the Integration Pipeline Data Capture and Data Staging Transformation Processes Transformation Sequence and Dependencies End-to-End Data Flow Work Flow Design Extending Data Flow with Events Service Level Design Performance and More Process Management Design Metadata Capture and Event Logging Balancing and Audits Error and Exception Handling Communication Module Five - Construction, Deployment, and Operation Construction, Deployment, & Operations Concepts Building Data Integration Systems Tools and Technology Standards, Frameworks, Templates, and Reuse System Management and Data Integration System Testing and Data Integration Implementing Data Integration Systems One-Time Data Consolidation Ongoing Data Consolidation Operating Data Integration Systems Integration System Operations Customer and User Support Change Management Module Six - Summary and Conclusion Best Practices in Data Integration Learned through Experience References and Resources o For More Information 5