Getting Started with Oracle

Transcription

1 Getting Started with Oracle Data Integrator 11g: A Hands-On Tutorial Combine high volume data movement, complex transformations and real-time data integration with the robust capabilities of ODI in this practical guide Peter C. Boyd-Bowman Christophe Dupupet Denis Gray David Hecksel Julien Testut Bernard Wheeler enterp I %\a* I ^ I I professional oxper PUBLISHING i;ro fr>s$ionni «xoo>t rise 88 BIRMINGHAM - MUMBAI

2 Preface 1 Chapter 1: Product Overview 11 ODI product architecture 13 ODI repository 15 Repository overview 15 Repository location 16 Master repository 16 Work repository 17 Execution repository 17 Lifecycle management and repositories 18 Studio 19 Agent 22 Console 24 Oracle Enterprise Manager 26 ODI key concepts 26 Execution Contexts 27 Knowledge Modules 28 Models 30 Interfaces 31 Interface descriptions 31 Interface mappings 31 Interface flow tab 32 Interface controls 34 Packages and Scenarios 34 Summary 34 Chapter 2: Product Installation 35 Prerequisites 35 Prerequisites for the repository 36 Prerequisites for the Oracle Universal Installer 36

3 Prerequisites for the Studio 36 Prerequisites for the Standalone Agent 37 Installing OD111 g 37 Two installation modes 37 Creating the repository with RCU 38 Installing the ODI Studio and the ODI Agent 50 Starting the ODI Studio for the first time 67 Post installation parameter files review 69 Summary 70 Chapter 3: Using Variables 71 Defining variables 71 Variable location and scope 71 Variable definitions 72 Refreshing variables 73 Variable history 74 Using variables for dynamic information 74 Assigning a value to a variable 75 Setting a hardcoded value 75 Refresh Variable 76 Passed as a parameter (Declare Variable) 76 Referencing variables 77 Variables in interfaces 77 Variables in models 79 Variables in topology 80 Using variables to alter workflows 80 Packages 80 Load Plans 82 Summary 83 Chapter 4: ODI Sources, Targets, and Knowledge Modules 85 Defining Physical Schemas, Logical Schemas, and Contexts 86 Defining physical data servers 86 Defining Physical Schemas 90 Data schemas and work schemas 90 Defining Logical Schemas and Contexts 92 Non-database technologies 94 Reverse-engineering metadata into ODI models 100 Standard reverse-engineering 101 Custom reverse-engineering 102 File reverse-engineering 103 XML reverse-engineering 104

4 Examining the anatomy of the interface flow 105 Example 1: Database and file to database 105 Example 2: File and database to second file 108 Example 3: File to Enterprise Application 110 Importing and choosing Knowledge Modules 112 Choosing Knowledge Modules 112 Importing a Knowledge Module 114 KMs A quick look under the hood 115 Configuring behavior with KM options 117 Examining ODI Interfaces 119 Overview tab 120 Mapping tab 121 Flow tab 123 Quick-Edit tab 125 Summary 126 Chapter 5: Working with Databases 127 Sample scenario description 128 Integration target 128 Integration source 129 Integration mappings 129 Data flow logistics 130 Exercise 1: Building the Load_Customer interface 131 Building the topology 131 Reverse-engineering the model metadata 141 Moving the data using an ODI interface 148 Checking the execution with the Operator Navigator 165 Summary 175 Chapter 6: Working with MySQL 177 What you can and can't do with MySQL 178 Working with MySQL 178 Obtaining and installing the software 179 Overview of the task 179 Integrating the product data 180 Product data target, sources, and mappings 180 Product interface flow logistics 181 Integrating inventory data 182 Inventory target, sources, and mappings 182 Inventory interface flow logistics 183 Using MySql with ODI 183 Adding the MySQL JDBC driver 184

5 Expanding the topology 185 Reverse-engineering revisited 188 Preparing to move the product data 190 Using simulation and execution 197 Moving the inventory data 201 Summary 209 Chapter 7: Working with Microsoft SQL Server 211 Example: Working with SQL Server 211 Overview of the task 212 Integrating the Sales data 212 Source 212 Target 213 Integrations 213 Sample scenario 215 Expanding the ODI topology 215 Setting up the topology 215 Reverse-engineering the Model metadata 219 Creating interfaces and mappings 221 Load Sales Person interface 221 Load Sales Person mapping 223 Automatic Temporary Index Management 227 Load Sales Region interface 229 Checking the execution with the Operator Navigator 232 Execute the Load Sales Person interface 232 Verify and examine the Load Sales Person results 233 Verify and examine Load Sales Region results 236 Summary 237 Chapter 8: Integrating File Data 239 Working with flat files 240 Scope 240 Prerequisites for flat files 240 Integrate the file data into an Oracle table 241 Partner data target, source, and mappings 241 Partner interface flow logistics 242 Step-by-step example 243 Expanding the topology for file handling 244 Integrating the Partner data 247 Creating and preparing the project 255 Creating the interface to integrate the Partner data 256 Running the interface 258 Summary 261

6 Chapter 9: Working with XML Files 263 Introduction to XML 263 Introducing the ODIJDBC driver for XML 265 ODI and its XML driver basic concepts 265 Example: Working with XML files 268 Requirements and background 268 Scope 269 Overview of the task 269 Integrating a Purchase Order from an XML file 269 Creating models from XML files 270 Integrating the data from a single Purchase Order 270 Single order interface flow logistics 272 Sample scenario: Integrating a simple Purchase Order file 274 Expanding the Topology 274 Reverse-engineering the metadata 278 Creating the Interface 280 Creating procedures 288 Summary 293 Chapter 10: Creating Workflows Packages and Load Plans 295 Packages 295 Creating a package 295 Adding steps into a package 297 Adding tools in a package 299 Changed Data Capture 299 Event Detection 299 Files 299 Internet 299 Metadata 300 ODI Objects 300 Plugins 300 SAP 300 Utilities 300 Adding tools to a package 300 Using ODI Tools 300 Retry versus fail 301 Best practice: No infinite loop 302 Generating a scenario from a package 302 Load Plans 303 Serial and parallel steps 304 Objects that can be used in a Load Plan 304 Exception handling 305 Using Packages and Load Plans 307 Summary 307

7 Chapter 11: Error Management 309 Managing Detecting and diverting data errors 310 data errors 310 Data quality with ODI constraints 310 ODI error table prefix 313 Contents of an error table 314 Using flow control and static control 314 Using error thresholds 316 Correcting and recycling data errors 316 Recycling errors and ODI update keys 318 Managing execution errors 319 Handling anticipated errors 319 Causing a deliberate benign error with OdiBeep 320 Handling unexpected design-time errors 321 More detailed error investigation in Operator Navigator 322 Handling unexpected runtime errors 324 Handling operational errors 326 Summary 327 Chapter 12: Managing and Monitoring ODI Components 329 Scheduling with Oracle Data Integrator 329 Overview 330 Illustrating the schedule management user interface 332 Using third-party schedulers 334 Fusion Middleware Console Control 335 Launching and accessing the FMCC 336 Domain 336 Agent 337 Starting and stopping 338 Performance summary 338 Log file visibility and aggregation 339 Visibility 339 Aggregation 340 Repository visibility 341 Session statistics 341 Oracle Data Integrator Console 342 Launching and accessing ODI Console 343 Data Lineage 343 Flow Map 346 Summary 347 Chapter 13: Concluding Remarks 349 Index 351