Oracle Data Integrators for Beginners. Presented by: Dip Jadawala Company: BizTech Session ID: 9950

Transcription

1 Oracle Data Integrators for Beginners Presented by: Dip Jadawala Company: BizTech Session ID: 9950

2 Please silence your cell phones

3 Overview Introductions Architecture of ODI Topology Manager Designer Operator Security Manager Q & A

4 About Dip Jadawala Senior Technical Consultant with BizTech 6 Years of Data warehousing, ETL, Reporting experience Experience with different ETL tools like ODI, Informatica, Pentaho and OWB. Ad hoc Reporting using OBIEE experience in all phases of the system development and software development lifecycle.

5 About BizTech Leading Mid-Atlantic Oracle Platinum Partner and IT Services firm focused on Oracle Applications and Technology solutions Over 400 successful Oracle implementations over the past 15 years Based in King of Prussia, PA with offices in New Jersey, New York City and Washington DC Service Fortune 500 companies, organizations and government agencies Oracle certified and experienced consultants

6 Client-Centric Practice Areas Oracle Applications - Full Portfolio of Oracle Applications Solutions - Implementation, Upgrade, Migration - Since 1990 MPL6 to R12 Experience - Over 400 successful implementations to date Oracle Applications Oracle Technology BI/EPM Oracle Technology and Business Intelligence - End to end service offering in BI and EPM - Fully staffed team of Data Architects and DBAs - Solid experience in RAC, HA and HS designs - Understand full Oracle technology stack Managed Services and IT Outsource - Remote or Onsite services - Full portfolio of Oracle Applications and Technologies - World-Class Data Center with 24x7 Support - Instant capacity, operational focused business model Managed Services ITO Clients Oracle Software Provider Oracle Software Provider - Full Portfolio of Oracle License Resell - Help Clients Optimize License models - RapidApp BI Software for the agile enterprise - RapidApp Auditor to manage change and GRC

7 Prerequisites 1. The system should include the following installed products: Oracle Database 11g Oracle Data Integrator 11gR1 2. Creation of ODI Master Repositoryand Work Repository has been done.

8 Architecture of ODI

9 Architecture of ODI Oracle Data Integratoris built around a modular repository architecture and accessed in client/server or thin client mode by components that are developed entirely in Java. Repositories. ODI Studio. Weblogic/Application Server.

10 Architecture of ODI Repositories The central component of the architecture is the Oracle Data Integrator Repository. It stores configuration information about the IT infrastructure, metadata of all applications, projects, scenarios, and the execution logs. There are 2 types of repositories in ODI. 1. Master repository. 2. Work repository. Work repositories can be 1 or more. There should be only one master repository talking to all work repositories. According to best practices we should have only 1 master repository and multiple work repositories like DEV, TEST and PROD.

11 Architecture of ODI Master Repository Master Repository consists of 1. Security information including users, profiles and rights for the ODI platform. 2. Topology information including technologies, server definitions, schemas, contexts, languages and so forth. 3. Versioned and archived.

12 Architecture of ODI Work Repository A Work Repository consists of 1. Models, including schema definition, data stores structures and metadata, fields and columns definitions, data quality constraints, cross references, data lineage etc. 2. Projects, including business rules, packages, procedures, folders, Knowledge Modules, variables etc. 3. Scenario execution, including scenarios, scheduling information and logs.

13 Architecture of ODI ODI Studio Administrators, Developers and Operators use the Oracle Data Integrator Studio to access the repositories. This Fusion Client Platform (FCP) based UI is used for administering the infrastructure (security and topology), reverseengineering the metadata, developing projects, scheduling, operating and monitoring executions. ODI Studio provides four Navigators for managing the different aspects and steps of an ODI integration project

14 Architecture of ODI

15 Architecture of ODI Run-Time Agent At design time, developers generate scenarios from the business rules that they have designed. The code of these scenarios is then retrieved from the repository by the Run-Time Agent. This agent then connects to the data servers and orchestrates the code execution on these servers. 2 types of agents comes with ODI. 1. The Java EE. 2. Standalone Agent.

16 Topology Manager Topology Navigatoris used to manage the data describing the information system's physical and logical architecture. Through Topology Navigator you can manage the topology of your information system, the technologies and their data types, the data servers linked to these technologies and the schemas they contain, the contexts, the languages and the agents, as well as the repositories. The site, machine, and data server descriptions will enable Oracle Data Integrator to execute the same integration interfaces in different physical environments.

17 Topology Manager Connection to Master Repository and Topology Manager User snmp1 has been created on database already. You can use any Driver depending on target Database.

18 Topology Manager Work repository connection to Master Repository

19 Topology Manager Work repository connection to Master Repository

20 Topology Manager Work repository connection to Master Repository

21 Topology Manager Agent Setup

22 Topology Manager Agent Setup

23 Topology Manager Start Agent Services Command Prompt agent -NAME=localagent for windows machine Command Prompt./agent.sh NAME=localagent for linux/unix machine

24 Topology Manager SCHEMA Schema where ODI inserts and updates the target tables. WORK_SCHEMA Schema where all the ODI Temporary table ( C$_,I$_ etc ) are created and dropped by ODI.

25 Topology Manager Contexts are bridge between Physical and Logical Architecture. The contexts in ODI are basically labels that you can create as needed. ODI comes with only one context called Global. Typical implementations would actually have three contexts:development, QA, Production.

26 Designer Designer Navigatoris used to design data integrity checks and to build transformations such as for example: Automatic reverse-engineering of existing applications or databases Graphical development and maintenance of transformation and integration interfaces Visualization of data flows in the interfaces Automatic documentation generation Customization of the generated code 3 things in Designer where development is required are Interface Package Scenario

27 Designer Main Tabs on Designer 1. Projects The Projects accordion contains the developments made with Designer Navigator. 2. Models The Models accordion contains the descriptions of the data and applications structures. 3. Others The Others accordion contains the Global User Functions, Variables, Markers, and Sequences. 4. Solutions The Solutions accordion contains the Solutions that have been created when working with version management.

28 Designer Knowledge Module Description Reverse-engineering KM Check KM Loading KM Integration KM Journalizing KM Retrieves metadata to the Oracle Data Integrator work repository Checks consistency of data against constraints Loads heterogeneous data to a staging area Integrates data from the staging area to a target Creates the Change Data Capture framework objects in the source staging area Usage Used in models to perform a customized reverse-engineering Used in models, sub models and datastores for data integrity audit Used in interfaces for flow control or static control Used in interfaces with heterogeneous sources Used in interfaces Used in models, sub models and datastores to create, start and stop journals and to register subscribers. Service KM Generates data manipulation web services Used in models and datastores

29 Designer Connecting to Designer

30 Designer Data store creation in Model

31 Designer You can see Flat File is also reverse engineered in right most snapshot.

32 Designer Interface creation

33 Designer

34 Designer

35 Designer

36 Designer Execution of Interface Right click on the interface and click Execute You can see interface running in Operator.

37 Designer 2 nd example

38 Designer CKM Use

39 Designer Package Creation

40 Designer Scenario Creation

41 Designer Parallel Processing of Interfaces

42 Operator The Operator Navigator has the following accordions: Session List The Session List accordion displays all sessions organized per date, physical agent, status, keywords, and so forth. Hierarchical Sessions The Hierarchical Sessions accordion displays the execution sessions organized in a hierarchy with their child sessions. Scheduling The Scheduling accordion displays the list of physical agents and schedules. Scenarios The Scenarios accordion displays the list of scenarios available Solutions The Solutions accordion contains the Solutions that have been created when working with version management.

43 Operator

44 Operator In Designer Navigator Select the TRG_CUSTOMER datastore. Right-click and selectcontrol>errors... 2records that did not satisfy the FK_CUST_CITY constraint (for example, the CITY_ID value does not exist in the table of cities SRC_CITY table). 7records that did not satisfy the business rule acting on customers under 21 (AGE > 21 constraint). The invalid records were saved into an error table and were not integrated into the target table. Correct the data and Re run it.

45 Operator Scheduling the Scenario You can schedule the scenario either in operator or in Designer

46 Operator Scheduling the Scenario

47 Operator Scheduling the Scenario

48 Operator Update Scheduling in Topology

49 Operator Update Scheduling in Topology

50 Operator View the scheduler in Topology

51 Security Manager Security Navigatoris the tool for managing the security information in Oracle Data Integrator. Through Security Navigator you can create users and profiles and assign user rights for methods (edit, delete, etc) on generic objects (data server, data types, etc), and fine-tune these rights on the object instances (Server 1, Server 2, and so forth)

52 Security Manager User Creation

53 Security Manager Manage Profiles

54 Security Manager Manage Profiles

55 Security Manager Open Designer and drag /drop folder.

56 Security Manager Allow/Deny to appropriate Methods that User can access

57 Security Manager

58 Summary Architecture of ODI is made of Java Programming. Core part is Repository. Topology Manager has source and Target connection details. Designer has interfaces, packages, scenarios, variables, KM, procedures and so on. All the development components. Operator is monitoring tool which keeps the logs of interfaces, packages and scenarios. Security Manager keep the Users and their respective profiles to access different modules of ODI Studio.

59 References

60 Any Questions? Dip Jadawala Senior Technical Consultant Booth #1252 on Main Street