Data Integrator: Object Naming Conventions

Transcription

1 White Paper Data Integrator: Object Naming Conventions Data Integrator: Object Naming Conventions 1

2 Author: Sense Corp Contributors: Peter Siegel, Alicia Chang, George Ku Audience: ETL Developers Date : October 2004 Based on BusinessObjects Data Integrator version 6.5 and earlier Data Integrator: Object Naming Conventions 2

3 Contents CONTENTS...3 EXECUTIVE SUMMARY...4 OBJECTS...5 PROJECT...5 JOB...6 DATASTORE...6 WORKFLOW...6 DATAFLOW...6 SCRIPTS...7 FLAT FILES...7 TABLES...7 REPOSITORIES...7 MISCELLANEOUS...8 CONCLUSION...9 ABOUT SENSE CORP...10 ABOUT BUSINESS OBJECTS...11 Data Integrator: Object Naming Conventions 3

4 Executive Summary Data Integrator utilizes multiple objects in creating jobs to extract, transform, and load transactional data into data warehouses. To keep these objects organized and easily identifiable, Business Objects recommends you establish naming conventions for each object type. This paper will provide guidance on standard naming conventions used to create Data Integrator objects. This paper is based on BusinessObjects Data Integrator 6.5 and is compatible with all currently supported versions. Objects will contain upper case conventions to identify object owners. Specific Business Objects Data Integrator (BODI) objects and their naming conventions have been included as examples. Also, please note that this document is intended to be a supplement, not a replacement, to existing BODI technical documentation. Data Integrator: Object Naming Conventions 4

5 Objects Data Integrator objects adhere to a hierarchical structure. This structure aids in organization and logical development. The following diagram illustrates the hierarchy: Repository Projects Jobs Work Flows Data Flows R3 Data Flows Datastores Source Database Target Database Project Format: DW_DESCRIPTION _PJ Example: DW_SALES_PJ Projects are the highest level of reusable object in Data Integrator. They allow you to group jobs that have dependent schedules or belong to the same application. DW stands for Data Warehouse. DESCRIPTION is the general application or subject area to which the jobs in the project relate (The sales area in the example about). PJ is an abbreviation for Project. Data Integrator: Object Naming Conventions 5

6 Job Format: TARGETSCHEMANAME_JB Example: SALES_LOAD_JB A job is a group of objects that you can schedule and execute together. Multiple jobs can be included in a single project. When naming a job, include a description of data managed within the job. In the example above, the job loads sales data to a data mart. JB is an abbreviation for Job. When designing jobs, you may want to limit the scope of the job to a particular area of your business, so that you can easily understand the purpose of the job from the name. Keeping the jobs limited in scope also makes it easier to re-use the jobs in other projects. Datastore Format: DB_SOURCENAME_DS Example: ORA_BOBJDEV _DS A Datastore provides connection information for a source or target database. Data Integrator is able to import metadata through the datastore connection. DB stands for database and you can create abbreviations for various databases in your environment. The example ORA refers to an Oracle database. The SOURCENAME could be the schema name of the database. DS stands for datastore. Workflow Format: TARGETTABLE_WF Example: SALES_DIM_WF CUSTOMER_WF Workflows can contain other workflows as well as dataflows. TARGETTABLE refers to the final table in the workflow that is being loaded. In instances where a workflow contains multiple workflows, it can refer to a category or group of tables. In the examples, SALES_DIM_WF hold all the dimension table loads for the SALES data mart application. One of the workflows is the CUSTOMER_WF, which loads only customer data. WF stands for workflow. Dataflow Format: TARGETTABLE_DF Example: CUSTOMER_DF Dataflows are the lowest granularity objects in Data Integrator. Multiple dataflows can reside in a workflow. Dataflows hold the specific query or transformation that is being applied to the source data. TARGETTABLE refers to the specific table that the dataflow is loading. DF stands for dataflow. In the example, the Customer table is being loaded by CUSTOMER_DF. Data Integrator: Object Naming Conventions 6

7 Scripts Format: DESCRIPTION_SC Example: CLEANUP_TEMP_TABLES_SC Scripts can be created in Jobs or Workflows. They are single-use objects and are not stored in the Data Integrator object library. They can be used to declare and assign variables, call a SQL function or perform SQL statements. DESCRIPTION can refer to the function or task that the script is performing or the name of a table that the script is loading. SC stands for script. Note: Descriptive Names could include (SQL update table X, Print Message, etc.) The example describes a script that truncates the temp tables for the job. Flat Files Format: DESCRIPTION_TARGETTABLE_FF Example: ORA_APPS_SALES_DETAIL_FF Flat files are data sources that are not in database table form. They are typically used in dataflows to create tables or as a source of metadata. The example describes the target table that the file will populate from the Oracle application. FF stands for flat file. Tables Format: DESCRIPTION Example: CUSTOMER Tables can be used as a source or target in a dataflow. DESCRIPTION refers to the data content in the table. The example describes the database table that will house CUSTOMER data. Repositories DIUSR1 Data Integrator Developer 1 Repository DIUSR2 Data Integrator Developer 2 Repository DIDEV Data Integrator Development Central Repository DIQA Data Integrator Quality Assurance Repository DIPRD Data Integrator Production Repository Repositories store definitions of Data Integrator objects (i.e. Projects, Jobs, Workflows, and Dataflows) as well as source and target metadata. In a development environment, developers would be assigned their own repository (DIUSR1, DIUSR2). Developers would migrate their code to a central development repository (DIDEV). After unit testing is complete in development, the code is migrated to a quality assurance repository (DIQA) for data testing. Finally, the code is migrated to production (DIPRD). Data Integrator: Object Naming Conventions 7

8 Miscellaneous The following objects are contained within container objects. There is no potential for a naming conflict with these objects from another repository object. There is no restrictive reason to force a naming convention, thus it is helpful to be as descriptive as possible. The following are contained object types: Transforms Descriptive Names could include map, filter, cleanse, update, transpose, etc. Try/Catch Descriptive Names could include Error Type X, Error Number #, etc. Conditionals - Descriptive Names could include Good Record Path, Bad Record Path, etc. Variables* - Descriptive Names could include Global Flag, Local Flag) * NOTE: A centralized Data Integrator competency team should control Variable Names because they can be included in all jobs. This allows flexibility to move workflows between different jobs when load balancing. Load balancing is the process of structuring the job schedule so that all jobs will run in approximately the same time duration.or this is because of the design need of job level global variables and the fact that workflows should have the ability to move into different jobs depending on load balancing needs. Data Integrator: Object Naming Conventions 8

9 Conclusion Business Objects Data Integrator offers a robust environment, which offers multiple objects for quick and easy development of ETL code to create a data warehouse. The use of a common naming convention allows developers to identify and organize various BODI objects, which ultimately saves time and effort spent in development, testing, and debugging situations. Beyond the BODI environment, a consistent naming convention facilitates object identification across all systems in your enterprise. This allows you to more easily work with metadata across all facets of a data warehouse such as data modeling, ETL, and reporting. Data Integrator: Object Naming Conventions 9

10 About Sense Corp Sense Corp is a full services consulting firm that helps clients improve their performance utilizing business intelligence, system integration, and data warehousing solutions. We develop long-term relationships, which bind us to the quality of our work. Our team members combine business and technical expertise as well as structured methodologies to implement common sense solutions. Customer success dictates Sense Corp success. For further information about Sense Corp, visit its web site at: Data Integrator: Object Naming Conventions 10

11 About Business Objects Business Objects is the world's leading business intelligence (BI) software company. Business intelligence enables organizations to track, understand, and manage enterprise performance. The company's solutions leverage the information that is stored in an array of corporate databases, enterprise resource planning (ERP), and customer relationship management (CRM) systems. Popular uses of BI include enterprise reporting, management dashboards and scorecards, customer intelligence applications, financial reporting, and both customer and partner extranets. These solutions enable companies to gain visibility into their business, acquire and retain profitable customers, reduce costs, optimize the supply chain, increase productivity, and improve financial performance. In December 2003, Business Objects completed the acquisition of Crystal Decisions, the leader in enterprise reporting. The combined product line includes software for reporting, query and analysis, performance management, analytic applications, and data integration. In addition, Business Objects offers consulting and education services to help customers effectively deploy their business intelligence projects. Business Objects has more than 24,000 customers in over 80 countries. The company s stock is traded under the ticker symbols NASDAQ: BOBJ and Euronext Paris (ISIN: FR BOB). It is included in the SBF 120 and IT CAC 50 French stock market indexes. Business Objects can be reached at and Data Integrator: Object Naming Conventions 11