WHITE PAPER SAP Exchange Infrastructure and Informatica PowerCenter: Complementary Data Integration Capabilities to Support SAP Data Migration Projects
This document contains Confidential, Proprietary and Trade Secret Information ( Confidential Information ) of Informatica Corporation and may not be copied, distributed, duplicated, or otherwise reproduced in any manner without the prior written consent of Informatica. While every attempt has been made to ensure that the information in this document is accurate and complete, some typographical errors or technical inaccuracies may exist. Informatica does not accept responsibility for any kind of loss resulting from the use of information contained in this document. The information contained in this document is subject to change without notice. The incorporation of the product attributes discussed in these materials into any release or upgrade of any Informatica software product as well as the timing of any such release or upgrade is at the sole discretion of Informatica. Protected by one or more of the following U.S. Patents: 6,032,158; 5,794,246; 6,014,670; 6,339,775; 6,044,374; 6,208,990; 6,208,990; 6,850,947; 6,895,471; or by the following pending U.S. Patents: 09/644,280; 10/966,046; 10/727,700. This edition published June 2006
White Paper Table of Contents Executive Summary..........................................2 Architectural and Application Data Integration Services Components of SAP XI........................................3 SAP Implementation vs. Data Migration...........................4 Addressing the Requirements of SAP Data Migrations: SAP XI and PowerCenter......................................4 Data Analysis/Profiling of Legacy Data.....................................5 Data Quality.........................................................5 Data Volumes........................................................5 Data Transformation/Conversion..........................................6 Legacy Data Access....................................................6 Services-based Approach: SAP XI and PowerCenter..................7 About Informatica...........................................9 SAP Exchange Infrastructure and Informatica PowerCenter 1
Executive Summary When evaluating technology options to meet the needs of enterprise data integration, many CIOs and data architects consider enterprise application integration (EAI). After all, with features to help address many data integration requirements, EAI toolsets seem like the right choice. SAP XI and PowerCenter don't provide competing capabilities when it comes to data migration; they complement one another, providing a comprehensive set of capabilities that ensure efficient and successful SAP implementations. When considering a long-term data integration solution, CIOs and data architects whose technology infrastructure is based on SAP solutions should evaluate SAP Exchange Infrastructure (SAP XI). SAP XI is native SAP data integration technology that offers capabilities common to EAI tools and seamless integration into the SAP technology stack. They should also evaluate Informatica PowerCenter, a single, unified enterprise data integration platform that allows companies and government organizations of all sizes to access, discover, and integrate data from virtually any business system, in any format, and deliver that data throughout the enterprise at any speed. The purpose of this white paper is to compare and contrast the role of SAP XI and PowerCenter in data migration projects. This paper examines the differences between the requirements for integrating business processes and the requirements for integrating data in the context of data migration into SAP. This paper also explores the ways in which the capabilities of SAP XI and PowerCenter complement one another and serve to ensure efficient and successful data migrations and to reduce implementation risks. 2
White Paper Architectural and Application Data Integration Services Components of SAP XI SAP's vision for SAP XI is to have the native data integration technology serve as a hub for integrating both data and business processes to help orchestrate both internal and crossenterprise business events. SAP XI is well suited to support this vision, primarily for its ability to meet transactional- and XMLbased data integration requirements. Figure 1 shows an illustration of SAP XI's architecture and key components. Key capabilities of SAP XI include: Business process management features essential in orchestrating services and cross application business events Document- and message-oriented data integration Data movement-based XML message processing Data transformation based on XSLT Web services-based integration (either a consumer or provider of Web services) Presentation and Analysis (SAP, GUI, SAP Portals) Business Process Management (SAP XI) Application Services Business Process Integration SAP XI Reliable Messaging Event Processing Transaction Processing Message Routing Service Repository Service Container Transactional Data Access (via SAP XI adapters) PDF Adobe Applications Databases Messages Flat Files XML Unstructured Mainframe Data Figure 1: Architectural and Application Data Integration Services Components of SAP XI Examples of business processes requiring data integration supported by SAP XI include: Integrate inventory and procurement systems (e.g., when inventory drops below 500 units, send an order of 300 more to procurement system) Integrate order, financial, and distribution systems (e.g., when each order is entered, generate an invoice and ship the products) Integrate two transactional systems across the firewall (B2B) (e.g., when an order is placed in SAP of Partner A, send a electronic purchase order document to the fulfillment application of Partner B.) : SAP Exchange Infrastructure and Informatica PowerCenter 3
SAP Implementation vs. Data Migration Every SAP implementation involves data migration. Most data migrations into SAP involve more than one source system because organizations are increasing the scope of the business processes supported with SAP solutions. SAP implementations focus on reengineering and evolving enterprise business processes. The challenge during the implementation is to determine how to configure SAP to support each company's unique processes. Data migration is about data readiness. A data migration is not just about moving the data into SAP; it's about aligning the data to the business process within SAP. The data migrated into SAP must be accurate and trustworthy to motivate the business users to transition from their legacy applications and adopt the SAP solution. It's a common assumption that migrating data from legacy applications to SAP is simply another integration task that can be easily handled by SAP XI. However, an efficient and successful migration of data into SAP involves massive amounts of data. This data must be accessed, discovered, cleansed, and integrated before it's migrated to the SAP application. All tasks within a migration-from profiling and analyzing source data, to data transformations and conversionsmust be applied to very large data sets. To best support data migration into SAP, the selected data integration tool should be based on an integrated, metadata-driven architecture to provide visibility into all data integration changes across the entire migration process, including data access, extraction, profiling, transformation, and loading. The data integration tool should provide the following data migration capabilities: Upfront analysis and profiling of the complete data set across all sources of data Extraction and movement of bulk data including both master data and current and historical transactional data-from heterogeneous, legacy applications Set-oriented transformations and conversions, since complex transformations must be applied when data from multiple sources are joined and made to align with the new business process Centrally-managed, native source system access because migration projects require access to a wide range of source systems from homegrown mainframe applications to legacy clientserver applications Data audit and lineage capabilities that enable migration teams to trace and verify data, which gives the user community confidence in the quality of the data and encourages adoption of the new system Addressing the Requirements of SAP Data Migrations: SAP XI and PowerCenter This section describes when and how PowerCenter's capabilities can complement those of SAP XI during data migration a critical phase of an SAP implementation that's often underestimated. Since both SAP XI and PowerCenter extract, convert, and write data to targets, why should two products be considered for SAP data migration? SAP XI and PowerCenter don't provide competing capabilities when it comes to data migration; they complement one another, providing a comprehensive set of capabilities that ensure efficient and successful SAP implementations. 4
White Paper Data Analysis/Profiling of Legacy Data Data profiling is the analysis of data to understand its content, structure, quality, and dependencies. As SAP XI does not offer native data profiling functionality, SAP data migration teams traditionally take a manual approach to profiling legacy data. Manual profiling techniques range from spot inspections of the legacy applications or sample data extracts, to analysis via custom-coded reports or elaborate spreadsheets. These data profiling methods typically sample data in a few key fields to get a sense of what the data is like in these columns, but the results are often inconsistent and incomplete. PowerCenter's data profiling capabilities provide comprehensive and accurate information about the content, quality, and structure of data in virtually any operational system. Organizations can reduce implementation risks and increase data confidence by automatically assessing the initial and ongoing quality of data regardless of its location or type. PowerCenter can help SAP migration teams move beyond wondering if their data is dirty and onto fixing data issues. Data Quality Most organizations attempt to tackle data quality problems by implementing tactical solutions to improve data quality within a single application or within a single business process. While this approach may mitigate the problem for part of the organization in the short term, such limited initiatives generally fail to achieve long-term data quality improvement on a broad scale. Organizations need a solution to improve and maintain data quality enterprise-wide. PowerCenter's data cleansing, parsing, and matching capabilities enable organizations to standardize, validate, enhance, and correct enterprise data such as customer contact information and corporate or third-party data. Powerful, algorithmic data matching capabilities identify relationships between data records for de-duplication or group-based processing. These capabilities: Improve the accuracy of data, giving end users confidence in the quality of their data Increases developer productivity, reducing the effort necessary to cleanse and transform data Maintains data consistency and integrity throughout the enterprise by eliminating data duplication SAP XI excels at architecting and driving enterprise business processes. Data quality is another area which SAP XI does not natively support. By acknowledging the challenge of establishing and maintaining enterprise-wide data quality, organizations can use PowerCenter and SAP XI to model a common set of business rules to drive their data quality initiatives. Data Volumes Legacy data migration projects involve extracting, converting, and importing data from multiple, heterogeneous applications into SAP. When you consider that data migration requires the movement of master data (e.g., material/customer/vendor master, account codes, etc.), as well as the movement of current and historical transactional data (e.g., purchase and sales orders, etc.), the data volumes are massive-millions of rows and hundreds of gigabytes of data. SAP migration teams are under pressure to minimize the amount of time critical applications will be down in order to migrate large data volumes into them. SAP Exchange Infrastructure and Informatica PowerCenter 5
SAP XI's data movement capabilities are based exclusively on XML-based document processing. While beneficial when addressing messaging-based interface scenarios, these capabilities may not be robust enough to support the overhead of converting every row of data-whether that data is coming from flat files, databases, or a business applications-to an XML format. Architected to support bulk movement of data, PowerCenter gives SAP migration teams the confidence that they can meet their limited cutover windows for migration data into SAP. Data Transformation/Conversion Both SAP XI and PowerCenter offer mappings, which are graphical representations of data integration processes, including the source, transformation of data, and target. Differences between the two products arise when examining how mappings can be used for data migration. As SAP XI is built primarily for transactional- and XML-based data integration, its data transformation is based on XSLT. SAP XI's data transformation capabilities rely on creating the XSLT-based transformations within the SAP XI toolset and are ideal for message-to-messagebased conversions. SAP XI does an excellent job in converting data from one XML format to another. However, the transformation requirements of data migration projects are likely to exceed the design parameters of XSLT. It's true that any transformation logic can be supported through custom coding, but this approach can be costly when maintenance and reuse are considered. PowerCenter complements SAP XI by exposing any of its transformations as a Web service. This means that SAP users have the option to run PowerCenter data migration processes within the SAP XI business process management framework. PowerCenter offers SAP data migration teams powerful data transformation capabilities that can: Improve the accuracy of data, giving end users confidence in the quality of their data Address non-existing fields in legacy applications required by SAP Fix name and address duplication Join disparate tables and files (even across disparate applications) Route data to heterogeneous applications in single mapping (e.g., one mapping can target data to SAP, mainframe, database table[s], and spreadsheet[s]) Legacy Data Access SAP XI and PowerCenter provide different methods and approaches for sourcing legacy data. SAP XI relies on a combination of native connectors for SAP APIs, along with a library of thirdparty adapters. These adapters support integration via such message formats and protocols as EDI, flat file, JDBC, and JCA. Any data sourced via SAP XI is always converted to an XML payload prior to processing within SAP XI. PowerCenter, in contrast, does not require all data to be converted to XML; it can access the legacy application data directly. Nor does PowerCenter rely on third-party adapters to get to non-sap data sources. PowerCenter offers an extensive library of connectivity options. Figure 2 illustrates the diversity in the systems, message formats, and protocols natively supported by PowerCenter. 6
White Paper Real-Time Data Sources TIBCO IBM WebSphere MQ JMS SAP MSMQ WEBM Web Services Unstructured Data PDF Word Excel Vertical Standards (e.g., HL7, SWIFT, ACORD) Print Stream BLOBs Any proprietary data format/standard Informatica PowerCenter Enterprise Software Sources Mainframe AS/400 JDE PeopleSoft Siebel SAP SAS Essbase Lotus Notes Across the Firewall/WAN Open and Relational Data Sources Oracle IBM Microsoft Sybase Informatix Teradata Flat Files XML Web Logs Remote Data Access Remote or Outsourced Business Applications Figure 2: PowerCenter Provides Access to All Sources of Enterprise Data Without considering the context for specific data integration requirements, SAP XI and PowerCenter seem to offer redundant levels of data access. However, each product offers unique capabilities that offer specific advantages in the context of data migration. Let's use a real-world business process to showcase these differences: purchase orders (PO). Let's say that purchase orders created by a customer need to be sent hourly to the vendor. Each PO is sent to the vendor via the EDI 850 message format across a VAN and needs to be sourced, parsed, and mapped into the vendor's SAP ERP system. In this case, SAP XI is the ideal data integration tool, since this process is a message-oriented scenario and involves EDI, which is a source type supported by third-party SAP XI adapters. Let's look at another example. This one involves six years of historical purchase order data that need to be migrated from disparate legacy applications based on relational databases, mainframes, and flat files. Here PowerCenter complements SAP XI. PowerCenter provides native access to challenging sources of data like mainframe applications. PowerCenter can perform bulk movement of large data volumes in a limited window of time. Also, PowerCenter can perform referential integrity checks for example, PowerCenter can ensure that each PO has valid vendor master and material master values and that each vendor master address entry is deduped and cleansed. This process can be supported in a single mapping with codeless PowerCenter transformation objects. Trying to support this type of scenario using only SAP XI would be challenging and time-consuming, requiring significant code to be created to address performance and transformation requirements. Services-based Approach: SAP XI and PowerCenter SAP XI is the enabling technology for the Enterprise Services Architecture (ESA), SAP's blueprint for a Service-oriented Architecture (SOA). The purpose of the ESA is to show SAP architects/developers how to use of SOA/Web services technology to better conform to business goals. 1 1 Mattern, Thomas and Matthias Haendly. "ESA: A 2005 'Business-Savvy' Take on SOAs," Integration Developer News, February 9, 2005. SAP Exchange Infrastructure and Informatica PowerCenter 7
PowerCenter's data services-based approach supports the vision of SAP's ESA. Each data integration requirement specific to SAP data migrations supported by PowerCenter can be exposed to SAP XI via a data service. Building on SAP XI's architectural and application data integration services components shown earlier in Figure 1, Figure 3 shows how PowerCenter's data services complement and co-exist with SAP XI. Presentation and Analysis (SAP, GUI, SAP Portals) Business Process Management (SAP XI) Data Migration Services Application Services Enterprise Data Services PowerCenter Data Integration Services for Data Migration Business Process Integration SAP XI Data Cleansing Data Profiling Reliable Messaging Event Processing Data Transformation Data Lineage Transaction Processing Message Routing Data Movement Data Quality Service Repository Service Container Data Access for Migration Transactional Data Access (via SAP XI adapters) PDF Adobe Applications Databases Messages Flat Files XML Unstructured Mainframe Data Figure 3: Enterprise Data Services Components Illustrate Complementary Data Integration Between PowerCenter and SAP XI Just as SAP XI would be used as an information hub to orchestrate services for transactional and message-oriented business processes, it can also call out to any PowerCenter data service. Figure 4 shows how an SAP XI business process flow can call out to a PowerCenter data service for moving bulk legacy data or facilitating complex data transformations based on set based aggregations, for example. SAP XI Business Process Flow WS Tranformation Request WS Tranformation Response PowerCenter Web Service Provider S F(x) T PowerCenter Data Migration Service Figure 4: Detail of SAP XI Business Process Calling Out and Consuming a PowerCenter Data Service 8
White Paper Conclusion The chapter on "Analyzing and Cleansing Legacy Data" from SAP's online manual (available at www.help.sap.com) states that the analysis of legacy data is critical for successful migration into SAP and has the reader consider the following questions: Which data exists? How is the data structured (e.g., length, sequence)? Which data can be transferred unmodified, which must be converted, and which cannot be transferred at all? It does not elaborate how this analysis is to be done across the heterogeneous landscape of existing legacy applications. The assumption is that identifying and preparing the legacy data for input into SAP is the responsibility of the SAP implementation team. Assumptions such as this one can lead SAP implementation teams to underestimate the scope and effort required to successfully complete the data migration effort critical for any new or ongoing SAP implementation phase. While SAP XI is suited for transactional- and message-oriented-based data integration requirements, SAP's native data integration technology can be complemented to extend its value for data migration phases in SAP implementations. Informatica PowerCenter offers robust data integration and transformation capabilities as data services that can be exposed as Web services and leveraged on demand by SAP XI to ensure efficient and successful data migrations and reduce implementation risks. To find out more about PowerCenter can help you in your next SAP data migration project, please visit us at www.informatica.com or call us at (800) 653-3871. About Informatica Informatica Corporation delivers data integration software and services to solve a problem facing most large organizations: the fragmentation of data across disparate systems. Informatica helps organizations gain greater business value from their information assets by integrating their enterprise data. Informatica's open, platform-neutral software reduces costs, speeds time to results, and scales to handle data integration projects of any size or complexity. With a proven track record of success that extends back to 1993, Informatica helps companies and government organizations of all sizes realize the full business potential of their enterprise data. That's why Informatica is known as the data integration company. SAP Exchange Infrastructure and Informatica PowerCenter 9
Worldwide Headquarters, 100 Cardinal Way, Redwood City, CA 94063, USA phone: 650.385.5000 fax: 650.385.5500 toll-free in the US: 1.800.653.3871 www.informatica.com Informatica Offices Around The Globe: Australia Belgium Canada China France Germany Japan Korea the Netherlands Singapore Switzerland United Kingdom USA 2006 Informatica Corporation. All rights reserved. Printed in the U.S.A. Informatica, the Informatica logo, and, PowerCenter are trademarks or registered trademarks of Informatica Corporation in the United States and in jurisdictions throughout the world. All other company and product names may be tradenames or trademarks of their respective owners. J50949 6732 (06/26/06)