Applying OpenText Integration Center to Information Integration for the ECM Suite Enterprise Data Archiving
2 Information Integration for the ECM Suite Contents Introduction... 3 Enterprise Data Archiving... 5 Our Solution... 5 Implementing the OpenText Enterprise Data Archiving Solution... 6 Designing an Archiving Process... 8 Simplifying Archiving Through Reuse... 9 About OpenText Integration Center... 10 System Requirements... 10 Solution Components... 10 Access Methods for the ECM Suite... 12 OpenText Global Services... 15 Service Offerings... 15
3 Information Integration for the ECM Suite Introduction The ability to easily consolidate and interpret information from across the organization in a way that is seamless and transparent has unequivocal business value. However, in today s global economy, an ever-expanding proliferation of data sources both structured and unstructured, and in various formats and software applications has brought about a new mindset. No longer is information understood as simply data. It now includes content and other information sources, all of which need to be interpreted, integrated, and made more meaningful to corporate end users. OpenText Integration Center helps organizations fuse traditional data integration and enterprise content integration approaches into a single comprehensive information integration strategy, allowing organizations to realize the true value of their people, processes and information. OpenText Integration Center supports a complete 360-degree view of relevant information across the enterprise, ensuring that all business-critical decisions are based on current and complete information, regardless of its location or format, for the purposes of business intelligence, governance, and process optimization. Serving as the hub of the information system, OpenText Integration Center is a data and content integration-enabling technology that gives organizations the ability to quickly adapt to new and changing business processes. With the power and flexibility to transform information as it moves it from where it is to where it needs to be, Integration Center serves as the central hub to all data- and contentdriven systems. With its unique suite of features that help to reduce development cycles, it offers organizations the ability to break down information silos and cross application boundaries to enable access to data and content by providing a unified view of information located throughout the information ecosystem from leading-edge ERP, ECM and CRM systems to legacy applications.
4 Information Integration for the ECM Suite OpenText Integration Center includes a suite of tools that manage and deliver both structured and unstructured information across the entire business supply chain. It carries out all vital data exchange operations regardless of format, syntax, source, or target from XML support to mainframe connectivity, relational databases to multi-dimensional, on-line analytical processing to enterprise content repositories. Whatever the project, OpenText Integration Center enables tighter integration with any enterprise environment for projects such as Legacy Decommissioning, Data and Content Archiving, Data and Content Migration, and Data Lifecycle Management, as part of an Enterprise Content Management strategy. This document is part of a collection of papers that discuss leveraging the power of OpenText Integration Center for information integration to the OpenText ECM Suite in four business scenarios: Legacy Data Decommissioning, Data Archiving, Content Migration, and Data Lifecycle Management.
5 Information Integration for the ECM Suite Enterprise Data Archiving The challenge The amount of information that companies must manage is growing at an ever increasing rate. A large portion of that information is not needed for day-to-day operations, and it is not desirable to allow inactive data to monopolize valuable and costly resources across IT infrastructures. Simply discarding this information, however, is not a viable solution. Many business drivers require companies to retain legacy information: Corporate compliance Regulatory compliance E-discovery Historical reference Couple these retention drivers with the need to reduce both IT storage costs and the amount of information that an individual business unit must oversee, and it becomes evident that archiving enterprise information is not just important, it is critical. But information archiving is not a simple task. The systems and platforms that generate information are typically in active, day-to-day use and this cannot be compromised by the implementation of an archiving infrastructure. Furthermore, archiving processes involve both active and inactive information located on these business systems and repositories, which are often numerous and varied, including mainframes, ERPs, CRMs, SCMs, and business reporting platforms. In order to build an enterprise repository of archived information, the archiving processes must follow precise extraction rules to ensure that all relevant content and metadata is accurately extracted from its source to the target. The challenge is to schedule archive infrastructure implementation and processes in such a way as to have zero impact on operations. The goal of archiving must be to maximize cost and resource efficiencies, not to complicate business. Our Solution The OpenText enterprise data archiving solution uses rules to differentiate between active and inactive data previously generated by business applications, and relocates archive-ready data into the central OpenText Archive repository. The integrity of the remaining active data is never compromised. OpenText s solution retains all necessary records, ensuring that organizations can fulfill reporting, auditing, and compliance requirements securely.
6 Information Integration for the ECM Suite OpenText Integration Center is the heart of the data archiving solution. Working as the Business Logic Layer, Integration Center archives information into the OpenText Archive repository, or any other target system. A wide variety of flexible database adapters ensures that all of your archive-ready data is and remains easily accessible, regardless of the source and target system location. Implementing the OpenText Enterprise Data Archiving Solution A business scenario Manufacturing company, BestMade Inc., performs rigorous testing, reporting, auditing, and certification on a number of products to ensure that they are market ready and delivered on time. Each stage of product development generates a substantial data trail. Meeting regulatory and compliance requirements means that transaction records need to be retained within a retention schedule or Records Management policy that dictates when a record may be archived. Other retention schedules dictate that product-specific records that have not been referenced in three years need to be moved into the archive. The matrix of archiving triggers is complex and includes, for example: Record retention schedule triggers Seven years after an employee is terminated, related records are archived. Event triggers After closing a contract or the completing a purchase, related records are archived. Workflow or BPM triggers Achieving a certain stage in a workflow process can trigger the archiving action. For example, after an item receives all necessary sign-offs at multiple levels, an approval process is triggered. This in turn triggers an archiving process.
7 Information Integration for the ECM Suite Technical Architecture OpenText Integration Center provides a robust solution for archiving legacy data into the OpenText Archive repository. A Unified Master Repository OpenText Integration Center offers flexibility, enabling easy and accurate consolidation of BestMade Inc. s structured, semi-structured, and unstructured data from any source into the OpenText Archive repository, creating a central, unified storage location for archive-ready business content. From Any Source to the OpenText Archive OpenText Integration Center includes four groups of connector types to ensure connectivity to each of BestMade Inc. s source systems: Web Services Includes an OpenText proprietary connector for the ECM Suite Enterprise Library Services, as well as the ability to interface with all internal and external Web-Service-compliant applications, and to participate in Service Oriented Architectures (SOA). ECM Objects for OpenText Content Server Allows triggering of easy ECM related functions on OpenText CS such as create documents, folders and apply categories and attributes Database Adapters Provide the ability to connect to relational databases. Custom API Connectors Include pre-built connectors for middleware such as: FTP, MQ Series, Lotus Notes, and MS Exchange, and also provides a framework for building connectors against additional APIs.
8 Information Integration for the ECM Suite Text/XML Integration Center natively accesses fixed length files, flat (mainframe) files, delimited text (CSV), and large XML (SAX) files. Designing an Archiving Process OpenText Global Services works with BestMade Inc. to perform a full analysis of their business systems and design a process for extracting and archiving the required data/information. During this analysis, the following information is identified: record types outcome and usage expectations retention policies metadata that will describe records output format Working backwards from this analysis, OpenText and BestMade Inc. determine how best to extract the data from across the enterprise and consolidate it into a single archive. Indirect Data Access If gaining direct access to a live production system is problematic, BestMade Inc. may opt to implement an intermediary process to create a snapshot that Integration Center reads and processes, using extraction rules for archiving. Alternatively, Integration Center can create a request for a snapshot with a batch file or script in this case, the extraction rules are held within the Engine and do not need to be updated in two places. OpenText Integration Center archives information in many output formats, including but not limited to XML, CSV, and other text based formats. Rollback The archiving design process also considers the rollback capabilities of OpenText Integration Center. BestMade Inc. can build in allowances to restore archived data to their active systems. Enriching Data with Metadata As part of the analysis, OpenText and BestMade Inc. determine the metadata required to enrich the records. Using the Integration Center's Engine, OpenText creates a set of rules to ensure accurate extraction. Along with the metadata, OpenText Integration Center applies Records Management (RM) classifications to set retention and maintenance schedules designed to meet regulatory and corporate governance needs for the archived data. OpenText Integration Center uses a set of injection rules to collate the archived records with the appropriate metadata as it is moved into the archive.
9 Information Integration for the ECM Suite Content Access on Demand To give business users a more meaningful and useful view of the data in the archive, OpenText can design a template that transforms the XML into HTML using XSLT, or provide access through a portal such as LiveLink. Making Data Available for Analysis, Aggregation, and E-discovery The archive may not be the final resting place for data. Integration Center can extract the data just as easily as it archives it. BestMade Inc. can use OpenText Integration Center to extract the archive and load it into their data warehouse, where it can be used with BI to perform analysis and aggregation. The company can use this same extraction functionality to support any forensics, such as audits or e-discovery initiatives. Archived data is always accessible. Auditing and Reporting To provide proof of validity and lineage of the archived data, OpenText maintains a record of the archiving process by leveraging Integration Center system logging to generate reports and summaries. For ongoing reporting, OpenText Integration Center creates custom reports from the logs to show only the information BestMade Inc. is most interested in: for example, the number of transactions performed, the success rate, and the state of the data (valid or invalid, as determined by business rules). These reports can be generated as CSV files (for example) and can be stored in the archive with the records to which they refer, thus providing long-term tracking of data movement. Simplifying Archiving Through Reuse Archiving is often an ongoing process. As newer records age, they too will reach a point when they are no longer vital to the business. Once configured, Integration Center continues to run transparently in the background, listening for the triggers or events that will initiate the next archival cycle. End users never have to see or get hands-on with Integration Center, but they can rest assured that the Engine will move their content when and where they need it.
10 Information Integration for the ECM Suite About OpenText Integration Center OpenText Integration Center is a universal integration solution that accesses, transforms, enriches, cleanses, and directs information from across the entire spectrum of enterprise systems and applications. OpenText Integration Center includes a suite of tools that can manage and deliver both structured and unstructured information throughout the business supply chain. It can carry out all vital data exchange operations regardless of format, syntax, source, or target from XML support to mainframe connectivity, relational databases to multidimensional on-line analytical processing. Whatever the project, OpenText Integration Center enables tighter integration within any enterprise environment. System Requirements Currently, OpenText Integration Center natively supports six main platforms in either 32-bit or 64-bit mode: Windows, Sun Solaris, IBM AIX, HP/UX, SUSE and Red Hat Linux. Client Components run only on Windows, while Server components (Integration Center Services) can be installed on Windows or Linux/UNIX platforms. Solution Components OpenText Integration Center offers an integrated set of components that enables organizations to design, deploy, and maintain data transformation and exchange processes. Client Components Designer The core application for Integration Center. It provides an intuitive, graphical scripting environment that lets you easily design complex data transformation processes. User-defined business rules, functions, and procedures are stored as objects in the Repository. Designer lets you create and launch information manipulation tasks, such as retrieving data from heterogeneous sources, populating target data warehouses, and transforming the data using powerful instructions in relational and multidimensional systems. In Designer, you use projects to organize the objects that build data exchange processes. There are four main categories of objects: 1. Remote Objects are objects that can be imported from a remote site to preserve their original structures, including primary and foreign keys. For example, Tables and Views can be imported from relational databases, and hypercubes from OLAP sources. Remote Objects include: Connections, Tables, Views, EXE and SQL Procedures, SQL Functions, Stored Procedures, Web Procedures, Cubes, and XML Documents.
11 Information Integration for the ECM Suite 2. Local Objects are objects stored directly in the Repository. They are processed by Integration Center to help you create more efficient processes and avoid repetitive tasks. Local Objects include Lookup Tables, Variables, Macro, DLL, and Query Functions, Exceptions, and Messages. 3. Integration Center Objects are objects that actually process the data. Datasets allow various source objects to be grouped together. Modules transform data. Processes define a chain of conditional Module executions, and other tasks, in an easy and flexible script. Objects include: Datasets, Running Contexts, Events, Modules, Processes, Loaders, and Exporters. 4. Query Results are objects that do not comply with certain Designer rules. These objects have an invalid status and are listed in the Invalid Objects group. Administration Console The central management tool for Integration Center, which you can use to perform essential administrative tasks, including managing repositories, configuring Services, defining hosts, users, and projects, and importing and exporting projects. Now, can run in native 64 bit on 64bit O/S. Scheduler Provides a complete and flexible scheduling facility so you can plan for Processes to be executed at fixed times or under certain conditions. It also provides full history and audit trail reporting, as well as a detailed analysis of log files. Real-Time Administrator A real-time communication management application that provides an overview of Integration Center Services as well as administration and/or execution threads running on all host machines defined for a particular Integration Center Repository. Server Components The OpenText Integration Center Server Components are Services that run on the server platform: the Engine service, Scheduling service, Polling service, and BW service. Engine The Integration Center Engine is a scalable, multi-threaded transformation engine that brokers information from any source to any target. Integration Center architecture supports distribution and synchronization of data transformation and exchange processes over multiple engines, allowing Integration Center to leverage the power of existing distributed computing resources. Now, can run in native 64 bit on 64bit O/S.
12 Information Integration for the ECM Suite Repository A Neutral Architecture Repository that centralizes process development, creates accurate communication between applications, and stores advanced Designer transformation tools and user-generated Modules. The Repository stores and manages all technical metadata (business rules, data flows) and production metadata (programs, logs). It can reside on any of the following platforms: Oracle Microsoft SQL Server Sybase SQL IBM DB2 Informix MySQL Access Methods for the ECM Suite Any Source, Any Target OpenText Integration Center includes four groups of connector types to ensure connectivity to source systems: Web Services Connectors ECM MetaLink API Connectors Mainframe Intelligent Connectors Database Adapters (ODBC or Native DB drivers) Text/XML Connectors Integration Center Web Services Connectors Integration Center can interface with thousands of systems, by leveraging its Web Services connectivity. This universal and comprehensive Web Services support allows Integration Center to interface with all internal and external Web Services compliant applications. It also allows Integration Center to participate in Service Oriented Architectures (SOA), and integrate other application s functions in Data Integration processes. ECM MetaLinks Integration Center comes with connectors called Metalinks for ECM content repositories, such as the OpenText ECM Suite, Filenet, and Documentum, to name a few. Metalinks specialize in the integration of external information into the OpenText ECM and Archive services platform.
13 Information Integration for the ECM Suite Integration Center API Connectors Integration Center includes several types of intelligent API Connectors, which enable it to connect to applications/systems with very complex database schemas or lacking web services connectivity. They are pluggable metadata bridges embedded in Designer that enable the import of data structures from CASE tools, OpenText Content Server, ERP systems, XML Schema, or Web Services Description Language (WSDL) documents. Integration Center is certified by SAP for both CA-ALE and BW-STA interfaces. The solution also includes pre-built connectors for middleware, such as FTP, SecuredFTP, MQ Series, Lotus Notes and MS Exchange, as well as a framework for building connectors against additional APIs. Mainframe Intelligent Connectors Integration Center includes a set of Mainframe Connectors, which consist of two tiers: a dedicated piece of data access middleware (installed on the host), and an ODBC driver for the specific legacy system. This structure allows Integration Center to extract information from various systems on the mainframe, including VSAM, IMS/DB, Adabas, Image, Allbase, Eloquence, KSAM, and FDGen files. Integration Center Database Connectors Integration Center Database Connectors provide the ability to connect to the following relational databases: Generic ODBC, SQL, Oracle, ESSbase, IBM DB2, Informix, Sybase and Teradata, among others. Integration Center also offers native population of multi-dimensional databases, such as Essbase. With this functionality, you can directly create all hierarchies or members, set all necessary attributes, and load or refresh cubes. Through native access, you do not require an additional staging area or complex multi-layer tools from multiple vendors. There are two advantages to this approach namely much better performance due to the elimination of any staging area, and maintenance of programmatic control of multi-dimensional cubes within the transformation logic. Integration Center Text Connectors Integration Center is also able to natively access files, such as Fixed Length Files (including mainframe flat files), Delimited Files (CSV), or large XML (SAX) files, and allows processing of any complex files (such as EDI, IDoc, or WebLogs). Integration Center gives organizations the ability to access information stored in SAP applications, combine it with data from other sources, and then share it with other systems throughout the enterprise. It delivers native connectivity to extract
14 Information Integration for the ECM Suite SAP data, supports bi-directional data interchange with SAP through the SAP IDoc format, and populates SAP BW with data from other systems.
15 Information Integration for the ECM Suite OpenText Global Services From professional consulting and integration services, to training and education, and support programs, OpenText Global Services assists in every phase of your solution planning, deployment, and management to drive key benefits: Rapid Deployment Capitalize on our experience, methods, and Packaged Service Offerings (see below) combining proven services, utilities, and tools, to realize quick return on investment. Improved Success Draw upon a solid team of professionals (Project Managers, Business Consultants, Technical Consultants, Trainers, Developers, Deployment Specialists) to fortify and support the team in delivering and developing solutions with proven success. Adoption Benefit from proven methodology in Project Management, Solutions Development, and Education to drive adoption of tools that are designed for the business. At OpenText, we're committed to your success. And we're dedicated to maximizing your investment in our technology. Our goal is to ensure that you receive the performance you expect from your software investment, and we apply our skills and dedication fully to that end. Through state-of-the-art software and world-class service, we can provide you with the total solution. Service Offerings OpenText is committed to maximizing your technology investment by effectively leveraging and enhancing the OpenText Suite of Products and Solutions. Our comprehensive education programs and packaged service offerings are designed to meet the needs of all OpenText users, ensuring the long-term success of your deployment. Our goal is to help you develop both the technical know-how and the creative vision it takes to meet your most demanding business challenges. Packaged Professional Services Consulting Services - With a methodology honed over years of deploying integration solutions for enterprise or departmental use, OpenText Integration Center consultants orchestrate the strategic planning, implementation and governance of Integration Center solutions, or deliver rapid implementation and upgrade services to accelerate your return on investment. Our consulting group can help you build solutions that enable you to leverage your investment in our technology, and in existing enterprise systems. Consulting engagements are based on the needs of our customers and include tasks and deliverables defined on a per-engagement basis. The implementation of these services can range from simple modifications to meet specific departmental needs, to enterprise applications that integrate with multiple existing systems. www. o p e n text.com For more information about OpenText products and services, visit www.opentext.com. OpenText is a publicly traded company on both NASDAQ (OTEX) and the TSX (OTC). Copyright 2009 by OpenText Corporation. OpenText and The Content Experts are trademarks or registered trademarks of OpenText Corporation. This list is not exhaustive. All other trademarks or registered trademarks are the property of their respective owners. All rights reserved.
16 Information Integration for the ECM Suite About OpenText OpenText is a leader in Enterprise Content Management (ECM). With two decades of experience helping organizations overcome the challenges associated with managing and gaining the true value of their business content, OpenText stands unmatched in the market. Together with our customers and partners, we are truly The Content Experts supporting 46,000 organizations and millions of users in 114 countries around the globe. We know how organizations work. We have a keen understanding of how content flows throughout an enterprise, and of the business challenges that organizations face today. It is this knowledge that gives us our unique ability to develop the richest array of tailored content management applications and solutions in the industry. Our unique and collaborative approach helps us provide guidance so that our customers can effectively address business challenges and leverage content to drive growth, mitigate risk, increase brand equity, automate processes, manage compliance, and generate competitive advantage. Organizations can trust the management of their vital business content to OpenText, The Content Experts. connectivity.opentext.com Sales: Support: connsales@opentext.com 1 877 359 4866 connsupport@opentext.com 1 800 486 0095 www. o p e n text.com For more information about OpenText products and services, visit www.opentext.com. OpenText is a publicly traded company on both NASDAQ (OTEX) and the TSX (OTC). Copyright 2009 by OpenText Corporation. OpenText and The Content Experts are trademarks or registered trademarks of OpenText Corporation. This list is not exhaustive. All other trademarks or registered trademarks are the property of their respective owners. All rights reserved.