FileTek, Inc. 9400 Key West Avenue Rockville, Maryland 20850 USA Phone: 301.251.0600 Fax: 301.251.1990 E-Mail: info@filetek.com Web: www.filetek.com The StorHouse Database Extension and Relational Repository for mysap BI BW
2003 FileTek, Inc. All rights reserved. FileTek and StorHouse are U.S. registered trademarks of FileTek, Inc. Other trademarks included herein are the properties of their respective owners. The following U.S. patents protect StorHouse: 4,864,572; 5,247,660; 5,727,197; 6,049,804.
Table of Contents Executive Summary... 1 Introducing the StorHouse Solution... 2 StorHouse Operation... 2 SAP BW and StorHouse Advantages... 2 SAP Interface for StorHouse... 3 StorHouse as an Alternative to ADK... 4 Determining an Appropriate SAP BW Environment for StorHouse... 7 Data Volume... 7 Business Rules Implemented in the Current R3 and SAP BW Environments... 7 Scope and Goals of the mysap BI Environment... 8 Industrial, Institutional, and Legally Imposed Data Retention Requirements... 8 Business Needs Retention Requirements... 9 SAP BW Design Considerations for StorHouse... 9 Implemented and Planned ODS and InfoCube (ODS/IC) Designs... 10 Acceptable Response Time for Accessing Aged Data... 10 Key Factors in a Successful StorHouse/SAP BW Implementation... 10 In Conclusion... 11 FileTek, Inc. iii
iv FileTek, Inc.
The StorHouse Database Extension and Relational Repository for mysap BI BW Executive Summary As requirements for databases grow in size, it is becoming more difficult for SAP Business Information Warehouse (BW) users to store and manage the vast amount of less frequently accessed aged data. Hence, SAP BW users in all market sectors need innovative and affordable strategies to keep their aged data readily accessible. Enterprises using SAP R3 and mysap BI typically deploy SAP s Archive Development Kit (ADK) to offload and manage their aged data. In this environment, optical disk is often the archive storage medium of choice. While an ADK/optical strategy can accomplish enterprise archiving objectives, there are alternative ways to manage the data that are not only faster and more cost-effective, but also more sensible to implement from a business perspective. These methods offer transparent access to all the data and support data warehouse user requirements for data access over time. FileTek s StorHouse data management system is an SAP BW 3.0b-compliant alternative data management strategy that provides an economical, seamless, and efficient database extension to SAP BW. StorHouse uses integrated relational technology to enable speedy and direct SQL access to aged SAP BW data on a blend of StorHouse-managed media including high-speed RAID, IDE disk, optical, and tape. The result is faster data access with lower total storage and data management costs. This paper introduces StorHouse and explains how it can be used as an alternative to an archiving strategy and a relational database extension to SAP BW. It also discusses the issues and methodologies associated with successfully integrating StorHouse and SAP BW, including: The concept of aging data and its impact on the mysap BI s primary database performance and information storage Why StorHouse is increasingly becoming the preferred alternative to ADK FileTek, Inc. 1
The concept and advantages of using alternative storage for database extension How to determine the appropriateness for a StorHouse/SAP integration based on business/technical needs In addition, this paper describes the following benefits of the StorHouse database extension and relational repository solution: Faster SAP BW performance Ability to run all reports (including R3) using SAP BW as a centralized data repository Up to a 70 percent reduction in data storage and management costs Improved system availability Fast, complete access to aged data Introducing the StorHouse Solution StorHouse is FileTek s integrated storage and data management system for storing and rapidly accessing gigabytes to petabytes of relational and non-relational enterprise data. StorHouse technology combines Open System processors and industry-leading storage devices including high-speed RAID, low-cost IDE disk, optical jukeboxes, and nearline automated tape libraries with specialized storage management and relational database management system components. StorHouse Operation StorHouse automatically manages data, devices, and media on all layers of the storage hierarchy. In addition, unlike conventional hierarchical storage management (HSM) systems, it provides direct SQL row-level access to data on all storage layers, including tape. With StorHouse as a database extension to SAP BW, there is no need to reload data into the primary SAP database before it can be accessed relationally. Instead, users can seamlessly drill down to detailed aged data that was migrated to cost-effective alternative storage media. SAP BW and StorHouse Advantages SAP selected FileTek as a partner because it recognized the benefits of having relational access to aged data on StorHouse-managed alternative media. These benefits include: Off-the-shelf seamless integration with SAP BW Up to a 70 percent reduction in data storage and management costs over exclusively disk-based systems when factoring in personnel and hardware expenses Enhanced SAP BW performance for large-scale systems Fast access to aged data 2 FileTek, Inc.
Cost savings and performance gains Automatically secure aged data, with no additional action required from BW Figure 1 lists the benefits of the StorHouse database extension and relational archive for SAP BW. Faster SAP BW Performance Lower Data Storage Costs Improved System Availability No loading and unloading of data to the primary database management system Promotion of faster business processes such as analysis, regulation compliance, and marketing Reduced burden on mysap BI processors because aged data is migrated to StorHouse Faster load and query times Diversified economical storage hierarchy (addition of alternative storage versus RAID) Reduced hardware overhead CPU, memory, etc. Fewer database administrator resources required Continued cost-effective system growth Direct SQL relational access to data migrated out of SAP BW Faster load and query times No downtime for backups Figure 1 StorHouse Benefits SAP Interface for StorHouse SAP developed an interface for StorHouse to provide seamless product integration. The interface supports SAP BW versions 3.0x and higher and is also available by request for SAP BW 2.x. SAP BW administrators access this interface through an easy-to-use, frontend graphical user interface (GUI). Using this GUI, they: Select the data to be migrated Create the corresponding SAP BW database, tables, and indexes on StorHouse Migrate data from SAP BW to the appropriate StorHouse tables Establish the database links that enable mysap BI queries to directly access the migrated data Once data is migrated to StorHouse, it is always available for direct SQL access. This capability enables users to perform data drill downs and/or full or selective data reloads. The SAP interface for StorHouse also supports the migration of SAP BW ODS objects and InfoCubes to StorHouse (complete or partial migration). If users select partial migration, the interface creates a StorHouse relational table as a unique SAP BW data object with the same characteristics as the original SAP BW table but with a new userdefined name. FileTek, Inc. 3
Figure 2 illustrates how users access data using SAP BW OLAP queries. BW Application Server OLAP Query ( BEx Crystal Reports arcplan Cognos Business Objects Brio etc ) ODS MultiCube InfoProvider MultiProvider Remote Cube InfoSet DataManager InfoCube ODS Oracle/DB2 Oracle/DB2 SQL Drill Down StorHouse Relational Archive Figure 2 Data Access StorHouse as an Alternative to ADK StorHouse is more efficient than ADK for managing aged data. Furthermore, it enables SAP BW data management functionality previously not possible. Unlike ADK, StorHouse is a database extension to SAP BW. It permits SAP BW users to submit direct SQL queries against aged data on any StorHouse-managed media, including affordable IDE disk, optical, and tape. In contrast, ADK users must always reload and re-index archived data before it can be accessed relationally. This time- and labor-intensive process results in slow response times and high data management, storage administration, and personnel-related costs. 4 FileTek, Inc.
Figure 3 illustrates the efficiency of StorHouse compared to ADK for data retrieval. As indicated, StorHouse requires only one step to access aged SAP BW data while ADK takes eight. This is based on the assumption that ADK requires no restarts due to initial incorrect data retrieval (see steps 5a and 5b). With restarts, the retrieval process must begin again, lengthening the entire query procedure. StorHouse Process User: Directly query the StorHousemanaged BW data object with a BW query object ADK Process Tape 2 3 4 DBA Task: 1 Select the archive file User talks to that contains DBA the required data using the archive administrator DBA Task: Perform step 8 and repeat the process at step 2 5b No DBA Task: DBA Task: Copy the Reload the original BW archive file data object to into the new create a new data target target User Task: Verify loaded data accuracy 5 DBA calls user saying data is ready for query 5a Primary DBMS 8 DBA Task: Delete the new data target to reclaim the space Yes 6 User Task: Access the new data target with a BW query object 7 User calls DBA to delete Figure 3 The Different Steps to Access Data using StorHouse and ADK Apart from performance, the ADK process requires more data management staff and more disk and computing resources than the StorHouse process requirements that set FileTek, Inc. 5
the stage for escalating IT budgets. Every additional step converts to additional costs with lower returns to data access. StorHouse is the best alternative to the ADK archiving method because it provides: Enhanced mysap BI BW performance (load and query times) for large-scale systems Reduction in data management costs over disk-only storage Transparent user access to StorHouse-managed detail data, both current and historical Timely access to specific data Access to data precisely when it is needed, resulting in a higher level of user satisfaction Better value, because StorHouse segregates volumes of data without any disconnect to analytics. This unique combination enhances BI system performance while simultaneously providing seamless access to all aged data. Automatic data loads? Minimal administration requirements Little to no end-user training requirements Figure 4 illustrates why StorHouse is the preferred strategy for storing and managing aged SAP BW data. StorHouse enables SAP BW users to capture the economic benefits of early migration while extending the power of SAP BW to all aged data. Decreasing TCO Frequency of access reads Move data to StorHouse updates Age of data Figure 4 SAP BW Data Aging Strategy 6 FileTek, Inc.
Determining an Appropriate SAP BW Environment for StorHouse Common characteristics identify which SAP BW systems are most appropriate for use with StorHouse. These characteristics include: Data volume of the primary database system Size of Data Targets Business rules implemented in the current R3 and SAP BW environments Scope and goals of the SAP BW environment Industrial, institutional, and legally imposed data retention requirements Business data retention needs based on data access patterns and frequencies The following sections provide high-level guidelines to help determine if StorHouse is suitable for a particular SAP BW deployment. They are presented for analytical purposes only. As with all technology endeavors, further qualification by SAP, a systems integrator, or FileTek is necessary to ensure applicability for systems architecture and business requirements. Data Volume StorHouse is the optimal relational data management solution for SAP BW versions 3.0x and higher environments with a data volume of at least (or nearing) one terabyte. StorHouse is designed to easily scale from this level into petabytes with the same fast relational access across different storage media. Size of Data Targets InfoCubes and ODS objects, which are all relevant physical objects when modeling a data model and loading data, can become bloated over time. StorHouse relieves these data targets by reducing their size and enhances them by increasing response. Business Rules Implemented in the Current R3 and SAP BW Environments In SAP R3 environments, data management parameters are generally time-based (SAP time periods) and data source controlled. In traditional SAP BW deployments, end-users access detailed data from the R/3 OLTP system, or they run transactional reports on the OLTP environment. The SAP strategy is to merge all process and management reporting from SAP BW and store detailed data at the ODS layer. This strategy has made the management of large data volumes an even bigger concern for corporations and the data warehouse administrators who serve them. FileTek, Inc. 7
Current BW design trends support an architecture where the ODS becomes the data retention layer, or the normalized data store, which provides the detailed list reporting required for operational reporting. The InfoCube becomes the data warehouse layer and stores de-normalized data in a more summarized manner, primarily for management analytics. A clear pattern emerges if you align this architecture with the fact that close to 90 percent of all corporate analytics are conducted on data that is less than 12-18 months old. Administrators must plan a data archiving system that enables all analytics to run efficiently without sacrificing primary data warehouse performance. In a well-architected data warehouse, executives and business users can analyze data for trends and patterns over a specific time period. However, because of storage costs and performance considerations, aged data is typically archived out of BW and stored on offline media. Users must reload this data before they can query it. In contrast, StorHouse permits frequent loads by data target and provides SQL access to all data migrated to StorHouse on any storage media, including tape. With StorHouse, the location of aged data is transparent to queries, which still use all BW business rules and intelligence. In addition, in most cases, StorHouse can also lower data management costs (hardware and database administrator) by allowing the removal of some disk-based storage systems. The time-period criteria established within R3 and SAP BW can help determine when to migrate data to StorHouse. Refer to the section, Implemented and Planned ODS and InfoCube (ODS/IC) Designs, on page 10 for more information about migration. Scope and Goals of the mysap BI Environment Database administrators generally address data retention and availability when they initially create an SAP BW environment. However, as that database environment grows, changing business rules and increased data volumes create new user demands. These demands often modify the scope and goals of the original design. For example, administrators may initially implement a database system that satisfies the reporting of operational business needs for the most current 18 months. However, suppose that users subsequently require access to 36 months of data and/or additional data types. In this case, it is a reasonable strategy for StorHouse to manage the oldest 24 months of data while SAP BW-managed online media keep the most current 18 months. This plan frees up resources on the primary database server so that it can satisfy more high performance demands. Industrial, Institutional, and Legally Imposed Data Retention Requirements Organizations in government-regulated industries such as brokerage and financial services, telecommunications, utilities, banking, air and defense, and pharmaceuticals are often required to save data (for example, e-mail, tax records, stock trades, and so on) for a 8 FileTek, Inc.
specified period. This aged data, often many years old, can burden the SAP BW environment just by its inherent size. StorHouse economically relieves SAP BW of this liability by offloading the data, yet still enabling relational access to all required compliance information regardless of its age. Business Needs Retention Requirements Business users of an SAP BW environment generally fit into one of three categories: Operational users Tactical planners Strategic planners and regulatory compliance officers Table 1 shows the unique historical data retention and access requirements for each of these categories. Table 1: Types of Business Users Type of Business Users Characteristic Operational Tactical Planners Business unit Sales, billing, manufacturing, customer service Sales management, marketing, purchasing Type of study Structured reporting Canned studies quarterly comparisons Strategic Planners and Compliance Officers Legal, corporate planners Primarily special studies performed by power users Required data Most current 120 days summary data Most current 15 months of detail data 2-7 years of detail data Ad hoc frequency Occasional ad hoc access Increased ad hoc access Frequent ad hoc access For each of these business user categories, data becomes a prime candidate for migration to StorHouse when it ages and its access demands decrease. SAP BW Design Considerations for StorHouse System designers and database administrators should examine the following design considerations when integrating StorHouse into an SAP BW environment: Implemented and planned ODS and InfoCube (ODS/C) designs Acceptable response time for accessing aged data Other key factors in a successful StorHouse/SAP BW implementation FileTek, Inc. 9
Implemented and Planned ODS and InfoCube (ODS/IC) Designs The current or planned design of SAP BW data objects (ODS/IC) has a significant impact on how a StorHouse system can best be implemented to satisfy end-user access requirements. If SAP BW data objects are based on time periods (for example, all data for a given quarter exists in a unique object), they can be moved in their entirety to StorHouse. Users will continue to access these non-partitioned data objects just as if they were still contained in the primary SAP BW database. The object s name remains the same, and there is no change to the SAP BW metadata. Scheduling migration is the only requirement. In contrast, when the SAP BW data object is open-ended/permanent (for example, all data for all months is stored in the same object), administrators must establish a horizontal partitioning migration strategy. Horizontal partitioning uses two unique SAP BW data objects and stores the older information in the second object. Refer to the section, Identifying Different Methods to Access Aged Data on StorHouse, on page 11 for more information about determining a strategy for accessing horizontally partitioned data. Acceptable Response Time for Accessing Aged Data Acceptable response time the balance between cost and query performance is typically established as the adequate time to access specific data. It varies depending on query type and frequency of execution. For example, for analytics that execute once a year, an acceptable response time of three hours to one day may be suitable. For reports that run quarterly or half-yearly, an acceptable response time of three hours may be more appropriate. In contrast, analytics that run more frequently (for example, twice daily) may require acceptable response times ranging from only several seconds to minutes. Because of its diversified storage architecture (sub-second network attached storage (NAS) disk to automated tape libraries), StorHouse can satisfy all response time requirements from a single install. Performance also depends on the size and complexity of the transaction and the number of simultaneous queries (which also determines the number of required tape drives, optical drives, or the amount of IDE disk and optical and tape media necessary in the StorHouse configuration). Thus, data with diverse acceptable response times can be stored on different media and storage layers and remain accessible from the same StorHouse solution. Key Factors in a Successful StorHouse/SAP BW Implementation Various factors influence cost verses performance when looking at different storage media options. These factors include, but are not limited to, determining when data 10 FileTek, Inc.
should be migrated to StorHouse and identifying different methods for accessing aged data on StorHouse. Once these issues have been determined, the StorHouse hierarchy can be configured with the proper mixture of IDE disk, optical, and tape media to meet expected performance goals. Determining When to Migrate Data to StorHouse. Typically, as data ages, users access it less frequently and can tolerate longer query response times. It is time to migrate data to StorHouse when access frequency and query response time intersect at an acceptable threshold (for example, data is accessed twice monthly and a query response time of one minute is acceptable). Because StorHouse provides seamless data access, administrators can maintain a minimal amount of data in SAP BW and frequently archive aged data to StorHouse. By following this methodology, administrators can run efficient data warehouse environments without sacrificing any potential analytical requirements. Identifying Different Methods to Access Aged Data on StorHouse. For nonpartitioned data migrated to StorHouse, users will continue to access data the same way they access it on the primary database. No changes are required. Horizontally partitioned data can be accessed in three ways. Each of these methods may have a different effect on performance and require varying levels of development effort. The following options demonstrate the functionality of the existing SAP BW StorHouse interface as implemented in current SAP BW releases. 1. Choice of Two Objects. This is the easiest solution to implement and provides the best performance. In this scenario, there are two unique mysap BI query objects that access a specific data partition (current or aged). Users choose to access current data or aged data and use the corresponding query object. The development effort is a simple matter of copying existing query objects and changing the data source. 2. Dynamic IF. This solution provides the intelligence to direct a query to the correct data source (primary database or StorHouse). This method is easiest on users and provides good performance (only the data required is accessed). 3. Always Both. This option requires that SAP BW query objects be developed to access both data partitions to perform a UNION of data (multi-cube or multiprovider). Both data partitions are searched automatically, which may limit performance. NOTE: For educated power users who need to perform analysis, a combination of option one (Choice of Two Objects) and option three (Always Both) may be the best strategy. In Conclusion With the definitive growth of enterprise historical data volumes, SAP BW users will find it ever more challenging to economically and physically manage their corporate information. Most are approaching these new challenges as a matter of when rather than if they will happen. Thus, forward-looking SAP BW professionals are choosing FileTek, Inc. 11
to implement an innovative data management strategy today to meet tomorrow s pressing needs. At the current rate of data growth, data warehouses will double in size every 24 to 36 months. Therefore, reaching the critical mass of 1 terabyte InfoCubes is not far off for most SAP BW administrators. Adopting a strategy that combines advanced storage management and relational database technology with SQP BW is a sound way to manage this growth. This paper introduces StorHouse and explains how it can be advantageously used to control data growth. When deployed with SAP BW, StorHouse provides faster SAP BW performance, up to 70 percent reduction in data storage and management costs, improved system availability, and fast access to aged data. When compared with traditional archiving methods, the StorHouse approach lessens not only the data processing weight from the SAP BW system, but also the workload for the IT professional. To learn more about the StorHouse database extension and relational repository for mysap BI BW, please contact FileTek at one of the following offices: North America, Latin America, Asia: FileTek, Inc. 9400 Key West Avenue Rockville, MD 20850 Tel: 301-251-0600 Fax: 301-251-1990 France, Sweden, Denmark, Finland, Norway, and Benelux: FileTek SPRL Stephanie Square Centre Avenue Louise 65 1050 Bruxelles, Belgium Tel: +32 2 535 7876 Fax: +32 2 535 7700 MEA, Europe (except France, Sweden, Denmark, Finland, Norway, and Benelux): FileTek UK Limited 1 Northumberland Avenue London WC2N 5BW Tel: +44 207 872 5583 Fax: +44 207 753 2829 Information is also available at: www.filetek.com. 12 FileTek, Inc.