SAP NetWeaver BW Archiving with Nearline Storage (NLS) and Optimized Analytics

Transcription

1 SAP NetWeaver BW Archiving with Nearline Storage (NLS) and Optimized Analytics corp.com

2 Copyright 2011 Dolphin, West Chester PA All rights are reserved, including those of duplication, reproduction, use or disclosure of the contents of this documentation, or any part of it. No part of it may be reproduced in any form, passed on to third parties or particularly by electronic means, processed, reproduced, distributed or used for publication without the written permission of Dolphin. We reserve the right to update or modify the contents. Trademarks SAP, R/3, SAP NetWeaver BW, SAP ERP are trademarks and registered trademarks of SAP AG in Germany and several other countries. PBS archive add ons is a registered trademark of PBS Software GmbH. Other product names are used only to identify the products and are registered trademarks of the relevant manufacturers.

3 Table of Contents Introduction... 4 The BW Conundrum... 6 Accomplishing ILM for SAP NetWeaver BW with Nearline Storage... 9 Compression and Speed: How Columns Support Analytics Implementing the Data Archiving Process Case Study: Implementation of BW Archiving with NLS and Column-based Database for SAP NetWeaver BW About Dolphin... 18

4 Introduction Knowledge is power. The more in-depth, actionable information your business can extract from its data warehouse, the greater the opportunity to optimize sales, inventory, supply chain and other processes. And the better this is, the more ways you can find to lower costs and increase profits. In the world of SAP business intelligence (BI), businesses use the SAP NetWeaver BW platform to consolidate data from multiple business systems into a data warehouse. As BI tools for analysis, process improvement, reporting and predictive analytics gain higher adoption rates within Dolphin is at the forefront of applying ILM principles to SAP NetWeaver BW with Nearline Storage optimizing for both cost and performance, while reducing administrative load on the IT department. organizations, information technology departments are feeling the heat: business users want greater access to more data, the ability to run more queries and to facilitate deeper analytics. And everyone wants all of it faster. Rising data and query volumes take a toll on data warehouse hardware, software and administration and drive up costs. BI applications feed on data, and through replication, aggregation and analysis create even more data that needs to be updated, accessed, analyzed and archived. The demand for fast access to core business analytics for example, daily sales across SKUs and channels; financial reporting; or trend analysis requires high data volumes to be kept online in high-speed production databases. The larger these core InfoCubes are, the more strain is put on online database performance. To maintain service levels, IT must either perform constant juggling and tweaking, or commit an even higher percentage of the IT budget to adding more storage. This pressure from rising data volume and traffic can only increase. SearchDataManagement.com asked a number of experts and analysts to predict BI trends for 2010; many cited the growth of self service. James Kobielus, a senior analyst at Forrester Research, remarked: Enterprises have begun to adopt self-service BI to cut costs, unclog the analytics development backlog, and improve the velocity of practical insights. Users are demanding tools to do interactive, deeply dimensional exploration of information pulled from enterprise data warehouses, data marts, transactional applications, and other systems.

5 Meeting the demand requires systems that are fast, flexible and adaptable to enable standard and ad-hoc queries and drill-downs. The response? For IT departments, it has long been throw more storage at it and so they add more data storage. Alternatively, strategic, long-term focused programs implement data archiving and ideally some form of Information Lifecycle Management (ILM). Dolphin has long been at the forefront of applying ILM principles to SAP systems for ERP archiving and business warehousing. Dolphin helps customers using SAP solutions design and implement data archiving and ILM strategies to take data offline using archiving and Nearline Storage (NLS). For SAP NetWeaver Business Warehouse (BW), this permits access and use of the data for reporting without reloading it back into the online SAP system. We are helping customers streamline for both cost and performance, while reducing the overall administrative load on the IT department. Today, advances in column-based database architecture add another dimension to the traditional benefits of data archiving. Companies using SAP NetWeaver BW systems now have multiple new options with archiving to nearline storage for optimizing both cost and query performance, gaining faster access to archived data, reducing database administration requirements and lowering total costs. This white paper presents a case study for NLS optimized for analytics using columnar database architecture and outlines how a nearline storage solution makes it possible for IT departments using SAP NetWeaver BW to manage costs, reduce overhead and increase system performance. Benefits of Nearline Storage for SAP NetWeaver BW Significantly reduced data volume and growth in online BW database Great query response for all data without administrative overhead Smaller BW Accelerator footprint needed due to lower online data volume Easier backup and disaster recovery Improved performance/cost ratio Lower Total Cost of Ownership

6 The BW Conundrum Real-time and historical data on business performance is available from across the enterprise. SAP NetWeaver Business Warehouse (BW) collects all of this information so that the business can have the information it needs in a single place to gain insight and make informed business decisions. But the more you use and rely on SAP NetWeaver BW, the more it generates exponential increases in database volume which in turn takes a toll on system performance. In mitigating the performance issue, IT must consider the cost of continuous data storage expansion to contain all of that information with associated cost of maintenance and administration. The first reaction of most organizations is to add SAP NetWeaver BW Accelerator. BW Accelerator essentially adds massive amounts of fast CPU memory to boost SAP Netweaver BW performance. The result is that you can keep more data online for efficient queries and reporting. Many companies also create a set of standard InfoCubes that contain and maintain queries needed by the business every day for example, daily sales by SKU, channel and customer; inventory levels; days-salesoutstanding; financial reports; or materials supply and cost. And, business intelligence applications generate data replication. Source data loaded Figure 1: BW database volumes and costs rise while performance drops. The answer is to apply ILM principles to the SAP NetWeaver BW system to reverse this. into SAP NetWeaver BW is replicated on different layers as an example, on PSA, DSO, Cubes and Indices. Administrators develop aggregates to speed query performance for specific types of data and requests. However, developing aggregates takes a great deal of time, and the process further multiplies the data growth problem.

7 As the information volume grows, so do the administrative and storage challenges, as illustrated in Figure 1. The problem is that all the data is stored in the same way regardless of its value and how often it used. And, the BW database itself is built on a traditional relational database management system designed for transaction processing, not for analytics. Using only the SAP NetWeaver BW Accelerator to support the rising information tide can quickly become an expensive proposition. The right ILM strategy can help prevent an overload of the SAP NetWeaver BW Accelerator s license and service levels. By using SAP NetWeaver BW Accelerator optimized with nearline storage, IT is able to more easily and efficiently manage these daily queries against ad-hoc queries and less time-sensitive requests. The Challenge of Information Lifecycle Management for BW Using SAP functionality to archive BW data presents its own challenges. SAP Data Archive Process (DAP) enables data archiving from the BW system. The archiving process removes data from the BW database and sends it to a nearline storage system to pare down the amount of data in the production database. But standard SAP NetWeaver BW archiving functionality does not permit direct reporting against archived data. To access and use the archived data, it needs to be reloaded back into the BW database to enable reporting. Data reload requires labor resources and database space in the system to handle the additional volume. And the reloaded data must be removed again from the database after historical data reporting. The lack of direct access to archived data using DAP, and a cumbersome retrieval process, often leads to long online residence periods and huge BW system databases. When addressing archiving as part of managing the information lifecycle of your data, there are some core differences between ILM for ERP and for BW. In ERP, static data data records that are business complete and will no

8 longer be changed can often be moved relatively quickly to lower cost storage tiers. By contrast, although data and historical information in the business warehouse may be static, it is still used for reporting, comparisons and other analytics. For example, a consumer food and beverage business that supplies grocery stores might access and analyze year-over-year, month-over-month and week-over-week sales data across customers, stores and SKUs on a weekly or even daily basis to track changes in performance. This illustrates that old data is highly valuable on a regular basis and therefore needs to be where it can be accessed efficiently. Archiving improves the performance speed of the online system, and both archiving and nearline storage makes it possible to reduce online database size. In the last few years, nearline storage solutions have garnered a lot of interest because access and retrieval are quicker than that of traditional archive media. The cost is also much lower than keeping the data online. NLS enables space-saving compression, improved data access and faster reporting time. Coupling nearline storage with column-based technology, data access is faster still, and it has become a viable alternative to keeping data online; and the cost savings are significant. The solution to assuring real-time data availability and being mindful of the performance/cost ratio is to establish a data archiving strategy that maximizes storage and network resources, yet ensures efficient business information access.

9 Accomplishing ILM for SAP NetWeaver BW with Nearline Storage Where a typical ILM strategy is to separate static data from the online production database and archive it on a less expensive, slower access storage tier, a better strategy for BW is to implement nearline storage, optimized for analytics, that while not online delivers significant cost and performance advantages needed for SAP NetWeaver BW operations and administration. Dolphin implements an SAP-certified solution from German SAP software developer, PBS - PBS CBW Nearline Storage (NLS), designed for use with SAP NetWeaver BW. CBW NLS allows archiving from SAP NetWeaver BW to nearline storage and the data can be A column-oriented DBMS is a database management system (DBMS) which stores its content by column rather than by row. This has advantages for databases such as data warehouses and library catalogues. - From Wikipedia transparently accessed and used by business users. Perhaps more important to controlling database growth and online database performance, the nearline archived data does not have to be reloaded into the online database. The solution requires no IT resources and, since users access the nearline data the same way they access online data, no additional user training is required. Fig. 2: Access methods for archive data with PBS nearline solutions using the NLS interface. Although compression is the same for both NLS solutions, the analytics server provides faster retrieval.

10 To further enhance the PBS CBW NLS solution and meet rising data volumes and the demand for higher speed the NLS solution offered by Dolphin uses an advanced column-based database optimized for business analytics. A columnar database differs from a traditional relational database structure by compressing related information into columns. The result is dramatically faster search and query performance. Data is compressed by 95% or more in the NLS database, which lowers storage costs and enhances overall performance. There are three types of storage systems available to manage data for SAP NetWeaver BW; which one is implemented depends on the type of data being managed: Figure 3: Using nearline storage for archived data. Integration for nearline interface for SAP BW 7.x 1. Online Production Database. This is the high-performance disk storage where data in the BW environment originates it could be daily sales, inventory, HR or financial data while it is still being updated. InfoCubes queried and updated frequently should be kept online; users can deploy SAP NetWeaver BW Accelerator to cache the most recent or most frequently used data to improve query performance. On the other hand, staging data not actively used in queries should be offloaded from this database to archive storage as quickly as possible to reduce strain on the online database and the SAP NetWeaver BW Accelerator. 2. Nearline Storage System. The nearline system archives read-only data on a fast-access disk system, and, if the business warrants, it could use a column-based database optimized for business analytics. An example would be the previous year s sales data, enabling quick comparison with online data for the same period describing current sales trends.

11 Ideally, this kind of reporting data can be stored in the nearline system for a long period of time until it is no longer needed for frequent queries. A nearline system using a column-based database delivers significant data compression 95% or more so that storage costs are lower while reporting performance is better than what is typically seen in online databases not using SAP NetWeaver BW Accelerator. 3. Long-term Archive Storage. This is where information resides that is not used for queries, but still needs to be accessible for data transfer processes. This data can be kept on low-cost, slower storage media archival disk systems or even tape. The data is indexed for seamless access via the SAP application. Using ILM principles, these systems allow you to manage the data across its useful lifecycle, as illustrated in Figure 4. Figure 4: Access speed, performance and cost over the data's lifecycle are maximized when using nearline storage architecture optimized for business intelligence analytics. ILM strategy varies based on the type of data being managed. InfoCubes that are frequently used spend their most active periods cached with the SAP NetWeaver BW Accelerator for high performance. They go to nearline storage when appropriate. Staging data, once static, should be moved from the online database directly to long-term archives.

12 Compression and Speed: How Columns Support Analytics Why are columnar databases better for analytic applications? Query speed and compression. In relational databases, data is stored in rows. This is an ideal structure for transactional processing organizing data in tables by rows simplifies recording and organizing customer sales records, for example, and attach to each record variable such as sales location, price, product SKU and so on. The data is well-organized and easy to enter and understand. But this structure is inefficient for the kind of queries performed in analytic applications, including drill-downs on specific variables, developing theories and specialized reporting. The reason is that in row-based relational databases, getting and reporting information from the system requires a high number of input and output (I/O) operations. In many cases, the query or report only requires a small number of the fields stored for each record. For example, if you want to determine average salaries across a departmental job function, finding the information requires reading every field name, ID numbers, benefits and more when the only data necessary for the calculation is the salary field. Creating a departmental index can speed the process, but creating this index requires additional administrative time and overhead. A column-based database literally turns this query on its side. In a columnar database optimized for analytics, as implemented by Dolphin for SAP NetWeaver BW, data is compressed into relevant columns. The values for each variable, or field, are stored sequentially. Thus, in the departmental salary query example, all salary data is stored together, so that the query only has to read a single field. The search that might require 500,000 I/O operations in a relational database may require less than half that many in a columnar database. This difference can dramatically improve performance. Moreover, this kind of database engine requires little-to-no indexing, reducing administrative costs. Data compression improves operational speed and reduces archiving costs. Dolphin SAP NetWeaver BW archiving projects deliver data compression in the range of

13 85%-95% or more for the read-only data on the nearline system. This instantly and dramatically reduces strain on the online database and makes it easier for IT to manage longterm database growth. Implementing the Data Archiving Process Moving the Data So how do we move static data out of the SAP production database and into the nearline analytical archive? The process begins by specifying a nearline connection for an InfoProvider s data archiving process (DAP). Once this nearline connection is established, it can be used as a destination for archiving any InfoProvider in the SAP system. Running the DAP for a time period consists of writing the selected data to the NLS system, verifying that the data was correctly stored on the nearline system and finally deleting the data from the online database. With the PBS solution, BW administrators are capable of monitoring the archiving process on a real-time basis, including obtaining statistics on how much data is being archived, how much space it takes up in the nearline system, and compression rates. This information is essential to manage costs and measure ROI. Data Retrieval Nearline Queries There are two options for querying nearline data. First, the nearline flag can be set in the query properties to add nearline data to a query from a single InfoProvider. If the query is used against multiple InfoProviders, a virtual provider is added to the base cubes so that the query can access the nearline database. Either way, the user sees no difference whether the query is run through the SAP production database or the data comes from the nearline system. The results simply arrive faster.

14 Case Study: Implementation of BW Archiving with NLS and Columnbased Database for SAP NetWeaver BW Findings from an implementation at an international consumer products company illustrates how applying ILM principles and adding column-based NLS storage to the BW architecture can dramatically improve SAP NetWeaver BW performance. The customer sells globally through grocery chains and convenience stores. The customer s data environment included: A multi-provider data set with about 17 million records in two base cubes (one with 14 million records; the other with 3 million) Nearly 2 million customer master data records More than 63,000 master data table records and 800,000 items in the navigational hierarchy Dolphin s tests compared different types of queries under normal conditions: First, a typical online relational database using the SAP NetWeaver BW Accelerator to boost performance; next, using aggregates to speed relational database performance; and then using the PBS CBW NLS the nearline storage solution using a column-based database optimized for analytics. The results were dramatic. In the first test, a high-level query into the 17 million records, the results were:

15 The results were even more dramatic in a drill-down query the kind of query a business user might run to see year-over-year results for a single product brand, or sales through a single grocery chain customer: Billing Documents Test. Another test was performed with the same customer on a billing documents database that was even more telling. Here, Dolphin conducted queries on a multi-provider data-set of more than 57 million records and a master data table with more than 1.5 million records. The results were:

16 In the drill-down query, the results were, again, even more impressive: In both cases, the NLS solution was able to deliver impressive results, while the online database without aggregates timed out after 2,000 seconds and was unable to complete the query. The Bottom Line Business Intelligence continues to add value for more and more organizations. Concurrently, the data management challenge will continue to grow exponentially. Applying the principles of Information Lifecycle Management can put IT in the right position to tame this growth as well as manage database performance and storage to meet short- and long-term business needs. If NLS technology is the entry to effective ILM for SAP NetWeaver Business Warehouse, the column-based database opens that door all the way. Fast access, disk-based storage combined with columnar database engines designed for analytic applications make it possible to archive data from the online database without sacrificing performance. Moreover, moving to an NLS-based system reduces time spent on database administration, creating aggregates and the overall reduction in cost of ownership for the SAP NetWeaver BW infrastructure.

17 Figure 5: With NLS archiving, you can maintain BW performance while controlling costs. The benefits are clear and measurable: Significantly reduced data volume and growth in online BW database the NLS solution removes static data from the online SAP NetWeaver BW database and compresses it by 95% or more, reducing storage costs and maintaining performance. Faster query response for all data without administrative overhead (no aggregates necessary) this saves significant time in terms of database administration and reduces overhead costs. Smaller SAP NetWeaver BW Accelerator footprint needed due to lower online data volume less data held online lets you get the most benefit out of your investment in accelerators. Efficient backup and disaster recovery significant compression in the nearline database means less data to backup and restore. Improved performance/cost ratio and lower Total Cost of Ownership using the right storage for the right data saves costs now and in the future.

18 Dolphin is in a position to know. We have been helping companies make their critical business operations run in SAP better and smarter through data archiving and an ILM strategy since Our solutions combine unique tools, best practices and in-depth SAP and data management expertise so that they meet the performance needs of our customers businesses. About Dolphin Dolphin makes crucial business operations like accounts payable, accounts receivable, order management and data archiving for ERP and BW run better and smarter for organizations using SAP solutions. Founded in 1995, Dolphin produces the right solution for each customer, faster, through its unmatched experience in SAP solutions, and its proven best practices, tools and SAP add-on applications. Dolphin solutions improve business and IT performance, lower total cost of ownership and deliver high return on investment. Based in Philadelphia, PA and San Jose, CA, Dolphin solutions are implemented across North America and around the world. Contact Dolphin Phone: 1 (888) contact@dolphin-corp.com Website: Blog: