How to Archive Data from SAP NetWeaver BW to SAP Sybase IQ as Near line Storage

Transcription

1 SAP How-to Guide Database & Technology SAP NetWeaver Business Warehouse SAP HANA Appliance How to Archive Data from SAP NetWeaver BW to SAP Sybase IQ as Near line Storage Applicable Releases: SAP NetWeaver BW 7.3x SP09 SAP HANA 1.0 SPS 06 Version 1.0 September 2013

2 No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP AG. The information contained herein may be changed without prior notice. Some software products marketed by SAP AG and its distributors contain proprietary software components of other software vendors. Microsoft, Windows, Excel, Outlook, and PowerPoint are registered trademarks of Microsoft Corporation. by Netscape. SAP, R/3, SAP SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP AG in Germany and other countries. Sybase and Adaptive Server, 365, SQL Anywhere, and other Sybase products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of Sybase, Inc. All other product and service names mentioned are the trademarks of their respective companies. Data contained in this document serves informational purposes only. National product specifications may vary. The information in this document is proprietary to SAP. No part of this document may be reproduced, copied, or transmitted in any form or for any purpose without the express prior written permission of SAP AG. This document is a preliminary version and not subject to your license agreement or any other agreement with SAP. This document contains only intended strategies, developments, and functionalities of the SAP product and is not intended to be binding upon SAP to any particular course of business, product strategy, and/or development. Please note that this document is subject to change and may be changed by SAP at any time without notice. SAP assumes no responsibility for errors or omissions in this document. SAP does not warrant the accuracy or completeness of the information, text, graphics, links, or other items contained within this material. This document is provided without a warranty of any kind, either express or implied, including but not limited to the implied warranties of merchantability, fitness for a particular purpose, or non-infringement. SAP shall have no liability for damages of any kind including without limitation direct, special, indirect, or consequential damages that may result from the use of these materials. This limitation shall not apply in cases of intent or gross negligence. The statutory liability for personal injury and defective products is not affected. SAP has no control over the information that you may access through the use of hot links contained in these materials and does not endorse your use of third-party Web pages nor provide any warranty whatsoever relating to third-party Web pages. Copyright 2014 SAP AG. All rights reserved. Business Objects and the Business Objects logo, BusinessObjects, Crystal Reports, Crystal Decisions, Web Intelligence, Xcelsius, and other Business Objects products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of Business Objects Software Ltd. Business Objects is an SAP company. SAP How-to Guides are intended to simplify the product implementation. While specific product features and procedures typically are explained in a practical business context, it is not implied that those features and procedures are the only approach in solving a specific business problem using SAP NetWeaver. Should you wish to receive additional information, clarification or support, please refer to SAP Consulting. Any software coding and/or code lines / strings ( Code ) included in this documentation are only examples and are not intended to be used in a productive system environment. The Code is only intended better explain and visualize the syntax and phrasing rules of certain coding. SAP does not warrant the correctness and completeness of the Code given herein, and SAP shall not be liable for errors or damages caused by the usage of the Code, except if such damages were caused by SAP intentionally or grossly negligent. Disclaimer Some components of this product are based on Java. Any code change in these components may cause unpredictable and severe malfunctions and is therefore expressively prohibited, as is any decompilation of these components. Any Java Source Code delivered with this product is only to be used by SAP s Support Services and may not be modified or altered in any way.

3 Document History Document Version Description 1.00 First official release of this guide

4 Typographic Conventions Type Style Example Text Description Words or characters quoted from the screen. These include field names, screen titles, pushbuttons labels, menu names, menu paths, and menu options. Cross-references to other documentation Icons Icon Description Caution Note or Important Example Recommendation or Tip Example text Example text Example text <Example text> EXAMPLE TEXT Emphasized words or phrases in body text, graphic titles, and table titles File and directory names and their paths, messages, names of variables and parameters, source text, and names of installation, upgrade and database tools. User entry texts. These are words or characters that you enter in the system exactly as they appear in the documentation. Variable user entry. Angle brackets indicate that you replace these words and characters with appropriate entries to make entries in the system. Keys on the keyboard, for example, F2 or ENTER.

5 Table of Contents 1. Introduction Step-by-Step Procedure Configuring Sybase IQ as a Near-Line Storage Solution The configuration steps in SAP NetWeaver BW Configuration Steps in Sybase IQ (optional) Create Data Archiving Process in BW Read Archived Data BEx Query on Archived Data Reading NLS data in BW Transformations NLS data access for InfoProviders as sources within APD InfoProvider Type Specific Features Archiving of non-compressed data for InfoCubes Archiving of Write-Optimized DSOs Write into NLS with Process Chain Process Chain with Archiving Steps Write Phase Verification Phase Deletion Phase... 33

6 1. Introduction This paper s focus is the procedure of setting up and executing the archiving process with the Sybase IQ NLS from the BW system. For positioning, prerequisites and other relevant information please visit: new SCN page (Roland s current page: The aim of this paper is to illustrate with screenshots the setup of the NLS-IQ solution for BW as described in the online documentation Sybase IQ as a Near-Line Storage Solution Please also always refer to the latest version of the documentation when executing the steps described in this paper. April

7 ... How to Archive Data from SAP BW to Sybase IQ as Near Line Storage 2. Step-by-Step Procedure First, you will need to Create Database Connection and Configure Sybase IQ. Next you will use the SAP BW system to create the archiving process to Archive SAP BW Data to Sybase IQ. Finally you will check the archived request and test query access to data in the Sybase IQ for older data. For detailed information on the Sybase IQ installation please refer the SAP First Guidance - SAP- NLS Solution with Sybase IQ 2.1 Configuring Sybase IQ as a Near-Line Storage Solution The configuration steps in SAP NetWeaver BW For details refer to the SAP Online documentation: Configuring Sybase IQ as a Near-Line Storage Solution Create Database Connection First create DB Connect - call transaction DBCO and create a database connection to Sybase IQ. For more details, refer to SAP Note CAUTION The DBCO user and the Sybase IQ user both have to be created with uppercase letters. Parameter SYBASE_IQ_CESU-8 under Connection Information must be set to 0 if you use UTF8 as the character set. Connection Info that works is: SYBASE_SERVER=MELN A.dhcp.wdf.sap.corp SYBASE_PORT=2638 SYBASE_IQ_ENGINE=MELN A_iqdemo SYBASE_DBNAME=BDFNLS SYBASE_CONTYPE=IQ SYBASE_IQ_LOAD=1 SYBASE_IQ_CESU-8=1 SYBASE_SQL_TRACE=1 SYBASE_IQ_LOCKWAIT=600 SYBASE_IQ_BUFFER_SIZE= April

8 Note Ensure that Sybase IQ is up and running. (Refer to Appendix A). The default password for user DBA delivered is sql. This needs to be changed for security reasons. Create Near-Line Storage Connection 1. Call transaction SPRO SAP NetWeaver Business Warehouse General Settings Process Near-Line Storage Connection 2. Choose New Entries 3. Enter a name for the near-line storage connection. This name is then offered to you for selection when you create a data archiving process. April

9 4. As the name, enter class CL_RSDA_SYB_CONNECTION. 5. Leave the Destination field empty 6. Enter a connection string under Connection Parameters. The connection string generally comprises a list of name/value pairs, separated by a semi-colon. Note Each name/value pair must be entered as follows: <Name of the parameter>=<value of the parameter> The following parameters are supported: DBCON (required entry): The value of the parameter should be the name of the database connection that you created in the first step. BACKUP_REQUEST_EVENT (optional): The value should be either DUMMY or the name of an event that has been defined in the Sybase IQ database and which triggers a new data backup. If this parameter is entered, and all data has been copied to Sybase IQ, the end of the write phase triggers this event. This makes it possible to perform the data backup in Sybase IQ parallel to the verification phase. Before the actual deletion process begins, the system checks for a backup in Sybase IQ. The deletion phase cannot continue until this has happened. April

10 A special event called DUMMY triggers an event in the Sybase IQ database. It checks for a more recent data backup however before the data in the request is deleted from the BW database. The DUMMY event should therefore be used if database backups are detected before the data is deleted and the data backups are automatically created by the Sybase IQ database. If the parameter is left out or does not contain any values, no event is triggered, and the system does not check for data backups. You should only choose this setting in non-productive systems. BACKUP_WAIT_SEC (optional) If the BACKUP_REQUEST_EVENT has been specified, and no data backup is detected before the start of the deletion phase, the process waits for a period specified here in seconds. If no data backup is detected during this period, the archiving request terminates with error message RSDA_SYB197: "No new data backup since the end of the copy phase. The current data backup ID is still <backup ID> of <timestamp>. The parameters <backup ID> and <timestamp> refer to the columns bu_id and bu_time in the sybase IQ system view SYSIQBACKUPHISTORY. All rows with the condition selective_type in (1,2) are taken into account. This excludes all data backups that only refer to read-only files. The archiving request only be restarted once the data backup has been created. PACKET_SIZE_MB (optional) The package size of a single LOAD statement in MB. If this parameter is left out or does not contain any values, the package size of the LOAD statement is exactly the same as the package size of the FETCH of the online table. This is currently set to 20 MB. A package size greater than 2000 is ignored and replaced by 2000 MB. 7. Save your settings. Tip Use program ADBC_TEST_CONNECTION to test the DB connection to Sybase IQ configured above: Confirm the ODBC connection settings. In our example the ODBC connection has been defined as follows: April

11 April

12 April

13 ... How to Archive Data from SAP BW to Sybase IQ as Near Line Storage Configuration Steps in Sybase IQ (optional) For details refer to the SAP Online documentation: Configuring Sybase IQ as a Near-Line Storage Solution These configuration steps in Sybase IQ are optional in order to integrate an automated data backup mechanism. 1. Log on to Sybase IQ with user DBA via Sybase Central or IQISQL. 2. Create an event that triggers a new data backup. As the name of the event, enter the name that you entered in step 6 for BACKUP_REQUEST_EVENT. The way the event is handled depends on your data backup and data recovery strategy. Example The following event definition shows a very simple strategy. Here, a data backup is always started whenever the event is triggered. The data backup files are stored in a directory in the database's Home directory. The name of the database forms part of the path name and of the file name: CREATE EVENT BACKUP_REQUEST HANDLER BEGIN -- Declare and set unique filename DECLARE filename varchar(255); SET filename = '../backup/' db_name() '/' db_name() '_' dateformat(now(),'yyyymmdd_hhnnss.ssssss'); -- Perform incremental backup BACKUP DATABASE FULL TO filename; END April

14 ... How to Archive Data from SAP BW to Sybase IQ as Near Line Storage 2.2 Create Data Archiving Process in BW For details refer to the SAP Online documentation: Creating Data Archiving Processes Step-by-step procedure: 1. From the context menu of the InfoProvider (e.g. DataStore Object), choose Create Data Archiving Process 2. Deselect the ADK-Based Archiving checkbox, enter the Near-Line Connection BDFNLS and Check this connection. April

15 3. Check the status information for the selected Near-Line Connection and choose Continue (Enter). 4. Enter selection profile if applicable Note The fields that are available for selection are only the InfoProvider key fields (highlighted in red). If there is no time characteristic then you may not archive the data in the DSO. Where there is no time characteristic in the key of the DSO then the characteristic for time slice refers to a time characteristic in the data field of the DSO (see pictures below). The primary partitioning characteristic refers to the selection defined in the Data Archiving Process. The way archiving works is that the system will select all primary partitioning characteristics that are older than the date of the characteristic for time slice. April

16 The DSO key doesn t contain any time characteristic. The characteristic for time slice refers to a time characteristic in the data field of the DSO. April

17 5. Select semantic group if applicable 6. Accept the default settings on the Near-Line storage tab. Check and Activate your Data Archiving Process. April

18 7. Check the Activation Log for the Data Archiving Process and leave the log by choosing Continue (Enter). 8. Create Archiving Request. From the context menu of the InfoProvider choose Manage then go to the Archiving tab and choose Create Archiving Request Note An icon appears next to the InfoProvider indicating that a Data Archiving Process was created successfully. April

19 For additional details refer to Creating and Executing Archiving Requests 9. Simulate Execution of Data Archiving Request in Dialog Note The period referred to in Only Data Records Older Than is the number of days/weeks/years relative to today (this is dynamic so can easily be used in process chains). The absolute value is a static fixed selection period April

20 The selection options provided in the restriction is based on the definition of the Data Archiving Process Selection. Refer to step 4 above. 10. You can simulate the Data Archiving Process from here in the background (user BWREMOTE is used) or in the foreground (user logged into BW) Simulate the execution of the Data Archiving Request by choosing Only Simulate Request Processing in Dialog 11. Ignore the warning No restrictions yet defined by choosing Continue (Enter) Confirm Do you want to continue processing in the dialog? with Yes April

21 Check the displayed log for errors and choose Continue (Enter). 12. Create and execute the Data Archiving Request by choosing Create Request and Execute in Background April

22 Note The Data Archiving Process consists of 5 steps. This is reflected in the screen dump above. You can run this in single steps or you can do this in one step meaning you can simulate step 10 of the process, then step 20. This will assist in error detection. Setting the Continue Processing until Target Status to 70 will execute all steps in one go. It is recommended to use 70 for the manual approach. Alternatively split this into step 40, 50 and 70 within a process chain. The reason for the split within the process chain is to better assist with error handling 13. View the results of scheduled Job BI_ARCH-NLSDS01-CREATE April

23 Check the Job s Status. You should see a successfully finished archiving job (BI_ARCH- NLSDS01-CREATE). Leave the job overview by choosing Back 14. Check the Archiving Request Click Refresh to refresh the Data Archiving Request list. Your Data Archiving Request should now appear in the list. April

24 ... How to Archive Data from SAP BW to Sybase IQ as Near Line Storage Leave the DataStore Object administration and the Data Warehousing Workbench by choosing Back twice. 2.3 Read Archived Data The archived data is accessible via BEx Queries, Transformations for DSO lookup and Analysis Process BEx Query on Archived Data With BW on HANA 7.30, you can now set up queries on MultiProvider and read the NLS partitions of the PartProviders. Configuration for NLS use is possible on four different levels: 1. In the properties of the InfoProvider 2. In the properties of the PartProvider (from MultiProvider) 3. In the properties of the BEx Query Designer 4. Definition of a near-line storage variable for the query selection screen The settings for NLS use on the four different levels are described below. For additional details refer to the SAP Online documentation: Read Stored Data 1. Properties of the InfoProvider: In order to access archived data at InfoProvider level you have to change/adjust the InfoProvider settings appropriately. For all InfoProvider object basically two NLS settings are possible: April

25 Near-Line Access Switched Off (Default) X Near-Line Access Switched On For MultiProviders, an additional option is available: P Near-Line Access Set the Same as for PartProvider Depending on the type of the InfoProvider different settings are available. 2. Properties of the PartProvider: NLS usage can also be delegated from the query level to the PartProvider level by using the following settings: BEx Query Designer: Use Near-Line Storage According to Provider Settings MultiProvider: Near-Line Access Set the Same as for PartProvider PartProvider: Near-Line Access Switched On/Off This configuration allows you to switch specific NLS PartProvider partitions on and off. April

26 3. Properties of the BEx Query Designer The query definition is defined with NLS on the extended tab. Options available for Near-Line Storage include: Use Near-Line Storage According to Provider Setting mode Do Not Read Near-Line Storage Read Near-Line Storage April

27 4. Definition of a near-line storage variable for the query selection screen: In the BEx Query Designer, you can also define an NLS variable to let a specific query run either in Read Near-Line Storage mode or Do Not Read Near-Line Storage mode or Use Near-Line Storage According to Provider Setting mode April

28 Start the Query Monitor You can start the Query Monitor from the SAP Easy Access Menu (SAP menu -> Business Explorer -> BEx Monitor -> Query Monitor (= transaction RSRT1). The NLS read access is the name of the selection variable. Now you can decide during query execution if you want to report on archived data or not. April

29 This example illustrates results of the query using variable NLS set and without NLS access. Example: Read Near-Line Storage: Do Not Read Near-Line Storage The results of the query monitor are as follows: Note In the Statistics Data for the Query Runtime (transaction RSRT), you can see how many PartProviders, including aggregates and near-line partitions (suffix $N), effectively contribute to the query result. The following is another example showing how data is queried on from NLS and live data The $X reflects the active data. The $N reflects Near-line Storage (Sybase IQ). April

30 Values show 96 records which were archived and 48 remained in the DSO in for this selection Note Two tables are generated in Sybase IQ for the InfoProvider. Once contains the actual data. The other contains the MetaData. You can view data in Sybase IQ to validate that the data is indeed correct The screen shot display data archived using the Data Arching Process The following screen shot displays the Data Arching Process Metadata. April

31 This above Sybase IQ table reflects the BW DAP below Reading NLS data in BW Transformations For additional details refer to the SAP Online documentation: Reading Stored Data Please refer to SAP notes and for additional information. April

32 2.3.3 NLS data access for InfoProviders as sources within APD As of SAP NetWeaver BW 7.30, the Analysis Process Designer is enabled to read data transparently from the online partitions and near-line partitions. This makes it easy to include historically archived data in your analysis process. In the example shown in the following picture an APD process (TC: RSANWB) data is to be extracted from Cube which is partly archived. Select the source cube with the cursor and click the right mouse button. In the context menu you can select the Properties.. Now you can select the option Read Data from Near-Line Storage. April

33 3. InfoProvider Type Specific Features For additional details refer to the SAP Online documentation: Creating Data Archiving Processes 3.1 Archiving of non-compressed data for InfoCubes The option to archive non-compressed data was requested by customers for whom data compression was a time consuming operation and did not really lead to significant compression rates (highly granular data). Data compression was a precondition for archiving InfoCube data since release BW Before data can be compressed, it must be: distributed to subsequent data marts rolled up to existing aggregates As of SAP NetWeaver BW 7.30, it is no longer necessary to compress data before archiving. To ensure data consistency in different data layers, the two preconditions above must be met before data can be archived. The flag Allow Archiving of Non-Compressed Data is a central setting in the data archiving process. Note The flag can be changed in the Data Archiving Process (DAP) even if data has already been archived. If you change this flag (Allow Archiving of Non-Compressed Data) in the DAP definition, all archiving requests that are not completely finished (Status 70 Deletion Phase Confirmed and Request Completed ) must be finished in the old Compression Mode or must be set to invalid before you can archive data in the new Compression Mode April

34 3.2 Archiving of Write-Optimized DSOs For additional details refer to the SAP Online documentation: Special Features of Write-Optimized DataStore Objects The NLS solution allows for archiving of data from Write Optimized DSOs. The main difference between settings for DAP for standard DSOS and write optimized DSOs is: data from write optimized DSOs are archived request-based. On the archiving request (context menu of an InfoProvider -> Manage-> Tab: Archiving) you can still create a time related restriction: April

35 After starting the archiving process an additional request related restriction is generated: As a result all request which contain data older than Time-restriction are archived in full. You can check it in the Requests tab, April

36 ... How to Archive Data from SAP BW to Sybase IQ as Near Line Storage 4. Write into NLS with Process Chain For additional details refer to the SAP Online documentation: Scheduling Data Archiving Processes Using Process Chains Prerequisite is an Active, executable Data Archiving Process for your desired Info Provider. 4.1 Process Chain with Archiving Steps The sequence of process chain using NLS is as follows (see screen): 1. Write Phase 2. Verification Phase 3. Deletion Phase It is recommended to at least keep the delete process separately. April

37 4.1.1 Write Phase Apply the following settings: Continue Processing Until Target Status: 40 Write Phase Completed Successfully Auom- Request Invalidation After Error: None Verification Phase You need the following settings: Continue Open Archiving Request(s) : Check Continue Processing Until Target Status: 50 Verification Phase Ended Successfully Auom- Request Invalidation After Error: None April

38 Note The Continue Open Archiving Request(s) Check box refer to whether there are any preceding steps that have been executed If the All Open Archiving Requests check box is selected then all open requests for this InfoProvider will be processed including manually managed DAP s as well as those that have not been processed by other processed chains Deletion Phase You need following settings: Continue Open Archiving Request(s) : Check Continue Processing Until Target Status: 70 Deletion Phase Confirmed and Request Completed Auom- Request Invalidation after Error: None Important For InfoCubes with compressed data please note special archiving setting for the noncompressed data (Allow Archiving of Non-Compressed Data ). It the flag is not turned on you have to compress your data in the cube before starting the archiving process. E.g. include the compression step in the process chain prior the archiving steps. April

39