BUSINESS CONTINUITY BEST PRACTICES FOR SAP HANA TAILORED DATACENTER INTEGRATION WITH EMC SYMMETRIX VMAX

BUSINESS CONTINUITY BEST PRACTICES FOR SAP HANA TAILORED DATACENTER INTEGRATION WITH EMC SYMMETRIX VMAX Data protection and availability enabled by EMC Symmetrix Remote Data Facility EMC TimeFinder EMC Solutions Abstract This white paper provides a comprehensive set of EMC recommendations and procedures for data protection and availability using SAP HANA with EMC Symmetrix VMAX 10K, 20K, and 40K arrays in a Tailored Datacenter Integration (TDI) deployment. This deployment includes EMC Symmetrix Remote Data Facility (SRDF) and EMC Timefinder. June 2014

Disclaimer This document outlines a general product direction and should not be relied on in making a purchase decision. This document is not subject to your license agreement or any other agreement with EMC or SAP. Neither EMC nor SAP has an obligation to pursue any course of business outlined in this document or to develop or release any functionality mentioned in this document. This document and the strategy and possible future developments of EMC or SAP are subject to change and may be changed by EMC or SAP at any time for any reason without notice. This document is provided without a warranty of any kind, either express or implied, including but not limited to, the implied warranties of merchantability, fitness for a particular purpose, or noninfringement. EMC and SAP assume no responsibility for errors or omissions in this document, except if such damages were caused by EMC or SAP intentionally or through gross negligence. No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of EMC and SAP AG. Copyright 2014 SAP. All Rights Reserved. SAP and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP AG in Germany and in other countries. See http://www.sap.com/corporate-en/legal/copyright/index.epx for additional trademark information and notices. Copyright 2014 EMC Corporation. All Rights Reserved. EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without notice. The information in this publication is provided as is. EMC Corporation makes no representations or warranties of any kind with respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose. EMC 2, EMC, and the EMC logo are registered trademarks or trademarks of EMC Corporation in the United States and other countries. All other trademarks used herein are the property of their respective owners. For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on EMC.com. Part Number H13149 2

Table of contents Executive Summary... 5 Business case... 5 Solution overview... 5 Key results... 5 Audience... 5 Introduction... 5 Disaster Recovery support in SAP HANA... 6 Backups... 6 HANA System Replication... 6 HANA Storage Replication... 6 SAP HANA storage replication with VMAX... 7 SAP HANA database clones and snapshots with TimeFinder... 7 Business continuity best practice use cases for local and remote SAP HANA storage replications... 8 When to use SRDF/S and SRDF/A... 8 General requirements... 9 Software requirements... 9 Test environment for use cases... 9 Maintain the SAP HANA global.ini file at the remote site... 10 Use case 1: Establishing remote mirroring for disaster protection (synchronous and asynchronous) 11 Process overview... 12 Detailed steps... 12 Use case 2: Moving the SAP HANA persistence devices to the remote site if the source site is unavailable for either planned or unplanned outages... 14 Process overview... 15 Detailed steps... 15 Use case 3: Moving or failing back the HANA persistent devices to the source site... 16 Process overview... 16 Detailed steps... 16 Use case 4: Creating remote restartable and writeable database clones for repurposing (using a HANA storage snapshot)... 18 Process overview... 18 Detailed steps... 19 3

Use case 5: Creating remote restartable and writeable database clones for repurposing (using consistent clones)... 21 Process overview... 21 Detailed steps... 21 Use case 6: Creating clones of source HANA persistent devices at local site for rapid database recovery... 22 Process overview... 23 Detailed steps... 23 Conclusion... 26 Summary... 26 Findings... 26 References... 27 EMC Documentation... 27 SAP documentation... 27 4

Executive Summary Business case Customers deploy SAP databases and integrated applications in many of the most mission-critical functions, including manufacturing, financial accounting, inventory management, sales, and marketing. These and other functions are the life blood of a business, and interruptions or loss of data can be catastrophic. To assure the availability of these systems, for the businesses that depend on them, a comprehensive approach to business continuity planning and execution is required. SAP landscapes include many separate business critical modules that all interact and communicate with critical cross dependencies of data. In addition, every complex business IT landscape includes not only many SAP systems, but non-sap systems that feed SAP, receive data from SAP, or are critical standalone systems. Recovering these many inter-related systems, or periodically testing the effectiveness of this recovery for business and audit requirements, is a unique requirement of federated systems such as SAP. True cross database, cross business system recovery is required for a single point in time in contrast to recovering many individual databases separately. Solution overview Key results EMC VMAX is widely used in SAP landscapes, in mission-critical applications, with traditional databases. The EMC VMAX family now provides the same reliable platform for the software and hardware layer for the SAP HANA database as for traditional databases. In this solution, EMC SRDF with consistency technology and combined with TimeFinder clones, enables SAP customers to sync consistently across databases to reliably restore or test the many business functions in the total SAP landscape. Symmetrix SRDF and Timefinder enable complete high availability and business continuity for mission-critical environments for synchronous and asynchronous data protection. The use cases in this white paper provide best practices for implementing SAP HANA storage replication on VMAX arrays, using proven tools that are already in wide use by customers. The use cases illustrate recovery using TimeFinder and SRDF and show how to implement recoverable and restartable local and remote database replicas. Audience This white paper is for database and system administrators, storage administrators, and system architects who are responsible for implementing, maintaining, and protecting robust SAP HANA in-memory databases and storage systems. Readers should have some familiarity with SAP HANA in-memory databases and EMC software and be interested in achieving higher database availability and protection. Introduction When the SAP HANA in-memory database is used in Tailored Datacenter Integration (TDI) deployments with VMAX enterprise storage arrays, most customers must protect their mission-critical applications and the SAP HANA in-memory database against disasters, hardware or software failures, or human errors, by maintaining a copy of the data either locally or remotely. 5

While SAP HANA offers disaster recovery support with backups or HANA system replication, the HANA storage replication allows customers to seamlessly integrate SAP HANA into their existing business continuity solutions, based on VMAX replication technologies. Note: Storage Configuration Recommendations for SAP HANA TDI on EMC VMAX Storage Systems provides details about TDI. Disaster Recovery support in SAP HANA SAP HANA offers three levels of disaster recovery support backups, storage replication, and system replication. Each solution addresses different recovery point objectives (RPO) within the required recovery time objective (RTO) where: RTO denotes the time allowed for a recovery of the HANA appliance to a specified point of consistency. RPO denotes the point of consistency to which HANA need s to recover. Backups Backups are used to protect the primary HANA persistence against a storage failure. Therefore, the HANA backup target should never be on the same storage array as the primary HANA persistence. Backup systems can typically replicate the backup storage to a remote system to protect against a site failure. With HANA backups, the RPO depends on the frequency at which the customer does HANA backups, which can be from minutes to several hours. The RTO with backups can be several hours because a HANA backup needs to be restored to the primary persistence and from there read into memory before the database is available. EMC offers HANA backup solutions based on the EMC NetWorker and EMC Data Domain product lines. HANA System Replication HANA system replication is an application-based disaster recovery solution where a secondary standby HANA system is configured as an exact copy of the active primary system. HANA system replication allows a multitier system replication where one primary system can have multiple secondary systems. As with storage replication, HANA system replication requires a reliable connection between the primary and secondary sites. The technology for replication supports multiple replication modes, synchronous, synchronous in-memory, and asynchronous. With HANA System replication, only the database content is replicated to the secondary site. HANA Storage Replication HANA Storage Replication in TDI deployments provides a convenient method to protect HANA against a site failure. The HANA primary persistence is replicated to a secondary site. Depending on RTO and RPO requirements by customers and the distance between the primary and secondary site, the implementation of synchronous storage replication (RPO=0), asynchronous storage replication (RPO>0), or a combination of both is possible. If a disaster occurs, the RTO is typically the time it takes to start up the HANA database at the secondary site. 6

HANA Storage Replication provides several benefits to customers, compared to HANA System Replication. These benefits include: Replicate not only to a consistent point-in-time image of the HANA database, but to the secondary site. Include applications and data outside of HANA in the consistency technology of the storage arrays, thus creating a consistent point-in-time image of the overall business applications at the secondary site. In addition to the HANA persistence, with storage replication, you can replicate HANA storage using methods other than the database persistence, such as operating system boot volumes, HANA binaries, and moving configuration files to the secondary site. This white paper includes use cases for SAP HANA local and remote storage replication on VMAX storage arrays. Note: The SAP HANA Administration Guide provides more details about the different disaster recovery solutions and their advantages and disadvantages. SAP HANA storage replication with VMAX The demand for database protection and availability increases as data grows in size and as databases become more interconnected, and it is essential to have continuous access to all databases and applications. Datacenters face disasters that are caused by human errors, hardware and software failures, and natural disasters. When disaster occurs, an organization is measured by its ability to resume operations quickly, seamlessly, and with the minimum amount of data loss. A valid backup and restartable image of the entire information infrastructure helps achieve the desired level of RPO, RTO, and service level agreement (SLA). SAP HANA database clones and snapshots with TimeFinder Every mission-critical system has a need for multiple copies of data to support test system copies and copies as a base for business and IT testing of remote site disaster recovery validation while still maintaining the protection of the primary production databases. With VMAX, using Timefinder software in combination with SAP HANA storage snapshots of the in-memory database, you can create or restore multiple copies of the SAP HANA database persistence (Data and Log) in seconds, either full volume clones or snapshots, regardless of the database size. As soon as TimeFinder activates (or restores) a replica, the target device data is immediately available to the SAP HANA nodes, even as data copy operations continue in the background. 7

Business continuity best practice use cases for local and remote SAP HANA storage replications SAP HANA TDI can be enhanced with different business continuity solutions. EMC SRDF can extend the SAP HANA persistent devices to a remote site. The remote site can be close or at a long distance. The business continuity solutions use EMC SRDF in synchronous mode for close distances or asynchronous mode for long distances. These solutions provide disaster recovery protection. Note: For SRDF replications, a single consistency group was used that included all the SAP HANA persistent devices (data and log), as shown in Table 1. Adding EMC TimeFinder extends the database recovery protection to include a remote replica of SAP HANA persistent devices for repurposing that is restartable and writeable. Database recovery can use SAP HANA database technology or EMC consistency technology. On the source site, EMC TimeFinder can be used for a rapid recovery of the database. The following business continuity solutions include descriptions and command examples for reference: Use case 1: Establishing remote mirroring for disaster protection (synchronous and asynchronous) Use case 2: Moving the SAP HANA persistence devices to the remote site if the source site is unavailable for either planned or unplanned outages Use case 3: Moving or failing back the HANA persistent devices to the source site Use case 4: Creating remote restartable and writeable database clones for repurposing (using a HANA storage snapshot) Use case 5: Creating remote restartable and writeable database clones for repurposing (using consistent clones) Use case 6: Creating clones of source HANA persistent devices at local site for rapid database recovery When to use SRDF/S and SRDF/A SRDF/S is a limited distance solution that is typically implemented in campus or metro environments: In campus solutions, data is typically transmitted over fibre-optic cable, using Symmetrix and SAN equipment with a distance of 66 km or less and using channel extenders or long-distance fibre optic cable. In metro solutions, the distance is typically less than 100 km. Extended Distance Wide Area Network (WAN) provides SRDF connectivity over long distances, using telecommunication networks such as IP, synchronous optical networking (SONET), or asynchronous transfer mode (ATM). SRDF/A must be used in all WAN implementations. Whether SRDF/S can be used in campus or metro implementations depends on the bandwidth available for the SAP HANA persistence and the latency of I/Os. SAP 8

provides a hardware configuration and check tool (hwcct), which must be used to validate the bandwidth and latency of the HANA persistent storage. The results of using this tool must meet the SAP HANA key performance indicators (KPIs) for the persistent storage layer, which are also available from SAP. This applies to nonreplicated environments. Consult your local SAP representative for exceptions for the KPIs for replicated environments. General requirements Software requirements Test environment for use cases Before a SAP HANA disaster recovery solution in a SAP HANA TDI scenario can be established, the following requirements must be met: An SRDF license is applied on both the source and destination VMAX. An SRDF connection exists between the source and the destination. SRDF links are logical connections between Symmetrix SRDF groups and their ports, which are physically connected by cables, routers, extenders, switches, and other network devices. The destination array must be configured with the same HANA volumes and use the EMC TDI guidelines as the source or production array. The use cases have been tested using the following software revisions. Ensure that your environment is at least at these levels: SAP HANA 1.0 SPS07 rev73 SUSE Linux SLES11 for SAP Applications SP2 kernel 3.0.93-5 The use cases described in this document have been tested in a scale-out SAP HANA deployment with three worker nodes and one standby node on a local VMAX 10K array, connected with SRDF to a remote VMAX 10K array, as shown in Figure 1. Figure 1. Test environment for use cases 9

Each VMAX array has a management server connected with Solutions Enabler command line (SYMCLI) installed. The use cases described in the following sections use SYMCLI commands. Table 1 shows the primary devices (also TimeFinder and SRDF source devices) used for the SAP HANA installation and the Solutions Enabler device group. The same number of SAP HANA nodes must be available at the remote site or standby HANA appliance. The nodes use the R2 devices for their HANA persistence when a failover occurs. Based on the storage configuration best practices for SAP HANA in TDI deployments on EMC VMAX, MetaLUNs were created for the primary devices and the SRDF target devices and use Symmetrix RAID1 (2-way-mirror) protection. Table 1. SAP HANA storage layout and VMAX device and composite groups HANA persistence VMAX devices Local device ID Local clone Remote device ID Remote clone SRDF consistency group (CG) RDF group Server01 Data 1.5TB MetaLUN 0048 0268 0108 0268 HANA_CG and HANA_CG_Remote HANA_RDF Server01 Log 512GB MetaLUN 0028 0248 00e8 0248 Server02 Data 1.5TB MetaLUN 0058 0278 0118 0278 Server02 Log 512GB MetaLUN 0030 0250 00f0 0250 Server03 Data 1.5TB MetaLUN 0068 0278 0128 0278 Server03 Log 512GB MetaLUN 0038 0258 00f8 0258 HANA_CG and HANA_CG_Remote HANA_RDF Maintain the SAP HANA global.ini file at the remote site The storage section in the HANA global.ini file contains the references from the HANA storage partitions to the storage LUNs. EMC uses the UUID of the LUN to identify the correct storage devices. The UUIDs can be identified using the following two methods: Method 1, from the SAP HANA node Run the following command on the HANA node to identify the UUID of a 1.5 TB data LUN: server01:~ # multipath -ll grep -B1 1.5T 360000970000298700613533030313038 dm-5 EMC SYMMETRIX size=1.5t features='1 queue_if_no_path' hwhandler='0' wp=ro The string 360000970000298700613533030313038 is the UUID of the corresponding storage LUN. Linux adds a preceding 3 to the storage UUID. 10

Method 2, from a management server using SYMCLI If the VMAX device-id is known (0108 in this example), the UUID can be displayed by running the following command: C:\Program Files\EMC\SYMCLI\bin>symdev -sid 613 list -wwn findstr 0108 0108 Not Visible RDF1+TDEV (M) 60000970000298700613533030313038 Note: The symcli commands display the UUID without the preceding 3. The HANA global.ini file should then look like this example: [storage] ha_provider = hdb_ha.fcclient partition_*_* prtype = 5 partition_1_data wwid = 360000970000298700613533030313038 partition_1_log wwid = 360000970000298700613533030304538 partition_2_data wwid = 360000970000298700613533030313138 partition_2_log wwid = 360000970000298700613533030304630 partition_3_data wwid = 360000970000298700613533030313238 partition_3_log wwid = 360000970000298700613533030304638 Note: If HANA needs to start at the secondary site with the mirrored R2 devices or the clone devices, ensure that the storage section of the global.ini file contains the UUIDs pointing to the R2 or clone devices. Use case 1: Establishing remote mirroring for disaster protection (synchronous and asynchronous) Use case 1 shows the steps to create remote disaster recovery protection using EMC SRDF in a synchronous or an asynchronous mode. The protection is performed on the SAP HANA database persistent (Data and Log) devices. The R1 devices in the source VMAX will be replicated to the R2 devices in the target VMAX. While the replication is active, writing to the R2 devices is not possible. Figure 2 illustrates this use case. Figure 2. Establish remote mirroring 11

Process overview The following steps summarize the process for establishing remote mirroring from the source site to the target site. The Detailed steps section provides command examples. 1. Create an RDF group between the two sites. 2. Create the device pairs text file. 3. Create device pairs and set the RDF mode. 4. Create the composite group HANA_CG and add the devices. 5. Perform initial synchronization of SRDF in adaptive copy mode. 6. After the SRDF target is synchronized with the source, change the replication mode to SRDF/S or SRDF/A. 7. Enable consistency using composite group HANA_CG. 8. Optionally, associate the DSE pool for SRDF/A. Detailed steps Use the following command examples for this use case: Note: EMC Solutions Enabler Symmetrix SRDF Family CLI Product Guide provides a full explanation of the command structure. 1. Create an RDF group between the two sites: symrdf addgrp label HANA_RDF rdfg 1 sid 460 -dir 1f,2f remote_rdfg 1 remote_sid 613 -remote_dir 1f,2f 2. Create a text file on the management server which lists the source and the target device pairs. Map the source data1 LUN with the target data1 LUN, the source data2 with the target data2, and so on. # cat hana_pairs.txt 0048 0108 0058 0118 0068 0128 0028 00e8 0030 00f0 0038 00f8 3. Create the device pairs in the HANA_RDF group and perform and set the RDF mode to adaptive copy disk mode. The target devices must be write_disabled. symdev sid 613 dev 0108,0118,0128,00e8,00f0,00f8 write_disable symrdf file hana_pairs.txt sid 460 -rdfg 1 createpair type rdf1 invalidate r2 rdf_mode acp_disk 4. Create composite group HANA_CG and add the local HANA devices to this group. 12

Ensure that the same group HANA_CG is created on the management server at the remote site as follows: symcg type RDF1 create HANA_CG rdf_consistency symcg sid 460 cg HANA_CG add dev 0048 data1 symcg sid 460 cg HANA_CG add dev 0058 data2 symcg sid 460 cg HANA_CG add dev 0068 data3 symcg sid 460 cg HANA_CG add dev 0028 log1 symcg sid 460 cg HANA_CG add dev 0030 log2 symcg sid 460 cg HANA_CG add dev 0038 log3 Run the following command to display the device group details: symrdf cg HANA_CG query 5. Perform initial synchronization of SRDF in adaptive copy mode: symrdf cg HANA_CG establish Run the query command to verify the status of the synchronization. After the values for R2 Inv Tracks are close to zero, continue with the next step. 6. When the replication nears completion and there are limited number of invalid tracks outstanding between the source and the target, change the replication mode to SRDF/S or SRDF/A: For SRDF/S, set protection mode to synchronous: symrdf cg HANA_CG set mode sync For SRDF/A, set protection mode to asynchronous: symrdf cg HANA_CG set mode async Run the query command to verify the new status of the RDF pairs: For SRDF/S, the RDF pair state is synchronized. For SRDF/A, the state is consistent. 7. Enable consistency for the group HANA_CG: symcg cg HANA_CG enable 8. For SRDF/A, associate a DSE pool to the RDF group HANA_RDF. A DSE pool contains save devices where delta set data is offloaded to protect SRDF/A sessions from a longer duration of unbalance. DSE pools must be created on both arrays before the following commands can be used: symrdf sid 460 rdfg 1 set rdfa_dse fba_pool HANA_DSE autostart on symrdf sid 613 rdfg 1 set rdfa_dse fba_pool HANA_DSE autostart on Note: HANA_DSE is the name of the preconfigured DSE pool for HANA. 13

Use case 2: Moving the SAP HANA persistence devices to the remote site if the source site is unavailable for either planned or unplanned outages Use case 2 explains the options to move or failover the HANA persistent devices to the remote site by using SRDF/S or SRDF/A. If a disaster occurs, use this use case to enable database persistence (R2 devices) for the standby HANA installation at the remote site. Figure 3 illustrates the scenario when the devices have been failed over to the target site. Figure 3. Failover to the remote site After the source site is repaired and is back online, as shown in Figure 4, establish replication from the destination to the source array by swapping the personality of the R1 and R2 devices. This ensures data protection of production while running at the remote site. Figure 4. Enable remote protection with production running on a target site 14

Process overview The following steps summarize the process for failing over to the remote site. The Detailed steps section provides command examples. 1. Create composite group HANA_CG_remote and add devices on the remote management server 2. Failover composite group HANA_CG_remote to write-enable the R2 devices 3. If it is a scheduled failover to the remote site and the source VMAX is still available: a. Swap the personality of the composite group HANA_CG_remote. b. Establish the composite group HANA_CG_remote. Detailed steps Use the following command examples for this use case: Note: EMC Solutions Enabler Symmetrix SRDF Family CLI Product Guide provides a full explanation of the command structure. 1. Failover composite group HANA_CG_remote to write-enable the R2 devices: symrdf cg HANA_CG_remote failover The R2 devices in the composite group HANA_CG_remote are now in read/write enabled state and SAP HANA can be restarted at the remote site. Ensure that the global.ini file of the standby HANA appliance contains the partition entries with the pointers to the UUIDs of the R2 devices. 2. If the failover to the remote site is scheduled and the source VMAX is still available, or the source site is back online after a disaster, swap the personality of the devices; this changes R2 to an R1 device and vice versa and ensures that production can run on the remote site during replication to the source site: symrdf cg HANA_CG_remote query Run the query command to verify the status of the device. When the values for RDF Pair STATE are Failed Over, the personality of the devices can be swapped and the RDF link re-established to continue storage replication. a. Swap personality of the composite group HANA_CG_remote: symrdf cg HANA_CG_remote swap The personality of the devices is changed from R2 to R1 and vice versa. b. Establish the composite group HANA_CG_remote. symrdf cg HANA_CG_remote establish The RDF link is re-established: symrdf cg HANA_CG_remote query Run the query command to verify the status of the synchronization. 15

Use case 3: Moving or failing back the HANA persistent devices to the source site Use case 3 illustrates moving or failing back the HANA persistence devices to the source site using SRDF/S or SRDF/A. Note: Be absolutely sure which data is needed. In this use case, the data at the remote site is restored to the source VMAX system. Perform these steps from the management server at the source site and using the composite group HANA_CG. Process overview The following steps summarize the process for failing back to the source site. The Detailed steps section provides command examples. 1. If the source site was not available for a period of time, this process will return the configuration to the source site for composite group HANA_CG: a. Set the RDF mode to adaptive copy to minimize the performance impact of a full restore. b. Restore data from the remote VMAX to the source site. c. Set data protection to synchronous or asynchronous. d. Re-enable consistency using composite group HANA _CG. 2. A scheduled failover will have minimal changes between the source and target site. This process will swap the personality of the devices to return the configuration to the source site for composite group HANA_CG: a. Failover composite group HANA_CG. b. Swap the personality of the composite group HANA_CG. c. Establish the composite group HANA_CG. Detailed steps Use the following command examples for this use case: Note: EMC Solutions Enabler Symmetrix SRDF Family CLI Product Guide provides a full explanation of the command structure. 1. To minimize impact when the source site is unavailable, use this process to return the composite group HANA_CG to the source site. To display the device group details, run: symrdf cg HANA_CG query 16

Run the query command to verify the status of the device. If a new or recovered VMAX system is on the source site, the value for RDF Pair State is Split and a full data restore is needed: a. To reduce the impact of a full restore, set the RDF mode to adaptive copy disk mode: symrdf cg HANA_CG set mode acp_disk b. Restore data from the remote VMAX to the source site: symrdf cg HANA_CG restore -full The R1 devices in the composite group HANA_CG are now read/writeenabled. c. Restart SAP HANA at the source site: symrdf cg HANA_CG query Run the query command to verify the status of the synchronization. When the values for R1 Inv Tracks are close to zero, continue with the next step. d. When the replication nears completion and there are limited number of invalid tracks outstanding between the source and the target, change the replication mode to SRDF/S or SRDF/A For SRDF/S, set protection mode to Synchronous: symrdf cg HANA_CG set mode sync For SRDF/A, set protection mode to Asynchronous: symrdf cg HANA_CG set mode async e. Re-enable consistency for the group HANA_CG: Symcg cg HANA_CG enable 2. In a scheduled failover process, make a personality swap of the devices to return to the source site for HANA_CG: symrdf cg HANA_CG query Run the query command to verify the status of the device. When the replication is synchronized for RDF Pair STATE, continue with the following steps: a. Failover composite group HANA_CG: symrdf cg HANA_CG_remote failover b. Swap the personality of the composite group HANA_CG: symrdf cg HANA_CG swap The personality of the devices is changed from R2 to R1 and vice versa. c. Establish the composite group HANA_CG: symrdf cg HANA_CG establish 17

Use case 4: Creating remote restartable and writeable database clones for repurposing (using a HANA storage snapshot) Use the SAP HANA database to clone a database while it is running, using the underlying storage system. This use case ensures a consistent database by creating a database-internal snapshot from HANA. This use case details the steps required to set up and activate clones of the R2 devices at the remote site for test, development, or QA purposes using TimeFinder clones. Figure 5 illustrates the scenario where production is running at the source site and the production data (R1 devices) is replicated to the target site (R2 devices) while clones of the R2 devices are used to start a HANA database at the target site for development, test, or QA purposes. Figure 5. Using database clones at the target site for repurposing Process overview The following steps summarize the process using the HANA storage snapshot for creating remote restartable and writeable database clones for repurposing. The Detailed steps section provides command examples. 1. Add clone or snap devices to the composite group HANA_CG_remote. 2. Create the device pairs file hana_clone_txt. 3. Pair the source R2 and target clone devices in the composite group HANA_CG_remote. 4. Prepare the SAP HANA database for a storage snapshot. 5. Activate the HANA_CG_remote clone with consistent. 6. Close the SAP HANA database storage snapshot. 18

Detailed steps Use the following command examples for this use case: Note: EMC Solutions Enabler Symmetrix SRDF Family CLI Product Guide provides a full explanation of the command structure. 1. Add clone or snap devices to the composite group HANA_CG_remote. Note: Even though only the HANA data devices must be cloned or snapped because a full restartable HANA snapshot is located in the data volume, we also clone the Log devices, because a HANA database restart at the target site requires both Data and Log. For a restart from a snapshot, the cloned Log devices will be cleared. symcg sid 613 cg HANA_CG_remote add dev 0268 tgt symcg sid 613 cg HANA_CG_remote add dev 0278 tgt symcg sid 613 cg HANA_CG_remote add dev 0288 tgt symcg sid 613 cg HANA_CG_remote add dev 0248 tgt symcg sid 613 cg HANA_CG_remote add dev 0250 tgt symcg sid 613 cg HANA_CG_remote add dev 0258 tgt 2. Create a text file on the management server which lists the source and the target device pairs. Map the R2 target data1 LUN with the cloned data1 LUN, the R2 target data2 with the clone data2, and so on. # cat hana_clone.txt 00e8 0248 00f0 0250 00f8 0258 0108 0268 0118 0278 0128 0288 3. Pair the source R2 and target clone devices in the composite group HANA_CG_remote: symclone sid 613 f hana_clone.txt create copy diff -nop Run the following command to display the device group details: symclone cg HANA_CG_remote query The query command verifies the status of the clone devices in the composite group HANA_CG_remote. The state of the devices must be Created. 4. Prepare the SAP HANA database for a storage snapshot. Type the following command from the HANA nameserver node server01 and the <sid>adm user: server01:/usr/sap/tdi/hdb00> hdbsql -n server01 -i TDI -u system -p <system password> BACKUP DATA CREATE SNAPSHOT where server01: nodname of first HANA node (master nameserver) TDI: HANA sid HDB00: 00 is the HANA instance number 19

system: password: Database user with snapshot privileges Password of database user Note: The SAP HANA Administration Guide provides further information about HANA storage snapshots. If SRDF/A is used, use an SRDF checkpoint command to ensure that the SRDF target R2 devices and the clones also have the snapshot prepared: symrdf cg HANA_CG checkpoint 5. Activate the HANA_CG_remote clone with consistent. After the device group is activated, use the clones of the remote R2 devices for HANA. The cloned database is restored during the first HANA restart: symclone cg HANA_CG_remote tgt consistent activate Run the following command to display the device group details: symclone cg HANA_CG_remote query The state of the devices should be CopyInProg. 6. Close the SAP HANA database storage snapshot. To close the SAP HANA storage snapshot, the BACKUP_ID of the snapshot must be retrieved using the following command: server01:/usr/sap/tdi/hdb00> hdbsql -n server01 -i TDI -u system -p <system password> "SELECT * FROM "PUBLIC"."M_BACKUP_CATALOG" WHERE ENTRY_TYPE_NAME = 'data snapshot' AND STATE_NAME = 'prepared'" Output: ENTRY_ID,ENTRY_TYPE_NAME,BACKUP_ID,SYS_START_TIME,UTC_START_ TIME,SYS_END_TIME,UTC_END_TIME,STATE_NAME,COMMENT,MESSAGE 1398776580303,"data snapshot",1398776580303,"2014-04-29 06:03:00.303000000","2014-04-29 13:03:00.303000000",?,?,"prepared","","" Use the Backup_ID to close the storage snapshot by running this command: server01:/usr/sap/tdi/hdb00> hdbsql -n server01 -i TDI -u system -p <system password> "BACKUP DATA CLOSE SNAPSHOT BACKUP_ID 1398776580303 SUCCESSFUL 'Remote Storage Clone'" The Data and Log clones are now usable at the remote site to start with a point-in-time copy of the production database (the SAP HANA snapshot) at the remote site, while still running production at the source site and while SRDF/S or SRDF/A replication to the destination site is active. When SAP HANA starts from a data persistence (in this case from the clones) that contains a storage snapshot, this snapshot is restored during the first restart and then removed. 20

If the VMAX clone in step 5 failed, abandon the SAP HANA storage snapshot by running the following command: server01:/usr/sap/tdi/hdb00> hdbsql -n server01 -i TDI -u system -p <system password> "BACKUP DATA CLOSE SNAPSHOT BACKUP_ID 1398776580303 UNSUCCESSFUL 'Storage Clone Failed'" Use case 5: Creating remote restartable and writeable database clones for repurposing (using consistent clones) Use case 5 is similar to use case 4. The difference is that a SAP HANA database snapshot is not required. The consistency technology in the VMAX array enables the cloning of a group of devices while ensuring a dependent-write consistent image across all devices. Process overview The following steps summarize the process for using consistent clones for creating remote restartable and writeable database clones for repurposing. The Detailed steps section provides command examples. 1. Add clone or snap devices to the composite group HANA_CG_remote. 2. Create the device pairs file hana_clone_txt. 3. Pair the source R2 and target clone devices in the composite group HANA_CG_remote. 4. Activate the HANA_CG_remote clone with consistent. Detailed steps Use the following command examples for this use case: Note: EMC Solutions Enabler Symmetrix SRDF Family CLI Product Guide provides a full explanation of the command structure. 1. Add clone or snap devices to the composite group HANA_CG_remote. Note: Even though only the HANA data devices must be cloned or snapped because a full restartable HANA snapshot is located in the data volume, we also clone the Log devices, because a HANA database restart at the target site requires both Data and Log. For a restart from a snapshot, the cloned Log devices will be cleared. symcg sid 613 cg HANA_CG_remote add dev 0268 tgt symcg sid 613 cg HANA_CG_remote add dev 0278 tgt symcg sid 613 cg HANA_CG_remote add dev 0288 tgt symcg sid 613 cg HANA_CG_remote add dev 0248 tgt symcg sid 613 cg HANA_CG_remote add dev 0250 tgt symcg sid 613 cg HANA_CG_remote add dev 0258 tgt 2. Create a text file on the management server which lists the source and the target device pairs. Map the R2 target data1 LUN with the clone data1 LUN, the R2 target data2 with the clone data2, and so on. # cat hana_clone.txt 00e8 0248 00f0 0250 21

00f8 0258 0108 0268 0118 0278 0128 0288 3. Pair the source R2 and target clone devices in the composite group HANA_CG_remote: symclone sid 613 f hana_clone.txt create copy diff -nop Run the following command to display the device group details: symclone cg HANA_CG_remote query This query command verifies the status of the clone devices in the composite group HANA_CG_remote. The state of the devices must be Created. 4. Activate the HANA_CG_remote clone with consistent. After the device group is activated, you can use the clones of the remote R2 devices for HANA. The cloned database is restored during the first HANA restart: symclone cg HANA_CG_remote tgt consistent activate Run the following command to display the device group details: symclone cg HANA_CG_remote query The state of the devices must be CopyInProg. After the activate command is run, start the SAP HANA database at the target site by using the clone devices. Unlike use case 4, HANA will go through a recovery process once it is started. This is similar to an event when HANA is restarted after a power failure. Use case 6: Creating clones of source HANA persistent devices at local site for rapid database recovery Use case 6 outlines the steps required to create clones of the source HANA persistent (R1) devices at the local site for a rapid database recovery. In situations where the database must be recovered to a given point-in-time, for example if the database is being loaded from an external source and it is not clear whether or not the database load was successful, create a clone of the R1 devices before loading the database. If the load failed or was incomplete, by restoring the clones to the original R1 devices, a status of the database before the data load will be restored. Figure 6 illustrates using a local TimeFinder replica while replication from the source R1 devices to the target R2 devices is continuing. 22

Figure 6. Using a local restartable TimeFinder replica for database recovery Process overview The following steps summarize the process for creating clones of R1 devices at the local site for rapid database recovery. The Detailed steps section provides command examples. 1. Add clone (or snap) devices to the composite group HANA_CG. 2. Create the device pairs hana_clone_local.txt file. 3. Pair the source R1 and local clone devices in the composite group HANA_CG. 4. Prepare the HANA database for a storage snapshot. 5. Activate the HANA_CG clone with consistent. 6. Close the HANA database storage snapshot. Detailed steps Use the following command examples for this use case: Note: EMC Solutions Enabler Symmetrix SRDF Family CLI Product Guide provides a full explanation of the command structure. 1. Add clone or snap devices to the composite group HANA_CG: symcg sid 460 cg HANA_CG add dev 0268 tgt symcg sid 460 cg HANA_CG add dev 0278 tgt symcg sid 460 cg HANA_CG add dev 0288 tgt symcg sid 460 cg HANA_CG add dev 0248 tgt symcg sid 460 cg HANA_CG add dev 0250 tgt symcg sid 460 cg HANA_CG add dev 0258 tgt 2. Create a text file on the management server which lists the source and the clone device pairs. Map the source data1 LUN with the clone data1 LUN, the source data2 with the clone data2, and so on: # cat hana_clone_local.txt 0028 0248 0030 0250 0038 0258 0048 0268 23

0058 0278 0068 0288 3. Pair the source R1 and local clone devices in the composite group HANA_CG: symclone sid 460 f hana_clone_local.txt create copy diff -nop Run the following command to display the device group details: symrdf cg HANA_CG query You can use the query command to verify the status of the clone devices in the composite group HANA_CG. The state of the devices must be Created. 4. Prepare the HANA database for a storage snapshot: server01:/usr/sap/tdi/hdb00> hdbsql -n server01 -i TDI -u system -p manager BACKUP DATA CREATE SNAPSHOT Note: The SAP HANA Administration Guide provides further information about the HANA database snapshot mode. 5. Activate the HANA_CG clone with consistent: symclone cg HANA_CG tgt consistent activate Run the following command to display the device group details: symclone cg HANA_CG_remote query The query command verifies the status of the clone devices in the composite group HANA_CG. The state of the devices should be CopyInProg. 6. Close the HANA database storage snapshot: server01:/usr/sap/tdi/hdb00> hdbsql -n server01 -i TDI -u system -p <system password> "SELECT * FROM "PUBLIC"."M_BACKUP_CATALOG" WHERE ENTRY_TYPE_NAME = 'data snapshot' AND STATE_NAME = 'prepared'" Output: ENTRY_ID,ENTRY_TYPE_NAME,BACKUP_ID,SYS_START_TIME,UTC_START_ TIME,SYS_END_TIME,UTC_END_TIME,STATE_NAME,COMMENT,MESSAGE 1398776580303,"data snapshot",1398776580303,"2014-04-29 06:03:00.303000000","2014-04-29 13:03:00.303000000",?,?,"prepared","","" Use the Backup_ID to close the snapshot mode. Use the Backup_ID format from the output when running the following command: server01:/usr/sap/tdi/hdb00> hdbsql -n server01 -i TDI -u system -p <system password> "BACKUP DATA CLOSE SNAPSHOT BACKUP_ID 1398776580303 SUCCESSFUL 'Storage Clone'" 7. Restore local clone devices to the R1 source devices. 24

When the HANA database needs to be recovered to the point-in-time of the previous HANA storage snapshot, restore the clone devices to the source R1 devices using the following command: Note: The SAP HANA database must be stopped before the R1 devices can be restored and the composite group must be in the Copied state. symclone cg HANA_CG tgt restore 8. Restart the HANA database. 25

Conclusion Summary Customers who deploy SAP landscapes, including those who use the SAP HANA database, can be assured that their mission-critical data is protected and continuously available. EMC SRDF with consistency technology and combined with TimeFinder clones enables SAP customers to sync consistently across databases to reliably restore or test the many business functions in the total SAP landscape. The EMC VMAX family now provides the same reliable platform for the software and hardware layer for the SAP HANA database as for traditional databases. Symmetrix SRFDF and Timefinder enable complete high availability and business continuity for mission-critical environments for synchronous and asynchronous data protection. Findings The use cases in this white paper provide best practices for implementing SAP HANA storage replication on VMAX arrays, using proven tools that are already in wide use by customers. The use cases illustrate recovery using TimeFinder and SRDF and show how to implement recoverable and restartable local and remote database replicas. 26

References EMC Documentation SAP documentation You can find the following documents on EMC Online Support: EMC Symmetrix Remote Data Facility (SRDF) Product Guide Provides SRDF product family, concepts, and theory of operation for the Symmetrix platforms, including VMAX 40K, VMAX 20K/VMAX, VMAX 10K, VMAXe, and DMX. Storage Configuration Recommendations for SAP HANA TDI on EMC VMAX Storage Systems Revokes limitations of the current SAP HANA Appliance Model. Using Tailored Data Center Integration (TDI) on EMC VMAX, move SAP HANA into a private cloud and integrate into an existing, data center infrastructure. EMC Solutions Enabler Symmetrix SRDF Family CLI Product Guide provides a full explanation of the command structure in the Detailed steps sections for the use cases. Solutions Enabler Symmetrix CLI 7.6 Command Reference Provides a reference for command-line users and script programmers by describing all of the SYMCLI commands, daemons, error codes, and option file parameters of the EMC Solutions Enabler software. Solutions Enabler Symmetrix Array Management CLI 7.6 Product Guide Provides a guide for command-line users and script programmers. The manual describes how to manage Symmetrix arrays and their devices for an enterprise storage environment using the SYMCLI commands of the EMC Solutions Enabler software. It includes information on basic monitoring and control, storage devices and device grouping, group name services, access control, statistics, gatekeepers, and managing track changes. You can find the following SAP documentation at http://help.sap.com/hana_appliance SAP HANA Technical Operations Manual (TOM) Provides an entry point for administering and operating your SAP HANA system landscape. SAP HANA Administration Guide Describes all administrative tasks for SAP HANA that system administrators perform regularly and on demand. 27