SAP HANA TDI Configuration Guidelines for Lenovo System x Servers and EMC VMAX Storage using IBM GPFS and XFS File Systems

Transcription

1 TECHNICAL WHITEPAPER SAP HANA TDI Configuration Guidelines for Lenovo System x Servers and EMC VMAX Storage using IBM GPFS and XFS File Systems ABSTRACT This white paper describes the advantages of using EMC VMAX enterprise storage and Lenovo System x servers in an SAP HANA TDI environment. Included are best practices and configuration guidelines for EMC VMAX storage and Lenovo System x Server with IBM GPFS and XFS file systems in a SAP HANA TDI implementation that meets or exceeds all SAP performance requirements and ensures the highest availability for database persistence on disk. April 2015 and EMC VMAX Storage Using IBM GPFS and XFS File System 1

2 Table of contents Abstract... 1 Table of contents... 2 Executive Summary... 4 Introduction... 5 Purpose... 5 Scope... 5 Audience... 5 Lenovo System x server and EMC VMAX storage overview... 6 Lenovo System x offerings in SAP HANA TDI compute area... 6 Lenovo System x Enterprise solution services for SAP HANA... 7 EMC Enterprise Storage offerings in SAP HANA TDI scenarios... 8 Disaster Tolerance using EMC VMAX SRDF System x UEFI settings for MAX performance in SAP HANA environment Lenovo System x UEFI Settings Intel Xeon E5 and E7 Based Systems Intel Xeon E5 V2 Processor Based Systems Intel Xeon E7 V2 Processor Based Systems Intel Xeon E5 V3 Processor Based Systems Setting C-States Using the Advanced Settings utility Using the System x UEFI Boot Screen SAP HANA System Check IBM GPFS Installation and Configuration Network Configuration Customize /etc/hosts file SSH Configuration Install GPFS Packages Update to the latest GPFS Maintenance release Build the GPFS Portability Layer Prepare GPFS Configuration GPFS Cluster configuration XFS Configuration for SAP HANA scale-out using storage connector API Network Configuration and EMC VMAX Storage Using IBM GPFS and XFS File System 2

3 XFS file system configuration for 2+1 HANA Scale out SAP HANA global.ini file HANA DB Installation HA tests (Scale-out node failover test) Worker Node failover Master/Worker Node Failover Hardware failure simulation Appendix-A HANA compute node details SAP HANA Database version used for the test s SAN Details Appendix-B Lenovo Compute building blocks for SAP HANA TDI General capacity considerations for VMAX storage in TDI QLogic Fibre Channel Adapters for Lenovo System x Servers HWCCT Resources Notices Trademarks Disclaimer This document outlines a general product direction and should not be relied on in making a purchase decision. This document is not subject to your license agreement or any other agreement with Lenovo, EMC or SAP. Lenovo, EMC or SAP has no obligation to pursue any course of business outlined in this document or to develop or release any functionality mentioned in this document. This document and the strategy and possible future developments of Lenovo, EMC or SAP are subject to change and may be changed by Lenovo, EMC or SAP at any time for any reason without notice. This document is provided without a warranty of any kind, either express or implied, including but not limited to, the implied warranties of merchantability, fitness for a particular purpose, or non-infringement. Lenovo, EMC and SAP assume no responsibility for errors or omissions in this document, except if such damages were caused by Lenovo, EMC or SAP intentionally or grossly negligent. No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of Lenovo, EMC and SAP AG. Copyright 2015 SAP. All Rights Reserved. SAP and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP AG in Germany and in other countries. See for additional trademark information and notices. and EMC VMAX Storage Using IBM GPFS and XFS File System 3

4 Executive Summary In today's fast-paced world, you need to know what's going on with your business, your marketplace, and your customers so you can respond quickly. Many enterprises are looking to obtain real business insights using SAP HANA. However, the move to SAP HANA can be complex, so choosing your business and technology partners wisely is an important first step of this journey. Lenovo and EMC are working closely with SAP to provide you with seamless, secure, contextaware experiences for your SAP environment. For SAP HANA, Lenovo and EMC offer customers a complete set of fully integrated solutions and services with Lenovo's System x servers and EMC Enterprise Storage, for customers that wish to deploy their own SAP HANA environments. By default, a SAP HANA appliance includes integrated storage, compute, and network components. SAP has defined key performance indicators for storage subsystems, such as I/O throughput and latency, ensuring that the overall infrastructure forms a HANA-ready environment. You can also deploy SAP HANA appliances using a Tailored Data Center Integration (TDI) approach. With this approach, you can connect SAP certified servers to existing enterprise storage systems. This reduces time-to-value, risk, and costs for an overall HANA adoption. With this approach, you can leverage existing datacenter processes, as well as storage infrastructure. The SAP HANA configuration outlined in this white paper is based on Lenovo's System x servers and an EMC VMAX enterprise storage system. The document covers SAP HANA scale-out architecture using IBM General Parallel File System (GPFS) and XFS in an SAP HANA scale-out implementation. The solution described in this white paper combines the technologies of EMC and Lenovo to create a best-in-class SAP HANA TDI implementation. Lenovo and EMC engineering performed many performance tests using the SAP HANA hardware configuration and check tool (HWCCT) on an EMC VMAX enterprise storage system and System x severs using IBM GPFS and XFS. Based on these tests, this white paper describes a scale-out SAP HANA configuration that meets or exceeds all SAP performance requirements and ensures the highest availability for database persistence on disk. This is one option of many that customers can define to build a HANA database. Refer to Appendix A for more information on the VMAX storage and System x servers used in these tests. SAP recommends that TDI customers run the HWCCT tool in their environment to ensure that the customer specific HANA TDI implementation meets the SAP performance criteria. This technical white paper gives you the details of how you can integrate EMC VMAX Storage and System x servers with SAP HANA TDI using SAP specifications for enterprise storage. This paper describes key solution components, configuration, and implementation details, plus provides information on the unique architectural advantages of EMC VMAX Storage and Lenovo's System x servers for TDI. This paper includes step-by-step instructions to configure the IBM GPFS file system and the XFS file system on a VMAX array for SAP HANA scale-out configuration in TDI deployments on System x servers. and EMC VMAX Storage Using IBM GPFS and XFS File System 4

5 Introduction Purpose To meet customers needs for SAP HANA, Lenovo offers two distinct delivery alternatives: complete appliance delivery or TDI. The Lenovo System x Solution for SAP HANA is a fully integrated, highly optimized appliance solution that offers the performance and robustness needed to handle the challenges of a combined transaction and analytics platform. It is easy to implement with convenient preconfigured hardware and preinstalled software. The solution is fully validated by both Lenovo and SAP. However, when a more customized data center solution is desired as an alternative to appliance-based solutions, Lenovo and EMC offers solutions for SAP HANA TDI. EMC VMAX storage integration is fully certified and authorized by SAP to give you the flexibility to specify key components within your SAP HANA infrastructure. Although customers take responsibility for the validation of their SAP HANA environment with TDI, this approach offers a few key benefits: Reduced costs through the use of existing hardware components Mitigated risk and improved time to value by enabling existing operation and management processes Scope This white paper serves describes the following: IBM GPFS Installation and configuration for SAP HANA scale out in 1F XFS file system configuration for SAP HANA scale out. TDI. SAP HANA High Availability using SAP HANA storage connector API using VMAX shared storage and System x servers. Key benefits of using VMAX enterprise storage and System x servers with SAP HANA. Audience This whitepaper is intended for Lenovo and EMC system integrators, systems and storage administrators, partners, customers, members of Lenovo Lab Services and EMC professional services who will be involved in a SAP HANA TDI implementation. 1 In this paper we primarily discuss SAP HANA scale-out architecture and high-availability feature tests on IBM GPFS and XFS, but a SAP HANA scale-up (single node) implementation for Business Warehouse (BW) on HANA and Suite on HANA (SoH) implementation is a tested and validated configuration by Lenovo and EMC. and EMC VMAX Storage Using IBM GPFS and XFS File System 5

6 Lenovo System x Server and EMC VMAX This section provides information on EMC VMAX enterprise storage and Lenovo System x servers. The System x server portfolio for SAP HANA offers unprecedented scalability. Large memory configurations are available for SAP Business Suite, powered by SAP HANA. And for SAP Business Warehouse (BW) and data mart implementations, you can easily create multinode scale-out configurations by networking multiple nodes together to enable support for larger SAP HANA memory sizes. This modular approach enables you to invest in an optimized solution for SAP HANA and grow your infrastructure as your SAP HANA environment grows. In addition, the solution, using System x servers and EMC VMAX storage, integrates high availability and disaster recovery with automatic failover, by using EMC SRDF (Symmetrix Remote Data Facility), IBM GPFS file system, or the XFS file system. Landscape optimization and datacenter readiness You can choose from a preconfigured solution or a custom TDI solution. Either way, you can effectively protect your data with integrated high availability and synchronous disaster recovery capabilities using IBM General Parallel File System (GPFS) software and EMC SRDF. Additionally, VMware virtualization technology can be used for optimizing the deployment and management of your SAP HANA application environment using virtual machines via a single, SAP HANA certified system. Lenovo System x offerings in the SAP HANA TDI compute area Choice of base memory size: 128GB, 256GB, 512GB, 1TB or 2TB Scalable memory support up to 6TB production and 12TB non-production for SAP Business Suite powered by SAP HANA Flex node Partitioning support enables you to maximize your IT resources and provide flexible capacity planning for non-production environments. It allows you to remotely and automatically partition your 8-socket x3950 X6 (Lenovo MTM: 6241) SAP HANA System x as one 8-socket system or two x3850 X6 4-socket systems, and change back and forth as needed. Software: IBM General Parallel File System (GPFS) on x86 Single Server license. To use these server models in a scale-out cluster, additional GPFS Server and GPFS FPO licenses will need to be purchased. The new System x X6 enterprise servers with enterprise X-Architecture are designed to provide faster performance and support for more memory. Using self contained compute books and Intel and EMC VMAX Storage Using IBM GPFS and XFS File System 6

7 Xeon E7 product family processors, the System x Solution for SAP HANA on X6 is purpose-built to support large-scale SAP HANA deployments by incorporating up to 120 cores per server, for maximum CPU performance and proven expansion capacity to scale up to a 100 TB SAP HANA system. The Lenovo Flex System x880 is the world s first 8-socket Intel Xeon processor-based blade server. It provides up to 2x the capacity of competitive blade servers and offers scalability from 2- to 4- or 8-sockets, enabling you to scale up processors and memory for large SAP Business Suite on HANA databases and add IT resource to your environment as your demands increase. It offers high-end server resiliency and availability to help maximize application uptime for critical SAP database and business applications. In addition, it provides the ability to integrate and optimize compute, storage and networking resources with simplified management using the no-comprise building blocks of the Flex System infrastructure. Figure 1: Flex System x880 X6 8-socket x3950 X6 (MTM: 6241) The solution is delivered on Flex System x880 X6 compute nodes that provide leading performance and scalability. SAP HANA efficiently processes and analyzes massive amounts of data by packaging SAP s use of in-memory technology, columnar database design, data compression and massive parallel processing combined with essential tools and functionality, such as data replication and analytic modeling. Using SAP HANA as a transactional single-node RDBMS (e.g., SAP Business Suite), customers can benefit from the in-box memory scalability of X6 technology. and EMC VMAX Storage Using IBM GPFS and XFS File System 7

8 System x enterprise solution services for SAP HANA The System x Lab Services team helps clients worldwide with deployment of System x solutions and technologies. Locally or globally, we offer installation, configuration, integration, migration and other professional services in support of System x servers and solutions. Whether training staff to help a customer site to get up to speed, performing health check services for an existing environment, or architecting the next design, we have offerings and expertise to meet the customer s needs. SAP HANA managed services Lab Services will help you configure, validate hardware and software stack (OS, GPFS), and SAP, configure networking, provide basic skills transfer and post-install documentation for SAP HANA on System x hardware. HULearn moreu SAP HANA managed services This service offers constant remote monitoring of HANA hardware and software. It also includes monitoring of HANA software stack (OS, GPFS, and SAP), ongoing health checks, scheduled upgrades, ownership and resolution of related problems, and maintenance of records. HULearn moreu SAP HANA health check services This offering provides health check services for SAP HANA. It includes on-site assessment of HANA software stack (OS, GPFS, SAP), networking, solution configuration, and post-install documentation on System x hardware. This service includes maintaining and updating firmware and software for the SAP HANA solution on System x hardware to SAP and Lenovo best practices levels. HULearn moreu EMC VMAX enterprise storage offerings in SAP HANA TDI scenarios The specific SAP HANA I/O workload requires special considerations for the configuration of persistence for data and log volumes on a shared EMC VMAX storage array. The VMAX system delivers the high-performance requirements for the persistent storage of a SAP HANA database, without impacting other applications concurrently using the same array. The I/O workload on SAP HANA has two major components: Random I/Os on the data volume Sequential I/Os on the log volume SAP HANA uses different block sizes for log volumes and data volumes. I/O for data volumes uses 4 K, 16 K, 64 K, 1 M, 16 M, and 64 M block sizes. and EMC VMAX Storage Using IBM GPFS and XFS File System 8

9 I/O for log volumes uses 4 K, 16 K, and 1 M block sizes. EMC VMAX scalability In a 10K, 20K, or 40K VMAX array, the scalability of SAP HANA primarily depends on the number of available engines in the array. Table 1 shows the VMAX models and the estimated maximum number of HANA worker nodes that can be connected according to the number of available engines. Table 1: EMC VMAX 10K, 20K and 40K scalability and EMC VMAX Storage Using IBM GPFS and XFS File System 9

10 When EMC Symmetrix Remote Data Facility (SRDF) is used for SAP HANA storage replication, a reduced number of front-end FA-ports will be available and the number of HANA worker nodes that can be connected to the array must be adjusted accordingly. EMC VMAX3 scalability In a 100K, 200K, or 400K VMAX3 array, the scalability of SAP HANA primarily depends on the number of available engines in the array. Table 2 shows the VMAX3 models and the estimated maximum number of HANA nodes that can be connected according to the number of available number of engines: Table 2: EMC VMAX 100K, 200K and 400K scalability and EMC VMAX Storage Using IBM GPFS and XFS File System 10

11 If you use SRDF for SAP HANA storage replication, a reduced number of front-end FA-ports are available, and the maximum number of HANA worker nodes that can be connected to the array must be adjusted accordingly. Disaster tolerance using EMC VMAX SRDF Leveraging VMAX SRDF and TimeFinder to Protect the Entire SAP Landscape Storage replication offers advantages over application-based disaster recovery solutions where only the database content is replicated to the secondary site. Leveraging SRDF provides a disaster tolerance solution for the entire SAP landscape. EMC VMAX SRDF consistency technology can be leveraged to include all relevant HANA and non-hana volumes, even across multiple arrays at each site, ensuring fast recovery of the entire SAP landscape, not only the database. See Figure 2. Figure 2: EMC VMAX SRDF for disaster recovery EMC TimeFinder can be used for a rapid recovery of the database and to augment remote replication for specific use cases. and EMC VMAX Storage Using IBM GPFS and XFS File System 11

12 System x UEFI Settings for Max Performance System x UEFI Settings In order to obtain optimal performance for the SAP HANA configuration, it is necessary for some of the System x UEFI settings to be modified from their original defaults. We recommend changing the following parameters to the described settings based upon the System x type you are installing, while leaving all others at their default values. Please note that if you update any of the firmware of the System x machine, the UEFI settings may be reset. Therefore, we recommend rechecking the UEFI settings after each firmware update and change the values as necessary to the documented values. Intel Xeon E7 processor-based systems Parameter Value Advanced Settings Utility (ASU) Setting x3690 ex5 Hyper Threading Enable UEFI.HyperThreading Operating Mode Performance Mode x3850 / x3950 ex5 UEFI.OperatingMode QPI Link Speed Max Performance UEFI.QPISpeed Memory Patrol Scrub Enable UEFI.PatrolScrub Table 3: Required UEFI settings for System x ex5 servers based on E7 processors Intel Xeon E5 v2 processor-based systems Parameter Value ASU Tool Setting Choose Operating Mode Active Energy Manager Platform Controlled Type Workload Configuration Memory Power Management Custom Mode OperatingModes.ChooseOperating Mode Capping Disable Max Performance x3500 M4 x3550 M4 x3650 M4 x240, x440 HS23 Power.ActiveEnergyManager Power.PlatformControlledType I/O sensitive Power.WorkloadConfiguration Automatic Memory.MemoryPowerManagement C1 Enhanced Mode Disable Processors.C1EnhancedMode Direct Cache Access (DCA) Enable Processors.DirectCacheAccessDCA Table 4: Required UEFI settings for System x M4 servers based in Intel Xeon E5 v2 processors and EMC VMAX Storage Using IBM GPFS and XFS File System 12

13 Intel Xeon E7 v2 processor-based systems Parameter Value ASU Tool Setting Choose Operating Mode x3850 X6 x3950 X6 x280 X6 x480 X6 Custom Mode OperatingModes.ChooseOperatingMode Active Energy Manager Capping Disable Power.ActiveEnergyManager Platform Controlled Type Max Performance Power.PlatformControlledType Workload Configuration I/O sensitive Power.WorkloadConfiguration Memory Power Management Automatic Memory.MemoryPowerManagement C1 Enhanced Mode Disable Processors.C1EnhancedMode Enable SMK Disable Processors.EnableSMK Direct Cache Access (DCA) Enable Processors.DirectCacheAccessDCA Table 5: Required UEFI settings for System x X6 servers based on Intel Xeon E7 v2 processors x880 X6 Intel Xeon E5 v3 processor-based systems Parameter Value ASU Tool Setting Choose Operating Mode x3500 M5 x3550 M5 x3650 M5 X240 M5 / x440 Custom Mode OperatingModes.ChooseOperatingMode Active Energy Manager Capping Disable Power.ActiveEnergyManager Platform Controlled Type Max Performance Power.PlatformControlledType Workload Configuration I/O sensitive Power.WorkloadConfiguration Memory Power Management Automatic Memory.MemoryPowerManagement C1 Enhanced Mode Disable Processors.C1EnhancedMode Direct Cache Access (DCA) Enable Processors.DirectCacheAccessDCA Table 6: Required UEFI settings for System x M5 servers based on Intel Xeon E5 v3 processors nx360 M5 and EMC VMAX Storage Using IBM GPFS and XFS File System 13

14 Setting C-States As recommended in the SAP Notes SAP HANA DB: Recommended OS settings for SLES 11 / SLES for SAP Applications 11 SP2 and SAP HANA DB: Recommended OS settings for SLES 11 / SLES for SAP Applications 11 SP3 and additionally described in IBM RETAIN Tip H Linux Ignores C-State Settings in Unified Extensible Firmware Interface (UEFI), the control ( C ) states of the Intel processor should to be turned off for the most reliable performance of SAP HANA. By default C-States are enabled in the UEFI due to the fact that we set the processor to Custom Mode. With C-States being turned on you might see performance degradations with SAP HANA. We recommend turning off the processor C-States using the Linux kernel boot parameter: processor.max_cstate=0 The Linux kernel used by SAP HANA includes a built-in driver ( intel_idle ) which will ignore any C- State limits imposed by Basic Input/Output System (BIOS)/Unified Extensible Firmware Interface (UEFI) when it is active. This driver may cause issues by enabling C-States even though they are disabled in the BIOS or UEFI. This can cause minor latency as the CPUs transition out of a C-State and into a running state. This is not the preferred state for the SAP HANA configuration and must be changed. To prevent the intel_idle driver from ignoring BIOS or UEFI settings for C-States, add the following Start parameter to the kernel s boot loader configuration file: intel_idle.max_cstate=0, Append both parameters to the end of the kernel command line of your boot loader (/boot/grub/menu.lst) and reboot the server. You can use the SUSE yast tool to configure the same. and EMC VMAX Storage Using IBM GPFS and XFS File System 14

15 Figure 3: EMC VMAX SRDF for disaster recovery Using the Advanced Settings utility Lenovo has a tool named the Advanced Settings Utility (ASU) for System x servers. This tool can be installed on top of a running operating system to view and change settings in the System x UEFI firmware. This tool can be downloaded from HUhttp://IBM.com/support/entry/portal/docdisplay?lndocid=LNVO-ASUU H We recommend using this tool to properly set the UEFI settings. 1. To install from the rpm file: 1. Copy the rpm to a directory on the hard drive. 2. Run one of the following commands as the root user: To install if the package is not already installed # rpm -i <ASU distribution>.rpm To install or update the package, removing other versions. # rpm -U <ASU distribution>.rpm 2. Display current values of the running system # /opt/ibm/toolscenter/asu/asu64 show 3. Save changed settings to file # /opt/ibm/toolscenter/asu/asu64 show --kcs > /tmp/myasu.txt 4. Modify values with a text editor 5. Store to UEFI # /opt/ibm/toolscenter/asu/asu64 set < /tmp/myasu.txt Using the System x UEFI boot screen If you wish to change the settings before you install the operating system, or need to modify the settings after an update of the System x firmware, then you may also enter the System x UEFI boot screen to manually change the settings at boot time. In order to enter the boot screen, you must press the <F1> function key while booting the server. Then when the setup screen is displayed, you may enter each of the above settings from their respective groupings. and EMC VMAX Storage Using IBM GPFS and XFS File System 15

16 SAP HANA System Check An optional but very important tool is the SAP HANA System Check tool which ensures that SAP HANA has been installed on valid and certified hardware. This tool does not check whether the system can perform adequately, it only ensures that all the hardware used is acceptable to the standards set out by SAP SE for its SAP HANA software. To run this tool, you will need to enter the directory DATA_UNITS/HDB_SERVER/server and execute the following command: HANAtdi:{HANA dump}/data_units/hdb_server_linux_x86_64/server# python HANAHwCheck.py Using this tool can give you an initial overview if those settings, as checked by SAP, are correct and usable for SAP HANA. If this tools shows errors, or you have questions regarding its output, you should open an SAP Customer Message on SAP Service Marketplace to obtain help from SAP. IBM GPFS Installation and Configuration Network Configuration Before you start GPFS cluster configuration you will need to gather the following networking details. and EMC VMAX Storage Using IBM GPFS and XFS File System 16

17 Network Description IP Addresses Host Name Net Mask Gateway SAP HANA Internodes Communication GPFS Internodes Communication X HANAnode0X None X GPFSnode0X None IMM2 Customer Defined Customer Defined Customer Defined Customer Defined Customer Network Customer Defined Customer Defined Customer Defined Customer Defined Table 7: GPFS network configuration requirements in a SAP HANA scale-out implementation X = Node number SAP HANA and GPFS cluster communication private IPs are configured on the 10Gb NIC ports. Example: In a 3 node (2 worker + 1 standby) cluster setup you will have the below network configuration: Node 1: (HANAnode01), (GPFSnode01) Node 2: (HANAnode02), (GPFSnode02) Node 3: (HANAnode03), (GPFSnode03) Note: It is mandatory to use 10Gb NICs to meet the SAP-defined KPIs in the TDI network. Customize /etc/hosts file Add GPFS host names and IP addresses of all cluster nodes to the /etc/hosts file GPFSnode01 GPFSnode GPFSnode02 GPFSnode GPFSnode03 GPFSnode03 : : : GPFSnode16 GPFSnode16 : : : and EMC VMAX Storage Using IBM GPFS and XFS File System 17

18 Add SAP HANA host names and IP addresses of all cluster nodes to the /etc/hosts file HANAnode01 HANAnode HANAnode02 HANAnode HANAnode03 HANAnode03 : : : HANAnode16 HANAnode16 : : : Add the SAP HANA external host names and IP addresses of all cluster nodes to the /etc/hosts file. This is the hostname that will be used for the installation of the SAP HANA database and the connection with tools, such as the SAP HANA Studio. SSH Configuration Configure SSH keys to enable ssh and scp for passwordless GPFS and SAP HANA communication: #ssh-keygen -t rsa -f ~/.ssh/id_rsa -N '' # ssh-copy-id -i ~/.ssh/id_rsa.pub root@gpfsnodenn # ssh-copy-id -i ~/.ssh/id_rsa.pub root@hananodenn Note: NN is the node number within the cluster. In a clustered configuration, exchange SSH keys with all nodes using a script such as the following. #!/bin/bash for host in `cat /etc/hosts grep GPFSnode awk '{print $3}'`; do ping -c 1 -s 1 $host if [ $? -eq 0 ]; then ssh-copy-id -i ~/.ssh/id_rsa.pub root@$host fi done for host in `cat /etc/hosts grep HANAnode awk '{print $3}'`; do ping -c 1 -s 1 $host if [ $? -eq 0 ]; then ssh-copy-id -i ~/.ssh/id_rsa.pub root@$host fi done and EMC VMAX Storage Using IBM GPFS and XFS File System 18

19 Install GPFS Packages Install GPFS as described in the "Concepts, Planning and Installation Guide", available HUhereU Make sure to select the correct version for your GPFS release, Login as user: root. Install GPFS (base package): # cd /var/tmp/install/gpfs-<gpfs-release> # rpm -ivh GPFS.base-<GPFS-RELEASE>-0.x86_64.rpm Verify the GPFS version installed. Update to the latest GPFS Maintenance release Install the following three packages for the latest (X) maintenance release: # rpm -Uvh GPFS.base-<GPFS-RELEASE>.x86_64.update.rpm # rpm -Uvh GPFS.docs-<GPFS-RELEASE>.noarch.rpm # rpm -Uvh GPFS.gpl-<GPFS-RELEASE>.noarch.rpm # rpm -Uvh GPFS.msg.en_US-<GPFS-RELEASE>.noarch.rpm Build the GPFS portability layer Follow the instructions in /usr/lpp/mmfs/src/readme: and EMC VMAX Storage Using IBM GPFS and XFS File System 19

20 # cd /usr/lpp/mmfs/src # make Autoconfig # make World # make InstallImages Prepare the GPFS configuration 1) In a clustered configuration these commands must be executed on all nodes. Add the GPFS executables to the path by creating a file /etc/profile.local with this content: 2) Change the file permission: # chmod 644 /etc/profile.local 3) Activate the new PATH variable: # source /etc/profile.local 4) Create a dump-directory for GPFS: mkdir /tmp/gpfsdump 5) Create a configuration-directory for GPFS: # mkdir /var/mmfs/config GPFS cluster configuration 6) Create a GPFS disk configuration file in /var/mmfs/config/ directory for SAP HANA persistent storage. In the below example it is named as disk.list.data2. 7) Have the disk paths populated in the GPFS disk definitions file, using the syntax show below: durxha238129:~ # cat /var/mmfs/config/disk.list.data2 %nsd: device=/dev/dm-10 Device Mapper file (Storage Volumes) nsd=mddata012 usage=dataandmetadata failuregroup=1001 %nsd: device=/dev/dm-11 nsd=mddata022 usage=dataandmetadata failuregroup=1001 and EMC VMAX Storage Using IBM GPFS and XFS File System 20

21 %nsd: device=/dev/dm-12 nsd=mddata032 usage=dataandmetadata failuregroup=1001 %nsd: device=/dev/dm-13 nsd=mddata042 usage=dataandmetadata failuregroup=1001 Note: Device paths can be verified using the multipath ll command as shown below: EMC configuration guides for VMAX and VNX can be downloaded from the following links: HUhttp:// SAP-HANA-tdi-VMAX-VMAX3-wp.pdfU HUhttps://community.EMC.com/docs/DOC-36203U 8) On the first node, create the node configuration file /var/mmfs/config/nodes.cluster and add a line containing the internal GPFSnodenn host name for each node in the cluster. Append ":quorum" to the first three entries. Example for a 4-node cluster: GPFSnode01:quorum GPFSnode02:quorum GPFSnode03:quorum GPFSnode04 and EMC VMAX Storage Using IBM GPFS and XFS File System 21

22 9) Create a cluster: Single node # mmcrcluster -n /var/mmfs/config/nodes.cluster -p GPFSnode01 -r /usr/bin/ssh -R /usr/bin/scp -C HANAcluster A Cluster mmcrcluster -n /var/mmfs/config/nodes.cluster -p GPFSnode01 -s GPFSnode02 -r /usr/bin/ssh -R /usr/bin/scp -C HANAcluster -A Note: All cluster commands must be run on the first node of the cluster. Example: 10) Assign a GPFS server license to the node: # mmchlicense server --accept -N all 11) Adjust various GPFS settings and run the following command on the first node of the cluster: # mmchconfig datastructuredump=/tmp/gpfsdump,pagepool=4g,maxmbps=2048, maxfilestocache=4000,skipdiowritelogwrites=1,nsdinlinewritemax=1m,prefetchaggressiv enesswrite=2,readreplicapolicy=local,enablelinuxreplicatedaio=yes,restripeondiskfailur e=yes,nsdthreadsperdisk=24 Note: Enter 999 when asked to press Enter due to Unknown attribute specified: skipdiowritelogwrites message Note: Enter 999 when asked to press Enter due to Unknown attribute specified: prefetchaggressivenesswrite message and EMC VMAX Storage Using IBM GPFS and XFS File System 22

23 12) Create GPFS disks: On Node 1 run the following command, with NN being the node number: # mmcrnsd F /var/mmfs/config/disk.list.data.gpfsnodenn Note: If the command fails because the disk is already part of a cluster, append -v no to the command to disable the check. 13) Create the GPFS file system and run the following command on first node of the cluster: # mmcrfs sapmntdata -F /var/mmfs/config/disk.list.data2 -A yes -B 64K -N v no - m 1 M 3 -r 1 -R 3 -Q yes -T /sapmnt --block-group-factor 1 write-affinity-depth 1 Note: The file system mount point is /sapmnt and the associated device name is /dev/sapmntdata. and EMC VMAX Storage Using IBM GPFS and XFS File System 23

24 For the directories /sapmnt/data, /sapmnt/log and /sapmnt/shared, GPFS filesets will be used instead of separate mounted file systems. 1) Verify the GPFS file system attributes/parameter values: #mmlsfs all 2) Mount the sapmntdata file system # mmmount /sapmnt a 3) Create filesets HANAdata, HANAlog and HANAshared. Link the filesets to the file system. # mmcrfile`set sapmntdata HANAdata -t "Data Volume for HANA database" and EMC VMAX Storage Using IBM GPFS and XFS File System 24

25 # mmcrfileset sapmntdata HANAlog -t "Log Volume for HANA database" # mmcrfileset sapmntdata HANAshared -t "Shared Directory for HANA database" # mmlinkfileset sapmntdata HANAdata -J /sapmnt/data # mmlinkfileset sapmntdata HANAlog -J /sapmnt/log # mmlinkfileset sapmntdata HANAshared -J /sapmnt/shared 4) Start GPFS on all nodes and check the status: # mmstartup a and # mmgetstate a XFS Configuration for SAP HANA Scale-out using the Storage Connector API Network Configuration Gather the following network details before you configure an XFS-based HANA scale-out cluster: and EMC VMAX Storage Using IBM GPFS and XFS File System 25

26 Network Description IP Addresses Host Name Net Mask Gateway SAP HANA Internodes Communication IMM X HANAnode0X None Customer defined Customer defined Customer defined Customer defined Customer Network Customer defined Customer defined Customer defined Customer defined Table 8: XFS Network configuration requirements in a SAP HANA scale out deployment. Configure SAP HANA cluster communication private IPs on the 10Gb NIC ports. Example: In a 3 node (2 workers + 1 standby) cluster setup, you will have the following network configuration. Node 1: (HANAnode01) Node 2: (HANAnode02) Node 3: (HANAnode03), Note: It is mandatory to use 10Gb NICs to meet the SAP defined KPIs in a TDI network. BXFS file system configuration for 2+1 HANA scale out XFS can be used as an underlying file system for both HANA Data and Log volumes. The NFS file system is used to create a shared file system, which is mandatory in an SAP HANA scale-out scenario. SAP HANA storage connector API will act as an agent for managing the shared volumes from the EMC storage array to System x servers, which will manage the automatic mounting/unmounting of devices on HANA nodes. SCSI-3 Persistent Reservations are written to the devices to prevent any data corruption due to multiple hosts accessing a device at the same time. For storage configuration best practices on EMC storage in a SAP HANA environment, please refer to the following technical whitepapers. Hhttps://community.EMC.com/docs/DOC-34450U HUhttp:// HANA-tdi-VMAX-VMAX3-wp.pdfU HUhttps://community.EMC.com/docs/DOC-36203U 1) Once the storage volumes are presented to the HANA Nodes we can start the XFS file system creation on the block devices. Run the following command to discover the shared disks to be used for HANA Data and Log volumes for all the nodes: and EMC VMAX Storage Using IBM GPFS and XFS File System 26

27 # multipath ll Note: In this test, we have used a 3-node (2 worker + 1 standby) scale-out configuration 2) Create the XFS file system using the command: #mkfs.xfs <block device> Example: #mkfs.xfs /dev/mapper/ #mkfs.xfs /dev/mapper/ ) Create directories for mounting the above file systems: #mkdir /hana/data #mkdir /hana/log #mkdir /hana/shared 4) Verify whether you can mount the file system on /hana/data and /hana/log mount points with the below options. and EMC VMAX Storage Using IBM GPFS and XFS File System 27

28 Node 1 (HANA role: worker) log volume: #mount /dev/mapper/ /hana/log o rw,inode64, nobarrier Node 2 (HANA role: worker) data volume: #mount /dev/mapper/ /hana/data o rw, inode64 Similarly you need to verify the XFS file system creation on shared block devices from Node 2 (HANA role: worker). Node 3 (HANA role: standby): It will not have any XFS mount points as that is taken care of by the SAP HANA storage connector API. 5) After verifying the XFS file system, unmount the file systems as this will be managed automatically by the SAP HANA storage connector API. 6) Make sure all 3 nodes have a NFS file system shared between them and mounted on /hana/shared, as shown below: and EMC VMAX Storage Using IBM GPFS and XFS File System 28

29 SAP HANA global.ini file The storage connector API is controlled in the storage section of the SAP HANA global.ini file. This section contains entries for the block devices with optional mount options. The WWIDs of the partition entries can be determined using the multipath ll command on the HANA hosts. This global.ini was used in the test: 7) Copy the global.ini file to the /hana/shared directory, which is hosted on an NFS share and available to all 3 nodes. 8) Make sure the passwordless ssh works between all 3 nodes. Refer to the SSH configuration section, above, under GPFS, for steps to configure this. HANA DB installation 1) Based on the SID to be selected for HANA installation, create a directory with the SID (System Instance ID) name under /hana/data and /hana/log. In this particular test we have SID= ANA and set the execute permission to directory: #mkdir /hana/data/ana #mkdir /hana/log/ana # chmod 777 /hana/data/ana # chmod 777 /hana/log/ana and EMC VMAX Storage Using IBM GPFS and XFS File System 29

30 2) Change the directory to where the HANA bits are located. 3) Run the following command from the master/worker node to start multihost HANA DB server installation: #./hdblcm action=install storage_cfg=/hana/shared addhosts=<worker node hostname>:role=worker,<standby node hostname>:role=standby Note: In this test we are using SAP HANA SPS09. 4) Details required for the HANA DB server installation: Which component needs to be installed? Select option 1 for Server. Enter Local hostname. Usually it s populated with the hostname. If not, enter the master/worker node hostname. Root password: <customer defined> and EMC VMAX Storage Using IBM GPFS and XFS File System 30

31 and EMC VMAX Storage Using IBM GPFS and XFS File System 31

32 5) Enter the SID. (In this test we used ANA ) 6) Enter the Instance Number. By default it will be listed based on the number of instances loaded in the system. 7) Depending on the Single DB/Multitenant DB configuration you can opt for options. In this test we chose 1 : Single Container. 8) Next, Depending on the DB usage you can opt. In this test we chose 4 : Custom. 9) The Certificate Host Name can be accepted as in the options. 10) DB admin user is created with name <sid>adm. The password needs to be set. Please refer to the HANA server installation guide for password rules. After the summary, the HANA DB installation starts. and EMC VMAX Storage Using IBM GPFS and XFS File System 32

33 11) Follow the HU SAP HANA server installation guide UH for more details. Troubleshooting tip: If there is an installation error due to directory access, make sure there is <sid>adm:sapsys permission assigned for the ANA directory. #chown <sid>adm:sapsys /hana/data/ana #chown <sid>adm:sapsys /hana/log/ana 12) Once HANA is installed and the SAP HANA storage connector API is active on the nodes, this is how the mount points for SAP HANA data, log volumes would look: Node 1 (Master/worker): and EMC VMAX Storage Using IBM GPFS and XFS File System 33

34 Node 2 (Worker): and EMC VMAX Storage Using IBM GPFS and XFS File System 34

35 Node 3 (Standby): The /hana/data and /hana/log mount points will not be mounted on a standby node prior to a failover. Should a failover occur, the SAP HANA storage connector API will mount the respective failed-node HANA partitions in the standby node. 13) Run the following commands to check HANA services status: #su - <sid>adm SAPcontrol nr 00 function GetSystemInstanceList. You should see the output as follows; color designates the status of the HANA DB in each node: and EMC VMAX Storage Using IBM GPFS and XFS File System 35

36 HA tests (Scale-out Node Failover Test) The SAP HANA scale-out node failover test was performed on an SAP HANA storage connector API-based configuration, as discussed earlier. Worker node failover 1) Trigger HANA services stop on one of the worker nodes in the scale-out configuration to trigger a node failure. Run, # su - <sid>adm # SAPcontrol nr <instance_number> -function Stop HDB 2) Before HDB stop was issued, below was the status of the scale-out configuration. 3) Post failover, you can see, below, that the standby node has taken over the role of worker. 4) The /hana/data and /hana/log mount points have failed-over from the failed worker node to the standby node. and EMC VMAX Storage Using IBM GPFS and XFS File System 36

37 Master/Worker node failover hardware failure simulation 1) In the test, we used the command #echo b > /proc/sysrq-trigger, which is a SIGKILL to all running processes without a graceful file system unmount. 2) After running the above command, check the status of HANA services. 3) Once the HANA services failed over to the standby node, the standby node would have inherited the role of master or worker node. and EMC VMAX Storage Using IBM GPFS and XFS File System 37

38 Appendix A Here we discuss the System x servers and EMC VMAX storage array model details used in the tests above. HANA compute node details Lenovo System x3850 X6 servers with Xeon CPU E GHz and 512GB RAM - QLogic ISP2532-based 8Gb Fibre Channel to PCI Express HBA (rev 02) - Emulex OneConnect 10Gb NIC (be3) (rev 02) - SUSE Linux 11 SP3 for SAP application with SAN Boot (64GB LUN for root and 32GB LUN for swap file system) - IBM GPFS DM-MPIO Lenovo Flex System x880 X6 servers with Xeon E GHz and 256GB RAM - Flex System FC GB SAN scalable switch installed - (2) Onboard EN port 1GB Ethernet adapters - SUSE Linux 11 SP3 for SAP application with SAN Boot (64GB LUN for root and 32GB LUN for swap file system) - IBM GPFS DM-MPIO - Dual port 16GB QLogic HBAs, model QM12672-IBM-BK SAP HANA database version used for the tests Version Node scale-out configuration (2 worker + 1 stand-by) SAN details EMC VMAX-40K ( ). 2 engines (using 6 FA processors) connected via Brocade DS GB port speed departmental switch and EMC VMAX Storage Using IBM GPFS and XFS File System 38

39 Appendix B Lenovo compute building blocks for SAP HANA TDI Table 9 shows which System x servers are supported in an SAP HANA TDI environment: Machine Type CPU Type Memory FC HBAs Ethernet cards System x3850 X6 (Type 6241) System x3950 X6 (Type 6241) Flex System x240 (Type 7863) Flex System x240 (Type 8737) NeXtScale nx360 M5 (Type 5465) System x3500 M4 Type 7383 System x3550 M4 Type 7914 System x3650 M4 Type 7915 BladeCenter HS23 Type 7875 Flex System x240 M5 Compute Node (9532) Intel Ivy Bridge EX E7-8880v2, 2S and 4S Intel Ivy Bridge EX E7-8880v2, 4S and 8S Intel Xeon E5-26xx v2, 2S Intel Xeon E5-26xx v2, 2S Intel Xeon E5-26xx v3, 2S Intel Xeon E5-26xx v2, 2S Intel Xeon E5-26xx v2, 2S Intel Xeon E5-26xx v2, 2S Intel Xeon E5-26xx v2, 2S Intel Xeon E5-26xx v3, 2S Maximum 6TBF2 HUClick HereU HUClick HereU Maximum 12TBF3 HUClick HereU HUClick HereU Maximum 768GB HUClick HereU HUClick HereU Maximum 768GB HUClick HereU HUClick HereU Maximum 512GB HUClick HereU HUClick HereU Maximum 384GB HUClick HereU HUClick HereU Maximum 768GB HUClick HereU HUClick HereU Maximum 768GB HUClick HereU HUClick HereU Maximum 512GB HUClick HereU HUClick HereU Maximum 1.5TB HUClick HereU HUClick HereU System x3500 M5 Intel Xeon E5- Maximum 1.5TB HUClick HereU HUClick HereU 2 6TB for single-node SoH implementations only, 8S 2TB is the maximum BW on HANA scale-out supported configuration in TDI. For more information please check HUSCNU 3 12TB support only for non-production environment and EMC VMAX Storage Using IBM GPFS and XFS File System 39

40 Machine Type CPU Type Memory FC HBAs Ethernet cards (5464) 26xx v3, 2S System x3550 M5 (5463) System x3650 M5 (5462) Flex System x880 X6 Intel Xeon E5-26xx v3, 2S Intel Xeon E5-26xx v3, 2S Intel Xeon E5-26xx v2, 2S,4S and 8S Maximum 1.5TB HUClick HereU HUClick HereU Maximum 1.5TB HUClick HereU HUClick HereU Maximum 6TB HUClick HereU HUClick HereU Table 9: Lenovo servers and components certified for SAP HANA TDI General capacity considerations for VMAX storage in TDI In general, the required capacity for the SAP HANA persistence on disk depends on the inmemory database size and the RAM size of the HANA servers. Every SAP HANA customer must perform memory and CPU sizing as the first step to sizing a SAP HANA deployment. For new SAP HANA implementations, size the memory and CPU for a SAP HANA system using the HANA version of the SAP Quick Sizer tool, available on the SAP Service Marketplace website, or consult SAP for assistance. For systems that are migrating to SAP HANA, SAP provides tools and reports for proper HANA memory sizing. After you determine the memory requirements, you can estimate the disk capacity requirements by using the sizing rules in the SAP white paper, HUSAP HANA Storage RequirementsU H. File systems must include: 1x RAM for the data file system ½x RAM for the log file system with 512 GB or less, or at least 512 GB for the log file system with 512 GB RAM or higher SAP refers to RAM size as the size of the database, in contrast to the physical memory size of the servers. Example: the HANA database can consume 1.3 TB of RAM on a single host, but the host has 2 TB of physical RAM capacity. SAP recommends sizing based on the actual database size, in this example 1.3 TB. Note: SAP sizing requirements do not consider future growth of the database. and EMC VMAX Storage Using IBM GPFS and XFS File System 40

41 However, in certain situations, you may need to expand the size of a data or log file system. Lenovo and EMC recommends sizing the database persistence (the data and log file systems) at a minimum on the physical RAM size of the HANA hosts. To calculate the required usable storage capacity for the HANA persistence of a scale-up (singlehost) or scale-out (multihost) appliance, the following details are required: (A) RAM size of a HANA worker host (B) Number of HANA worker hosts Example: Use the following formulas to calculate the required capacity for a 6+1 HANA scale-out appliance where each server has 2 TB RAM: Total capacity for data = (A) * (B) = 2 TB * 6 = 12 TB Total capacity for Log = 512 GB * (B) = 512 GB * 6 = 3 TB Total usable capacity for the HANA persistence = 12 TB + 3 TB = 15 TB For more information on disk considerations, etc., please refer to HUVMAX best practices guide for HANA TDI U QLogic Fibre Channel adapters for System x servers The QLogic 16 Gb FC Single-port and Dual-port Host Bus Adapters (HBAs) for System x offer 16 Gbps line-rate performance at extremely low CPU utilization with full hardware offloads. The Flex System FC port 16Gb FC Adapter enables high-speed access for Flex System compute nodes to connect to a Fibre Channel storage area network (SAN). This adapter is based on the proven QLogic 16Gb ASIC design and works with the 8 Gb and 16 Gb Flex System Fibre Channel switches and pass-thru modules. Management software: The QLogic QConvergeConsole (QCC) management software delivers a unified webbased single-pane-of-glass management console across the QLogic family of storage and networking adapters. A graphical user interface (GUI) or command-line interface (CLI) are available. A VMware vcenter plug-in is also available. Common IT tasks, such as VLAN configuration and teaming, can be easily accomplished either through the QConvergeConsole or through native OS tools, minimizing IT training and deployment costs. Role-based authentication allows for separate logins and access for SAN and LAN administrators. This eliminates the need to change your organizational structure as you converge your network. and EMC VMAX Storage Using IBM GPFS and XFS File System 41

42 for Figure 4: QLogic Single & Dual port FC HBA s Figure 4 shows the QLogic adapters used in the above SAP HANA TDI tests. These support nearly twice the throughput and 2.5 times the I/O operations per second (IOPS) per port of 8Gb adapters. The QLogic 16 Gb HBAs are ideal for high-bandwidth I/O-intensive applications, such as SAP HANA. Through rigorous testing carried out via the ServerProven program, you can maintain a high degree of confidence that your storage subsystem is compatible and functions reliably when using these 16 Gb HBA adapters. Refer HUhereU H more information on the QLogic Fibre channel adapters supported with Lenovo System x servers. HWCCT The SAP HANA HW Configuration Check Tool allows you to check the interoperability of SAP HANA with your existing enterprise storage in production environments. The SAP HANA HW Configuration Check Tool is a framework that provides tests and reports for new single-host and scale-out systems to determine whether the hardware you intend to use meets the minimum performance criteria required to run SAP HANA in production use. HUClick hereu H for more details on HWCCT and EMC VMAX Storage Using IBM GPFS and XFS File System 42

43 Resources Lenovo documentation: HUSystem x solution for SAP HANAU HFlex System X6 Solution for SAP HANAU HUSystem x solutions for SAP environmentsu HUIn-memory computing on Lenovo ex5 and X6U HULenovo System x solution for SAP HANA and VMware vsphere on X6U EMC documentation: HUEMC VNX best practices for HANA TDI U HUEMC-Lenovo whitepaper on HANA TDIU HUEMC SRDF whitepaperu HUEMC technical reference guides for SAP HANA TDIU SAP documentation: HUSAP HANA Server Installation GuideU HUBest Practice Document Technical Deployment Options for SAP Systems with SAP HANAU HUWhite Paper - SAP HANA System Landscape GuideU HUSAP HANA Appliance HUSAP HANA OneU HUSAP HANA Enterprise CloudU HUSAP HANA tailored data center integration (integration of enterprise storage systems with SAP HANA)U HUSAP HANA High Availability PaperU HUSAP SCN page for TDI certified servers and storageu QLogic documentation: HUQLogic products for Lenovo System xu and EMC VMAX Storage Using IBM GPFS and XFS File System 43

44 Notices Lenovo may not offer the products, services, or features discussed in this document in all countries. Consult our local Lenovo representative for information on the products and services currently available in your area. Any reference to a Lenovo product, program, or service is not intended to state or imply that only that Lenovo product, program, or service may be used. Any functionally equivalent product, program, or service that does not infringe any Lenovo intellectual property right may be used instead. However, it is the user's responsibility to evaluate and verify the operation of any other product, program, or service. Lenovo may have patents or pending patent applications covering subject matter described in this document. The furnishing of this document does not give you any license to these patents. You can send license inquiries, in writing, to: Lenovo (United States), Inc Think Place - Building One Morrisville, NC U.S.A. Attention: Lenovo Director of Licensing LENOVO PROVIDES THIS PUBLICATION AS IS WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some jurisdictions do not allow disclaimer of express or implied warranties in certain transactions, therefore, this statement may not apply to you. This information could include technical inaccuracies or typographical errors. Changes are periodically made to the information herein; these changes will be incorporated in new editions of the publication. Lenovo may make improvements and/or changes in the product(s) and/or the program(s) described in this publication at any time without notice. The products described in this document are not intended for use in implantation or other life support applications where malfunction may result in injury or death to persons. The information contained in this document does not affect or change Lenovo product specifications or warranties. Nothing in this document shall operate as an express or implied license or indemnity under the intellectual property rights of Lenovo or third parties. All information contained in this document was obtained in specific environments and is presented as an illustration. The result obtained in other operating environments may vary. Lenovo may use or distribute any of the information you supply in any way it believes appropriate without incurring any obligation to you. Any references in this publication to non-lenovo Web sites are provided for convenience only and do not in any manner serve as an endorsement of those Web sites. The materials at those Web sites are not part of the materials for this Lenovo product, and use of those Web sites is at your own risk. Any performance data contained herein was determined in a controlled environment. Therefore, the result obtained in other operating environments may vary significantly. Some measurements may have been made on development-level systems and there is no guarantee that these measurements will be the same on generally available systems. Furthermore, some measurements may have been estimated through and EMC VMAX Storage Using IBM GPFS and XFS File System 44

45 extrapolation. Actual results may vary. Users of this document should verify the applicable data for their specific environment. Trademarks 2015 Lenovo, All rights reserved. The Lenovo logo, BladeCenter, Flex System, NeXtScale, ServerProven, System x, ex5, X6, and X-Architecture, For Those Who Do are trademarks or registered trademarks of Lenovo in the United States, other countries, or both. Visit periodically for the latest information on safe and effective computing. Intel, Intel Xeon are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries. IBM and IBM GPFS are trademarks or registered trademarks of IBM Corporation or its subsidiaries in the United States and other countries. EMC, EMC VMAX, and SRDF are trademarks or registered trademarks of EMC Corporation or its subsidiaries in the United States and other countries. SAP logo, SAP, and SAP HANA are trademarks or registered trademarks of SAP or its subsidiaries in the United States and other countries. Linux is a trademark of Linus Torvalds in the United States, other countries, or both. Other company, product, or service names may be trademarks or service marks of others. and EMC VMAX Storage Using IBM GPFS and XFS File System 45