The Host Server. Linux Configuration Guide. July The Data Infrastructure Software Company

Transcription

1 The Host Server Linux Configuration Guide July 2016 The Data Infrastructure Software Company

2 Table of contents Overview 3 Recent changes made to this document 3 Linux compatibility lists 4 RedHat Enterprise Linux 4 SUSE Linux Enterprise Server 5 Ubuntu 6 Other Linux distributions 8 Self-qualifying 'other' Linux distributions 8 The DataCore Server's settings 9 The Linux Host's settings 11 Operating system settings 11 Multipath configuration settings 12 Linux applications 16 SAP HANA 16 Known issues 19 Appendix A 21 Preferred Server & Preferred Path settings 21 Appendix B 23 Configuring Disk Pools 23 Appendix C 24 Reclaiming storage 24 Previous changes 26 Page 2

3 Overview This document provides Linux-specific settings when serving Virtual Disks from a DataCore Server. Fundamental Linux administration skills are assumed; including how to connect Linux Hosts to storage array target ports (i.e. DataCore Server Front End ports) via either Fibre Channel or iscsi, along with the process of discovering, mounting and formatting disk devices in general. Recent changes made to this document New information added since last update (June 2016) Updated This document has been reviewed for SANsymphony 10.0 PSP 5. No updates were required. Added April 2016 Linux compatibility lists Red Hat Enterprise Linux Versions Previous changes made to this document Please see page 26 Page 3

4 Linux compatibility lists RedHat Enterprise Linux SANsymphony 9.0 PSP 4 Update 4 (1) 10.0 (all versions) Version With ALUA Without ALUA With ALUA Without ALUA 5.4 and earlier Not Supported Not Supported Not Supported Not Supported Not Supported Not Supported Not Supported Not Supported Supported Supported Not Qualified Not Qualified 6.4 Supported Supported Not Qualified Not Qualified Not Qualified Not Qualified Supported Not Qualified Not Supported Not Supported Not Qualified Not Qualified 7.0 Not Supported Not Supported Supported Not Qualified Not Supported Not Supported Not Qualified Not Qualified Compatibility notes Please see page 7 for specific details on the differences between 'Not Supported', 'Not Qualified' and 'Supported'. Fibre Channel and iscsi Connections DataCore Software supports RedHat Linux Hosts using either Fibre Channel or iscsi Connections to SANsymphony Front-End (FE) Ports for any Virtual Disk type. SCSI UNMAP RedHat Linux Hosts are supported using SCSI UNMAP commands - see 'Appendix C - Reclaiming Storage' on page SANsymphony-V 8.x and all versions of SANsymphony 9.x before PSP4 Update 4 are now End of Life. Please see: End of Life Notifications Page 4

5 Linux compatibility lists SUSE Linux Enterprise Server SANsymphony 9.0 PSP 4 Update 4 (1) 10.0 (all versions) Version With ALUA Without ALUA With ALUA Without ALUA 10.x and earlier Not Supported Not Supported Not Supported Not Supported 11.0 (no SP) Not Supported Supported Not Supported Not Supported 11.0 SP 1 Not Supported Not Supported Not Supported Not Supported 11.0 SP 2 Supported Supported Not Qualified Not Qualified 11.0 SP 3 Not Qualified Not Qualified Supported Not Qualified 11.0 SP 4 Not Supported Not Supported Not Qualified Not Qualified 12.x Not Supported Not Supported Not Qualified Not Qualified Compatibility notes Please see page 7 for specific details on the differences between 'Not Supported', 'Not Qualified' and 'Supported'. Fibre Channel and iscsi Connections DataCore Software supports RedHat Linux Hosts using either Fibre Channel or iscsi Connections to SANsymphony Front-End (FE) Ports for any Virtual Disk type. SCSI UNMAP SUSE Linux Hosts are not supported using SCSI UNMAP commands. 1 SANsymphony-V 8.x and all versions of SANsymphony 9.x before PSP4 Update 4 are now End of Life. Please see: End of Life Notifications Page 5

6 Linux compatibility lists Ubuntu SANsymphony 9.0 PSP 4 Update 4 (1) 10.0 (all versions) Version With ALUA Without ALUA With ALUA Without ALUA 13.x and earlier Not Supported Not Supported Not Supported Not Supported LTS Not Supported Not Qualified Not Supported Supported Not Supported Not Qualified Not Supported Not Supported 15.x Not Supported Not Qualified Not Supported Not Qualified 16.x Not Supported Not Supported Not Qualified Not Qualified Note: This table applies only to the Linux distribution provided by Canonical Ltd and does not include any Debian Linux distribution (or other distributions that are themselves either based on Debian or Ubuntu s own codebases). Compatibility notes Please see page 7 for specific details on the differences between 'Not Supported', 'Not Qualified' and 'Supported'. Fibre Channel and iscsi connections DataCore Software supports RedHat Linux Hosts using either Fibre Channel or iscsi Connections to SANsymphony Front-End (FE) Ports for any Virtual Disk type. Note: If the Host is running SAP HANA then only Fibre Channel connections are supported. Please see page 16 for more information. SCSI UNMAP Ubuntu Linux Hosts have not been tested using SCSI UNMAP commands also see 'Appendix C - Reclaiming Storage' on page Please see the compatibility notes on page 7 for specific details on the differences between 'Not Supported', 'Not Qualified' and 'Supported'. Page 6

7 Linux compatibility lists Compatibility notes Supported These Linux/SANsymphony combinations have been tested using all the host-specific settings listed in this document against all Virtual Disk types. Mirrored and Dual Virtual Disks have been tested for 'high availability' in all possible failure scenarios. Not Qualified These Linux/SANsymphony combinations have not been tested against any Mirrored or Dual Virtual Disks types. DataCore therefore cannot guarantee 'high availability' in any failure scenario (even if all host-specific settings listed in this document are followed) however, selfqualification may be possible. For more information on this please see: Support for any Linux versions that are considered End of Life by the vendor and are listed as 'Not Qualified' can still be self-qualified but only if there is an agreed support contract with your Linux vendor. In this case though, DataCore Technical Support will not do root-cause analysis for SANsymphony in the case of any future issues but will offer 'best effort' support to get Hosts accessing any SANsymphony Virtual Disks. Note: Non-mirrored Virtual Disks are always considered as supported even for 'Not Qualified' combinations of Linux/SANsymphony. Not Supported These Linux/SANsymphony combinations have usually failed one or more of our 'high availability' tests when using Mirrored or Dual Virtual Disks types; but also may simply be where an Operating System's own requirements (or limitations) due to its age make it impractical to test. Entries marked as 'Not Supported' can never be self-qualified. Mirrored or Dual Virtual Disks types are configured at the end-user's own risk. Note: Non-mirrored Virtual Disks are always considered as supported even for 'Not Supported' combinations of Linux/SANsymphony. End of Life Linux distributions Support for any Linux distribution that is considered End of Life by the vendor or has no active development/long Term Support can still be self-qualified but only if there is an agreed support contract with the supplier/vendor or the Linux Distribution. In this case, DataCore Technical Support will help the customer to get the Host Operating system accessing Virtual Disks, but will not then do any root-cause analysis. Page 7

8 Other Linux distributions All other Linux distributions that are not listed in this document are considered as unqualified for mirrored or dual Virtual Disks when using SANsymphony and, just like all the unqualified versions of Linux that are listed in this document, the behavior with SANsymphony cannot be guaranteed. These other distributions can also self-qualified by following the same process mentioned in the compatibility notes on page 7. Self-qualifying 'other' Linux distributions DataCore recommends using the same configuration settings as those listed for Ubuntu (i.e. the less complex, non-alua settings) for self-qualifying. Then assuming this passes all the tests, either of the ALUA-based configuration settings can then be tested. Note: Non-mirrored Virtual Disks are always considered as supported even for 'other' Linux distributions used with SANsymphony. Page 8

9 The DataCore Server's settings These are the Host-specific settings that need to be configured directly on the DataCore Server. Operating system type See the Registering Hosts section from the SANsymphony Help: RedHat Enterprise Linux When registering the Host choose the 'Linux (all other distributions)' menu option. SUSE Linux Enterprise Server When registering the Host choose the 'Linux SUSE Enterprise Server 11 ' menu option. Ubuntu Linux When registering the Host choose the 'Linux (all other distributions)' menu option. Port roles Ports used for serving Virtual Disks to Hosts should only have the Front End (FE) role enabled. Mixing other Port Role types may cause unexpected results as Ports that only have the FE role enabled will be turned off when the DataCore Server software is stopped (even if the physical server remains running). This helps to guarantee that any Hosts do not still try to access FE Ports, for any reason, once the DataCore Software is stopped but where the DataCore Server remains running. Any Port with the Mirror and/or Back End role enabled do not shut off when the DataCore Server software is stopped but still remain active. Multipathing support The Multipathing Support option should be enabled so that Mirrored Virtual Disks or Dual Virtual Disks can be served to Hosts from all available DataCore FE ports. Also see the Multipathing Support section from the SANsymphony Help: Webhelp/Hosts.htm Non-mirrored Virtual Disks and Multipathing Non-mirrored Virtual Disks can still be served to multiple Hosts and/or multiple Host Ports from one or more DataCore Server FE Ports if required; in this case the Host can use its own multipathing software to manage the multiple Host paths to the Single Virtual Disk as if it was a Mirrored or Dual Virtual Disk. Note: Hosts that have non-mirrored Virtual Disks served to them do not need Multipathing Support enabled unless they have other Mirrored or Dual Virtual Disks served as well. Page 9

10 The DataCore Server's settings Asymmetrical Logical Unit Access (ALUA) support The ALUA support option should be enabled if required and if Multipathing Support has been also been enabled (see above). Please refer to the compatibility list for your distribution of Linux on page 4 to see which combinations of Linux and SANsymphony support ALUA. More information on 'Preferred Servers' and 'Preferred Paths' used by the ALUA function can be found on in Appendix A on page 21. Serving Virtual Disks to the Hosts for the first time DataCore recommends that before serving Virtual Disks for the first time to a Host, that all DataCore Front-End ports on all DataCore Servers are correctly discovered by the Host first. Then, from within the SANsymphony Console, the Virtual Disk is marked Online, up to date and that the storage sources have a host access status of Read/Write. Virtual Disks LUNs and serving to more than one Host or Port DataCore Virtual Disks always have their own unique Network Address Authority (NAA) identifier that a Host can use to manage the same Virtual Disk being served to multiple Ports on the same Host Server and the same Virtual Disk being served to multiple Hosts. See the SCSI Standard Inquiry Data section from the online Help for more information on this: While DataCore cannot guarantee that a disk device's NAA is used by a Host's operating system to identify a disk device served to it over different paths generally we have found that it is. And while there is sometimes a convention that all paths by the same disk device should always using the same LUN 'number' to guarantees consistency for device identification, this may not be technically true. Always refer to the Host Operating System vendor s own documentation for advice on this. DataCore's Software does, however always try to create mappings between the Host's ports and the DataCore Server's Front-end (FE) ports for a Virtual Disk using the same LUN number (1) where it can. The software will first find the next available (lowest) LUN 'number' for the Host- DataCore FE mapping combination being applied and will then try to apply that same LUN number for all other mappings that are being attempted when the Virtual Disk is being served. If any Host-DataCore FE port combination being requested at that moment is already using that same LUN number (e.g. if a Host has other Virtual Disks served to it from previous) then the software will find the next available LUN number and apply that to those specific Host- DataCore FE mappings only. 1 The software will also try to match a LUN 'number' for all DataCore Server Mirror Port mappings of a Virtual Disk too, although the Host does not 'see' these mirror mappings and so this does not technically need to be the same as the Front End port mappings (or indeed as other Mirror Path mappings for the same Virtual Disk). Having Mirror mappings using different LUNs has no functional impact on the Host or DataCore Server at all. Page 10

11 The Linux Host's settings Operating system settings Disk Timeouts Set the SCSI Disk timeout value to 80 seconds for any DataCore Virtual Disk devices. echo 80 > /sys/block/[disk_device]/device/timeout For example if the two DataCore Virtual Disks are using /dev/sda and /dev/sdb respectively: echo 80 > /sys/block/sda/device/timeout echo 80 > /sys/block/sdb/device/timeout This will change the SCSI Disk timeout value to 80 seconds for those particular disk devices. The SCSI Disk timeout can then be verified by running the cat command on the disk device directly cat /sys/block/sda/device/timeout Note: The SCSI Disk timeout may revert to the system default after a reboot. Please contact your Linux vendor or consult their documentation for details on creating a rules file in /etc/udev/ to help resolve this behavior. Fibre Channel device timeouts Do not specify any fibre channel-specific device (i.e. lpfc_devloss_tmo for Emulex or, qlport_down_retry for the QLogic) but use the values in the multipath configuration file (see the next page). Page 11

12 The Linux Host's settings Multipath configuration settings Polling Interval In the defaults section of the multipath.conf file, the polling_interval must be set to 60 defaults { } polling_interval 60 This is a DataCore-required value which helps prevent excessive Host attempts to check for a Virtual Disk Storage Source s Host Access value, after a failure, is set as Offline. Smaller interval settings will interfere with overall Host performance. Do not add this parameter to the 'device' section (discussed on the next page) by mistake; else the setting will not work as expected. Blacklist exceptions Usually all storage vendor devices are specified under a separate device sub-section within the blacklist_exceptions section. blacklist_exceptions { } vendor product "DataCore" "Virtual Disk" Page 12

13 The Linux Host's settings The 'device' section ALUA-enabled Hosts Please refer to the compatibility list for your distribution of Linux on page 4 to see which combinations of Linux and SANsymphony support ALUA device { vendor "DataCore" product "Virtual Disk" path_checker tur prio alua failback 10 no_path_retry fail dev_loss_tmo infinity fast_io_fail_tmo 5 rr_min_io_rq 100 # Alternative option See notes below # rr_min_io 100 path_grouping_policy group_by_prio # Alternative policy - See notes below # path_grouping_policy failover } # optional - See notes below # user_friendly_names yes Without ALUA enabled Please refer to the compatibility list for your distribution of Linux on page 4 to see which combinations of Linux and SANsymphony support ALUA device { } vendor "DataCore" product "Virtual Disk" path_checker tur failback 10 dev_loss_tmo infinity fast_io_fail_tmo 5 no_path_retry fail # optional - See notes below # user_friendly_names yes Page 13

14 Device section notes Note: All entries listed are required by SANsymphony and are case sensitive. The Linux Host's settings dev_loss_tmo infinity Also requires the fast_io_fail_tmo 5 setting (see next). The dev_loss_tmo setting controls the length of time (normally indicated in seconds) before a PATH to a Virtual Disk, that has since become unavailable to the Linux Host, is removed from the Linux operating system. For example; when a DataCore Server is stopped or a PATH to a DataCore Server s Virtual Disk is set to Host Access: Not Allowed. Once a PATH to a Virtual Disk has been removed by the Linux operating system it can then only usually be re-established by manual intervention (e.g. user-initiated rescan on the Linux Host). The infinity value prevents the PATH from being removed. If the fast_io_fail_tmo 5 setting is not present in the multipath.conf file, the infinity setting is ignored and the dev_loss_tmo value defaults to 600 (10 minutes). Note: Some older kernels of RedHat Enterprise Linux and SUSE Linux Enterprise Server 11 SP2 and earlier do not support the infinity value and it will be ignored (and may also post an error in syslog). In that case, a default value - usually 600 seconds - will be applied instead. Use the cat command to verify that any DataCore Virtual Disks detected by the Linux Host are using the infinity value correctly. A simple example: sleshost3:~ # cat /sys/class/fc_remote_ports/rport-*\:*-*/dev_loss_tmo Note: The value of indicates a dev using the infinity setting. fast_io_fail_tmo This is required by the dev_loss_tmo infinity setting (see previous). Do not use any other value other than 5, otherwise the dev_loss_tmo setting will use a larger, default value (usually 600 seconds). failback This adds an extra wait period (10 seconds) that helps to prevent unnecessary failback attempts to a Virtual Disk whose Storage Source s Host Access value is still set as Not Allowed. Page 14

15 The Linux Host's settings no_path_retry Required for any Linux Hosts configured for ALUA and want to use the Preferred Server setting set to ALL. See Appendix A on page 21 about information regarding the ALL setting. path_checker This is a DataCore-required value. No other value should be used. path_grouping_policy group_by_prio or path_grouping_policy group_by_failover One of two possible values is required group_by_prio or failover - to avoid any other, non- DataCore, setting in the multipath.conf (i.e. from other storage arrays attached to the Linux Host) from taking precedence. Note: The failover value is unqualified for RHEL 6.5 and greater. In either case, make sure the Preferred Server setting on the DataCore Server is either set to 'Auto Select' or an explicit Server Name. See Appendix A on page 21 about information regarding the 'Auto Select' setting. rr_min_io_rq This is a DataCore-required value for Linux Hosts that are running kernels newer than Older versions must use the option rr_min_io 100 instead (see next). rr_min_io This is a DataCore-required value for Linux Hosts that are running kernels older than Newer versions must use the option rr_min_io_rq 100 instead (see previous). user_friendly_names This is an optional setting and simply specifies that the operating system should use the /etc/multipath/bindings file to assign a persistent and unique alias to the multipath device, in the form of mpathn. If this value is set to 'no' (or omitted completely) the operating system will use a WWID as an alias for the multipath device instead. Page 15

16 Linux applications SAP HANA Note: For more detailed information about HANA operation and configuration always consult the HANA configuration documents available from SAP. Sizing guidelines Please refer to the documents: SANsymphony with SAP HANA - Sizing Guidelines SAP HANA versions Versions SPS09 and SPS10 have both been certified. DataCore Software version requirements SANsymphony 10.0 PSP 3 or greater using Fibre Channel connections for Front-end, Mirror and Back-end connections. ISCSI connections are not certified and should not be used. Host operating system requirements SUSE Linux Enterprise Server 11.0 Service Pack 3 with the additional updates (or later) installed: kpartx ptf x86_64.rpm multipath-tools ptf x86_64.rpm Host settings requirements The Linux Host must be configured for ALUA. Hosts without ALUA are not certified. Mount point and filesystem requirements /hana/log /hana/data /hana/shared Use XFS Use XFS Use OCFS2 Additional file system settings /hana/log: /hana/data: /hana/shared: None async_write_submit_active = on async_write_submit_blocks = all None Example: Setting the /hana/data mount point: Page 16

17 Linux Applications SAP HANA su <SID>adm cd /usr/sap/<sid>/hdb<instance> source hdbenv.sh hdbparam --paramset "fileio[data].async_write_submit_active=on" hdbparam --paramset "fileio[data].async_write_submit_blocks=all" exit The global.ini setting Use the PRES Type 5 by adding the following line to the storage section partition_*_* prtype = 5 Page 17

18 Linux Applications SAP HANA An example of a HANA 3-node configuration In this example, two of the HANA nodes are active and the third node is standby. Active Server host1 /hana/log/db1/mnt00001 /hana/data/db1/mnt00001 /hana/log/db1/mnt00002 /hana/data/db1/mnt00002 /hana/shared <- Mounts host1 Log_vDisk <- Mounts host1 Data_vDisk <- Mounts host2 Log_vDisk <- Mounts host2 Data_vDisk <- Mounts shared OCFS2 filesystem vdisk Active Server host2 /hana/log/db1/mnt00001 /hana/data/db1/mnt00001 /hana/log/db1/mnt00002 /hana/data/db1/mnt00002 /hana/shared <- Mounts host1 Log_vDisk <- Mounts host1 Data_vDisk <- Mounts host2 Log_vDisk <- Mounts host2 Data_vDisk <- Mounts shared OCFS2 filesystem vdisk Standby Server host3 (in failover host role) /hana/log/db1/mnt00001 /hana/data/db1/mnt00001 /hana/log/db1/mnt00002 /hana/data/db1/mnt00002 /hana/shared <- Mounts host1 Log_vDisk <- Mounts host1 Data_vDisk <- Mounts host2 Log_vDisk <- Mounts host2 Data_vDisk <- Mounts shared OCFS2 filesystem vdisk Notes: Filesystem entries hi-lighted in red are those that are mounted on that particular host when the database is started. Those that are not hi-lighted in red are still available to the host but are not mounted. In the case of a failover event the HANA application will determine which of the other hosts will then mount and use the failed host s file systems. Because any host can, potentially, require access to any of the other hosts file systems all SANsymphony Virtual Disks used for the SAP HANA data or log filesystems must be served to all SAP HANA hosts active and standby. Page 18

19 Known issues The following is intended to make DataCore Software customers aware of any issues that may affect performance, access or generally give unexpected results under certain conditions when Linux Hosts are used with SANsymphony. Some of the issues here have been found during DataCore s own testing but many others are issues reported by DataCore Software customers, where a specific problem had been identified and then subsequently resolved. DataCore cannot be held responsible for incorrect information regarding Linux products. No assumption should be made that DataCore has direct communication with any of the Linux vendors regarding the issues listed here and we always recommend that users contact their own Linux vendor directly to see if there are any updates or fixes since they were reported to us. For Known issues for DataCore s own Software products, please refer to the relevant DataCore Software Component s release notes. Formatting a Virtual Disk may take longer than expected Since SANsymphony-V 9.0 PSP4, Linux Hosts have been able to take advantage of SANsymphony s SCSI-UNMAP support - See the Appendix C - Reclaiming Storage' on page 24. However for RHEL and SLES Hosts, this will result in the mkfs command sending additional discard commands during the format process, resulting in longer format times. Ubuntu has not been tested with SCSI UNMAP so is considered unqualified. Use the -K option while formatting to disable the discard command during formatting. Also see: and Refer to your own man page to be sure your installation supports this option. Ext3 filesystems use excessive storage allocations The Ext3 filesystem will use significant amounts of SAUs during the Writing superblocks and filesystem accounting information phase of the filesystem creation. Care must be taken so as not to completely use all the SAUs from the Disk Pool; and if Ext3 is required, then use a small (i.e. 4MB) SAU size. Other filesystem types do not seem to exhibit this behavior and use only a few SAUs during filesystem creation. Also refer to Appendix B - Configuration Disk Pools on page 23. Page 19

20 Known issues Manual rescans are needed to update previously failed paths for Ubuntu Hosts DataCore have not been able to get Ubuntu to automatically re-detect paths to mirrored Virtual Disks that have failed or have been removed (e.g. after stopping a DataCore Server) and are then subsequently made available again. Manual intervention is required. Use the command multipath -ll Establish which paths are currently failed and then use one of the commands rescan-scsi-bus From the scsitools package usually installed by apt-get install scsitools or run the command echo 1 > /sys/block/[disk_device_name]/device/rescan For example echo 1 > /sys/block/sdb/device/rescan will send a signal to the disk device and force the operating system to update the path s status properly. Page 20

21 Appendix A Preferred Server & Preferred Path settings See the Preferred Servers and Preferred Paths sections from the SANsymphony Help: Without ALUA enabled If Hosts are registered without ALUA support, the Preferred Server and Preferred Path settings will serve no function. All DataCore Servers and their respective Front End (FE) paths are considered equal. It is up to the Host s own Operating System or Failover Software to determine which DataCore Server is its preferred server. With ALUA enabled Setting the Preferred Server to Auto (or an explicit DataCore Server), determines the DataCore Server that is designated Active Optimized for Host IO. The other DataCore Server is designated Active Non-Optimized. If for any reason the Storage Source on the preferred DataCore Server becomes unavailable, and the Host Access for the Virtual Disk is set to Offline or Disabled, then the other DataCore Server will be designated the Active Optimized side. The Host will be notified by both DataCore Servers that there has been an ALUA state change, forcing the Host to re-check the ALUA state of both DataCore Servers and act accordingly. If the Storage Source on the preferred DataCore Server becomes unavailable but the Host Access for the Virtual Disk remains Read/Write, for example if only the Storage behind the DataCore Server is unavailable but the FE and MR paths are all connected or if the Host physically becomes disconnected from the preferred DataCore Server (e.g. Fibre Channel or iscsi Cable failure) then the ALUA state will not change for the remaining, Active Nonoptimized side. However, in this case, the DataCore Server will not prevent access to the Host nor will it change the way READ or WRITE IO is handled compared to the Active Optimized side, but the Host will still register this DataCore Server s Paths as Active Non-Optimized which may (or may not) affect how the Host behaves generally. Page 21

22 Appendix A - Preferred Server & Preferred Path settings In the case where the Preferred Server is set to All, then both DataCore Servers are designated Active Optimized for Host IO. All IO requests from a Host will use all Paths to all DataCore Servers equally, regardless of the distance that the IO has to travel to the DataCore Server. For this reason, the All setting is not normally recommended. If a Host has to send a WRITE IO to a remote DataCore Server (where the IO Path is significantly distant compared to the other local DataCore Server), then the WAIT times accrued by having to send the IO not only across the SAN to the remote DataCore Server, but for the remote DataCore Server to mirror back to the local DataCore Server and then for the mirror write to be acknowledged from the local DataCore Server to the remote DataCore Server and finally for the acknowledgement to be sent to the Host back across the SAN, can be significant. The benefits of being able to use all Paths to all DataCore Servers for all Virtual Disks are not always clear cut. Testing is advised. For Preferred Path settings it is stated in the SANsymphony Help: A preferred front-end path setting can also be set manually for a particular virtual disk. In this case, the manual setting for a virtual disk overrides the preferred path created by the preferred server setting for the host. So for example, if the Preferred Server is designated as DataCore Server A and the Preferred Paths are designated as DataCore Server B, then DataCore Server B will be the Active Optimized Side not DataCore Server A. In a two-node Server group there is usually nothing to be gained by making the Preferred Path setting different to the Preferred Server setting and it may also cause confusion when trying to diagnose path problems, or when redesigning your DataCore SAN with regard to Host IO Paths. Where there are three or more Servers in a Server Group, and where one or more of these DataCore Servers shares Mirror Paths between different DataCore Servers then setting the Preferred Path makes more sense. So for example, DataCore Server A has two mirrored Virtual Disks, one with DataCore Server B, and one with DataCore Server C and DataCore Server B also has a mirrored Virtual Disk with DataCore Server C then using just the Preferred Server setting to designate the Active Optimized side for the Host s Virtual Disks becomes more complicated. In this case the Preferred Path setting can be used to override the Preferred Server setting for a much more granular level of control. Page 22

23 Appendix B Configuring Disk Pools See Creating Disk Pools and Adding Physical Disks from the SANsymphony Help: The smaller the SAU size, the larger the number of indexes are required, by the Disk Pool driver, to keep track of the equivalent amount of allocated storage compared to a Disk Pool with a larger SAU size; e.g. there are potentially four times as many indexes required in a Disk Pool using a 32MB SAU size compared to one using 128MB the default SAU size. As SAUs are allocated for the very first time, the Disk Pool needs to update these indexes and this may cause a slight delay for IO completion and might be noticeable on the Host. However this will depend on a number of factors such as the speed of the physical disks, the number of Hosts accessing the Disk Pool and their IO READ/WRITE patterns, the number of Virtual Disks in the Disk Pool and their corresponding Storage Profiles. Therefore, DataCore usually recommend using the default SAU size (128MB) as it is a good compromise between physical storage allocation and IO overhead during the initial SAU allocation index update. Should a smaller SAU size be preferred, the configuration should be tested to make sure that a potential increased number of initial SAU allocations does not impact the overall Host performance. Page 23

24 Appendix C Reclaiming storage Using SCSI UNMAP commands Please refer to the compatibility list for your distribution of Linux on page 4 to see which combinations of Linux and SANsymphony support SCSI UNMAP commands from the Host. As of SANsymphony-V 9.0 PSP4 there is support for SCSI UNMAP commands which, when used in conjunction the Linux fstrim command or the mount o discard option (for some Linux file systems), allows Hosts to trigger the Automatic Reclamation function on served Virtual Disks. See the following online resources: The Linux man-pages project From RedHat s Storage Administration Guide: Section: 2.5. Discard unused blocks Always refer to your Linux Vendor to determine which file systems are supported for either the fstrim command and/or the mount o discard option for the version of RHEL or SLES that you currently have installed on the Host. It is also important to note that using the mount o discard option may affect overall performance again refer to your Linux Vendor for their recommendations. Automatic reclamation Since SANsymphony-V 9.0 PSP4 the DataCore Server will, in the background, continuously scan each Physical Disk in a Pool for any SAUs that contain all-zero data which are then reclaimed back into the Disk Pool. The Automatic Reclamation process runs at a low priority, so as to not potentially interfere with overall Disk Pool performance, and so will generally take longer to scan a Physical Disk in a Pool compared to a Manual Reclamation request (see below). However no user intervention is required. Manual reclamation For Linux Hosts that either do not support the fstrim command, the mount o discard option, or are using Virtual Disks served from DataCore Servers running SANsymphony-V 9.0 PSP3 Page 24

25 Appendix C Reclaiming storage update2 or earlier (including all versions of SANsymphony-V 8.x); a suggestion would be to create a sparse file of an appropriate size (If there is enough free space available in the file system) and then zero-fill it using the dd command. This example creates an empty file (called my_file ) of 2GB in size: fallocate --length my_file Then write zeroes within the file using dd: dd if=/dev/zero of=my_file bs=1024 count= Then either wait for an Automatic Reclamation to take place or run a Manual Reclamation. See the Performing Reclamation section from the SANsymphony Help: Note that it is also possible to script manual reclamation using the Start- DcsVirtualDiskReclamation PowerShell Command. How much storage will be reclaimed? It is impossible to predict exactly how many Storage Allocation Units (SAUs) will be reclaimed. For reclamation of an SAU to take place, it must contain only all-zero block data over the entire SAU else it will remain allocated and this is entirely dependent on how and where the Host has written its data on the DataCore LUN. For example, if the Host has written the data in such a way that every allocated SAU contains a small amount of non-zero block data, even if the total amount of data is significantly less than the total amount of SAU allocation, then no SAUs can be reclaimed. It may be possible, in some cases, to use the Host Operating System s own defragmentation tools to force the non-zero block data to be moved to a more contiguous pattern, so leaving the end of the DataCore LUN full of SAUs that no longer have non-zero data on them that can then be reclaimed. However care should be taken that the act of defragmenting the data does not itself cause more SAU allocation as the block data is moved around during the re-organization. DataCore Software can offer no guarantees. Page 25

26 Previous changes 2016 April Added Linux compatibility list Red Hat Enterprise Linux Versions SUSE Linux Enterprise Server 12 (no Service Pack) and 12 Service Pack 1 January Updated Linux compatibility list Red Hat Enterprise Linux Version Previously this version was listed as 'not qualified' with ALUA enabled Hosts and SANsymphony-V 9.x. This has now been changed to 'Not Supported with Mirrored or Dual Virtual Disks'. Red Hat Enterprise Linux Version 6.6 Previously this version was listed as 'Not Qualified' with ALUA enabled Hosts and SANsymphony-V 10.x. This version is now 'Qualified' with ALUA enabled Hosts and SANsymphony-V 10.x November Added Linux Applications SAP HANA Including an example of a 3-node SAP HANA configuration. Updated SANsymphony-V 8.x and all versions of SANsymphony 9.x before PSP4 Update 4 are now End of Life. Please see: End of Life Notifications September Added Linux Applications SAP HANA A new section with settings specific to SAP HANA. Updated Linux compatibility list SUSE Linux Enterprise Server 11.0 Service Pack 3 using ALUA are certified with SAP HANA versions SPS09 and SPS10. Multipath Configuration Settings The multipath.conf settings for all qualified Linux distributions are now identical so there is only one section for all Linux distributions listed here. The setting dev_loss_tmo infinity requires the additional setting fast_io_fail_tmo to be present in the multipath.conf file this was omitted previously from the SUSE Linux Enterprise Server requirements; and that the fast_io_fail_tmo value is set to 5 this was previously set to 30 for Red Hat Enterprise Linux requirements. Page 26

27 Previous changes June Added Linux compatibility list Ubuntu LTS has now been qualified with SANsymphony-V 10.x. Known Issues Ubuntu requires manual intervention to redetect failed paths to a DataCore Server. April Added Known Issues All RHEL or SUSE Linux specific known issues with SANsymphony-V will now be documented here. March Updated Linux compatibility lists SUSE Linux Enterprise Server 11.0 with Service Pack 3 is now qualified with SANsymphony-V 10.x using ALUA-only settings. February Updated Linux compatibility lists Red Hat Enterprise Linux 7.0 is now qualified with SANsymphony-V 10.x using ALUA-only settings. Multipath Configuration Settings Red Hat Enterprise Linux Added new entry in the multipath.conf file no_path_retry fail This is required for any Oracle RAC/GFS configured Hosts Hosts using ALUA with the Preferred Server setting configured for ALL November Updated Linux compatibility lists Updated the table for all Red Hat Enterprise Linux and SUSE Linux Enterprise Server Versions Single Virtual Disks are now always considered supported. July Updated Linux compatibility lists Updated the table for all Red Hat Enterprise Linux Versions It was previously listing versions only. Host Settings Disk Timeouts Added an example for use of the cat command to determine the current SCSI Disk Timeout and a note that some versions of Linux may revert to the default settings after a reboot and how to resolve this. Page 27

28 Previous changes June Updated Linux compatibility lists Updated the table for SANsymphony-V 10.x May Added Linux compatibility lists Red Hat Enterprise Linux 6.5 with ALUA is now qualified for use with SANsymphony-V 9.x Appendix A - Reclaiming Storage from Linux Hosts Added information for ATA Trim commands and Automatic Reclamation from Disk Pools. Updated Multipath Configuration Settings Red Hat Enterprise Linux only Added a new Multipath.conf entry requirement - fast_io_fail_tmo - which is required to support the dev_loss_tmo value of infinity else the value may default back to 10 minutes instead. NB: Check with your Vendor to make sure your specific Linux Kernel version supports these options. Red Hat Enterprise Linux and SUSE Enterprise Linux DataCore recommends that the Multipath.conf entry path_grouping_policy be group_by_prio instead of failover. NB: The failover option is considered unqualified for RHEL version 6.5. Added an optional Multipath.conf entry user_friendly_names which will create simpler, easier to read, disk device names for users to work with. NB: Check with your Vendor to make sure your specific Linux Kernel version supports this option. April This document combines all of DataCore s Linux-related information from older Technical Bulletins into a single document including: Technical Bulletin 2a: " Other Linux Hosts" Technical Bulletin 2b: "Redhat Enterprise Linux 6.x Hosts" Technical Bulletin 2c: "SUSE Linux Enterprise Server 11.x Hosts" Technical Bulletin 8: "Formatting Host s File Systems on Virtual Disks created from Disk Pools" Technical Bulletin 11: "Disk Timeout Settings on Hosts" Technical Bulletin 16: "Reclaiming Space in Disk Pools" Added Linux compatibility lists Added new tables to show which versions are explicitly qualified, unqualified and not supported with either SANsymphony-V 8.x or 9.x, and if the configuration is with or without ALUA enabled Hosts. Note that the minimum requirement for SANsymphony-V 8.x is now 8.1 PSP1 Update 4 to guarantee expected behavior for qualified versions of Linux. Appendix A This section gives more detail on the Preferred Server and Preferred Path settings with regard to how it may affect a Host. Appendix B Page 28

29 Previous changes This section incorporates information regarding "Reclaiming Space in Disk Pools" (from Technical Bulletin 16) that is specific to Linux Hosts. Updated Host Settings - SUSE Linux Enterprise Server with/without ALUA enabled dev_loss_tmo - For SLES 11 SP2 or greater, please make sure that multipath-tools or greater is installed for the infinity setting to work properly. Improved explanations to most of the required Host Settings and DataCore Server Settings generally and earlier Technical Bulletin 2a: " Other Linux Hosts" July 2013 Removed all references to SANmelody as this is now End of Life as of December Completely updated what is considered qualified, not qualified and not supported in this document. January 2013 Updated the section for the multipath.conf sections specifically for the polling_interval line. July 2012 Added SANsymphony-V 9.x. Updated the Notes section for the multipath.conf sections specifically for the getuid_callout line. May 2012 Updated DataCore Server and Host minimum requirements. Removed all references to End of Life SANsymphony and SANmelody versions that are no longer supported as of December Updated the configuration settings and entries for /etc/multipath.conf (with additional values and explanations) for all DataCore products. Users should re-check their existing configurations and make any appropriate changes. Removed all (out of date) step-by-step instructions on how to manage and configure/format disk devices on the Host. March 2011 Added SANsymphony-V December 2009 Initial Publication Technical Bulletin 2b: "Redhat Enterprise Linux 6.x Hosts" July 2013 Removed all references to RHEL version 5.3 as this were never tested with ALUA-enabled hosts and so may cause confusion. Please use Technical Bulletin 2a for this earlier version. This Bulletin is now only for RHEL versions 6.x. Updated Host Requirements. Added additional information in the Notes section of the multipath.conf section. June 2013 Improved blacklist_exceptions example for multipath.conf. April 2013 Removed all references to SANmelody as this is now End of Life as of December Updated settings for multipath.conf including additional settings and explanations. March 2013 RHEL versions 6.0, 6.1, 6.2 and 6.3 explicitly stated. Page 29

30 Previous changes January 2013 Updated the section for the multipath.conf sections specifically for the polling_interval line. This was previously stated to go in the devices. This was incorrect, and should be added to defaults section instead. Technical Bulletin 2c: "SUSE Linux Enterprise Server 11.x Hosts" July 2013 This Bulletin is now only for SLES versions 11.x. Updated Host Requirements. Completely updated what is considered qualified, not qualified and not supported in this document. Added additional information in the Notes section of the multipath.conf section. June 2013 Improved blacklist_exceptions example for multipath.conf. April 2013 Removed all references to SANmelody as this is now End of Life as of December Updated settings for multipath.conf including additional settings and explanations. February 2013 Added notes for SLES 11 SP2; including an additional multipath.conf setting. Updated that SLES 11 SP1 is no longer supported as this was found not work correctly in all failure conditions. Users on SLES 11 SP1 should update to SP2. January 2013 Updated the section for the multipath.conf sections specifically for the polling_interval line. This was previously stated to go in the devices. This was incorrect, and should be added to defaults section instead. Page 30

31 COPYRIGHT Copyright 2016 by DataCore Software Corporation. All rights reserved. DataCore, the DataCore logo and SANsymphony are trademarks of DataCore Software Corporation. Other DataCore product or service names or logos referenced herein are trademarks of DataCore Software Corporation. All other products, services and company names mentioned herein may be trademarks of their respective owners. ALTHOUGH THE MATERIAL PRESENTED IN THIS DOCUMENT IS BELIEVED TO BE ACCURATE, IT IS PROVIDED AS IS AND USERS MUST TAKE ALL RESPONSIBILITY FOR THE USE OR APPLICATION OF THE PRODUCTS DESCRIBED AND THE INFORMATION CONTAINED IN THIS DOCUMENT. NEITHER DATACORE NOR ITS SUPPLIERS MAKE ANY EXPRESS OR IMPLIED REPRESENTATION, WARRANTY OR ENDORSEMENT REGARDING, AND SHALL HAVE NO LIABILITY FOR, THE USE OR APPLICATION OF ANY DATACORE OR THIRD PARTY PRODUCTS OR THE OTHER INFORMATION REFERRED TO IN THIS DOCUMENT. ALL SUCH WARRANTIES (INCLUDING ANY IMPLIED WARRANTIES OF MERCHANTABILITY, NON- INFRINGEMENT, FITNESS FOR A PARTICULAR PURPOSE AND AGAINST HIDDEN DEFECTS) AND LIABILITY ARE HEREBY DISCLAIMED TO THE FULLEST EXTENT PERMITTED BY LAW. No part of this document may be copied, reproduced, translated or reduced to any electronic medium or machine-readable form without the prior written consent of DataCore Software Corporation