IBM Platform LSF. Best practices. Setting up firewall rules for IBM Platform LSF. Yuxing Ren LSF Development Systems & Technology Group

IBM Platform LSF Best practices Setting up firewall rules for IBM Platform LSF Yuxing Ren LSF Development Systems & Technology Group Issued: September 2014

Setting up firewall rules for IBM Platform LSF... 1 Executive Summary... 3 Introduction... 4 Setting up LSF firewall rules in Linux... 5 Firewall rules for outgoing packets... 5 Firewall rules for incoming packets... 5 In a single-cluster environment... 5 Open specific ports on every host... 5 Special requirements for LSF features on every host... 6 A simple firewall rule example... 6 In a Platform MultiCluster environment... 7 Setting up LSF firewall rules in Windows... 7 Opening ports in LSF for Windows... 8 Set up LSF firewall rules by using Windows firewall with advanced security... 8 Example Windows firewall configuration... 9 Best practices... 13 Conclusion... 14 Further reading... 15 Contributors... 15 Notices... 16 Trademarks... 17 Contacting IBM... 17 Setting up firewall rules for IBM Platform LSF Page 2 of 17

Executive Summary IBM Platform LSF is a workload management platform and job scheduler for distributed HPC environments. It is based on networked OS and needs communication between different hosts through a network. Since firewall services can block any unsolicited network traffic, setting firewall rules for LSF is required when a firewall service is running. This document presents guidelines for how to set up firewall rules for LSF in both Linux and Windows. Setting up firewall rules for IBM Platform LSF Page 3 of 17

Introduction This document serves as a best practice guide for how to set up firewall rules for LSF. This document describes the following: Setting up LSF firewall rules in Linux Firewall rules for outgoing packets Firewall rules for incoming packets In a single-cluster environment Open specific ports on every host Special requirements for LSF features on every host An simple firewall rule example In a Platform MultiCluster environment Setting up LSF firewall rules in Windows Opening ports in LSF for Windows Set up LSF firewall rules by using Windows firewall with advanced security Example Windows firewall configuration Setting up firewall rules for IBM Platform LSF Page 4 of 17

Setting up LSF firewall rules in Linux Firewall rules for outgoing packets Do not block outgoing packets sent from the machine itself. You should allow all LSF binaries to send out packets and open all ports that control outgoing packets. Firewall rules for incoming packets Incoming packets destined for the host computer can be blocked by the firewall. Most LSF binaries do not need the firewall to open certain ports that control incoming packets, but bsub I, lsrun and lsgrun may use specific ports to receive incoming packets from other hosts. Note: Some LSF commands, such as bsub I, lsrun, lsgrun, or bpeek require incoming packets from LSF on every host. See Special requirements for LSF features on every host below for details. The following steps are based on iptables in Linux. In a single-cluster environment If you use a firewall like iptables to block packets between hosts in the same cluster, follow the principles below. Open specific ports on every host Open the ports shown in Table 1. Table 1. LSF ports Port Configuration file Protocol LSF_LIM_PORT lsf.conf TCP+UDP LSB_MBD_PORT lsf.conf TCP LSF_RES_PORT lsf.conf TCP LSB_SBD_PORT lsf.conf TCP LSB_QUERY_PORT lsf.conf TCP Setting up firewall rules for IBM Platform LSF Page 5 of 17

If you set LSF_ENABLE_EGO=Y in lsf.conf to enable Platform EGO, you must open the following EGO-related ports shown in Table 2. Table 2. EGO-related ports Port Configuration file Protocol EGO_KD_PORT ego.conf TCP EGO_PEM_PORT ego.conf TCP ESC_PORT egosc_conf.xml TCP WSG_PORT wsg.conf TCP 53 (the default port of named) named.conf TCP+UDP Special requirements for LSF features on every host For the busb -I, lsrun and lsgrun commands, if you set the environment variable LSF_NIOS_PORT_RANGE, you must open a range of ports with TCP protocol whose range is equal to LSF_NIOS_PORT_RANGE. Because these features use random ports, you should disable the firewall or allow all incoming packets that come from LSF hosts. The bpeek command also uses random ports for communication. Either disable the firewall or allow all incoming packets that come from LSF hosts. A simple firewall rule example This example is based on iptables in Linux and enables network communication for the bjobs command. The OUTPUT chain controls the outgoing packets and the INPUT chain controls the incoming packets that are locally delivered. 1. At a minimum, you should open all ports in the OUTPUT chain first. 2. For the INPUT chain, open specific ports on non-master hosts listed in Table 3, and open specific ports on master hosts listed in Table 4. When you finish configuring the firewall rules in the INPUT chain that are described in Table 3 and Table 4, use iptables L INPUT to check whether you have set the rules successfully. Table 3. Ports to open on non-master hosts Condition Port type Port Configurat Protocol ion file If you configure source LSB_QUERY_PORT lsf.conf TCP LSB_QUERY_PORT If you do not source LSB_MBD_PORT lsf.conf TCP configure LSB_QUERY_PORT N/A source LSF_LIM_PORT lsf.conf TCP N/A destination LSF_LIM_PORT lsf.conf TCP N/A destination LSF_LIM_PORT lsf.conf UDP N/A destination LSF_LIM_PORT lsf.conf UDP Setting up firewall rules for IBM Platform LSF Page 6 of 17

Table 4. Ports to open on master hosts Condition Port type Port Configurat Protocol ion file If you configure destination LSB_QUERY_PORT lsf.conf TCP LSB_QUERY_PORT If you do not destination LSB_MBD_PORT lsf.conf TCP configure LSB_QUERY_PORT N/A source LSF_LIM_PORT lsf.conf TCP N/A destination LSF_LIM_PORT lsf.conf TCP N/A source LSF_LIM_PORT lsf.conf UDP N/A destination LSF_LIM_PORT lsf.conf UDP In a Platform MultiCluster environment If you only use the firewall to block packets between different clusters, you need to open only the two ports in Table 5 on the master host. If you also use the firewall between hosts in the same cluster, follow the In a single-cluster environment steps above. Table 5. Ports between clusters Port Configuration file Protocol LSF_LIM_PORT lsf.conf TCP+UDP LSB_MBD_PORT lsf.conf TCP Setting up LSF firewall rules in Windows This section describes two ways of setting firewall rules in Windows: By opening ports A more strict binary-based method using the built-in Windows firewall The binary-based firewall method is safer, because it assures that ports can only be used by specific LSF binaries. The LSF 9.1.3 Windows installer automatically sets binary-based firewall rules for LSF. Setting up firewall rules for IBM Platform LSF Page 7 of 17

Opening ports in LSF for Windows These rules are similar to setting up LSF firewall rules in Linux. Do not block outgoing packets from all LSF binaries and allow some specific binaries to accept incoming packets. The ports that need to be opened are listed in Table 6. Table 6. LSF ports Binary name Port Configuration file Protocol lim LSF_LIM_PORT lsf.conf TCP+UDP mbatchd LSB_MBD_PORT lsf.conf TCP res LSF_RES_PORT lsf.conf TCP sbatchd LSB_SBD_PORT lsf.conf TCP If you set LSF_ENABLE_EGO=Y in lsf.conf to enable EGO, open the EGO-related ports listed in Table 7. Table 7. EGO-related ports Binary name Port Configuration file Protocol vemkd EGO_KD_PORT ego.conf TCP pem EGO_PEM_PORT ego.conf TCP egosc ESC_PORT egosc_conf.xml TCP wsgserver WSG_PORT wsg.conf TCP named 53 (default port of named) named.conf TCP+UDP If you do not set the LSF_NIOS_PORT_RANGE environment variable, allow bsub and lsrun to use any port to get incoming packets. If you do set LSF_NIOS_PORT_RANGE, allow bsub and lsrun to use the specified range of ports with the TCP protocol. The bpeek and pam commands should be allowed to use any port to get incoming packets. And if you use bpeek command, res also should be allowed to use any port to get incoming packets. Set up LSF firewall rules by using Windows firewall with advanced security The following example applies to the Windows firewall on Windows server 2008 and later, which allows you to block both inbound and outbound network traffic. You must configure the following properties for each Windows firewall rule for LSF: 1. The firewall rule name. 2. A full path of the specific program. 3. Profile properties: Set properties to all active profiles. Windows advanced firewall has three types of profiles: domain, private, and public. Setting the profile properties to all active profiles enables LSF to work no matter which profile the firewall currently uses. Setting up firewall rules for IBM Platform LSF Page 8 of 17

4. Network direction properties. Specify whether the rule is an inbound rule or an outbound rule. Inbound rules filter traffic passing from the network to the local computer based on the filtering conditions specified in the rule. If an LSF binary needs to get incoming information from the network, it needs an inbound firewall rule. Outbound rules filter traffic passing from the local computer to the network based on the filtering conditions specified in the rule. An outbound firewall rule is needed when an LSF program wants to send requests out to the network. 5. Protocol properties and port number properties. Only the inbound rules for some LSF daemon binaries need these two properties. Example Windows firewall configuration An example configuration is shown in Table 8. The example is based on the following assumptions: 1. This example uses LSF ports defined in lsf.conf (LSF_LIM_PORT, LSF_RES_PORT, LSB_MBD_PORT and LSB_SBD_PORT). The example assumes the following default port numbers: LSF_LIM_PORT=6879 LSB_MBD_PORT=6881 LSB_MBD_PORT=6882 You cannot limit the port number of RES because commands like bpeek use random ports to exchange information with RES. 2. The example also uses EGO-related ports defined in ego.conf (EGO_KD_PORT and EGO_PEM_PORT), in egosc_conf.xml (ESC_PORT), and in wsg.conf (WSG_PORT). The example assumes the following default port numbers: EGO_KD_PORT=6777 EGO_PEM_PORT=6778 ESC_PORT=6779 WSG_PORT=9090 3. The example assumes that the DNS server named binary uses the default port number 53. 4. Only lim and the named binary need two kinds of inbound firewall rules and the only difference is the protocol property. 5. If you set LSF_NIOS_PORT_RANGE for NIOS communication, you must set up a new firewall rule for NIOS whose protocol property is TCP and open the port range at the firewall accordingly. By default, there is no limit for the port property of the firewall rule for NIOS and NIOS-related binaries. Setting up firewall rules for IBM Platform LSF Page 9 of 17

Table Key: Any: the binary can receive network packets with matching any value in protocol field and port field. Y: You must set up a new inbound or outbound firewall rule. N: You do not need to set up a new inbound or outbound firewall rule. Table 8. Example of LSF firewall rules based on Windows firewall with advanced security Binary name Inbound firewall rule Outbound firewall rule egosc Y Y TCP: 6779 esub.password N Y N/A lim Y Y TCP: 6879 UDP:6879 mbatchd Y Y TCP: 6881 named Y Y TCP: 53 UDP:53 nios Y Y Any pem Y Y TCP: 6778 res Y Y Any sbatchd Y Y TCP: 6882 TSJobHelper Y Y Any vemkd Y Y TCP: 6777 wsgserver Y Y TCP: 9090 bacct N Y N/A badmin N Y N/A bapp N Y N/A bbot N Y N/A bchkpnt N Y N/A bclusters N Y N/A bconf N Y N/A bgadd N Y N/A bgbroker N Y N/A bgdel N Y N/A bgmod N Y N/A bgpinfo N Y N/A bhist N Y N/A bhosts N Y N/A bhpart N Y N/A bjdepinfo N Y N/A bjgroup N Y N/A bjobs N Y N/A bkill N Y N/A blaunch N Y N/A blimits N Y N/A Protocol and port number for inbound rules Setting up firewall rules for IBM Platform LSF Page 10 of 17

Binary name Inbound firewall rule Outbound firewall rule bmg N Y N/A bmgroup N Y N/A bmig N Y N/A bmod N Y N/A bparams N Y N/A bpeek N Y Any bpost N Y N/A bqueues N Y N/A bread N Y N/A brequeue N Y N/A bresize N Y N/A bresources N Y N/A brestart N Y N/A bresume N Y N/A brsvadd N Y N/A brsvdel N Y N/A brsvmod N Y N/A brsvs N Y N/A brun N Y N/A bsla N Y N/A bslots N Y N/A bstatus N Y N/A bstop N Y N/A bsub N Y Any bswitch N Y N/A btop N Y N/A bugroup N Y N/A busers N Y N/A egosh N Y N/A lsadmin N Y N/A lsclusters N Y N/A lseligible N Y N/A lsgrun N Y N/A lshosts N Y N/A lsid N Y N/A lsinfo N Y N/A lsload N Y N/A lsloadadj N Y N/A lslogin N Y N/A lsmail N Y N/A lsmon N Y N/A lspasswd N Y N/A lsplace N Y N/A Protocol and port number for inbound rules Setting up firewall rules for IBM Platform LSF Page 11 of 17

Binary name Inbound firewall rule Outbound firewall rule lsrcp N Y N/A lsrtasks N Y N/A lsrun N Y Any lssmtp N Y N/A pam Y Y Any ppmsetvar N Y N/A TaskStarter N Y N/A tspeek N Y N/A tssub N Y N/A Protocol and port number for inbound rules Setting up firewall rules for IBM Platform LSF Page 12 of 17

Best practices This document has described best practices for the following: Provides guidelines for setting up LSF firewall rules in Linux. Provides guidelines for setting up LSF firewall rules in Windows. Setting up firewall rules for IBM Platform LSF Page 13 of 17

Conclusion This document provides guidelines for setting up LSF firewall rules in both Linux and Windows and provides an integrated example of setting up LSF firewall rules based on Windows firewall with advanced security. Setting up firewall rules for IBM Platform LSF Page 14 of 17

Further reading Administering Platform LSF Version 9 Release 1.3: o o PDF format: http://publibfp.dhe.ibm.com/epubs/pdf/c2753023.pdf IBM Knowledge Center: http://www.ibm.com/support/knowledgecenter/ssetd4_9.1.3/lsf_kc_ad min.dita Contributors Dianpeng Liang LSF Architect Xun Pan LSF Developer Xiaofeng Ye LSF Developer Setting up firewall rules for IBM Platform LSF Page 15 of 17

Notices This information was developed for products and services offered in the U.S.A. IBM may not offer the products, services, or features discussed in this document in other countries. Consult your local IBM representative for information on the products and services currently available in your area. Any reference to an IBM product, program, or service is not intended to state or imply that only that IBM product, program, or service may be used. Any functionally equivalent product, program, or service that does not infringe any IBM intellectual property right may be used instead. However, it is the user's responsibility to evaluate and verify the operation of any non-ibm product, program, or service. IBM may have patents or pending patent applications covering subject matter described in this document. The furnishing of this document does not grant you any license to these patents. You can send license inquiries, in writing, to: IBM Director of Licensing IBM Corporation North Castle Drive Armonk, NY 10504-1785 U.S.A. The following paragraph does not apply to the United Kingdom or any other country where such provisions are inconsistent with local law: INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES THIS PUBLICATION "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON- INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states do not allow disclaimer of express or implied warranties in certain transactions, therefore, this statement may not apply to you. Without limiting the above disclaimers, IBM provides no representations or warranties regarding the accuracy, reliability or serviceability of any information or recommendations provided in this publication, or with respect to any results that may be obtained by the use of the information or observance of any recommendations provided herein. The information contained in this document has not been submitted to any formal IBM test and is distributed AS IS. The use of this information or the implementation of any recommendations or techniques herein is a customer responsibility and depends on the customer s ability to evaluate and integrate them into the customer s operational environment. While each item may have been reviewed by IBM for accuracy in a specific situation, there is no guarantee that the same or similar results will be obtained elsewhere. Anyone attempting to adapt these techniques to their own environment does so at their own risk. This document and the information contained herein may be used solely in connection with the IBM products discussed in this document. This information could include technical inaccuracies or typographical errors. Changes are periodically made to the information herein; these changes will be incorporated in new editions of the publication. IBM may make improvements and/or changes in the product(s) and/or the program(s) described in this publication at any time without notice. Any references in this information to non-ibm websites are provided for convenience only and do not in any manner serve as an endorsement of those websites. The materials at those websites are not part of the materials for this IBM product and use of those websites is at your own risk. IBM may use or distribute any of the information you supply in any way it believes appropriate without incurring any obligation to you. Any performance data contained herein was determined in a controlled environment. Therefore, the results obtained in other operating environments may vary significantly. Some measurements may have been made on development-level systems and there is no guarantee that these measurements will be the same on generally available systems. Furthermore, some measurements may have been estimated through extrapolation. Actual results may vary. Users of this document should verify the applicable data for their specific environment. Setting up firewall rules for IBM Platform LSF Page 16 of 17

Information concerning non-ibm products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products and cannot confirm the accuracy of performance, compatibility or any other claims related to non-ibm products. Questions on the capabilities of non-ibm products should be addressed to the suppliers of those products. All statements regarding IBM's future direction or intent are subject to change or withdrawal without notice, and represent goals and objectives only. This information contains examples of data and reports used in daily business operations. To illustrate them as completely as possible, the examples include the names of individuals, companies, brands, and products. All of these names are fictitious and any similarity to the names and addresses used by an actual business enterprise is entirely coincidental. COPYRIGHT LICENSE: Copyright IBM Corporation 2014. All Rights Reserved. This information contains sample application programs in source language, which illustrate programming techniques on various operating platforms. You may copy, modify, and distribute these sample programs in any form without payment to IBM, for the purposes of developing, using, marketing or distributing application programs conforming to the application programming interface for the operating platform for which the sample programs are written. These examples have not been thoroughly tested under all conditions. IBM, therefore, cannot guarantee or imply reliability, serviceability, or function of these programs. Trademarks IBM, the IBM logo, and ibm.com are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. If these and other IBM trademarked terms are marked on their first occurrence in this information with a trademark symbol ( or ), these symbols indicate U.S. registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of IBM trademarks is available on the Web at Copyright and trademark information at www.ibm.com/legal/copytrade.shtml Windows is a trademark of Microsoft Corporation in the United States, other countries, or both. UNIX is a registered trademark of The Open Group in the United States and other countries. Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both. Other company, product, or service names may be trademarks or service marks of others. Contacting IBM To provide feedback about this paper, write to renyux@cn.ibm.com. To contact IBM in your country or region, check the IBM Directory of Worldwide Contacts at http://www.ibm.com/planetwide To learn more about IBM Information Management products, go to http://www.ibm.com/software/data/ Setting up firewall rules for IBM Platform LSF Page 17 of 17