Supported Platform. 2 Installation. 3 Configuration. Application Definition. Sitew ide Settings. Argument Choice

Transcription

1 Contents 1 Table of Contents Part I 2 1 Introduction... to PBS Application Services 3 Copyrights, Tradem... arks, and Third Party Licenses 4 Supported Platform... s 5 System Requirem... ents 6 2 Installation... 8 Install... 9 Com m and Line... Install 19 Uninstall Configuration Application... Definitions 36 Overview of Application... Definition Com ponents 37 Sitew ide Settings Adm inistration... of Application Definitions 40 Add Application... Definitions 41 Update Application... Definitions 43 Remove Application... Definitions 44 Application Input... File 45 Argument Choice ArgumentString ArgumentInt ArgumentFileName ArgumentDirectoryName Application Converter... File 53 Environment Resources Job Wide and Chunk... Resources 57 Using Site Policies Data Staging Conditional... Expressions in Application Definitions 62 Application Runtim... e Scripts and Actions 66 Start Epilogue Restart Terminate Runtime Actions... - Common UNIX/Linux Example 71 Custom Application Definition... PBSJobApp 75 Reserved Words Troubleshooting License... Agreement 82 Index 90 1

2 2 1 PBS Professional Application Services A PBS Works Solution Introduction to Application Services Installation Configuration Application Definitions Troubleshooting License Agreement

3 1.1 3 Introduction to PBS Application Services Today, doing more with less is simply business as usual. Enterprises and institutions are placing higher demands upon their high performance computing infrastructure larger and more complex applications and dynamic end-user work loads. Optimizing end-user productivity and getting the most from limited resources and complex application work loads can seem to be an impossible challenge. The Application Services (formerly known as AIF) component of PBS Professional allows users to quickly and cost effectively customize their PBS Works configuration in response to dynamic market, user and application requirements. PBS Application Services uses a Service Oriented Architecture (SOA) design to provide an array of customizable services for creating an integrated application aware solution. Specifically, PBS Application Services communicates with your PBS Professional servers and compute nodes. Application administrators can use PBS Application Services to create custom, solver specific application definitions. Leveraging key industry standards like the Open Grid Forum High Performance Computing Basic Profile, PBS Application Services makes it easy to respond to dynamic changes to your users and applications. Benefits of PBS Application Services: Define the services needed to integrate custom or third party vertical applications. Standards-based service interfaces and APIs for web access, security, and HPC Basic Profile. Create and provide a customized but cost effective Software as a Service (SaaS) based access and delivery. Customized application definition files can be shared across all supported platforms. Easier integration into Enterprise SOA and business process management (BPM.) Standards-based implementation using Web Services, WS-Security, high performance computing basic profile (HPCBP.) See also Copyrights, Trademarks, and Third Party Licenses Supported Platforms System Requirements

4 PBS Professional Application Services A PBS Works Solution Copyrights, Trademarks, and Third Party Licenses All rights reserved. No part of this online help may be reproduced, transmitted, transcribed, stored in a retrieval system, or translated to another language without the written permission of Altair Engineering, Inc. Copyright Altair Engineering Inc. All Rights Reserved for: PBSWorks, PBS, PBS Professional, PBS Pro, PBS, Personal PBS, and Portable Batch System are trademarks of ALTAIR ENGINEERING INC. Patent #6,859,792 and other patents pending. All are protected under U.S. and international laws and treaties. Copyright [b1]. ALTAIR ENGINEERING INC. Proprietary and Confidential. Contains Trade Secret Information. Not for use or disclosure outside ALTAIR and its licensed clients. Information contained herein shall not be decompiled, disassembled, duplicated, or disclosed in whole or in part for any purpose. Usage of the software is only as explicitly permitted as stated in the end user software license agreement. Except as expressly permitted in the Altair Software License Agreement, any use or disclosure, in whole or in part, of this information without the express prior written permission of ALTAIR is prohibited. Copyright notice does not imply publication. All other copyrights and trademarks are the property of their respective owners. License terms governing use of third party software provided with PBS is governed by the terms and conditions in the owner s license agreements.

5 Supported Platforms The PBS Application Services Server is supported on the following platforms: Novell SUSE Linux Enterprise Server (SLES) 9.X, 10.X, and 11.X on x86 and x86_64 Red Hat Enterprise Linux (RHEL) 4 (AS, ES, and WS) on x86 and x86_64 Red Hat Enterprise Linux (RHEL) 5 and 6 (server and desktop) on x86 and x86_64 Windows XP Professional on x86 and x86_64 Windows Vista on x86 and x86_64 Windows Server 2003 on x86 and x86_64 Windows Server 2008 on x86 and x86_64 Windows Server 2008 R2 on x86_64 CentOS 5.4, 5.5, and 5.6 on x86 and x86_64 Note PBS Application Services does not preclude the client applications from executing on other platforms not listed above. Supported Configuration The PBS Application Services server and the PBS Professional Server are required to be installed on the same host.

6 PBS Professional Application Services A PBS Works Solution System Requirements Minimum Hardware Requirements Hardware Requirement CPU 1 CPU Memory (Physical) 1 GB Disk Space 1.2 GB Minimum Software Requirements PBS Professional Server For further information on the installation of PBS Professional, please refer to the PBS Professional Installation and Administrator s Guide. On Linux systems, the Open SSH daemon using the SSH2 protocol is required. PBS Application Services also includes three third party software packages. PBS Application Services will install and configure these three packages during installation: - JRE 1.6 (Java Runtime Environment) - Apache Tomcat 6.0x - Python 2.5.x Note Python 2.5.x must be installed on every execution node, and so is included with the PBS Professional execution node installation option. See Advance Configuration Parameters Security Supported authentication using NIS and UNIX/Linux password security infrastructure Supported Windows NT LAN Manager (NTLM) authentication Supported High Performance Computing Basic Profile/Basic Execution Services (HPCBP/BES)compatible Web Services Interoperability (WS-I), Web Services Architecture (WS-A), and WSsecurity message format

7 7 Assumptions, Constraints, and Dependencies The PBS Application Services server and the PBS Professional server are installed on the same host PBS Professional infrastructure for file staging: cp or passwordless rcp or scp Windows users username and password exactly match those in the UNIX/Linux security setup

8 8 1.2 PBS Professional Application Services A PBS Works Solution Installation PBS Application Services is supported on Linux and Windows platforms. Therefore, separate steps for Linux and Windows are referenced in the install and uninstall process. GUI install or command line install Uninstall

9 Install Installation of this product requires: System administrator privileges (Windows Administrator or root on Linux) Existing installation of PBS Professional Existing PBS Application Services Administrator account (default is pbsadmin) Removal of any old instances of PBS Application Services (formerly known as AIF) Removal of any instances of Python and Tomcat installed with previous versions of PBS Application Services NOTE: The first step in the installation process is the extraction of the installer to the %tmp% directory on Windows, or the /tmp directory on Linux. There should be enough space in this directory to extract the installer (approximately 3 times the installation size). Also, while the installer is being extracted, nothing should be deleted from this directory. To install PBS Application Services, follow the steps below: 1. Download the installer appropriate for your platform from the user area. 2. For Windows, invoke the installer by double clicking on the exe file. For Linux, invoke the installer by executing the binary file. The installer will be extracted. This may take a few moments. Once the extraction process is complete, a PBS Professional Application Services dialog box will be displayed. You may cancel the install at any time during the installation process by clicking on the Cancel button.

10 10 PBS Professional Application Services A PBS Works Solution 3. To begin the installation process, click Next. A License Agreement dialog will be displayed.

11 11 4. Click on the button next to I accept the terms of the License Agreement and click Next. A Select Install Directory dialog will be displayed. You may install PBS Application Services in the default directory, or you may select a custom directory by clicking on the Choose button. To restore the default directory, click on the Restore Default Folder button. Default installation directory is: on Windows x86: C:\Program Files\PBSWorks\ on Windows x86_64: C:\Program Files (x86)\pbsworks\ on Unix/Linux: /opt/pbsworks/ Select a directory where PBS Applications Services will be installed and click Next. Once a location has been established, a dialog box will be displayed. Please wait until all prechecks have been validated. Note If you get any warning or error messages, see the Troubleshooting section.

12 12 PBS Professional Application Services A PBS Works Solution For Windows, after the pre-checks have been validated a Enter the pbsadmin password dialog box will be displayed. The pbsadmin password will be the one established when you installed PBS Professional. 6. For Windows, enter the pbsadmin password and click Next. Note See the Troubleshooting section if you get a warning message.

13 13 A Select Job File Stage Area dialog will be displayed. You may select a custom directory by clicking on the Choose button. Select the Restore Default Folder button to reset the directory to the default location. The default staging directory is: on Windows: C:\stage on Linux : /stage 7. Select a directory where user's job files will be staged prior to execution and click Next. Note The Stage Area can grow quite large, depending on the size of the average job. Give careful consideration to the disk usage and disk capacity. A License Server dialog box will be displayed.

14 14 PBS Professional Application Services A PBS Works Solution 8. Enter the License Server and click Next. r A Pre-Installation Summary will be displayed. Review all your locations before initiating the installation. You may click Previous to go back and make any changes. 9. Install PBS Application Services by clicking on the Install button.

15 15 A Installing PBS Professional Application Services dialog box will be displayed. You can monitor the progression of your installation via the installation progress toolbar. Once the components of PBS Application Services are installed, you will be asked to wait while the system is being configured. Once installation and configuration of the PBS Application Services is complete, a Start Services dialog box is displayed.

16 16 PBS Professional Application Services A PBS Works Solution 10.Start PBS Application Services. - Click Yes, if you want to start PBS Application Services now. - Click No, to manually start, see below: For Windows: Start the service from the command prompt (Start > Run > cmd). net start PBSProAppSvc For Linux: Start the daemon from a command prompt: /etc/init.d/aif start An Install Complete dialog will be displayed indicating your installation is complete.

17 17 11.Click the Done button to complete the installation. A dialog box will be displayed asking you to restart your computer. The computer must be restarted to complete the installation. 12.Restart the computer to complete the installation process by clicking on Yes. See also Uninstall

18 18 PBS Professional Application Services A PBS Works Solution Troubleshooting

19 1.2.2 Command Line Install Installation of this product requires: System administrator privileges (root privileges) Existing installation of PBS Professional 11.1 or newer Existing PBS Application Services Administrator account (default is pbsadmin) Removal of any old instances of PBS Application Services (formerly known as AIF) Removal of any instances of Python and Tomcat installed with previous versions of PBS Application Services Installation To install PBS Application Services using the command line installer, follow the steps below: 1. Run the command line installer by executing the appropriate installer binary: For example:./pbsproappsvc_11.0.1_linux_x86_64.bin 19

20 20 PBS Professional Application Services A PBS Works Solution The command line installer will launch and you will be prompted to either cancel or continue the installation. Note: You may cancel the install at any time during the installation process by entering "quit" and pressing <Enter>. 2. To begin the installation process, press <Enter>.

21 3. 21 Several pages of License Agreement information will be displayed. Page through the License Agreement by continuing to press <Enter> until you are prompted to accept the terms of the License Agreement.

22 22 PBS Professional Application Services A PBS Works Solution 4. To accept the terms of the License Agreement enter "y" and press <Enter>.

23 5. 23 Enter a directory where PBS Applications Services will be installed and press <Enter>. To accept the default directory, /opt/pbsworks/11.0.1, just press <Enter>.

24 24 PBS Professional Application Services A PBS Works Solution 6. Enter a directory where user's job files will be staged prior to execution and press <Enter>. To accept the default staging directory, /stage, just press <Enter>. Note The Stage Area can grow quite large, depending on the size of the average job. Give careful consideration to the disk usage and disk capacity.

25 7. 25 Enter the License Server and press <Enter>. A Pre-Installation Summary will be displayed. Review all your locations before initiating the installation.

26 26 PBS Professional Application Services A PBS Works Solution 8. To install PBS Application Services press <Enter>. The installation of PBS Application Services will begin.

27 9. 27 Once the installation is complete a message will be displayed asking if PBS Application Services should be started. Enter "1" and press <Enter> to start PBS Application Services. Note: You may manually start PBS Application Services by entering "2", pressing <Enter> and starting the PBS Application Services daemon from a command prompt.: /etc/init.d/aif start An Installation Complete message will be displayed indicating your installation is complete. 10. Press <Enter> to complete the installation. See also Uninstall Troubleshooting

28 PBS Professional Application Services A PBS Works Solution Uninstall To uninstall PBS Application Services, select your platform and complete the following steps below: For Linux For Windows For Linux 1. As root, change directory to Uninstall PBS Application Services directory. This directory will be located in the base install directory, or the location you provided during install. cd /opt/pbsworks/11.0.0/ Uninstall\ PBS\ Professional\ Applications\ Services 2. Execute the Uninstall_PBS_Professional_Applications_Services script./uninstall_pbs_professional_applications_services 3. An Uninstall PBS Professional Applications Services dialog will be displayed. 4. Click Next to start the uninstall process.

29 5. A dialog box will be displayed indicating the uninstall process has started. 6. The uninstall is complete. 29 Any items that could not be removed will be listed, as shown below. 7. Click Done. Note If the installer could not remove files or directories, please take a moment to manually remove them.

30 30 PBS Professional Application Services A PBS Works Solution For Windows 1. As the Windows Administrator navigate to Add or Remove Programs (Start > Control Panel > Add or Remove Programs). 2. Select PBS Professional Application Services and click on the Change/Remove button. 3. An Uninstall PBS Professional Application Services dialog will be displayed. 4. Click Next to start the uninstall process. A dialog box will be displayed indicating the uninstall process has started.

31 31 The uninstall is complete. Any items that could not be removed will be listed, as shown below. 6. Click Done. Note If the installer could not remove files or directories, please take a moment to manually remove them. See also Install Troubleshooting

32 PBS Professional Application Services A PBS Works Solution Configuration Standard server.conf parameters The PBS Application Services (formerly known as AIF) server has a few configurable parameters available to you. PBS Application Services stores its configuration data in AIF_HOME/conf/server. conf. All parameters are stored in key=value pairs. Spaces and tabs are preserved (as spaces) and providing quotes around values is optional. As spaces are preserved, do not use key = value (notice the space surrounding the =) unless spaces are intentional. Empty lines beginning with a comment symbol (ie. #) are ignored. LOGGING_CONFIG_FILE_RELATIVE The PBS Application Services logging configuration file (Log4J). Since the PBS Application Services uses Log4J as its logging and tracing API, you can access additional information on Log4J via the Apache Log4J website. Default Value: server-log.xml DEBUG The PBS Application Services debugging property. If enabled true, PBS Application Services will send intermittent results to files stored in AIF_HOME/temp. This can be an effective debugging mechanism. Default Value: false ADMIN_USER The PBS Application Services administration account (for example, pbsadmin) that has complete control over the PBS Application Services server. This account is required for any remote PBS Application Services administration tasks. On Windows this will be the same pbsadmin account as specified during the installation of PBS Professional and cannot be changed. On Linux the administration account can be a non-privileged user account that has been granted manager status for PBS Professional, and can be changed after installation. Default Value: pbsadmin LOGIN_MODULE_NAME The PBS Application Services authentication module. PBS Application Services uses the Java Authentication and Authorization Service (JAAS) as its authentication abstraction layer. Default Value - Linux: UnixLogin Default Value - Windows: WindowsLogin

33 33 STAGE_ROOT The PBS Application Services file staging location. The staging directory is where the necessary files for jobs submitted, via the clients (for example, PBS Portals), are transferred prior to the portal submitting the job to PBS Application Services, for transfer to PBS Professional for execution. When PBS Professional selects the job for execution, the files are staged-in to the execution host from the staging directory. Upon job completion, any files the job wants to retain are staged-out by PBS Professional to the staging directory, and the portal retrieves them from there when it is ready to do so. Customer written portals may choose to do the same, but are not required to do so. Default Value - Linux: /stage Default Value - Windows: C:/stage FILE_TRANSFER_PROTOCOL The PBS Application Services file transfer mechanism. Default Value: http FILE_EXPIRATION_TIME The PBS Application Services stage area file expiration time. That is, the age in days after which data is removed from "STAGE_ROOT" and its subdirectories with the exception of STAGE_ROOT/profiles. Files in STAGE_ROOT/profiles are never removed by PBS Application Services. Default Value: 14 DYN_RES_PATTERN The PBS Application Services dynamic resource string. Pattern for creation of server_dyn_res records in the PBS Scheduler configuration file (PBS_HOME/sched_priv/sched_config). The following elements are substituted with literal values defined by PBS Application Services admin server $FEATURE, $PYTHON_PATH. Default Value: server_dyn_res: '$FEATURE!"$PYTHON_PATH" "$LICSTATUS_PATH" --get-licenses -F $FEATURE' AIF_SCHEDULING The PBS Application Services scheduling policy. This Boolean attribute determines if PBS Professional should enable an application resource on a given execution host. If set to true, PBS Application Services will: request that a given application be enabled true via a custom Boolean resource of the same name on that node. activate the DYN_RES_PATTERN mechanism which enables users to run application jobs when

34 34 PBS Professional Application Services A PBS Works Solution they have FLEXlm-based licensed applications If set to false, PBS Application Services will consider all available nodes for job execution, regardless if the physical application exists on that node or not. Default Value: true Advanced Configuration Parameters The PBS Application Services (formerly known as AIF) server also has a few optional parameters available to you that are appropriate for advanced configurations. These parameters would also be set in AIF_HOME/conf/server.conf and follow the same formatting requirements as the standard configuration parameters. PYTHON_PATH The PBS Application Services Python binary. The PBS Application Services require that each PBS execution host provide a standard Python binary. The full path to your systems Python binary can be changed here. Note The PBS Professional daemon pbs_mom will look for a binary called pbs_python in PBS_EXEC/bin. If using custom actions or any other operations that involve the pbs_mom accessing this pbs_python (aka python binary) there may be problems if the python libraries are not located in the std locations relative to PBS_EXEC/bin/pbs_python. If Python is installed elsewhere a link to (Linux) or copy of (Windows) the actual python binary must exist as PBS_EXEC/bin/pbs_python. Note As of PBS Professional 11.0 a Python distribution is included as part of the pbs_mom installation. Default Value: $PBS_EXEC/bin/python LICSTATUS_PATH When the PBS Scheduler needs to know the number of licenses available with which to run jobs submitted via PBS Application Services, it uses the lic_status.py utility to query the license server. This parameter can be configured to point to the location of lic_status.py. See the DYN_RES_PATTERN parameter above. Default Value: AIF_EXEC/scripts/lic_status.py ALTAIR_LICENSE_PATH The location of the Altair license server. This is the license server from which PBS Application Services itself attempts to pull license(s). Default Value: Specified during the installation of PBS Applications Services

35 35 FEATURE The application licensing feature, as known to the FLEXlm server. The custom resource created for PBS Professional must be the same as this feature name. Default Value: none

36 PBS Professional Application Services A PBS Works Solution Application Definitions PBS Application Services (formerly known as AIF) requires a predefined set of instructions, called application definitions, to describe your application parameters to users, store their responses, and prepare those responses for job execution via PBS Professional. An application definition provides a flexible set of instructions that can be manipulated to allow for precise control over all aspects of application-specific parameters and job execution. These application definitions are stored in a central repository and will make PBS Professional aware of each of them upon server startup. The following sections will provide you with an overview of an application definition and how to manually administer your application definitions. A working example of an application definition is also included. Overview of Application Definition Components Administration of Application Definition Application Definition PBSJobApp

37 Overview of Application Definition Components An application definition is comprised of a set of Python scripts and two XML files which could include references to site wide configuration settings. A separate application definition with application-specific details is required for each application that will be integrated into PBS Application Services. Application definitions are compliant with the Open Grid Forum High Performance Computing Basic Profile (HPCBP), Business Process Execution Language (BPEL), and Job Submission Description Language (JSDL) standards. site-config.xml, Sitewide Information File The site config file is meant to make application definitions more portable by consolidating data that may change from cluster to cluster in a central location. It is where administrators can define things like policies, version, and path information for all the available applications, billing account information, etc. The data in this file is used by the input and converter files. app-inp-appname.xml, Application Input File The input file is where administrators can define the allowed arguments for a given application. This file is also used by Graphical, Web Based and even Command-Line tools to display these arguments to users for job submission. app-conv-appname.xml, Application Converter File (The HPCBP Converter) The converter file is where administrators take the values received by the user via the input file and communicates this information to PBS Application Services and PBS Professional. This file allows the PBS Application Services administrator to configure the job submission environment. start.py, Application Runtime Scripts The scripts in the runtime directory are what really get executed on the execution hosts. The startup script is what the WLM will execute as the job script. Administrators can directly edit these runtime script(s), taking full advantage of Python to add further inspection and complexity to job submission and finally execution of the application itself. This adds tremendous flexibility as nearly infinite possibilities for job control exist at this phase of job description

38 PBS Professional Application Services A PBS Works Solution Sitewide Settings Sitewide Settings (site-config.xml) By putting some site specific application information in the site-config.xml, application definitions can be made more portable and reusable among different PBS Professional complexes: for example, putting binaries locations and version availability information here, makes the rest of the application definition reusable on a different cluster just by modifying that cluster's site-config.xml. This file can hold virtually any sort of information, since it's used by inclusion from other XML files composing the application definitions. Altair has included certain data in the site-config.xml file for the integration and support of other products from the PBS Professional family. The example below shows a sample of a site-config.xml with the sections currently used by the PBS Professional product suite: XML header - It is mandatory to have the XML header. Applications - This section hold application specific information. Application versions - For each application, you can insert site supported versions and for each version its binary pathname. Job projects (billing accounts) - This section is for integrating PBS Application Services with other products from Altair. Here you can list a set of strings to be used as accounting information to be attached to jobs. Policies - Site policies are values that can be used in a site s application definition XML constituents and/or as values available in job runtime environment (policies are included as environment variables for the jobs). <?xml version="1.0" encoding="utf-8"?> <SiteConfiguration xmlns=" xmlns:site-config=" xmlns:xsi=" xsi: schemalocation=" schemas/site-config.xsd"> <Applications> <Application id="app1"> <ApplicationVersions> <ApplicationVersion> <Option>1.0</Option> <Executable>/apps/bin/bogus_v1.0.sh</Executable> </ApplicationVersion> <ApplicationVersion> <Option>1.1</Option> <Executable>/aps/bin/bogus_v1.1.sh</Executable> </ApplicationVersion> </ApplicationVersions> </Application> <Application id="app2"> <ApplicationVersions> <ApplicationVersion>

39 <Option>1.0</Option> <Executable>/apps/Altair/v1.0/bin/test</Executable> </ApplicationVersion> <ApplicationVersion> <Option>2.2</Option> <Executable>/apps/Altair/v2.2/bin/testing</Executable> </ApplicationVersion> </ApplicationVersions> </Application> </Applications> <JobProjects id="billing_account"> <Option>PDD</Option> <Option>Test</Option> <Option>NotBillable</Option> <Option>Customer</Option> </JobProjects> <Policies> <Policy> <Option>POLICY_APP1</Option> <Value>single_precision</Value> </Policy> <Policy> <Option>MPROC_APP2</Option> <Value>2</Value> </Policy> <Policy> <Option>MPIPROCS</Option> <Value>4</Value> </Policy> </Policies> </SiteConfiguration> 39

40 PBS Professional Application Services A PBS Works Solution Administration of Application Definitions The PBS Application Services has a central location for keeping application definitions and will make PBS Professional aware of each of them upon server startup. You may also wish to perform various administrative tasks against your newly installed application definitions. Add Application Definitions Update Application Definitions Remove Application Definitions Application Definition PBSJobApp If application definitions fail to meet key criteria, they will be moved to an invalid application directory. The directory will be created if it does not exist: AIF_HOME/invalid_applications The criteria the PBS Application Services server will check for include: 1. Application converter file is missing 2. Application input file is missing 3. Application Runtime Script start.py is missing 4. Licensing feature element is present but contains an empty string or a white space in application converter file. The move of the invalid application definitions will show in the server logs. The application definitions can be retrieved from the invalid application directory and modified to meet the criteria necessary to be considered valid. Converter file does not exist for <application> Or Application <application> has <ApplicationLicensingFeature> element it is an empty string An invalid application definition will not prevent the PBS Application Services server from starting up.

41 Add Application Definitions Adding an application definition into the applications directory, followed by a restart of PBS Application Services, will expose your application definition to the user, as well as notify PBS Professional. For Linux 1. Shutdown the PBS Application Services server. /etc/init.d/aif stop 2. Navigate to AIF_HOME/applications. cd AIF_HOME/applications Note Location of AIF_HOME can be found in the /etc/aif.conf file (i.e., AIF_HOME=/ var/spool/aif). 3. Create a directory called appname, and subdirectory called runtime. mkdir appname appname/runtime 4. Place app-inp-appname.xml, app-conv-appname.xml in appname. 5. Place any runtime scripts into appname/runtime. 6. Start the PBS Application Services server. /etc/init.d/aif start For Windows 1. Shutdown the PBS Application Services server by opening a command prompt (Start > Run > cmd ). net stop PBSProAppSvc 2. Navigate to AIF_HOME\applications. cd AIF_HOME\applications Note Location of AIF_HOME can be found in the C:\Program Files\PBSWorks\ \AIF\aif.conf file 3. Create a directory called appname, and subdirectory called runtime. mkdir appname appname\runtime 4. Place app-inp-appname.xml, app-conv-appname.xml in appname. 5. Place any runtime scripts into appname\runtime. 6. Start the PBS Application Services server. net start PBSProAppSvc

42 42 PBS Professional Application Services A PBS Works Solution See also Update Application Definitions Remove Application Definitions Application Definition PBSJobApp

43 Update Application Definitions You can easily modify an existing application definition using your favorite XML editor. No restart is required for modifications that do not modify the name or ID of your application definition (core changes). If the name or ID of your application definition has changed (this includes the name of app-inp-appname as well as app-conv-appname), you will need to restart PBS Application Services before any changes will take effect. For Linux 1. Restart the PBS Application Services server. /etc/init.d/aif restart For Windows 1. Stop and Start the PBS Application Services server. net stop PBSProAppSvc net start PBSProAppSvc See also Add Application Definitions Remove Application Definitions Application Definition PBSJobApp

44 PBS Professional Application Services A PBS Works Solution Remove Application Definitions Removing an application definition from PBS Application Services is also very simple. Simply remove the application definition directory and restart PBS Application Services to remove your application definition from both Application Services and PBS Professional. For Linux 1. Navigate to AIF_HOME/applications. cd AIF_HOME/applications Note Location of AIF_HOME can be found in the /etc/aif.conf file (i.e., AIF_HOME=/ var/spool/aif). 2. Remove application definition directory; for example, appname. rm -rf appname 3. Restart the PBS Application Services server. /etc/init.d/aif restart For Windows 1. Navigate to AIF_HOME\applications via a command prompt (Start > Run > cmd). cd AIF_HOME\applications Note Location of AIF_HOME can be found in the C:\Program Files\PBSWorks\ \AIF\aif.conf file. 2. Remove the application directory and all of its content; for example appname. rd /s appname 3. Stop and Start the PBS Application Services server. net stop PBSProAppSvc net start PBSProAppSvc See also Add Application Definitions Update Application Definitions Application Definition PBSJobApp

45 Application Input File XML character entity references and special characters Unlike traditional HTML with its large range of character entity references, in XML there are only five predefined character entity references. If any of the five special characters are used, for example: a. in an <expression> to be evaluated for convertif function, like: converterif($ncpus>2? <argument1>:<argument2>), b. as part of an argument for convertif function, like: use che apos;ck as <argument1>, c. or are used within an XML element, like: <Description>This does this & that!</description> we need to use the W3C standard on special characters to escape them. The 5 special characters and their XML character entity references are: & (ampersand)? & < (less-than sign)? < > (greater-than sign)? > " (quotation mark)? " ' (apostrophe)? &apos; The above examples would be entered as: a. converterif($ncpus>2?<argument1>:<argument2>) b. use che&apos;ck as <argument1> c. <Description>This does this & that!</description> These XML character entity references should be used when authoring XML documents related to application definitions. The general structure of an input file is very simple. It is an XML document, which contains the name of the application as well as any options (arguments) you would like to make available to the user for submission. Root XML Element with XML Namespace Inclusions An input file must contain the usual XML encoding directive, as well as the root element with XML Namespace inclusions. The Namespace and Schema inclusions configured here should not be modified, as they articulate the w3c and Altair schema information required for use with an input file. <?xml version="1.0" encoding="utf-8"?> <TemplateApplicationInput xmlns=" xmlns:xi=" xmlns:xsi="

46 46 PBS Professional Application Services A PBS Works Solution xmlns:schemalocation=" </TemplateApplicationInput> Application ID and Name Application definitions require an internal identifier with a formally displayed name. The ApplicationId is an internal name used by PBS Application Services for the application definition. It is case sensitive, allows no spaces, and should be considered the true name of your application definition from inside PBS Application Services. The ApplicationName is a formal name, and the name displayed to users and administrators via GUI's or administrative tools. <ApplicationId>appname</ApplicationId> <ApplicationName>appname</ApplicationName> See also Argument Choice ArgumentString ArgumentInt ArgumentFileName ArgumentDirectoryName

47 Argument Choice Arguments can be displayed to the user in various forms, with multiple options or even default values assigned to those options. Arguments can require a value to be input by the user, or possibly require no input at all. The XML element ArgumentChoice is used to define an argument segment within the input file. An ArgumentChoice contains an ArgumentType, which determines the data type of the argument. An ArgumentType can have special extensions which can enable Enumeration of multiple predefined options for a single argument to the user, or the ability to receive Multiple inputs from the user. <ArgumentChoice> <ArgumentStringEnumerated> <Name>VERSION</Name> <Description>Select a version to use.</description> <DisplayName>Version</DisplayName> <InputRequired>true</InputRequired> <option>1.0</option> <option>2.0</option> <option default='true'>3.0</option> </ArgumentStringEnumerated> </ArgumentChoice> The following example shows typical argument elements. Here you will see an ArgumentType labeled as ARG_NAME, using the Name element inside ArgumentType. You can think of the value of Name as a case sensitive, no spaces allowed variable name used by PBS Application Services to hold the value input by the user (ala Python: variable = 'value' ). The DisplayName element is the name of your argument as displayed to the user. The Description element provides a mechanism to explain the purpose of your argument to front-end users. The InputRequired element is a Boolean, and lets PBS Application Services know if user input for this argument is required or not. The Value element enables you to predefine a default value for your argument, which can be overwritten by the user if necessary. <ArgumentChoice> <ArgumentType> <Name>ARG_NAME</Name>

48 48 PBS Professional Application Services A PBS Works Solution <Description>An explanation of my argument.</description> <DisplayName>Argument Name</DisplayName> <InputRequired>true</InputRequired> <Value>default_val</Value> </ArgumentType> </ArgumentChoice> See also Application Definition PBSJobApp ArgumentString ArgumentInt ArgumentFileName ArgumentDirectoryName ArgumentString String arguments can be configured to require a single value using ArgumentString, receive consecutive stringed inputs using ArgumentStringMulti, and to display multiple predefined options using ArgumentStringEnumerated. The options available to Enumerated strings can have a default option already selected for the user by using the default Boolean attribute to option as demonstrated below. <ArgumentChoice> <ArgumentString> <Name>JOB_NAME</Name> <Description>Input the name of your job.</description> <DisplayName>Job Name</DisplayName> <InputRequired>true</InputRequired> <Value>My Job</Value> </ArgumentString> </ArgumentChoice>

49 <ArgumentChoice> <ArgumentStringMulti> <Name>SOLVER_OPTIONS</Name> <Description>Advanced solver options</description> <DisplayName>Solver Option(s)</DisplayName> </ArgumentStringMulti> </ArgumentChoice> <ArgumentChoice> <ArgumentStringEnumerated> <Name>EXTRA_OPTIONS</Name> <DisplayName>Extra Options</DisplayName> <Description>Select additional option.</description> <InputRequired>false</InputRequired> <option>command_1</option> <option default='true'>command_2</option> <option>command_3</option> </ArgumentStringEnumerated> </ArgumentChoice> 49

50 50 PBS Professional Application Services A PBS Works Solution ArgumentInt Integer arguments can be configured to require a single value using ArgumentInt, and to display multiple predefined options using ArgumentIntEnumerated. The options available to Enumerated integers can have a default option already selected for the user by assigning the default Boolean attribute to option as demonstrated in the following example. Integer arguments can also have specific ranges (minimum and maximum integer) defined using the LowerBound and UpperBound elements. <ArgumentChoice> <ArgumentInt> <Name>MEMORY</Name> <Description>The desired memory to use.</description> <DisplayName>Memory (MB)</DisplayName> <InputRequired>true</InputRequired> <LowerBound>100</LowerBound> <UpperBound>2000</UpperBound> <Value>512</Value> </ArgumentInt> </ArgumentChoice> <ArgumentChoice> <ArgumentIntEnumerated> <Name>NCPU</Name> <Description>Number of CPUs</Description> <DisplayName>Cores</DisplayName> <InputRequired>true</InputRequired> <Option default="true">1</option> <Option>2</Option> <Option>3</Option> <Option>4</Option>

51 </ArgumentIntEnumerated> </ArgumentChoice> ArgumentFileName File name arguments can be configured to require a single value using ArgumentFileName, and receive consecutive file inputs using ArgumentFileNameMulti. You may also associate an ApplicationExtension with a given input file. <ArgumentChoice> <ArgumentFileName> <Name>MASTER</Name> <Description>Upload your solver master file.</description> <DisplayName>Master File</DisplayName> <ApplicationExtension>.k</ApplicationExtension> <ApplicationExtension>.dyn</ApplicationExtension> <InputRequired>true</InputRequired> </ArgumentFileName> </ArgumentChoice> <ArgumentChoice> <ArgumentFileNameMulti> <ApplicationExtension>.txt</ApplicationExtension> <Name>INPUT</Name> <Description>Add input files.</description> <DisplayName>Input File(s)</DisplayName> </ArgumentFileNameMulti> 51

52 52 PBS Professional Application Services A PBS Works Solution </ArgumentChoice> ArgumentDirectoryName Directory name arguments can be configured to require a single value using ArgumentDirectoryName. <ArgumentChoice> <ArgumentDirectoryName> <Name>SUBMISSION_DIRECTORY</Name> <Description>Local directory to return output files.</ Description> <DisplayName>Local Directory</DisplayName> <InputRequired>true</InputRequired> </ArgumentDirectoryName> </ArgumentChoice>

53 Application Converter File The converter file describes the information obtained from the user via the input file to the workload manager. The values of arguments selected by the users are available within this file as variables containing their assigned values. It is in this XML document that you begin to prepare your applications resources and job execution environment. This configuration file is consists of several sections: - Header - Root XML Element - Application names references - Software licenses request - Job runtime environment configuration - Run time scripts parameters - Environment variables definitions - Resources requests - Data staging directives Software Licenses Request This section contains the needed information to generate license aware job submissions. This section is meant for integrating application definitions with other tools from Altair Engineering and should not be edited manually;if this is manually populated and modified, customizations in other PBS Application Services configuration files and PBS Professional configuration is necessary to accomplish a successful job submission and execution. <ApplicationLicensing> <ApplicationLicensingFeature>PBSWorks</ApplicationLicensingFeature> </ApplicationLicensing> Root XML Element with XML Namespace Inclusions The converter file must contain the usual XML encoding directive, as well as the root element with XML Namespace inclusions. The Namespace and Schema inclusions configured here should not be modified, as they articulate the w3c and Altair schema information required for use with an input file. <?xml version="1.0" encoding="utf-8"?> <TemplateApplicationConverter

54 54 PBS Professional Application Services A PBS Works Solution xmlns=" xmlns:app-def=" xmlns:jsdl=" xmlns:jsdl-hpcp=" xmlns:xsi=" schemalocation=" schemas/app-conv.xsd"> </TemplateApplicationConverter> Application ID and Name Application definitions require an internal identifier, with a formally displayed name. The ApplicationId is an internal name used by PBS Application Services for the application definition. It is case sensitive, allows no spaces, and should be considered the true name of your application definition from inside PBS Application Services. The ApplicationName is a formal name, and the name displayed to users and administrators via GUI's or administrative tools. <ApplicationId>appname</ApplicationId> <ApplicationName>appname</ApplicationName> See also Environment Resources Data Staging Conditional Expressions in Application Definitions

55 Environment The HPC Profile will enable you to define your job submission environment. The HPCProfileApplication element will open this section, and enable you to use JSDL submission elements such as jsdl-hpcp:executable and jsdl-hpcp:argument, which tells PBS Application Services to use the Python interpreter and the name of the Runtime Script as its first argument. The jsdl-hpcp:environment element allows you to define OS environment variables for use within your job submission environment. This adds amazing flexibility and ease when authoring your job scripts, as any value received from your input file by way of a Name associated with an ArgumentType can be accessed as an environment variable. Using Python, for instance inside start.py, you can access such variables like: if os.environ['ncpu'] == 1: print "Serial processing requested." else: print "Parallel processing requested." <HPCProfileApplication> <jsdl-hpcp:executable>$pythonpath</jsdl-hpcp:executable> <jsdl-hpcp:argument>runtime/start.py</jsdl-hpcp:argument> <jsdl-hpcp:environment name="master">$master</jsdl-hpcp: Environment> <jsdl-hpcp:environment name="input">$input</jsdl-hpcp: Environment> <jsdl-hpcp:environment name="memory">$memory</jsdl-hpcp: Environment> <jsdl-hpcp:environment name="ncpu">$ncpu</jsdl-hpcp:environment> </HPCProfileApplication>

56 PBS Professional Application Services A PBS Works Solution Resources The JSDL provides a mechanism for describing the resources required to properly schedule your application with a workload manager. The jsdl:resources element will open up this section, allowing you to describe your resources. The TotalCPUCount and TotalPhysicalMemory reflect the corresponding values received from your input file. The CustomResource element is used to enable your application definition within the workload manager. In regards to TotalPhysicalMemory, it is important to remember that PBS Professional expresses memory units in bytes. This is relevant within the Input File since you will need to know how memory is being requested of PBS Professional. This is especially relevant here, within the Converter File as you will need to convert from your desired unit type into bytes using a simple math expression as demonstrated below. <jsdl:resources> <TotalCPUCount> <Exact>$NCPUS</Exact> </TotalCPUCount> <TotalPhysicalMemory> <Exact>$MEMORY * 1024 * 1024</Exact> </TotalPhysicalMemory> <CustomResource> <Name>aif_applications_enabled</Name> <Value>appname</Value> </CustomResource> </jsdl:resources> See also Job Wide and Chunk Resources Using Site Policies

57 Job Wide and Chunk Resources Speaking in terms of PBS Professional resource handling, when we specify something like: <CustomResource> <Name>aif_applications_enabled</Name> <Value>appname</Value> </CustomResource> we are using a "job level" resource; this means it could be defined and matched against cluster wide resources, defined as properties of the PBS complex (ie. Licensing resources), or nodal resources (ie. Memory, ncpus). When we are submitting a job spawning to a single node it doesn't make much difference where the resource is defined, but when multiple execution hosts are involved, it is important to distinguish job wide resources from nodal resources. For instance, we might ask for multiple execution hosts with multiple cpus, but we would probably want to check a floating license availability just for the job itself, and not for every execution host. Instead we might ask for specific nodes, matching against properties (ie. Installed operating system, application availability, architecture). For more details on resources in PBS Professional and how to customize them, refer to the PBS Professional Administration Guide. Jobs required to spawn on multiple execution hosts must be composed of multiple chunks. A chunk, in PBS Professional, is the smallest allocated job constituent; an execution host can hold several chunks from the same job, but a chunk can be only on a single execution host. Node level resources are defined as: <CustomResource> <Name>aif_applications_enabled</Name> <Value>RadiossSPMD</Value> <NodeLevel>true</NodeLevel> </CustomResource> <CustomResource> <Name>mpiprocs</Name> <Value>2</Value> <NodeLevel>true</NodeLevel> </CustomResource> The above node level requests, combined in the same converter file, will result in a submission request equivalent to: qsub -l select=1:aif_applications_enabled=radiossspmd:mpiprocs=2 Multiple chunks jobs are generated specifying the number of chunks we need and then customizing our chunks attributes. Here's an example combining job level and chunk level resources: <Chunks>$NCPUS/2</Chunks> <CustomResource> <Name>place</Name> <Value>scatter</Value>

58 58 PBS Professional Application Services A PBS Works Solution </CustomResource> <CustomResource> <Name>ncpus</Name> <Value>2</Value> <NodeLevel>true</NodeLevel> </CustomResource> <CustomResource> <Name>mpiprocs</Name> <Value>2</Value> <NodeLevel>true</NodeLevel> </CustomResource> <CustomResource> <Name>mem</Name> <Value>( $MEMORY * 1024 * 1024 ) / ( $NCPUS / 2 ) </Value> <NodeLevel>true</NodeLevel> </CustomResource> <CustomResource> <Name>aif_applications_enabled</Name> <Value>RadiossSPMD</Value> <NodeLevel>true</NodeLevel> </CustomResource> The above, assuming we input in the submission interface 8 as NCPUS and 16 as MEMORY, will result in a submission like: qsub -l place=scatter -l select=4:ncpus=2:mpiprocs=2:mem= : aif_applications_enabled=radiossspmd See also Resources Using Site Policies We can further polish conversion rules applying PBS Application Services Site Policies: they are defined in site-config.xml and are site wide constants we can reuse during conversion and at runtime. If we insert policies definitions in site-config.xml: <Policies> <Policy> <Option>MPROC_VALUE</Option> <Value>2</Value> </Policy> <Policy> <Option>ALLOCATIONPOLICY</Option> <Value>BigDisk</Value> </Policy> </Policies> we can later use the values from the converter file:

59 59 <CustomResource> <Name>node_property</Name> <Value>$ALLOCATIONPOLICY</Value> <NodeLevel>true</NodeLevel> </CustomResource> <Chunks>$NCPUS/$MPROC_VALUE</Chunks> <CustomResource> <Name>ncpus</Name> <Value>$MPROC_VALUE</Value> <NodeLevel>true</NodeLevel> </CustomResource> And also at runtime, inside the runtime scripts; this is sample code from an hypothetical Python startup runtime script (note the AIF_ prefix): cmd="/opt/bin/run_application.sh -threads " + os.environ['aif_mproc_value'] print cmd os.system(cmd) Site policies are useful to make application definitions more portable: if properly built they can be exported to a new (compatible) PBS Professional complex and adapted to it just by customizing the site configuration, not by touching again the application definition itself. See also Resources

60 PBS Professional Application Services A PBS Works Solution Data Staging The JSDL provides a mechanism for describing the way workload managers should handle data staging for your application. The jsdl:datastaging element will open up this section allowing you to describe the file names and directory paths required. You will want to configure how each file and directory received are to be staged. The jdsl:filename element should contain the name of your file or directory as defined by the user via the input file. A jdsl:creationflag needs to be specified, as well as the jdsl:source of the file or location of the jdsl:target directory. <jsdl:datastaging> <jsdl:filename>name($master)</jsdl:filename> <jsdl:creationflag>overwrite</jsdl:creationflag> <jsdl:source> <jsdl:uri>$master</jsdl:uri> </jsdl:source> </jsdl:datastaging> <jsdl:datastaging> <jsdl:filename>name($include)</jsdl:filename> <jsdl:creationflag>overwrite</jsdl:creationflag> <jsdl:source> <jsdl:uri>$include</jsdl:uri> </jsdl:source> </jsdl:datastaging> <jsdl:datastaging> <jsdl:filename>*</jsdl:filename> <jsdl:creationflag>overwrite</jsdl:creationflag> <jsdl:target> <jsdl:uri>$submission_directory/*</jsdl:uri> </jsdl:target>

61 </jsdl:datastaging> 61

62 PBS Professional Application Services A PBS Works Solution Conditional Expressions in Application Definitions Many application job scripts expect specific values for the arguments passed by a client. For example, a job script expects Boolean -check/-nocheck values for the argument that is represented by checkbox element of GUI client (and Boolean-based xml element in the PBS Application Services framework) The resulting values could be: 1. When a value of Boolean is passed as true user intends to send -check option to the application where as when the value of Boolean is false user intends to pass -nocheck as the option to the application. 2. User wants to send -check for true and nothing for false and vice-versa. 3. User wants to send -check=true or -check=false 4. User wants to send -check=1 or -check=0 The place to control this transformation is app-conv-<applicationid>.xml. Boolean variables will be evaluated and their values will be replaced with user defined string values. In app-conv-<applicationid>. xml, an hpcbp application s job arguments are represented by <jsdl-hpcp:argument> elements. A Boolean based function could be represented as: <jsdl-hpcp:argument>converterif(<expression>)</jsdl-hpcp:argument> The converterif() function has one argument and one or two return options. Using the example scenario above, the converterif() function would be evaluated like: converterif ($ARGVAL?-check:-nocheck) Here we have variable $ARGVAL as an argument, and return options are -check and nocheck. The expected result will be -check if $ARGVAL is evaluated to True because: - xsd:int is defined in user input - xsd:boolean true - xsd:string and not empty The expected result will be -nocheck if $ARGVAL is evaluated to false because: - xsd:boolean false - xsd:string and empty - variable is not defined so its type is unknown and variable is treated as an empty xsd:string If they are part of the <expression> or part of an <argument> we need to escape them with the backslash to let the parser know this is not a separator, but the part of the <expression> or the

63 63 <argument>. Other examples of the expression syntax: Pass check=true if $ARGVAL evaluates to True, nothing if $ARGVAL evaluates to False: <jsdl-hpcp:argument>converterif($argval?-check=true) </jsdl-hpcp:argument> Pass nothing if $ARGVAL evaluates to True and nocheck if $ARGVAL evaluates to False: <jsdl-hpcp:argument>converterif($argval?:-nocheck) </jsdl-hpcp:argument Various examples of what the result would be if $SOME_ARG were to evaluate as: <jsdl-hpcp:argument>converterif($some_arg?-topt $SOME_ARG:-fopt) </jsdl-hpcp:argument> a. $SOME_ARG of xsd:string evaluated to hello : Result is (-topt hello) b. $SOME_ARG of xsd:string evaluated to true : Result is (-topt true) c. $SOME_ARG of xsd:string evaluated to false : Result is (-topt false) d. $SOME_ARG of xsd:string is empty(optional value not passed) : Result is (-fopt) e. $SOME_ARG of xsd:boolean evaluated to true : Result is (-topt true) f. $SOME_ARG of xsd:boolean evaluated to false : Result is (-fopt) g. $SOME_ARG of xsd:int evaluated to 1 : Result is (-topt 0) h. $SOME_ARG of xsd: int evaluated to 0 : Result is (-topt 1) A mathematical expression can be used in the function: <jsdl-hpcp:argument>converterif(1 * ?-check:-nocheck) </jsdl-hpcp:argument> Or in setting a minimum value for a resource: <Exact>converterIf($NCPUS<2?2:$NCPUS) List of the xml elements that can use the converterif(): - <jsdl-hpcp:argument> converterif(<expression>) </jsdl-hpcp:argument> - <jsdl-hpcp:environment> converterif(<expression>) </jsdl-hpcp:environment> - <Exact> element of <TotalPhysicalMemory> element of <jsdl:resources> <jsdl:resources> <TotalPhysicalMemory> <Exact> converterif(<expression>)</exact> </TotalPhysicalMemory> </jsdl:resources> - <Exact> element of <TotalCPUCount> element of <jsdl:resources> <jsdl:resources> < TotalCPUCount>

64 64 PBS Professional Application Services A PBS Works Solution <Exact> converterif(<expression>) </Exact> </TotalCPUCount> </jsdl:resources> - <Chunks> element of <jsdl:resources> <jsdl:resources> <Chunks> converterif(<expression>) </Chunks> </jsdl:resources> - <jsdl:filename> element of <jsdl:filestaging> <jsdl:datastaging> <jsdl:filename> converterif(<expression>) </jsdl:filename> </jsdl:datastaging> conveterif() can use nested functions. The possible nested functions in converterif() are: - name() - namemerge() - Jep math functions - Jep math expressions - aif variables Nested functions in name(): - Jep math functions - Jep math expressions - aif variables Nested functions in nammerge(): - Jep math functions - Jep math expressions - aif variables An example of a nested function used for determining the FileName for file staging: <jsdl:filename> converterif($use_master?name($master):name($restart)) </jsdl:filename> The order of evaluation: 1. All variables (e.g. $MASTER) get evaluated first. a. Variables that could not be resolved are replaced with empty strings ( ).

65 2. 65 For each xml element (see the list of possible xml elements above) the innermost function is found (converterif, name(), or namemerge()). a. If this function argument is math expression or Jep math function, it will be resolved with Jep. b. If the Jep evaluation results in an exception the exception will be propagated up and will show as an OperationFailed fault in the response message. Job submission will be interrupted. 3. Successful evaluation of the function argument results in some function output. 4. The function output value may be an argument for the next outermost function, and so on the process works recursively until the most outermost function is resolved. Any errors will result in error messages in PBS Application Services server log files: If one of the brackets is missing or separator? is missing: Invalid syntax in expression <expression> in function <function_name> In case of unresolved variables or math expressions: "Error while parsing expression <expression> in function <function name>" - AIF ignores the variable and replaces it with an empty string. - The job submission will be interrupted if evaluation of a math expression results in an exception. WS method will return OperationFailed fault. See also Environment Resources Data Staging

66 PBS Professional Application Services A PBS Works Solution Application Runtime Scripts and Actions An application definition uses Python runtime scripts associated with runtime actions for controlling various aspects of job execution. Runtime scripts can take full advantage of core Python modules, OSspecific commands or virtually any programming task that may be required for job execution. Runtime Actions PBS Application Services checks for the existence of runtime scripts in a directory named runtime, located inside your application definition. The start and epilogue actions enable you to control the beginning and end of your job. The start action is the primary action, and should be considered the main job script for your application definition as it actually executes the application associated with the job. The epilogue action gives you the ability to conduct post job operations. The restart and terminate actions provide you with the ability to customize the restart and termination of the job, should these actions be needed by the job's submitting user. The epilogue, restart, and terminate actions are achieved with additional PBS-specific configurations. It is important to note that any dependency scripts that runtime actions might need can also be included inside the runtime directory of your application definition. Below is a complete list of runtime actions that you can create: start epilogue restart terminate custom

67 Start The start runtime action should be considered the primary job execution script. The start action, file name start.py, is responsible for bringing together all information received prior to this point and executing the application associated with your application definition. As with any runtime action, you can use the Python 'os' module to retrieve environment variables defined in the converter file. The values retrieved can be used to build a system command line that executes your application. import os # for grabbing environment variables... input = os.environ['input'] memory = os.environ['memory'] ncpu = os.environ['ncpu'] # for job execution... os.system( '/usr/bin/appname ' + input + ' -ncpu=' + ncpu + ' --memory=' + memory ) Go to Runtime Actions - Common UNIX/Linux Example See also Runtime Action Scripts and Actions Epilogue Restart Terminate Custom

68 PBS Professional Application Services A PBS Works Solution Epilogue The epilogue runtime action enables post job operations. This is achieved by modifying the PBS Professional epilogue mechanism (on each execution host) to execute your application's epilogue.py script when necessary. To learn more about configuring PBS Professional for use with an epilogue script, please see the Integrating & Administration section of the PBS Professional Administrators Guide. Go to Runtime Actions - Common UNIX/Linux Example See also Runtime Action Scripts and Actions Start Restart Terminate Custom

69 Restart The restart runtime action enables the restarting of your application during its runtime phase. This is achieved by way of PBS Professional $action directives inside each nodes configuration file (please see Configuring MOM for Site-Specific Actions inside the PBS Professional Administrators Guide). As with the PBS Professional epilogue script used to execute your application's own 'epilogue' action, you can create a restart wrapper that can do the same. Go to Runtime Actions - Common UNIX/Linux Example See also Runtime Action Scripts and Actions Start Epilogue Terminate Custom

70 PBS Professional Application Services A PBS Works Solution Terminate The terminate action enables the killing of your application during any phase. This is achieved by way of PBS $action directives inside each nodes configuration file (please see Configuring MOM for Site-Specific Actions inside the PBS Professional Administrators Guide). As with the PBS Professional epilogue script used to execute your application's own 'epilogue' action, you can create a terminate wrapper that can do the same. Go to Runtime Actions - Common UNIX/Linux Example See also Runtime Action Scripts and Actions Start Epilogue Restart Custom

71 Runtime Actions - Common UNIX/Linux Example This example demonstrates how PBS Professional-specific runtime actions can safely wrap against your application-specific runtime action (such as epilogue, restart, and terminate) written in bourne shell for UNIX/Linux. You can find additional examples inside of the PBS Application Services unsupported directory, located at $AIF_EXEC/unsupported/actions-unix/. #!/bin/sh action_name="epilogue" jobid=$1 jobname=$4 user=$2 job_dir=$pbs_jobdir if [ -z "$jobid" -o -z "$jobname" -o -z "$user" ]; then echo "`basename $0`: No arguments: exiting." exit 1 fi state=/tmp/$action_name${jobid} if [! -f $state ]; then touch $state if [ -e $job_dir/runtime/$action_name.py ]; then for var in `cat $job_dir/pbsjob.env grep AIF`; do export $var; done su - $2 -c "$AIF_PYTHON_PATH $PBS_JOBDIR/runtime/$action_name.py" fi rm -f $state fi until [! -f $state ]; do sleep 5 done exit 0

72 PBS Professional Application Services A PBS Works Solution Custom PBS Application Services allows for custom runtime actions. A custom runtime action is an action that can be instantiated by the submission user after the job has already been submitted. Custom runtime actions can be a powerful mechanism for custom job control. You can create custom runtime actions against jobs when they are in the RUNNING job state. To use custom runtime actions with your application definition, you will need to create an additional XML file known as app-actions-appname.xml. You will also need to place any required runtime scripts into your application definition's 'runtime' directory. As with the input and converter files, the actions file begins with the typical XML encoding directive, along with the required XML namespace inclusions. You will also provide the ApplicationId of your application definition. <?xml version="1.0" encoding="utf-8"?> <ApplicationActions> xmlns=" xmlns:xsi=" xsi:schemalocation=" <ApplicationId>appname</ApplicationId> </ApplicationActions> Every custom runtime action for your application may be included within this file. The CustomAction element opens up the custom runtime action block. The JobStates element can contain a single or multiple JobState elements. The Name, DisplayName, and Description elements, as with the input and converter files, contain the name as known to PBS Application Services, the display name as known to users, and the description of the custom runtime action. As with an input file, your custom runtime action can receive arguments. For a more detailed explanation of ArgumentChoice and its available types, please consult the Application Input File section of this document. Finally, near the end of your individual custom runtime action, you will notice the Executable element, which contains the Language and Name of the runtime script associated with your action. PBS Application Services will automatically provide the values for each ArgumentChoice by way of an environment variable, which can be retrieved by the runtime script associated with your custom runtime action. <CustomAction> <JobStates> <JobState>RUNNING</JobState> </JobStates>

73 <Name>CREATE_FILE</Name> <DisplayName>Create File</DisplayName> <Description>Create a file with a string.</description> <ArgumentChoice> <ArgumentString> <Name>FILE</Name> <Description>Name the file to create.</description> <DisplayName>File Name</DisplayName> <InputRequired>true</InputRequired> </ArgumentString> </ArgumentChoice> <ArgumentChoice> <ArgumentString> <Name>STRING</Name> <Description>String to input to the file.</description> <DisplayName>String</DisplayName> <InputRequired>true</InputRequired> </ArgumentString> </ArgumentChoice> <Executable> <Language>PYTHON</Language> <Name>CreateFile.py</Name> </Executable> <ShowOutput>true</ShowOutput> </CustomAction> 73

74 74 PBS Professional Application Services A PBS Works Solution See also Runtime Action Scripts and Actions Start Epilogue Restart Terminate Runtime Actions - Common UNIX/Linux Example

75 Application Definition PBSJobApp PBSJobApp This is an example of how an administrator could create an application definition enabling users to use custom job scripts akin to the more common PBS Professional job script. PBSJobApp allows these job scripts to be written in any language by using its start.py to parse the first line (shebang line) to determine the correct interpreter to execute your job script against. PBSJobApp's versatility allows the following: extra arguments to be passed to the job script specification of advanced multi-node placement options the inclusion of additional files To view start.py and its associated xml files, please look in the applications directory of your implementation of PBS Applications Services: AIF_HOME/applications/PBSJobApp/runtime/start.py AIF_HOME/applications/PBSJobApp/app-inp-PBSJobApp.xml AIF_HOME/applications/PBSJobApp/app-conv-PBSJobApp.xml

76 PBS Professional Application Services A PBS Works Solution Reserved Words The table below illustrates the reserved words used by the PBS Application Service. The reserved words enable unification of application definitions and automatic rendering of GUIs based on these definitions. The following list of names are identified. BILLING_ACCOUNT PBS Professional "account" attribute CHUNK PBS Professional "requested chunks count" attribute CPUTIME PBS Professional "cput" attribute EXECUTABLE path to application executable JOB_NAME PBS Professional "job_name" attribute MASTER master file MEMORY PBS Professional "mem" attribute NCPUS PBS Professional "ncpus" attribute PYTHON_PATH Python Interpreter QUEUE PBS Professional "queue" attribute SOFTWARE PBS Professional "software" attribute SUBMISSION_DIRECTORY job submission directory VERSION application version VMEMORY PBS Professional "vmem" attribute WALLTIME PBS Professional "walltime" attribute

77 Troubleshooting PBS Professional version incompatible Question: I get a PBS Professional version incompatible error box, as shown below, what do I do? Answer: The installer has detected you have an incompatible version and will abort. Make sure you have PBS Professional version installed. Execute the PBS qstat Bf command from a command prompt and look for pbs_version attribute to yield For instance: > qstat -Bf Server: server01 server_state = Active server_host = server01 scheduling = True total_jobs = 0 state_count = Transit:0 Queued:0 Held:0 Waiting:0 Running:0 Exiting:0 Begun:0 default_queue = workq log_events = 511 mail_from = adm query_other_jobs = True resources_available.aif_applications = <Applications>,</Applications> resources_available.aif_billing_accounts = <BillingAccounts>,</BillingAccounts> resources_default.ncpus = 1 default_chunk.ncpus = 1 scheduler_iteration = 600 FLicenses = 0 resv_enable = True node_fail_requeue = 310 max_array_size = single_signon_password_enable = True pbs_license_file_location = 7788@license-svr pbs_license_min = 0 pbs_license_max = pbs_license_linger_time = 3600 license_count = Avail_Global:0 Avail_Local:0 Used:0 High_Use:0

78 78 PBS Professional Application Services A PBS Works Solution pbs_version = PBSPro_ eligible_time_enable = False job_history_enable = True max_concurrent_provision = 5 PBS Professional is not installed Question: I get a PBS Professional is not installed error message, as shown below, what do I do? Answer: You must install PBS Professional. If PBS Professional is installed, verify that PBS_CONF_FILE environment variable is defined and that you can execute the PBS qstat command from the command prompt. PBS Professional Server is not accessible Question: I get a PBS Professional Server is not accessible error message, as shown below, what do I do? Answer: The installation requires the PBS Professional Server to running, otherwise it will abort. Make sure your PBS Professional Server is running. Remember that the PBS Application Service must be installed on the same machine as the PBS Professional Server. You can verify if the PBS Professional Server is running via

79 79 On Linux: Verify that the pbs_server is running ps ef grep pbs_server If the pbs_server process is not running, execute the following command /etc/init.d/pbs start On Windows: Verify that the pbs_server process is running via the Windows Task Manager. If the pbs_server process is not running, execute the following command through Windows Command Shell (Start > Run > cmd.exe) net start pbs_server Empty password Question: I get an Empty password error warning message, as shown below, what do I do? Answer: You must use your password from the PBS Application Services Administrator account to proceed with the installation. Service account does not exist (Windows) Question: Using Windows, I get a Administration account does not exist error message, as shown below, what do I do?

80 80 PBS Professional Application Services A PBS Works Solution Answer: Verify that the PBS Professional services account that the PBS Application Service relies on has been created correctly. Please refer to the PPAS Administration Guide, Configuration section for more information. Insufficient Privileges for installation (Linux) Question: Using Linux, I get an Insufficient Privileges for installation error message, as shown below, what do I do? Answer: Ensure that you are installing as root. Existing instance of Python found (Windows) Question: Using Windows, I get an Existing instance of Python found error message, as shown below, what do I do?

81 Answer: Uninstall existing Python installation and restart PBS Application Services installation. See also Install Uninstall 81

82 PBS Professional Application Services A PBS Works Solution License Agreement CAUTION! PRIOR TO INSTALLATION OR USE OF THE SOFTWARE YOU MUST CONSENT TO THE FOLLOWING SOFTWARE LICENSE TERMS AND CONDITIONS BY CLICKING THE "I ACCEPT" BUTTON BELOW. YOUR ACCEPTANCE CREATES A BINDING LEGAL AGREEMENT BETWEEN YOU AND ALTAIR. IF YOU DO NOT HAVE THE AUTHORITY TO BIND YOUR ORGANIZATION TO THESE TERMS AND CONDITIONS, YOU MUST CLICK "I DO NOT ACCEPT" AND THEN HAVE AN AUTHORIZED PARTY IN THE ORGANIZATION THAT YOU REPRESENT ACCEPT THESE TERMS. IF YOU, OR THE ORGANIZATION THAT YOU REPRESENT, HAS A MASTER SOFTWARE LICENSE AGREEMENT ( MASTER SLA ) ON FILE AT THE CORPORATE HEADQUARTERS OF ALTAIR ENGINEERING, INC. ( ALTAIR ), THE MASTER SLA TAKE PRECEDENCE OVER THESE TERMS AND SHALL GOVERN YOUR USE OF THE SOFTWARE. MODIFICATION(S) OF THESE SOFTWARE LICENSE TERMS IS EXPRESSLY PROHIBITED. ANY ATTEMPTED MODIFICATION(S) WILL BE NONBINDING AND OF NO FORCE OR EFFECT UNLESS EXPRESSLY AGREED TO IN WRITING BY AN AUTHORIZED CORPORATE OFFICER OF ALTAIR. ANY DISPUTE RELATING TO THE VALIDITY OF AN ALLEGED MODIFICATION SHALL BE DETERMINED IN ALTAIR S SOLE DISCRETION. Altair Engineering, Inc. - Software License Agreement THIS SOFTWARE LICENSE AGREEMENT, including any Additional Terms (together the Agreement ), shall be effective as of the date of YOUR acceptance of these software license terms and conditions (the Effective Date ) and is between ALTAIR ENGINEERING, INC., 1820 E. Big Beaver Road, Troy, MI , USA a Michigan corporation ( Altair ), and YOU, or the organization on whose behalf you have authority to accept these terms (the Licensee ). Altair and Licensee, intending to be legally bound, hereby agree as follows: 1. DEFINITIONS. In addition to terms defined elsewhere in this Agreement, the following terms shall have the meanings defined below for purposes of this Agreement: Additional Terms. Additional Terms are those terms and conditions which are determined by an Altair Subsidiary to meet local market conditions. Documentation. Documentation provided by Altair or its resellers on any media for use with the Products. Execute. To load Software into a computer's RAM or other primary memory for execution by the computer. Global Zone: Software is licensed based on three Global Zones: the Americas, Europe and AsiaPacific. When Licensee has Licensed Workstations located in multiple Global Zones, which are connected to a single License (Network) Server, a premium is applied to the standard Software License pricing for a single Global Zone. ISV/Independent Software Vendor. A software company providing its products, ( ISV Software ) to Altair s Licensees through the Altair License Management System using Altair License Units. License Log File. A computer file providing usage information on the Software as gathered by the Software. License Management System. The license management system (LMS) that accompanies the Software and limits its use in accordance with this Agreement, and which includes a License Log

83 83 File. License (Network) Server. A network file server that Licensee owns or leases located on Licensee s premises and identified by machine serial number and/or HostID on the Order Form. License Units. A parameter used by the LMS to determine usage of the Software permitted under this Agreement at any one time. Licensed Workstations. Single-user computers located in the same Global Zone(s) that Licensee owns or leases that are connected to the License (Network) Server via local area network or Licensee s private wide-area network. Maintenance Release. Any release of the Products made generally available by Altair to its Licensees with annual leases, or those with perpetual licenses who have an active maintenance agreement in effect, that corrects programming errors or makes other minor changes to the Software. The fees for maintenance and support services are included in the annual license fee but perpetual licenses require a separate fee. Order Form. Altair's standard form in either hard copy or electronic format that contains the specific parameters (such as identifying Licensee s contracting office, License Fees, Software, Support, and License (Network) Servers) of the transaction governed by this Agreement. Products. Products include Altair Software, ISV Software, and/or Suppliers software; and Documentation related to all of the forgoing. Proprietary Rights Notices. Patent, copyright, trademark or other proprietary rights notices applied to the Products, packaging or media. Software. The Altair software identified in the Order Form and any Updates or Maintenance Releases. Subsidiary. Subsidiary means any partnership, joint venture, corporation or other form of enterprise in which a party possesses, directly or indirectly, an ownership interest of fifty percent (50%) or greater, or managerial or operational control. Suppliers. Any person, corporation or other legal entity which may provide software or documents which are included in the Software. Support. The maintenance and support services provided by Altair pursuant to this Agreement. Templates. Human readable ASCII files containing machine-interpretable commands for use with the Software. Term. The term of licenses granted under this Agreement. Annual licenses shall have a 12-month term of use unless stated otherwise on the Order Form. Perpetual licenses shall have a term of twenty-five years. Maintenance agreements for perpetual licenses have a 12-month term. Update. A new version of the Products made generally available by Altair to its Licensees that includes additional features or functionalities but is substantially the same computer code as the existing Products. 2. LICENSE GRANT. Subject to the terms and conditions set forth in this Agreement, Altair hereby grants Licensee, and Licensee hereby accepts, a limited, non-exclusive, non-transferable license to: a) install the Products on the License (Network) Server(s) identified on the Order Form for use only at the sites identified on the Order Form; b) execute the Products on Licensed Workstations in accordance with the LMS for use solely by Licensee s employees, or its onsite Contractors who have agreed to be bound by the terms of this Agreement, for Licensee s internal business use on

84 84 PBS Professional Application Services A PBS Works Solution Licensed Workstations within the Global Zone(s) as identified on the Order Form and for the term identified on the Order Form; c) make backup copies of the Products, provided that Altair s and its Suppliers and ISV s Proprietary Rights Notices are reproduced on each such backup copy; d) freely modify and use Templates, and create interfaces to Licensee s proprietary software for internal use only using APIs provided that such modifications shall not be subject to Altair s warranties, indemnities, support or other Altair obligations under this Agreement; and e) copy and distribute Documentation inside Licensee s organization exclusively for use by Licensee s employees and its onsite Contractors who have agreed to be bound by the terms of this Agreement. A copy of the License Log File shall be made available to Altair automatically on no less than a monthly basis. In the event that Licensee uses a third party vendor for information technology (IT) support, the IT company shall be permitted to access the Software only upon its agreement to abide by the terms of this Agreement. Licensee shall indemnify, defend and hold harmless Altair for the actions of its IT vendor(s). 3. RESTRICTIONS ON USE. Notwithstanding the foregoing license grant, Licensee shall not do (or allow others to do) any of the following: a) install, use, copy, modify, merge, or transfer copies of the Products, except as expressly authorized in this Agreement; b) use any back-up copies of the Products for any purpose other than to replace the original copy provided by Altair in the event it is destroyed or damaged; c) disassemble, decompile or unlock, reverse translate, reverse engineer, or in any manner decode the Software or ISV Software for any reason; d) sublicense, sell, lend, assign, rent, distribute, publicly display or publicly perform the Products or Licensee s rights under this Agreement; e) allow use outside the Global Zone(s) or User Sites identified on the Order Form; f) allow third parties to access or use the Products such as through a service bureau, wide area network, Internet location or time-sharing arrangement except as expressly provided in Section 2(b); g) remove any Proprietary Rights Notices from the Products; h) disable or circumvent the LMS provided with the Products; or (i) link any software developed, tested or supported by Licensee or third parties to the Products (except for Licensee s own proprietary software solely for Licensee's internal use). 4. OWNERSHIP AND CONFIDENTIALITY. Licensee acknowledges that all applicable rights in patents, copyrights, trademarks, service marks, and trade secrets embodied in the Products are owned by Altair and/or its Suppliers or ISVs. Licensee further acknowledges that the Products, and all copies thereof, are and shall remain the sole and exclusive property of Altair and/or its Suppliers and ISVs. This Agreement is a license and not a sale of the Products. Altair retains all rights in the Products not expressly granted to Licensee herein. Licensee acknowledges that the Products are confidential and constitute valuable assets and trade secrets of Altair and/or its Suppliers and ISVs. Licensee agrees to take the same precautions necessary to protect and maintain the confidentiality of the Products as it does to protect its own information of a confidential nature but in any event, no less than a reasonable degree of care, and shall not disclose or make them available to any person or entity except as expressly provided in this Agreement. Licensee shall promptly notify Altair in the event any unauthorized person obtains access to the Products. If Licensee is required by any governmental authority or court of law to disclose Altair s or its ISV s or its Suppliers confidential information, then Licensee shall immediately notify Altair before making such disclosure so that Altair may seek a protective order or other appropriate relief. Licensee s obligations set forth in Section 3 and Section 4 of this Agreement shall survive termination of this Agreement for any reason. Altair s Suppliers and ISVs, as third party beneficiaries, shall be entitled to enforce the terms of this Agreement directly against Licensee as necessary to protect Supplier s intellectual property or other rights. Altair and its resellers providing support and training to licensed end users of the Products shall keep confidential all Licensee information provided to Altair in order that Altair may provide Support and training to Licensee. Licensee information shall be used only for the purpose of assisting Licensee in its use of the licensed Products. Altair agrees to take the same precautions necessary

85 85 to protect and maintain the confidentiality of the Licensee information as it does to protect its own information of a confidential nature but in any event, no less than a reasonable degree of care, and shall not disclose or make them available to any person or entity except as expressly provided in this Agreement. 5. MAINTENANCE AND SUPPORT. Maintenance. Altair will provide Licensee, at no additional charge for annual licenses and for a maintenance fee for paid-up licenses, with Maintenance Releases and Updates of the Products that are generally released by Altair during the term of the licenses granted under this Agreement, except that this shall not apply to any Term or Renewal Term for which full payment has not been received. Altair does not promise that there will be a certain number of Updates (or any Updates) during a particular year. If there is any question or dispute as to whether a particular release is a Maintenance Release, an Update or a new product, the categorization of the release as determined by Altair shall be final. Licensee agrees to install Maintenance Releases and Updates promptly after receipt from Altair. Maintenance Releases and Updates are subject to this Agreement. Altair shall only be obligated to provide support and maintenance for the most current release of the Software and the most recent prior release. Support. Altair will provide support via telephone and to Licensee at the fees, if any, as listed on the Order Form. If Support has not been procured for any period of time for paid-up licenses, a reinstatement fee shall apply. Support consists of responses to questions from Licensee s personnel related to the use of the then-current and most recent prior release version of the Software. Licensee agrees to provide Altair with sufficient information to resolve technical issues as may be reasonably requested by Altair. Licensee agrees to the best of its abilities to read, comprehend and follow operating instructions and procedures as specified in, but not limited to, Altair s Documentation and other correspondence related to the Software, and to follow procedures and recommendations provided by Altair in an effort to correct problems. Licensee also agrees to notify Altair of a programming error, malfunction and other problems in accordance with Altair s then current problem reporting procedure. If Altair believes that a problem reported by Licensee may not be due to an error in the Software, Altair will so notify Licensee. Questions must be directed to Altair s specially designated telephone support numbers and addresses. Support will also be available via at Internet addresses designated by Altair. Support is available Monday through Friday (excluding holidays) from 8:00 a.m. to 5:00 p.m local time in the Global Zone where licensed, unless stated otherwise on the Order Form. Exclusions. Altair shall have no obligation to maintain or support (a) altered, damaged or Licensee-modified Software, or any portion of the Software incorporated with or into other software not provided by Altair; (b) any version of the Software other than the current version of the Software or the immediately prior release of the Software; (c) problems caused by Licensee s negligence, abuse or misapplication of Software other than as specified in the Documentation, or other causes beyond the reasonable control of Altair; or (d) Software installed on any hardware, operating system version or network environment that is not supported by Altair. Support also excludes configuration of hardware, non- Altair Software, and networking services; consulting services; general solution provider related services; and general computer system maintenance. 6. WARRANTY AND DISCLAIMER. Altair warrants for a period of ninety (90) days after Licensee initially receives the Software that the Software will perform under normal use substantially as described in then current Documentation. Supplier software included in the Software and ISV Software provided to Licensee shall be warranted as stated by the Supplier or the ISV. Copies of the Suppliers and ISV s terms and conditions of warranty are available on the Altair Support website. Support services shall be provided in a workmanlike and professional manner, in accordance with the prevailing standard of care for consulting support engineers at the time and place the services are performed. ALTAIR DOES NOT REPRESENT OR WARRANT THAT THE PRODUCTS WILL MEET LICENSEE S REQUIREMENTS OR THAT THEIR OPERATION WILL BE UNINTERRUPTED OR

86 86 PBS Professional Application Services A PBS Works Solution ERROR-FREE, OR THAT IT WILL BE COMPATIBLE WITH ANY PARTICULAR HARDWARE OR SOFTWARE. ALTAIR EXCLUDES AND DISCLAIMS ALL EXPRESS AND IMPLIED WARRANTIES NOT STATED HEREIN, INCLUDING THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NON-INFRINGEMENT. THE ENTIRE RISK FOR THE PERFORMANCE, NON-PERFORMANCE OR RESULTS OBTAINED FROM USE OF THE PRODUCTS RESTS WITH LICENSEE AND NOT ALTAIR. ALTAIR MAKES NO WARRANTIES WITH RESPECT TO THE ACCURACY, COMPLETENESS, FUNCTIONALITY, SAFETY, PERFORMANCE, OR ANY OTHER ASPECT OF ANY DESIGN, PROTOTYPE OR FINAL PRODUCT DEVELOPED BY LICENSEE USING THE PRODUCTS. 7. INDEMNITY. Altair will defend and indemnify, at its expense, any claim made against Licensee based on an allegation that the Software infringes a patent or copyright ( Claim ); provided, however, that this indemnification does not include claims which are based on Supplier software or ISV software, and that Licensee has not materially breached the terms of this Agreement, Licensee notifies Altair in writing within ten (10) days after Licensee first learns of the Claim; and Licensee cooperates fully in the defense of the claim. Altair shall have sole control over such defense; provided, however, that it may not enter into any settlement binding upon Licensee without Licensee s consent, which shall not be unreasonably withheld. If a Claim is made, Altair may modify the Software to avoid the alleged infringement, provided however, that such modifications do not materially diminish the Software's functionality. If such modifications are not commercially reasonable or technically possible, Altair may terminate this Agreement and refund to Licensee the prorated license fee that Licensee paid for the then current Term. Perpetual licenses shall be prorated over a 36-month term. Altair shall have no obligation under this Section 7, however, if the alleged infringement arises from Altair s compliance with specifications or instructions prescribed by Licensee, modification of the Software by Licensee, use of the Software in combination with other software not provided by Altair and which use is not specifically described in the Documentation, and if Licensee is not using the most current version of the Software, if such alleged infringement would not have occurred except for such exclusions listed here. This section 7 states Altair's entire liability to Licensee in the event a Claim is made. No indemnification is made for Supplier and/or ISV Software. 8. LIMITATION OF REMEDIES AND LIABILITY. Licensee s exclusive remedy (and Altair s sole liability) for Software that does not meet the warranty set forth in Section 6 shall be, at Altair s option, either (i) to correct the nonconforming Software within a reasonable time so that it conforms to the warranty; or (ii) to terminate this Agreement and refund to Licensee the license fees that Licensee has paid for the then current Term for the nonconforming Software; provided, however that Licensee notifies Altair of the problem in writing within the applicable Warranty Period when the problem first occurs. Any corrected Software shall be warranted in accordance with Section 6 for ninety (90) days after delivery to Licensee. The warranties hereunder are void if the Software has been improperly installed, misused, or if Licensee has violated the terms of this Agreement. Altair s entire liability for all claims arising under or related in any way to this Agreement (regardless of legal theory), shall be limited to direct damages, and shall not exceed, in the aggregate for all claims, the license and maintenance fees paid under this Agreement by Licensee in the 12 months prior to the claim on a prorated basis, except for claims under Section 7. ALTAIR AND ITS SUPPLIERS AND ISVS SHALL NOT BE LIABLE TO LICENSEE OR ANYONE ELSE FOR INDIRECT, SPECIAL, INCIDENTAL, OR CONSEQUENTIAL DAMAGES ARISING HEREUNDER (INCLUDING LOSS OF PROFITS OR DATA, DEFECTS IN DESIGN OR PRODUCTS CREATED USING THE SOFTWARE, OR ANY INJURY OR DAMAGE RESULTING FROM SUCH DEFECTS, SUFFERED BY LICENSEE OR ANY THIRD PARTY) EVEN IF ALTAIR OR ITS SUPPLIERS OR ITS ISVS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. Licensee acknowledges that it is solely responsible for the adequacy and accuracy of the input of data, including the output generated from such data, and agrees to defend, indemnify, and hold harmless

87 87 Altair and its Suppliers and ISVs from any and all claims, including reasonable attorney s fees, resulting from, or in connection with Licensee s use of the Software. No action, regardless of form, arising out of the transactions under this Agreement may be brought by either party against the other more than two (2) years after the cause of action has accrued, except for actions related to unpaid fees. 9. UNITED STATES GOVERNMENT RESTRICTED RIGHTS. This section applies to all acquisitions of the Products by or for the United States government. By accepting delivery of the Products except as provided below, the government or the party procuring the Products under government funding, hereby agrees that the Products qualify as commercial computer software as that term is used in the acquisition regulations applicable to this procurement and that the government s use and disclosure of the Products is controlled by the terms and conditions of this Agreement to the maximum extent possible. This Agreement supersedes any contrary terms or conditions in any statement of work, contract, or other document that are not required by statute or regulation. If any provision of this Agreement is unacceptable to the government, Vendor may be contacted at Altair Engineering, Inc., 1820 E. Big Beaver Road, Troy, MI ; telephone (248) If any provision of this Agreement violates applicable federal law or does not meet the government s actual, minimum needs, the government agrees to return the Products for a full refund. For procurements governed by DFARS Part (OCT 1998), the Software, except as described below, is provided with only those rights specified in this Agreement in accordance with the Rights in Commercial Computer Software or Commercial Computer Software Documentation policy at DFARS (a) (OCT 1998). For procurements other than for the Department of Defense, use, reproduction, or disclosure of the Software is subject to the restrictions set forth in this Agreement and in the Commercial Computer Software Restricted Rights FAR clause (June 1987) and any restrictions in successor regulations thereto. Portions of Altair s PBS Professional Software and Documentation are provided with RESTRICTED RIGHTS. Use, duplication, or disclosure by the Government is subject to restrictions as set forth in subdivision(c)(1)(ii) of the rights in the Technical Data and Computer Software clause in DFARS , or in subdivision (c)(1) and (2) of the Commercial Computer Software-Restricted Rights clause at 48 CFR , as applicable. 10. CHOICE OF LAW AND VENUE. This Agreement shall be governed by and construed under the laws of the state of Michigan, without regard to that state's conflict of laws principles except if the state of Michigan adopts the Uniform Computer Information Transactions Act drafted by the National Conference of Commissioners of Uniform State Laws as revised or amended as of June 30, 2002 ( UCITA ) which is specifically excluded. This Agreement shall not be governed by the United Nations Convention on Contracts for the International Sale of Goods, the application of which is expressly excluded. Each Party waives its right to a jury trial in the event of any dispute arising under or relating to this Agreement. Each party agrees that money damages may not be an adequate remedy for breach of the provisions of this Agreement, and in the event of such breach, the aggrieved party shall be entitled to seek specific performance and/or injunctive relief (without posting a bond or other security) in order to enforce or prevent any violation of this Agreement. 11. [RESERVED] 12. Notice. All notices given by one party to the other under the Agreement or these Additional Terms shall be sent by certified mail, return receipt requested, or by overnight courier, to the respective addresses set forth in this Agreement or to such other address either party has specified in writing to the other. All notices shall be deemed given upon actual receipt. Written notice shall be made to: Altair: Licensee Name & Address:

88 88 PBS Professional Application Services A PBS Works Solution Altair Engineering, Inc E. Big Beaver Rd Troy, MI Attn: Tom M. Perring Attn: 13. TERM. For annual licenses, or Support provided for perpetual licenses, renewal shall be automatic for each successive year ( Renewal Term ), upon mutual written execution of a new Order Form. All charges and fees for each Renewal Term shall be set forth in the Order Form executed for each Renewal Term. All Software licenses procured by Licensee may be made coterminous at the written request of Licensee and the consent of Altair. 14. TERMINATION. Either party may terminate this Agreement upon thirty (30) days prior written notice upon the occurrence of a default or material breach by the other party of its obligations under this Agreement (except for a breach by Altair of the warranty set forth in Section 8 for which a remedy is provided under Section 10; or a breach by Licensee of Section 5 or Section 6 for which no cure period is provided and Altair may terminate this Agreement immediately) if such default or breach continues for more than thirty (30) days after receipt of such notice. Upon termination of this Agreement, Licensee must cease using the Software and, at Altair s option, return all copies to Altair, or certify it has destroyed all such copies of the Software and Documentation. 15. GENERAL PROVISIONS. Export Controls. Licensee acknowledges that the Products may be subject to the export control laws and regulations of the United States and other countries, and any amendments thereof. Licensee agrees that Licensee will not directly or indirectly export the Products into any country or use the Products in any manner except in compliance with all applicable U.S. and other countries export laws and regulations. Notice. All notices given by one party to the other under this Agreement shall be sent by certified mail, return receipt requested, or by overnight courier, to the respective addresses set forth in this Agreement or to such other address either party has specified in writing to the other. All notices shall be deemed given upon actual receipt. Assignment. Neither party shall assign this Agreement without the prior written consent of other party, which shall not be unreasonably withheld. All terms and conditions of this Agreement shall be binding upon and inure to the benefit of the parties hereto and their respective successors and permitted assigns. Waiver. The failure of a party to enforce at any time any of the provisions of this Agreement shall not be construed to be a waiver of the right of the party thereafter to enforce any such provisions. Severability. If any provision of this Agreement is found void and unenforceable, such provision shall be interpreted so as to best accomplish the intent of the parties within the limits of applicable law, and all remaining provisions shall continue to be valid and enforceable. Headings. The section headings contained in this Agreement are for convenience only and shall not be of any effect in constructing the meanings of the Sections. Modification. No change or modification of this Agreement will be valid unless it is in writing and is signed by a duly authorized representative of each party. Conflict. In the event of any conflict between the terms of this Agreement and any terms and conditions on a Licensee Purchase Order or comparable document, the terms of this Agreement shall prevail. Moreover, each party agrees any additional terms on any Purchase Order or comparable document other than the transaction items of (a) item (s) ordered; (b) pricing; (c) quantity; (d) delivery instructions and (e) invoicing directions, are not binding on the parties. In the event of a conflict between the terms of this Agreement, and the Additional Terms, the Agreement shall take precedence. Entire Agreement. This Agreement, the Additional Terms, and the Order Form(s) attached hereto constitute the entire understanding between the parties related to the subject matter hereto, and supersedes all proposals or prior agreements, whether written or oral, and all other communications between the parties with respect to such subject matter. This Agreement may be executed in one or more counterparts, all of which together shall constitute one and the same instrument. Execution. Copies of this Agreement

89 executed via original signatures, facsimile or shall be deemed binding on the parties. 89

90 90 PBS Professional Application Services A PBS Works Solution Index -I- -A- Input File 45 Install 9 Installation 8 Add Application Definitions 41 Administration of Application Definitions 2 Application Converter File 53 Application Definitions 36 Application Definitions - Removing 44 Application Definitions - Updating 43 Application Input File 45 Application Runtime Scripts 66 Argument Choice 47 ArgumentDirectoryName 52 ArgumentFileName 51 ArgumentInt 50 Arguments 47 ArgumentString Bboolean -JJob Submission - Data Staging 60 Job Submission - Environment 55 Job Wide and Chunk Resources 57 -LLicense Agreement 82 -OOverview of Application Definition Components 37 -R62 -Cchunk 57 Conditional Expressions in Application Definitions 62 Configuration 32 Converter File 53 copyright 4 Custom Runtime Action 72 -DData Staging Remove Application Definitions 44 Reserved Words 76 Resources 56 Restart Runtime Action 69 Runtime Actions 66 Runtime Actions - Common UNIX Example Runtime Scripts SSitewide Settings 38 Start Runtime Action 67 Supported Platforms 5 System Requirements T- -EEnvironment 55 Epilogue Runtime Action Example - JLaunch Terminate Runtime Action third party licenses 4 trademark 4 Troubleshooting 77 70

91 Index -UUninstall 28 Update Application Definitions Using Site Policies