CA AutoSys Workload. Troubleshooting

Transcription

1 CA AutoSys Workload Automation r11 Troubleshooting Presenters: Dan Shannon, Elizabeth Dexter

2 Terms of This Presentation This presentation was based on current information and resource allocations as of October 2009 and is subject to change or withdrawal by CA at any time without notice. Notwithstanding anything in this presentation to the contrary, this presentation shall not serve to (i) affect the rights and/or obligations of CA or its licensees under any existing or future written license agreement or services agreement relating to any CA software product; or (ii) amend any product documentation or specifications for any CA software product. The development, release and timing of any features or functionality described in this presentation remain at CA s sole discretion. Notwithstanding anything in this presentation to the contrary, upon the general availability of any future CA product release referenced in this presentation, CA will make such release available (i) for sale to new licensees of such product; and (ii) to existing licensees of such product on a when and if-available basis as part of CA maintenance and support, and in the form of a regularly scheduled major product release. Such releases may be made available to current licensees of such product who are current subscribers to CA maintenance and support on a when and if-available basis. In the event of a conflict between the terms of this paragraph and any other information contained in this presentation, the terms of this paragraph shall govern. 2 October 2009 [CA AutoSys Workload Automation r11 Troubleshooting] Copyright 2009 CA

3 For Informational Purposes Only Certain information in this presentation may outline CA s general product direction. All information in this presentation is for your informational purposes only and may not be incorporated into any contract. CA assumes no responsibility for the accuracy or completeness of the information. To the extent permitted by applicable law, CA provides this document as is without warranty of any kind, including without limitation, any implied warranties or merchantability, fitness for a particular purpose, or non-infringement. In no event will CA be liable for any loss or damage, direct or indirect, from the use of this document, including, without limitation, lost profits, lost investment, business interruption, goodwill, or lost data, even if CA is expressly advised of the possibility of such damages. 3 October 2009 [CA AutoSys Workload Automation r11 Troubleshooting] Copyright 2009 CA

4 Agenda: >Preparing for the Installation >Installation and Initial Configuration >Customisation and Tuning >Common Issues 4 October 2009 [CA AutoSys Workload Automation r11 Troubleshooting] Copyright 2009 CA

5 Preparing for Installation >Architecture What are you going to install, and where? What are you planning to support? >System Machine sizing i > OS requirements Patches, libraries > Networking Can the machines communicate? 5 October 2009 [CA AutoSys Workload Automation r11 Troubleshooting] Copyright 2009 CA

6 Installation > Verify that the common component installs were successful > Verify that you can communicate between machines 6 October 2009 [CA AutoSys Workload Automation r11 Troubleshooting] Copyright 2009 CA

8 WCC Ports Services Related Applications Start Stop SSL Launcher Server: WCC Main WCC Main Login Web Services Web Services Samples Application Credential Application Configuration Server: Configuration Event Server: Event Console Quick Edit Job Status Console Servant: Job Status Console Server: Job Status Console Job Status Views Monitoring Server: Job Flow Design Job Flow Monitoring Critical Path Monitoring Scheduling Server: Job Editor Job Editor Plus (BLOB editor) Application Editor UI Framework Server: Quick View Quick Start Enterprise Command Line HA Server: High Availability Reporting Collectors October 2009 [CA AutoSys Workload Automation r11 Troubleshooting] Copyright 2009 CA

9 Installation and Initial Configuration > Installing the Common Components SSA EEM CCS/Event Management > Connectivity it EEM AutoSys/WCC/SSA / > Multicast (WCC) 9 October 2009 [CA AutoSys Workload Automation r11 Troubleshooting] Copyright 2009 CA

10 Common Issues > Collecting Information for Support Release, maintenance level, logs, system information > SSA Issues > EEM Issues > AutoSys Issues > WCC Issues > Multiple NIC cards > Command Sponsor is not installed 10 October 2009 [CA AutoSys Workload Automation r11 Troubleshooting] Copyright 2009 CA

12 CA Workload Automation AutoSys Logging > ISDBGACTIV >Log Files univagent.out as_server.ins event_demon.ins auto_rem.joid.runnum 12 October 2009 [CA AutoSys Workload Automation r11 Troubleshooting] Copyright 2009 CA

13 EEM Logging (SDK) > New With SDK 8.4 SR01 >EIAMCONFIG Environment > Configuration Files eiam.config logger.config 13 October 2009 [CA AutoSys Workload Automation r11 Troubleshooting] Copyright 2009 CA

14 WCC Logging > WCC Logging > Location: <WCC Install Root> Logs folder Uses log4j > Tomcat Logs Access (not currently used) Application Admin, config, event, haserver, jobcontrolservant, jobcontrolserver, launcher, monitoring, reporting, uiframework Log4j.properties Install Services <WCC Install Root>/xxx/logs 14 October 2009 [CA AutoSys Workload Automation r11 Troubleshooting] Copyright 2009 CA

17 Directory Layout /opt/c A /U nicentera utos ysjm $ A U T O U S E R $AU TO SYS o u t a r c h iv e b in a g e n t o u t lib IN G tx jo b s t d b o b j snm p O R A S Y B d o c m a n 8 r e q u e s t r e s p o n s e in s ta ll d a ta m e s s a g e s b in C o n fig u ra tio n F ile s : * c o n f ig. $ A U T O S E R V * autosys.sh.host * a u t o s y s. csh. h o s t * a u t o s y s. k s h. h o s t * a u t o s y s. b a s h. h o s t e t c. te s t b in_b a k... lib_b a k.. jil d e m o n ite ly O utthed oor 17 October 2009 [CA AutoSys Workload Automation r11 Troubleshooting] Copyright 2009 CA

23 AutoSys Communication Problems > CAUAJM_E_10029 Communication attempt with the Unicenter AutoSys JM Application Server has failed! [host:port]. Can you ping the App Server host? Is the App Server running? Are AutoServer/AutoServerPort values correct? Is port 7163 (SSA) blocked by firewall? Are SSA port configurations the same on client & App Server machines the same? EnablePmux EnableSSL 23 October 2009 [CA AutoSys Workload Automation r11 Troubleshooting] Copyright 2009 CA

24 AutoSys Communication Problems > CAUAJM_E_10527 Timed out waiting for response from the Unicenter AutoSys JM Application Server [host:port] Is the App Server running? Are there messages in the App Server log file? ISDBGACTIV=LIGHT,LOG Is the SSA PortRange for configured correctly? EnablePmux EnableSSL Do the App Server and RDBMS machines have adequate resources? Does increasing DB_CONNECTIONS reduce problem? 24 October 2009 [CA AutoSys Workload Automation r11 Troubleshooting] Copyright 2009 CA

25 Troubleshooting the Scheduler Job stuck in STARTING > Did Scheduler complete the Agent handshake? CAUAJM_I_10082 [POLAL02-GX280 connected for job ] > What does corresponding Job Agent log file show? CAUAJM_E_10527 Timed out waiting for response from the Unicenter AutoSys JM Application Server. CAUAJM_E_10221 Exhausted list of application servers. Failing request. 25 October 2009 [CA AutoSys Workload Automation r11 Troubleshooting] Copyright 2009 CA

26 Troubleshooting the Agent CAUAJM_E_50026 ERROR: AutoPing WAS NOT SUCCESSFUL! > Does univagent log file show activity under ISDBGACTIV=LIGHT,JOB traces? > Do OS network tools reach Agent machine? (ping, nslookup, tracert/traceroute) > Is firewall blocking SSA physical port 7163? > Do SSA settings for AutoServerPort match (autoping D)? > Do SSA settings for PortRange= match? EnablePmux EnableSSL 26 October 2009 [CA AutoSys Workload Automation r11 Troubleshooting] Copyright 2009 CA

27 EEM Troubleshooting > Performance Issues Check logging levels Active Directory connectivity Linux kernel Cache settings > Unexpected permissions errors Check for mixed case directory entries > Active Directory Group Retrieval Filters 27 October 2009 [CA AutoSys Workload Automation r11 Troubleshooting] Copyright 2009 CA

28 EEM Troubleshooting CAUAJM_E_10448 FATAL SECURITY ERROR: Tampering or corruption of the security-related database keys detected > Did someone install a new instance using an MDB from a previous instance installation? > Did someone directly modify the ujo_alamode or ujo_keymaster keys? > What is the state of the database tables? Did some unknown corruption of keys take place? 28 October 2009 [CA AutoSys Workload Automation r11 Troubleshooting] Copyright 2009 CA

29 EEM Troubleshooting CAUAJM_E_10436 Security server unreachable or invalid authentication certificate file > Is the EEM backend running? > Do OS network tools reach EEM backend machine? (ping, nslookup, tracert/traceroute) > Is firewall blocking EEM physical port 5250? > Was the proper EEM backend host name entered? > Are you able to connect to the EEM backend via the as_safetool? safetool? > Does the AUTOUSER folder contain a valid certificate file? 29 October 2009 [CA AutoSys Workload Automation r11 Troubleshooting] Copyright 2009 CA

30 EEM Troubleshooting Policy problems > Does the EEM web UI Permission Check work as expected? > Does autosec_test test work as expected? > Does as_safetool work? 30 October 2009 [CA AutoSys Workload Automation r11 Troubleshooting] Copyright 2009 CA

31 WCC - Other > Job Flow Monitoring Monitor ID Effective limit on the number of jobs in a flow > Job Status Console Global l Session/Global l ID Prior to WCC r11.1 SP1, check alarms/alerts filters > Unexpected login prompts IAM Security setting in server definition 31 October 2009 [CA AutoSys Workload Automation r11 Troubleshooting] Copyright 2009 CA

32 Command Sponsor Troubleshooting > Required for Quick View and ECLI in WCC >Requires a credential user that is authorised on the OS to execute the command > Restricted to the commands in the AutoSysCommandISponsorFilters.txt txt file > Uses igateway 32 October 2009 [CA AutoSys Workload Automation r11 Troubleshooting] Copyright 2009 CA