halcyon software guide to templates

Transcription

1 halcyon software guide to templates covering: Windows, AIX, Linux, IBM i and customizations

2 Copyright Copyright Halcyon Software Limited. All rights reserved. IBM, iseries, Power/System i, IBM i and i5/os are trademarks of International Business Machines. Windows is a registered trademark of Microsoft Corporation. All other trademarks are respective of their own companies. This document is intended as a guide to the reports generated by using Halcyon software. This documentation contains Halcyon proprietary and confidential information and may not be disclosed, used, or copied without the prior consent of Halcyon Software Limited, or as set forth in the applicable license agreement. Users are solely responsible for the proper use of the software and the application of the results obtained. Although Halcyon Software Limited has tested the software and reviewed the documentation, the sole warranty for the software may be found in the applicable license agreement between Halcyon Software Limited and the user. Publication Revision: May 2012 Company Headquarters Head Office Halcyon Software Limited 5 The Forum Minerva Business Park Lynch Wood Peterborough Cambridgeshire United Kingdom PE2 6FT Tel: +44 (0) For details of your local office or reseller, please visit and access the Contact Us, Support or Partners options. You may also us directly using the following addresses: Technical Support:support@halcyonsoftware.com Sales: sales@halcyonsoftware.com Accounts: accounts@halcyonsoftware.com

3 Table of Contents Copyright Company Headquarters Introduction 1 Network Server Suite Templates 3 Using the default templates Windows templates Active Directory template Exchange Server (Performance) template... 6 Exchange Server (Services) template M3 Monitoring M3 Grid Monitoring template M3 Monitoring template Web Application Monitor Service Monitor TCP HTTP Monitor Server Performance (Advanced) template.. 16 Server Performance (Standard) template.. 18 SQL Server template Terminal Services template AIX Templates AIX System Monitoring (Standard) Template23 AIX Error Report Monitor AIX Subsystem Report Monitor Logical Volume Monitor Script Monitor AIX File & Folder Monitor Log File Monitor CPU, Filesystem and Memory Monitor.. 26 AIX Process Monitor AIX Ping Monitor AIX System Monitoring (Advanced) Template AIX Error Report Monitor AIX Subsystem Monitor AIX Logical Volume Monitor Script Monitor AIX File & Folder Monitor Log File Monitor CPU, Filesystem and Memory Monitor.. 34 Process Monitor Linux Templates RED HAT System Monitoring Template (Standard) Linux Logical Volume Monitor Script Monitor File & Folder Monitor Log File Monitor CPU, Filesystem and Memory Monitor.. 38 Process Monitor Ping Monitor RED HAT System Monitoring Template (Advanced) Linux Logical Volume Monitor Script Monitor File & Folder Monitor Log File Monitor CPU, Filesystem and Memory Monitor.. 42 Process Monitor Ping Monitor SUSE System Monitoring Template (Standard) Log File Monitor SUSE System Monitoring Template (Advanced) Windows Reporting Templates Power/System i Rules 47 Overview Message Queue Rules Example Message Queue Rules No Global Exclusions Defined FTP Activity monitoring Queue Rule Group System History Log System Critical Messages Queue QSYSOPR Message Queue TCP/IP Rules Example TCP/IP Rules Output Queue Rules Example Output Queue Rules Device Rules Example Device Rules Job Queue Rules Example Job Queue Rules Performance Rules Example Performance Rules User Profile Rules Example User Profile Rules Distribution Queue Rules Example Distribution Queue Rules Inactivity Rules

4 Example Inactivity Rules Object Rules Example Object Rules Restricted Task Rules Example Restricted Task Rules Audit Journal Rules Authority Failure Audit Journal Rules Example Audit Journal Rules Security Audit Rules Service Audit Rules SYSMGT Audit Rules Specialist Templates 76 Overview Installation of Customized Environment SAP Customization Template Message Queue Rules QHST Message Queue QSYSOPR Message Queue Device Rules Performance Rules JDE Customization Template Message Queue Rules JDE World Messages Output Queue Rules JDEPRT01 Output Queue Rules Device Rules JDE Rules Job Queue Rules JDE Daytime JOBQ JDE Nighttime JOBQ Performance Rules JDE Rules User Rules JDE Rules Inactivity Rules JDE Rules Object Rules JDE Rules Audit Journal Rules JDE Rules JDE Windows monitoring HTTP Monitor Service Monitor Event Log Monitor CPU, Disk and Memory Monitor File and Folder Monitor Ping Monitor Generic TCP/UDP monitor Lawson Movex(M3) Templates Message Queue Rules QHST Message Queue TCP/IP Rules M Performance Rules M Lawson Movex(M3) Windows monitoring.. 90 MAXAVA (NOMAX) customization templates.. 90 Message Queue Rules QSYS MAXAVA messages QSYSOPR QSYS Qsysopr Message Queue TCP/IP Rules MAXAVA Check all partitions are available 91 Performance Rules MAXAVA MAXAVA alerts and monitoring. 92 Quick-EDD customization templates Additional Installation Routine Message Rules PMEDH PMEDHUSR QuickEDD Message Monitoring TCP/IP Rules QSL Source/Target System Halcyon System Changes JOBD User Authority Halcyon System Defaults MISYS customization templates Message Queue Rules KAPSTATUS End of Day Monitoring QSYSOPR Qsysopr Message Queue Performance Group Rules EQUATION MISYS Equation Rules Object Group Rules EQUATION MISYS Equation Rules Misys Midas Plus Customization Templates Message Queue Rules KAPSTATUS Midas Plus End of Day Monitoring MOPERQ Midas Plus QSYSOPR Qsysopr Message Queue TCP/IP Group Rules MIDAS Plus Windows Servers Active Performance Group Rules MIDAS PLUS MIDAS Plus Rules User Group Rules MIDAS PLUS Midas Plus Rules WebSphere MQ Monitoring 113 MQ Monitoring Record & Playback Scripts

5 CHKMQM (WebSphere MQ Manager Checking) Script CHKMQMCHL (Active MQ Channels Running) Script CHKMQMCHLG (ALL MQ Channels Running) Script CHKMQMQ (Message Queue Depth) Script Setting MQ Monitoring Rules Creating the MQ Message Rule Adding Actions Creating Performance Rules Checking the WebSphere MQ Subsystem is active Checking MQ Listener Jobs Checking WebSphere MQ Jobs Scheduling the Running of the Record & Playback Scripts Creating a TCP/IP rule to schedule the Record & Playback scripts Checking/Editing CL Source and Message File 128 Viewing the CL Source Files Viewing the Message File Template Assignment 132 Overview Sample Form Message Queue Example Rules 134 Overview FTP Activity Monitoring Queue QHST System History Log System Critical Messages Queue QSYSOPR Message Queue TCP/IP Example Rules 142 Output Queue Example Rules 143 Device Example Rules 144 Job Queue Example Rules 145 Performance Example Rules 146 User Profile Example Rules 149 Distribution Queue Example Rules 150 Inactivity Example Rules 151 Object Example Rules 152 Restricted Tasks Example Rules 154 Audit Journal Rules 155 Network Server Suite Windows Templates 172 Active Directory (Monitoring & Performance) Template Exchange Server (Performance) Template. 173 Exchange Server (Services) Template M3 Grid Monitoring TCP HTTP Monitor M3 Monitoring Template Web Application Monitor Service Monitor TCP HTTP Monitor Server Performance (Advanced) Template. 179 Server Performance (Standard) Template SQL Server Template Terminal Services template AIX Templates 184 AIX System Monitoring (Standard) Template 184 AIX System Monitoring (Advanced) Template Linux Templates 193 RED HAT System Monitoring Template (Standard) RED HAT System Monitoring Template (Advanced) SUSE System Monitoring Template (Standard) SUSE System Monitoring Template (Advanced) Specialist Templates- SAP Customization 199 Message Queue Rules Device Group Rules Performance Group Rules Specialist Templates - JDE Customization 203 Message Queue Rules Output Queue Rules Device Group Rules Job Queue Rules Performance Rules User Rules Inactivity Rules Object Rules Audit Journal Rules Specialist Templates - Lawson M3/Movex Customization 209 Message Queue Rules TCP/IP Rules

6 Performance Rules Specialist Templates - MAXAVA (NOMAX) Customization 214 Message Queue Rules TCP/IP Rules Performance Rules Specialist Templates - Quick/EDD Customization 218 Message Queue Rules TCP/IP Rules Specialist Templates - Misys Customization 220 Message Queue Rules Performance Group Rules Object Group Rules Specialist Templates - Misys Midas Plus Customizations 227 Message Queue Rules TCP/IP Group Rules Performance Group Rules User Group Rules

7 C HAPTER CHAPTER 1INTRODUCTION Halcyon Templates are designed to provide the same level of monitoring across a number of similar devices by applying a set of user-defined filters with a single-click. This greatly reduces set-up time and ensures all systems are covered by at least a basic level of monitoring. Should you need to make a system-wide change at a later date, a single update covers all systems using the template. Halcyon Templates are available in the following products: Network Server Suite Windows AIX Linux RED HAT SUSE IBM i Level 1 - Message Management Suite Level 2 - Systems Operations Suite Level 3 - Advanced Automation Suite Level 4 - Operations Center Suite Customization Templates SAP JDE Lawson M3 (Movex) Maxava (NOMAX) Misys Misys Midas Plus Quick-EDD Introduction 1-1

8 Note: IMPORTANT! All of the templates contained within this User Reference are examples only. Once installed into your environment, please review each rule and if necessary, amend with local configuration data, such as: Times of Operation Specific Machine Identification Number of occurrences before alerting Where appropriate actions are taken, also remember to configure: Addresses Mobile Phone numbers Escalation polices 1-2 Halcyon Templates

9 C HAPTER 2NETWORK SERVER SUITE TEMPLATES CHAPTER Network Server Suite comes supplied with seven default monitoring templates for Windows that cover the majority of everyday scenarios that your organization is likely to encounter. Additionally, there are two pre-configured templates that can be applied to AIX devices and two templates each to cover the RED HAT and SUSE versions of the Linux operating system. Templates are created using the Central Configuration Manager and can then be quickly applied to all systems. More than one template can be applied to a system at any one time and it is also possible to have individual filters running alongside the template filters on any system. Basic templates which monitor devices for routine issues and concerns such as low disk space, memory and so on can be deployed enterprise-wide. Business critical machines may require the application of an advanced template additionally covering, for example, application event log and service monitoring. A basic reporting template covering the five key performance measurements is shipped with Network Server Suite as standard. Using the default templates Launch the Central Configuration Manager and select the Templates tab. The following Windows monitor templates are defined: Active Directory Exchange Server (Performance) Exchange Server (Services) M3 Grid and M3 Monitoring Server Performance (Advanced) Server Performance (Standard) SQL Server Terminal Services Network Server Suite Templates 2-3

10 The following AIX templates are defined: System Monitoring (Advanced) System Monitoring (Standard) The following Linux templates are defined: RED HAT System Monitoring (Advanced) RED HAT System Monitoring (Standard) SUSE System Monitoring (Advanced) SUSE System Monitoring (Standard) The Reporting template: Reporting (Performance) is also included. Figure 2.1 Templates dialog 2-4 Halcyon Templates

11 Windows templates Active Directory template Active Directory is a centralized and standardized system that automates network management of user data, security, and distributed resources, and enables inter-operation with other directories. Active Directory is designed especially for distributed networking environments. The Active Directory template contains seven generic performance filters: Counter(DRA Inbound Bytes Total/sec) Trigger(>=100) This counter shows the number of bytes received through inbound Active Directory related replication. If this number is consistently equal to zero, it means that replication is not occurring. Low numbers may indicate that there is a network bottleneck or that the server's NIC is too busy with other traffic to receive the requests in a timely manner. If the number is too high, it may indicate that the server cannot cope with the volume of requests. This filter generates an alert if the number of DRA Inbound bytes received is equal to or greater than 100. Counter(DRA Inbound Object Updates Remaining in Packet) Trigger(>=5) This counter displays the number of Active Directory objects that have been received through replication, but that have not yet been applied. This number may start high, but should diminish very quickly. If this value takes a while to diminish, it is a clue that the server's hardware might not be fast enough to keep up with the demand. This filter generates an alert if the number of objects remaining in packet is greater or equal to five. Counter(DRA Outbound Bytes Total/sec) Trigger(>=1,024,000) This counter displays the total number of bytes (compressed and uncompressed) that are being transmitted each second as a result of the replication process. This filter generates an alert if the number of DRA Outbound Bytes is equal or greater than 1,024,000 in any one second. Counter(DRA Pending Replication Synchronizations) Trigger(>=50) This number indicates the number of objects which must be synchronized. As with the DRA Inbound Object Updated Remaining in Packet counter, this value may start high, but should quickly fall. If this counter's value remains high, it usually means that the hardware is having trouble keeping pace with the demands under which it is operating. This filter generates an alert if the number of DRA Pending Replication Synchronizations is greater than or equal to 50. Network Server Suite Templates 2-5

12 Counter(DS Name Cache hit rate) Trigger(<=99) Shows the percentage of directory object name component lookups that are satisfied out of the DSA's name cache. An alert is generated if this percentage falls below 99 percent. Counter(LDAP Client Sessions) Trigger(>=250) This figure indicates the number of Lightweight Directory Access Protocol (LDAP) sessions that are connected to the domain controller at the present time. This filter monitors the current number of LDAP client sessions and generates an alert if this figure is greater than or equal to 250. Note: Counter(NTLM Authentications) Trigger(>=10,000) Shows the number of NT Lan Manager (NTLM) authentications per second serviced by this domain controller. This filter monitors the number of NTLM Authentications per second and generates an alert if this figure is greater than or equal to 10,000. All actions for each of the above filters within this template are set to a default of sending an alert to the Enterprise Console. You must manually change this setting if you require an alternative action to be taken upon the generation of an alert. Exchange Server (Performance) template This template covers the performance of the Exchange Server, including mail server, client and groupware applications (such as shared calendars). The Exchange Server (Performance) template contains six generic performance filters: IS - Counter(Active User Count) Trigger(>=200) Sometimes the number of connected users can be deceptive when measuring the server s workload. This counter only includes the users who have actually done something Exchange-related in the past 10 minutes, therefore registering the number of active users on the Exchange Server. If the number of active users is equal to or exceeds 200, an alert is generated by this filter. IS Public - Instance(_Total) Counter(Send Queue Size) Trigger(>=10) This counter measures the number of messages that are waiting to be sent. This number can be greater than zero during periods of heavy activity, but it should return to zero quickly after the period of heavy use has passed. If it does not it may indicate that the SMTP service is down. This filter generates an alert if the size of the Send Queue is greater than or equal to 10 messages waiting to be sent. 2-6 Halcyon Templates

13 Mailbox - Instance(_Total) Counter(Average Delivery Time) Trigger (=10,000) This counter provides an indication of how long it usually takes the public information store to deliver messages. This filter monitors the delivery time and generates an alert if the average delivery time is equal to 10,000 milliseconds. Mailbox - Instance(_Total) Counter(Send Queue Size) Trigger(>=20) This counter measures the number of messages that are waiting to be sent. This number can be greater than zero during periods of heavy activity, but it should return to zero quickly after the period of heavy use has passed. This filter generates an alert if the size of the Send Queue is greater than or equal to 20 messages waiting to be sent. MTA - Counter(Messages/Sec) Trigger(>=10) This counter tallies the number of messages that flow through the Mail Transfer Agent (MTA) each second. This is a good method for measuring the server s overall workload. This filter monitors the number of messages being processed through the MTA and generates an alert if the figure is greater than or equal to 10 messages per second. MTA - Counter(Work Queue Length) Trigger(>=10) This counter measures the number of messages that are collectively waiting for delivery. It looks at the number of messages in the MTA queue rather than basing its results on the private or the public information store queue. This filter generates an alert if the work queue length is greater than or equal to 10 messages waiting for delivery. Note: All actions for each of the above filters within this template are set to a default of sending an alert to the Enterprise Console. You must manually change this setting if you require an alternative action to be taken upon the generation of an alert. Exchange Server (Services) template Microsoft Exchange Server is a client/server messaging system in which active server processes interact with client processes. You can view these server processes in the Services tool, found within the Administrative Tools program group of Windows. Most Exchange Server services have a name that starts with Microsoft Exchange. This template covers the main services used by Exchange Server. The Exchange Server (Services) template contains seven system service filters and would be deployed on your Exchange Server. Network Server Suite Templates 2-7

14 Service(Microsoft Exchange IMAP4) Status(<> Started ) Startup(= Automatic) Logon A/C(= Local System ) Microsoft Exchange IMAP4 service provides internet message access protocol 4 services to clients. If this service is stopped clients are unable to connect to the computer using IMAP4. This filter generates an alert if the service is found to be in any status other than started. Service(Microsoft Exchange Information Store) Status(<> Started ) Startup(= Automatic ) Logon A/C(= Local System ) The Information Store is the most critical component of Microsoft Exchange Server as it manages the Exchange store. The service makes mailbox stores and public folder stores available. If this service is stopped, mailbox stores and public folder stores on this computer are unavailable. If this service is disabled, any services that explicitly depend on it cannot start. The Microsoft Exchange System Attendant service is dependent on the Information Store service running. This filter generates an alert if the service is found to be in any status other than started. Service(Microsoft Exchange Management) Status(<> Started ) Startup (= Automatic ) Logon A/C(= Local System ) The purpose of this service is to run Directory Service Access (DsAccess) and Message Tracking. If the service is stopped or will not start, neither the DsAccess interface nor the Message Tracking Center will work. This filter generates an alert if the service is found to be in any status other than started. Service(Microsoft MTA Stacks) Status(<> Started ) Startup(= Automatic ) Logon A/C(= Local System ) The Message Transfer Agent (MTA) Stacks service is primarily used to provide backward-compatible message transfer services between Exchange 2000 Server or Exchange Server 2003 servers and Microsoft Exchange Server 5.5 servers in a mixed-mode environment. This filter generates an alert if the service is found to be in any status other than started. Service(Microsoft Exchange POP3) Status(<> Started ) Startup (= Automatic ) Logon A/C(= Local System ) This Microsoft Exchange Service provides Post Office Protocol version 3 (POP3) services to clients. If this service is stopped, clients cannot connect to this computer using POP3. This filter generates an alert if the service is found to be in any status other than started. 2-8 Halcyon Templates

15 Service(Microsoft Exchange Routing Engine) Status(<> Started ) Startup (= Automatic ) Logon A/C(= Local System ) The Microsoft Exchange Routing engine, a service in the Default Exchange Services, is responsible for determining the least expensive available path for message delivery. It supplies this information to the advanced queuing engine as part of the message delivery process. This filter generates an alert if the service is found to be in any status other than started. Note: Service(Microsoft System Attendant) Status(<> Started ) Startup (= Automatic Logon A/C(= Local System ) The Microsoft Exchange System Attendant service is primarily a collection of subcomponents that work together to proxy Active Directory requests and to regulate internal Exchange Server functions. It is critical to Exchange Server's performance, since many Exchange-related services will not function without it. This filter generates an alert if the service is found to be in any status other than started. All actions for each of the above filters within this template are set to a default of sending an alert to the Enterprise Console. You must manually change this setting if you require an alternative action to be taken upon the generation of an alert. M3 Monitoring Lawson M3 (Movex) is an ERP application capable of running on a multi-tiered platform, typically IBM i and Windows. Using Network Server Suite v8.0 upwards, the following Lawson M3 Windows components can be monitored using the M3 Grid and M3 Monitoring templates. M3 Grid Monitoring template The Lawson M3 Grid provides a common point of entry for all Intel and IBM i based server and systems operating within the M3 Application. The Lawson Grid is a distributed runtime architecture that supports rapid deployment of Lawson Appliances running as Java Virtual Machines (JVMs) A typical view of an M3 Grid main information panel is shown overleaf. Although this grid display provides a top-level view of the Lawson applications, the template monitoring is performed on the underlying information behind the main display. A typical view of this information is shown in the second screen shot displayed overleaf. Network Server Suite Templates 2-9

16 Figure 2.2 Typical M3 Grid Main Information Panel Figure 2.3 M3 Grid Monitoring Details 2-10 Halcyon Templates

17 The M3 Grid Monitoring template contains the following eleven filters, all contained within the TCP HTTP Monitor: Note: Filters marked with an asterisk * are only available in XML v1.0 Check CONNECT Site - URL( Timeout(5 Secs) Include( News ) NotInclude( 404 ) This filter checks that the Lawson M3 News site is available. To pass the filter check, the page to which this filter connects, must contain the word 'News' and must not contain '404 (the numerical error ID code for Page Not Found errors). If 'News' is not found or '404' is present, a message is sent to the local Enterprise Console. Check Jvm locked - URL( =jvms) Timeout(5 Secs) NotInclude( status= locked ) Within the Monitoring Details, this filter checks within M3 Grid monitor category JVMS to ensure that the M3 Coordinator program is running successfully, is not locked and is in a status of started. An alert is triggered should this program be locked and a message sent to the local Enterprise Console. Checking for looping M3 Auto Job - URL( monitor?category=news) Timeout(5 Secs) NotInclude( Job may be looping ) This filter checks to ensure that none of the M3 auto jobs (identified in M3 Grid monitor category News ), that operate via background batch processing are looping, preventing any subsequent jobs from running. An alert is raised on the first matching instance being found, sending a message to the local Enterprise Console. High Severity for News page - URL( monitor?category=news) Timeout(5 Secs) This filter triggers an alert whenever a high severity issue is reported within the M3 NEWS page, as monitored in the M3 Grid monitor category News. A message is sent to the local Enterprise Console. *M3 Excessive Interactive Job CPU - URL( monitor?category=interactivejobs) Timeout(5 Secs) This filter monitors for high CPU usage (greater than 80%) against any M3 interactive job logged within the M3 Grid monitoring category interactive jobs. An alert is raised for any instance being found, sending a message to the local Enterprise Console. Network Server Suite Templates 2-11

18 M3 Excessive Subsystem CPU - URL( monitor?category=counters) Timeout(5 Secs) This filter monitors for high CPU usage (greater than 75%) against any M3 subsystem logged within the M3 Grid monitoring category counters. An alert is raised for any instance being found, sending a message to the local Enterprise Console. Supervisor Status = Critical - URL( category=status) Timeout(5 Secs) NotInclude ( supervisorstatus= critical ) This filter monitors for the Supervisor interactive M3 job within the M3 Grid monitoring category status and triggers an alert if a status of critical is returned. The alert message is sent to the Enterprise Console. Transaction Server Check - URL( monitor?category=services) Timeout(5 Secs) NotInclude( ) This filter monitors for the Transaction Server within the M3 Grid monitoring category services and triggers an alert if the server is not active. The alert message is sent to the Enterprise Console. XML Autojobs count - URL( monitor?category=autojobs) Timeout(5 Secs) Include( Autojobs jobs currently running in the system count=52 ) This filter ensures that all of your M3 autojobs are active.the M3 Grid monitoring category autojobs is checked to ensure that the system count of autojobs is equal to 52. (This number can be amended by editing the filter). An alert is raised if number of autojobs fails to equal 52 and a message is sent to the Enterprise Console. Note: XML Autojobs List - URL( =autojobs) Timeout(5 Secs) This filter works in the same way in the previous filter in that it ensures that all of your M3 autojobs are active, but in this instance, the M3 Grid monitoring category autojobs is checked against a list of autojobs rather than a system count. (The list can be amended by editing the filter). An alert is raised if an autojob is missing from the provided list and a message is sent to the Enterprise Console. You only need to use one of the previous two filters as they both achieve the same end result Halcyon Templates

19 *XML Job Queue Length >25 - URL( monitor?category=status) Timeout(5 Secs) Include(OneOf jobqueuelength= 0 to 25 ) This filter checks the M3 Grid monitoring category status to ensure that there are no M3 job queues with a length of more than 25 present. An alert is raised if an M3 job queue exceeds 25 and is sent to the Enterprise Console. M3 Monitoring template Note: The M3 Monitoring template includes monitors for Lawson M3 applications and uses Web Application Monitor, Service and TCP HTTP monitoring components. If you have not purchased the Web Application Monitor, this template only includes the Service and TCP HTTP elements. The M3 Monitoring template contains the following components: Web Application Monitor Checking for looping M3 auto job - URL( showlog?addr= &port6101) Timeout(5 Secs) TriggerType(first Matching) TriggerObject(Table) This filter checks to ensure that non of the M3 auto jobs, that operate via background batch processing, are looping, preventing any subsequent jobs from running. An alert is raised on the first matching instance being found, sending a message to the local Enterprise Console. Excessive CPU - URL( Secs) TriggerType(Each Matching) TriggerObject(Row) This filter monitors for high CPU usage for any M3 interactive job on the M3 Server View main page. An alert is raised on the first matching instance being found, sending a message to the local Enterprise Console. High severity for NEWS page - URL( Timeout(5 Secs) TriggerType(Each Matching) TriggerObject(Row) This filter triggers an alert for each high severity issue reported within the M3 NEWS page. A message is sent to the local Enterprise Console. Network Server Suite Templates 2-13

20 Instances of an interactive job for a specified user - URL( :6666/) Timeout(15 Secs) TriggerType(Each Matching) TriggerObject(Row) This filter monitors for end-users who start up an excessive number of M3 work sessions. This filter can also be configured to monitor for specific end-users whom often start up the same occurrence of a job multiple times which can result in data integrity issues. An alert is raised if more than 5 interactive jobs for the same user are found and a message sent to the Enterprise Console. Interaction auto job validation - URL( Timeout(15 Secs) TriggerType(First Matching) TriggerObject(Row) This filter ensures that all of your M3 background jobs are active at the times specified. An alert is raised if any of the expected jobs fail the check and a message is sent to the Enterprise Console. Interactive job CPU% Check - URL( Timeout(15 Secs) TriggerType(Each Matching) TriggerObject(Row) This filter monitors CPU usage and currently validates the CPU usage of the M3 auto jobs. However, this criteria setting can easily be changed to review any M3 related page. An alert is raised is CPU usage exceeds 80% for any interactive job and a message sent to the Enterprise Console. ServerView Counters Check - URL( Timeout(15 Secs) TriggerType(First Matching) TriggerObject(Row) This filter monitors CPU usage within any of the M3 counter pages and sends an alert to the Enterprise Console if any counters are found to be using excessive CPU. ServerView Dumplogs in NEWS page - URL( news) Timeout(5 Secs) TriggerType(Each Matching) TriggerObject(Row) This filter triggers an alert and sends a message to the Enterprise Console whenever any new dumplogs are generated in the M3 NEWS page that have a severity level of 4 or greater. ServerView Status Check - URL( Timeout(15 Secs) TriggerType(Each Matching) TriggerObject(Row) This filter monitors for any job displayed on the M3 Server View main page that has an invalid status. Should any job be found to be in an invalid status, a message is sent to the Enterprise Console Halcyon Templates

21 Supervisor Check - URL( Timeout(25 Secs) TriggerType(First Matching) TriggerObject(Table) This filter monitors for the Supervisor interactive M3 job and triggers an alert if the job is not active. A message is sent to the Enterprise Console. Transaction Server Check - URL( Timeout(25 Secs) TriggerType(First Matching) TriggerObject(Table) This filter monitors for the Transaction interactive M3 job and triggers an alert if the job is not active. A message is sent to the Enterprise Console. Service Monitor Backup Exec Services Started - Service(Backup Exec Remote Agent for Windows Systems) Status(<> Started ) This filter checks that the essential Backup Exec Services used for the protection and recovery of key Lawson M3 Windows elements is running. If found to be in a status of stopped, a message is sent to the local Enterprise Console. You may also consider creating an action that attempts to restart this service automatically. MapGenServer - Service(MapGenServer) Status(<> Started ) This filter checks that the critical MapGenServer service is started. If this service is found to be stopped, a message is sent to the local Enterprise Console. You may also consider creating an action that attempts to restart this service automatically. MecService - ecollaborator - Service(MECServer) Status(<> Started ) The Lawson M3 Enterprise Collaborator provides message-based integration between functions in the Lawson M3 Enterprise Management System and other external applications. This filter checks that the ecollaborator is running as normal. If this service is found to be stopped, a message is sent to the local Enterprise Console. You may also consider creating an action that attempts to restart this service automatically. Print Spooler Service <> Started - Service(printSpooler) Status(<> Started ) This filter checks that the print spooler service is running as normal. If this service is found to be stopped, a message is sent to the local Enterprise Console. You may also consider creating an action that attempts to restart this service automatically. Network Server Suite Templates 2-15

22 StreamServe - Production - Service(StreamServe Prod) Status(<> Started ) This filter checks that the Lawson M3 Stream Server service used for the production environment is running as normal. If this service is found to be stopped, a message is sent to the local Enterprise Console. You may also consider creating an action that attempts to restart this service automatically. StreamServe - Standard environment - Service(StreamServe zstd) Status(<> Started) This filter checks that the Lawson M3 Stream Server service used for the standard environment is running as normal. If this service is found to be stopped, a message is sent to the local Enterprise Console. You may also consider creating an action that attempts to restart this service automatically. StreamServe - Test - Service(StreamServe Test) Status(<> Started ) This filter checks that the Lawson M3 Stream Server service used for the test environment is running as normal. If this service is found to be stopped, a message is sent to the local Enterprise Console. StreamServe Repository Server <> Started - Service(StreamServe Repository Server) Status(<> Started ) This filter checks that the Lawson M3 Stream Server service used for the repository server is running as normal. If this service is found to be stopped, a message is sent to the local Enterprise Console to notify you of the error. TCP HTTP Monitor Check CONNECT Site - URL( Timeout(5 Secs) Include( News ) NotInclude( 404 ) This filter checks that the Lawson M3 News site is available. To pass the filter check, the page to which this filter connects, must contain the word 'News' and must not contain '404 (the numerical error ID code for Page Not Found errors). If 'News' is not found or '404' is present, a message is sent to the local Enterprise Console. Server Performance (Advanced) template The Server Performance Advanced template covers the more technical aspects of server performance aside from those covered by the basic CPU, Disk and Memory template. The Server Performance Advanced template contains six generic performance filters: 2-16 Halcyon Templates

23 Memory - Counter(% Committed Bytes In Use) Trigger(>= 90) This filter measures the ratio of Committed Bytes to the Commit Limit - in other words, the amount of virtual memory in use. This indicates insufficient memory if the number is greater than 80 percent. The filter generates an alert if the amount of %Committed Bytes in use at any one time equals or exceeds 90 percent. Memory Available Bytes - Counter(Available Mbytes) Trigger(<=50) This filter measures the amount of physical memory, in Megabytes, available for running processes. If this value is less than 5 percent of the total physical RAM, it means that there is insufficient memory, and paging activity can increase. The filter generates an alert if the amount of memory available at any one time equals or falls below 50Mb. Memory Counter(Pages/sec) Trigger(>=20) This filter measures the rate, per second, at which pages are read from or written to disk to resolve hard page faults. If the value is greater than 1,000, as a result of excessive paging, there may be a memory leak. The filter generates an alert if the amount of memory pages read from or written to disk is greater than or equal to 20 per second. Physical Disk - Instance(_Total) Counter(% Disk Time) Trigger(>= 99) Disk time is represented as a percentage of time that the hard disk is in use. If the hard disk is running around 80 percent of the time, for example, it is a sure sign that system performance is suffering. This filter generates an alert if the percentage disk time in use is greater than or equal to 99 percent. Processor - Instance(_Total) Counter(% Processor Time) Trigger(>=90) This filter measures the percentage of elapsed time that the processor spends executing a non-idle thread. If the percentage is greater than 85 percent, the processor is overwhelmed and the server may require a faster processor. The filter generates an alert if the percentage processor time is greater than or equal to 90 percent. Note: System - Counter(Processor Queue Length) Trigger(>= 10) This measurement indicates the number of threads in the processor queue. The server doesn't have enough processor power if the value is more than two times the number of CPU s for an extended period of time. This filter generates an alert if the processor queue length contains 10 or more threads. All actions for each of the above filters within this template are set to a default of sending an alert to the Enterprise Console. You must manually change this setting if you require an alternative action to be taken upon the generation of an alert. Network Server Suite Templates 2-17

24 Server Performance (Standard) template The Server Performance (Standard) template provides three summary performance filters to cover standard performance indicators of servers and workstation devices. CPU Utilization - Group(CPU) Instance(_Total) Type(% Processor Time) Trigger(>= 98%) This measures the percentage of elapsed time the processor spends executing a non-idle thread. If the percentage is greater than 85 percent, the processor is overwhelmed and the server may require a faster processor. This filter generates an alert if the CPU load of the server on which this template is active is equal to or exceeds 98 percent. Disk Space C drive - Group(Disk) Instance(C) Type(Drive Space Used %) Trigger(>= 75%) This measures the percentage of free space on the selected logical disk drive. Take note if this falls below 15 percent, as you risk running out of free space for the OS to store critical files. One obvious solution here is to add more disk space. This filter monitors systems to ensure that disk space used on the main (C) drive is kept to under 75 percent of the maximum available. You can change the drive setting by opening the template and using the Edit Filter option. An alert is generated if this threshold is reached or exceeded. Note: Physical memory Group(Memory) Type(Physical Memory Used %) Trigger (>= 90%) This filter monitors systems to ensure that the physical memory being used at any given time does not exceed 90 percent. An alert is generated if this threshold is reached or exceeded. All actions for each of the above filters within this template are set to a default of sending an alert to the Enterprise Console. You must manually change this setting if you require an alternative action to be taken upon the generation of an alert Halcyon Templates

25 SQL Server template The SQL Server template contains sixteen generic performance filters to monitor the integrity and performance of an SQL Server device. SQL Server Access Methods - Counter(Full Scans/sec) Trigger(>=50) This counter monitors the number of full scans on base tables or indexes made per second. Values greater than 1 or 2 indicate table/index page scans. A few of the main causes of high Full Scans/sec are: Missing indexes Too many rows requested Queries with missing indexes or too many rows requested result in a large number of logical reads and an increased CPU time. This filter generates an alert if the number of full scans is greater than or equal to 50 per second. SQL Server Access Methods - Counter(Page Splits/sec) Trigger(>=10) This counter measures the number of page splits per second that occur as the result of overflowing index pages. This value needs to be low as possible. This filter generates an alert if the number of page splits exceeds or equals 10 per second. SQL Server Access Methods - Counter(Table Lock Escalations/sec) Trigger(>=10) This counter measures the number of times a table lock was requested in any given second. A high number needs a revisit to the query and the indexes on the table. This filter generates an alert if the number of table locks exceeds or equals 10 requests per second. SQL Server Buffer Manager - Counter(Buffer Cache Hit Ratio) Trigger(<=99) This measurement indicates the ratio between the cache hits and misses. This counter is a good indicator of the caching mechanism in SQL Server. This value needs to be high in the system. This filter generates an alert if the Buffer Cache Hit Ratio equals or falls below 99. SQL Server Buffer Manager - Counter(Stolen Pages) Trigger(>=10,000) This counter measures the number of pages stolen from the buffer cache to satisfy other memory requests of SQL. This filter generates an alert if the number of stolen pages exceeds or equals 10,000 pages. Network Server Suite Templates 2-19

26 SQL Server Cache Manager - Instance(First Triggered Instance) Counter(Cache Hit Ratio) Trigger (<=75) This counter monitors for the percentage of pages found in the procedure cache on an individual SQL instance without having to read from disk. This filter generates an alert on the first instance of the cache hit ratio equalling or falling under 75 percent. SQL Server Cache Manager - Instance(_Total) Counter(Cache Hit Ratio) Trigger(<=75) This counter monitors for the total number of pages found in the procedure cache across all all SQL instances without having to read from disk. This filter generates an alert if the total of the cache hit ratio equals or falls under 75 percent. SQL Server Databases - Instance(_Total) Counter(Active Transactions) Trigger(>=100) This counter monitors for the number of active update transactions for the database. This filter generates an alert if the number of active transactions exceeds or equals 100 transactions at any given time. SQL Server Databases - Instance(_Total) Counter(Percent Log Used) Trigger(>= 75) This counter checks the percentage of free space in the SQL log that is in use. This filter generates an alert in the percentage of free space is equal to or less than 25 percent of the total available. SQL Server Databases - Instance(_Total) Counter(Transactions/sec) Trigger(>= 25) This counter checks against the number of transactions per second started for the database. These transactions come in the form of requests from client machines that are serviced by the database. This filter generates an alert if the number of transactions exceeds or equals 25 per second. SQL Server General Statistics - Counter(User Connections) Trigger(>=100) This counter keeps a tally of the number of users currently connected to the SQL Server. It is recommended to review this counter along with Batch Requests/Sec. A surge in user connections may result in a surge of Batch Requests/Sec. So if there is a disparity (one going up and the other staying flat or going down), then that may be a cause for concern. With a blocking problem, for example, you might see user connections, lock waits and lock wait time all increase while batch requests/sec decreases. This filter generates an alert if the number of user connections exceeds or equals 100 at any given time Halcyon Templates

27 SQL Server Locks - Instance(Database) Counter(Number of Deadlocks/sec) Trigger(>1) This counter checks the number of lock requests that resulted in a deadlock. A deadlock occurs when there is a cyclic dependency between two or more threads, or processes, for some set of resources within SQL Server. This filter generates an alert if the number of deadlocks at any one time is greater than 1. SQL Server Memory Manager - Counter(Connection Memory (KB)) Trigger(>=20480) Connection memory is that used to store the data structures holding the connection context for each user connection. This also includes the memory for the network send and receive buffers that are associated with a connection. This value is the total number of 8Kb buffers. Some of these may be stolen buffers from the buffer pool, and others that may be from MemToLeave. This filter generates an alert if the connection memory counter is greater than or equal to 20480Kb. SQL Server Memory Manager - Counter(Optimizer Memory (KB)) Trigger(>=10240) SQL Server uses the Optimizer memory manager to allocate and manage metadata and tree structures related to query optimization. The server limits this manager to 80% of the server s total memory. This filter generates an alert if the optimizer memory exceeds or equals Kilobytes. Note: SQL Server SQL Statistics - Counter(SQL Compilations/sec) Trigger(>=5) This counter measures the number of times per second that SQL Server compilations have occurred. This value needs to be as low as possible. If you see a high value such as over 100, it is an indication that there are lots of adhoc queries that are running. This filter generates an alert if the number of SQL Compilations exceeds or equals 5 per second. All actions for each of the above filters within this template are set to a default of sending an alert to the Enterprise Console. You must manually change this setting if you require an alternative action to be taken upon the generation of an alert. Terminal Services template Terminal Services is a component of Microsoft Windows operating systems (both client and server versions) that allows a user to access applications or data stored on a remote computer over a network connection without the need to install the application locally. The Terminal Services template provides six generic performance filters: Network Server Suite Templates 2-21

28 Terminal Services - Counter(Active Sessions) Trigger(>=2) This counter measures users who are actively working with the session. This, and the Inactive Sessions counter, in combination with other monitors, are useful in helping you identify the maximum number of users your server can handle before it becomes overloaded and the user experience diminishes as a result. This filter generates an alert if 2 or more active sessions are currently in progress. Terminal Services - Counter(Inactive Sessions) Trigger(>=2) This counter is the reverse of the previous filter as it monitors for users whose session is idle or those who have disconnected. This filter generates an alert if 2 or more inactive sessions are found to be currently in progress. Terminal Services Session - Instance(Console) Counter(% Processor Time) Trigger(>=20) This counter effectively shows how much the processor is being used by productive work. It represents the percentage of time the processor was busy executing a thread other than the thread of the Idle process. This counter has an instance for each system processor available to the operating system. You can use it to verify that each system processor is contributing equally to processing waiting threads. This filter generates an alert if the processor time exceeds or equals 20 percent. Terminal Services Session - Instance(Console) Counter(Total Bytes) Trigger (>=100,000) This counter measures the total number of bytes on this session that includes all protocol overhead. This filter generates an alert if the total number of bytes exceeds or equals 100,000. Terminal Services Session - Instance(Console) Counter(Total Timeouts) Trigger(>=100,000) This counter measures for the total number of timeouts on the communication line from both the host and client sides of the connection. This filter generates an alert if the total number of timeouts equals or exceeds 100,000. Note: Terminal Services Session - Instance(Console) Counter(Total Errors) Trigger(>=5) This counter measures for the total number of errors across all terminal service types. This filter generates an alert if there are 5 or more errors currently being reported across all terminal services. All actions for each of the above filters within this template are set to a default of sending an alert to the Enterprise Console. You must manually change this setting if you require an alternative action to be taken upon the generation of an alert Halcyon Templates

29 AIX Templates AIX System Monitoring (Standard) Template The AIX System Monitoring (Standard) template contains filters covering all of the AIX Monitors with the exception of the System Monitor. The following filters are defined: AIX Error Report Monitor This monitor checks against the AIX Error Report, which contains a list of logged errors. This monitor contains the following two filters: Hardware Errors - Errpt(Class=H) This filter checks for any hardware errors that are reported within the AIX Error Report. These are identified as being of class=h. An alert is generated if any class=h errors are found within the AIX Error Report. Software Errors - Errpt(Class=S) This filter checks for any software errors that are reported within the AIX Error Report. These are identified as being of class=s. An alert is generated if any class=s errors are found within the AIX Error Report. AIX Subsystem Report Monitor This monitor checks for critical subsystems being present and active. It contains the following six filters: Critical Subsystem (inetd) Does Not Exist - Subsystem Does Not Exist(inetd) This filter checks for the existence of the AIX subsystem inetd. An alert is generated if this critical subsystem is not found. Critical Subsystem (inetd) is Inoperative - Subsystem is Inoperative(inetd) This filter checks that the AIX subsystem inetd is active. An alert is generated if this critical subsystem is in an inoperative state. Critical Subsystem (qdaemon) Does Not Exist - Subsystem Does Not Exist(qdaemon) This filter checks for the existence of the AIX subsystem qdaemon. An alert is generated if this critical subsystem is not found. Network Server Suite Templates 2-23

30 Critical Subsystem (qdaemon) is Inoperative - Subsystem is Inoperative(qdaemon) This filter checks that the AIX subsystem qdaemon is active. An alert is generated if this critical subsystem is in an inoperative state. Critical Subsystem (syslogd) Does Not Exist - Subsystem Does Not Exist(syslogd) This filter checks for the existence of the AIX subsystem syslogd. An alert is generated if this critical subsystem is not found. Critical Subsystem (syslogd) is Inoperative - Subsystem is Inoperative(syslogd) This filter checks that the AIX subsystem syslogd is active. An alert is generated if this critical subsystem is in an inoperative state. Logical Volume Monitor The AIX Logical Volume monitors the status of Logical Groups, Logical Volumes and Physical Volumes of the AIX system and contains the following two filters: Alert when Quorum is Set To Off When Disk Mirroring is Active( rootvg) - Measure(Quorum) Trigger(=0) A quorum is a state in which 51 percent or more of the physical volumes in a volume group are accessible. A quorum is a vote of the number of Volume Group Descriptor Areas and Volume Group Status Areas (VGDA/VGSA) that are active. A quorum ensures data integrity in the event of a disk failure. When a quorum is lost, the volume group varies itself off so that the disks are no longer accessible by the Logical Volume Manager (LVM). This filter checks to ensure that the Quorum is available when disk mirroring is active and raises an alert if set to off. Volume Group (rootvg) Does Not Exist - Volume Group rootvg Does Not Exist Root Volume Group (rootvg) is a volume group containing the Base Operating System (BOS). This filter checks to ensure that the Root Volume Group exists and raises an alert if it is not found to be present Halcyon Templates

31 Script Monitor The AIX Script Monitor runs custom AIX scripts and commands and checks the output against Regular Expressions. This contains the following two filters: Check for Failed Logins - Script(/var/lib/halcyon/logfails.sh denied) This filter checks for any failed login attempts. Any failed login attempts found are entered in the failedlogins.log. The filter, Monitor for Failed Logins within the Log File Monitor, then looks for any entries within this file. Check for Missing or Removed Disks - Script(lspv missing removed) This filter runs the lspv command and checks to see if any disks are reported as missing or removed. An alert is raised upon discovery of either of these two conditions. AIX File & Folder Monitor The AIX File & Folder Monitor checks the current status of specific files within the AIX system and contains the following seven filters: File (/etc/aixmibd.conf) Has Changed - File(/etc/aixmibd.conf) Trigger(Has Changed) This filter checks the file; /etc/aixmibd.conf. This file is used to configure the thresholds for many AIX monitors. An alert is generated if a change is detected to the modified date of this file. File (/etc/inetd.conf) Has Changed - File(/etc/inetd.conf) Trigger(Has Changed) This filter checks the file; /etc/inetd.conf. This file, also known as the super server, loads a network program based upon a request from the network. An alert is generated if a change is detected to the modified date of this file. File (/etc/inittab) Has Changed - File(/etc/inittab) Trigger(Has Changed) This filter checks the file; /etc/inittab. This file is a script that controls most of the boot sequence. It dictates what programs and scripts to launch and at what runlevels. An alert is generated if a change is detected to the modified date of this file. File (/etc/profile) Has Changed - File(/etc/profile) Trigger(Has Changed) This filter checks the file; /etc/profile. This file contains system wide environment details and startup programs. An alert is generated if a change is detected to the modified date of this file. Network Server Suite Templates 2-25

32 File (/etc/security/login.cfg) Has Changed - File(/etc/security/ login.cfg) Trigger(Has Changed) This filter checks the file; /etc/security/login.cfg. The /etc/security/login.cfg file is an ASCII file that contains stanzas of configuration information for login and user authentication. An alert is generated if a change is detected to the modified date of the file. File (/etc/sendmail.cf) Has Changed - File(/etc/sendmail.cf) Trigger(Has Changed) This filter checks the file; /etc/sendmail.cf. This is a lengthy and detailed configuration file and direct editing of this file should be avoided. An alert is generated if a change is detected to the modified date of the file. File (/var/spool/cron/crontabs/root) Has Changed - File(/var/spool/ cron/crontabs/root) Trigger(Has Changed) The /var/spool/cron/crontabs/root file contains commands needed for basic system control. This filter checks this file and raise an alert if any changes have been made to the file since the last time the filter ran. Log File Monitor The AIX Log File Monitor checks AIX Log Files for failed user login attempts and new entries being posted to the Cron Log file. This monitor contains the following two filters: Monitor for Failed Logins - LogFile(/var/lib/halcyon/failedlogins.log) Expression(.*) This filter checks the failedlogins.log file for any entries. An alert is generated if any failed logins are reported within the /var/lib/halcyon/failedlogins.log file. Monitor for New Entries in Cron Log - LogFile(/var/adm/cron/log) Expression(.*) This filter checks the /var/adm/cron/log for any new entries. The cron daemon, that controls the automatic running of commands creates a log of its activities in the /var/adm/cron/log file. An alert is generated if any new entries are recorded in this log file. CPU, Filesystem and Memory Monitor The AIX CPU, Filesystem and Memory Monitor template contains the following sixteen filters, that measure filesystem, memory and CPU performance and alerts if thresholds are breached Halcyon Templates

33 Filesystem (/) Disk Space Used >=80% - Group(Filesystem) Volume(/ ) Type(UsedPercent) Trigger(>=80%) This filter checks that the root filesystem / on volume / has more than 20% free space available at all times. An alert is generated if the available disk space on filesystem / equals or exceeds 80 percent. Filesystem (/) Does Not Exist - Group(Filesystem) Volume(/) Trigger(Does Not Exist) This filter checks that the root filesystem / is in existence on volume /. An alert is generated if the root filesystem / is not found on volume /. Filesystem (/) Inode Used >=90% - Group(Filesystem) Volume(/) Type(I-Nodes Used %) Trigger(>=90%) This filter checks the percentage Inode used on root filesystem /. An inode is a data structure in UNIX operating systems that contains important information pertaining to files within a file system. When a file system is created in UNIX, a set amount of inodes are also created. Usually, about 1 percent of the total file system disk space is allocated to the inode table. An alert is generated if the percentage Inode used on root filesystem / equals or exceeds 90 percent. Filesystem (/home) Disk Space Used >=80% - Group(Filesystem) Volume(/home) Type(UsedPercent) Trigger(>=80%) This filter checks that the filesystem /home on volume /home has more than 20% free space available at all times. An alert is generated if the available disk space on filesystem /home equals or exceeds 80 percent. Filesystem (/home) Does Not Exist - Group(Filesystem) Volume(/ home) Trigger(Does Not Exist) This filter checks that the filesystem /home is in existence on volume /home. An alert is generated if the root filesystem /home is not found on volume / home. Filesystem (/home) Inode Used >=90% - Group(Filesystem) Volume(/ home) Type(I-Nodes Used %) Trigger(>=90%) This filter checks the percentage Inode used on filesystem /home. An alert is generated if the percentage Inode used on root filesystem /home equals or exceeds 90 percent. Filesystem (/tmp) Disk Space Used >=80% - Group(Filesystem) Volume(/tmp) Type(UsedPercent) Trigger(>=80%) This filter checks that the filesystem /tmp on volume /tmp has more than 20% free space available at all times. An alert is generated if the available disk space on filesystem /tmp equals or exceeds 80 percent. Network Server Suite Templates 2-27

34 Filesystem (/tmp) Does Not Exist - Group(Filesystem) Volume(/tmp) Trigger(Does Not Exist) This filter checks that the filesystem /tmp is in existence on volume /tmp. An alert is generated if the root filesystem /tmp is not found on volume /tmp. Filesystem (/tmp) Inode Used >=90% - Group(Filesystem) Volume(/ tmp) Type(I-Nodes Used %) Trigger(>=90%) This filter checks the percentage Inode used on filesystem /tmp. An alert is generated if the percentage Inode used on root filesystem /tmp equals or exceeds 90 percent. Filesystem (/usr) Disk Space Used >=80% - Group(Filesystem) Volume(/usr) Type(UsedPercent) Trigger(>=80%) This filter checks that the filesystem /usr on volume /usr has more than 20% free space available at all times. An alert is generated if the available disk space on filesystem /usr equals or exceeds 80 percent. Filesystem (/usr) Does Not Exist - Group(Filesystem) Volume(/usr) Trigger(Does Not Exist) This filter checks that the filesystem /usr is in existence on volume /usr. An alert is generated if the root filesystem /usr is not found on volume /usr. Filesystem (/usr) Inode Used >=90% - Group(Filesystem) Volume(/ usr) Type(I-Nodes Used %) Trigger(>=90%) This filter checks the percentage Inode used on filesystem /usr. An alert is generated if the percentage Inode used on root filesystem /usr equals or exceeds 90 percent. Filesystem (/var) Disk Space Used >=80% - Group(Filesystem) Volume(/var) Type(UsedPercent) Trigger(>=80%) This filter checks that the filesystem /var on volume /var has more than 20% free space available at all times. An alert is generated if the available disk space on filesystem /var equals or exceeds 80 percent. Filesystem (/var) Does Not Exist - Group(Filesystem) Volume(/var) Trigger(Does Not Exist) This filter checks that the filesystem /var is in existence on volume /usr. An alert is generated if the root filesystem /var is not found on volume /usr Halcyon Templates

35 Filesystem (/var) Inode Used >=90% - Group(Filesystem) Volume(/ var) Type(I-Nodes Used %) Trigger(>=90%) This filter checks the percentage Inode used on filesystem /var. An alert is generated if the percentage Inode used on root filesystem /var equals or exceeds 90 percent. Sustained CPU >95% - Group(CPU) CPU(0) Type(Load) Trigger(>95%) This filter checks the sustained usage of the CPU. An alert is generated if the sustained CPU load exceeds 95% at any one time. AIX Process Monitor The AIX Process Monitor is used to check that critical processes exist on the system. This monitor contains the following eight filters: Critical Process (biod) Does Not Exist - Type(Process By Name) Process(biod) Trigger(DoesNotExist) The biod daemon is required on systems that are either mounting (as a client) or exporting (as a server) filesystems via NFS. This filter checks that the critical process, biod, exists on the system and generates an alert if it is not present. Critical Process (cron) Does Not Exist - Type(Process By Name) Process(cron) Trigger(DoesNotExist) The cron daemon runs shell commands at specified dates and times. This filter checks that the critical process, cron, exists on the system and generates an alert if it is not present. Critical Process (errdemon) Does Not Exist - Type(Process By Name) Process(errdemon) Trigger(DoesNotExist) This critical process starts the error logging daemon (errdemon) and writes entries to the error log. This filter checks that this critical process exists on the system and generates an alert if it is not present. Critical Process (inetd) Does Not Exist - Type(Process By Name) Process(inetd) Trigger(DoesNotExist) The inetd critical process provides Internet service management for a network. This filter checks that the inetd process exists on the system and generates an alert if it is not present. Critical Process (portmap) Does Not Exist - Type(Process By Name) Process(portmap) Trigger(DoesNotExist) The portmap daemon helps clients map program number and version number pairs to the port number of a server. This filter checks that the portmap process exists on the system and generates an alert if it is not present. Network Server Suite Templates 2-29

36 Critical Process (qdaemon) Does Not Exist - Type(Process By Name) Process(qdaemon) Trigger(DoesNotExist) The qdaemon tracks both job requests and the resources necessary to complete the jobs, where the resources may be a real printer, some other real device, or even a file. This filter checks that the qdaemon process exists on the system and generates an alert if it is not present. Critical Process (syncd) Does Not Exist - Type(Process By Name) Process(syncd) Trigger(DoesNotExist) A file synchronization program, the syncd daemon flushes pages at regularly scheduled intervals. This filter checks that the syncd process exists on the system and generates an alert if it is not present. Critical Process (writesrv) Does Not Exist - Type(Process By Name) Process(writesrv) Trigger(DoesNotExist) The writesrv daemon enables the ability of users on remote machines to use the write command to send messages to users on the local system. This filter checks that the writesrv process exists on the system and generates an alert if it is not present. AIX Ping Monitor This monitor contains a single filter: Note: Check Server Can Ping Router - Host( ) Timeout(2000) Attempts(4) Success(50%) TTL(128) This filter checks that the server can ping a defined user-defined router. The host address of this template needs to be amended to the actual IP address of the router that you wish to monitor. An alert is generated if the success rate is less than 50% over the four attempts that the ping makes to communicate with the router. All actions for each of the above filters within this template are set to a default of sending an alert to the Enterprise Console. You must manually change this setting if you require an alternative action to be taken upon the generation of an alert. AIX System Monitoring (Advanced) Template The AIX System Monitoring (Advanced) template contains all of the filters available in the standard template and adds additional filter coverage. AIX Error Report Monitor This contains the standard filters plus these additional two filters: 2-30 Halcyon Templates

37 Error Logger Errors - Errpt(Class=O) This filter checks for any hardware errors that are reported within the AIX Error Report. These are identified as being of class=o. An alert is generated if any class=h errors are found within the AIX Error Report. Undetermined Errors - Errpt(Class=U) This filter checks for any hardware errors that are reported within the AIX Error Report. These are identified as being of class=u. An alert is generated if any class=u errors are found within the AIX Error Report. AIX Subsystem Monitor This contains the standard filters plus these additional six filters that cover noncritical subsystems: Optional Subsystem (dhcpcd)does Not Exist - Subsystem Does Not Exist(dhcpcd) This filter checks for the existence of the AIX subsystem dhcpcd. An alert is generated if this optional subsystem is not found. Optional Subsystem (dhcpcd) is Inoperative - Subsystem is Inoperative(dhcpcd) This filter checks that the AIX subsystem syslogd is active. An alert is generated if this optional subsystem is in an inoperative state. Optional Subsystem (lpd)does Not Exist - Subsystem Does Not Exist(lpd) This filter checks for the existence of the AIX subsystem lpd. An alert is generated if this optional subsystem is not found. Optional Subsystem (lpd) is Inoperative- Subsystem is Inoperative(lpd) This filter checks that the AIX subsystem lpd is active. An alert is generated if this optional subsystem is in an inoperative state. Optional Subsystem (sendmail)does Not Exist - Subsystem Does Not Exist(sendmail) This filter checks for the existence of the AIX subsystem sendmail. An alert is generated if this optional subsystem is not found. Optional Subsystem (sendmail) is Inoperative- Subsystem is Inoperative(sendmail) This filter checks that the AIX subsystem sendmail is active. An alert is generated if this optional subsystem is in an inoperative state. Network Server Suite Templates 2-31

38 AIX Logical Volume Monitor This contains the standard filters plus these additional six filters: Stale Physical Partitions Exist (/Mounted on rootvg hd4) - Measure(Stale Physical Partitions) Trigger(=0) A STALE physical partition is a physical partition that contains data you cannot use. This filter checks to see if any stale physical partitions exist on the Root Volume Group hd4. Stale Physical Partitions Exist (/home Mounted on rootvg hd1) - Measure(Stale Physical Partitions) Trigger(=0) This filter checks to see if any stale physical partitions exist on the Root Volume Group hd1. Stale Physical Partitions Exist (/tmp Mounted on rootvg hd3) - Measure(Stale Physical Partitions) Trigger(=0) This filter checks to see if any stale physical partitions exist on the Root Volume Group hd3. Stale Physical Partitions Exist (/usr Mounted on rootvg hd2) - Measure(Stale Physical Partitions) Trigger(=0) This filter checks to see if any stale physical partitions exist on the Root Volume Group hd2. Stale Physical Partitions Exist (/usr Mounted on rootvg hd9) - Measure(Stale Physical Partitions) Trigger(=0) This filter checks to see if any stale physical partitions exist on the Root Volume Group hd9. Stale Physical Volumes on rootvg - Measure(Stale Physical Volumes) Trigger(=0) A physical volume is stale if its contents reflect a version of the logical volume that is known to VOL to be earlier than the current version. This filter checks for the presence of stale physical volumes on the Root Volume Group. Script Monitor This contains the standard filters plus these additional five filters: Console Log - Script(/var/lib/halcyon/conslog.sh.+) This filter checks the console log for any errors and raises an alert should any be found Halcyon Templates

39 Disk I/O Busy >75% Possible I/O Bound System This filter checks the disk for input and output activity and raises an alert if the activity percentage is higher than 75% which would suggest that the disk is too busy. Disk I/O Wait >25% Possible I/O Bound System This filter checks the disk for input and output activity and raises an alert if the disk is waiting for more than 25% for activity which would suggest that the disk is too idle. Sustained Disk Utilization >=80% This filter checks the percentage of time that the physical disk is active and raises an alert if the activity level is constantly higher than 80 percent. Verify Operations Status of Path to MPIO Device Beginning with AIX 5.2, all SCSI SCSD disk drives are automatically configured as MPIO devices and a select number of Fibre Channel disk drives can be configured as MPIO Other disk. Other devices can be supported, providing the device driver is compatible with the MPIO implementation in AIX. This filter checks to ensure that MPIO is enabled. AIX File & Folder Monitor This contains the standard filters plus these additional four filters: File (/etc/environment) Has Changed - File(/etc) Trigger(Exists) The /etc/environment file contains variables specifying the basic environment for all processes. When a new process begins, the exec subroutine makes an array of strings available that have the form Name = Value. This array of strings is called the environment. This filter checks this file and raises an alert if any changes have been made. File (/etc/hostmibd.conf) Has Changed - File(/etc) Trigger(Exists) The hostmibd.conf file provides the configuration information for the hostmibd dpi2 sub-agent. This filter checks this file and raises an alert if any changes have been made. File (etc/netsvc.conf) Has Changed - File(/etc) Trigger(Exists) The /etc/netsvc.conf file is used to specify the ordering of name resolution for the sendmail command, gethostbyname subroutine, gethostaddr subroutine, and gethostent subroutine and alias resolution for the sendmail command. This filter checks this file and raises an alert if any changes have been made. Network Server Suite Templates 2-33

40 File(/etc/resolv.conf) Has Changed - File(/etc) Trigger(Exists) The etc/resolv.conf configuration file contains information that determines the operational parameters of the DNS resolver routines installed in the operating system, typically the IP addresses of nameservers available to a host. This filter checks this file and raises an alert if any changes have been made. Log File Monitor This contains the same two filters available in the AIX System Monitoring (Standard) template. CPU, Filesystem and Memory Monitor This contains the standard filters plus these additional three filters: Page File Used <30% (Suggests Too Much Paging Space) - Group(Memory) Type(Page File Used %) Trigger(=30%) This filter checks the page file used memory and raises an alert if the percentage figure falls under 30 percent, which would suggest that there is too much paging space assigned. Page File Used >70% (Suggests Not Enough Paging Space) - Group(Memory) Type(Page File Used %) Trigger(=70%) This filter checks that page file used memory and raises an alert if the percentage figure exceeds 70 percent, which would suggest that there is too little paging space assigned. Paging Space >95% - Group(Memory) Type(Page File Used %) Trigger(=95%) This filter checks that the paging space available for use does not exceed 95%. An alert is raised if this figure is exceeded. Process Monitor This contains the standard filters plus these additional eight filters checking optional processes: Optional Process (aixmibd) Does Not Exist - Type(Process By Name) Process(aixmibd) Trigger(Does Not Exist) The process aixmibd provides the AIX Enterprise Management Information Base (MIB) extension subagent, for use with the Simple Network Management Protocol (SNMP), that collects data from system for variables defined in the AIX Enterprise Specific MIB. This filter checks that the aixmibd process exists on the system and generates an alert if it is not present Halcyon Templates

41 Optional Process (hostmibd) Does Not Exist - Type(Process By Name) Process(hostmibd) Trigger(Does Not Exist) The hostmibd command starts the hostmibd dpi2 sub-agent. The hostmibd daemon is normally executed during system startup when /etc/rc.tcpip shell script is called. This filter checks that the hostmibd process exists on the system and generates an alert if it is not present. Optional Process (rpc.lockd) Does Not Exist - Type(Process By Name) Process(rpc.lockd) Trigger(Does Not Exist) This daemon processes lock requests and asks the rpc.statd daemon to monitor those requests. This filter checks that the rpc.lockd daemon exists on the system and generates an alert if not present. Optional Process (rpc.statd) Does Not Exist - Type(Process By Name) Process(rpc.statd) Trigger(Does Not Exist) This daemon works with the rpc.lockd daemon to provide locking functions. This filter checks that the rpc.statd daemon exists on the system and generates an alert if not present. Optional Process (sendmail) Does Not Exist - Type(Process By Name) Process(sendmail) Trigger(Does Not Exist) The sendmail command receives formatted text messages and routes the messages to one or more users. Used on a network, the sendmail command translates the format of a message's header information to match the requirements of the destination system. The program determines the network of the destination system by using the syntax and content of the addresses. This filter checks that the sendmail process exists on the system and generates an alert if not present. Optional Process (snmpd) Does Not Exist - Type(Process By Name) Process(snmpd) Trigger(Does Not Exist) The snmpd process starts the Simple Network Management Protocol (SNMP) agent as a background process. This filter checks that the snmpd process exists on the system and generates an alert if not present. Optional Process (snmpmibd) Does Not Exist - Type(Process By Name) Process(snmpmibd) Trigger(Does Not Exist) The snmpmibd command starts the snmpmibd Distributed Protocol Interface (DPI) version 2 (dpi2) sub-agent. This command may only be issued by a user with root privileges or by a member of the system group. This filter checks that the snmpmibd process exists on the system and generates an alert if not present. Network Server Suite Templates 2-35

42 Linux Templates Optional Process (sshd) Does Not Exist - Type(Process By Name) Process(sshd) Trigger(Does Not Exist) The Open Secure Shell daemon helps secure network connections. This filter checks that the sshd process exists on the system and generates an alert if not present. Ping Monitor This contains the same filter as available in the AIX System Monitoring (Basic) template. Network Server Suite templates are currently available for two versions of the Linux operating system, RED HAT and opensuse. These are very similar and only variations of the RED HAT templates are listed in under the SUSE templates headings. RED HAT System Monitoring Template (Standard) The RED HAT system monitoring (Standard) template includes the following filters: Linux Logical Volume Monitor This contains the following two filters: Logical volume (LogVol00) status <> available - Measure(Status) Trigger(=available) This filter checks that logical volume LogVol00 is always available. If the status of LogVol00 is in any status other than available, an alert is raised. Logical volume (LogVol01) status <> available - Measure(Status) Trigger(=available) This filter checks that logical volume LogVol01 is always available. If the status of LogVol00 is in any status other than available, an alert is raised. Script Monitor This contains the following filter: Check For Failed Raid Drives - Script(mdadm) -D /dev/md0 grep Failed Devices This filter checks for up to 3 failed RAID drives. An alert is raised if any one of the RAID devices is found to be in a failed status Halcyon Templates

43 File & Folder Monitor This contains the following five filters: File(/etc/crontab) Has Changed - File (/etc) Trigger(Exists) Crontab is the program used to install, remove or list the tables used to drive the cron daemon. This filter checks to ensure that crontab has not been amended and raises an alert if any changes have been made. File(/etc/inittab) Has Changed - File (/etc) Trigger(Exists) The inittab file describes which processes are started at bootup and during normal operation. This filter checks to ensure that the inittab file has not been amended and raises an alert if any changes have been made. File(/etc/mail/sendmail.cf) Has Changed - File (/etc/mail) Trigger(Exists) This filter checks the file; /etc/sendmail.cf. This is a lengthy and detailed configuration file and direct editing of this file should be avoided. An alert is generated if a change is detected to the modified date of the file. File(/etc/profile) Has Changed - File (/etc) Trigger(Exists) This filter checks the file; /etc/profile. This file contains system wide environment details and startup programs. An alert is generated if a change is detected to the modified date of this file. File(/etc/xinetd) Has Changed - File (/etc) Trigger(Exists) Xinetd is a secure replacement for inetd, the Internet services daemon. Xinetd provides access control for all services based on the address of the remote host and/or on time of access and can prevent denial-of-access attacks. Each service has its own specific configuration file for Xinetd; the files a relocated in the /etc/xinetd.d directory. An alert is generated if a change is detected to the modified date of this file. Log File Monitor This contains the following filter: Monitor for Failures in Secure Log - LogFile(/var/log/secure) Expression(Failure) This filter checks for any authentication failure errors present in the /var/log/ secure log file and raises an alert if an error is found. Network Server Suite Templates 2-37

44 CPU, Filesystem and Memory Monitor This contains the following five filters: Filesystem (/) Disk Space Used >=80% - Group(Filesystem) Volume (/) Type(Filesystem Space Used %) Trigger(=80%) This filter checks that the root filesystem / on volume / has more than 20% free space available at all times. An alert is generated if the available disk space on filesystem / equals or exceeds 80 percent. Filesystem(/) Does Not Exist - Group(Filesystem) Volume(/) Trigger(Does Not Exist) This filter checks that the root filesystem / is in existence on volume /. An alert is generated if the root filesystem / is not found on volume /. Filesystem(/) Inode Used >=90% - Group(Filesystem) Volume(/) Type(I-Nodes Used %) Trigger (=90%) This filter checks the percentage Inode used on root filesystem /. An inode is a data structure in UNIX operating systems that contains important information pertaining to files within a file system. When a file system is created in UNIX, a set amount of inodes are also created. Usually, about 1 percent of the total file system disk space is allocated to the inode table. An alert is generated if the percentage Inode used on root filesystem / equals or exceeds 90 percent. Paging Space >95% - Group(Memory) Type(Page File Used %) Trigger(=95%) This filter checks that the paging space available for use does not exceed 95%. An alert is raised if this figure is breached. Sustained CPU >95% - Group(CPU) CPU(0) Type(CPU Load) Trigger (=95%) This filter checks the sustained usage of the CPU. An alert is generated if the sustained CPU load exceeds 95% at any one time Halcyon Templates

45 Process Monitor This contains the following filters: Critical Process (crond) Does Not Exist - Type(Process By Name) Process(crond) Trigger(Does Not Exist) Cron jobs are managed by a daemon named crond. When cron schedules are added, deleted or modified by crontab, any changes are enacted by the crond daemon. Crond daemon runs constantly in the background and checks once a minute to see if any of the scheduled jobs need to be executed. If the crond process is not running, no cron jobs will be executed. This filter checks that the crond process exists and raises an alert if it is not found. Critical Process (gdm-binary) Does Not Exist - Type(Process By Name) Process(gdm-binary) Trigger(Does Not Exist) Gdm (the GNOME Display Manager) is a configurable re implementation of xdm, the X Display Manager. Gdm allows you to log into your system with the X Window System running and supports running several different X sessions on your local machine at the same time. This filter checks that the gdm-binary process exists and raises an alert if it is not found. Critical Process (sshd) Does Not Exist - Type(Process By Name) Process(sshd) Trigger(Does Not Exist) OpenSSH Daemon (sshd) is the daemon program for ssh. Together these programs provide secure encrypted communications between two untrusted hosts over an insecure network. The SSH daemon listens for connections from clients. It starts a new daemon for each incoming connection. These daemons then handle key exchange, encryption, authentication, command execution, and data exchange. This filter checks that the sshd process exists and raises an alert if it is not found. Critical Process (syslogd) Does Not Exist - Type(Process By Name) Process(syslogd) Trigger(Does Not Exist) Syslogd provides the logging of system events. Every logged message contains at least a time and a hostname field and normally a program name field. This filter checks that the syslogd process exists and raises an alert if it is not found. Critical Process (xfs) Does Not Exist - Type(Process By Name) Process(xfs) Trigger(Does Not Exist) X font server xfs is a daemon that listens on a network port and serves X fonts to X servers (and thus to X clients). This daemon makes it possible to have a central repository of fonts on a networked machine running xfs so that all the machines running X servers on a network do not require their own set of fonts. This filter checks that the xfs process exists and raises an alert if it is not found. Network Server Suite Templates 2-39

46 Critical Process (xinetd) Does Not Exist - Type(Process By Name) Process(xinetd) Trigger(Does Not Exist) Xinetd is a secure replacement for inetd, the Internet services daemon. Xinetd provides access control for all services based on the address of the remote host and/or on time of access and can prevent denial-of-access attacks. This filter checks that the xinetd process exists and raises an alert if it is not found. Ping Monitor This monitor contains a single filter. Note: Check Server Can Ping Router - Host( ) Timeout(2000) Attempts(4) Success(50%) TTL(128) This filter checks that the server can ping a defined user-defined router. The host address of this template needs to be amended to the actual IP address of the router that you wish to monitor. An alert is generated if the success rate is less than 50% over the four attempts that the ping makes to communicate with the router. All actions for each of the above filters within this template are set to a default of sending an alert to the Enterprise Console. You must manually change this setting if you require an alternative action to be taken upon the generation of an alert. RED HAT System Monitoring Template (Advanced) The RED HAT system monitoring template (Advanced) includes all of the filters available in the Standard template plus these additional filters: Linux Logical Volume Monitor This monitor contains the same filters as those available in the Standard template. Script Monitor This monitor contains the same filter as the one available in the Standard template Halcyon Templates

47 File & Folder Monitor This monitor contains the same filters as the Standard template and includes these additional three filters: File (/etc/resolv.conf) Has Changed - File(/etc) Trigger(Exists) The configuration file resolv.conf contains information that allows a computer connected to the Internet to resolve names into addresses. The resolv.conf file typically contains the IP addresses of nameservers (DNS name resolvers) that attempt to translate names into addresses for any node available on the network.this filter raises an alert if any changes are made to the resolv.conf file. File (/etc/sysconfig/iptables) Has Changed - File(/etc/sysconfig) Trigger(Exists) Iptables is a generic table structure that defines rules and commands as part of the netfilter framework that facilitates Network Address Translation (NAT), packet filtering, and packet mangling in the Linux 2.4 and later operating systems. This filter raises an alert if any changes are made to iptables. File (/etc/vsftpd/vsftpd.conf) Has Changed - File(/etc/vsftpd) Trigger(Exists) Vsftpd.conf is used to control various aspects of vsftpd's behaviour. Vsftpd is the Very Secure File Transfer Protocol Daemon. This filter raises an alert if any changes are made to vsftpd.conf. Log File Monitor This monitor contains the same filter as the Standard template and includes these additional two filters: Monitor for MySQL Errors - LogFile(/var/log/mysqld.log) Expression (error failure) This filter checks the MySQL database server log file for any failure errors and raises an alert should any be found. Monitor for Samba Errors - LogFile(/var/log/samba/smbd.log) Expression(error failed) The Samba log files can help diagnose the vast majority of the problems that Samba administrators are likely to encounter. This filter checks the smbd.log file and raises an alert if any failure errors are found. Network Server Suite Templates 2-41

48 CPU, Filesystem and Memory Monitor This monitor contains the same filters as the Standard template plus these additional five filters: Filesystem (/boot) Disk Space Used >=80% - Group(Filesystem) Volume(/boot) Type(Filesystem Space Used %) Trigger(=80%) This filter checks to ensure that the disk space used in the filesystem /boot remains at least 20% free at all times. If the level of disk space available falls below 20% an alert is raised. Filesystem (/boot) Does Not Exist - Group(Filesystem) Volume(/boot) Trigger(Does Not Exist) This filter checks to ensure that the file system /boot exists and raises an alert if it is not found. Filesystem (/boot) Inode Used >=90% - Group(Filesystem) Volume(/ boot) Type(I-Nodes Used %) Trigger(=90%) This filter checks the percentage Inode used on root filesystem /boot. An inode is a data structure in UNIX operating systems that contains important information pertaining to files within a file system. When a file system is created in UNIX, a set amount of inodes are also created. Usually, about 1 percent of the total file system disk space is allocated to the inode table. An alert is generated if the percentage Inode used on root filesystem / equals or exceeds 90 percent. Page File Used <30% (Suggests Too Much Paging Space) - Group(Memory) Type(Page File Used %) Trigger (=30%) This filter checks the page file used memory and raises an alert if the percentage figure falls under 30 percent, which would suggest that there is too much paging space assigned. Page File Used >70% (Suggests Not Enough Paging Space) - Group(Memory) Type(Page File Used %) Trigger(=70%) This filter checks that page file used memory and raises an alert if the percentage figure exceeds 70 percent, which would suggest that there is too little paging space assigned. Process Monitor This monitor contains the same critical process filters as the Standard templates and adds another ten filters to monitor for the existence of the following optional processes Halcyon Templates

49 Optional Process (httpd) Does Not Exist - Type(Process By Name) Process(httpd) Trigger(Does Not Exist) The httpd process is the HyperText Transfer Protocol Daemon. This filter checks for the existence of this daemon and raises an alert if it is not found. Optional Process (mysqld) Does Not Exist - Type(Process By Name) Process(mysqld) Trigger(Does Not Exist) The mysql daemon launches the MySQL database server. This filter checks for the existence of this daemon and raises an alert if it is not found. Optional Process (postmaster) Does Not Exist - Type(Process By Name) Process(postmaster) Trigger(Does Not Exist) Postmaster is the PostgreSQL multiuser database server. In order for a client application to access a database it connects (over a network or locally) to a running postmaster. The postmaster then starts a separate server process ("postgres") to handle the connection. The postmaster also manages the communication among server processes. This filter checks for the existence of this process and raises an alert if it is not found. Optional Process (rpc.idmapd) Does Not Exist - Type(Process By Name) Process(rpc.imapd) Trigger(Does Not Exist) Rpc.idmapd is the NFSv4 ID name mapping daemon. It provides functionality to the NFSv4 kernel client and server, to which it communicates via upcalls, by translating user and group IDs to names, and vice versa. This filter checks for the existence of this daemon and raises an alert if it is not found. Optional Process (rpc.statd) Does Not Exist - Type(Process By Name) Process(rpc.statd) Trigger(Does Not Exist) The rpc.statd server implements the NSM (Network Status Monitor) RPC protocol. NSM implements a reboot notification service. It is used by the NFS file locking service, rpc.lockd, to implement lock recovery when the NFS server machine crashes and reboots. This filter checks for the existence of this daemon and raises an alert if it is not found. Optional Process(sendmail) Does Not Exist - Type(Process By Name) Process(sendmail) Trigger(Does Not Exist) Sendmail sends a message to one or more recipients, routing the message over whatever networks are necessary. Sendmail does internet work, forwarding as necessary to deliver the message to the correct place. This filter checks for the existence of this process and raises an alert if it is not found. Network Server Suite Templates 2-43

50 Optional Process(smbd) Does Not Exist - Type(Process By Name) Process(smbd) Trigger(Does Not Exist) This program is part of the samba(7) suite. smbd is the server daemon that provides filesharing and printing services to Windows clients. The server provides filespace and printer services to clients using the SMB (or CIFS) protocol. This filter checks for the existence of this daemon and raises an alert if it is not found. Optional Process(spamd) Does Not Exist - Type(Process By Name) Process(spamd) Trigger(Does Not Exist) The program spamd is a service designed to reduce the flow of spam to your inbox. This filter checks for the existence of this process and raises an alert if it is not found. Optional Process(squid) Does Not Exist - Type(Process By Name) Process(squid) Trigger(Does Not Exist) Squid is an Internet Object Cache developed by the National Laboratory for Applied Networking Research (NLANR) and Internet volunteers. This filter checks for the existence of this file and raises an alert if it is not found. Optional Process(vsftpd) Does Not Exist - Type(Process By Name) Process(vsftpd) Trigger(Does Not Exist) Vsftpd is The Very Secure FTP Daemon. Vsftpd supports both anonymous and non-anonymous FTP, PAM authentication, bandwidth limiting, and the Linux sendfile() facility. This filter checks for the existence of this file and raises an alert if it not found. Ping Monitor This monitor contains the same filter as the one available in the Standard template. SUSE System Monitoring Template (Standard) The SUSE system monitoring template (Standard) includes the same filters as RED HAT system monitoring template (Standard) with the following exceptions: 2-44 Halcyon Templates

51 Log File Monitor Monitor for Failure in Messages Log - LogFile(/var/log/messages) Expression (error fail) This filter checks for any error or failure messages in the /var/log/messages log file and raises an alert if any such messages are found. SUSE System Monitoring Template (Advanced) The SUSE system monitoring template (Advanced) includes the same filters as RED HAT system monitoring template (Advanced). Windows Reporting Templates Reporting templates allow you to apply the same reporting criteria across multiple systems in your enterprise. By using a reporting template you ensure that you are generating like-for-like reports across the same generic performance measurements of your systems. As with filter templates, a change made at filter level is reflected across all systems where that filter is implemented. Reporting templates are created by adding a new template to the Templates tab of the Central Configuration Manager. It is good practice to give a reporting template an identification label that distinguishes it from any filter templates that may already exist. A basic reporting template covering CPU, Disk and Memory Performance is included by default. Activate the reporting schedule of the WMI Monitor and edit the system defaults to your own reporting requirements. You can now edit the actual fields (in addition to the seven pre-defined fields) on which generic performance data is gathered and reported. Use of a reporting template disables the ability to manually add further reporting items to the reporting schedule for the selected device. Applying a Reporting Template Report templates are applied from the Systems tab of the Central Configuration Manager at Server Manager level. The reporting template is assigned from the Reporting Templates section. Note that all templates (Filter and Reporting appear in the drop-down choice menu so be careful when making your selection. The default Reporting template is called Reporting (Performance) - CPU, Disk and Memory Reporting. Network Server Suite Templates 2-45

52 Figure 2.4 Applying Reporting Templates Repeat the process for each server manager on which the reporting template is to be installed Halcyon Templates

53 C HAPTER 3POWER/SYSTEM I RULES CHAPTER Overview Each suite supplied by Halcyon Software comes with pre-defined example rules that are based upon common monitoring requirements. These rules enable you to commence system monitoring with the minimum of effort and knowledge. All example rules are in an initial status of held and must be released in order to begin monitoring. Unless specified under an individual rule heading, each rule has an predetermined action applied via an action schedule which sends an alert to the local system console. Message Queue Rules Example Message Queue Rules Example Message Queue rules are split into four key rule groups and are available within: Level 1 - Message Management Suite Level 2 - Systems Operations Suite Level 3 - Advanced Automation Suite Level 4 - Operations Center Suite No Global Exclusions Defined Sequence number 0 is a special sequence number which cannot be deleted, even if you do not define any global exclusions for the named message queue. Power/System i Rules 3-47

54 If you have messages that are to be totally excluded from any type of action, take option 2=Change against sequence 0 to define the exact criteria for the exclusion. If a global exclusion exists for a message, then none of the other sequence numbers for this message queue are searched for a match and no action is taken. FTP Activity monitoring Queue Rule Group This rule group contains seven example rules (plus the global exclusions rule) that are used to monitor FTP activity. FTP Create Directory The FTP Create Directory rule checks for the creation of a directory within the current environment on message queue HMMFTP. This rule checks for the existence of message ID FTP1001 and if present, generates an alert with the message text; Directory (name) created by user (user profile). FTP Delete Directory The FTP Delete Directory rule checks for the deletion of a directory within the current environment on message queue HMMFTP. This rule checks for the existence of message ID FTP1002 and if present, generates an alert with the message text; Directory (name) deleted by user( user profile). FTP File Received The FTP File Received rule checks for any file received within the current environment on message queue HMMFTP. There are three message ID s that this rule checks against; FTP0007, FTP1007 and FTP3007. FTP Generates an alert with message text; File (name) received by user (user profile) to IP Address (IP Address in DNS protocol format) FTP Generates an alert with message text; File (name) received by user (user profile) to IP Address (IP Address in DNS protocol format) FTP Generates an alert with the message text; File (name) received by user (user profile) to IP Address (IP Address in DNS protocol format) using TFTP. FTP File Sent The FTP File Sent rule checks for any file sent from within the current environment on message queue HMMFTP. There are three message ID s that this rule checks against; FTP0006, FTP1006 and FTP3006. FTP Generates an alert with message text; File (name) sent by user (user profile) to IP Address (IP address in DNS protocol format) FTP Generates an alert with message text; File (name) sent by user (user profile) to IP Address (IP address in DNS protocol format) FTP Generates an alert with the message text; File (name) sent by user (user profile) to IP Address(IP address in DNS protocol format) using TFTP Halcyon Templates

55 FTP File Deleted The FTP File Deleted rule checks for any file deleted from within the current environment on message queue HMMFTP. This rule checks for the existence of message ID FTP1005 and if present, generates an alert with the message text; File (name) deleted by user (user profile). FTP File Renamed The FTP File Renamed rule checks for any file that has been renamed within the current environment on message queue HMMFTP. This rule checks for the existence of message ID FTP1008 and if present, generates an alert with the message text; File (name) renamed by user (user profile). FTP File Command Executed The FTP File Command Executed rule checks for any command run using FTP in the current environment on message queue HMMFTP. There are two message ID s that this rule checks against; FTP1009 and FTP2009. FTP Generates an alert with the message text; Command (name) run by user (user profile) using FTP. FTP Generates an alert with the message text; Command (name) run by user (user profile) using REXEC. System History Log This rule group contains three (plus the global exclusions rule) example rules that are used to monitor system events that have been logged. Backup Objects NOT Saved Checks for message ID s CPF3701 and CPF3702 in the current environment within message queue QHST. These error messages indicate the number of objects saved and the number of objects not saved. For message ID CPF3702, the number of objects not included is also specified. An alert is generated if either of these messages are present Equivalent omission criteria, which exist for both message ID s, ensure that an alert is not generated if the number of objects not saved is zero. Password invalid for QSECOFR Checks for message ID CPF2234 in the current environment within message queue QHST and generates an alert if the message ID is present. This error message indicates that an invalid password has been entered for the Security Administrator user ID. Power/System i Rules 3-49

56 Job Ended Abnormally Checks for message ID CPF1164 in the current environment within message queue QHST. CPF1164 is the job completion message and an alert is raised if a job ends with abnormally or with errors. Omission criteria apply to jobs that complete normally or that completed normally during controlled ending. System Critical Messages Queue This rule group contains two example rules (plus the global exclusions rule) that are used to monitor both generic and specific critical messages. All Critical Messages Acts as a catch-all for all critical messages that may be present on message queue QSYSMSG within the current environment. There are two omission criteria that apply to this rule: CPF1269 A request to start a program on a remote system failed HMM0066 Message Manager monitoring queue startup message Example Critical Messages This rule provides examples of critical messages for which you may wish to monitor. The following message ID s are included: CPF0907 Serious storage condition may exist CPF111C System scheduled to power down CPF111D System is powering down CPF1269 A request to start a program on a remote system failed CPF1393 Subsystem disabled user profile on device CPI0920 Error occurred on disk unit CPI0945 Mirrored protection is suspended on disk unit CPI0946 Mirrored protection is suspended on disk unit CPI0947 Mirrored protection is suspended on disk unit CPI0948 Mirrored protection is suspended on disk unit CPI0949 Mirrored protection is suspended on disk unit CPI0950 Storage unit now available CPI0953 ASP Storage threshold reached CPI0954 Storage limit exceeded for ASP CPI0955 System ASP unprotected storage limit exceeded. CPI0956 Mirrored protection is suspended on disk unit CPI0957 Mirrored protection is suspended on disk unit 3-50 Halcyon Templates

57 CPI0958 CPI0959 CPI0964 CPI0965 CPI0966 CPI0970 CPI0988 CPI0989 CPI0992 CPI0996 CPI0997 CPI0998 CPI1117 CPI1136 CPI1138 CPI1139 CPI1153 CPI1154 CPI2209 CPI2283 CPI2284 CPI8A13 CPI8A14 CPI8898 CPI9014 CPI94A0 Mirrored protection is suspended on disk unit Mirrored protection is suspended on disk unit Weak battery condition exists Failure of battery backup feature in system unit Failure of battery backup feature in expansion unit Disk unit not operating Mirrored protection resuming on disk unit Mirrored protection resumed on disk unit Error occurred on disk unit Error occurred on disk unit Nearly all available machine addresses used Error occurred on disk unit Damaged job schedule in library deleted Mirrored protection still suspended Storage overflow recovered for ASP Storage overflow recovery failed for ASP System password bypass period ended System password bypass period will end in days User Profile deleted because it was damaged QAUDCTL system value changed to *NONE QAUDCTL system value changed to *NONE QDOC library nearing system object limit QDOC library has exceeded system object limit An optical system bus failure is detected Password received from device not valid Disk error on device CPI94CE Error detected in bus expansion adapter, system processor or cables CPI94CF Main storage card failure is detected CPI94FC Disk error on device CPI9490 Disk error on device CPP0DDA A system processor failure is detected in slot 9 CPP0DDB A system processor failure is detected in slot 10 CPP0DDC A system processor error is detected CPP0DDD A system processor diagnostic code detected an error CPP0DDE A system processor error is detected Power/System i Rules 3-51

58 CPP0DDF CPP0DD9 CPP29BA CPP29B0 CPP29B8 CPP29B9 CPP951B CPP9522 CPP955E CPP9575 CPP9576 CPP9589 CPP961F CPP9616 CPP9617 CPP9618 CPP962B CPP9620 CPP9621 CPP9622 CPP9623 A system processor is missing A system processor failure is detected Hardware error on device under controller Recovery threshold exceeded on device under controller RAID protection suspended on controller Power protection suspended on controller Battery power unit fault Battery power unit fault A battery power unit is not installed in the 9406 system unit The battery power unit in the 9406 needs to be replaced The battery power unit in the 9406 needs to be replaced Test of the battery power unit is complete DC bulk module 3 fault A battery power unit is not installed A battery power unit needs to be replaced A battery power unit needs to be replaced DC Bulk module 3 fault DC Bulk module 2 fault DC Bulk module 1 fault DC Bulk module 1 fault DC Bulk module 2 fault QSYSOPR Message Queue This rule group contains example rules of important system messages that appear on the QSYSOPR message queue. All inquiry messages Raises an alert for each inquiry message that is generated on the QSYSOPR message queue. High severity messages >=90 Raises an alert for each message generated on the QSYSOPR message queue with a severity level of 90 or greater. Job BACKUPS NOT completed Raises an alert for each message with ID CPF1241 for job BACKUPS that is NOT received on the QSYSOPR message queue Halcyon Templates

59 Job MTHEND completed normally Raises an alert for each message with ID CPF1241 for job MTHEND that is received on the QSYSOPR message queue. All RPG error messages Raises an alert for each message beginning with R* from within the QRPGMSGE message file that is received on the QSYSOPR message queue. Note: System power failed Raises an alert for any message with ID CPF1816; System Utility Power Failed that is received on the QSYSOPR message queue. The actions for this rule are different in that the system initially sends an advisory text message to an operational rota group, then sends a message on the local system to advise all active users that the system is powering down. A final action then power downs the system. Warning: This rule may cause issues if released by itself, as the system may go into a loop, meaning that as soon as the machine powers up, the action restarts and powers the system back down again. Always release the System Power Restored rule (below) in combination with this rule to safeguard against this. System power restored Raises an alert for any message with ID CPF1817; System Utility Power Restored that is received on the QSYSOPR message queue. The actions for this rule are again different from the usual default action, in that an advisory text message is sent to the operational rota group, advising that the system power is now restored and then a final action closes the alert. Controller CTL999 failed Raises an alert for any message with ID CPI7F70; Controller CTL999 failed that is received on the QSYSOPR message queue. There are five actions applied to this rule. Initially, the system attempts to reply to this message up to three times. The reply action is then cancelled. Subsequent actions send a command to vary off the controller, wait sixty seconds before sending another command to vary on the controller and then reset the rule so that further failures are captured. Controller CTL999 OK Raises an alert for any message with ID CPF5908; Controller CTL999 OK that is received on the QSYSOPR message queue. The applied rule action simply resets the rule. Power/System i Rules 3-53

60 TCP/IP Rules File is Full Raises an alert for any message with ID CPA5305; Record Not Added. Member is Full. that is received on the QSYSOPR message queue. The default rule action is set to reply with I (ignore this message, increase the file size by one increment and add the record to the file). Cache Battery Hardware Errors Raises an alert for any message with ID CPPEA13; *Attention* Contact Your Hardware Service Provider that is received on the QSYSOPR message queue. Critical Hardware Errors Raises an alert for any message with ID CPP8988; A critical system hardware problem has occurred. Critical message handler has been run that is received on the QSYSOPR message queue. Halcyon SMS/Pager Errors Raises an alert for any message with ID HMC0303 in message file HMCMSGF on the QSYSOPR message queue. This message is raised by Message Communicator scripts via the ADMINMSG function, whenever a failure occurs. GSM Service Provider Messages Advises when any GSM Service Provider messages are received. The rule raises an alert when the specialist Halcyon message ID HMC0191 is received on the QSYSOPR message queue. Temp Maint Code Expiry Advises when the Halcyon Software Maintenance code is due to expire. The rule raises an alert when the specialist Halcyon message ID HAL0125 is received on the QSYSOPR message queue. Example TCP/IP Rules These rules are supplied by default under the EXAMPLE TCP/IP Rule Group, and are available within: Level 1 - Message Management Suite Level 2 - Systems Operations Suite Level 3 - Advanced Automation Suite Level 4 - Operations Center Suite 3-54 Halcyon Templates

61 Check Server is Active Checks that your mail server remains active at all times. An alert is raised if the attempted TCP/IP connection fails at any time on a 24/7 basis. Uses MAILSRV01 as the default Remote System - either change this entry to the actual name of your mail server or to *INTNETADR and supply an actual IP address. Check Main Router is Active Checks that the main router used in your network environment is operating and active on a 24/7 basis. Uses ROUTER01 as the default remote system - change to proper name of your main router or change entry in this field to *INTNETADR and supply an actual IP Address. Check i5 Partition is Active Checks that a partition within a Power/System i device is active on a 24/7 basis. Uses a default setting of a pre-supplied internet address and a default description of i570 Partition 2 that will need changing in order for the rule to be fully functional. Output Queue Rules Example Output Queue Rules These rules are supplied by default under the EXAMPLE Output Queue Rule Group, and are available within: Level 1 - Message Management Suite Level 2 - Systems Operations Suite Level 3 - Advanced Automation Suite Level 4 - Operations Center Suite Backlog >=100 Spoolfiles Checks to ensure that there is no more than 100 spoolfiles waiting on a specified Output Queue at any on time on a 24/7 basis. The example output queue of QGPL/EXAMPLE is specified but can be amended. To do this, use option 3=Copy against the example rule and then specify the new queue and library details for your own environment. Power/System i Rules 3-55

62 Device Rules Check Output Queue is Ready Checks that the specified output queue is in a Ready status on a 24/7 basis. The example output queue of QGPL/EXAMPLE is specified but can be amended. To do this, use option 3=Copy against the example rule and then specify the new queue and library details for your own environment. Spoolfiles >=2000 pages Checks that no one spoolfile on the specified output queue at any time contains more than 2000 pages. The example output queue of QGPL/EXAMPLE is specified but can be amended. To do this, use option 3=Copy against the example rule and then specify the new queue and library details for your own environment. Example Device Rules These rules are supplied by default under the EXAMPLE Device Rule Group, and are available within: Level 1 - Message Management Suite Level 2 - Systems Operations Suite Level 3 - Advanced Automation Suite Level 4 - Operations Center Suite Ethernet Lines OK? This rule checks to ensure that all ethernet line connections are active and are varied on. This rule operates on a 24/7 basis. Check TAP01 Varied on This rule checks to ensure that the Tape Drive TAP01 is varied on and available on a 24/7 basis. The device name can be overridden if required using option 2=Change Halcyon Templates

63 Job Queue Rules Example Job Queue Rules These rules are supplied by default under the EXAMPLE Job Queue Rule Group, and are available within: Level 1 - Message Management Suite Level 2 - Systems Operations Suite Level 3 - Advanced Automation Suite Level 4 - Operations Center Suite Check Job Queue is Ready Checks to ensure that the specified job queue is ready and available on a 24/7 basis. The example job queue QGPL/EXAMPLE is specified but can be amended. To do this, use option 3=Copy against the example rule and then specify the new queue and library details for your own environment. This rule uses an action schedule that sends an alert to the local console should the job queue fail to be in an active status. If the Job Queue is not ready a command is also sent as part of the action schedule to automatically start the job queue. Backlog of Jobs >=10 Checks to ensure that there are no more than 10 jobs on the specified job queue at one time. The rule operates on a 24/7 basis. The example job queue QGPL/EXAMPLE is specified but can be amended. To do this, use option 3=Copy against the example rule and then specify the new queue and library details for your own environment. Jobs on Queue >=30 minutes Checks to ensure that no job has been waiting on the specified job queue for a period of no more than 30 minutes. This rule operates on a 24/7 basis. The example job queue QGPL/EXAMPLE is specified but can be amended. To do this, use option 3=Copy against the example rule and then specify the new queue and library details for your own environment. Power/System i Rules 3-57

64 Performance Rules Example Performance Rules These rules are supplied by default under the EXAMPLE Performance Rule Group, and are available within: Level 2 - Systems Operations Suite Level 3 - Advanced Automation Suite Level 4 - Operations Center Suite *ASP Any ASP Percentage Used 90% or Higher Checks to ensure that the used storage of any Auxiliary Storage Pool (ASP) is no more than 90%. This rule operates on a 24/7 basis. *JOB Jobs Using 50% of CPU or Higher Checks to ensure that no one job on the system is using or exceeding more than 50% of the available CPU resource and operates on a 24/7 basis. *JOB Jobs using 600 seconds CPU time or higher Checks for any jobs that are using the equivalent of or higher than 600 seconds (10 minutes) of CPU time to complete and operates on a 24/7 basis *JOB Interactive jobs with 5 seconds response time Checks to ensure that any interactive jobs have a response time of less than 5 seconds and operates on a 24/7 basis. *JOB Jobs with run priority of less than 20 This rule reports on any jobs that are running in the system with a priority of less than 20. Any job with a priority of less than 20 has the potential to significantly reduce the response time thus affecting system performance. This rule operates on a 24/7 basis. *JOB Any job in a status of lock wait Checks for any job that is reported in being in a status of lock wait. A lock wait occurs when one transaction (composed of one or more SQL statements) tries to acquire a lock whose mode conflicts with a lock held by another transaction. Excessive lock wait time often translates into poor response time. This rule operates on a 24/7 basis and is suspended for 3 minutes when triggered Halcyon Templates

65 Uses an action that sends an alert to the local console should any job be reported as being in a status of lock wait. A second action sends the details of the job user, number and name and job lock status to print. *POOL Pools with main storage greater than 1000MB Checks for any pool with a main storage capacity of greater than 1000Mb. Any pool that uses a high amount of storage can have potential performance issues. This rule operates on a 24/7 basis. *SUBSYSTEM Check if Subsystem QSPL is *INACTIVE Checks that the spooled file subsystem QSPL is always active. Subsystem QSPL is used for processing the printer writer programs and must be active when printer writer programs are active. This rule operates on a 24/7 basis. *SYSTEM Alert if 1000 or more active jobs Checks to see if there more than 1000 active jobs running in the system at any one time. A large number of active jobs running at the same time can slow down the system. This rule operates on a 24/7 basis. *SYSTEM System ASP is 95% or higher Checks to see if the System Auxiliary Storage Pool is operating at 95% of its capacity or higher. This rule operates on a 24/7 basis. *SYSTEM System level CPU% use of 95% or higher Checks the system CPU to ascertain that the percentage of usage does not exceed 95%. This rule operates on a 24/7 basis. *SYSTEM Database I/O percentage of 95% or higher Interrogates the system database to check that concurrent inbound and outbound transactions are not running at 95% or greater than capacity. This rule operates on a 24/7 basis. *SYSTEM System response time 5 seconds or higher Checks that system response time is being kept to under 5 seconds. This rule operates on a 24/7 basis. Power/System i Rules 3-59

66 User Profile Rules Example User Profile Rules These rules are supplied by default under the EXAMPLE User Profile Rule Group, and are available within: Level 2 - Systems Operations Suite Level 3 - Advanced Automation Suite Level 4 - Operations Center Suite Disabled User Profiles Checks the user profiles for any users that have been disabled from the system.this rule operates on a 24/7 basis. Users with *SECADM Special Authority Checks user profiles to see if any have the *SECADM Special Authority assigned. This authority is usually reserved for the Security Administrator. This rule operates on a 24/7 basis. Distribution Queue Rules Example Distribution Queue Rules These rules are supplied by default under the EXAMPLE Distribution Queue Rule Group, and are available within: Level 2 - Systems Operations Suite Level 3- Advanced Automation Suite Level 4 - Operations Center Suite Queue Depth >5 Checks the specified distribution queue to ascertain that the distribution queue depth is less than 5 at any given time. This rule operates on a 24/7 basis and is in it ally set up to report on the example distribution queue; QSMPTQ. Queue Not Ready Checks the status of a specified distribution queue to ensure that it is in a ready status. This rule operates on a 24/7 basis and is set up to report on all distribution queues within the system Halcyon Templates

67 Inactivity Rules Object Rules Example Inactivity Rules These rules are supplied by default under the EXAMPLE Inactivity Rule Group and are available within: Level 2 - Systems Operations Suite Level 3 - Advanced Automation Suite Level 4 - Operations Center Suite Interactive Job Inactive >60 minutes Checks all interactive jobs for a period of inactivity in excess of 60 minutes. This rule operates on a 24/7 basis. Example Object Rules These rules are supplied by default under the EXAMPLE Object Rule Group and are available within: Level 2 - Systems Operations Suite Level 3 - Advanced Automation Suite Level 4 - Operations Center Suite *DTAARA - Data Area Value is Y Checks that the specified data area (DTAARA) exists on the system. The default setting for this rule is to check for the Test DTAARA and library QGPL. This rule operates on a 24/7 basis. *DIR - Directory contains *STMFs Checks that the specified directory path exists and contains at least one *STMF object. This rule operates on a 24/7 basis. *DTAQ - Data Queue with >10 items in queue Checks a named Data Queue for greater than 10 items waiting to be processed. The default setting for this rule is to check the data queue named DTAQ with a library of QGPL. This rule operates on a 24/7 basis. Power/System i Rules 3-61

68 *FILE - File exists and contains records Checks for the existence of a named file in a specified library and that it contains records. The default setting for this rule is to check for the existence of a file named File in the library QGPL and that the records contained within are greater than zero. This rules operates on a 24/7 basis. *OBJECT - Library does not exist Checks for the existence of a specified library. The default setting for this rule is check for the existence of TESTLIB in the library QSYS. This rule operates on a 24/7 basis. *STMF - File exists with > 100 records Checks a specified object path for the existence of a stream file (STMF) containing more than 100 records and raises an alert if found. This rule operates on a 24/7 basis. Restricted Task Rules Example Restricted Task Rules These rules are supplied by default under the EXAMPLE Restricted Task Rule Group, and are available within: Level 2 - Systems Operations Suite Level 3 - Advanced Automation Suite Level 4 - Operations Center Suite Note: Note: All monthly save completion messages Checks for any message type *COMP in the restricted task group MONTHLYSAV. To use this example rule without amendment, the task group; MONTHLYSAV must exist on your system. All monthly save completion inquiry messages This rule checks for any message type *INQ in the restricted task group MONTHLYSAV. To use this example rule without amendment, the task group; MONTHLYSAV must exist on your system. All Restricted State messages This rule checks for any restricted state message in any restricted state task group Halcyon Templates

69 Alerts to send via SMS This rule checks for the following messages and when received, sends a message by SMS to the device listed as OPSPHONE1. Once sent, the alert is closed. ALL messages received in file QCPFMSG (with the exception of messages CPF2456 and CPF5908) HRT8006 received in file HRTMSGF HRT8008 received in file HRTMSGF HRT8010 received in file HRTMSGF HRT8019 received in file HRTMSGF Audit Journal Rules These rules are supplied by default within Audit Journal Rules, and are available within: Level 2 - Systems Operations Suite Level 3 - Advanced Automation Suite Level 4 - Operations Center Suite Authority Failure Audit Journal Rules These rules are supplied by default under the AUTFAIL Audit Journal Rule Group and are designed to check for any type of authority failure within the system. AF: Authority failures System-wide rule that checks for any audit journal type AF (Authority Failures) entry across all jobs, objects and users. CV: Connection ended abnormally System-wide rule that checks for any audit journal type CV (Connection Ended Abnormally - Specific Reason Code E) entry across all jobs and actioning users. CV: Connection rejected System-wide rule that checks for any audit journal type CV (Connection Rejected - Specific Reason Code R) entry across all jobs and actioning users. Power/System i Rules 3-63

70 DI: Authority failures System-wide rule that checks for any audit journal type DI (Directory Server - Specific Reason Code L) across all jobs and actioning users. LDAP operation type - AF (Authority Failure). The authority failure reason code of L represents the LDAP operation. DI: Password failures System-wide rule that checks for any audit journal type DI (Password Failures - Specific Reason Code L) across all jobs and actioning users. LDAP operation type - PW (Password). The authority failure reason code of L represents the LDAP operation. GR: Function registration operations System-wide rule that checks for any audit journal type GR (Generic Record - Specific Reason Code F) across all jobs and actioning/affected users. The reason code F represents function registration operations. KF: Incorrect password was entered System-wide rule that checks for any audit journal entry type KF (Key Ring File - Specific Reason Code P) across all jobs, actioning users, certificate, key ring and trusted root operations and all named objects. The reason code P represents an incorrect password. IP: Authority failure for an IPC request System-wide rule that checks for any audit journal entry type IP (Interprocess Communication - Specific Reason Code F) across all jobs, actioning users and owners. The reason code F represents an authority failure. PW: Invalid passwords System-wide rule that checks for any audit journal entry type PW (Invalid Password) across all jobs and users (both actioning and affected). VC: Connection rejected incorrect password System-wide rule that checks for any audit journal entry type VC (Starting or Ending a Connection - Specific Reason Code R) across all jobs, job users, actioning users, server name, computer name and rejection reason. The reason code R represents a rejection. VO: Unsuccessful validation System-wide rule that checks for any audit journal entry type VO (Validation List Actions - Specific Reason Code U) across all jobs, job users, actioning users, validation list and library names. The reason code U represents the unsuccessful verification of a validation list Halcyon Templates

71 VN: Network logon rejected System-wide rule that checks for any audit journal entry type VN (Logging on and off the Network - Specific Reason Code R) across all jobs, users, server and computer names, user privileges and rejection reasons. The reason code R represents a rejected logon VP: Incorrect network password System-wide rule that checks for any audit journal entry type VP (Network Password Error - Specific Reason Code P) across all jobs, users, server and computer names. The reason code P represents a password error. X1: Delegate of identity token failed System-wide rule that checks for any audit journal entry type X1 (Identity Token - Reason Code F) across all jobs, users and failed request reasons. The reason code F represents Delegate of identity token failed. Example Audit Journal Rules These rules are supplied by default under the EXAMPLE Audit Journal Rule Group. System Value Changes System-wide rule that checks for occurrences of audit journal type SV (System Value Changes). System Service Tools Accessed System-wide rule that checks for occurrences of audit journal type ST (Use of Service Tools). An omission rule exists for any service tools accessed by job user QSRVAGT. Job Description User Changed System-wide rule that checks for occurrences of audit journal type JD (Change to user parameter of job description). New Libraries created on system System-wide rule that checks for occurrences of audit journal type CO (Create Objects) against object type *LIB. Programs Changed to adopt QSECOFR user authority System-wide rule that checks for occurrences of audit journal type PA (Program changed to adopt authority), against Object Owner QSECOFR. Power/System i Rules 3-65

72 Track use of PWRDWNSYS command System-wide rule that checks for occurrences of audit journal type CD (Command String Audit) against command PWRDWNSYS. Security Audit Rules This group of rules is designed to check for the breach or attempt to change any security related aspect of the IBM i. AD: Auditing of a DLO changed with CHGDLOAUD System-wide rule that checks for occurrences of audit journal type AD (Auditing Changes) against a Document Library Object (DLO) changed with the CHGDLOAUD command. AD: Object changed with CHGOBJAUD or CHGAUD System-wide rule that checks for occurrences of audit journal type AD (Auditing Changes) against a Document Library Object (DLO) changed with the CHGOBJAUD or CHGAUD commands. AD: Scan attribute change by CHGATR or API System-wide rule that checks for occurrences of audit journal type AD (Auditing Changes) involving a scan attribute change by CHGATR or API. AD: Auditing for user changed with CHGUSRAUD System-wide rule that checks for occurrences of audit journal type AD (Auditing Changes) against any user changed with the CHGUSRAUD command. X1: Delegate of identity token successful System-wide rule that checks for occurrences of audit journal type X1 (Identify Token), activated when the delegation of an identity token within Enterprise Identity Mapping (EIM) is successful. X1: Get user from identity token successful System-wide rule that checks for occurrences of audit journal type X1 (Identify Token), activated when a user has been successfully recognized from an identity token within Enterprise Identity Mapping (EIM). AU: EIM configuration change System-wide rule that checks for occurrences of audit journal type AU (Attribute Changes), against any Enterprise Identity Management (EIM) configuration changes to any attributes Halcyon Templates

73 CA: Changes to autl or object authority System -wide rule that checks occurrences of audit journal type CA (Authority Changes), for changes of authority against any jobs, users and objects. Accompanying Omit rules mean that alerts are not raised from this rule if the authority change affects an *STMF object or for any *DIR changes actioned by user QMSF. CP: User profile changed by QSYRESPA API System-wide rule that checks occurrences of audit journal type SP (User profile changed, created or restored), for any changes made to any user profile. CQ: A *CRDQ object was changed System-wide rule that checks for occurrences of audit journal type CQ (Change of *CRQD object), of a change against any *CRQD object. CV: Connection established System-wide rule that checks occurrences of audit journal type CV (Connection Verification) for any connection established via any protocol. CV: Connection ended normally System-wide rule that checks occurrences of audit journal type CV (Connection Verification) for any connection that is ended normally. CV: Connection rejected System-wide rule that checks occurrences of audit journal type CV (Connection Verification) for any connection that is rejected. CY: Access Control function System-wide rule that checks occurrences of audit journal type CY (Cryptographic configuration) for any instance of a cryptographic coprocessor access control function. CY: Facility Control function System-wide rule that checks occurrences of audit journal type CY (Cryptographic configuration) for any instance of a cryptographic coprocessor facility control function. CY: Master Key function System-wide rule that checks occurrences of audit journal type CY (Cryptographic configuration) for any instance of a cryptographic coprocessor master key function. Power/System i Rules 3-67

74 DI: Audit change System-wide rule that checks occurrences of audit journal type DI (Directory Server) for any Lightweight Directory Access Protocol (LDAP) audit attribute changes. DI: Successful bind System-wide rule that checks occurrences of audit journal type DI (Directory Server) for any Lightweight Directory Access Protocol (LDAP) successful bind operations. DI: Authority change System-wide rule that checks occurrences of audit journal type DI (Directory Server) for any Lightweight Directory Access Protocol (LDAP) authority changes. DI: Password change System-wide rule that checks occurrences of audit journal type DI (Directory Server) for any Lightweight Directory Access Protocol (LDAP) password changes. DI: Ownership change System-wide rule that checks occurrences of audit journal type DI (Directory Server) for any Lightweight Directory Access Protocol (LDAP) changes of ownership. DI: Policy change System-wide rule that checks occurrences of audit journal type DI (Directory Server) for any Lightweight Directory Access Protocol (LDAP) policy changes. DI: Successful unbind System-wide rule that checks occurrences of audit journal type DI (Directory Server) for any Lightweight Directory Access Protocol (LDAP) successful unbind operations. DO: Object deleted not under commit control System-wide rule that checks occurrences of audit journal type DO (Delete Object) for any object that is deleted but not under commitment control. DO: Pending object delete was committed System-wide rule that checks occurrences of audit journal type DO (Delete Object) for any pending object delete that was committed Halcyon Templates

75 DO: Pending object create rolled back System-wide rule that checks occurrences of audit journal type DO (Delete Object) for any pending object creation that was rolled back. DO: Object delete is pending System-wide rule that checks occurrences of audit journal type DO (Delete Object) for any pending object deletions. DO: Pending object delete rolled back System-wide rule that checks occurrences of audit journal type DO (Delete Object) for any pending object deletions that have been rolled back. DS: DST QSECOFR password reset request System-wide rule that checks occurrences of audit journal type DS (DST Security Password Reset) for any reset of service tools user password. DS: DST profile changed System-wide rule that checks occurrences of audit journal type DS (DST Security Password Reset) for any user profile changed to a service tools user ID. EV: Add System-wide rule that checks occurrences of audit journal type EV (System Environment Variables) for the addition of any system environment variables. EV: Change System-wide rule that checks occurrences of audit journal type EV (System Environment Variables) for any system environment variables that have been changed. EV: Delete System-wide rule that checks occurrences of audit journal type EV (System Environment Variables) for the deletion of any system environment variables. EV: Initialize environment variable space System-wide rule that checks occurrences of audit journal type EV (System Environment Variables) for any initialization of an environment variable space. GR: Exit program added System-wide rule that checks occurrences of audit journal type GR (Generic Record) for any added exit programs. Power/System i Rules 3-69

76 GR: Exit program removed System-wide rule that checks occurrences of audit journal type GR (Generic Record) for any exit programs that have been removed. GR: Function registration operation System-wide rule that checks occurrences of audit journal type GR (Generic Record) for any function registration operations. GR: Exit program replaced System-wide rule that checks occurrences of audit journal type GR (Generic Record) for any exit program that has been replaced. GS: Socket descriptor given to another job System-wide rule that checks occurrences of audit journal type GS (Socket Descriptor Given To Another Job) for any instances of a socket descriptor being given to another job. GS: Receive descriptor System-wide rule that checks occurrences of audit journal type GS (Socket Descriptor Given To Another Job) for any instances of a socket descriptor being received GS: Unable to use descriptor System-wide rule that checks occurrences of audit journal type GS (Socket Descriptor Given To Another Job) for any instances of being unable to use a descriptor. IP: Create an IPC object System-wide rule that checks occurrences of audit journal type IP (Interprocess Communications) for any instances of an Interprocess Communication (IPC) object being created. IP: Delete an IPC object System-wide rule that checks occurrences of audit journal type IP (Interprocess Communications) for any instances of an Interprocess Communication (IPC) object being deleted. IP: Get an IPC object System-wide rule that checks occurrences of audit journal type IP (Interprocess Communications) for any instances of an Interprocess Communication (IPC) object being retrieved Halcyon Templates

77 JD: User parameter of a jobd was changed System-wide rule that checks occurrences of audit journal type JD (Change User Parameter Of Job Description) for any instances of a user parameter of a job description being changed. KF: Certificate operation System-wide rule that checks occurrences of audit journal type KF (Key Ring File) for any instances of any certificate operation being performed. KF: Key ring file operation System-wide rule that checks occurrences of audit journal type KF (Key Ring File) for any instances of any key ring file operation being performed. KF: Trusted root operation System-wide rule that checks occurrences of audit journal type KF (Key Ring File) for any instances of a trusted root operation being performed. NA: Network attribute was changed System-wide rule that checks occurrences of audit journal type NA (Network Attribute Changed) for any instances of network attributes that have been changed. OW: Ownership object was changed System-wide rule that checks occurrences of audit journal type OW (Object Ownership Changed) for instances of a change of object owner. PA: Program changed to adopt owner authority System-wide rule that checks occurrences of audit journal type PA (Program Changed To Adopt Authority) for instances of a program that has been changed to adopt the owner s authority. PG: Primary group for an object was changed System-wide rule that checks occurrences of audit journal type PG (Change Of An Object s Primary Group) for any instances of a change in an object s primary group. PS: Profile swap during pass-through System-wide rule that checks occurrences of audit journal type PS (Profile Swap) for any instances of a profile swap taking place during pass-through. PS: User ended work on behalf of another System-wide rule that checks occurrences of audit journal type PS (Profile Swap) for any instances of a user profile ending work on behalf of another user with which a relationship is held. Power/System i Rules 3-71

78 PS: Profile handle generated through API System-wide rule that checks occurrences of audit journal type PS (Profile Swap) for any instances of a user profile handle being generated through API. PS: All profile tokens were invalidated System-wide rule that checks occurrences of audit journal type PS (Profile Swap) for all user profile tokens being invalidated. PS: Maximum profile tokens generated System-wide rule that checks occurrences of audit journal type PS (Profile Swap) for the maximum number of profile tokens being generated. PS: Profile token generated for user System-wide rule that checks occurrences of audit journal type PS (Profile Swap) for any instances of a profile token being generated for a user. PS: Profile tokens for user removed System-wide rule that checks occurrences of audit journal type PS (Profile Swap) for any instances of profile tokens for users being removed. PS: Work started on behalf on another user System-wide rule that checks occurrences of audit journal type PS (Profile Swap) for any instances of a user profile starting work work on behalf of another user with which a relationship is held. PS: User profile authenticated System-wide rule that checks occurrences of audit journal type PS (Profile Swap) for any instances of a user profile being authenticated. SE: Subsystem routing entry was changed System-wide rule that checks occurrences of audit journal type SE (Subsystem Routing Entry Changed) for any instances of a subsystem routing entry being changed. SO: Add entry System-wide rule that checks occurrences of audit journal type SO (Server Security User Information Actions) for any instances of an entry being added to Server Security User Information. SO: Change entry System-wide rule that checks occurrences of audit journal type SO (Server Security User Information Actions) for any instances of an entry being changed within Server Security User Information Halcyon Templates

79 SO: Remove entry System-wide rule that checks occurrences of audit journal type SO (Server Security User Information Actions) for any instances of an entry being removed from Server Security User Information. SV: System value was changed System-wide rule that checks occurrences of audit journal type SV (System Value Changed) for any instances of a system value being changed. SV: Service attributes were changed System-wide rule that checks occurrences of audit journal type SV (System Value Changed) for any instances of service attributes being changed. SV: Change to system clock System-wide rule that checks occurrences of audit journal type SV (System Value Changed) for any instances of a change being made to the system clock. SV: Change to option System-wide rule that checks occurrences of audit journal type SV (System Value Changed) for any instances of a change being made to a system value option. SV: Change to system-wide journal attribute System-wide rule that checks occurrences of audit journal type SV (System Value Changed) for any instances of a change being made to a system-wide journal attribute. VA: Access control list changed successfully System-wide rule that checks occurrences of audit journal type VA (Changing an Access Control List) for any instances of an access control list being successfully changed. VA: Change to access control list failed System-wide rule that checks occurrences of audit journal type VA (Changing An Access Control List) for any instances of an access control list change attempt failing. VO: Successful verify validation list entry System-wide rule that checks occurrences of audit journal type VO (Validation List Actions) for any instances of the successful verification of a validation list entry. Power/System i Rules 3-73

80 VU: Group record changed System-wide rule that checks occurrences of audit journal type VU (Changing a Network Profile) for any instances of a group record being changed. VU: User profile global information changed System-wide rule that checks occurrences of audit journal type VU (Changing a Network Profile) for any instances of the global information of a user profile being changed. VU: User record changed System-wide rule that checks occurrences of audit journal type VU (Changing a Network Profile) for any instances of a user record being changed. X0: Network authentication System-wide rule that checks occurrences of audit journal type X0 (Network Authentication) for any instances of network authentication failures. Service Audit Rules This group of rules is designed to check for the use of any IBM i Service Commands and API s. ST: Service Tool Used System-wide rule that checks for occurrences of audit journal type ST (Service Tool Used - Specific Reason Code A), across all jobs, users, service tools, objects and SST user profile. The reason code A represents a service record. VV: Changing Service Status System-wide rule that checks for any occurrences of audit journal type VV (Changing Service Status) across all jobs, users, servers, computers, statuses, service and return codes. SYSMGT Audit Rules This group of rules is designed to check for any changes to IBM i System Management Tasks. DI: Configuration changes System-wide rule that checks for any occurrences of audit journal type DI (Directory Server - Specific Reason Code L) across all jobs, users and LDAP Operation type of Configuration Change. The reason code L represents LDAP operation Halcyon Templates

81 DI: Create instance System-wide rule that checks for any occurrences of audit journal type DI (Directory Server - Specific Reason Code L) across all jobs, users and LDAP Operation type of Create Instance. The reason code L represents LDAP operation. DI: Delete instance System-wide rule that checks for any occurrences of audit journal type DI (Directory Server - Specific Reason Code L) across all jobs, users and LDAP Operation type of Delete Instance. The reason code L represents LDAP operation. DI: Replication management System-wide rule that checks for any occurrences of audit journal type DI (Directory Server - Specific Reason Code L) across all jobs, users and LDAP Operation type of Replication Management. The reason code L represents LDAP operation. SM: Systems management changes System-wide rule that checks for any occurrences of audit journal type SM (Systems management changes) across all jobs, users, access types, sequence numbers and message IDs. VL: Account limit exceeded System-wide rule that checks for any occurrences of audit journal type VL (Account limit exceeded) across all jobs, users, servers and computers. Power/System i Rules 3-75

82 C HAPTER 4SPECIALIST TEMPLATES CHAPTER Overview Halcyon Software supplies templates with each of its product suites that can be used to monitor application specific areas within the following specialist environments: SAP JDE Lawson M3 (Movex) MAXAVA (NOMAX) Quick/EDD MISYS Midas Plus In most cases, the templates supplied can be used immediately upon completion of installation, but there may be instances where you need to change rule properties to match those of your own environment. This can be done by taking option 2=Change against the rule and making the required changes. Similarly, should you require multiple rules for different message queues or devices and so on you can use option 3=Copy against the rule and then make the required amendments. Note: Actions Unless otherwise specified, all template rules are implemented with a default action schedule which sends an alert message to your local console (option 10=Message Console from the main menu). Should you wish to amend this option, take option 2=Change against the action within the rule and make the amendments as required. Please refer to the user reference guide for your Halcyon solution for details of actions that may be applied to rules Halcyon Templates

83 Installation of Customized Environment Note: Follow these instructions to install templates to a customized environment. 1 Install the Halcyon solution, using the appropriate installation guide. 2 Once successfully installed, log into the environment to which you wish to apply the customized template, for example, HALPROD/HALCYON. 3 From the command line run ENDMON and press F4. Follow the prompts to complete the ending of the monitors. The installation of the customized environment fails if the monitors are not stopped. 4 From the command line type CSTENV and press F4. 5 Type the required authorization code for the template you wish to apply and press Enter. Note: Each customized environment requires an authorization code. Please contact technicalservices@halcyonsoftware.com or your local Halcyon office or reseller for details on how to obtain this code. The Customized environment is now installed. 6 From the main menu of your Halcyon solution, select option 5=Work with Rules. The template rules applicable to the customized environment that you installed can be found in the listed queue and rule groups. Default action schedules are installed and additionally, where appropriate, changes to system defaults may also be made. Specialist Templates 4-77

84 SAP Customization Template Once applied, via entry of the applicable customization code, the following customization templates are available for SAP application monitoring: Message Queue Rules QHST Message Queue This rule group contains two SAP customization rules that can be used to monitor system events that have been logged. The rules within this group are run every 60 seconds with an immediate alert level (message severity) of 50. SAP:SAPSTART started by 0600 Checks that the SAPSTART script has been run by checking for message ID CPF1124 with job parameter SAPSTART being present in message queue QHST. The SAPSTART program starts the R/3 services and processes associated with each instance. An alert is generated if this message is not received between 02:00 and 06:00 on a daily basis. SAP:Clean BACKUP1 obtained Checks that job Backup1 has been secured by checking for message ID CPF3701 being present in message queue QHST and alerts if it is not received and the number of objects not saved is greater than zero. An equivalent omission criteria ensures that an alert is not generated if the number of objects missing from the backup is zero. An alert is generated if this message is not received between 00:00 and 02:00 on a daily basis. QSYSOPR Message Queue This rule group contains two SAP customization rules that can be used to monitor important system messages that appear on the QSYSOPR message queue. The rules within this group are run every 60 seconds with an immediate alert level (message severity) of 50. SAP:All inquiry messages An alert is raised for each inquiry message that is generated on the QSYSOPR message queue. SAP:High severity messages >=90 An alert is raised for each message generated on the QSYSOPR message queue with a severity level of 90 or greater Halcyon Templates

85 Device Rules This rule is available within the Device Rules > SAP Monitoring Rule Group. The rule within this group is run every 60 seconds. SAP: Ethernet line OK? Checks to ensure that all ethernet line connections are active and are varied on. Performance Rules These rules are available within the Performance Rules > SAP Monitoring Rule Group. SAP:Check for excessive CPU load on system Checks to ensure that the average of the sixty second interval during which the processing units were in use is no more than 90%. SAP:Verify subsystem R3_03 is active Checks to ensure that the subsystem R3_03 is continuously active on a 24/7 basis. SAP R/3 work processes on the iseries server are jobs within the R3_nn subsystem that actually perform the work. SAP:Check Memory for ACT_INEL Checks the transition of threads from active to an ineligible condition. An alert is raised if the rate of transition of threads per minute is higher than or equal to 0.1. SAP:Check Memory for WAIT_INEL Checks the transition of threads from wait to an ineligible condition. An alert is raised if the rate of transition of threads per minute is higher than or equal to 0.1. SAP:ASP1 <20% available Checks that ASP 1 has at least 20 percent of its capacity available for use. SAP:Ensure job DISP_WORK is active in R3_03 Checks that the specified job DISP_WORK is active in subsystem R3_03 and generates an alert if an active job is not found. Alerts can still be generated by this rule when the R3_03 subsystem is inactive. Specialist Templates 4-79

86 SAP:Ensure job ICMAN is active in R3_03 SAP's J2EE Engine is activated by the ICMAN (ICM monitor) service that forwards requests to the J2EE server. This rule checks that the specified job ICMAN is active in subsystem R3_03 and generates an alert if an active job is not found. Alerts can still be generated by this rule when the R3_03 subsystem is inactive. SAP:Ensure job MSG_SERVER is active in R3_03 The task of the message server is to inform all the servers (instances) belonging to an SAP System of the existence of the other servers. This rule checks that the specified job MSG_SERVER is active in subsystem R3_03 and generates an alert if an active job is not found. Alerts can still be generated by this rule when the R3_03 subsystem is inactive. SAP:Ensure job SAPSTART is active in R3_03 The SAPSTART program starts the R/3 system. This rule checks that the specified job SAPSTART is active in subsystem R3_03 and generates an alert if an active job is not found. Alerts can still be generated by this rule when the R3_03 subsystem is inactive. SAP:Ensure job GWRD is active in R3_03 Gateway read (GWRD) is the main process in the gateway system. This rule checks that the specified job GWRD is active in subsystem R3_03 and generates an alert if an active job is not found. Alerts can still be generated by this rule when the R3_03 subsystem is inactive. SAP:Ensure job QXDAEDRSQL is active in QSYSWRK The server job QXDAEDRSQL that runs on a remote database host must be active to accept new database connections from clients. QSYSWRK is a special IBM subsystem that is used to run i5/os server jobs, including many of the system's TCP/IP servers. This rule checks that the specified job QXDAEDRSQL is active in subsystem QSYSWRK and generates an alert if an active job is not found. Alerts can still be generated by this rule when the R3_03 subsystem is inactive. SAP:Check for SEMW, MSGW & LCKW job within R3_03 This rule checks for any job on subsystem R3_03 in a SEMW (Semaphore Wait), MSGW (Message Wait), or LCKW (Lock Wait) status Halcyon Templates

87 JDE Customization Template Once applied, via entry of the applicable customization code, the following customization templates are available for JDE application monitoring: Message Queue Rules JDE World Messages This rule group contains a single JDE customization rule that can be used to monitor JDE World messages. The rules within this group are run every 60 seconds. Note: No Global Exclusions See the No Global Exclusions Defined section for more information. All JDE World Messages Checks for the existence of any message received in the message queue JDEWORLD within library QUSRSYS. Output Queue Rules JDEPRT01 Output Queue Rules This rule group contains a single JDE customization rule that can be used to monitor JDE output queues. The rules within this group are run every 60 seconds. JDEPRT01 queue released with JDEPRT01 started Checks that the output queue JDEPRT01 is in a status of released and is attached to writer JDEPRT01 which is started. The rule is suspended for 30 minutes if an alert is generated. Device Rules JDE Rules This rule group contains two JDE customization rules that can be used to monitor devices. The rules within this group are run every 60 seconds. Ethernet lines active Checks to ensure that all ethernet line connections are active and are varied on. The rule is suspended for 30 minutes if an alert is generated. Specialist Templates 4-81

88 TAP01 not active/varied on Checks to ensure that the Tape Drive TAP01 is varied on. The device name can be overridden if required using option 2=Change against the rule. The rule is suspended for 30 minutes if an alert is generated. Job Queue Rules JDE Daytime JOBQ This rule group contains two JDE customization rules that can be used to monitor the daytime job queue. The rules within this group are run every 60 seconds. JDE Daytime queue released 0800 to 1800 hours Checks to ensure that the job queue JDEDAYQ is in a released status between the hours of 0800 and 1800 on a daily basis. The rule is suspended for 30 minutes if an alert is generated. JDE Daytime queue held 1801 to 0759 hours Checks to ensure that the job queue JDEDAYQ is in a held status between the hours of 1801 and 0759 on a daily basis. The rule is suspended for 30 minutes if an alert is generated. JDE Nighttime JOBQ This rule group contains two JDE customization rules that can be used to monitor the nighttime job queue. The rules within this group are run every 60 seconds. JDE Night queue held 0800 to 1800 hours Checks to ensure that the job queue JDENIGHTQ is in a held status between the hours of 0800 and 1800 on a daily basis. The rule is suspended for 30 minutes if an alert is generated. Night queue released 1801 to 0759 hours Checks to ensure that the job queue JDENIGHTQ is in a released status between the hours of 1801 and 0759 on a daily basis. The rule is suspended for 30 minutes if an alert is generated Halcyon Templates

89 Performance Rules JDE Rules This rule group contains eight JDE customization rules that can be used to monitor JDE application specific performance statistics. System ASP>=80% Checks that the percentage of the system storage pool does not exceed more than 80% of the available space. The rule is suspended for 30 minutes if an alert is generated. Excessive sustained overall CPU% Checks that the percentage usage of the CPU does not exceed 90%. An alert is generated if the usage of the CPU is found to exceed 90% on 5 occasions during a 6 minute period. Response times > 2 seconds Checks that the average interactive response time during the previous 5 Performance Monitor base processing periods does not exceed 2 seconds. The rule is suspended for 30 minutes if an alert is generated. Subsystem QSERVER inactive Checks that subsystem QSERVER is constantly active. The QSERVER subsystem holds jobs related to communication and TCP/IP servers. The rule is suspended for 30 minutes if an alert is generated. Subsystem QSYSWRK inactive Checks that subsystem QSYSWRK is constantly active. The QSYSWRK subsystem holds system-related jobs. Jobs in this subsystem are generally pre-start jobs.the rule is suspended for 30 minutes if an alert is generated. Subsystem QBATCH inactive Checks that subsystem QBATCH is constantly active. The QBATCH subsystem handles all of the OneWorld batch processing. The rule is suspended for 30 minutes if an alert is generated. Subsystem JDEB733 inactive Checks that subsystem JDEB733 is constantly active. The JDEB733 subsystem handles the data base source. The rule is suspended for 30 minutes if an alert is generated. Specialist Templates 4-83

90 Subsystem NIGHTBATCH inactive Checks that subsystem NIGHTBATCH is constantly active. The NIGHTBATCH subsystem is used to handle night time workloads. The rule is suspended for 30 minutes if an alert is generated. User Rules JDE Rules This rule group contains a single JDE customization rule that can be used to monitor the JDE user profile. The rule within this group is run every 60 seconds. Ensure JDE user profile is enabled Checks that the user profile JDE is enabled at all times. The rule is suspended for 30 minutes if an alert is generated. Inactivity Rules JDE Rules This rule group contains two JDE customization rules that can be used to monitor types of JDE inactivity. The rules within this group are run every 60 seconds. QZDASOINIT jobs in MSGW Checks that there are no QZDASOINIT jobs in MSGW status. The prestart job QZDASOINIT performs the access to the database that OneWorld applications request. The rule is suspended for 30 minutes if an alert is generated. Inefficient QZDASOINIT job running Checks that the QZDASOINIT job is running efficiently. An alert is raised if the period of inactivity is equal to 5 minutes or longer. The rule is suspended for 30 minutes if an alert is generated Halcyon Templates

91 Object Rules JDE Rules This rule group contains a single JDE customization rule that can be used to monitor JDE objects. The rule within this group is run every 60 seconds. Track data area JDFOBJ/QJDF existence Checks that library object library QJDF exists within the JDFOBJ data area. This library contains the execution objects required by the initial sign-on program. This field is also used by JDE s PTF procedures to know where to replace the object code. The rule is suspended for 30 minutes if an alert is generated. Audit Journal Rules JDE Rules This rule group contains three JDE customization rules that can be used to monitor JDE audit journal activity. The rules within this group are constantly in force. Track changes to JDE user profile Checks audit journal entry type; CP (User profile changed, created or restored) for any changes made to the JDE user profile. System value changed Checks for any occurrence of audit journal entry type; SV (System Value Changes). JDFOBJ authority failures Checks for audit journal entry type; AF (Authority Failure) in object library JDFOBJ. JDE Windows monitoring JDE can also be run on a Windows server. Using Halcyon Network Server Suite the following can also be monitored: HTTP Monitor Enterprise One Portal is available Check any JDE webpage/portal page is available Specialist Templates 4-85

92 Service Monitor Ensure critical services are running Event Log Monitor Any errors that are in the event logs CPU, Disk and Memory Monitor Disk space usage of server and application CPU and Memory File and Folder Monitor Files are being updated (or otherwise) within a pre-determined time Ping Monitor Ping other servers Generic TCP/UDP monitor Check applications are running on specified ports Lawson Movex(M3) Templates Lawson Movex(M3) is an ERP application capable of running on a multi-tiered platform, typically IBM i and Windows servers to provide end-users with systems. The IBM i platform is served by two different architectures; RPG and JAVA. RPG is the more traditional format providing good visibility of system activity via WRKACTJOB and WRKJOBQ amongst others. JAVA is now more commonplace but makes viewing subsystem and job information more difficult when just using traditional iseries commands. Lawson provide a management console called Server View which runs on the HTTP server on the IBM i, which provides greater visibility of system activity. Both of these architectures use the same approach, usually consisting of: Database - usually installed and managed on an IBM server. User Interface - Can be any one of the following green screen interface (older systems only) Movex Explorer GUI interface Move Workplace (true HTTP interface) 4-86 Halcyon Templates

93 MOM- Movex Output Manager (Serverstream) installed on a Windows server and handles all the system output (print, fax, ) Other variations may include: ecollaborator - B2B transaction translation utility installed on a Windows server. LBI (Lawson Business Intelligence) data analysis and reporting utility Message Queue Rules QHST Message Queue This rule group contains five Lawson Movex(M3) customization rules that can be used to monitor system events that have been logged. The rules within this group are run every 60 seconds. M3: Password invalid - MSRVADM Checks if an invalid password has been entered for user MSRVADM, the Movex server administrator, by checking for message ID CPF2234 (comparator MSRVADM) being present in message queue QHST. M3: Password invalid - QSECOFR Checks if an invalid password has been entered for user QSECOFR, the IBM security administrator, by checking for message ID CPF2234 (comparator QSECOFR) being present in message queue QHST. M3: Monitor Login by MSRVADM Checks for login by user profile MSRVADM to subsystems QCTL, QINTER and QPGMR by checking for message ID CPF1124 (with these subsystem comparators) being present in message queue QHST. M3: Backup finished by 04:00 Checks that job M3BACKJOB has completed by checking for message ID CPF1164 being present in message queue QHST. This rule runs Monday to Friday only between 02:00 and 04:00 hours M3: Backup objects NOT saved Checks for message ID s CPF3701 and CPF3702 in the current environment within message queue QHST. These error messages indicate the number of objects saved and the number of objects not saved. For message ID CPF3702, the number of objects not included is also specified. An alert is generated if either of these messages are present. Equivalent omission criteria, which exist for both message ID s, ensure that an alert is not generated if the number of objects not saved is zero. Specialist Templates 4-87

94 TCP/IP Rules M3 This rule group contains a single TCP/IP customization rule that can be used to monitor Lawson M3/Movex. The rule within this group is run every 5 minutes. Note: M3: Check MOM Server Checks that the Movex Output Management Server is active and can be contacted. A success ratio of 75% (allowing 1 failure every 24 hours) determines if an alert is generated. In order to use this template rule, you must amend the example IP Address of this rule to the IP address of your MOM server. This can be done by taking option 2=Change against this rule from the Work with Rules main display. Performance Rules M3 This rule group contains seventeen Performance customization rules that can be used to monitor Lawson M3/Movex performance. M3: Verify subsystem QEJBAS51 is active This subsystem is commonly used for hosting the Websphere Application Server, the application in which Movex runs. This rule checks that this subsystem is active at all times. M3: Verify subsystem MVXLICSVR is active This subsystem hosts the Movex License Server, required for the Workplace to run. This rule checks that this subsystem is active at all times. M3: Verify subsystem MVXJVA is active The MVXJVA subsystem is the supervisor subsystem where the first instance of JVM is started. This rule checks that this subsystem is active at all times. M3: Verify subsystem MVXJVAINT is active This subsystem is used to serve interactive connections to Movex Java from the Movex Explorer client in a multiple JVM setup. This rule checks that this subsystem is active at all times. M3: Verify subsystem MVXJVAASJ is active This subsystem is responsible for running autostart jobs in a Movex Java multiple JVM setup. This rule checks that this subsystem is active at all times Halcyon Templates

95 M3: Verify subsystem MVXJVABCH is active This subsystem is used for batch job type work in a Movex Java multiple JVM setup. This rule checks that this subsystem is active at all times. M3: Verify subsystem MVXC is active This subsystem denotes an earlier version of the software (where C is the version). Older versions may be running in subsystem MVXB. This rule checks that this subsystem is active at all times. M3: Verify subsystem MVXCINT is active This subsystem is used to serve interactive connections to Movex Java from the Movex Explorer client in a multiple JVM setup. This rule checks that this subsystem is active at all times. M3: Verify subsystem MVXCASJ is active This subsystem is responsible for running autostart jobs in a Movex Java multiple JVM setup. This rule checks that this subsystem is active at all times. M3: Verify subsystem MVXCBCH is active This subsystem is used for batch job type work in a Movex Java multiple JVM setup. This rule checks that this subsystem is active at all times. M3: Verify subsystem MVXCMEX is active This subsystem runs the Movex Explorer which is the Movex GUI. This rule checks that this subsystem is active at all times. M3: Verify subsystem MVXJVAMIJ is active This subsystem is the Machine Interface subsystem which handles the communication with the Lawson M3 APIs. This rule checks that this subsystem is active at all times. M3: Ensure job MWP is active in QHTTPSVR Checks to ensure that job MWP exists and is active in subsystem QHTTPSVR. M3: Ensure at least 1 job S.* active in MVXJVA Checks to ensure there is at least one active job beginning with S. on subsystem MVXJVA. M3: Ensure at least 1 job I.* active in MVXJVAINT Checks to ensure there is at least one active job beginning with I. on subsystem MVXJVAINT. Specialist Templates 4-89

96 M3: Ensure at least 1 job A.* active in MVXJVAASJ Checks to ensure there is at least one active job beginning with A. on subsystem MVXJVAASJ. M3: Ensure at least 1 job B.* active in MVXJVABCH Checks to ensure there is at least one active job beginning with B. on subsystem MVXJVABCH. Lawson Movex(M3) Windows monitoring Note: Lawson Movex(M3) also utilizes Windows servers for several of its applications. See M3 Monitoring for more information on the M3 monitoring capabilities within Windows. MAXAVA (NOMAX) customization templates MAXAVA products offer protection for critical business information within your IBM i (i5/os, OS/400) environment, using comprehensive and secure High Availability / Disaster Recovery solutions. MAXAVA messages go to QSYSOPR by default but can go to any message queue and NOMAX in QSYS is commonly used. Message Queue Rules QSYS MAXAVA messages This rule group contains a single MAXAVA customization rule that can be used to monitor MAXAVA messages. The rules within this group are run every 60 seconds with an immediate alert level (message severity) of 90. Note: No Global Exclusions See the No Global Exclusions Defined section for more information. Catch all MAXAVA Messages This rule checks for the existence of any message received in the message message queue NOMAX within library QSYS. QSYSOPR QSYS Qsysopr Message Queue This rule group contains a three MAXAVA customization rules that can be used to monitor MAXAVA messages. The rules within this group are run every 60 seconds with an immediate alert level (message severity) of Halcyon Templates

97 MAXAVA: All Inquiry Messages Raises an alert for each inquiry message that is received on the QSYSOPR message queue. MAXAVA: High Severity Messages > 90 Raises an alert for each message generated on the QSYSOPR message queue with a severity level of 90 or greater. MAXAVA Messages (MAX*) Raises an alert for each message generated on the QSYSOPR message queue that begins with the letters MAX. TCP/IP Rules MAXAVA Check all partitions are available This rule group contains four TCP/IP customization rules that can be used to monitor MAXAVA. The rules within this group are run every 5 minutes. Note: Note: Note: Ping Production Partition 1 Checks that production partition 1 is always available by sending a PING command to remote system PRODPART1. One PING failure within a 24 hour period causes an alert to be generated. You must change the name of the remote system to replicate the actual name of your own production partition. Ping Production Partition 2 Checks that production partition 2 is always available by sending a PING command to remote system PRODPART2. One PING failure within a 24 hour period causes a alert to be generated. You must change the name of the remote system to replicate the actual name of your own production partition. Ping HA Partition 1 Checks that high availability partition 1 is always available by sending a PING command to remote system HAPART1. One PING failure within a 24 hour period causes a alert to be generated. You must change the name of the remote system to replicate the actual name of your own high availability partition. Specialist Templates 4-91

98 Note: Ping HA Partition 2 Checks that high availability partition 2 is always available by sending a PING command to remote system HAPART2. One PING failure within a 24 hour period causes a alert to be generated. You must change the name of the remote system to replicate the actual name of your own high availability partition. Performance Rules MAXAVA MAXAVA alerts and monitoring This performance rule group contains nine customization rules that can be used to monitor aspects of MAXAVA performance. Alert if ASP % used is more than 80% on any ASP Checks against the level of usage on all ASP s and raises an alert if the used percentage on any ASP is found to be greater than 80%. This rule is suspended for 60 minutes upon the generation of an alert. Note: Note: Note: Check that MAXAVASBS is started 24x7 Checks that the MAXAVA subsystem is started and available on a 24x7 basis. This rule is suspended for 10 minutes upon the generation of an alert. In addition to sending the alert to the local console, this rule has a secondary action that attempts to start this subsystem on the local system. The MAXAVA subsystem is typically configured to MAXAVASBS but could be any name as it is manually created at installation time. xxxcfg Ensure that control job exists Checks that there is at least 1 active control job (currently named xxxcfg) exists in the current environment. This rule is suspended for 10 minutes upon the generation of an alert. This rule must be manually amended for each environment where the job names that can be chosen may differ. xxx001 Ensure correct number of APPLY Groups exists Checks that there are at least 10 APPLY Groups (currently named xxx*) that exist in the current environment. A MAXAVA APPLY group is a channel used to replicate changes for the Files and/or Objects. Multiple APPLY Groups allow MAXAVA to spread the replication workload to make efficient use of system resources. This rule is suspended for 10 minutes upon the generation of an alert. This rule must be manually amended for each of the APPLY Group names that can be chosen may differ Halcyon Templates

99 Note: DFT_APYGRP Ensure default APPLY group exists Checks that the default APPLY group (DFT_APYGRP) exists in the current environment. This rule is suspended for 10 minutes upon the generation of an alert. This rule must be manually amended for each environment where the default APPLY group name that can be chosen may differ. IFSCFGCTL Ensure IFS replication is active (1 of 2) Checks that the replication job IFSCFGCTL is active within the current environment. No suspension limits apply to this rule should an alert be generated. IFSCFGRPL Ensure IFS replication is active (2 of 2) Checks that the replication job IFSCFGRPL is active within the current environment. No suspension limits apply to this rule should an alert be generated. Note: Note: MXPATROL Ensure PatrolMax job exists As a module of MAXAVA, PatrolMAX automatically detects and fixes out-ofsynch conditions. This rule checks that the PatrolMAX job is active. No suspension limits apply to this rule should an alert be generated. The PatrolMAX job may not exist as it is a monitoring tool provided by MAXAVA. If it is present and running then this rule should be applied to monitor it. This template can only be applied to the source machine on which PatrolMAX is installed. RMTJRNMON Ensure remote journaling job exists Checks that the remote journaling job RMTJRNMON is active. MAXAVA uses Remote Journaling functionality to obtain the information required to replicate changes made on a primary database to backup database copy or copies. No suspension limits apply to this rule should an alert be generated. This template can only be applied to the source machine on which the remote journaling job is running. Specialist Templates 4-93

100 Quick-EDD customization templates Note: This template is specifically designed to apply to IBM i machines running Quick- EDD. The application of this customized template also causes widespread system changes to apply. These are also documented under the section Halcyon System Changes on page A generic version of this template is currently in development and will be available shortly. Please contact technicalservices@halcyonsoftware.com for more information. Additional Installation Routine As part of the installation routine of the Quick-EDD customized template, five valid settings must be entered which the template then uses to set default rules and actions within the template. Figure 4.1 Setting Quick-EDD defaults SNDMAIL recipient Enter a valid address for the default recipient of all system messages sent via the SNDMAIL action. SNDMAIL sender Enter a valid address for the default sender all all system messages sent via the SNDMAIL action Halcyon Templates

101 SNDMAIL host IP address Enter the IP address of the default device used to send system messages using the SNDMAIL action. Source system IP address Enter the IP address of the device to be used as the default source system for use within Quick-EDD template rules. Target system IP address Enter the IP address of the device to be used as the default target system for use within Quick-EDD template rules. Note: Press Enter to apply the entered system defaults to the customization template. If non-valid data is entered, the installation of the customized Quick-EDD template fails and you must restart the process. See Installation of Customized Environment on page Message Rules Halcyon is set to monitor message queue PMEDHUSR/PMEDH for the following messages PMEDH PMEDHUSR QuickEDD Message Monitoring This rule group contains a four customizations rules that can be used to monitor specific Quick-EDD messages. The rules within this group are run every 60 seconds with an immediate alert level (message severity) of 90. No Global Exclusions Note: See the No Global Exclusions Defined section for more information. Severity 00 message ID EDH* Rule is active 24/7 for all message types and with any message severity. Alerts when any message is received on message queue PMEDH, for message ID s EDH0038, EDH0039 or beginning EDH*. Severity 40 auto messages Rule is active 24/7 for all message types with message severity 40. Alerts when any message is received on message queue PMEDH for message ID s EDH0733, EDH0735, EDH0741, EDH1408, EDH1409, EDH1424, EDH1426, EDH1432, EDH1433, EDH1434, EDH1435, EDH1438, EDH1452, EDH1453, EDH1456, EDH1457, EDH1458, EDH1466, EDH1474, EDH0635 and EDH0040. Specialist Templates 4-95

102 The action schedule for this rule performs the following actions: Sends an using SNDMAIL (24/7) Sends a local console alert (24/7) Severity 40 manual messages Rule is active 24/7 for all message types with message severity 40. Alert when any message is received on message queue PMEDH for message ID s EDH0731, EDH0801, EDH1813, EDH1832 and EDH0041 The action schedule for this rule performs the following actions: Sends an using SNDMAIL (9am - 7pm every day) Sends a local console alert (24/7) Sends an SMS to OPSPHONE1 using the following schedule: Monday Tuesday Tuesday Wednesday Wednesday Thursday Thursday Friday Friday Monday Severity 40 manual messages (escalated) Rule is active 24/7 for all message types and with message severity 40. Alerts when any message is received on message queue PMEDH for message ID EDH1745, EDH1881, EDH1883, EDH3061 and EDH3066 The action schedule for this rule performs the following actions: Sends a text to ESCLIST1 using the following schedule: Mon - Sun 24 hours TCP/IP Rules QSL Source/Target System This rule group contains two TCP/IP customization rules that can be used to monitor the QSL Source and Target systems. The rules within this group are run every 5 minutes. Note: Source System Rule is active 24/7 and sends a PING to the source system identified by the entered IP Address. A single PING failure in a 24 hour period generates an alert. The default IP Address must be amended to that of your source system in order for this rule to run successfully Halcyon Templates

103 The following actions are performed for this rule: Sends an SMS to OPSGROUP1 Sends an using SNDMAIL to default recipient Note: Target System Rule is active 24/7 and sends a PING to the source system identified by the entered IP Address. A single PING failure in a 24 hour period generates an alert. The default IP Address must be amended to that of your remote system in order for this rule to run successfully. The following actions are performed for this rule: Sends an SMS to OPSGROUP1 Sends an using SNDMAIL to default recipient Halcyon System Changes In addition to applying the customized rules, the following Halcyon system changes are also applied when selecting the Quick-EDD customized template: JOBD STRMON User profile set to PMSOFTICF Initial Library List entries added for PMEDH & PMEDHTOOLS STRSBS User profile set to PMSOFTICF STRSBSAVL User profile set to PMSOFTICF STRSUPER User profile set to PMSOFTICF Initial Library List entries added for PMEDH & PMEDHTOOLS Specialist Templates 4-97

104 User Authority PMSOFTICF granted all user authority for the following products: HMM Halcyon Message Manager HMC Halcyon Message Communicator HEM Halcyon Event Manager Halcyon System Defaults Specific Halcyon system defaults for the Quick-EDD customization have been applied: HAL MONUSRPRF = PMSOFTICF MISYS customization templates Misys is the leading application software and services provider to the financial services and healthcare industries. In financial services, their services are used by over 1,200 customers, including all of the world s top 50 banks. Message Queue Rules KAPSTATUS End of Day Monitoring This rule group contains rules to ensure that start and end messages for critical end of day Misys routines. This daily cycle is broken down into phases that complete a sequence that runs between end of business of one day and the start of business on the next and uses the following schedule: Phase C05 C10 C15 C20 C35 C40 (C41) Phase Name End of Input and File Backup Deal Reporting Maturities Input Reports Housekeeping and Statements Extract Interest End of Day Profit Transfer 4-98 Halcyon Templates

105 Phase C43 C45 C50 (C65) Phase Name Statistics Reconciliation End of Day backup End of Day Reports Phase numbers within brackets are not necessarily run each day, but only when the control system has been instructed to run them. The rules within this group are run every 60 seconds with an immediate alert level (message severity) of 90. Note: No Global Exclusions See the No Global Exclusions Defined section for more information. C05 Phase Started - End of Input & File Backup This rule looks for a message from Job MISYSEOD containing the text EQC05 Phase Started. Once received, a start watch action commences to check that the EQC05 Phase Ended message is received within 30 minutes. Additionally, a second start watch action commences to check that the entire end of day phase is completed within 4 hours. The alert is automatically closed after one minute of the message being received. C05 Phase Ended - End of Input & File Backup This rule partners rule 10 and raises an alert, using a standard Action Schedule, if the EQC05 Phase Ended message from Job MISYSEOD is not received. C10 Phase Started - Deal Reporting This rule looks for a message containing the text EQC10 Phase Started. Once received, a start watch action commences to check that the EQC10 Phase Ended message is received within 30 minutes. The alert is automatically closed after one minute of the message being received. C10 Phase Ended - End of Input & File Backup This rule partners rule 30 and raises an alert, using a standard Action Schedule, if the EQC10 Phase Ended message is not received. Specialist Templates 4-99

106 C15 Phase Started - Maturities This rule looks for a message containing the text EQC15 Phase Started. Once received, a start watch action commences to check that the EQC15 Phase Ended message is received within 30 minutes. The alert is automatically closed after one minute of the message being received. C15 Phase Ended - Maturities This rule partners rule 50 and raises an alert, using a standard Action Schedule, if the EQC15 Phase Ended message is not received. C20 Phase Started - Input Reports This rule looks for a message containing the text EQC20 Phase Started. Once received, a start watch action commences to check that the EQC20 Phase Ended message is received within 30 minutes. The alert is automatically closed after one minute of the message being received. C20 Phase Ended - Input Reports This rule partners rule 70 and raises an alert, using a standard Action Schedule, if the EQC20 Phase Ended message is not received. C35 Phase Started - Housekeeping & Statements Extract This rule looks for a message containing the text EQC35 Phase Started. Once received, a start watch action commences to check that the EQC35 Phase Ended message is received within 30 minutes. The alert is automatically closed after one minute of the message being received. C35 Phase Ended - Housekeeping & Statements Extract This rule partners rule 90 and raises an alert, using a standard Action Schedule, if the EQC35 Phase Ended message is not received. C40 Phase Started - Interest This rule looks for a message containing the text EQC40 Phase Started. Once received, a start watch action commences to check that the EQC40 Phase Ended message is received within 30 minutes. The alert is automatically closed after one minute of the message being received Halcyon Templates

107 C40 Phase Ended - Interest This rule partners rule 110 and raises an alert, using a standard Action Schedule, if the EQC40 Phase Ended message is not received. C41 Phase Started - End of Day Profit Transfer This rule looks for a message containing the text EQC41 Phase Started. Once received, a start watch action commences to check that the EQC41 Phase Ended message is received within 30 minutes. The alert is automatically closed after one minute of the message being received. C41 Phase Ended - End of Day Profit Transfer This rule partners rule 130 and raises an alert, using a standard Action Schedule, if the EQC41 Phase Ended message is not received. C42 Phase Started - Statements and Balance Rollup This rule looks for a message containing the text EQC42 Phase Started. Once received, a start watch action commences to check that the EQC42 Phase Ended message is received within 30 minutes. The alert is automatically closed after one minute of the message being received. C42 Phase Ended - Statements and Balance Rollup This rule partners rule 150 and raises an alert, using a standard Action Schedule, if the EQC42 Phase Ended message is not received. C43 Phase Started - Statistics This rule looks for a message containing the text EQC43 Phase Started. Once received, a start watch action commences to check that the EQC43 Phase Ended message is received within 30 minutes. The alert is automatically closed after one minute of the message being received. C43 Phase Ended - Statistics This rule partners rule 170 and raises an alert, using a standard Action Schedule, if the EQC43 Phase Ended message is not received. C45 Phase Started - Reconciliation This rule looks for a message from containing the text EQC45 Phase Started. Once received, a start watch action commences to check that the EQC45 Phase Ended message is received within 30 minutes. The alert is automatically closed after one minute of the message being received. Specialist Templates 4-101

108 C45 Phase Ended - Reconciliation This rule partners rule 190 and raises an alert, using a standard Action Schedule, if the EQC45 Phase Ended message is not received. C50 Phase Started - End of Day Backup This rule looks for a message from containing the text EQC50 Phase Started. Once received, a start watch action commences to check that the EQC50 Phase Ended message is received within 30 minutes. The alert is automatically closed after one minute of the message being received. C50 Phase Ended - End of Day Backup This rule partners rule 210 and raises an alert, using a standard Action Schedule, if the EQC50 Phase Ended message is not received. C65 Phase Started - End of Day Reports This rule looks for a message from containing the text EQC65 Phase Started. Once received, a start watch action commences to check that the EQC65 Phase Ended message is received within 30 minutes. The alert is automatically closed after one minute of the message being received. C65 Phase Ended - End of Day Reports This rule partners rule 230 and raises an alert, using a standard Action Schedule, if the EQC65 Phase Ended message is not received. All Phases Ended This rule checks to ensure that all end of day phases have completed and raises an alert, using a standard Action Schedule, if the EQC50 Phase Ended message is not received. QSYSOPR Qsysopr Message Queue EQUATION: All Inquiry Messages Raises an alert for each inquiry message that is received on the QSYSOPR message queue. EQUATION: High Severity Messages >=90 Raises an alert for each message generated on the QSYSOPR message queue with a severity level of 90 or greater Halcyon Templates

109 Performance Group Rules EQUATION MISYS Equation Rules EQASYNC subsystem active Checks to ensure that the subsystem EQASYNC is active on a 24/7 basis. An alert is raised if the subsystem is found to be in an inactive status. EQASYNC subsystem jobs in message wait Checks to ensure that there are no jobs within subsystem EQASYNC in message wait status. An alert is raised if any jobs in this subsystem are found in this status. EQASYNC ensure key jobs running Checks to ensure that the key jobs; #EQ_MQ_INV, #MQACC_INV, #MQPAY_INV, #UN_MT_INV, OLBACS_INV are running. An alert is raised if any of these jobs are found to be inactive. REPEQNINV subsystem inactive Checks to ensure that the subsystem REPEQNINV is active on a 24/7 basis. An alert is raised if the subsystem is found to be in an inactive status. MQ Series channel jobs inactive #1 Checks to ensure that MQ Series channel jobs; MMISINV, QJVAEXEC and QZSHSH are running. An alert is raised if any of these jobs are found to be inactive. MQ Series channel jobs inactive #2 Checks to ensure that 2 MQ Series channel jobs named QP0ZSPWT are running. An alert is raised if the number of QP0ZSPWT jobs is found to be less than or greater than 2. MQ Series channel jobs in message wait Checks to ensure that the MQ Series channel jobs; MMISINV, QJVAEXEC, QZSHSH and QP0ZPWT are not in message wait status. An alert is raised if any of this job are found to be in this status. ASP >80% Checks against the level of usage on all ASP s and raises an alert if the used percentage on any ASP is found to be greater than 80%. This rule is suspended for 60 minutes upon the generation of an alert. Specialist Templates 4-103

110 Runtime of MISYS End of Day >4 hours Checks that the job MISYSEOD completes within 4 hours of being started. An alert is generated if the runtime is outside this time period. Object Group Rules EQUATION MISYS Equation Rules Data Area KLIBINV/KAPPHS <> Day (Current Phase) Checks that the Data Area KAPPHS exists within library KLIBINV and that the text value DAY is always present during the online working period. An alert is raised if the data area is not found or if the text value DAY is not found. Data Area KLIBINV/KQIMODE (Day NORM = Input Enabled) Checks that the Data Area KQIMODE exists within library KLIBINV and that the text value NORM is always present during the online working period which is set between 06:00 and 18:00. An alert is raised if the data area is not found or if the text value NORM is not found within this time period. Data Area KLIBINV/KQIMODE (Eve SUSP = Input Disabled) Checks that the Data Area KQIMODE exists within library KLIBINV and that the text value SUSP is always present during the end of day processing (offline) period which is set between 18:01 and 05:59. An alert is raised if the data area is not found or if the text value SUSP is not found within this time period. Misys Midas Plus Customization Templates Midas Plus is the latest generation of the Misys Midas core banking system. Misys Midas Plus offers a single, integrated platform for delivering a wide range of services to corporate customers, including Treasury, Commercial Lending, Capital Markets, Cash Management and Wealth Management. The following template rules are designed to monitor specific areas of the Misys Midas Plus system to ensure that all functions are performing as expected. Message Queue Rules KAPSTATUS Midas Plus End of Day Monitoring This rule group contains rules to ensure that start and end messages for critical end of day Midas Plus routines. This daily cycle is broken down into phases that complete a sequence that runs between end of business of one day and the start of business on the next and uses the following schedule: Halcyon Templates

111 Phase C05 C10 C15 C20 C35 C40 (C41) C43 C45 C50 (C65) Phase Name End of Input and File Backup Deal Reporting Maturities Input Reports Housekeeping and Statements Extract Interest End of Day Profit Transfer Statistics Reconciliation End of Day backup End of Day Reports Phase numbers within brackets are not necessarily run each day, but only when the control system has been instructed to run them. The rules within this group are run every 60 seconds with an immediate alert level (message severity) of 90. Note: No Global Exclusions See the No Global Exclusions Defined section for more information. C05 Phase Started - End of Input & File Backup This rule looks for a message from Job MISYSEOD containing the text EQC05 Phase Started. Once received, a start watch action commences to check that the EQC05 Phase Ended message is received within 30 minutes. Additionally, a second start watch action commences to check that the entire end of day phase is completed within 4 hours. The alert is automatically closed after one minute of the message being received. C05 Phase Ended - End of Input & File Backup This rule partners rule 10 and raises an alert, using a standard Action Schedule, if the EQC05 Phase Ended message from Job MISYSEOD is not received. Specialist Templates 4-105

112 C10 Phase Started - Deal Reporting This rule looks for a message containing the text EQC10 Phase Started. Once received, a start watch action commences to check that the EQC10 Phase Ended message is received within 30 minutes. The alert is automatically closed after one minute of the message being received. C10 Phase Ended - End of Input & File Backup This rule partners rule 30 and raises an alert, using a standard Action Schedule, if the EQC10 Phase Ended message is not received. C15 Phase Started - Maturities This rule looks for a message containing the text EQC15 Phase Started. Once received, a start watch action commences to check that the EQC15 Phase Ended message is received within 30 minutes. The alert is automatically closed after one minute of the message being received. C15 Phase Ended - Maturities This rule partners rule 50 and raises an alert, using a standard Action Schedule, if the EQC15 Phase Ended message is not received. C20 Phase Started - Input Reports This rule looks for a message containing the text EQC20 Phase Started. Once received, a start watch action commences to check that the EQC20 Phase Ended message is received within 30 minutes. The alert is automatically closed after one minute of the message being received. C20 Phase Ended - Input Reports This rule partners rule 70 and raises an alert, using a standard Action Schedule, if the EQC20 Phase Ended message is not received. C35 Phase Started - Housekeeping & Statements Extract This rule looks for a message containing the text EQC35 Phase Started. Once received, a start watch action commences to check that the EQC35 Phase Ended message is received within 30 minutes. The alert is automatically closed after one minute of the message being received. C35 Phase Ended - Housekeeping & Statements Extract This rule partners rule 90 and raises an alert, using a standard Action Schedule, if the EQC35 Phase Ended message is not received Halcyon Templates

113 C40 Phase Started - Interest This rule looks for a message containing the text EQC40 Phase Started. Once received, a start watch action commences to check that the EQC40 Phase Ended message is received within 30 minutes. The alert is automatically closed after one minute of the message being received. C40 Phase Ended - Interest This rule partners rule 110 and raises an alert, using a standard Action Schedule, if the EQC40 Phase Ended message is not received. C41 Phase Started - End of Day Profit Transfer This rule looks for a message containing the text EQC41 Phase Started. Once received, a start watch action commences to check that the EQC41 Phase Ended message is received within 30 minutes. The alert is automatically closed after one minute of the message being received. C41 Phase Ended - End of Day Profit Transfer This rule partners rule 130 and raises an alert, using a standard Action Schedule, if the EQC41 Phase Ended message is not received. C42 Phase Started - Statements and Balance Rollup This rule looks for a message containing the text EQC42 Phase Started. Once received, a start watch action commences to check that the EQC42 Phase Ended message is received within 30 minutes. The alert is automatically closed after one minute of the message being received. C42 Phase Ended - Statements and Balance Rollup This rule partners rule 150 and raises an alert, using a standard Action Schedule, if the EQC42 Phase Ended message is not received. C43 Phase Started - Statistics This rule looks for a message containing the text EQC43 Phase Started. Once received, a start watch action commences to check that the EQC43 Phase Ended message is received within 30 minutes. The alert is automatically closed after one minute of the message being received. C43 Phase Ended - Statistics This rule partners rule 170 and raises an alert, using a standard Action Schedule, if the EQC43 Phase Ended message is not received. Specialist Templates 4-107

114 C45 Phase Started - Reconciliation This rule looks for a message from containing the text EQC45 Phase Started. Once received, a start watch action commences to check that the EQC45 Phase Ended message is received within 30 minutes. The alert is automatically closed after one minute of the message being received. C45 Phase Ended - Reconciliation This rule partners rule 190 and raises an alert, using a standard Action Schedule, if the EQC45 Phase Ended message is not received. C50 Phase Started - End of Day Backup This rule looks for a message from containing the text EQC50 Phase Started. Once received, a start watch action commences to check that the EQC50 Phase Ended message is received within 30 minutes. The alert is automatically closed after one minute of the message being received. C50 Phase Ended - End of Day Backup This rule partners rule 210 and raises an alert, using a standard Action Schedule, if the EQC50 Phase Ended message is not received. C65 Phase Started - End of Day Reports This rule looks for a message from containing the text EQC65 Phase Started. Once received, a start watch action commences to check that the EQC65 Phase Ended message is received within 30 minutes. The alert is automatically closed after one minute of the message being received. C65 Phase Ended - End of Day Reports This rule partners rule 230 and raises an alert, using a standard Action Schedule, if the EQC65 Phase Ended message is not received. All Phases Ended This rule checks to ensure that all end of day phases have completed and raises an alert, using a standard Action Schedule, if the EQC50 Phase Ended message is not received Halcyon Templates

115 MOPERQ Midas Plus These template rules are designed to monitor the MOPERQ (MIDAS Operations message queue) within library P1XLIB. By default, rules in this group are run every 60 seconds and override any paging schedules should the message severity be greater than or equal to 90. Note: No Global Exclusions Defined See the No Global Exclusions Defined section for more information. MIDASPLUS: All Messages This rule raises an alert for any message received on the MOPERQ message queue within library P1XLIB. MIDASPLUS: All Midas Database Errors This rule raises an alert for any message that is received on the MOPERQ message queue within library P1XLIB that contains the text A Midas database error in position 1. QSYSOPR Qsysopr Message Queue These template rules are designed to monitor any inquiry and high severity messages that are received on the general purpose QSYSOPR message queue.by default, rules in this group are run every 60 seconds and override any paging schedules should the message severity be greater than or equal to 50. MIDASPLUS: All Inquiry Messages Raises an alert for each inquiry message that is received on the QSYSOPR message queue. MIDASPLUS: High Severity Messages >=90 Raises an alert for each message generated on the QSYSOPR message queue with a severity level of 90 or greater. TCP/IP Group Rules Note: MIDAS Plus Windows Servers Active This group of template rules is designed to ensure that key Windows Servers as used by MIDAS Plus are constantly available. For the purpose of these rules, you must replace the actual Remote System Name and Internet Address with the correct equivalents from your own network environment. Specialist Templates 4-109

116 MIDAS Plus: Ensure Windows Server 1 is active This rule raises an alert when the remote system on the network, identified as Windows 1 by default, returns a status of Not Available. The rule is active 24/7 and the server is checked for availability every 5 minutes. MIDAS Plus: Ensure Windows Server 2 is active This rule raises an alert when the remote system on the network, identified as Windows 2 by default, returns a status of Not Available. This rule is active 24/7 and the server is checked for availability every 5 minutes. Performance Group Rules MIDAS PLUS MIDAS Plus Rules This group of rule templates is designed to monitor important MIDAS Plus subsystems and key jobs to ensure that all are running as expected. Unless specified otherwise, all of these rules are active on a 24/7 basis. MIDAS Plus: MP1SBSC Zone subsystem available Checks to ensure that the critical subsystem MP1SBSC is active on a 24/7 basis. An alert is raised if this subsystem is found to be in an inactive status. MIDAS Plus: MPGGSBS Global subsystem available Checks to ensure that the critical subsystem MPGGSBS is active on a 24/7 basis. An alert is raised if this subsystem is found to be in an inactive status. MIDAS Plus: MP1SBSC Zone jobs in MSGW or LCKW status Checks to ensure that all jobs (except subsystem monitor jobs) within the Midas Plus subsystem; MP1SBSC, are not in a status of MSGW or LCKW. MIDAS Plus: MPGGSBS Global jobs in MSGW or LCKW status Checks to ensure that all jobs (except subsystem monitor jobs) within the Midas Plus subsystem; MPGGSBS Global, are not in a status of MSGW or LCKW. MIDAS Plus: MQ Series channel jobs inactive #1 Checks to ensure that MQ Series channel jobs; MMISINV, QJVAEXEC and QZSHSH are running. An alert is raised if any of these jobs are found to be inactive. MIDAS Plus: MQ Series channel jobs inactive #2 Checks to ensure that 2 MQ Series channel jobs named QP0ZSPWT are running. An alert is raised if the number of QP0ZSPWT jobs is found to be less than or greater than Halcyon Templates

117 MIDAS Plus: MQ Series channel jobs in message wait Checks to ensure that the MQ Series channel jobs; MMISINV, QJVAEXEC, QZSHSH and QP0ZPWT are not in message wait status. An alert is raised if any of this job are found to be in this status. MIDAS Plus: ASP Used >80% Checks against the level of usage on all ASP s and raises an alert if the used percentage on any ASP is found to be greater than 80%. MIDAS Plus: Runtime of MISYS End of Day >4 hours Checks that the job MISYSEOD completes within 4 hours of being started. An alert is generated if the runtime is outside this time period. MIDAS Plus: QINTER subsystem available Checks to ensure that the critical subsystem QINTER is active on a 24/7 basis. An alert is raised if this subsystem is found to be in an inactive status. MIDAS Plus: QSPL subsystem available Checks to ensure that the critical subsystem QSPL is active on a 24/7 basis. An alert is raised if this subsystem is found to be in an inactive status. MIDAS Plus: QUSRWRK subsystem available Checks to ensure that the critical subsystem QUSRWRK is active on a 24/7 basis. An alert is raised if this subsystem is found to be in an inactive status. MIDAS Plus: QSYSWRK subsystem available Checks to ensure that the critical subsystem QSYSWRK is active on a 24/7 basis. An alert is raised if this subsystem is found to be in an inactive status. MIDAS Plus: Sustained high CPU jobs in MP1SBSC Checks to ensure that any job (except subsystem monitor jobs) is not taking more than 20% of the LPAR processing capacity over any 60 second time period. An alert is raised if this level is reached 4 times within a 5 minute period. MIDAS Plus: Sustained high CPU jobs in MPGGSBS Checks to ensure that any job (except subsystem monitor jobs) is not taking more than 20% of the LPAR processing capacity over any 60 second time period. MIDAS Plus: Track QZDASOINIT jobs Tracks QZDASOINIT jobs in subsystem QUSRWRK that are in a status of INEL, MSGW or LCKW and raises an alert accordingly if any instances are found. QZDASONIT jobs can slow down your system when there are too many of them active, especially when running batch processing. Specialist Templates 4-111

118 User Group Rules MIDAS PLUS Midas Plus Rules This rule group contains a single rule that checks that the critical MIDASEOD User Profile is enabled. Ensure MIDASEOD User Profile is enabled Checks that the User Profile MIDASEOD is enabled 24/7. If this user profile is found in a state of *DISABLED, an alert is raised Halcyon Templates

119 C HAPTER 5WEBSPHERE MQ MONITORING CHAPTER WebSphere MQ, a member of the WebSphere family from IBM, is the most popular system for cross- platform messaging, providing assured delivery of messages across 35 plus IBM and non-ibm platforms, including IBM mainframe and midrange, Windows, Linux, and UNIX. WebSphere MQ is often referred to as "MQ" or "MQ Series". Key components of WebSphere MQ that require monitoring are: "MQ Channels (up/down) "Listeners (up/down) "MQ Queue Depth (alert if a large build up of messages) "Dead Letter Queue "Individual MQ elements (jobs) Halcyon's Record & Playback utility can be used to record key events, keystrokes, command entry and activity checking routines on these critical areas of WebSphere MQ. The resulting scripts can then be replayed using the session play command PLAYSSN from within the IBM Job Scheduler. For added flexibility and to control the scheduled running of the Record and Playback scripts, the Halcyon Advanced Job Scheduler can be deployed. Alternatively, the session can be replayed using the COMMAND action from within any Halcyon Rule (supplied as standard within any of our IBM i Product Suites). Should an error be found, an alert can be sent to a message queue, via , SMS or to Halcyon's centralized GUI Enterprise Console. WebSphere MQ Monitoring 5-113

120 MQ Monitoring Record & Playback Scripts Halcyon s Record & Playback is used to run 4 specially created scripts to monitor WebSphere s MQ Channels. Providing that you have Record & Playback installed on your system, at the command line type: GO HRP400/HRP Figure 5.1 Starting Halcyon Record & Playback The Record & PlayBack main menu is displayed Halcyon Templates

121 Figure 5.2 Record & Playback main menu Select Option 3=Edit Scripts to open the Edit Scripts display. Figure 5.3 Edit Scripts display WebSphere MQ Monitoring 5-115