JBoss Operations Network 1 Users Guide

Transcription

1 JBoss Operations Network 1 Users Guide Users Guide for JBoss Operations Network 1.x Edition 2 Landmann

2 JBoss Operations Network 1 Users Guide Users Guide for JBoss Operations Network 1.x Edition 2 Landmann rlandmann@redhat.co m

3 Legal Notice Copyright , 2012 Red Hat, Inc. T his document is licensed by Red Hat under the Creative Commons Attribution-ShareAlike 3.0 Unported License. If you distribute this document, or a modified version of it, you must provide attribution to Red Hat, Inc. and provide a link to the original. If the document is modified, all Red Hat trademarks must be removed. Red Hat, as the licensor of this document, waives the right to enforce, and agrees not to assert, Section 4d of CC-BY-SA to the fullest extent permitted by applicable law. Red Hat, Red Hat Enterprise Linux, the Shadowman logo, JBoss, MetaMatrix, Fedora, the Infinity Logo, and RHCE are trademarks of Red Hat, Inc., registered in the United States and other countries. Linux is the registered trademark of Linus Torvalds in the United States and other countries. Java is a registered trademark of Oracle and/or its affiliates. XFS is a trademark of Silicon Graphics International Corp. or its subsidiaries in the United States and/or other countries. MySQL is a registered trademark of MySQL AB in the United States, the European Union and other countries. Node.js is an official trademark of Joyent. Red Hat Software Collections is not formally related to or endorsed by the official Joyent Node.js open source or commercial project. T he OpenStack Word Mark and OpenStack Logo are either registered trademarks/service marks or trademarks/service marks of the OpenStack Foundation, in the United States and other countries and are used with the OpenStack Foundation's permission. We are not affiliated with, endorsed or sponsored by the OpenStack Foundation, or the OpenStack community. All other trademarks are the property of their respective owners. Abstract Installation, Inventory, Monitoring, Alerts, Managed Products, Response T ime, Control, Administration, Developing Plugins.

4 Table of Contents Table of Contents. Preface Document Conventions T ypographic Conventions Pull-quote Conventions Notes and Warnings 12. Chapter Welcome Overview of JBoss Operations Network Auto Discovery Monitor Alert Control Architecture Supported Platforms Supported Platforms - JBoss ON the Road 17. Chapter Installation Install Guide Planning and Prerequisites Sizing X Libraries Hardware JRE Clock Synchronization Database Preparation Oracle Preparation Advanced Oracle Configuration Oracle Asynchronous I/O PostgreSQL Preparation Installing JBoss Operations Network Download JBoss ON Available Distributions Full Installer vs. Agent-Only Installer Platforms Windows GUI Installer T ext Installers Components JON Server Installation JON Agent Installation JON Shell Installation JON Server Startup JON Agent Startup JON Agent Shutdown Laptop Installations QuickStart Guide Check for supported platforms Download JBoss ON Unpack the JBoss ON package Install the JBoss ON Server and Agent Start up the JBoss ON Server Start up the JBoss ON Agent Login to JBoss ON 34 1

5 JBoss Operations Network 1 Users Guide Import a new platform View platform metrics Complete 2.3. Agent-Only Installation Agent-Only Install on Windows Agent-Only Install on UNIX General Notes 2.4. High Availability Installation 2.5. No-JRE Installer Overview Using the No-JRE JBoss ON Installer PostgreSQL Server Installation and Configuration 2.6. Installing on a Laptop What to do if not on a network 2.7. Uninstalling JBoss Operations Network Chapter JBoss ON... Inventory Auto-Discovery Helpful Hints Platforms Servers Services Groups Application Inventory XML Descriptor Syntax 46. Chapter Monitoring Monitoring Basics Metrics Baselines Problem Metrics Metric Display Range Monitoring Visibility Navigating to Current Health Resources Child Resources Health Portlets Host Resources Health Portlet Group Member Resources Health Portlet Problem Metrics Portlet Indicators Availability and T imeline Indicator Charts and Views Managing Metric Data Metric Categories Availability Usage Performance Utilization Viewing Metrics Modifying Metrics Working With Autogroup Metrics T o Remove Metrics Working with Baselines T o manually establish a metric baseline Availability Charting Metric Data 60 2

6 Table of Contents Chart Legend 61. Chapter Alerts Using Alerts to Manage Resources Defining Alerts Defining Recovery Alerts SNMP Traps as Alert Actions T utorial on generating SNMP traps from JBoss ON alerts Environment Installing an SNMP client Listening for traps Configuring JBoss ON to generate SNMP traps Common SNMP settings on the alert SNMP v1 - Generic trap SNMP v1 - Enterprise Specific trap SNMP v2c - Generic trap 69. Chapter Managed Products Operating Systems Linux Management Support Linux Supported Versions Linux Monitoring Specification Linux Administration Specification Linux Metrics Linux Process Metrics Linux User Metrics Network Interface Network IP Service Metrics Filesystem Directory Metrics FileSystem Mount File Service Overview Configuration Properties Metrics Control Actions HP-UX Management Support HP-UX Supported Versions HP-UX Monitoring Specification HP-UX Administration Specification HP/UX Metrics HP/UX Process Metrics HP UX User Metrics Network Interface Network IP Service Metrics Filesystem Directory Metrics FileSystem Mount File Service Overview Configuration Properties Metrics Control Actions Sun Solaris Management Support Solaris Supported Versions Solaris Monitoring Specification Solaris Administration Specification Solaris Metrics 79 3

7 JBoss Operations Network 1 Users Guide Solaris Process Metrics Solaris User Metrics Network Interface Network IP Service Metrics Filesystem Directory Metrics FileSystem Mount File Service Overview Configuration Properties Metrics Control Actions IBM AIX Management Support IBM AIX Supported Versions IBM AIX Monitoring Specification IBM AIX Administration Specification IMB AIX Metrics IBM AIX Process Metrics IBM AIX User Metrics Network Interface Network IP Service Metrics Filesystem Directory Metrics FileSystem Mount File Service Overview Configuration Properties Metrics Control Actions Microsoft Windows Management Support Windows Supported Versions Windows Monitoring Specification Windows Administration Specification Windows Metrics Windows Process Metrics Network Interface Network IP Service Metrics Filesystem Directory Metrics FileSystem Mount File Service Overview Configuration Properties Metrics Control Actions Windows Service Metrics Defining a Windows Service Windows Service Control 6.2. JEMS Products JBoss AS Management Support JBoss AS Supported Versions JBoss Monitoring Specification JBoss Administration Specification Autodetection General Server Metrics JMS Destinations JMS Destination Metrics Administering JMS Destinations

8 Table of Contents Restrictions Features Updates to configuration files Messages in Queue Messages in T opic Availability Receivers Count Subscriptions Count Entity EJB Metrics Stateless Session EJB Metrics] Stateful Session EJB Metrics Message Driven EJB Metrics JCA Connection Pool Metrics JCA DataSources JCA DataSource Metrics Administering JCA DataSources Restrictions Features Updates to configuration files Hibernate Support JBoss AS Supported Versions Hibernate Monitoring Specification JGroups Support JGroups Supported Versions JGroups Monitoring Specification Control Actions Configuration Actions Dynamic JBoss Launcher Java Service Wrapper Install JBoss Launcher Service Start JBoss Launcher Service Stop JBoss Launcher Service Uninstall JBoss Launcher Service File Deployment Actions Deploy File Undeploy File List Deployed Files Undeploy Selected File T omcat Management Support T omcat Supported Versions T omcat Monitoring Specification T omcat Administration Specification General Server Metrics Reliability Metrics Resource Utilization Metrics Connector Metrics Webapp Metrics Servlet Metrics Control Actions Hibernate Support JBoss AS Supported Versions Hibernate Monitoring Specification JGroups Support JGroups Supported Versions JGroups Monitoring Specification

9 JBoss Operations Network 1 Users Guide 6.3. Web Servers Apache Management Support Apache Supported Versions Apache Monitoring Specification Apache Administration Specification Apache Product Configuration Resource Utilization Metrics Reliability Metrics T hroughput Metrics Request Metrics Response Metrics VHost Metrics Control Actions Microsoft IIS Management Support Microsoft IIS Supported Versions Microsoft IIS Monitoring Specification Microsoft IIS Administration Specification Reliability Metrics Connection Metrics T ransaction Metrics Performance Metrics Session Metrics I/O Metrics T hroughput Metrics Request Processing Metrics VHost Metrics iplanet/sunone Management Support iplanet/sunone Supported Versions iplanet/sunone Monitoring Specification iplanet/sunone Administration Specification Reliability Metrics Connnection Queue Metrics Resource Utilization Metrics Process Metrics T hroughput Metrics T hread Pool and VHost 6.4. File System Services FileSystem Mount Filesystem Directory Metrics File Service Overview Configuration Properties Metrics Control Actions 6.5. Network Services Network Interface Network IP Service Metrics 6.6. System Process Services MultiProcess Metrics Process Metrics Windows Service Metrics 6.7. Script Services Script Service Overview Configuration Properties

10 Table of Contents Metrics Control Actions Nagios Plugin Service Overview Configuration Properties Metrics Control Actions Chapter Response T. ime What is Response Time? Configuring Response T ime Apache Servlet Charting Response T ime 127. Chapter Control Controlling Resources 128. Chapter Administration JBoss Operations Network Administration JBoss ON Users and Roles Server Configuration JON Base URL Configuration Properties JON Configuration Properties Data Manager Configuration Properties Automatic Baseline Configuration Properties LDAP Configuration Properties SNMP Server Configuration Properties Monitoring Defaults Configuration Agent Configuration License Management Patch Installation What is a Cumulative Patch (CP)? Current Limitations Installing a Patch Review the Installation Steps Installing a Patch on a server that has previously been patched Installing a Patch on a server with multiple configurations Example patch installation instructions View the Result of Installation Performing Manual Steps Uninstall the patch if necessary Manually Uninstalling a Patch Uninstalling a patch Uninstalling a patch from a server with multiple configurations How to reverse each step Reversing a successful "Backup and replace" step Reversing a skipped "Backup and replace" step Reversing a failed "Backup and replace" step Reversing a successful "Stop server" step Reversing a failed "Stop server" step Reversing a successful "Start server" step Reversing a failed "Start server" step Reversing the "Download file" step Reversing the "Calculate digest" step Reversing the "Unzip" step 143 7

11 JBoss Operations Network 1 Users Guide Viewing Past Patch Installation Attempts Securing Patch Installation T roubleshooting Patch Installation Example Chapter Developing Plugins Plugin Overview Product Plugin Component Measurement Plugin Component Introduction Measurement XML Attributes MeasurementPlugin.getValue Method Control Plugin Component Introduction ControlPlugin.doAction Method Autoinventory Plugin Component Overview Simple autoinventory Plugin Components Plugin XML Descriptor Plugins are supported in the following formats: Top-level plugin Tag server and service plugin T ag Config Schema Plugin Classpath filter T ag property T ag metrics T ag help Tag server T ag scan Tag service T ag Plugin XML Descriptor Syntax Invoking Plugins Standalone Examples Plugin Build Script Adding an MBean attribute as a JBoss Server Metric Adding a New JBoss Service via the JON PDK 178 8

12 Table of Contents 9

13 JBoss Operations Network 1 Users Guide Preface 1. Document Conventions T his manual uses several conventions to highlight certain words and phrases and draw attention to specific pieces of information. In PDF and paper editions, this manual uses typefaces drawn from the Liberation Fonts set. T he Liberation Fonts set is also used in HTML editions if the set is installed on your system. If not, alternative but equivalent typefaces are displayed. Note: Red Hat Enterprise Linux 5 and later include the Liberation Fonts set by default Typographic Conventions Four typographic conventions are used to call attention to specific words and phrases. T hese conventions, and the circumstances they apply to, are as follows. Mono-spaced Bold Used to highlight system input, including shell commands, file s and paths. Also used to highlight keys and key combinations. For example: T o see the contents of the file m y_next_bestselling_novel in your current working directory, enter the cat m y_next_bestselling_novel command at the shell prompt and press Enter to execute the command. The above includes a file, a shell command and a key, all presented in mono-spaced bold and all distinguishable thanks to context. Key combinations can be distinguished from an individual key by the plus sign that connects each part of a key combination. For example: Press Enter to execute the command. Press Ctrl+Alt+F2 to switch to a virtual terminal. T he first example highlights a particular key to press. T he second example highlights a key combination: a set of three keys pressed simultaneously. If source code is discussed, class s, methods, functions, variable s and returned values mentioned within a paragraph will be presented as above, in m ono-spaced bold. For example: File-related classes include filesystem for file systems, file for files, and dir for directories. Each class has its own associated set of permissions. Proportional Bold T his denotes words or phrases encountered on a system, including application s; dialog box text; labeled buttons; check-box and radio button labels; menu titles and sub-menu titles. For example: Choose System Preferences Mouse from the main menu bar to launch Mouse Preferences. In the Buttons tab, select the Left-handed m ouse check box and click Close to switch the primary mouse button from the left to the right (making the mouse suitable for use in the left hand). 10

14 Preface T o insert a special character into a gedit file, choose Applications Accessories Character Map from the main menu bar. Next, choose Search Find from the Character Map menu bar, type the of the character in the Search field and click Next. T he character you sought will be highlighted in the Character T able. Double-click this highlighted character to place it in the T ext to copy field and then click the Copy button. Now switch back to your document and choose Edit Paste from the gedit menu bar. T he above text includes application s; system-wide menu s and items; application-specific menu s; and buttons and text found within a GUI interface, all presented in proportional bold and all distinguishable by context. Mono-spaced Bold Italic or Proportional Bold Italic Whether mono-spaced bold or proportional bold, the addition of italics indicates replaceable or variable text. Italics denotes text you do not input literally or displayed text that changes depending on circumstance. For example: T o connect to a remote machine using ssh, type ssh user@ domain. at a shell prompt. If the remote machine is example.com and your user on that machine is john, type ssh john@ exam ple.com. T he m ount -o rem ount file-system command remounts the d file system. For example, to remount the /home file system, the command is mount -o remount /home. T o see the version of a currently installed package, use the rpm -q package command. It will return a result as follows: package-version-release. Note the words in bold italics above user, domain., file-system, package, version and release. Each word is a placeholder, either for text you enter when issuing a command or for text displayed by the system. Aside from standard usage for presenting the title of a work, italics denotes the first use of a new and important term. For example: Publican is a DocBook publishing system Pull-quote Conventions T erminal output and source code listings are set off visually from the surrounding text. Output sent to a terminal is set in mono-spaced roman and presented thus: books Desktop documentation drafts mss photos stuff svn books_tests Desktop1 downloads images notes scripts svgs Source-code listings are also set in m ono-spaced rom an but add syntax highlighting as follows: 11

15 JBoss Operations Network 1 Users Guide package org.jboss.book.jca.ex1; import javax.naming.initialcontext; public class ExClient { public static void main(string args[]) throws Exception { InitialContext inictx = new InitialContext(); Object ref = inictx.lookup("echobean"); EchoHome home = (EchoHome) ref; Echo echo = home.create(); System.out.println("Created Echo"); } } System.out.println("Echo.echo('Hello') = " + echo.echo("hello")); 1.3. Notes and Warnings Finally, we use three visual styles to draw attention to information that might otherwise be overlooked. Note Notes are tips, shortcuts or alternative approaches to the task at hand. Ignoring a note should have no negative consequences, but you might miss out on a trick that makes your life easier. Important Important boxes detail things that are easily missed: configuration changes that only apply to the current session, or services that need restarting before an update will apply. Ignoring a box labeled 'Important' will not cause data loss but may cause irritation and frustration. Warning Warnings should not be ignored. Ignoring warnings will most likely cause data loss. 12

16 Chapter 1. Welcome Chapter 1. Welcome 1.1. Overview of JBoss Operations Network JBoss Operations Network is an IT management framework which gives its users the ability to manage many different types of technologies from a single interface Auto Discovery Auto-discover resources (Platforms, Servers and Services, refer to Chapter 3, JBoss ON Inventory) Define Applications and view supporting resource maps (refer to Section 3.6, Application ) Manage platform, server, service, and application inventories Monitor Measure end-to-end application service levels T rack performance, availability, resource utilization, throughput and events Identify root cause of application problems by drilling down through resource hierarchies Correlate cause and effect Analyze historical data Alert Alert on problems as soon as they occur Configure multi-condition alerts Send alerts to , cell phone or pager Setup auto-recovery alerts that are linked to control actions Control Centrally exercise a full range of resource specific commands Perform pre/post control impact analysis Perform one-to-many corrective actions over groups of resources Auto-recover from service level outages and degradations 1.2. Architecture JBoss Operations Network is a distributed infrastructure management system. T he major pieces of the system architecture are shown below. JON Repository The data repository forms the bedrock of the JON system. It stores all JON-related data in a relational schema and is responsible for ensuring data integrity and coordinating concurrent access to data. Access to the data repository is regulated by the Management Server. JON Server 13

17 JBoss Operations Network 1 Users Guide As JON's central nervous system, the JON server coordinates all system functions, including: Processing incoming monitoring data Detecting alert conditions and sending out alerts Managing inventory, including merging auto-discovery information into current inventory Enforcing security Maintaining JON operational schedules (for control actions and auto-discovery scans) Processing user-driven actions initiated via the GUI Console or command line interface In large environments, the management server can be clustered for enhanced fault tolerance and to share the overall system load across multiple machines. JON Agent Acting as the sensory facilities of the management system, agents are deployed throughout the network infrastructure to provide points-of-presence for discovering inventory, gathering data, controlling software, and other "in the trenches" tasks. JON GUI Console T he web-based graphical user interface makes it easy to manage your infrastructure from anywhere. T he console interacts with the management server and exposes server functionality through a rich, user-friendly web-based interface. T he console is implemented as a portal, which makes it easy to customize the content and look-and-feel on a per-user basis. JON Command Line T he command line interface (CLI) provides a text-based interface to the management server. T he command line can be driven programmatically, giving power users the ability to automate JON activities by writing shell scripts. JON Agent Plugins A plugin-oriented design allows JON to support a wide variety of products and platforms. Each plugin allows JON agents to interface with a specific product. Plugins are packaged as a single ".jar" file, for ease of deployment. Each plugin has a declarative component and a functional component. T he declarative component is deployed within the JON Server to describe how the product's organizational concepts fit into the JON data model, and what kinds of metrics and control actions are supported. T he functional component is deployed within the agent and is responsible for the actual product-specific work of auto-discovery, metric collection and resource control. JON T opology JON seamlessly monitors, manages, and controls the various servers in the enterprise. Below are two views of the same topology, to help illustrate how the product does this. The User View <--> The Component View On the left we have what JON would look like from a user's point of view. The JON Web Console is gathering metrics and executing certain controls on the servers it know's about. T he diagram on the right shows how the JON Server communicates with these servers in the enterprise. An agent is installed on each of the machines that JON should monitor or manage. The agent is able to 14

18 Chapter 1. Welcome discover resources - platforms, servers, and services - installed on that machine by using its plugins as delegates. Information flows from the plugins through the agent and up to the JON server. Likewise, information from the JON server will flow down to the agent, which will distribute the request to the appropriate plugin. By default, the agent comes pre-packaged with all of our standard product plugins. T he diagram above simply emphasizes the plugins that would be active on each particular machine in that particular 3-tier topology. However, it's perfectly possible to have multiple products installed on the same machine, and the JON Agent will successfully handle those scenarios as well. Things to note The JON Agent is J2SE only, not a full-blown J2EE application. T he JON Server installation process is self-contained; it does not require an existing application server 15

19 16 JBoss Operations Network 1 Users Guide

20 Chapter 1. Welcome 1.3. Supported Platforms Platform Server Agent Windows (refer to Section 6.1.5, Microsoft Windows Management Support ) Linux (refer to Section 6.1.1, Linux Management Support ) Solaris (refer to Section 6.1.3, Sun Solaris Management Support ) HP-UX (refer to Section 6.1.2, HP-UX Management Support ) AIX (refer to Section 6.1.4, IBM AIX Management Support ) Warning.MSI,.ZIP.tar w/setup.tar w/setup.tar w/ setup not supported.msi,.zip.tar w/ setup, deploy tool(ssh).tar w/ setup, deploy tool(ssh).tar w/setup, deploy tool(ssh).tar w/ setup, deploy tool(ssh) Click on a platform for information on exactly which architectures and OS versions are supported for that platform Supported Platforms - JBoss ON the Road JBoss ON the Road is a day-long hands-on workshop presented by the JBoss ON team. In order to run the labs, attendees will need to bring a laptop with the following specifications: Operating system - Windows 2000/XP, or some flavor of x86 Linux/Solaris, in particular: NOT a Mac 17

21 JBoss Operations Network 1 Users Guide and NOT Windows Vista. (A full list of supported platforms is listed in T able 3.1, Supported Platforms, however not all installers are available at the workshop.) A CD-rom drive to install the JBoss Operations Network software. 600MB of free hard disk spce. A wireless network adaptor (WiFi) It is recommended for optimal performance to have at least 512Mb of RAM and a 1GHz or faster processor To find out about upcoming JBoss ON the Road Workshops visit JBoss ON the Road registration or contact JBoss Sales. Slides used for the workshop are available upon request. Please contact sales@jboss.com 18

22 Chapter 2. Installation Chapter 2. Installation 2.1. Install Guide JBoss Operations Network is software designed to be quickly installed and integrated into your existing IT infrastructure. T his chapter is the installation guide and will give a complete overview on the JON installer. Tip: To get up and running quickly, see Section 2.2, QuickStart Guide T he activities associated with installing JBoss Operations Network are: 1. Planning and Prerequisites 2. Database preparation (If necessary) 3. Install JBoss Operations Network Planning and Prerequisites Sizing Prior to installing JBoss ON, there are some planning and prerequisite activities that should be undertaken. First, the size of the infrastructure environment to be managed should be considered. A small environment would consist of 2-5 platforms (hosts) running 5-15 servers, a medium sized environment is up to 25 platforms, and a large environment is anything larger. The size of the environment is one factor in determining which backend database will best suit the JBoss ON installation for your particular environment. Other factors include personal preference and existing infrastructure. T he JON built-in embedded database is the easiest to install. Next, check that the machines and operating systems you are installing JBoss ON onto are supported. See Section 1.3, Supported Platforms for a detailed breakdown. Embedded Database on MS-Windows If you are installing on Windows in your production environment, you should not use the embedded database option. Currently, there is no way to upgrade this setup with a new JBoss ON. It is possible this limitation will be removed in future versions of JBoss ON, but as it stands, if you use the embedded database on Windows and you want to upgrade to a newer version of JBoss ON, you will lose all data in your database, including inventory, users/roles, metric data, etc. Actually, this may be a little strongly worded - you could upgrade if you take a pg_dump of your embedded database and transfer it to a standalone database that will be used by the new version, but most people don't want to go through the hassle. If you are evaluating the product, it is highly recommended that you use the embedded database option to get up and running as quickly as possible. However, if your infrastructure already includes installed and configured Oracle or PostgreSQL database servers, these can be used just as easily. If the administrator's personal preference is to maintain separate hosts to run JBoss ON and the database server, that is also a supported configuration. In special case environments that are especially demanding, a combination of advanced JBoss ON and Oracle configuration can be very effective. 19

23 JBoss Operations Network 1 Users Guide Regardless of the size of your environment, your JBoss ON performance can be greatly enhanced with multiple processors and higher memory X Libraries UNIX X Windows Requirement If the machine that is to run the ON server runs a UNIX-based operating system, an important prerequisite for that machine is that it have the X libraries installed. A fully functional X Windows system is not necessary, but the X libraries need to be available so that ON can create the charts and graphs on the monitoring pages. Make sure all of these libraries exist on your system. The X libraries are are almost certainly available as a package for your Linux distro. Here are the packages required for some of the more common distros: RHEL 5 and Fedora Core 8: libxp package RHEL 4: xorg-x11-deprecated-libs package (v6.8.1 or later) RHEL 3: XFree86-devel package Fedora Core 2: xorg-x11-devel package Debian: xlibs-dev package Note: If you have installed JON using a non-64-bit distribution onto a machine with a 64-bit architecture, you should make sure that the 32-bit versions of the X libraries are installed. Note: If you have a base install of RHEL5, with NO X installed (e.g. prepping for a headless server) then you need to install more than just the libxp package. You also need to install: libx11, libice, libsm, libxt, libxext, libxtst, libxau and libxdmcp Here are the specific X shared libraries that the Sun JDK for Linux x86 (specifically, libawt.so) is linked against: /usr/x11r6/lib/libxp.so.6 /usr/x11r6/lib/libxt.so.6 /usr/x11r6/lib/libxext.so.6 /usr/x11r6/lib/libxtst.so.6 /usr/x11r6/lib/libx11.so.6 /usr/x11r6/lib/libsm.so.6 /usr/x11r6/lib/libice.so Hardware Next, hardware must be allocated to run the ON server. Minimum system requirements are listed in the tables below: 20

24 Chapter 2. Installation T able 2.1. Hardware Requirements (Small Deployment) T ype Operating System Processor Memory Storage Description Linux, Solaris, HP/UX, Windows 1 x 1GHz x86 or 1 x 450MHz SPARC or 1 x 440MHz PA MB 1 GB T able 2.2. Hardware Requirements (Medium Deployment) T ype Operating System Processor Memory Storage Description Linux, Solaris, HP/UX, Windows 2 x 2Ghz x86 or 2 x 450MHz SPARC or 2 x 440MHz PA GB 1 GB JRE If you are installing JBoss ON with an external JRE instead of the embedded JRE, ON requires the following versions: T able 2.3. JRE Requirements JBoss ON JRE version Server JRE Agent JRE or JRE Clock Synchronization All Clocks Must Be In Sync You must make sure that all platforms to be imported into your environment must have their clocks in sync with each other and with the ON Server. If the times are off, metric values/graphs will be skewed and problems will occur when you attempt to import resources into your inventory Database Preparation The options for a backend database for ON are: 1. JON Built-in database (i.e. the embedded Postgres DB) 2. Oracle 8 3. Oracle 9i/10g 4. PostgreSQL 8 All options except JON Built-in database require some further preparation. If the JON Built-in database is 21

25 JBoss Operations Network 1 Users Guide to be used, this section can be skipped Oracle Preparation Three things need to be done for Oracle for use with JBoss ON. More preparation is necessary if the advanced Oracle setup will be used. 1. Create or determine an Oracle instance to be used for the JBoss ON database. It is recommended that the Oracle server to be used by JBoss ON be running on its own hardware. Install Oracle on the machine to be used, and create a database (you can the database whatever you want). The Oracle instance will be ready for the next step. 2. Create the user JBoss ON will use to access Oracle. There are several ways to create a user in Oracle. One way is with SQL*Plus. First, log into the Oracle instance as the system user with SQL*Plus, then issue the CREAT E USER command: SQL> CREATE USER jon IDENTIFIED BY jon; The above command would create a user d 'jon' with a password of 'jon'. 3. Grant the required permissions to the Oracle user. The Oracle user must possess the connect and resource roles. This can be easily done in SQL*Plus with the GRANT command: SQL> GRANT connect, resource TO jon; Oracle is now ready to accept a JBoss ON installation Advanced Oracle Configuration T his is optional configuration that can help Oracle perform very well with very large JBoss ON environments. T his configuration is not necessary for small to medium environments. An example of an environment where this type of configuration would help performance is an environment with hundreds of JON Agents. Warning If you follow the recommended advanced Oracle instructions below, you must utilize the "Advanced Database Install" option within the installer. If you don't use this option, all tables will be placed into the Oracle user's default tablespace and the installation will fail. 1. Create a new database using Oracle Database Configuration Assistant. Select New Database (Includes datafiles = No). Decline to install the Example Schemas to save space. 2. If this advanced configuration is being used, Oracle should be installed on a dedicated host. So select Typical Memory configuration. Select OLTP as the type of database sizing to use. Allocate as high a percentage of system resources as you can afford. T his should be 70-90%, ideally in the higher range. 3. Create the following tablespaces: RESOURCE_01 (with 2 x 512MB datafiles) - used for all non-metric data 22

26 Chapter 2. Installation RESOURCE_IDX_01 (with 2 x 512MB datafiles) - used for indexes on above data MET RIC_01 (with 4 x 1024MB datafiles) - used for all metrics (including response-time) MET RIC_IDX_01 (with 4 x 2048MB datafiles) - used for all metric related indexes Locally Managed Freelists All tablespaces should have locally managed freelists. T he default behavior is automatically managed freelists, and that has an adverse effect on our ability to do concurrent inserts. So set Manual Segment Space Management. Redo Logging should be turned OFF for all of these tablespaces, by specifying the NOLOGGING clause when creating the tablespaces. In fact, Redo Logging should be turned off for ALL the tablespaces in the same database. This is a major bottleneck for the database, and, in our scenario, we need high throughput, which comes at the expense of recoverability. In other words, export data and back it up when you want to recover. 4. For each of the datafiles created: For RESOURCE_01 and RESOURCE_IDX_01 datafiles, set AUT OEXT END ON and increment by 128MB For METRIC_01 and METRIC_IDX_01 datafiles, set AUTOEXTEND ON and increment by 256MB 5. Create the JBoss ON user. Not all JBoss ON tables specify a tablespace at install time, those that do not will end up in the default tablespace for the user. Therefore, make sure to set RESOURCE_01 as the default tablespace when the user is created: CREATE USER jon IDENTIFIED BY jon DEFAULT TABLESPACE RESOURCE_01; 6. Grant permissions to the new user: GRANT CONNECT, RESOURCE TO jon; Oracle Asynchronous I/O Async I/O Asynchronous I/O is yet another optional configuration that can further increase the performance of an Oracle database serving a large JBoss ON environment. Follow the instructions in this section to enable it. Asynchronous I/O configuration is only supported by Oracle 9i. On Linux, Oracle 9i supports asynchronous I/O but it is disabled by default because some Linux distributions do not have libaio by default. For Solaris, the following configuration is not required - skip down to the section on enabling asynchronous I/O. On Linux, the Oracle binary needs to be relinked to enable asynchronous I/O. The first thing to do is shutdown the Oracle server. After Oracle has shutdown, follow the instructions from here: uninglinuxfororacle.shtml#settingasynchronousio - more information about asynchronous I/O on Oracle may be available there as well. 23

27 JBoss Operations Network 1 Users Guide Note for RedHat 9 Download the src rpm for libaio (libaio src.rpm) and patch it with Redhat Patch before building and installing. T hen, relink Oracle as described above and follow the rest of the instructions to enable asynchronous I/O. T here are two initialization parameters for asynchronous I/O: disk_asynch_io = true # This parameter defaults to true filesystemio_options = asynch # This parameter defaults to unset The second parameter may only apply if you use the file system rather than raw partitions. If your installation uses the init.ora file, or init_yourdatabase_.ora, add the above parameters to it and start the database. Alternatively, if your installation uses the spfile mechanism, which is new in Oracle 9, you have to start the server, log in as SYSDBA through sqlplus, then give the following command: SQL> ALTER SYSTEM SET filesystemio_options = ASYNCH; This default invocation changes the parameter value both in memory and on disk in the actual spfile (this is assumed to take effect immediately). Display a parameter's value by using the SHOW PARAMET ER parameter; command. You will know that your installation uses spfiles if: Editing init.ora doesn't accomplish anything there is a file d spfileyourdatabase.ora in \${ORACLE_HOME}/dbs PostgreSQL Preparation PostgreSQL requires a few changes to the database configuration. Edit the postgresql.conf file and change or add the following: ## not necessary if the database is started with the -i flag tcpip_socket = true ## performance changes for JBoss ON fsync = false shared_buffers = work_mem = 2048 statement_timeout = checkpoint_segments = 10 Additionally, depending on the OS you are using, you may need to adjust some kernel parameters as described here. You will need to create a role that JBoss ON will use to connect to PostgreSQL. For more details of how to do that see here. Finally you'll need to update the pg_hba.conf file to allow the role you just created to connect from the machine the JBoss ON Server will be installed on, e.g. localhost. For more details of how to do that see here Restart PostgreSQL, so the changes take effect. The database is now ready to support a JBoss ON 24

28 Chapter 2. Installation installation Installing JBoss Operations Network Download JBoss ON T he first step is to download JBoss ON from the JBoss Network Customer Support Portal. Once you've logged in, choose Software and then Operations Network from the menu on the left. From the list of software which is shown, choose the distributions which are appropriate for your environment. T he various distributions that are available are described below Available Distributions Full Installer vs. Agent-Only Installer The full installer includes all three components of JON - the server, the agent, and the shell. It is intended to be run on the machine that will act as the JON server (in the standard JON architecture, there is a single server and one or more agents). The server component also includes the JON GUI, which is Web-based. The shell component is a command-line interface to the JON server. It is provided as an optional alternative to the Web GUI. The agent-only installer is intended to be run on each machine that you want to be managed by JON. It is not necessary to run the agent-only installer on the server machine, since the full installer includes the agent, as well as the server. Refer to Section 2.3, Agent-Only Installation Platforms JON is supported on Windows, Linux, Solaris, and HP-UX (PA-RISC). For full details, see Section , Platforms. T here is a separate installer for each supported platform. Choose the installer that corresponds to your target platform. If you are installing the JON server, v1.2-sp1 or later, on Windows, you can choose between two installers - a GUI installer or a text-based installer. T he GUI installer is recommended for most cases, since, unlike the text installer, it configures Windows services for the JON server and agent. However, if you are upgrading a previous installation that used the embedded PostgreSQL database, the text installer must be used. If you are installing JON, v1.2 or earlier, the Windows text installer is not available. If you are installing JON on a UNIX platform, you will always use a text-based installer. Each platform-specific JON installer is bundled with a JRE and a PostgreSQL database that is built for the corresponding platform. In most cases, these installers should suffice. However, if you prefer to use your own JRE (v1.4.1 or later) instead of a bundled JRE, you can choose to use the no-jre installer instead of a platform-specific installer. Note, this means you will also need to use an external database, since the no-jre installer does not include a database. Refer to Section 2.5, No-JRE Installer Windows GUI Installer Run the installer executable to start it. Upon acceptance of the terms of the license, specify an install path and then select to either install everything (JON Server, JON Agent and JON Shell) or select custom to select specific components. It is recommended that you always install the agent component on the server machine, in addition to the server component. Next, the options are either to use the default installation settings (quick install) or be interviewed for the custom install. Most of the fields are the same as the text installer fields, which are described below. 25

29 JBoss Operations Network 1 Users Guide T ext Installers Unzip the installer zipfile to a temporary directory. Open a command prompt, and start the installer by running setup.bat on Windows or setup.sh on UNIX. The following is a table that documents the available options when running the setup script: option -upgrade -oracle -postgresql -full -highavail description T his argument will start the installer in upgrade mode. The installer will ask for the full path to the JON Server to be upgraded. This option will only upgrade a JON Server. To upgrade agents, simply install a new agent and configure it to connect to the new server. T his argument will invoke the installer in quick install mode for oracle. T his argument will invoke the installer in quick install mode for standalone PostgreSQL. NOT E: The above arguments will not overwrite an existing JON database if one exists on the target database instance. For upgrades and database overwrites, use -full. T his argument will cause the installer to prompt for almost everything. Ports to use, JON admin user, what type of database to use, etc. NOTE: This argument should be used for standalone JON Server upgrades T his argument will cause the installer to prompt for everything above and will have an additional prompt for the high availability server installation options. NOTE: This argument should be used for new high availability JON Server installations. No argument passed to setup.sh will invoke the installer in quick install mode for the built-in database. As there are minimal prompts, this is the option to get JON up and running as fast as possible. In all cases, the first thing the installer prompts for is what components to install. Choose any or all of JON Server, JON Agent or JON Shell depending on what the installation requirements are. It is recommended that you always install the agent component on the server machine, in addition to the server component. Do not use a custom Ant environment (applies only to versions of JON prior to 1.2.SP1) Make sure you do not override Apache Ant settings via system or user-level Ant configuration files. This can break the Ant scripts that are used by the installer. If you have an /etc/ant.conf, $HOME/.antrc, $HOME/.ant/ant.conf or other Ant configuration file, move them out of the way during the time you are running the JON installer. 26

30 Chapter 2. Installation Components The JBoss ON components available to install are JON Server, JON Agent and JON Shell. The JON Server is the heart of JBoss Operations Network and must be installed in one location. The JON Agent is an agent that sits on each managed platform and reports to the server. The JON Agent is installed in at least one location. The JON Shell is a command line interface to JBoss ON. It is useful for scripting multiple operations and general JBoss ON access from a command line environment. T he JON Shell does not need to be installed for JBoss ON to run, it is purely to facilitate access to the server. It is capable of connecting to the server remotely and can therefore be installed anywhere JON Server Installation When installing the JON Server using the -full option, a series of questions are asked to aid in configuration. T hese are detailed below: High Availability This is only an option if the UNIX installer is invoked with -highavail. This allows for installation of a distributed JBoss ON Server environment and requires an Oracle backend database (Postgres can be tried but may not be capable of handling the load in the typical environment in which this -highavail installation is needed). Create JON Database This will create a new JON database. If this is an upgrade of a JBoss ON installation with an Oracle backend, the correct answer would be 'no' so that the existing database will be used. This option defaults to yes. JON Server HTTP Port Allows for specification of an alternate HTTP port. Defaults to JON Server HTTPS Port Allows for specification of an alternate HTTPS port. Defaults to JON Server JNP Port Allows for specification of an alternate JNP port. Defaults to Any changes to this setting are currently ignored by the Windows Installer. JON Server MBean Port Allows for specification of an alternate MBean server port. Defaults to Any changes to this setting are currently ignored by the Windows Installer. JON Server Base URL Allows for specification of the URL used to access the JON Server. This value is used in alert notification s. T his value can also be changed from the Server Administration page by an JON administrator after the product is installed. Defaults to 27

31 JBoss Operations Network 1 Users Guide JON Server Alert Notification sender This is the 'From:' address on alert notification s sent from JBoss ON. Note that some mail servers require a valid domain in the From field. JON Backend Database Select the backend database JON Server will use. Options are the JON built-in database (PostgreSQL 8.1), Oracle 8, 9i, 10g and PostgreSQL. T his option defaults to Built-in Database. JON Administrator user Allows for specification of the user of the original JON administrator (the "root" user). Default is 'jonadmin'. If you are upgrading, be sure to enter the existing admin's user. JON Administrator password Specify a password for the JON administrator. The installer will not echo the password and will prompt for it twice so it can be verified. The default is only for use when the installer is invoked with one of the quick installation options. If -full or -highavail is used, there is no default and a password must be entered, otherwise the default is 'jonadmin'. If you are upgrading, be sure to enter the existing admin's password. JON Administrator address This is the address assigned to the JON administrator. This is the address alert notifications will be sent to if the administrator user gets configured to receive alert notifications. JON Server Authentication data source T his allows specification of an alternate LDAP data source for the JON user authentication. T he default is the JON Database JON Agent Installation For UNIX installations, the only questions the agent installer will ask is where to install the agent. If an agent installation exists in that location, the installer will ask if you want to overwrite it. However, the first time the JON Agent is started, there are some questions to be answered. The agent needs to know how to contact the JON Server, so that information must be supplied to it. For Windows installations, the installer asks where you want to install the JON Agent and, if you are doing an agent-only install, it also asks for all the JON Server connection information. For a full install (JON Server and JON Agent), the installer automatically configures the JON Agent to connect to the Server. Firewall Issue If you are doing an agent-only install on Windows and the Windows Firewall is active, open port 2144 in the firewall before installing the JON agent (2144 is the default port the agent will listen to). In the event that the port was not opened prior to the JON Agent installation, the problem can be fixed by 28

32 Chapter 2. Installation opening a DOS Command Shell and from the JON install directory agent-x.x.x directory run jon-agent.exe setup The above command only works if the JON Agent is running. If it is not running, start it from the Windows Service Manager first JON Shell Installation The only question the shell installation will ask is the location to install. After all questions have been correctly answered, the installer will install the selected JBoss ON components JON Server Startup Once the installer successfully installs the server, it requires no post-installation tasks; it can immediately be started. T he Windows installer starts the server automatically upon successful installation. T he JON Server and Agent both get installed as Windows services, so starts and stops can be managed from the Windows Service Manager. UNIX users start the server by executing 'serverx.x.x/bin/jon-server.sh start'. T he script backgrounds itself immediately, but will display some startup information on stdout. Both Windows and UNIX users can see the full server log (along with verbose startup information) at server-x.x.x/logs/server.log. When the server has started, the log will display that information and UNIX users will see a message displayed on stdout JON Agent Startup This section will cover the questions to be answered during the first start of an agent. But be aware that the JON Server must already be running before starting an agent for the first time. Windows users start the agent service in the Windows Service Manager. Starting the agent the first time is not necessary as the Windows GUI Installer does that automatically following a successful installation. When using the Windows Text Installer, it is necessary to first start the JON Server, configure the JON Agent, install it as a service and then start the service: start the JON Server (see Section , JON Server Startup ) configure the JON Agent: agent-x.x.x/jon-agent.exe -start after setup is complete, press Ctrl+C to stop the agent. install the JON Agent as service: agent-x.x.x/jon-agent.exe -i start the new JON Agent through the Windows Services tab On UNIX, the JON Server does not get started after installation, it must be started manually. If the server is not running yet, see the JON Server Startup section above. UNIX users start the agent by executing: agent-x.x.x/jon-agent.sh start When the JON Agent starts up for the first time, a series of questions are given to establish network communications with the JON Server. T hese are detailed below: JON Server IP Address Supply the IP address of the running JON Server. If the agent is on the same host as the server, the answer can be to communicate on the loopback interface. 29

33 JBoss Operations Network 1 Users Guide Server <-> Agent secure communication This option tells the Agent to communicate with the Server over HTTPS. If secure communication is not necessary (for instance, if the agent and server are on a private network), saying 'no' here will increase the performance of agent/server communication. T he default is to use secure communication. JON Server Port Supply the port that the JON Agent will communicate to the JON Server on. If the answer for the previous question was 'yes' to communicate securely, the HTTPS port must be supplied. If the answer was 'no' (not to communicate securely), the HTTP port must be supplied. JON Login Supply the user of a JON user with sufficient permissions to create resources in JON. T he JON administrator has such permissions. JON Password Supply the password for the user entered above. JON Agent IP Address Supply the IP address that the JON Server will use to contact this agent. If the agent is on the same host as the server, the answer can be to communicate on the loopback interface. If there is a firewall between the JON Server and Agent, this would be the IP address of the firewall. JON Agent Port Similar to the previous question, this is the port the JON Server will communicate to the agent on. Note that changing this value will not change the port that the agent binds to on the machine. The port that the agent binds to is set in the agent.properties file. The property to set is agent.listenport. T here is no agent.listenport specified by default, so the agent listens on 2144 by default. If the agent.listenport is changed, this question must be answered accordingly JON Agent Shutdown The JON Agent can be shutdown by executing agent-x.x.x/jon-agent.sh stop Connection information can be displayed from a running agent by executing agent-x.x.x/jon-agent.sh status Laptop Installations If you installed JBoss ON on a laptop or other computer that regularly disconnects from the network, dynamically obtains new IP addresses, connects to a VPN or experiences other network addressing changes then please see Section 2.1.4, Laptop Installations for additional setup that is required. Do so prior to starting the JON server or agent. 30

34 Chapter 2. Installation 2.2. QuickStart Guide JBoss Operations Network is designed to be management software that can be quickly installed and integrated into your existing IT infrastructure. T his document will detail how to get JBoss Operations Network up and working for you as quickly as possible Check for supported platforms See Table 3.1, Supported Platforms for a list of platforms on which we support the JBoss ON Server and Agents Download JBoss ON T he first step is to download JBoss ON from the JBoss Network Customer Support Portal Download Page. Once you've logged in, choose Software and then Operations Network from the menu on the left. From the list of software which is shown choose the distributions which you want and which are appropriate for your environment. T he "Full" distributions include the JBoss ON Server, Agent and Shell. T he JBoss Operations Network requires a Java Runtime Environment to run, so it comes bundled with platform specific JREs. If you already have a JRE (1.4.1 or higher), and want to use that, you can download the nojre or "Platform Neutral" version. If you are using the nojre package, be sure that your JAVA_HOME environment variable is set correctly. If you have a particular JRE you want JBoss ON to run on, you can also set JAVA_HOME to point to it Unpack the JBoss ON package Now that you have downloaded the package, you must unpack it. The.tgz packages are gnuzipped tar archive files and can be unpacked using gnu tar: tar zxvf <jon-installer.tgz file> Windows.zip files can be unpacked using any Windows zip utility. Windows.exe binaries can be executed directly on Windows Install the JBoss ON Server and Agent Unix platforms start the JBoss Operations Network installer with this command:./jon-installer/setup.sh On Windows, with the nojre JBoss ON package, you can execute setup.bat in a command shell:./jon-installer/setup.bat or double click the JBoss ON Windows installer executable. T he following instructions will explain the command line installation in detail. For Windows graphical installation, refer to the Install Guide. Initializing JBoss Operations Network Installation... Choose which software to install: 1: JON Server 2: JON Shell 3: JON Agent You may enter multiple choices, separated by commas. Install a JBoss ON Server and Agent by entering 31

35 JBoss Operations Network 1 Users Guide 1,3 Next, you need to decide where to install the JBoss ON Server: JON server installation path [default '/usr/local']: Install JBoss ON to any location you wish - you must have write access to that location, of course and the directory must already exist. JBoss ON agent installation path [default '/usr/local']: The agent install path can be the same as the server install path (they install themselves to different subdirectories). At this point, the installer will begin installing the JBoss Operations Network Server and Agent. When the installation has completed, the installer will exit and you can move on to the next step Start up the JBoss ON Server When the installer has successfully exited from the previous step, it will display the full path to the startup script. The command to start the JBoss ON Server will be: /usr/local/server-*/bin/jon-server.sh start Of course, if the install directory was not /usr/local, the path will be whatever the install directory was set to. Windows users can execute jon-server.exe start If the Windows graphical installer was used, there will be a Windows Service for the JBoss ON Server that will have been started automatically when the installer finished. For a successful command line startup, you should see this output after executing the startup script: Starting JON server... Initializing JON server configuration... Starting JON built-in database... JON built-in database started. Booting the JON server... JON server booted. Login to JON at: T hat should be displayed relatively quickly, and the server will start up in the background. Depending on the hardware, the JBoss ON Server can take several minutes to completely boot. The JBoss ON URL will display startup status until the server is completely booted and then it will display the login page. When the JBoss ON Server has booted, the next step is to start the JBoss ON Agent Start up the JBoss ON Agent Start up the JBoss ON Agent using the jon-agent.sh (or.bat) startup script: /usr/local/agent-*/bin/jon-agent.sh start The first time you start the JBoss ON Agent, it will ask you some questions so it knows how to connect to the JBoss ON Server: 32

36 Chapter 2. Installation Starting agent - Unable to load agent token file. Generating a new one... Done - Invoking agent Agent successfully started [ Running agent setup ] What is the JBoss ON server IP address: This Quick Start Guide is assuming the JBoss ON Server and JBoss ON Agent are installed on the same machine. T herefore, the correct answer to this question is Next, you must determine if the agent will communicate to the server via SSL. Should Agent communications to JBoss ON always be secure [default=no]: It is generally safe to take the default of 'no' here. In this case, since the agent and server are on the same machine, it is very safe to say 'no'. SSL would be useful if you agent and server needed to use the open internet to communicate to each other. What is the JBoss ON server port [default=7080]: 7080 is the default HTTP port for JBoss ON. Press enter to accept the default. - Testing insecure connection... Success What is your JBoss ON login [default=jonadmin]: What is your JBoss ON password: T he default user for JBoss ON is 'jonadmin' and the default password is also 'jonadmin'. Enter that information here. What IP should JBoss ON use to contact the agent [default= ]: Depending on the host configuration, the default here may be the IP address of the machine. Since the agent and server are on the same machine, communication over the loopback device is the most efficent. Enter T hen continue on with the installation: What port should JBoss ON use to contact the agent [default=2144]: 2144 is the default port the JBoss ON Agent listens on. Press enter to accept the default. 33

37 JBoss Operations Network 1 Users Guide - Received temporary auth token from agent - Registering agent with JBoss ON - JBoss ON gave us the following agent token Informing agent of new JBoss ON server - Validating - Successfully setup agent The JBoss Operations Network Server and Agent are now up and running Login to JBoss ON Using a web browser, navigate to port 7080 of the host you installed JBoss ON on. An example might be You will be presented with the JBoss ON Login screen. Enter the default JBoss ON Administrator user and password of jonadmin and jonadmin Upon successful login, you will be at the JBoss ON Start page. Refer to the "JBoss Operations Network Start page" section in the JBoss Operations Network 1 Reference Guide available from From there various sections of the console can be reached. Hitting the Dashboard link will bring up a page from where the following can be performed: Import a new platform T he Dashboard consists of several portlets that provide different types of information. T he one portlet that will stick out is the Auto-Discovery portlet. It will have several entries for the newly registered Agent on the local machine. The radio button next to this host should be selected. If there are other entries and the radio button is not selected, please select it. Now click IMPORT to import the platform and it's servers. The process adds the platform and servers to the JBoss ON inventory as well as enables the default metrics for collection on the newly imported resources. You will know this process has been started when you see the host for the platform in the Recently Added Resources dashboard portlet View platform metrics T he default metric collection intervals for platform metrics range between 1 minute and 10 minutes depending on the metric. So since we just scheduled these metrics for collection, we need to give it a few minutes to collect the first data points. After the first metrics have been collected, we can view them from the platform metrics page. To view the metric data, it is necessary to navigate to the platform current health page. There are several ways to do this. One way is to click the Browse Resources link at the top of most any page. Next click on the link of the platform you wish to view the metrics for. This is the Platform Current Health page. This page displays a wealth of information on the health of the current resource (in this case, a platform) as well as information on resources related to this resource. To get more specific metrics for this platform, click on the METRIC DATA mini-tab. This list of metrics represents all the metrics for which JBoss ON has data for in the display range. In our case, we just started metric collection a few minutes ago, so there will be very little metric data here. There are still a few things you can do to explore the possibilities in JBoss ON. Each metric has an info icon on the right side of the page. Clicking this will open a window with detailed information on the specific 34

38 Chapter 2. Installation metric. You can chart the data for any metric by clicking the chart icon to the right of each metric or simply clicking on the metric itself. Alternately, you chart multiple metrics at the same time by checking the checkbox next to the desired metrics and clicking CHART SELECTED METRICS Complete JBoss Operations Network is now up and running for your environment. At this point, you can add more resources to your JBoss ON Inventory (refer to Chapter 3, JBoss ON Inventory), install more JBoss ON Agents on other machines to import more platforms, configure metrics (refer to Section 4.3, Managing Metric Data ) and alerts (refer to Chapter 5, Alerts for your resources, control your servers (refer to Chapter 5, Alerts) and the list goes on. Continue reading this documentation for more on how JBoss ON can help manage your resources Agent-Only Installation JBoss ON provides agent-only packages to make it easier to deploy agents to multiple machines. T he packages are considerably smaller than the full install package, so you don't have to copy a huge installer around to your machines if you only want to install the agent. The agent-only installation is meant to be very quick and easy Agent-Only Install on Windows T o install a JON Agent on Windows using the agent-only installer, simply download the agent-only package for windows from the JBoss Customer Support Portal and run the.exe binary. Enter the necessary information so the agent can communicate with the JON Server and click OK to install. Note The values entered in the agent-only installer are not validated in any way before the agent is installed. T hey are simply put into the agent.properties configuration file so the agent can use them when it starts up. If the values need to be edited after the install, simply open agent.properties in a text editor and scroll to the bottom of the file. Another thing to be aware of is that changing the agent connection port from 2144 to another value does not change the port the agent actually listens on. It only changes the port the JON Server will use to contact the agent. This is useful if you have a different number port on a firewall between the server and agent mapped to port 2144 of your agent machine. To change the port the agent listens on, add a line like this to your agent.properties: agent.listenport = 2555 Start the agent by executing "jon-agent.exe start" from the agent installation directory Agent-Only Install on UNIX T o install a JON Agent on any UNIX platform using the agent-only installer, simply download the agentonly package for the platform you wish to install on from the JBoss Customer Support Portal and unpack the.tgz to the location you wish to install to. 35

39 JBoss Operations Network 1 Users Guide Warning When unpacking the HP_UX agent-only installer.tgz file, use GNU tar rather than HP-UX tar (/usr/bin/tar). T his is necessary because the.tgz file contains some file s that are longer than 256 characters, and HP-UX tar does not support long file s. Start the agent by executing the jon-agent.sh shell script from the agent installation directory and passing it a 'start' argument like this:./jon-agent.sh start The first time the agent is started, it will initiate a dialog and you will need to answer a few questions to tell it how to contact the JON Server. On subsequent startups, the dialog will be skipped. The agent listens on TCP port 2144 by default. To change this behavior, add a line like this to the agent.properties configuration file: agent.listenport = General Notes Each platform specific package comes with a JRE for that platform. If you already have a JRE, you can choose to download the no-jre package, as the size is significantly smaller. Then you just need to make sure the JAVA_HOME environment variable is set to the location of your JRE before starting the JON Agent. Any change to agent.properties while the agent is running will require a restart in order to pick up the changes High Availability Installation JBoss ON can be installed as a cluster in large scale environments. This is called a high availability installation. In a JBoss ON high availability installation, there are two node types: Cluster Master Data Processor The Cluster Master is what it sounds like - it is in charge of the JBoss ON Cluster. The Cluster Master handles automatic database maintenance, calculates automatic metric baselines and serves the JBoss ON GUI among other things. Each cluster may contain one and only one Cluster Master. T he Data Processor handles connections from JON Agents, processes the data received and fires alerts. A cluster can contain as many Data Processor nodes as required to manage an environment. The number of JON Agents a Data Processor node can handle depends on several factors (hardware, number of metrics each agent is collecting, etc), but a general guideline is that a Data Processor can handle between agents reporting in. All data processor nodes require direct access to the backend database. Agents never access the database, so they do not need connectivity to the database machine. T he JBoss ON High Availability cluster requires a standalone backend database. See the database preparation section for options. T he built-in database cannot be used for a High Availability installation. 36

40 Chapter 2. Installation When installation preparation is complete, installation of the Cluster Master can begin by executing the setup.sh installer with the -highavail flag. Select the JON Server component to install and enter the path to install to. Next is the selection for the node type: Choices: 1: Standalone JON Server 2: Clustered JON Server Node: Cluster Master (requires Oracle database) 3: Clustered JON Server Node: Data Processor (requires Oracle database) 4: Clustered JON Server Node: GUI (requires Oracle database) What kind of JON server should be installed? [default '1']: Select 2 for Cluster Master and the following options will be presented next: What IP address should the cluster server bind to? [default '*.*']: What is the cluster's multicast address? [default ' ']: What is the cluster's multicast port? [default '3030']: The defaults are fine or customized parameters can be specified, if the environment requires it. The remaining installation process is exactly like the full install process documented in the JON Installation section. Server's Agent Must Use True IP As you probably know, the JON server machines (that is, the platforms that the JON servers (master or data nodes) are running on) can be monitored, too. This means that there can be a JON Agent running as well as the JON Server on the machine. When you use this configuration, be sure to setup the agent's agent.setup.agentip property (the IP that the server uses to contact the agent) to its true IP, do not use or localhost. This is true for agents running on the master JON Server machine and all JON Server data node machines - all their agents must not use To be more specific, the agent IP addresses must be visible by both the agent's data node and the master node. All Clocks Must Be In Sync You must make sure that your data nodes and master node machine clocks are all in sync with each other. This is also true with all agents that you install in your environment. All clocks must be in sync. After installation is complete, but before starting all of your Data Processor nodes, you should first start up your Master node and make sure it starts up fully. This gives the master a chance to initialize the database. You should only have to worry about this the very first time you start up after a new HA install has been performed. After the Cluster Master has been installed and started, the Data Processor node(s) can be installed. Simply run the installer with the -highavail flag on the machines that will run the data processor nodes and select 3 for Data Processor when prompted. The interface to bind to must be specified, but after that there are no further High Availability specific questions to answer for the Data Processor installation. It is important to specify the correct JDBC URL, user and password for the database that the Cluster 37

41 JBoss Operations Network 1 Users Guide Master is on because that is the database the node will use when running and that is where the cluster configuration is kept. You need to make sure you elect to "upgrade the database" when the installer detects the already existing schema (the one that the master install created for you). T he addresses should be the same as what you entered in the master installer, but assuming you "upgrade the database" the ones you entered in the master database will override any addresses you specify when installing the data nodes. You can modify the addresses later in the GUI once the master has started up. Once the Data Processor node(s) are successfully installed and started, JON Agents can be pointed at them. Simply install JON Agents as normal and specify a Data Processor node information as the JON Server IP address and port. The agent will register itself with the Data Processor which will report into the other nodes in the cluster. Users must access the JBoss ON GUI from the Cluster Master. In the current versions, even the Data Processor nodes serve up a GUI but their behavior will not be correct. In a future version, we plan to disable the GUI in all but the Master node. All aspects of using JBoss ON will remain the same as a normal installation. A High Availability installation is completely transparent to the user. You will note based on the above description that the of this feature isn't quite describing what it does. High Availability to some people mean failover capabilities. That is, if a JON Server goes down, you might expect its agents to switch over to another JON Server. That isn't what this High Availability feature is or does. High Availability refers to having some agents operate properly if one JON Server goes down (because they are talking to another JON Server that is running). This also allows you to have a high number of agents enter into your JON environment. In retrospect, this feature should have been called High Scalability for these reasons. It provides a higher degree of scalability by allowing you to add many more agents to your environment that a single JON server can't manage on its own. It does provide some amount of HA capability in that if on JON Server goes down, your entire network of agents don't lose contact with a server. Only the subset of agents that were assigned the downed JON Server will fail to communicate with it. Note that even if an agent fails to communicate with its JON server, you will not lose metric data. The agent will spool metric data to the local file system until it can communicate with the JON server, at which point it will upload the spooled data No-JRE Installer Overview Unlike the other JON installers, the no-jre installer does not include a Java Runtime Environment (JRE) or the PostgreSQL database. The installer should be used only if you would like to use a separate v1.4 JRE installation to run JBoss ON instead of a bundled JRE. If you choose to use the no-jre installer, you will not have the option of using a preconfigured embedded database; you will need to install and configure either PostgreSQL or Oracle separately Using the No-JRE JBoss ON Installer First, make sure you have a suitable JRE installed. The Sun J2SE JRE version 1.4 or higher is recommended. The full SDK is not necessary, but will work just fine. Second, make sure you have PostgreSQL or Oracle installed and have created a database for JON. Instructions for installing and configuring PostgreSQL are included in Section 2.5.3, PostgreSQL Server Installation and Configuration. Next, download the No-JRE JBoss ON installer and unpack it. Then open a command prompt and 38

42 Chapter 2. Installation change directories to the unpacked installer. Execute setup.bat (Windows) or setup.sh (UNIX) to start the installer. The rest of the installation will proceed exactly the same way as the command line install invoked with - full. Please see Section , T ext Installers for details. Unless you have a mail server running on the local machine, you will need to supply a mail server that JBoss ON can use to send alerts. Aside from that, the only other thing you need to do is make sure to select the appropriate database option and enter the JDBC URL that will connect to the database you set up (e.g. jdbc:postgresql://localhost:5432/jon or jdbc:oracle:thin:@localhost:1521:jon). Warning In order to run the no-jre version of JON, the JAVA_HOME environment variable must be set to the location of a v1.4.1 or later JRE or JDK. The JON start script uses JAVA_HOME to locate the JRE that will be used to run JON PostgreSQL Server Installation and Configuration Download PostgreSQL Server from Unzip the package and run the msi installer. Follow the graphical installer to install PostgreSQL. It will need to create some operating system users, but the process is pretty straight forward. Once the installation is complete, the installer will start up your database. Next, you will need to create a database for JON and a user to connect with. An easy way to do this is to use the pgadmin III tool included in the PostgreSQL installation. Connect with the admin account you specified during installation. Create a user called jon with password jon (these are suggestions, you can use whatever values are appropriate for your environment; just be sure to remember the user and password). Grant the jon user permission to create databases. Next, create a database d jon and make the owner the jon user from the previous step. The final step is to edit the memory settings of the database as described in Section , PostgreSQL Preparation. Remember to restart the PostgreSQL Server after updating the configuration. PostgreSQL is now ready to accept an installation of JBoss ON Installing on a Laptop This page contains information if you are installing JBoss ON on a laptop. In fact, this information isn't limited solely to laptops - it is pertinent for installations on any computer that regularly sees changes with its network connectivity (e.g. disconnects from a network entirely, connects to a VPN, or obtains DHCP IP addresses). Usually this is for the purposes of evaluating or demo'ing JBoss ON. During the installation of JBoss ON, and initial import of resources, you should make sure to have a working network connection. An internet connection is not required, but a working network interface, other than , is necessary. Instructions for adding a new loopback network adapter on Windows can be found here. In order to run JBoss ON on a laptop, you'll want to configure it in a way that won't tie the inventoried servers to the IP address or host of the platform. This can be done by configuring the agent to use a hardcoded server as well as using loopback addresses for client and server communication end points. 39

43 JBoss Operations Network 1 Users Guide Before starting the agent and inventorying and services, edit the agent.properties file to add the hardcoded platform, such as "platform.fqdn=demo". You should always shutdown the agent before making these changes, and restart it after the changes are made. When configuring the agent, utilize " " for both the server connection address and the agent address. In other words, edit the agent.properties file and add the following section: # Don't tie inventoried servers to an IP address: platform.fqdn=demo agent.listenip= agent.setup.serverip= Finally, when moving your laptop between networks, or upon being assigned a new ip address, you should restart the JON server. T he server uses remoting that may utilize the old address, causing failures in connecting to the business tier. Once you've restarted, you should see an auto-discovery noting that the platform IP address changed. Run this import to update the platform configuration. If you follow these steps, you should be able to use the same inventoried local servers and configurations while on any network What to do if not on a network If, for some reason, your computer is not connected to any network (that is, none of your network interfaces have an active connection), then you can simulate the existence of a network in the agent with the "platform.ip" property. If you are not on a network, pick some dummy IP address you want to assign to your computer and assign that IP address to the agent.properties "platform.ip" property: platform.ip= In this example, after I start the agent, it will report that it is connected to a network with the IP address of Be Careful It is critical that the dummy ip address you choose is not one that your computer could be associated with. So don't make platform.ip= for instance. If, at some point, your computer reconnects to a network, you must be sure any IP addresses assigned to your computer does not match your platform.ip address. Otherwise, an error will occur within the server as it tries to add the new agent's IP address. Again, do not forget to shutdown the agent before making these changes and do not import any resources into the JON Server until these changes are made. An agent restart is required to pick up these changes Uninstalling JBoss Operations Network Uninstalling JBoss ON is very easy. On Windows, a user with administrator privileges can go to the Add/Remove Programs section of the Administrator T ools and select the JBoss Operations Network component for removal. Some of the files themselves remain on the disk - once you uninstall, you should then perform a file delete to remove the files from the actual file system. On UNIX, all the components of JBoss ON are contained in a single location, so a user with adequate 4 0

44 Chapter 2. Installation permissions can simply delete the entire installation directory and its subdirectories. 4 1

45 JBoss Operations Network 1 Users Guide Chapter 3. JBoss ON Inventory JBoss Operations Network stores information about your managed systems in an inventory repository. T he items in this repository are called resources. When you use JBoss ON's Auto-Discovery feature on your platforms, JBoss ON will discover servers and services running on those platforms, and you can then choose to import them into the inventory (refer to Section 3.1, Auto-Discovery ). If there has been a change in the managed resources on a platform since the last time you executed an Auto-Discovery, JBoss ON will alert you to the change. You can use the JBoss ON GUI Console to manage, monitor and control the systems you have placed in inventory. Starting from the Dashboard, you can view the status of all your managed systems Auto-Discovery JBoss Operations Network provides an intelligent and efficient Auto-Discovery feature to collect resource-specific details about your environment and include them in the JBoss ON inventory. Auto- Discovery discovers and collects platform (refer to Section , Platforms ), server (refer to Section 3.3, Servers, and service (refer to Section 3.4, Services ) property information about each of your managed systems that you can then add to the inventory repository. JBoss ON can auto-discover all of the resources running on these machines from OS level properties, to servers (such as Apache web servers), to services (such as Apache virtual hosts). T here are two Auto-inventory methods used within JBoss ON: Auto Scan Auto Scan is a special type of auto-discovery that requires no end user intervention. Auto Scans typically do not take long, and are not CPU intensive. Auto Scans are automatically run by the JON Agent on a daily schedule, as well as each time an agent is started. T he implemention for Auto Scan varies among platforms. On Unix type platforms, the Auto Scan is performed using a process table scan, which looks for processes that match a given pattern. On Windows platforms as well as a process table scan, a simple registry scan is performed looking for registry keys that installed products register during their installation process. File Scan The second method of auto-discovery is File Scan. This method involved scanning the file system for installed products. T he File Scan method must be initiated by the user using the New Auto-Discovery links found on all platform pages. T he File Scan auto-discovery is more configurable than Auto Scan. You can define parameters to specify which machines should be included in the auto-discovery, which directories should be searched, and which types of servers to discover. You can either perform File based Auto-Discovery on demand or schedule it for a later time. You can also schedule recurring auto-discoveries so that, for example, Auto-Discovery occurs on the last day of every month at 11:55 PM. You can check the status of completed, in-progress, and scheduled File Scan Auto-Discovery actions. When an Auto-Discovery is completed, you can then review the inventory information JBoss ON detected and choose to import it into the inventory repository. From there, you can group resources into applications or other logical groups and manage your inventory using the JBoss ON Console or CLI. 4 2

46 Chapter 3. JBoss ON Inventory Helpful Hints Note that if you have a product that you never want to manage, you can tell the auto-discovery feature to ignore that product during both the auto scan and file scan mechanisms. Create a file on the agent-side called <user home dir>/.jon/installdir.excludes with each line in that file being either the precise install directory to a resource you want ignored or an install prefix ending in '*'. Below is a sample file: # ignore this and other #-prefixed lines, as they are comments # for windows-based agents # ignore only the agent resource in the staging environment # this performs string-equals comparison against the resource's installdir F:\Staging\JBoss ON \agent # for * nix-based agents # ignore all JON resources in the dev environment (performs substring matching) # this performs substring matching against the resource's installdir /dev/jboss ON /* As you can see, the currrent implementation is platform sensitive, meaning that you'll need to use either '\' or '/' for the paths in the installdir.excludes file depending on the operating system of the agent Platforms As defined by JBoss ON, a platform is an operating system running on a physical machine. Each JON Agent is installed onto a platform, where it Auto-Discovers the platform's properties. Servers (refer to Section 3.3, Servers ) run on top of platforms, and Services (refer to Section 3.4, Services ) in turn are supported by servers. JBoss ON supports the following platforms: T able 3.1. Supported Platforms Platform Server Agent Windows (refer to Section 6.1.5, Microsoft Windows Management Support ) Linux (refer to Section 6.1.1, Linux Management Support ) Solaris (refer to Section 6.1.3, Sun Solaris Management Support ) HP-UX (refer to Section 6.1.2, HP-UX Management Support ) AIX (refer to Section 6.1.4, IBM AIX Management Support ).MSI,.ZIP.tar w/setup.tar w/setup.tar w/ setup not supported.msi,.zip.tar w/ setup, deploy tool(ssh).tar w/ setup, deploy tool(ssh).tar w/setup, deploy tool(ssh).tar w/ setup, deploy tool(ssh) 4 3

47 JBoss Operations Network 1 Users Guide Warning Click on a platform for information on exactly which architectures and OS versions are supported for that platform Servers Servers are software products that run on platforms, and support services. T hey provide a communications interface and respond to requests to perform specific tasks. JBoss ON supports the following servers: Application Servers JBoss AS 3.2.x (refer to Section 6.2.1, JBoss AS Management Support ) JBoss AS 4.x (refer to Section 6.2.1, JBoss AS Management Support ) JBoss AS 5.x (coming soon) (refer to Section 6.2.1, JBoss AS Management Support ) Web Servers Apache (refer to Section 6.3.1, Apache Management Support ) Microsoft IIS (refer to Section 6.3.2, Microsoft IIS Management Support ) Sun ONE Web Server (refer to Section 6.3.3, iplanet/sunone Management Support ) T omcat (refer to Section 6.2.2, T omcat Management Support ) T he ON platform is extensible and can be integrated with other server types including databases. Servers are hosted on a platform (refer to Section , Platforms ). A single platform can host a number of different servers. Servers, and consequently the services they provide, can be shared across a number of applications. For example, a single JBoss App Server may contain EJB components used by an online storefront Application as well as a separate order entry Application Services Services are supported by servers (refer to Section 3.3, Servers ), which in turn run on platforms (refer to Section , Platforms ). A service represent an individual piece of software which is dedicated to a particular task. Applications are composed of various services running on servers. An example of a service is an Apache virtual host that is dedicated to servicing requests that are directed to a specific Application. Other examples of services include: T omcat Application Contexts (Webapps) EJB Services Some services are provided by the application server - for example, JDBC connection pools and JMS queues. T hese services are internal to the application server and thus called internal services. Providing these services is the primarily value of an application server. Other services are provided by the application creator such as web apps and Enterprise Java Beans. 4 4

48 Chapter 3. JBoss ON Inventory T hese application services are deployed onto an application server and are called deployed services. Both deployed services and internal services are present in most applications. JBoss ON distinguishes between the two because they have very different roles in an application and significantly different levels of reliability in an Application. Internal services are generally more reliable because they are very extensively tested as part of the server product that provides them. JBoss ON allows you to define Applications that are composed of both deployed services and internal services, but it is generally more useful to define an Application as a collection of deployed services. An application can be composed of clusters of deployed services since most Applications involve clustering to enhance availability and performance. By defining an Application as a set of clusters, you also add a level of indirection, which enables you to add or remove resources from the cluster without having to change the definition of the Application Groups When JBoss ON has detailed inventory information about your managed systems, you can gather services (refer to Section 3.4, Services, servers (refer to Section 3.3, Servers ), and platforms (refer to Section , Platforms ) into useful groups. Grouping resources lets you easily monitor and control your applications and the servers, services, and platforms that make them up. You can create a group from any arbitrary set of resources, based on any attribute, such as geographical location, owner, business unit, and so on. If you group resources that share a common set of monitoring metrics and control actions, that group is considered a compatible group. For example, you can organize all Apache 2.x servers into a compatible group. JBoss ON can control, monitor, and inventory compatible groups. If you group resources that do not share a common set of monitoring metrics and control actions, that group is considered a mixed group. You can view inventory information for a mixed group, but you cannot perform control actions on, monitor, or edit the group's inventory properties as a whole. You can also make groups containing multiple compatible and mixed groups Application An Application is made up of services (refer to Section 3.4, Services ) that are provided by one or more servers (refer to Section 3.3, Servers ) and hosted on a platform (refer to Section , Platforms ). An Application is usually identified by the functionality it provides, such as a purchase order entry application, or an online banking application. T his is known as the Application T ype. When you specify a set of resources as an Application, JBoss ON collects measurement data about the availability, performance, and throughput of that set of resources and correlates it so you can see the service level of the entire Application from end-to-end. You can also easily drill down to the service levels of the individual resources to see how they are performing. T he application model exposes the following resources: the platforms (machine/os combinations), the servers (application, database and Web servers), services provided by those servers (including EJBs, servlets, VHosts and database instances) and most importantly the application that is made up of combinations of those services. T hrough this application model, you can improve service levels, increase utilization of your Web infrastructure and report service levels accurately. 4 5

49 JBoss Operations Network 1 Users Guide 3.7. Inventory XML Descriptor Syntax In addition to auto-discovery, JBoss ON allows for resources to be created using the JON CLI Shell. See the resource commands in the CLI reference for details on these commands. T he import and export process uses the following XML syntax: 4 6

50 Chapter 3. JBoss ON Inventory Tag <hq>: Sub Tag <platform> can be specified any # of times REQUIRED attributes: fqdn type OPTIONAL attributes: certdn comment cpucount cpuspeed dhcp dns gateway ram description location Sub Tag <controlconfig> OPTIONAL Sub Tag <metric> OPTIONAL Sub Tag <metricconfig> OPTIONAL Sub Tag <collect> can be specified any # of times REQUIRED attributes: metric OPTIONAL attributes: 4 7

51 JBoss Operations Network 1 Users Guide interval Sub Tag <resourceconfig> OPTIONAL Sub Tag <customprops> OPTIONAL Sub Tag <ip_addresses> REQUIRED Sub Tag <ip> REQUIRED at least ONCE REQUIRED attributes: address netmask OPTIONAL attributes: macaddress Sub Tag <agentconn> REQUIRED REQUIRED attributes: address OPTIONAL attributes: port Sub Tag <server> can be specified any # of times REQUIRED attributes: installpath type OPTIONAL attributes: autoinventoryidentifier description 4 8

52 Chapter 3. JBoss ON Inventory location Sub Tag <controlconfig> OPTIONAL Sub Tag <metric> OPTIONAL Sub Tag <metricconfig> OPTIONAL Sub Tag <collect> can be specified any # of times REQUIRED attributes: metric OPTIONAL attributes: interval Sub Tag <resourceconfig> OPTIONAL Sub Tag <customprops> OPTIONAL Sub Tag <service> can be specified any # of times REQUIRED attributes: type OPTIONAL attributes: parentservice description location Sub Tag <controlconfig> OPTIONAL Sub Tag <metric> OPTIONAL Sub Tag <metricconfig> OPTIONAL Sub Tag <collect> can be specified any # of times 4 9

53 JBoss Operations Network 1 Users Guide REQUIRED attributes: metric OPTIONAL attributes: interval Sub Tag <resourceconfig> OPTIONAL Sub Tag <customprops> OPTIONAL Sub Tag <application> can be specified any # of times REQUIRED attributes: OPTIONAL attributes: location description businesscontact engcontact opscontact Sub Tag <services> REQUIRED Sub Tag <service> REQUIRED at least ONCE REQUIRED attributes: Sub Tag <dependson> can be specified any # of times REQUIRED attributes: 50

54 Chapter 3. JBoss ON Inventory Sub Tag <group> can be specified any # of times REQUIRED attributes: type membertype OPTIONAL attributes: description location membertype Sub Tag <groupmember> can be specified any # of times REQUIRED attributes: OPTIONAL attributes: type Sub Tag <property> can be specified any # of times REQUIRED attributes: value 51

55 JBoss Operations Network 1 Users Guide Chapter 4. Monitoring T his section contains information about how to monitor resources with JBoss ON Monitoring Basics JBoss Operations Network collects data from Web, application, and database environments (services, servers, and platforms), as well as from clients making requests of an Application. T he data is stored in a centralized repository and correlated according to the resource hierarchy defined in JBoss ON Metrics Metric data is the information JBoss ON collects for a given platform, server or service. The data JBoss ON collects depends on what type of server or service is being monitored. For instance, if you monitor a Linux platform, you can see metric data about total, used, and free swap, total, used, shared, and free memory, total, idle, and user CPU, and more. For an instance of Tomcat, you can see things like JVM total memory, active thread count and thread group count, uptime, process shared memory time, and more. A metric can belong to one of the following categories. Availability (refer to Section , Availability ) Usage (refer to Section , Usage ) Performance (refer to Section , Performance ) Utilization (refer to Section , Utilization ) Baselines A metric baseline is a value that represents the norm for a particular metric. JBoss ON automatically calculates the baselines for dynamic metrics currently being collected for a resource. T he baseline is the average value of a metric based on a time range that you specify (see Section 9.3.4, Automatic Baseline Configuration Properties ). In addition, the high and low range values are also calculated for the metric. You can also simply specify the expected mean, high, and low range values. You can set and chart these values when you chart a metric for a resource. The use of a baseline as an analysis method compares changes in actual data against a baseline value. T his functionality allows you to perform trending analysis, manage SLAs, and monitor overall application health as a form of fault management Problem Metrics JBoss ON automatically tracks the occurrences of the metric value collected for any given metric falling outside (i.e. out-of-bounds, or O.O.B.) of the high and low range set in the baselining process (either automatic or user-defined), and will report the given metric as a problem metric. In addition, when an alert occurs as a result of a metric value collected, the alerting event is also tracked as a problem metric occurrence for that metric. Refer to Section 4.4, Working with Baselines. When you view the monitor visibility Current Health page for any resource, the problem metrics list is displayed in the RESOURCES section of the page. Each problem metric shows its O.O.B. and alerts counts, and can be charted in the INDICATORS section on the same page. T he problem metrics function is also the criteria for which resources are deemed to be problematic. Essentially, any resource that has problem metrics in the specified time range is considered to be a problem resource. 52

56 Chapter 4. Monitoring Metric Display Range T he metric display range specifies the time range JBoss ON considers for displaying the metric data. You can change the display range for metrics JBoss ON collects for all resources. The settings you select are used in all JBoss ON Visibility pages. To edit the metric display range On all monitoring pages, there is a Metric Display Range form. To specify the metric display range going backwards from the current time, select a number in the Last field and the time unit you want to use from the drop-down list, then press the right arrow button to have the changes take effect. To select a date range, click Advanced Settings... If you select Within a Date Range, enter the From and To dates and times. To save your changes, click the OK button Monitoring Visibility JBoss Operations Network provides a convenient way to view the overall health of the resources through the analysis of the metrics collected. T he main monitor visibility page of each resource is known as the Current Health page Navigating to Current Health The Current Health page is the default monitor visibility page that you land on whenever you navigate to the monitoring pages from elsewhere in the product. For example, you can click on any resource from any of the dashboard portlets, with the exception of Auto-Discovery, that displays resources. Similarly, clicking on any resource or clicking on the M icon next to the resource in Browse Resources will take you to the Current Health page of that resource. Refer to "Browse Resources" in the JBoss Operations Network 1 Reference Guide available from If you are already in the monitor visibility pages of the resource, you can click on the INDICATORS tab to reach the Current Health page. Lastly, if you are in the monitor configure pages of the resource, you can click on the Monitor Visibility Tab tab button to navigate to the Current Health page Resources There are several portlets in the RESOURCES column of the Current Health page. The number and types of portlets depend on the type of resource you are viewing Child Resources Health Portlets A Child Resources Health Portlet displays a list of the resource's child resources (as defined by JBoss ON's resource hierarchy) and a snapshot of their current health. There may be more than one child resources portlet displayed for given resource, depending on the type of the resource displayed. T he title of the portlet will reflect how the child resources are categorized. Platforms display two portlets: Platform Services and Deployed Servers. Servers and Applications display two portlets: Deployed Services and Internal Services. Services, Autogroups, and Groups have no child resources. Each resource in the list is displayed with either their or type, if there are more than one child resource of that type (this is an autogroup). If the resource is an autogroup, then the Total column will denote how many resources are included. T he Avail column displays the last known availability value of the resource. 53

57 JBoss Operations Network 1 Users Guide In the right most column of this list is the Up Arrow menu icon. Clicking on this icon or the resource/type will take you directly to the Current Health page of that resource. Hovering over the icon will bring up the Child Resource menu. The menu displays the resource type, last known Usage metric value, and a detailed breakdown of availability (applies to autogroups only) Host Resources Health Portlet The Host Resources Health Portlet displays one or more host resource(s) of the resource you are currently viewing. Each resource is displayed with its current availability, as well as an icon that displays the Host Resource menu. Following list is how the hosts are determined for each resource type: Servers and Platform Services display their host platforms. Services display their host servers. Applications display the servers that are host servers of the services included in the application. T he title of the portlet will specify the relationship described above. Clicking on the Down Arrow menu icon or the resource will take you directly to the Current Health page of that resource. Hovering over the icon will bring up the Host Resource menu, which displays the resource type and the last known Usage metric value Group Member Resources Health Portlet If you are viewing a Group or Autogroup, the Group Members Resources Health Portlet will be displayed. T he portlet lists the member resources of this group, along with each resource's current availability. T he right most column is an icon for the resource's menu. Clicking on the Up Arrow menu icon or the resource will take you directly to the Current Health page of that resource. Hovering over the icon will bring up the resource menu, which displays the resource type and the last known Usage metric value Problem Metrics Portlet T his portlet displays the list of metric display range (refer to Section 4.1.4, Metric Display Range ). By default, it displays only the problem metrics of the resource that you are viewing. However, you can view the problem metrics of other related resources by setting the checkboxes next to any of the other resources and resource types listed above the portlet and clicking the ADD RESOURCES button to display the problem metrics of the checked resources as well. Each problem metric is displayed with two counts: O.O.B. and Alerts. T he Right Arrow menu button appears to the right of the count columns. To display the metric data of the problem metric as a column chart in the INDICATORS workspace on the right-hand side of the page, simply click the Right Arrow icon. Otherwise, hovering over the icon will bring up the context-sensitive problem metrics menu. T he menu has the following data and function: Resource Type - the resource type of the metric. This value will reflect the varying resource types when other resources have been selected above. If the resource type represents more than one resource in the hierarchy, then this item will also display the number of resources that have the same problem metric in parenthesis. Problems Began - the time of the first occurrence of O.O.B. metric value or alert. Chart Metric in Indicators - click on this option to display a chart for this metric in the Indicators workspace on the right-hand side of the page. View Full Chart - click on this option to navigate to the full chart page of this metric. Refer to Section 4.6, Charting Metric Data. Metric Data - click on this option to display a popup window which contains the meta data information on this metric and its data. 54

58 Chapter 4. Monitoring You can also switch the list of metrics to include all metrics, instead of only problem metrics. To view all metrics, click on the Problem Metrics drop-down and select All Metrics. All of the functionality of the problem metrics display apply to the all metrics display Indicators The INDICATORS column on the Current Health page provides a visual workspace for you to analyze and compare metrics in-depth to gain insight into the current health of your resource(s) Availability and T imeline Near the top of the Indicators column is the availability history of the resource. T he availability values span across the defined metric display range (refer to Section 4.1.4, Metric Display Range ). T he overall average availability value appears after the Availability heading. Each availability light reflects the average availability over the time slice represented. T here are four possible color values: Table 4.1. Availability Values Color Table Color Green Red Definition 100% availability 0% availability Yellow Availability between 0% to 100% Gray No availability was collected in the time frame Clicking on any of the availability lights will bring up or move a highlight bar, which conveniently lines up all of the metrics in the indicator charts with the availability light that you are analyzing. Near the bottom of the column is the timeline display. The timeline has the same ability to bring up the highlight bar as the availability history. In addition, hovering over any of the time slice will bring up a popup that specifies the time represented by the square, giving you an idea what occurred in that time frame by lining up the availability and the metric data Indicator Charts and Views Below the availability history is an scrollable (if necessary) area where the metrics are charted, these are known as the indicator charts. T he indicator charts are displayed as column charts, where the area of each column indicates the value range (the high and low values) of the metric. The average value of the metric is indicated by the cross in the column. By default, charts of metrics that have been selected by the plugin as designated metrics for the type of resource you are viewing are displayed. You can choose additional metrics to be charted by selecting them from the Problem Metrics Portlet (refer to Section , Problem Metrics Portlet ). T he indicator charts are stacked vertically so that their X-axis (time) values line up. T his works in conjunction with the highlight bar from the availability and timeline to analyze the metric data across multiple metrics for any given time. T his is especially useful when you are trying to diagnose a problem at a specific time by correlating the important metrics and their metric values. When the number of charts displayed exceed the screen real estate, a vertical scrollbar appears. You can use the scrollbar to navigate to the metric charts that are not currently visible on the screen. The order of the charts can be easily changed by using the Up Arrow and Down Arrow buttons that appear in the upper-right hand corner of each of the charts. Furthermore, you can remove a chart from this display by clicking on the (X) chart close button. 55

59 JBoss Operations Network 1 Users Guide Once you have a set of metrics that you have arranged, you can persist the set and its order as a view. The view controls appear in the upper right-hand corner of the Indicators column. The of the current view appears in the text box. There are several actions you can take by selecting the appropriate option from the view drop-down. Note T he drop down displays only options that are currently applicable, therefore, not all of the following actions appear in the drop-down list at all times. Table 4.2. Indicator Charts View Actions Table View Update view Create New View Delete view Go to View Action Update the current view as displayed. Create a new view with the current display. You must enter a valid view in the text box. Delete the current view. Select one of the following view(s) to display a previously saved view. To execute any of the actions, select it in the drop-down and, if necessary, click on the right arrow button Managing Metric Data You can customize the way JBoss ON gathers and displays metric information. You can choose which metrics to display, what time intervals to use, over what length of time to display data, and more Metric Categories A metric can belong to one of the following categories Availability In JBoss ON, the definition of 'available' is 'ready to service requests'. A platform is available if the JBoss ON Server can reach it. JBoss ON issues a query or a request to other kinds of resources to determine their availability. If a resource that is part of an Application is unavailable, JBoss ON considers the Application to be unavailable. Availability status for a resource managed by JBoss ON is one of three things: Table 4.3. Availability Status Table Status Up Down Unknown Definition T he resource is ready to service requests T he resource was reachable, but did not respond T he resource was unreachable (possibly a network issue) 56

60 Chapter 4. Monitoring When JBoss ON notifies you that an Application is unavailable, you can then quickly drill down into the resources that make up that Application in order to determine which resource (such as a web server, application server, or database) is causing the availability problem Usage You can measure usage for each resource under management. For Web servers and application servers, usage is typically measured as bytes served, bytes received, number of requests, and number of responses over a period of time (minutes, hours, days) that you specify. If monitoring was extended for databases, usage would be measured as the number of requests processed or active connections over a specified period of time (minutes, hours, days) Performance JBoss ON provides end-to-end response time for components of your infrastructure that deal with end user perception. JBoss ON measures requests originating with the end user and then presents the information in a segmented format that separates out the response times of multiple tiers of the transaction, including end user, Web server, and application server. JBoss ON measures response times passively by leveraging how servers and services expose metrics, rather than applying invasive filters which interfere with inbound requests. Since this measurement takes place outside of the process of the application being measured, it results in accurate performance measurement with no impact on application performance - a critical requirement for applications in production. In addition, this information monitors real time transactions rather that synthetic or simulated transactions, providing a high level of visibility into production issues as the actually arise. JBoss ON shows response time data in seconds, and shows low, average, and peak values, as well as number of requests Utilization JBoss ON can measure utilization rates for the platforms and servers that make up an Application. Examples of utilization include number of sessions created and destroyed, number of loaded or reloaded servlets, JVM total, used, and free memory, EJB creates, removes, loads, stores, and so on. You can examine the capacity of an entire platform and measure individual utilization of the servers on those platforms. Using JBoss ON, you can pinpoint underutilized resources by establishing minimum utilization thresholds on a per platform basis. You can also determine where Application bottlenecks occur by examining utilization rates between disk, memory, CPU, and network, and then apply capacity appropriately Viewing Metrics To view metrics for a resource, navigate to the monitor visibility pages of that resource. Click the METRIC DATA tab of that resource. The displayed page lists all metrics currently being collected for the selected resource. In the RESOURCES column of this page, there are options to filter the complete list by metric categories (refer to Section 4.3.1, Metric Categories ), collection type, or keyword. The metrics are displayed with their low, average, peak, and last values, as well as alerts and out-ofbounds counts, for the specified metric display time range. You can chart (refer to Section 4.6, Charting Metric Data ) or set baselines (refer to Section 4.4.1, T o manually establish a metric baseline ) of one or more of the metrics from this list Modifying Metrics When you have added a resource to the Resource Hub and set its Configuration Properties, you can 57

61 JBoss Operations Network 1 Users Guide specify what metric data you want JBoss ON to collect and display for that resource. The metrics that are available for a particular resource will depend on that resource. You can see the list of available metrics in the Plugin Reference. To modify the metrics being collected for a resource, first navigate to the monitor visibility page of that resource. Then click on the CONFIGURE tab button to view the Collect Metrics page. To Add Metrics The metrics that are not currently being collected by JBoss ON are listed with Collection Interval of NONE. To enable metric collection, select the metrics you want to add by checking the checkbox next to the metric. Next, enter the value you want in the Collection Interval for Selected field, selecting a time unit from the drop-down list next to the field. Click the right arrow button to enable the metrics with the specified collection interval you specified. To Edit Metrics T he metrics that are currently being collected by JBoss ON are listed with their specified Collection Intervals. T o alter the collection interval for metrics, check the checkbox next to their s and enter the value you want in the Collection Interval for Selected field, selecting a time unit from the drop-down list next to the field. Click the right arrow button to update the collection interval with your new value Working With Autogroup Metrics For an autogroup, JBoss ON lists all metrics available for collection for that resource type. T he Members Collecting column of each metric denotes if any of the members of the autogroup is collecting that metric. The possible values are NO (none), YES (all), or SOME. If JBoss ON is currently collecting a given metric, it is listed with its specified Collection Interval, if all members that are collecting the metric have the same collection interval; otherwise, the column displays DIFFERENT. T o enable or alter the collection interval for metrics, check the checkbox next to their s and enter the value you want in the Collection Interval for Selected field, selecting a time unit from the drop-down list next to the field. Click the right arrow button to update the collection interval for that metric for all members of the autogroup with your new value To Remove Metrics To remove metrics that are currently being collected by JBoss ON, check the checkbox next to their s and click the DISABLE COLLECT ION button to disable the collection of these metrics Working with Baselines A good baseline is important when evaluating metric data gathered from your resources. By default, JBoss Operations Network automatically calculates the baseline values for all dynamic metrics. A baseline value for a metric becomes more accurate as more data is collected. There are a number of parameters that direct JBoss ON on how to calculate the baseline values, including the frequency of calculation, the dataset to consider, and the minimum number of data points to use for calculation. All of these parameters can be configured in the Automatic Baseline Configuration Properties section in the Administration page (refer to Section 9.3.4, Automatic Baseline Configuration Properties ). T he following details some ways you can use baselines in JBoss ON: T rending Analysis The most common use of baselining is as a performance management tool for trending analysis. Using JBoss ON, you establish and retain the same metric baseline value over a specific period of time, then include the baseline when you chart the current values of the 58

62 Chapter 4. Monitoring metric. You can then identify trends that will help you to estimate future performance or needs. Service Level Management T o manage service level agreements, you measure actual performance against agreed upon minimum service level values. Using JBoss ON, you specify the acceptable high and low range metric values, then include these range values when you chart the current values of the metric. Exception Management A third use of baselining is for monitoring application health (watching for changes in problem indicators), which is a proactive form of fault management. Using JBoss ON, you define an alert based on either the baseline, the high, or the low range value. For example, you can set up an alert that triggers when the metric value is more than 25% of the baseline value To manually establish a metric baseline While JBoss ON continually calculates and updates baseline values, there are two ways of manually establishing the baseline values from the actual metric values collected for the specified metric display range (refer to Section 4.1.4, Metric Display Range ). T his is especially useful if there were not enough metric data for JBoss ON to automatically calculate the baselines. Once established, JBoss ON saves the calculated metric baseline value and uses it each time you chart this metric, regardless of the currently specified display range. JBoss Operations Network uses this value until it is automatically recalculated after the specified baseline frequency or is explicitly recalculated. 1. Select the checkboxes next to the metrics whose baseline values you want to set. Click on the SET BASELINES button. This will set the baseline values of those metrics to the LOW, AVG, and PEAK values as displayed. 2. To set the baseline values individually, click on the metric whose baseline value you want to calculate. T he single metric chart page appears (refer to Section 4.6, Charting Metric Data ). From the Metric Baseline & Expected Range section, click Change Value next to the Baseline, High Range, or Low Range field. You can either accept the calculated value, or simply enter the value that you desire and click on Save Value. The chart is automatically redrawn with the Baseline chart option selected. Note: You must enter: the value as a decimal number including any decimal points. For example, 102,.5, the unit of measure appropriate to the selected metric. For example, MB, GB, %. the low value to be less than the high value, if you are entering both a high and low range value Availability Availability is gathered and recorded for every resource monitored. Green status means that availability is 100% for all items and for the entire duration of a time-block. 59

63 JBoss Operations Network 1 Users Guide Grey question marks mean that we're unable to collect the data at the usual interval. Red means one of two things. We either have a verified outage, where the agent recognizes that the resource was unavailable, or we have an unverified outtage that has lasted at least three collection intervals. This means that if an entire server machine goes down, the metrics on that machine would show up to three grey question marks. Once it got a fourth interval that was not collected, the first grey question mark would change to red. Yellow is shown on the historical Availability timeline. It means that the resource was available for some of that time period but not the entire time period. The timeline would show red if it was not available for the entire time period Charting Metric Data A metric chart is a graphical representation of the metric data that JBoss ON collects for each resource. Different resources support different types of metric data, so different charts are available for different resources. You can use charts to see trends in resource behavior over time. This can be useful for capacity planning, cost analysis, and other operational decision-making. A metric chart includes the option to chart the metric data values along with peak, low, average, and high values. If you are charting a single metric for a resource, you can also set (refer to Section 4.4.1, To manually establish a metric baseline ) and chart baseline values. Depending on the timeframe over which you are charting the metric data, the values may be averaged. A chart can contain up to 60 points. For dynamic metric data (such as CPU time, free memory, load average), each point on the chart (representing a time, such as 1:00pm, 12/22) represents the average value for a period. T he time listed is the end of the period. For a display range of 8 hours of data collected, each charted point represents the average of an 8 minute period, because 8 hours divided by 60 is 8 minutes. If you were to collect data every 60 seconds and then set the display range of your chart to be 60 minutes, you would see the actual raw value for each data point. The values for data that is cumulative (trending up or down), such as bytes served, uptime, minimum response time, number of transactions, and similar metric data are not averaged. A point charted for this kind of data shows the maximum (or minimum) value for the period that the point represents. Chart Types T here are three different kinds of charts. Single Metric Single Resource To chart any metric for a single resource, navigate to the METRIC DATA tab of that resource in the monitor visibility pages. T he displayed page lists all metrics currently being collected for the selected resource. To view a chart of a metric, click its or the chart icon icon next to that metric. Another way to view the chart page for a metric for a single resource is from the INDICATORS tab on the monitor visibility pages. If an indicator chart represents a metric for a single resource, 60

64 Chapter 4. Monitoring clicking on its will also display the full chart page for that metric and resource. Multiple Metrics Single Resource You can chart a maximum of ten metrics at a time for a resource. To chart multiple metrics for a single resource, navigate to the METRIC DATA tab of that resource in the monitor visibility pages. T he displayed page lists all metrics currently being collected for the selected resource. Check the checkbox next to each of the s of the metrics you want to chart and click the CHART SELECT ED MET RICS button. Single Metric Multiple Resources To chart the values of a specific metric for multiple resources at once, those resources must be part of a compatible group/cluster, or a set of resources of the same subtype (autogroup) running on the same parent. You can chart a maximum of ten resources at a time. 1. To chart a metric for multiple members of a compatible group/cluster, navigate to the METRIC DATA tab of that group. 2. To chart a metric for multiple members of an autogroup, first navigate to the parent resource that contains the autogroup. T hen from the parent resource's monitor visibility pages, click on the subtype, which leads you to the monitor visibility pages of the autogroup of that subtype. Click on the METRIC DATA tab of that group. To view a chart of a metric, click its or the chart icon next to that metric. Another way to view the chart page for a metric of an autogroup of resources is from the INDICATORS tab on the monitor visibility pages. If an indicator chart represents a metric for an autogroup of resources, clicking on its will also display the full chart page for that metric and the autogroup. If there are more than 10 resources in the group, you can select different resources to graph from the Participating resources section at the bottom of the page Chart Legend From the chart legend, select one or more items you want to graph: Actual Peak Average Low Low Range High Range Baseline Control Actions To update the chart based on your selections, click the REDRAW button. 61

65 JBoss Operations Network 1 Users Guide Chapter 5. Alerts 5.1. Using Alerts to Manage Resources The alerting functionality in JBoss ON is one of the more powerful and important pieces of the product. Once the power of JBoss ON's monitoring functionality is apparent, the natural next step is being able to set alerts and be notified if/when anything exceeds the threshold of your alert. Alerting is an important tool when it comes to managing resources. For this reason JBoss ON's alerting engine was designed to be powerful as well as flexible enough to allow for even the most demanding alert and notification scenarios. T he general steps for creating an alert are: 1. Determine which resource needs the alert Example: A Tomcat server keeps crashing. You need to be alerted about the resource usage of this server as well as its availability. 2. Determine which metric(s) the alerts will be associated with Our example T omcat server appears to be exhausting its available memory and then crashing. Alerts should be put on the the metrics JVM Free Memory and Availability 3. Determine the threshold(s) where alerts will fire A warning alert would be nice for when the JVM Free Memory gets low, so the threshold for that alert might be set at 10M. That could give you some time to investigate before your server crashes. Then a high severity alert should be set for Availability 100%. That will let you know that your T omcat server has gone down and needs immediate attention. 4. Determine when you want the alert to fire Sometimes you need to know immediately when a critical event happens. Sometimes you want to be alerted if a specific event has been happening frequently over a specified period of time. JBoss ON is flexible enough to allow these and many more alerting options. In our example, perhaps you want to know about it only when the JVM Free Memory is 10M for 30 minutes during a period of one hour. Almost certainly, you'd want to know immediately when the server goes down. 5. Determine the action(s) to be taken You'll want an notification sent when the server gets low on free memory. An to yourself might be appropriate, as well as an alert notification sent to the network operations center, so the on-duty administrator is notified. Perhaps a simple restart of the server is all that is necessary to put things back on track when it crashes. JBoss ON supports adding a control action as part of an alert event. You can have JBoss ON send you notification as well as restart the Tomcat server Defining Alerts After your resources are in the JBoss ON inventory and metrics are being collected, the first step in using the alerting functionality is defining the alerts. First, determine which resource the alert will be defined for and navigate to the Current HealthMonitoring Visibility page for that resource. Refer to Section 4.2.1, Navigating to Current Health. Next, click on the ALERT tab. This will bring you to the List Alerts page. Click CONFIGURE to take you to the List Alert Definitions page. Now click the NEW button to go to the New Alert Definition page. 62

66 Chapter 5. Alerts An alternate navigation to this page is to navigate to any metric chart and click the Define New Alert link in the upper right corner of the page. This has an added benefit of pre-populating the metric dropdown with the particular metric that was being charted. The first thing to do on the New Alert Definition page is to the alert. Give it a that will clearly tell you what the problem is. The of the alert will be displayed in the subject header of an alert notification like this: Subject: [JON]!! resource alert For example, if your resource was a JBoss AS server residing on a Linux platform d server.jboss.com, your T omcat resource is probably "server.jboss.com JBoss AS 3.2.x". If you wanted to set up an alert for if the JBoss AS server goes down, you might want to it simply "down!" and give it a high priority. Then the subject header of your alert notification mail would look like: Subject: [JON]!!! server.jboss.com JBoss AS 3.2.x down! Since you already know which resource the alert is being defined for, the next step is to determine which metric(s) the alert will be defined for and what the alert thresholds will be. Once that is done, select the metric from the Metric dropdown box, select the operator (, or =) from the operator dropdown and enter the threshold in the Absolute Value textfield. If the alert threshold will be based on the baseline or min/max range for the metric, select appropriate radio button and value (Baseline, Min, Max) from the baseline dropdown and enter the percentage in the textbox. If the alert will be based on multiple metrics, click the Add Another Condition link, select the appropriate AND or OR operator from the dropdown and add the new metric and threshold. Repeat for as many metric/threshold combinations as you need for your alert. Next, is the determination for when and how often the alert will fire when the defined conditions are met. Each time conditions are exceeded or met This is the selection that will apply to most alerts. This means "alert me immediately when the conditions for my alert have been met". However, with no further configuration this also means "continue alerting me every x minutes until the conditions are no longer met". x will vary depending on the collection interval for the metric(s) in the alert definition, but generally it will be either 1, 5 or 10. This selection is very effective when defined along with a Recovery Alert which will eliminate the 'alert storm' described above. Refer to Section 5.3, Defining Recovery Alerts. When conditions are exceeded for X within a time period of Y This is a fairly complex action and is most effective when the time periods represented by X and Y are relatively large. For instance, if you want to make Y anything less than 30 minutes, you probably want to use the "Each time conditions are exceeded or met" selection. T o really understand this configuration selection you need to be familiar with the concept of metric collection intervals and know the collection interval(s) of the metric(s) in your alert definition. Refer to Section 4.3.3, Modifying Metrics. Let's say your alert definition was for Free Memory 10M and you wanted to be alerted whenever this condition was met for 20 minutes within a time period of 1 hour. Its not absolutely necessary to know that the collection interval for the Free Memory metric is 5 minutes, but it helps because then you know that there are 12 collections per hour for that metric and if 4 of those 12 collections meet or exceed the threshold, the alert will fire. Once every X times conditions are exceeded within a time period of Y This selection is very similar to the previous one. With the previous option, it was nice to know the collection intervals for the metrics in the alert definition. With this option, it is absolutely necessary. It is necessary because it is possible to create an alert that is impossible to fire. If the metric collection interval is large enough that X collections will never be taken in the Y time period, 63

67 JBoss Operations Network 1 Users Guide the alert can never fire. For example, if we take our Free Memory metric with its 5 minute collection interval and configure an alert definition to fire "Once every 15 times conditions are exceeded within a time period of 1 hour." this alert will never fire. Because we know that only 12 collections will be taken per hour so we will never see 15 metrics per hour, much less 15 exceeding the alert threshold. The last section of the New Alert Definition page is the Enable Action Filters section. This section is entirely optional, but allows you to fine tune the alerting functionality JBoss ON offers. Disable alert until re-enabled manually or by recovery alert T his is the configuration option mentioned above that will prevent 'alert storms'. Selecting this option will disable the alert after it fires so it does not repeatedly fire. The alert will become reenabled if it is re-enabled manually within JBoss ON or if a Recovery Alert re-enables it automatically. Recovery Alerts are covered in greater detail in Section 5.3, Defining Recovery Alerts. Disregard control actions that are defined for related alerts. T his option will only appear on New Alert Definition pages for resources that support control actions. T his option only applies when these conditions are met: T he current alert definition will include an alert action The resource associated with the alert is a member of an application T here are other members of the same application with alerts that fire control actions (ideally the same control action) If the conditions are met, this configuration option will make it so that if multiple alerts are fired within a short period from resources that are members of the same application, only one control action will be executed. This is to prevent a server from being restarted several times in a short period of time for the same alert conditions. An Example would be if you had an alert to restart a Tomcat server if the JVM Free Memory got too low and then another alert to restart the same server if the JVM Active Thread count got too high. If both of these alerts fired at the same time and they were filtering control actions, only 1 restart control action would be executed and not two. Filter notification actions that are defined for related alerts. This option only has an effect on resources that are members of an application. It is very useful for cutting down on the number of alerts from a single application. An example of the functionality would be if you had several resources in an application and each with several alerts. If alerts started firing from several resources, and if each of the alerts was configured for filtering notifications, you would get a single alert with all the alerts from all the resources consolidated in it. Clicking OK will create the alert, enable it and take you to the View Alert Definition page. From this page, you can see the details of the alert definition you just created as well as configure notification. Control actions can also be configured from this page if the resource supports control. Notification can be set by Role, User or arbitrary address. The Roles tab is selected by default. To notify by role simply click the ADD TO LIST... button, select the role(s) to be notified and click OK. To notify by User, click the Notify JON Users tab, click ADD TO LIST... and select the user(s) to be notified. To notify by arbitrary address, click the Notify Other Recipients tab, click ADD TO LIST... and enter the address(es) to be notified. If the resource supports control you can configure a control action to be executed when the alert fires. In the Control Action section of the page, click the EDIT... button, select the control action from the Control Type dropdown and click OK. 64

68 Chapter 5. Alerts Notification is not required for an alert definition. Any combination of notifications and control action is allowed. You can have a single type of notification or all types. You can have a control action associated with an alert or not. The action part of the alert notification is meant to be as flexible as possible Defining Recovery Alerts Recovery Alerts are one of the more powerful and useful features of JBoss ON. The general idea is you set up one alert to let you know when something has gone bad and you set up another alert that lets you know when the problem has been corrected and things are normal again. A properly configured alert/recovery alert combination keeps you notified of problems without deluging you with alert notifications. An example of a good use for an alert/recovery alert combination would be an administrator who wants to know when a particular Linux platform is under a heavy CPU load. The initial alert to indicate a problem could be 1 Minute Load Avg 2.0. The recovery alert could be 1 Minute Load Avg 110% of Baseline. Refer to Section 4.4, Working with Baselines. In the event that the Linux 1 minute system load average goes over 2.0, the administrator would get an alert notification about this event. T hat alert would automatically become disabled in JBoss ON so no more alert notifications will be sent about that alert, no matter how long the system load average remains over 2.0. However, when the system load average drops to 110% of the baseline for the platform, two things will happen. First, the administrator will get another alert notification that says the system load average is back to normal. Second, the first alert will be automatically re-enabled so if the system load average goes back above 2.0 the administrator will get an alert notification. T o configure this type of alert combination, simply define the initial alert (1 Minute Load Avg 2.0) normally, but at the bottom of the New Alert Definition page in the Enable Action Filters section, check the checkbox for the Disable alert until re-enabled manually or by recovery alert. Click OK to create the alert, then set alert notification as appropriate. Refer to Section 5.2, Defining Alerts. Next, create the recovery alert by defining an alert with your recovery criteria (1 Minute Load Avg 110% of Baseline) normally, but in the Recovery Alert dropdown box, select the initial alert you just created. Click OK to create the alert and set alert notification as appropriate. You now have a properly configured Recovery Alert SNMP Traps as Alert Actions In addition to sending an when an Alert is triggered, you can cause an SNMP Trap to be emitted. There are two places this needs to be configured: SNMP Server Configuration Properties. Managed through Administration > Server Configuration > Edit page. At the bottom of this page you specify general information common to all SNMP traps which will be emitted by JBoss ON - for example, the SNMP protocol version. Alert-specific SNMP properties. T hese can be set in two places: Alert instances. When you create an Alert on a given resource, after you have specified the alert definition you can choose the SNMP Trap tab and specify details about the specific trap you want emitted. Alert templates. From the Administration > Monitoring Defaults Configuration > Policy Based Alert Definitions page you can create alert templates which will apply to all resources of the selected type. Similar to the Alert instance above, after you've entered the definition of the alert you can set information about the specific SNMP trap to be fired. 65

69 JBoss Operations Network 1 Users Guide General Restrictions All OIDs must be specified in the numeric dotted format, e.g SNMP Server Configuration Properties For SNMP Protocol Version 1, all ID attributes must be numeric. Alert-specific SNMP properties Support is available for sending SNMP traps via either TCP or UDP. For example: Address of the target SNMP engine: will send a trap via UDP to port 161 on Address of the target SNMP engine: localhost/162 will send a trap via UDP to port 162 on localhost Address of the target SNMP engine: tcp:foo.xyz.com/1621 will send a trap via TCP to port 1621 on foo.xyz.com Getting Started with SNMP Getting Started with SNMP will walk you through configuring SNMP alert actions and verifying traps are being emitted using the free Net-SNMP trap daemon Tutorial on generating SNMP traps from JBoss ON alerts This tutorial will show you how to configure JBoss ON to generate different sorts of SNMP traps Environment This tutorial used JBoss ON Server and Agent installed locally on Windows XP SP2 to generate the SNMP traps. To receive the traps we used Net-SNMP v , installed on the same machine Installing an SNMP client In order to make sure JBoss ON is really generating SNMP traps we need to install something capable of receiving and parsing such traps. For this tutorial we will use 'snmptrapd' which comes as part of the Net-SNMP toolset. We'll be using the Windows distribution. 1. Download the current version of Net-SNMP from T here is an.exe version which contains a Windows Installer. 2. Double click the installer (e.g. net-snmp win32.exe) and follow the instructions. When selecting components you just need the Base Components and the Net-SNMP T rap Service. 3. Once the install is complete open a command prompt and type: snmptrapd -h You should see a lot of commandline options fly past. 4. The final step is to configure snmptrapd to allow connections from our JBoss ON Server. To do this create a file called snmptrapd.conf in <Net-SNMP_HOME>\usr\etc\snmp\ and add the following line: authcommunity log public An example file is attached to this page. 66

70 Chapter 5. Alerts Listening for traps 1. Open a command prompt and type snmptrapd -Lo : The -Lo just tells it to output all traps received to stdout says we should listen on the loopback network interface and 1621 says use UDP port The default for snmptrapd is to try to listen on UDP port 162, but I've had problems with other programs, e.g. Skype and the Remote Procedure Call (RPC) service already listening on that port, so it's easier just to pick a different one Configuring JBoss ON to generate SNMP traps 1. There are two places within JBoss ON which need to be configured: a. Overall server configuration: Within the JBoss ON portal go to Administration > Server Configuration. At the bottom of this page you specify general information common to all SNMP traps which will be emitted by JBoss ON. Changes to these Server level settings do not require the JBoss ON Server to be restarted. Regardless of whether we're using SNMP protocol version 1 or 2c you must set the community property to be 'public' (without the quotes) in order to match the authentication instruction specified in snmptrapd.conf. This tutorial covers generating traps for SNMP protocol version 1 and 2c. Its also possible to generate SNMP v3 traps using JBoss ON. b. Alert specific SNMP properties. When you create an Alert on a given resource, after you have specified the alert definition you can choose the SNMP Trap tab and specify details about the specific trap you want emitted. These properties can also be set via the Alert Templates but that is outside the scope of this tutorial. 2. You will need to create an Alert on a given resource in order to follow along with the tutorial. It doesn't matter what alert is created as long as it continues to fire at a regular interval. For instance, when creating this tutorial, I imported the JON T omcat Server then created an alert for when the Availability metric is greater > 20%, i.e. all the time. The final step is to set the metric collection interval on the metric you are using to be 1 minute. This will cause the alert to be fired every minute and therefore the SNMP trap to be generated at that same frequency. The main part of this tutorial will consist of screenshots of these various pages and also the resulting output from snmptrapd Common SNMP settings on the alert The following screen shot shows the properties we set on the alert. This particular alert is d 'testsnmpt rap2'. T hese settings are unchanged across all the other configurations below. T his screen shows the following: 1. SNMP traps will be emitted over UDP (the default). 2. They will be sent to port 1621 on

71 JBoss Operations Network 1 Users Guide 3. T he trap associated to this particular alert is a Net-SNMP specific one, in particular NET-SNMP- AGENT-MIB::nsNotifyStart (oid: ) SNMP v1 - Generic trap T he Server configuration settings above will generate a generic Link Down trap and cause the following output from snmptrapd: The screen shot above shows four SNMP traps being emitted, with each one taking up three lines: 1. First Line Server Configuration: Agent Adress ( ) Server Configuration: SNMP Protocol Version (1) Server Configuration: Community (public) 2. Second Line Server Configuration: Generic ID (2, i.e. SNMPv2-SMI::zeroDotZ ero Link Down T rap) 3. T hird Line Alert Configuration: OID ( , i.e. NET-SNMP-AGENT-MIB::nsNotifyStart) Alert Configuration: Alert Name (testsnmpt rap2) SNMP v1 - Enterprise Specific trap T he Server configuration settings above will generate a generic Link Down trap and cause the following output from snmptrapd: 68

72 Chapter 5. Alerts The screen shot above shows four SNMP traps being emitted, with each one taking up three lines: 1. First Line Server Configuration: Agent Adress ( ) Server Configuration: SNMP Protocol Version (1) Server Configuration: Community (public) 2. Second Line Server Configuration: Generic ID (6, i.e. Enterprise Specific T rap) Server Configuration: Enterprise OID ( , i.e. NET-SNMP- MIB::netSnmpNotifications) Server Configuration: Specific ID (1, i.e. when combined with Enterprise OID gives NET-SNMP- AGENT-MIB::nsNotifyStart) 3. T hird Line Alert Configuration: OID ( , i.e. NET-SNMP-AGENT-MIB::nsNotifyStart) Alert Configuration: Alert Name (testsnmpt rap2) SNMP v2c - Generic trap With SNMP v2c there is really no distinction between Generic traps and Enterprise specific ones, you just specify the OID of the trap you want, in this case Link Down. T he Server configuration settings above will generate a generic Link Down trap and cause the following output from snmptrapd: The screen shot above shows four SNMP traps being emitted, with each one taking up two lines: 1. First Line N/A 2. Second Line Alert Configuration: OID ( , i.e. NET-SNMP-AGENT-MIB::nsNotifyStart) Alert Configuration: Alert Name (testsnmpt rap2) Server Configuration: T rap OID ( , i.e. IF-MIB::linkDown) 69

73 70 JBoss Operations Network 1 Users Guide

74 Chapter 6. Managed Products Chapter 6. Managed Products 6.1. Operating Systems Linux Management Support Linux Supported Versions Red Hat Enterprise Red Hat SUSE Enterprise 8.1 Fedora Core 2-6 Gentoo Linux Linux Monitoring Specification Linux Metrics Linux Process Metrics Linux User Metrics CPU Service Metrics Network Interface Network IP Service Metrics Filesystem Directory Metrics FileSystem File Metrics FileSystem Mount Metrics Script Service Metrics Linux Administration Specification Linux Metrics Cpu Idle Cpu Nice Cpu Usage Cpu Wait Free Memory Free Memory with buffers and cache Load Average 1 Minute Load Average 15 Minutes Load Average 5 Minutes Number of CPUs Shared Memory Swap Free Swap Total Swap Used 71

75 JBoss Operations Network 1 Users Guide System Cpu T otal Memory Used Memory Used Memory without buffers or cache User Cpu Linux Process Metrics Availability Virtual Memory Size Memory Size Cpu System T ime Cpu System T ime per Minute Cpu User Time Cpu User Time per Minute Cpu Total Time Cpu Total Time per Minute Cpu Usage Start Time Open File Descriptors Linux User Metrics Availability Virtual Memory Size Memory Size Cpu System T ime Cpu System T ime per Minute Cpu User Time Cpu User Time per Minute Cpu Total Time Cpu Total Time per Minute Cpu Usage Start Time Open File Descriptors Network Interface Availability Bytes Received Bytes Received per Minute Packets Received Packets Received per Minute Bytes T ransmitted Bytes T ransmitted per Minute 72

76 Chapter 6. Managed Products Packets T ransmitted Packets T ransmitted per Minute T ransmit Errors T ransmit Errors per Minute Receive Errors Receive Errors per Minute T ransmit Packets Dropped T ransmit Packets Dropped per Minute Receive Packets Dropped Receive Packets Dropped per Minute T ransmit Collisions T ransmit Collisions per Minute Network IP Service Metrics Availability Request T ime Filesystem Directory Metrics Last Modified T ime Last Change Time Last Access Time Permissions Owner User Id Owner Group Id Availability Regular Files Subdirectories Symbolic Links Character Devices Block Devices Sockets Total FileSystem Mount Availability Use Percent Total Bytes Used Capacity Total Bytes Free Total Bytes Avail Disk Reads Disk Reads per Minute 73

77 JBoss Operations Network 1 Users Guide Disk Writes Disk Writes per Minute Free Files Total Files File Service Overview A File service is associated with a platform. It allows a file on the Platform machine to be monitored and, via the Control actions, actually executed Configuration Properties Property Required Description path yes the full path to a file on the Agent machine prefix no a prefix to prepend to the command line when executing the script (e.g. sudo) timeout yes a timeout for the control action (in seconds) Metrics Last Modified T ime Last Change Time Last Access Time Permissions Owner User Id Owner Group Id Availability Size Control Actions Run: T ries to execute the actual file associated with this service HP-UX Management Support HP-UX Supported Versions HP-UX 11.x PA Warning HP-UX is not supported on the IA64 architecture. Future support is being evaluated HP-UX Monitoring Specification 74

78 Chapter 6. Managed Products HP UX Metrics HP UX Process Metrics HP UX User Metrics CPU Service Metrics Network Interface Network IP Service Metrics Filesystem Directory Metrics FileSystem File Metrics FileSystem Mount Metrics Script Service Metrics HP-UX Administration Specification TBD HP/UX Metrics Cpu Idle Cpu Nice Cpu Usage Cpu Wait Free Memory Free Memory with buffers or cache Load Average 1 Minute Load Average 15 Minutes Load Average 5 Minutes Number of CPUs Shared Memory Swap Free Swap Total Swap Used System Cpu T otal Memory Used Memory Used Memory without buffers or cache User Cpu HP/UX Process Metrics Availability Virtual Memory Size Memory Size Cpu System T ime Cpu System T ime per Minute Cpu User Time 75

79 JBoss Operations Network 1 Users Guide Cpu User Time per Minute Cpu Total Time Cpu Total Time per Minute Cpu Usage Start Time Open File Descriptors HP UX User Metrics Availability Virtual Memory Size Memory Size Cpu System T ime Cpu System T ime per Minute Cpu User Time Cpu User Time per Minute Cpu Total Time Cpu Total Time per Minute Cpu Usage Start Time Open File Descriptors Network Interface Availability Bytes Received Bytes Received per Minute Packets Received Packets Received per Minute Bytes T ransmitted Bytes T ransmitted per Minute Packets T ransmitted Packets T ransmitted per Minute T ransmit Errors T ransmit Errors per Minute Receive Errors Receive Errors per Minute T ransmit Packets Dropped T ransmit Packets Dropped per Minute Receive Packets Dropped Receive Packets Dropped per Minute T ransmit Collisions T ransmit Collisions per Minute Network IP Service Metrics 76

80 Chapter 6. Managed Products Availability Request T ime Filesystem Directory Metrics Last Modified T ime Last Change Time Last Access Time Permissions Owner User Id Owner Group Id Availability Regular Files Subdirectories Symbolic Links Character Devices Block Devices Sockets Total FileSystem Mount Availability Use Percent Total Bytes Used Capacity Total Bytes Free Total Bytes Avail Disk Reads Disk Reads per Minute Disk Writes Disk Writes per Minute Free Files Total Files File Service Overview A File service is associated with a platform. It allows a file on the Platform machine to be monitored and, via the Control actions, actually executed Configuration Properties Property Required Description path yes the full path to a file on the Agent machine 77

81 JBoss Operations Network 1 Users Guide prefix no a prefix to prepend to the command line when executing the script (e.g. sudo) timeout yes a timeout for the control action (in seconds) Metrics Last Modified T ime Last Change Time Last Access Time Permissions Owner User Id Owner Group Id Availability Size Control Actions Run: T ries to execute the actual file associated with this service Sun Solaris Management Support Solaris Supported Versions SPARC Solaris , 32-bit x86 Solaris (agent only - must use no-jre installer) Warning Solaris 64-bit is only supported when 32-bit compatibility mode is enabled. Future support for pure 64-bit mode is being evaluated Solaris Monitoring Specification Solaris Metrics Solaris Process Metrics Solaris User Metrics CPU Service Metrics Network Interface Network IP Service Metrics Filesystem Directory Metrics FileSystem File Metrics FileSystem Mount Metrics Script Service Metrics Solaris Administration Specification 78

82 Chapter 6. Managed Products TBD Solaris Metrics Cpu Idle Cpu Nice Cpu Usage Cpu Wait Free Memory Free Memory with buffers or cache Load Average 1 Minute Load Average 15 Minutes Load Average 5 Minutes Number of CPUs Shared Memory Swap Free Swap Total Swap Used System Cpu T otal Memory Used Memory Used Memory without buffers or cache User Cpu Solaris Process Metrics Availability Virtual Memory Size Memory Size Cpu System T ime Cpu System T ime per Minute Cpu User Time Cpu User Time per Minute Cpu Total Time Cpu Total Time per Minute Cpu Usage Start Time Open File Descriptors Solaris User Metrics Availability Virtual Memory Size Memory Size Cpu System T ime 79

83 JBoss Operations Network 1 Users Guide Cpu System T ime per Minute Cpu User Time Cpu User Time per Minute Cpu Total Time Cpu Total Time per Minute Cpu Usage Start Time Open File Descriptors Network Interface Availability Bytes Received Bytes Received per Minute Packets Received Packets Received per Minute Bytes T ransmitted Bytes T ransmitted per Minute Packets T ransmitted Packets T ransmitted per Minute T ransmit Errors T ransmit Errors per Minute Receive Errors Receive Errors per Minute T ransmit Packets Dropped T ransmit Packets Dropped per Minute Receive Packets Dropped Receive Packets Dropped per Minute T ransmit Collisions T ransmit Collisions per Minute Network IP Service Metrics Availability Request T ime Filesystem Directory Metrics Last Modified T ime Last Change Time Last Access Time Permissions Owner User Id Owner Group Id Availability 80

84 Chapter 6. Managed Products Regular Files Subdirectories Symbolic Links Character Devices Block Devices Sockets Total FileSystem Mount Availability Use Percent Total Bytes Used Capacity Total Bytes Free Total Bytes Avail Disk Reads Disk Reads per Minute Disk Writes Disk Writes per Minute Free Files Total Files File Service Overview A File service is associated with a platform. It allows a file on the Platform machine to be monitored and, via the Control actions, actually executed Configuration Properties Property Required Description path yes the full path to a file on the Agent machine prefix no a prefix to prepend to the command line when executing the script (e.g. sudo) timeout yes a timeout for the control action (in seconds) Metrics Last Modified T ime Last Change Time Last Access Time Permissions Owner User Id 81

85 JBoss Operations Network 1 Users Guide Owner Group Id Availability Size Control Actions Run: T ries to execute the actual file associated with this service IBM AIX Management Support IBM AIX Supported Versions IBM AIX , 32-bit Warning AIX is not supported on the PPC 64-bit architecture. Future support is being evaluated IBM AIX Monitoring Specification IBM AIX Metrics IBM AIX Process Metrics IBM AIX User Metrics CPU Service Metrics Network Interface Network IP Service Metrics Filesystem Directory Metrics FileSystem File Metrics FileSystem Mount Metrics Script Service Metrics IBM AIX Administration Specification TBD IMB AIX Metrics Cpu Idle Cpu Nice Cpu Usage Cpu Wait Free Memory Load Average 1 Minute Load Average 15 Minutes Load Average 5 Minutes Number of CPUs Shared Memory 82

86 Chapter 6. Managed Products Swap Free Swap Total Swap Used System Cpu T otal Memory Used Memory User Cpu IBM AIX Process Metrics Availability Virtual Memory Size Memory Size Cpu System T ime Cpu System T ime per Minute Cpu User Time Cpu User Time per Minute Cpu Total Time Cpu Total Time per Minute Cpu Usage Start Time Open File Descriptors IBM AIX User Metrics Availability Virtual Memory Size Memory Size Cpu System T ime Cpu System T ime per Minute Cpu User Time Cpu User Time per Minute Cpu Total Time Cpu Total Time per Minute Cpu Usage Start Time Open File Descriptors Network Interface Availability Bytes Received Bytes Received per Minute Packets Received Packets Received per Minute 83

87 JBoss Operations Network 1 Users Guide Bytes T ransmitted Bytes T ransmitted per Minute Packets T ransmitted Packets T ransmitted per Minute T ransmit Errors T ransmit Errors per Minute Receive Errors Receive Errors per Minute T ransmit Packets Dropped T ransmit Packets Dropped per Minute Receive Packets Dropped Receive Packets Dropped per Minute T ransmit Collisions T ransmit Collisions per Minute Network IP Service Metrics Availability Request T ime Filesystem Directory Metrics Last Modified T ime Last Change Time Last Access Time Permissions Owner User Id Owner Group Id Availability Regular Files Subdirectories Symbolic Links Character Devices Block Devices Sockets Total FileSystem Mount Availability Use Percent Total Bytes Used Capacity Total Bytes Free Total Bytes Avail 84

88 Chapter 6. Managed Products Disk Reads Disk Reads per Minute Disk Writes Disk Writes per Minute Free Files Total Files File Service Overview A File service is associated with a platform. It allows a file on the Platform machine to be monitored and, via the Control actions, actually executed Configuration Properties Property Required Description path yes the full path to a file on the Agent machine prefix no a prefix to prepend to the command line when executing the script (e.g. sudo) timeout yes a timeout for the control action (in seconds) Metrics Last Modified T ime Last Change Time Last Access Time Permissions Owner User Id Owner Group Id Availability Size Control Actions Run: T ries to execute the actual file associated with this service Microsoft Windows Management Support Windows Supported Versions Windows NT 4.0 Windows 2000 Windows XP, 32-bit Windows 2003, 32-bit 85

89 JBoss Operations Network 1 Users Guide Warning Windows Vista is not supported, nor is Windows supported on the x64 architecture. Future support is being evaluated Windows Monitoring Specification Windows Metrics Windows Process Metrics CPU Service Metrics Network Interface Network IP Service Metrics Filesystem Directory Metrics FileSystem File Metrics FileSystem Mount Metrics Script Service Metrics Windows Service Metrics Windows Administration Specification Windows Service Control Windows Metrics Availability Number of Processes Memory Size Cpu System T ime Cpu System T ime per Minute Cpu User Time Cpu User Time per Minute Cpu Total Time Cpu Total Time per Minute] Cpu Usage Windows Process Metrics Availability Virtual Memory Size Memory Size Cpu System T ime Cpu System T ime per Minute Cpu User Time Cpu User Time per Minute Cpu Total Time 86

90 Chapter 6. Managed Products Cpu Total Time per Minute Cpu Usage Start Time Open File Descriptors Network Interface Availability Bytes Received Bytes Received per Minute Packets Received Packets Received per Minute Bytes T ransmitted Bytes T ransmitted per Minute Packets T ransmitted Packets T ransmitted per Minute T ransmit Errors T ransmit Errors per Minute Receive Errors Receive Errors per Minute T ransmit Packets Dropped T ransmit Packets Dropped per Minute Receive Packets Dropped Receive Packets Dropped per Minute T ransmit Collisions T ransmit Collisions per Minute Network IP Service Metrics Availability Request T ime Filesystem Directory Metrics Last Modified T ime Last Change Time Last Access Time Permissions Owner User Id Owner Group Id Availability Regular Files Subdirectories Symbolic Links Character Devices 87

91 JBoss Operations Network 1 Users Guide Block Devices Sockets Total FileSystem Mount Availability Use Percent Total Bytes Used Capacity Total Bytes Free Total Bytes Avail Disk Reads Disk Reads per Minute Disk Writes Disk Writes per Minute Free Files Total Files File Service Overview A File service is associated with a platform. It allows a file on the Platform machine to be monitored and, via the Control actions, actually executed Configuration Properties Property Required Description path yes the full path to a file on the Agent machine prefix no a prefix to prepend to the command line when executing the script (e.g. sudo) timeout yes a timeout for the control action (in seconds) Metrics Last Modified T ime Last Change Time Last Access Time Permissions Owner User Id Owner Group Id Availability Size 88

92 Chapter 6. Managed Products Control Actions Run: T ries to execute the actual file associated with this service Windows Service Metrics Collects information on windows services. T o enable collection you must specify the of the windows service. Availability Defining a Windows Service. JBoss ON has the ability to control a Windows Service. In order to control the service, you first must define it as a resource of your platform. 1. Browse to your Windows platform, and click the "Inventory" tab. 2. Scroll down to the "Windows Service" section", and click "New" 3. Give the service a Name and optionally a description. Click "OK" You've now created a resource for a Windows Service, but the resource has not yet been configured. 4. Click the "Configuration Properties" link on the next page, or scroll to the bottom and click the last [Edit] button. 5. Enter the exact of the Windows Service you'd like to control in the "service_" field and click OK. You can now control this windows service through the "Control" tab of this resource Windows Service Control Start Service Stop Service Restart Service 6.2. JEMS Products JBoss AS Management Support JBoss AS Supported Versions JBoss AS 3.2 JBoss AS JBoss Monitoring Specification General Server Metrics JMS Destinations Entity EJB Metrics Stateless Session EJB Metrics Stateful Session EJB Metrics Message Driven EJB Metrics 89

93 JBoss Operations Network 1 Users Guide JCA Connection Pool Metrics JCA DataSources Hibernate JGroups JBoss Administration Specification Control Actions Configuration Actions Autodetection T he JBoss plugin is capable of detecting the configurations for running servers when installed via the JBoss Installer or when utilizing -c for a specific configuration or -b for a specific binding address. Autodetection will not be able to automatically configure for servers that either use secured JMX invokers or use the Binding Service to customize ports. In those cases, the server will be detected and can be imported, but must be manually configured after that General Server Metrics Active T hread Count Active T hread Group Count JVM Free Memory JVM T otal Memory JVM Max Memory JMS Message Cache Size JMS Message Cache Hits JMS Message Cache Hits per Minute JMS Message Cache Misses JMS Message Cache Misses per Minute Availability T ransactions Active T ransactions Committed T ransactions Committed per Minute T ransactions Rolledback T ransactions Rolledback per Minute JMS Message Cache Current Memory Usage JMS Message Cache High Memory Mark JMS Message Cache Max Memory Mark JMS Destinations JMS Destination Metrics 90 Messages in Queue Messages in Topic Availability Receivers Count

94 Chapter 6. Managed Products Subscriptions Count Administering JMS Destinations Four JMS Destination related actions are supported: Available from the Control tab for 'JBoss 4.0' servers is: Create JMS Destination Available from the Control tab for 'JBoss 4.0 JMS Queue' and 'JBoss 4.0 JMS Topic' services are: View JMS Destination Update JMS Destination Delete JMS Destination Restrictions The JMS Destination Create actions are only available on 'JBoss 4.0' servers and not on 'JBoss 3.2'. Similarly, the View/Update/Delete actions are only available on 'JBoss 4.0 JMS Queue' and 'JBoss 4.0 JMS Topic' services and not on their JBoss 3.2 counterparts. When creating a JMS Queue or a JMS Topic via the Create JMS Destination action, a JNDI must be specified for the JMS destination. T his is a departure from the convention followed natively by the JBoss server while deploying JMS destinations. If the definition of a JMS Queue d MyQueue does not contain a JNDI, the JBoss server assigns the JNDI "queue/myqueue" by default during deployment. In order to mimic this behavior while creating JMS destinations via JBoss ON, the "queue/" or "topic/" prefix must be included in the JNDI of the JMS destination. The JNDI specified while creating a JMS destination will be used as the unique identifier of the JMS destination. The definition of the JMS destination will be stored in a file d with the identifier in the format <JNDI Name>-service.xml under the target JBoss server's deploy directory. T his identifier will also be used to define the '' attribute of the MBean representing the created JMS destination. As a result, the JNDI must not contain characters that are forbidden in a file or in the unquoted value component of an MBean ObjectName (T he latter excludes characters such as comma, equals, colon, quote, asterisk, or question mark. Please see for details). While creating JMS destinations manually by deploying a file containing the definition of the JMS destination under a JBoss server, the value of the parameter 'Destination Manager' must be a valid MBean ObjectName specified in the format "jboss.mq.destination:service=< Queue/T opic>,=<destination->" with the components of the ObjectName appearing strictly in that order. T he Read/Update/Delete actions will not work on JMS destinations whose definitions do not conform to this. In order for JMS destinations created by the actions to be deployed and usable in the JBoss Server the DeploymentScanner on the server itself must be configured and running. In order for JMS destinations created by the actions to be successfully added to the inventory, the managed server configuration property 'Auto-Discovery for EJB3s, Entity EJBs, and other services' must be turned on for the JBoss Server itself. During the process of updating a JMS destination definition, the JMS destination itself will be unavailable for a brief period of time. In order to avoid any disruption to applications which depend on the JMS destination, appropriate steps should be taken to make sure requests are not made to the JMS destination while it is unavailable. For example: stop the application, update the JMS destination, restart the application. In order to prevent possible file corruption you should not schedule two Edit or Delete actions which 91

95 JBoss Operations Network 1 Users Guide will access the same physical -service.xml file to occur at the same time Features T he following sections describe in detail the functionality of the JMS Destination related actions. All actions can be either executed immediately or scheduled for execution at some specified time in the future. Creating a JMS Destination Choosing the Create JMS Destination action will render a page showing a dropdown of JMS Destination templates to choose from. Selecting one of JMS Queue or JMS Topic will present a form showing the attributes relevant to creating that type of JMS Destination. Once you've chosen the template you would like to use to create your JMS destination the form is quite straightforward with a description of each field appearing if you mouse-over the field. For fields that are specified by drop-down, the 'Default' entry means don't specify anything for this field just take the default specified by the JBoss Server. An indication of what is the default is provided in the description for each field. For example, when creating a JMS Queue or T opic, leaving the In Memory drop-down as Default, means that this JMS destination definition will not specify anything for this setting, so the default will be used, i.e. messges received at this JMS destination will be persisted to disk. If you over-ride the default values supplied for the 'Destination Manager' and 'Security Manager' attributes, be sure to specify values representing a valid MBean ObjectName. Hitting the Ok button will attempt to create a new -service.xml file containing the definition of the JMS Destination in the /deploy directory of the JBoss server. The JNDI attribute will be used as the unique identfier of the new JMS destination (it will be used as the basis for the ObjectName of the MBean representing the JMS destination). The JNDI attribute will also be used to construct the the -service.xml file. In the file all non letter or digit characters from the JNDI will be converted to underscores. Note: Currently when creating a JMS destination, e. g. with JNDI MyDest_test, which would map to the of an existing -service.xml file, e.g. MyDest_test-service.xml, then when you save the new JMS destination, the existing - service.xml file will get overwritten with the new one. A future release will fix this by not allowing existing -service.xml files to be overwritten. The JNDI attribute will also be used as the basis for the ObjectName of the MBean representing the created JMS Destination. The final step of the Create action is for it to add the JMS destination to the JBoss ON inventory. Shortly after the action has completed you should be able to go to the Browse Resource > Services page, choose 'JBoss 4.0 JMS Queue' or 'JBoss 4.0 JMS Topic' from the drop-down and see the new JMS destination you just created in the list. Note: Depending upon the metric collection interval you have configured for JMS queues and topics, its availability may show as unknown icon (a grey circle with a? in it). Once metrics have been collected for the JMS destination the availability metric should show green. Viewing a JMS Destination Choosing this action will render a read-only view of the attributes of the JMS Queue or Topic. Updating a JMS Destination Using this action will present the user with a form where changes can be made to the definition of the JMS destination. Hitting Ok will update the underlying -service.xml with the new information. 92

96 Chapter 6. Managed Products Deleting a JMS Destination Choosing this action will remove the definition of the JMS destination from the underlying - service.xml file. All other content of the -service.xml file will be preserved, and no attempt will be made to delete the file under any circumstances. T his means that 'orphaned' -service.xml files devoid of any or relevant content may be left behind in the aftermath of deleting a JMS destination. Finally, deleting a JMS Queue or JMS T opic will remove the corresponding 'JBoss 4.0 JMS Queue' or 'JBoss 4.0 JMS T opic' service respectively from the JBoss ON inventory Updates to configuration files If you select to Edit a JMS Destination, don't change any of the settings and hit the Ok button to save it, then the following changes will be made to the underlying -service.xml file: Whitespace changes. T hese will generally be in the form of empty lines being removed from between elements and line breaks removed from between attributes. Empty element removal. Most empty elements, e.g. <a/> or <a></a>, in the original -service.xml that are part of the definition of the JMS destination being updated will be removed in the saved version. This is safe for most elements since specifying an emtpy element is equivalent to not specifying the element at all Messages in Queue This metric is an alias for what JBoss more commonly refers to as QueueDepth, which is the number of messages that are currently on the queue. For more information, see section of the JBoss guide here Messages in Topic This metric is an alias for what JBoss more commonly refers to as AllMessageCount, which is the message count across all queue types associated with the topic. For more information, see section of the JBoss guide here Availability Availability is gathered and recorded for every resource monitored. Green status means that availability is 100% for all items and for the entire duration of a time-block. Grey question marks mean that we're unable to collect the data at the usual interval. Red means one of two things. We either have a verified outage, where the agent recognizes that the resource was unavailable, or we have an unverified outtage that has lasted at least three collection intervals. This means that if an entire server machine goes down, the metrics on that machine would show up to three grey question marks. Once it got a fourth interval that was not collected, the first grey question mark would change to red. 93

97 JBoss Operations Network 1 Users Guide Yellow is shown on the historical Availability timeline. It means that the resource was available for some of that time period but not the entire time period. The timeline would show red if it was not available for the entire time period Receivers Count This metric is an alias for what JBoss refers to as ReceiversCount, which is the number of receivers currently associated with the queue. For more information, see section of the JBoss guide here Subscriptions Count This metric is an alias for what JBoss more commonly refers to as AllSubscriptionsCount, which is the count of durable and non-durable subscriptions. For more information, see section of the JBoss guide here Entity EJB Metrics Create Calls Create Calls per Minute Remove Calls Remove Calls per Minute Ready Beans Pooled Beans CacheSize PassivatedCount PassivatedCount per Minute PoolSize MaxPoolSize Availability Stateless Session EJB Metrics] Create Calls Create Calls per Minute Remove Calls Remove Calls per Minute Method\-Ready Beans PoolSize MaxPoolSize Availability Stateful Session EJB Metrics Create Calls Remove Calls Method\-Ready Beans 94

98 Chapter 6. Managed Products Passive Beans Availability Message Driven EJB Metrics Create Calls Create Calls per Minute Remove Calls Remove Calls per Minute Messages Received Messages Received per Minute Availability JCA Connection Pool Metrics Active Connections Available Connections Connections Created Connections Created per Minute Connections Destroyed Connections Destroyed per Minute Max Connections Min Connections T otal Connections Availability JCA DataSources JCA DataSource Metrics Availability Administering JCA DataSources T here are four DataSource related actions available Available from the Control tab for 'JBoss 4.0' servers are: Create DataSource Available from the Control tab for 'JBoss 4.0 JCA Data Source' services are View DataSource Edit Datasource Delete DataSource Restrictions T he DataSource View/Edit/Delete actions will only work for DataSources which are defined using the shorter -ds.xml format, and where there is only one DataSource defined in that file. A future release will support the scenario where multiple DataSource definitions are deployed to a single -ds.xml file. T he DataSource Create actions are only available on 'JBoss 4.0' servers and not 'JBoss 3.2'. 95

99 JBoss Operations Network 1 Users Guide Similarly the View/Edit/Delete actions are only available on 'JBoss 4.0 JCA Data Source' services. T here is no corresponding DataSource service for JBoss 3.2 servers. Jar files containing Database Driver files cannot be deployed using the DataSource actions. Instead the Deployment Actions should be used. In order for DataSources created by the actions to be deployed and usable in the JBoss Server the DeploymentScanner on the server itself must be configured and running. In order for DataSources created by the actions to be successfully added to the inventory, the JBN EM setting 'Auto-Discovery for EJB3s, Entity EJBs, and other services' must be turned on for the JBoss Server itself. During the process of updating a DataSource definition, the DataSource itself will be unavailable for a brief period of time. In order to avoid any disruption to applications which depend on the DataSource, appropriate steps should be taken to make sure requests are not made to the DataSource while it is unavailable. For example: stop the application, update the DataSource, restart the application. In order to prevent possible file corruption you should not schedule two Edit or Delete actions which will access the same physical -ds.xml file to occur at the same time. It is not possible to use the Edit action to change the JNDI of an existing DataSource Features T he following sections describe in detail the functionality of the DataSource related action. All actions can be either executed immediately or scheduled for execution at some specified time in the future. Creating a DataSource Choosing the Create DataSource action will render a page showing a dropdown of DataSource templates to choose from. Choosing one of the Generic No T ransaction, Local T ransaction or XA templates will present a form showing only the attributes relevant to creating that type of DataSource. Choosing one of the MySQL or Oracle templates will show a form which has some of the basic attributes defaulted to values applicable to that type of Database. Once you've chosen the template you would like to use to create your DataSource the form is quite straightforward with a description of each field appearing if you mouse-over the field. For fields that are specified by drop-down, the 'Default' entry means don't specify anything for this field just take the default specified by the JBoss Server. An indication of what is the default is provided in the description for each field. For example, when creating an XA datasource, leaving the T rack Connection By T x drop-down as Default, means that this DataSource definition will not specify anything for this setting, so the default will be used, i.e. don't track connections by transaction. Hitting the Ok button will attempt to create a new -ds.xml file containing the DataSource definition in the /deploy directory of the JBoss server. The JNDI attribute will be used as the basis for the of the -ds.xml file. In the file all non letter or digit characters from the JNDI will be converted to underscores. Note: Currently when creating a DataSource if you specify a JNDI, e.g. myds-test, which would map to the of an existing -ds.xml file, e.g. myds_test-ds.xml, then when you save the new DataSource the existing -ds.xml file will get overwritten with the new one. A future release will fix this by not allowing existing -ds.xml files to be overwritten. The final step of the Create action is for it to add the DataSource to the JBN EM inventory. Shortly after the action has completed you should be able to go to the Browse Resource > Services page, choose 'JBoss 4.0 JCA Data Source' from the drop-down and see the new DataSource you just created in the list. Note: Depending upon the metric collection interval you have configured for DataSources its availability may show as unknown icon (a grey circle with a 96

100 Chapter 6. Managed Products? in it). Once metrics have been collected for the DataSource the availability metric should show green. Viewing a DataSource Choosing this action will render a read-only view of the attributes of the DataSource. Editing a DataSource Using this action will present the user with a form where changes can be made to the DataSource definition. Hitting Ok will update the underlying -ds.xml with the new information. Deleting a DataSource Choosing this action will remove the underlying -ds.xml file associated with the DataSource. Care should be taken when using this action as no 'Do you really want to do this' prompt will be given. Note: if other Mbean definitions are also included in the -ds.xml then those will be removed also. Finally deleting a DataSource will remove the corresponding 'JBoss 4.0 JCA Data Source' and 'JBoss 4.0 JCA Connection Pool' services from the JBoss ON inventory Updates to configuration files If you select to Edit a DataSource, don't change any of the settings and hit the Ok button to save it, then the following changes will be made to the underlying -ds.xml file: Whitespace changes. T hese will generally be in the form of empty lines being removed from between elements and line breaks removed from between attributes. Empty element removal. Most empty elements, e.g. <a/> or <a></a>, in the original -ds.xml will be removed in the saved version. This is safe for most elements since specifying an emtpy element is equivalent to not specifying the element at all. This will not be done for "boolean" type elements, i.e. use-java-context, track-connection-by-tx and no-tx-separate-pools, for which an empty element signifies the setting is true. Instead these will be written out as having the value true, e.g. <use-javacontext>true</use-java-context>. Niether of these changes are meant to change the underlying definition of the DataSource or the settings JBoss uses for the DataSource once it has read in the - ds.xml. Invalid element removal. Certain elements in the -ds.xml are supposed to be restricted having particular values, e.g. <transaction-isolation>t RANSACT ION_REPEAT ABLE_READ</transactionisolation> is fine, but <transaction-isolation>repeat ABLE_READ</transaction-isolation> isn't. For such restricted elements if the original -ds.xml has an illegal value, then it will be shown as having the 'Default' setting in the UI and when it is saved to the updated -ds.xml file that element will be removed. T he only elements this applies to are transaction-isolation, issamerm-override-value and track-statements. T he other elements which are restricted in what values they can take are use-javacontext, track-connection-by-tx and no-tx-separate-pools. Ignoring case, the only valid values for these are "true" and "false", and of course empty. However if an invalid value is specified in the original -ds.xml for one of these elements, then the DataSource will not deploy into JBossAS and so will not even appear in the JBN EM inventory. In fact because of after JBoss 4.0.3RC2 illegal values for transactionisolation will also prevent the DataSource from deploying Hibernate Support JBoss AS Supported Versions 97

101 JBoss Operations Network 1 Users Guide Latest version of Hibernate with JMX support Hibernate Monitoring Specification Availability Entity Insert Count Entity Insert Count per Minute Query Execution Max T ime Entity Update Count Entity Update Count per Minute Collection Update Count Collection Update Count per Minute Entity Load Count Entity Load Count per Minute Entity Fetch Count Entity Fetch Count per Minute Entity Delete Count Entity Delete Count per Minute Collection Recreate Count Collection Recreate Count per Minute Query Execution Count Query Execution Count per Minute Flush Count Flush Count per Minute Collection Load Count Collection Load Count per Minute Successful T ransaction Count Successful T ransaction Count per Minute Query Cache Hit Count Query Cache Hit Count per Minute Collection Remove Count Collection Remove Count per Minute Connect Count Connect Count per Minute Start Time Second Level Cache Put Count Second Level Cache Put Count per Minute Query Cache Put Count Query Cache Put Count per Minute Session Open Count Session Open Count per Minute T ransaction Count T ransaction Count per Minute 98

102 Chapter 6. Managed Products Collection Fetch Count Collection Fetch Count per Minute Session Close Count Session Close Count per Minute Query Cache Miss Count Query Cache Miss Count per Minute Second Level Cache Miss Count Second Level Cache Miss Count per Minute JGroups Support JGroups Supported Versions Latest version of JGroups with JMX support JGroups Monitoring Specification Availability Number of Messages Number of Messages per Minute Sent Messages Sent Messages per Minute Sent Bytes Sent Bytes per Minute Received Bytes Received Bytes per Minute Control Actions T he following are control actions for JBossAS servers: start stop restart Dynamic JBoss Launcher actions Configuration Actions T he following are configuration actions for JBossAS: Dynamic JBoss Launcher Deployment Actions Dynamic JBoss Launcher In the previous 1.0 release, there were three JBossAS control actions (start, stop, restart). T hey utilize some simple mechanisms to start and stop the JBossAS instances. T hese control actions still exist in the 1.2 version, however, the Dynamic JBoss Launcher (DJL) feature provides a new way to start and stop JBossAS instances. 99

103 JBoss Operations Network 1 Users Guide Note For a quick demo on the DJL, see the DJL Overview Demo Java Service Wrapper T he DJL utilizes the Java Service Wrapper open source project to launch and stop JBossAS instances. There are four new control actions that you can execute on any JBossAS instance: Install JBoss Launcher Service, Start JBoss Launcher Service, Stop JBoss Launcher Service, Uninstall JBoss Launcher Service Install JBoss Launcher Service Before you can start a JBossAS instance via the DJL, you first must install it. Warning Do not install or otherwise use the DJL on the JON Server resource. It is intended only for use with your own JBossAS server instances. Installing the DJL does two things: 1. configures the startup parameters for the JBossAS instance 2. (Windows only) installs a Windows Service so the JBossAS instance can be started automatically at boot time When you execute the install action, you will be asked for several parameters: javahome - the full path to the Java installation you want to run the JBossAS instance with (optional - default is the agent's VM) javaexe - the path (relative to javahome directory) of the Java executable that is to run (optional - default is the agent's VM) javat ools.jar - the path to the Java tools.jar (relative to the javahome directory) (optional - default is one (if any) in the agent VM) account - account of the Windows user which the service will run as - e.g. MYDOMAIN\user (optional, Windows only - default is the Local System Account) password - the password of the account (optional, Windows only) wrapperconfig - additional Java Service Wrapper configuration settings to apply when the instance is started. T his is a comma separated list of "wrapper.[rest of setting ]=[setting value]" settings. See the Java Service Wrapper configuration documentation at for more information on the different configuration settings allowed. By default, nothing is set. colocatewithjbossinstance - (only in JBoss ON 1.2.SP1 or later) enabling this checkbox will install the DJL binary and configuration files such that they are co-located with the JBossAS server instance itself. It is highly recommended that you select this feature since it will allow the DJL to survive agent uninstalls, reinstalls and upgrades. Deselecting this option will install the DJL directly in the agent - use this if you do not want the agent writing files directly to your JBossAS server install directories. 100

104 Chapter 6. Managed Products Agent /data Directory When not installed co-located with a JBossAS instance, the DJL stores its configuration files in the agent's /data directory. You should not destroy or otherwise move this directory. If you delete the /data directory, your DJL configurations will be lost. In addition, you will not be able to uninstall the Windows Services or start your JBoss Windows Service. Deleting this directory will also stop any JBossAS instances you currently have running under the DJL. Keep all of this in mind when you ugprade the agent (since it will start with an empty /data directory). When co-located, these problems go away and the DJL configuration will survive an agent uninstall or upgrade Start JBoss Launcher Service Once the DJL service is installed, you can then start the JBossAS instance using the "Start JBoss Launcher Service" action. Note that the original "start" control action still operates as it normally does - but it has nothing to do with the DJL. You can still use "start" if you want to start the JBossAS using the original way of starting JBoss instances (that is, with the run.bat/sh script). But, if you want to start your JBossAS instance with the DJL, you must execute the "Start JBoss Launcher Service" action Stop JBoss Launcher Service Once the DJL service has been both installed and started, you can stop the JBossAS instance using the "Stop JBoss Launcher Service" action. Note that the original "stop" control action still exists and can be used to stop the JBossAS instance. It simply won't go through the DJL mechanism to perform the stop Uninstall JBoss Launcher Service If you no longer wish to manage the JBossAS instance via the DJL, you may uninstall it. You should uninstall the DJL service prior to deleting a JBossAS server from the inventory. Uninstalling the DJL service will ensure the Windows Service is removed (for Windows platforms). If you are on Windows and you have not installed the DJL co-located with your JBossAS instances, you must uninstall the DJL before uninstalling or upgrading your agent; otherwise, your Windows Services will become unresponsive. If you want to manually uninstall the DJL wrapper (e.g. you have uninstalled your agent and have no way of using JON to uninstall it), you can execute the following command line (note that you won't have to do this on UNIX; this is really only for the case when you want to manually uninstall the Windows Service, which obviously is a Windows-only concern): wrapper.exe -r global-djl-wrapper.conf 'set.jboss_instance_env_config_path=<full path to wrapper.env file>' The wrapper.exe is found either in your agent's /bin directory or (if you installed the DJL co-located with the JBossAS instance) in the <jboss-as-install-dir>/jbosson directory. T he global-djl-wrapper.conf file is found in the agent's home directory or (if you installed the DJL colocated with the JBossAS instance) in the <jboss-as-install-dir>/jbosson directory. In the agent's /data directory or (if you installed the DJL co-located with the JBossAS instance) in the 101

105 JBoss Operations Network 1 Users Guide <jboss-as-install-dir>/jbosson directory, you will find all of the wrapper environment files (*wrapper.env) - one.env file exists for each DJL wrapper you have installed File Deployment Actions You can deploy and undeploy files to your JBossAS server instances. T here are four control actions that you can execute on any JBossAS server resource: 1. Deploy File 2. Undeploy File 3. List Deployed Files 4. Undeploy Selected File T hese four control actions are described below Deploy File Execute this action to deploy a file to a JBossAS server. The following action parameters can be supplied - note that one and only one of the File To Upload and Deploy or File to Deploy parameters must be specified: File To Upload and Deploy - this allows you to upload the file to be deployed from your local machine (using your browser file upload capabilities). File To Deploy - this allows you to specify the full path to the file that is to be deployed. This path must be relative to the JON Server. In other words, the file to be deployed must exist on and be accessible to the JON Server machine. Deploy Path - this is the remote directory where the file is to be deployed. This is a relative path that is relative to the JBossAS server's server configuration directory. For example, if your JBossAS server is running from the "default" server configuration, you typically set this parameter to "deploy/" Undeploy File Execute this action to undeploy a file from a JBossAS server. T he following action parameters can be supplied: File to Undeploy - this is the file of the file to be undeployed. This must a file that exists in the Deploy Path. Deploy Path - this is the remote directory where the file is currently deployed. This is a relative path that is relative to the JBossAS server's server configuration directory. For example, if your JBossAS server is running from the "default" server configuration, you typically set this parameter to "deploy/" List Deployed Files This will simply show you the list of all files deployed in the given deployment directory. The following action parameter can be supplied: Deploy Path - this is the remote directory from which you want to obtain the listing. This is a relative path that is relative to the JBossAS server's server configuration directory. For example, if your JBossAS server is running from the "default" server configuration, you typically set this parameter to "deploy/" Undeploy Selected File This action is used to undeploy a file that you select from a list of existing, deployed files. This is a twostep action. When you first execute this action, you will be given the opportunity to enter the deployment 102

106 Chapter 6. Managed Products directory from which you want to obtain a list of existing, deployed files (see the List Deployed Files action above). Once you specify the deploy directory and hit OK, you will then be shown a drop down options list of all files deployed in that location. Select the file you want to undeploy and click OK to undeploy that file Tomcat Management Support T omcat Supported Versions Tomcat 4.0 Tomcat 4.1 Tomcat 5.0 Tomcat T omcat Monitoring Specification General Server Metrics Reliability Metrics Resource Utilization Metrics Connector Metrics Webapp Metrics Servlet Metrics T omcat Administration Specification Control Actions Administration Actions Provisioning Actions General Server Metrics Number of Requests Served Number of Requests Served per Minute Total Processing Time Total Processing Time per Minute Reliability Metrics Uptime Availability Resource Utilization Metrics JVM Active T hread Count JVM Active T hread Group Count JVM Free Memory JVM T otal Memory Process CPU System T ime Process CPU User Time 103

107 JBoss Operations Network 1 Users Guide Process CPU User Time Process Memory Size Process Shared Memory Size Process Open File Descriptors Connector Metrics Tomcat 4.1 and 5.0 Support Only Availability Bytes Received Bytes Received per Minute Bytes Sent Bytes Sent per Minute Error Count Error Count per Minute Request Count Request Count per Minute Maximum Request T ime Processing Request T ime Processing Request T ime per Minute T hreads Allocated T hreads Active Webapp Metrics Tomcat 4.0, 4.1, 5.0, 5.5 Number of Requests Served Number of Requests Served per Minute Number of errors Number of errors per Minute Sessions Created Sessions Created per Minute Sessions Destroyed Sessions Destroyed per Minute Minimum Response T ime of a servlet Maximum Response T ime of a servlet Average Response T ime Total Processing Time for the webapp Total Processing Time for the webapp per Minute Availability Warning T he below metrics are not supported for JBoss-embedded T omcat Servers. Do not enable collection of any of these metrics for JBoss-embedded T omcat Servers. 104

108 Chapter 6. Managed Products Standalone Tomcat 4.1, 5.0 and 5.5 only Expired Sessions Expired Sessions per Minute Rejected Sessions Rejected Sessions per Minute Max Active Sessions Servlet Metrics Availability Request Count Request Count per Minute Error Count Error Count per Minute Maximum Response T ime Average Response T ime Min Response Time Processing Request T ime Processing Request T ime per Minute Control Actions Start T omcat Stop Tomcat Restart T omcat Hibernate Support JBoss AS Supported Versions Latest version of Hibernate with JMX support Hibernate Monitoring Specification Availability Entity Insert Count Entity Insert Count per Minute Query Execution Max T ime Entity Update Count Entity Update Count per Minute Collection Update Count Collection Update Count per Minute Entity Load Count Entity Load Count per Minute Entity Fetch Count 105

109 JBoss Operations Network 1 Users Guide Entity Fetch Count per Minute Entity Delete Count Entity Delete Count per Minute Collection Recreate Count Collection Recreate Count per Minute Query Execution Count Query Execution Count per Minute Flush Count Flush Count per Minute Collection Load Count Collection Load Count per Minute Successful T ransaction Count Successful T ransaction Count per Minute Query Cache Hit Count Query Cache Hit Count per Minute Collection Remove Count Collection Remove Count per Minute Connect Count Connect Count per Minute Start Time Second Level Cache Put Count Second Level Cache Put Count per Minute Query Cache Put Count Query Cache Put Count per Minute Session Open Count Session Open Count per Minute T ransaction Count T ransaction Count per Minute Collection Fetch Count Collection Fetch Count per Minute Session Close Count Session Close Count per Minute Query Cache Miss Count Query Cache Miss Count per Minute Second Level Cache Miss Count Second Level Cache Miss Count per Minute JGroups Support JGroups Supported Versions Latest version of JGroups with JMX support JGroups Monitoring Specification 106

110 Chapter 6. Managed Products Availability Number of Messages Number of Messages per Minute Sent Messages Sent Messages per Minute Sent Bytes Sent Bytes per Minute Received Bytes Received Bytes per Minute 6.3. Web Servers Apache Management Support Apache Supported Versions Apache 1.3 (not supported on Windows) Apache 2.0 Apache 2.2 (JON 1.4.SP2 on UNIX only) Apache Monitoring Specification Resource Utilization Metrics Reliability Metrics T hroughput Metrics Request Metric Response Metrics VHost Metrics Apache Administration Specification Control Actions Configuration Actions Provisioning Actions Apache Product Configuration In order for JBoss Operations Network to gather metrics about the Apache web server, SNMP must be enabled for your Apache server. T he SNMP module is an Apache Dynamically Shared Object (DSO). JON can also gather response time metrics for your Apache server. T his feature also requires another DSO known as the response time (RT) module. Response time is configured per Apache virtual host. The steps to enable a DSO module in the Apache server in general are: 1. Compile and install the module 2. Configure the Apache server to use the module 3. Restart the Apache server 107

111 JBoss Operations Network 1 Users Guide T hese steps are described in detail below. Note In the following instructions, JON_INST ALL and APACHE_INST ALL refer to the installation path of your JBoss ON and Apache web server installations respectively. 1. Compiling and installing modules Installation on Windows JON does not support Apache 1.3 or Apache 2.2 on Windows. Pre-compiled copies of the SNMP and RT modules for Apache 2.0 are included in the JON full and agent distributions on Windows. The modules are packaged in a zip file at the following location in your JON installation: JON_INSTALL/product_connectors/apache-2.0-x86-winnt.zip Unzip the above file to APACHE_INST ALL (e.g., C:/Program Files/Apache Group/Apache2). Installation on UNIX T he following describes the steps to install Apache 1.3: Warning Apache 1.3 needs to have been configured with shared module support (--enablemodule=so) when it was built! Compile and install the SNMP module. % tar -zxf JON_INSTALL/agent-jboss-X.X.X/product_connectors/snmp-jboss- X.X.X.tgz % cd snmp-jboss-x.x.x %./build_apache_snmp.sh 1.3 APACHE_INSTALL/bin/apxs % cp snmp_module_1.3/module/*.so APACHE_INSTALL/libexec % cp snmp_module_1.3/conf/snmpd.conf APACHE_INSTALL/conf % mkdir APACHE_INSTALL/var Compile and install the RT module: % tar -zxf JON_INSTALL/agent-jboss-X.X.X/product_connectors/rt-jboss-X.X.X.tgz % cd rt-jboss-x.x.x %./build_apache_module.sh 1.3 APACHE_INSTALL/bin/apxs % cp apache1.3/unix/mod_rt.so APACHE_INSTALL/libexec T he following describes the steps to install Apache 2.x: Warning Apache 2.0 needs to have been configured with shared module support (--enable-so) when it was built! 108

112 Chapter 6. Managed Products Compile and install the SNMP module. % tar -zxf JON_INSTALL/agent-jboss-X.X.X/product_connectors/snmp-jboss- X.X.X.tgz % cd snmp-jboss-x.x.x %./build_apache_snmp.sh 2.0 APACHE_INSTALL/bin/apxs % cp snmp_module_2.0/module/*.so APACHE_INSTALL/modules % cp snmp_module_2.0/conf/snmpd.conf APACHE_INSTALL/conf % mkdir APACHE_INSTALL/var Compile and install the RT module: % tar -zxf JON_INSTALL/agent-jboss-X.X.X/product_connectors/rt-jboss-X.X.X.tgz % cd rt-jboss-x.x.x %./build_apache_module.sh 2.0 APACHE_INSTALL/bin/apxs % cp apache2.0/unix/.libs/mod_rt.so APACHE_INSTALL/modules 2. Configuring the Apache server to use the modules T he following describes the steps to configure Apache 1.3: Edit APACHE_INST ALL/conf/httpd.conf, adding: LoadModule snmp_agt_module libexec/libsnmp_agt.so LoadModule rt_module libexec/mod_rt.so SNMPConf conf SNMPVar var # Add the below line only if the ClearModuleList directive is in your config file. AddModule covalent-snmpv13.c RtLog logs/rt_log T he following describes the steps to configure Apache 2.x: Edit APACHE_INST ALL/conf/httpd.conf, adding: LoadModule snmpcommon_module modules/libsnmpcommon.so LoadModule snmpagt_module modules/libsnmpmonagt.so LoadModule rt_module modules/mod_rt.so SNMPConf conf SNMPVar var # NOTE: The letter 'S' inside the quotes below must be capitalized! LogFormat "%S" rt_log CustomLog logs/rt_log rt_log a. ServerName Make sure the "main" Apache configuration section, as well as each VirtualHost configuration block, contains a ServerNam e directive, which includes a port. T he SNMP module uses this directive to uniquely identify the main server and each virtual host, so make sure all ServerNames have unique values. For example: ServerName main.example.com:80... <VirtualHost vhost1.example.com:80> ServerName vhost1.example.com:80... </VirtualHost> 3. Restarting the Apache server to load the modules A. UNIX Apache 1.3: % APACHE_INSTALL/bin/apachectl restart Apache 2.x: % APACHE_INSTALL/bin/apache2ctl restart B. Windows 109

113 JBoss Operations Network 1 Users Guide Apache 2.0: > APACHE_INSTALL\bin\Apache -k restart Resource Utilization Metrics Number of Concurrent Connections Reliability Metrics Availability Start Time T hroughput Metrics T otal Number of Bytes Received Total Number of Bytes Received per Minute Total Number of Bytes Sent Total Number of Bytes Sent per Minute T otal Number of Requests Total Number of Requests per Minute T otal Number of Responses T otal Number of Responses per Minute Request Metrics Bytes Received for GET Requests Bytes Received for GET Requests per Minute Bytes Received for HEAD Requests Bytes Received for HEAD Requests per Minute Bytes Received for POST Requests Bytes Received for POST Requests per Minute Bytes Received for PUT Requests Bytes Received for PUT Requests per Minute Number of GET Requests Number of GET Requests per Minute Number of HEAD Requests Number of HEAD Requests per Minute Number of POST Requests Number of POST Requests per Minute Number of PUT Requests Number of PUT Requests per Minute Response Metrics Bytes Sent for 200 Responses Bytes Sent for 200 Responses per Minute 110

114 Chapter 6. Managed Products Bytes Sent for 301 Responses Bytes Sent for 301 Responses per Minute Bytes Sent for 302 Responses Bytes Sent for 302 Responses per Minute Bytes Sent for 401 Responses Bytes Sent for 401 Responses per Minute Bytes Sent for 403 Responses Bytes Sent for 403 Responses per Minute Bytes Sent for 404 Responses Bytes Sent for 404 Responses per Minute Bytes Sent for 500 Responses Bytes Sent for 500 Responses per Minute Number of 200 Responses Number of 200 Responses per Minute Number of 301 Responses Number of 301 Responses per Minute Number of 302 Responses Number of 302 Responses per Minute Number of 401 Responses Number of 401 Responses per Minute Number of 403 Responses Number of 403 Responses per Minute Number of 404 Responses Number of 404 Responses per Minute Number of 500 Responses Number of 500 Responses per Minute VHost Metrics Supported for Apache 1.3, Apache 2.0, Apache-ERS 2.3 and Apache-ERS 2.4 Bytes Received for GET Requests Bytes Received for GET Requests per Minute Bytes Received for HEAD Requests Bytes Received for HEAD Requests per Minute Bytes Received for POST Requests Bytes Received for POST Requests per Minute Bytes Received for PUT Requests Bytes Received for PUT Requests per Minute Number of GET Requests Number of GET Requests per Minute Number of HEAD Requests Number of HEAD Requests per Minute Number of POST Requests Number of POST Requests per Minute 111

115 JBoss Operations Network 1 Users Guide Number of PUT Requests Number of PUT Requests per Minute Bytes Sent for 200 Responses Bytes Sent for 200 Responses per Minute Bytes Sent for 301 Responses Bytes Sent for 301 Responses per Minute Bytes Sent for 302 Responses Bytes Sent for 302 Responses per Minute Bytes Sent for 401 Responses Bytes Sent for 401 Responses per Minute Bytes Sent for 403 Responses Bytes Sent for 403 Responses per Minute Bytes Sent for 404 Responses Bytes Sent for 404 Responses per Minute Bytes Sent for 500 Responses Bytes Sent for 500 Responses per Minute Number of 200 Responses Number of 200 Responses per Minute Number of 301 Responses Number of 301 Responses per Minute Number of 302 Responses Number of 302 Responses per Minute Number of 401 Responses Number of 401 Responses per Minute Number of 403 Responses Number of 403 Responses per Minute Number of 404 Responses Number of 404 Responses per Minute Number of 500 Responses Number of 500 Responses per Minute Availability T otal Number of Bytes Received Total Number of Bytes Received per Minute Total Number of Bytes Sent Total Number of Bytes Sent per Minute T otal Number of Requests Total Number of Requests per Minute T otal Number of Responses T otal Number of Responses per Minute Control Actions start apache stop apache 112

116 Chapter 6. Managed Products restart apache graceful startssl configtest Microsoft IIS Management Support Microsoft IIS Supported Versions IIS 4.x IIS 5.x IIS 6.x Microsoft IIS Monitoring Specification Reliability Metrics Connection Metrics T ransaction Metrics Performance Metrics Session Metrics I/O Metrics T hroughput Metrics Request Processing Metrics VHost Metrics Microsoft IIS Administration Specification Control Actions Reliability Metrics Availability Service Uptime Connection Metrics Current Connections Maximum Connections T otal Connection Attempts T otal Connection Attempts per Minute T ransaction Metrics T ransactions Aborted T ransactions Committed T ransactions Pending T ransactions T otal T ransactions T otal per Minute 113

117 JBoss Operations Network 1 Users Guide Performance Metrics Request Execution T ime Request Wait Time Session Metrics Session Duration Sessions Current Sessions T imed Out Sessions T otal I/O Metrics [T otal Blocked Async I/O Requests] [T otal Blocked Async I/O Requests per Minute] [T otal Allowed Async I/O Requests] [T otal Allowed Async I/O Requests per Minute] [T otal Rejected Async I/O Requests] [T otal Rejected Async I/O Requests per Minute] [Current Blocked Async I/O Requests] [Measured Async I/O Bandwidth Usage] T hroughput Metrics Bytes Sent Bytes Sent per Minute Bytes Received Bytes Received per Minute Total Files Sent Total Files Sent per Minute T otal Files Received T otal Files Received per Minute T otal Files T ransferred Total Files Transferred per Minute Current Anonymous Users Current NonAnonymous Users T otal Anonymous Users T otal Anonymous Users per Minute T otal NonAnonymous Users T otal NonAnonymous Users per Minute Maximum Anonymous Users Maximum NonAnonymous Users T otal Logon Attempts T otal Logon Attempts per Minute 114

118 Chapter 6. Managed Products T otal Get Requests Total Get Requests per Minute T otal Post Requests T otal Post Requests per Minute T otal Head Requests T otal Head Requests per Minute T otal Put Requests Total Put Requests per Minute T otal Delete Requests T otal Delete Requests per Minute Total Trace Requests Total Trace Requests per Minute T otal Other Request Methods T otal Other Request Methods per Minute T otal Method Requests T otal Method Requests per Minute T otal CGI Requests Total CGI Requests per Minute T otal ISAPI Extension Requests T otal ISAPI Extension Requests per Minute Total Not Found Errors Total Not Found Errors per Minute T otal Locked Errors T otal Locked Errors per Minute Current CGI Requests Current CGI Requests per Minute Current ISAPI Extension Requests Maximum CGI Requests Maximum ISAPI Extension Requests Errors During Script Runtime Errors During Script Runtime per Minute Errors From ASP Preprocessor Errors From ASP Preprocessor per Minute Errors From Script Compilers Errors From Script Compilers per Minute Requests Disconnected Script Engines Cached T emplates Cached Template Cache Hit Rate Current CAL count for authenticated users Maximum CAL count for authenticated users T otal Options Requests T otal Options Requests per Minute 115

119 JBoss Operations Network 1 Users Guide T otal Propfind Requests T otal Propfind Requests per Minute T otal Proppatch Requests T otal Proppatch Requests per Minute T otal Search Requests T otal Search Requests per Minute T otal Unlock Requests T otal Unlock Requests per Minute T otal Mkcol Requests T otal Mkcol Requests per Minute T otal Move Requests T otal Move Requests per Minute T otal Copy Requests T otal Copy Requests per Minute T otal Lock Requests T otal Lock Requests per Minute Request Processing Metrics Debugging Requests Debugging Requests per Minute Request Bytes In Total Request Bytes In Total per Minute Request Bytes Out Total Request Bytes Out Total per Minute Requests Executing Requests Failed T otal Requests Failed T otal per Minute Requests Not Authorized Requests Not Authorized per Minute Requests Not Found Requests Not Found per Minute Requests Queued Requests Queued per Minute Requests Rejected Requests Rejected per Minute Requests Succeeded Requests Succeeded per Minute Requests T imed Out Requests Timed Out per Minute Requests T otal Requests T otal per Minute T emplate Notifications In Memory T emplates Cached 116

120 Chapter 6. Managed Products In Memory Template Cache Hit Rate Script Engine Cache Hit Rate Engine Flush Notifications Current CAL count for SSL connections Maximum CAL count for SSL connections VHost Metrics 4.x, 5.x and 6.x VHost Metrics [ Availability] Bytes Sent Bytes Sent per Minute Bytes Received Bytes Received per Minute Total Files Sent Total Files Sent per Minute T otal Files Received T otal Files Received per Minute T otal Files T ransferred Total Files Transferred per Minute Current Anonymous Users Current NonAnonymous Users T otal Anonymous Users T otal Anonymous Users per Minute T otal NonAnonymous Users T otal NonAnonymous Users per Minute Maximum Anonymous Users Maximum NonAnonymous Users Current Connections Maximum Connections T otal Logon Attempts T otal Logon Attempts per Minute T otal Get Requests Total Get Requests per Minute T otal Post Requests T otal Post Requests per Minute T otal Head Requests T otal Head Requests per Minute T otal Put Requests Total Put Requests per Minute T otal Delete Requests T otal Delete Requests per Minute Total Trace Requests Total Trace Requests per Minute 117

121 JBoss Operations Network 1 Users Guide T otal Other Request Methods T otal Other Request Methods per Minute T otal Method Requests T otal Method Requests per Minute T otal CGI Requests Total CGI Requests per Minute T otal ISAPI Extension Requests T otal ISAPI Extension Requests per Minute Total Not Found Errors Total Not Found Errors per Minute T otal Locked Errors T otal Locked Errors per Minute Current CGI Requests Current CGI Requests per Minute Current ISAPI Extension Requests Maximum CGI Requests Maximum ISAPI Extension Requests T otal Blocked Async I\-O Requests T otal Blocked Async I\-O Requests per Minute T otal Allowed Async I\-O Requests T otal Allowed Async I\-O Requests per Minute T otal Rejected Async I\-O Requests T otal Rejected Async I\-O Requests per Minute Current Blocked Async I\-O Requests Measured Async I\-O Bandwidth Usage 5.x and 6.x VHost Metrics only [ Current CAL count for authenticated users] Maximum CAL count for authenticated users T otal Options Requests T otal Options Requests per Minute T otal Propfind Requests T otal Propfind Requests per Minute T otal Proppatch Requests T otal Proppatch Requests per Minute T otal Search Requests T otal Search Requests per Minute T otal Unlock Requests T otal Unlock Requests per Minute Current CAL count for SSL connections Maximum CAL count for SSL connections T otal Mkcol Requests T otal Mkcol Requests per Minute 118

122 Chapter 6. Managed Products T otal Move Requests T otal Move Requests per Minute T otal Copy Requests T otal Copy Requests per Minute T otal Lock Requests T otal Lock Requests per Minute iplanet/sunone Management Support iplanet/sunone Supported Versions iplanet Admin 4.1 iplanet 4.1 iplanet 6.0 iplanet Admin iplanet/sunone Monitoring Specification Reliability Metrics Connection Queue Metrics Resource Utilization Metrics Process Metrics T hroughput Metrics Thread Pool and VHost iplanet/sunone Administration Specification Control Reliability Metrics Availability Uptime Connnection Queue Metrics Connection Queue Count Connection Queue Peak Connection Queue Max Connection Queue Overflows Connection Queue Overflows per Minute Resource Utilization Metrics Idle T hreads Active T hreads Active T hreads Resolving DNS Number of Running T hreads 119

123 JBoss Operations Network 1 Users Guide Number of Running Processes Status Number of Deaths Number of Deaths per Minute Total Number of Threads Keepalive Count Keepalive Count per Minute Keepalive Max Process Metrics Load Average 1 Minute Load Average 5 Minutes Load Average 15 Minutes Process Virtual Size Process Resident Size Process System Memory Usage Idle Time Idle Time per Minute User Time User Time per Minute Kernel Time Kernel Time per Minute T hroughput Metrics Number of Requests Number of Requests per Minute Number of Error Requests Number of Error Requests per Minute Bytes Received Bytes Received per Minute Bytes Sent Bytes Sent per Minute Network Bytes Received Network Bytes Received per Minute Network Bytes Sent Network Bytes Sent per Minute Success Responses Success Responses per Minute Redirect Responses Redirect Responses per Minute Client Error Responses Client Error Responses per Minute Server Error Responses 120

124 Chapter 6. Managed Products Server Error Responses per Minute Bad Requests Bad Requests per Minute Authentication Failures Authentication Failures per Minute Forbidden Errors Forbidden Errors per Minute Not Found Errors Not Found Errors per Minute T hread Pool and VHost Supported on iplanet 4.1 and 6.0 Availability 6.4. File System Services FileSystem Mount Availability Use Percent Total Bytes Used Capacity Total Bytes Free Total Bytes Avail Disk Reads Disk Reads per Minute Disk Writes Disk Writes per Minute Free Files Total Files Filesystem Directory Metrics Last Modified T ime Last Change Time Last Access Time Permissions Owner User Id Owner Group Id Availability Regular Files Subdirectories Symbolic Links 121

125 JBoss Operations Network 1 Users Guide Character Devices Block Devices Sockets Total File Service Overview A File service is associated with a platform. It allows a file on the Platform machine to be monitored and, via the Control actions, actually executed Configuration Properties Property Required Description path yes the full path to a file on the Agent machine prefix no a prefix to prepend to the command line when executing the script (e.g. sudo) timeout yes a timeout for the control action (in seconds) Metrics Last Modified T ime Last Change Time Last Access Time Permissions Owner User Id Owner Group Id Availability Size Control Actions Run: T ries to execute the actual file associated with this service Network Services Network Interface Availability Bytes Received Bytes Received per Minute Packets Received Packets Received per Minute Bytes T ransmitted 122

126 Chapter 6. Managed Products Bytes T ransmitted per Minute Packets T ransmitted Packets T ransmitted per Minute T ransmit Errors T ransmit Errors per Minute Receive Errors Receive Errors per Minute T ransmit Packets Dropped T ransmit Packets Dropped per Minute Receive Packets Dropped Receive Packets Dropped per Minute T ransmit Collisions T ransmit Collisions per Minute Network IP Service Metrics Availability Request T ime 6.6. System Process Services MultiProcess Metrics MultiProcess metrics collect metrics across a collection of processes. T he definition of which processes to select is based on a process table query language. Availability Virtual Memory Size Memory Size Cpu System T ime Cpu System T ime per Minute Cpu User Time Cpu User Time per Minute Cpu Total Time Cpu Total Time per Minute Cpu Usage Start Time Open File Descriptors Process Metrics Availability Virtual Memory Size Memory Size Cpu System T ime Cpu System T ime per Minute 123

127 JBoss Operations Network 1 Users Guide Cpu User Time Cpu User Time per Minute Cpu Total Time Cpu Total Time per Minute Cpu Usage Start Time Open File Descriptors Windows Service Metrics Collects information on windows services. T o enable collection you must specify the of the windows service. Availability 6.7. Script Services Script Service Overview A Script service is associated with a JON platform. It allows a script on the platform machine to be executed periodically by the JON Agent. A number of metrics corresponding to the filesystem attributes of the script file (i.e. timestamps, permissions, ownership, and size) can also be collected Configuration Properties Property Required Description path yes the full path to an executable script or binary on the platform machine prefix no a prefix to prepend to the command line when executing the script (e.g. sudo) Metrics Availability - based on the exit code of the script (0 translates to available, non-zero translates to unavailable) Execution Time - how long the script execution took Last Modified Time - the time the script file was last modified Last Change T ime - the time the script file's filesystem attributes were last modified Last Access Time - the time the script file was last accessed Permissions - the permissions of the the script file Owner User Id - the id of the user that owns the script file Owner Group Id - the id of the group that owns the script file Size - the size of the script file 124

128 Chapter 6. Managed Products NONE Nagios Plugin Service Overview A Nagios Plugin service is associated with a JON platform. It allows a Nagios plugin on the platform machine to be executed periodically by the JON Agent. A number of metrics corresponding to the filesystem attributes of the plugin file (i.e. timestamps, permissions, ownership, and size) can also be collected Configuration Properties Property Required Description path yes the full path to a Nagios plugin on the platform machine args no the number of arguments expected by the script prefix no a prefix to prepend to the command line when executing the plugin (e.g. sudo) Metrics Availability - based on the exit code of the plugin (0 translates to available, non-zero translates to unavailable) Execution Time - how long the plugin execution took Last Modified Time - the time the plugin file was last modified Last Change T ime - the time the plugin file's filesystem attributes were last modified Last Access Time - the time the plugin file was last accessed Permissions - the permissions of the the plugin file Owner User Id - the id of the user that owns the plugin file Owner Group Id - the id of the group that owns the plugin file Size - the size of the plugin file Control Actions NONE 125

129 JBoss Operations Network 1 Users Guide Chapter 7. Response Time 7.1. What is Response Time? Response T ime is the measurement of how fast your resources are responding to requests. JBoss ON response time measurements can help you pinpoint performance problems in your IT infrastructure Configuring Response Time Response T ime measurements can be collected for Apache, iplanet (SunONE), IIS, T omcat, and JBoss. Each product requires configuration to allow JBoss ON to collect response time data Apache In order to collect response time metrics for Apache, a module must be built and installed in the Apache installation. T he module is called mod_rt and is provided in the JON Agent directory in a subdirectory called product_connectors. Details on how to build and install the module can be found in the Section 6.3.1, Apache Management Support. Once the module is installed, navigate to the Resource Configuration section, of the Inventory page for the Apache virtual host you wish to collect response time data for. Make the appropriate changes to enable response time by turning it on and entering the correct path to the response time log Servlet Response time for servlets is relatively easy to configure. All of the servlet products supported by JBoss Operations Network require an additional XML fragment in their configuration to allow JON to collect metrics. This XML fragment has a section for enabling response time that is commented out by default. T he following comments in the XML fragment include information about how to enable response time, and how it works. <!-- Uncomment the following line to enable response time logging. The directory you specify as the param can include properties referenced from the System.properties of the vm. The ResponseTime log file will by default store the last 1 hour's worth of response time data. This file gets truncated as soon as data is succesfully sent into the server. The file is d uniquely for each webapp in the form: yourcontextname_jbnemresponsetime.log --> If this fragment is included in the global web.xml for the container, all webapps in it will generate response time data, and have logs following the format described above. You can enable it individually on each webapp as well if you dont want every webapp to generate response time data T o enable response time for servlets, simply uncomment the following XML in the fragment, and replace "/path/to/log/directory" with the appropriate directory on your system where servlet response time logs should be stored. 126

130 Chapter 7. Response Time <init-param> <param->responsetimelogdir</param-> <param-value>/path/to/log/directory</param-value> </init-param> For details on how to configure your specific servlet product please see the section on Servlet Product Configuration" in the JBoss Operations Network 1 Reference Guide available from Charting Response Time Charts of response time data can be found on the RESPONSE TIME tab. This will bring you to the Monitor Performace page. T his page will give you details on how each response has been performing. So for an Apache virtual host, you will see each URL that has been requested from that virtual host, the number of requests, the LOW (fastest response time), AVG (average response time of all requests) and PEAK (slowest response). On the right side of the page, these URLs will be charted. The data can be sorted by LOW, AVG or PEAK. 127

131 JBoss Operations Network 1 Users Guide Chapter 8. Control 8.1. Controlling Resources JBoss ON provides the capability to control resources one at a time, or issue controls to a group of similar resources at the same time. T he control actions JBoss ON makes available are resource-dependent. Servers support starting, stopping, and restarting. Services generally support starting, stopping, and reloading. Users can perform these control actions from a single central location via the JBoss ON GUI Console rather than having to log in to each machine's console or use multiple interfaces. 128

132 Chapter 9. Administration Chapter 9. Administration 9.1. JBoss Operations Network Administration T his chapter covers Administration of the JON Server. Administration tasks include defining authentication sources, managing users and roles, and managing how JON manages its metric data. All administration functions can be found using the Administration link from the top of each page in the JON GUI. T o access administration functions, the user must have JBoss Operations Network Administrator privileges JBoss ON Users and Roles JBoss Operations Network offers the ability to give fine grained access resources that are defined in the inventory. Users in the system can be added using the New User Link on the Administration page. To edit or remove users, use the List Users Link. By default, a new user does not have access to any of resources in the inventory. JON uses the concept of roles to manage the permissions individual users have to perform various tasks within JON. Roles determine whether a given user can view, create, modify, monitor, and control a specified Resource T ype. Resource T ypes include: Users Roles Groups Platforms Services Applications Because the platform, server, and service resources "stack" (in the sense that services run on servers that in turn run on platforms), permissions are also stacked. For instance, the permission to create a server (or any child object on a resource), is predicated on the parent resource allowing you to do so. For example, the permission to add a server is really associated with a platform. When you create a role, you assign users and resource groups to it. In order for a JON user to have permissions on a particular resource, that user must be assigned to a role that in turn is associated with a group that has that resource assigned to it. Note that the groups you associate with a role can be either Compatible Groups or Mixed Groups. If a user owns a platform, that user can create servers on that platform even though he does not have permission to create servers through a role. If that same user has permissions to create servers through a role, that user can create a server on any platform (whether he owns it or not) as long as that platform is a member of a group assigned to the same role Server Configuration JON Base URL Configuration Properties The Base URL is what users access to get to the JON GUI Console. This is also the URL used in the alert s that are sent to users (the URLs in alert s will allow users to click the link directly from 129

133 JBoss Operations Network 1 Users Guide their client and get to the information on the alert). The Base URL should not need to be changed unless your JBoss ON installation is fronted by a proxy server. The WebDAV URL points to the root of the WebDAV repository used to store topology content. Do not use localhost! Do not use localhost as the host in the URLs. If you do, remote agents will not be able to execute some control actions and alert s will not provide valid, "clickable" links. The HTTP Proxy URL and HTTP Proxy Port (JBoss ON 1.4 and higher) are used to define a proxy url and port used by the Software feed reader and the Cummulative patch downloader. (More information is available under "Accessing feed via HTTP Proxy" in the JBoss Operations Network 1 Reference Guide available from Ex: HTTP Proxy URL: HTTP Proxy Port: JON Configuration Properties T he configuration properties specify from who the alert s will be sent from Data Manager Configuration Properties JBoss ON stores monitoring data using a tiered model to reduce the amount of data that is stored and still provide the necessary granularity to provide useful historical metric data. The following outlines the main steps JBoss ON uses to archive its metric data: Raw metrics get aggregated over a one-hour window producing minimum, average and maximum values. Those one hour values get aggregated across a six hour window. Finally six hour values get aggregated across a 24 hour window. Old data gets deleted according to the following schedule: Raw metric data which has been aggregated and is older than 2 days (configurable between 1 and 7 days) One hour data which has been aggregated and is older than 14 days Six hour data which has been aggregated and is older than 31 days 24 hour data which is older than 365 days To modify the length of time raw metric data is retained, change "Delete Detailed Data Older Than" to the desired number of days. Note that the maximum allowed value for this setting is 7 days. The process which does the aggregate calculations and deletes old data runs once an hour. In addition to the above process database maintenance can be run hourly if you are using Postgres as the backend database to JBoss ON. To change the frequency this maintenance runs modify the "Run Database Maintenance Every" setting. You can also specify how long response time data and alerts are retained. By default, both are kept for 130

134 Chapter 9. Administration You can also specify how long response time data and alerts are retained. By default, both are kept for 31 days. Finally, you can set whether metric data tables should be reindexed nightly. By default, reindexing is enabled. When finished changing settings, apply them by clicking the OK button. Note that any changes to this section will require a server restart to take effect Automatic Baseline Configuration Properties JBoss ON's baselining feature allows for configuration of three main properties that contribute to the automatic calculation of metric baselines. Refer to Section 4.1.2, Baselines. T he Baseline Frequency property allows configuration of how often baselines are automatically calculated. The default value is 3 days. T he Baseline Dataset property allows configuration of the time interval used to calculate the baseline. The default for this setting is 7 days. T he Baseline Minimum Data Points property defines the minimum number of data points, collected during a Baseline Dataset time interval, required to calculate a baseline. By default, this value is set to 40. After making any changes, click the OK button to apply your changes LDAP Configuration Properties By default JBoss Operations Network uses its own database for storing information about users. JBoss ON also allows an external LDAP directory to be used, in addition to its own database, for authenticating users. T o enable LDAP-based authentication, check the Use LDAP Authentication checkbox. If LDAP authentication is enabled, it must be configured using the remaining settings, which are described below. Property Name Description Required if LDAP authentication is enabled URL SSL User the URL the JON Server will use to connect to the LDAP directory (e.g. "ldap://localhost/"); if no port is specified in the URL, 389 is used, or 636 if is SSL is enabled enable this boolean property if the LDAP directory requires SSL connections the LDAP distinguished (DN) of the user to use to connect to the LDAP server to perform the JON user search (e.g. "cn=manager, dc=jboss, dc=com"); this user must have permission to yes yes yes 131

135 JBoss Operations Network 1 Users Guide Password Search Base Search Filter Login Property search the subtree of the directory rooted at the Search Base the password to be used in conjunction with the above user the DN of the LDAP entry which is the root of the subtree to be searched (e.g. e.g., "o=jboss, c=us" or "ou=people, dc=jboss, dc=com"); this value is also commonly known as the suffix; if you are unsure of this setting, consult your LDAP administrator an LDAP filter, in the syntax defined by RFC 2254, to apply when doing the LDAP user search; this is useful if the population to authenticate can be identified via the values of one or more LDAP attributes (e.g. "(JONUser=T RUE)" or "( (memberof=cn=jboss Adm inistrators,ou=serve r Service Accounts,OU=Servers,OU=I nformation System s,dc=xyz,dc=com)( m em berof=cn=developm en t Departm ent,ou=developm e nt,ou=information System s,dc=xyz,dc=com))" ) the LDAP attribute that contains the value that should be used by JON as the user (e.g. "cn" or "uid"); if multiple matches are found, the first entry found is used When finished making any changes to the LDAP configuration properties, click the OK button to apply your changes. Your settings can be verified by logging out of the JBoss ON GUI using the Logout link at the top of the page. Log in using a user from the configured LDAP directory (i.e. the user should equal the value of the Login Property attribute in a directory entry within the subtree rooted at the Search Base entry). Upon successful authentication, ON will ask for additional information about that user (full, address, etc.); this information will be persisted to the ON database. yes yes no yes 132

136 Chapter 9. Administration Warning As with all new users in ON, LDAP users are not assigned to any roles by default and therefore they will not be able to see any resources in the ON GUI until they have been added to one or more roles. An ON administrator will need to add a new LDAP user to one or more JON roles after the user has logged in for the first time SNMP Server Configuration Properties To enable the option to emit SNMP traps as an alert notification mechanism, specify the SNMP protocol version that is implemented by the server that will be the destination for traps emitted by ON. ON supports SNMP versions 1, 2c, and 3. If SNMP is enabled, it must be configured using the remaining settings, which are described below. T he settings vary depending on which version of SNMP has been selected. TODO 9.4. Monitoring Defaults Configuration T he JON Monitoring Defaults Configuration page allows you to specify the default settings for metrics that are enabled by default. The settings include whether or not a metric should be enabled by default and what its default collection interval is. T he configuration is applied to the resource type, and only affects new resources that get imported or configured. To modify the settings, click on the EDIT METRIC TEMPLATE button next to the resource type. T he Monitoring Defaults Configuration page for the specified resource type is displayed. Each metric that can be collected by resources of this type is listed with two columns: Default On - whether or not the metric is enabled by default Collection Interval - what the collection interval is, if the metric is to be collected by default T o enable metrics for collection by default and/or modify the collection interval, select the metrics you want by checking the checkbox(es) next to the metric (s). Next, enter the value you want in the Collection Interval for Selected field, selecting a time unit from the drop-down list next to the field. T hen click the right arrow button to enable the metrics with the specified collection interval you specified. T o disable metrics for collection by default, simply select the metrics you want to disable by checking the checkbox(es) next to the metric (s). T hen click the DISABLE COLLECT ION button to prevent the metric(s) from being enabled when a new resource of the specified type is imported, created, or configured Agent Configuration T he JBoss ON agent is configured using the agent.properties file found in the top level agent directory. Optionally, properties may also be specified in a properties file located at.jon/agent.properties in the home directory of the user that runs the JON agent. Any settings specified in.jon/agent.properties override those specifed in the agent.properties file found in the top level agent directory. Preferrably, configuration changes for the agent should be placed in the.jon/agent.properties file so that the configuration survives agent upgrades. Note that changes to the agent.setup.xxx properties require the agent to be told to setup again. You do 133

137 JBoss Operations Network 1 Users Guide this by, first, ensuring that the agent is running and then executing the agent executable passing it the "setup" command line argument, simply restarting the agent will not affect the values which are used for these properties. Changes to any other properties take effect after restarting the agent. Note that if the agent's./data directory is empty, i.e. after install, then when the agent starts up it will also carry out a "setup" step. Laptop Installations If you are installing JBoss ON on a laptop or other computer that regularly disconnects from the network, dynamically obtains new IP addresses, connects to a VPN or experiences other network addressing changes, please see the Section 2.1.4, Laptop Installations. T he following agent configuration properties are available. You can also take a look at the agent.properties for documentation on more properties - that file is commented with descriptions of the lesser used properties. agent.listenport This property specifies the default port for the agent to bind to on starup. This port is used for commands sent from the JON server to the agent. The default value for this option is This is not necessarily the actual port that the server uses to connect to the agent (due to firewall port forwarding rules for example) - see agent.setup.agentport for that. agent.listenip This property specifies which IP address to bind to on startup. By default this value is '*', meaning that all IP addresses available on the machine will be bound. This is not necessarily the IP that the server uses to connect to the agent (due to firewall port forwarding rules for example) - see agent.setup.agentip for that. agent.javaopts T his property allows extra parameters to be passed to java when starting the agent process. agent.startupt imeout The time (in seconds) in which the agent is given to successfully setup its server socket. agent.maxbatchsize Maximum number of metrics sent in any one message to the server. agent.setup.serverip The IP address the agent will use to connect to the server. agent.setup.serverport The port that the agent will use to connect to the server in order to send unsecure messages to the server. agent.setup.serversslport 134

138 Chapter 9. Administration The port that the agent will use to connect to the server in order to send secure (via SSL) messages to the server. agent.setup.serversecure Tells the agent if it should send messages to the server using SSL. agent.setup.serverlogin The user that the agent will use to authenticate itself with the server. This is just part of the authentication scheme the agents use to identify itself with the server. There is also a secure token that the agent shares with the server to further identify itself. agent.setup.serverpword The password the agent will pass to the server to authenticate the login. agent.setup.servert imeout The time (in seconds) the agent will wait before determining that the server is down. This is only used during a setup - the server must be accessible in order for the agent to set itself up (which occurs the very first time an agent starts up or if the "setup" command argument is passed to the agent executable). agent.setup.agentip The IP address that the server will use to connect to the agent. This is not necessarily the same as agent.listenip. agent.setup.agentport The port that the server will use to connect to the agent. This is not necessarily the same as agent.listenport. agent.setup.resetupt okens If set to 'yes', this will tell the agent to regenerate another security token used to identify itself with the server License Management T he JON server requires a valid license that can be downloaded from on the software downloads page. Once installed, the software will redirect all requests to the license management page providing adminstrators the ability to install a license. 1. You'll be directed here after logging in as the administrator. 135

139 JBoss Operations Network 1 Users Guide Select "Update License" 2. When you click on update license, you'll see the following license entry form. 3. Clicking the link "JBoss Network Customer Service Portal" takes you directly to the network csp's software download page. Which license download links you see will depend upon your support contract. Contact your support sales representative for more information. 4. Choose your license and right-click save the license file to your hard disk. Then go back to the license update page, and use the dialog to upload the license file. The following screen will display, indicating the license has been successfully updated and that you can now start using the product. At any point after the license has been loaded, you can choose to upload a new license by returning to the licensing page from the administration section. 136