Java Management Extensions (JMX) and IBM FileNet System Monitor Derive J2EE statistics from FileNet System Monitor alerts Level: Introductory Steven J. Bass 01.Mar.2009 Scope: Does your customer want to understand Web Application metrics during peak usage times or have they deployed multiple applications to a single J2EE server? Often, slight degradation of performance over a finite period of time can herald an upcoming outage due to many factors. Having visibility into key statistics of the Java Virtual Machine (JVM) can help avert these outages before they happen. FileNet System Monitor (FSM) has these statistics available to it, and with a minor amount of configuration, provides them through the FSM Web Console, giving a single point of control for both System Availability and System Performance. This article provides an understanding of JMX based performance monitoring within the FSM framework.
Introduction IBM FileNet System Monitor (FSM) is an ECM Functional Expansion that enables organizations to effectively manage IBM FileNet P8, CMv8, and CMOD components and infrastructure while automating routine work and reducing cost in IT operations. With FSM, you will reduce downtime and costs while improving Management Productivity. FSM leverages the ability to monitor your Java Application Servers with JMX and integrates it into the FSM monitoring solution. This substantial integration provides visibility into the Java Application Server, and allows selection of key metrics that enable a very fine level of granularity to be observed. Proactive monitoring of these metrics with the FSM monitoring methodology can help avoid resource contention issues of an infinite magnitude. Java Management Extensions is a Java technology that provides tools for managing and monitoring applications, system objects, and devices. The Managed Bean (MBean) is the resource representing the object. The programming model of Java Beans is an official technology used to provide program parts with a plug and play mechanism. Architecture The infrastructure of JMX is a three-tiered architecture as shown below. FileNet System Monitor (FSM) The MBean is a type of JavaBean with extensions that enable it to be used as a conduit to the application configuration, statistic collection and notifying events. Statistics of interest to FSM would include performance, resource utilization, problems. This whitepaper will familiarize you with the technology incorporated within the IBM FileNet System Monitor product that enables you to leverage this technology, and use it to your advantage in proactively monitoring your ECM Platform.
FileNet System Monitor and JMX Assumptions The explanations below assume basic understanding of the FSM User Interface. The graphics below do not depict the entire UI; therefore subjects such as the Hosts view and invocation of the Monitoring Manager applet are not addressed. The Web Console interface for FSM shows a grouping of monitors under individual silos of functionality. This aggregation is customer configurable. The grouping shown below is strictly an example of how monitors might be arranged within a P8 environment. The third silo from the left is the Application Server silo. The green circle below the silo indicates the worst severity of the monitors within this group. If there was a warning, critical, or fatal event occurring within this group, the individual monitor would affect the entire silo s color (red is a warning event, black is a fatal event). Figure 1 Monitoring Web Console As we drill down into the Application Server silo, we see a single monitor. As shown below, we see the status of green, a harmless situation. Additionally shown is the value of OK. In this case OK for the JPSMonConfJMXMon indicates that the latest invocation of this monitor detected that all the interrogated values were within the tolerance range. The details of the monitor are presented on the next page. The remainder of this document will illustrate how this monitor was built and why it is important to use JMX monitors in your environment. Figure 2 JMX Monitor Output
Figure 3 Monitor Output Details As described in the monitor output above, the following conditions are evaluated with a single monitor invocation: Evaluating the JVM Runtime MBean The HeapFreeCurrent parameter has returned 32271648. The error condition would be satisfied when the HeapFreeCurrent returned less than 1000000 The HeapSizeCurrent parameter has returned 100663296. The error condition would be satisfied when the HeapSizeCurrent is greater than 110663296 Evaluating the Application Runtime MBean The name of the ApplicationRuntime parameter is assured to be p835ae_workplace. The error condition would be satisfied when the ApplicationRuntime parameter is any other value. In the case of an error condition, the monitor would not return ok as the Value, and the message would indicate which of the conditions has experienced the anomaly. FSM provides default knowledgebase entries as shown above. The knowledgebase can be augmented with additional detail of your experience, and/or contain hyperlinks to an existing PMR where notes from a prior problem can be retrieved. The entire contents of this monitor can be forwarded as an email or automatically sent as an email, SMS message, or pager alert.
Building the FSM JMX Monitor The Monitoring Manager tool is an applet within FSM that allows us to define the monitor. We have control over how frequently the monitor runs, as well as advanced scheduling considerations so that the monitor can be disarmed during certain known system outages. The escalation table provides flexibility to assign a range of status indicators, as well as an action to take when the monitor reaches the associated status. Additional information for this monitor is shown below. I have entered the necessary information for the monitor to establish connectivity with the Weblogic server, providing port information, user and password. A library path and Java path are also integral to a monitor of this type. The new feature for FSM 4.0.1 is shown by the arrow below. On the line where the configuration file name is represented by ae.xml, clicking the Edit button takes us to a new JMX browser feature incorporated into FSM. Figure 4 Monitor Definition in the Monitor Configuration Tool
Sample Use Case At this point, we have the information to create our JMX monitor. The nature of this JMX monitor is to assign thresholds relevant to our JVM. Once the JMX monitor is created, the resulting values are stored in a database. They can be viewed from the FSM Console, and analyzed for trends using the FSM reporting tool. The creation of this monitor has several steps, which we review below. The following interface is shown after Edit has been selected. This is the tool that enables us to select specific Java Beans to be interrogated via JMX monitoring. As depicted below, this monitor is ensuring the HeapFreeCurrent remains less than the value of 1000000, and the HeapSizeCurrent remains greater than 110663296. These values are specific to the environment for this demonstration and are not recommendations for your monitors. As each implementation is unique, the thresholds can be adjusted for your specific criteria. Figure 5 JMX Monitor Definition example in the JPS Monitor Configuration Tool
Browsing through the available beans within the Workplace application shows us that there are many statistics available to us for assigning thresholds and monitors. This becomes critical as more applications are running within the Application Engine component of the P8 Infrastructure. As seen below, there are additional metrics that are easily ascertained via the JMX browser. Information such as OpenSessionsHighCount and OpenSessionsCurrentCount give us visibility into the number of User Connections to the application. Metrics at this depth provide many useful purposes. We can now track usage of multiple applications running on the same Application engine, tracking contention of resources, memory affecting events, and many other situations that might ultimately cause an outage if these thresholds were crossed. The use of FSM as a proactive monitoring solution gives the System Administration team the detailed information it needs to make recommendations on future scalability, web farming, and other resource viability. With the FSM reporting tool, these metrics can be correlated over a period of time and provide graphical evidence of the performance of the system over time. Figure 6 JMX Monitor Definition continued - MBean browsing in the JPS Monitor Configuration Tool
The result of using the JPS Monitoring Configuration tool is saved into an xml file that is deployed to the respective server being monitored. In this case, the ae.xml file is shown below. No editing of the xml file is needed. It is shown for illustration purposes only. All selection of Managed Beans and assignment of thresholds is done via the JPS Monitoring Configuration tool. Figure 7 JMX Monitor Definition translation in XML Other Use Cases Many customer implementations are using Workplace, Workplace XT, BPF generated applications and other custom web applications. FileNet System Monitor has visibility into the Java MBeans from any of these applications. JMX monitors for these products are built into the FSM product itself. Utilization of FSM JMX monitoring for all web deployed applications alleviates much of the guesswork associated with understanding what is going on in the respective Java Virtual Machines (JVM). For example, utilization of FSM in monitoring Workplace provides analysis on full resource utilization within by that application, something that is not visible to FileNet administrators. This provides the administrators control over Websphere, Weblogic, JBoss, and other J2EE Servers that may be outside of the ECM administrators domain. Suggested scenarios for JMX monitoring include metrics as described above, including HeapFreeCurrent, Heap Size Current, Open Sessions and Servlet Sessions. The potential list of MBeans that can be ultimately included is beyond the scope of this document. Development groups should be encouraged to expose as much functionality at the Java Bean level as possible for discrete transaction analysis. Instrumentation of the Java Bean to provide visibility via JMX is a development topic, and is beyond the scope of this document.
Conclusion This article demonstrates that FileNet System Monitor can be leveraged as a JMX Monitoring tool, in addition to being an essential piece of the platform for system availability. The proactive monitoring feature of FSM has helped many customers avert system outages, and substantially reduces system downtime. Traditional system monitoring via third party tools is not integrated with the FileNet API set, as is the case with FSM. Therefore, it is the opinion of the author that FileNet System Monitor should be a solution for both the stability and assurance of the ECM platform availability. It is important to remember that this type of JMX monitoring can be extended across any web application within the ECM platform or any infrastructure where J2EE Web Applications are utilized. Contact Information IBM Steven Bass (sbass@us.ibm.com) Disclaimer The subject matter contained in this article is the opinion of the author. If you find any discrepancies or would like to discuss this topic, please contact me. Feedback Please provide feedback directly to my attention. This is a new concept for this particular product and I would like your input on how the field is receiving this contribution.