BSM and Enterprise IT Infrastructure Monitoring Guide & Reference

Transcription

1 BSM and Enterprise IT Infrastructure Monitoring Guide & Reference 1. Adopting Service-oriented Monitoring for Your Evolving IT Infrastructure 2. Giving Your Constituents Real-time Visibility to Business Service Health 3. Combining Bottom-up Analysis with Top-Down User Experience Metrics 4. Connecting Your IT Monitoring Software with Web Services APIs & Plug-ins 5. Making Sure Your IT Monitoring Solution is Built for Change 6. Taking Steps to Ensure Seamless Support for IPv6 and IPv4 7. Making Certain You Don t Have Scalability and Data Processing Bottlenecks 8. Including Your Remote Sites for an Organization-wide Performance View 1

2 1. Adopting Service-oriented Monitoring for Your Evolving IT Infrastructure Given the rapid adoption of shared and virtualized infrastructure in the data center environment, a new approach is needed to ensure the effective performance of the IT infrastructure. Rather than monitoring individual nodes and components in the data center in a piecemeal manner, organizations need to monitor the performance of supported services as well by correlating all the underlying components of the service. The monitoring approach for applications and services has to account for inter dependencies and impacts of all the IT infrastructure dimensions that can impact services. The early era of IT monitoring was centered around determining what s up and what s down, and generating events for NOC technicians to act on. This then gave way to capturing network performance data and doing historical analysis and future trending to proactively address problem areas. All of this was still focused on individual devices. The dynamic and complex environment today requires organizations to adopt an integrated IT monitoring and Business Service Management (BSM) approach, as this provides the ability to look across the entire environment to see what s working and what s not at a services level, and not just simply what s on and what s not. BSM solutions go beyond monitoring of individual components, and in addition identify impacted business services as well. The new approach starts with creating a real time service model that links and maps IT elements to supported business services. The service model can be hierarchical, and multiple underlying technical components or even individual performance metrics can be mapped to multiple services. Rules can be defined to automatically create the service model, and map devices to service containers or service descriptors. A fully integrated system that has the service management layer built on top of a technical IT event and performance management layer is key to being able to correlate and determine impacts to services. Deep performance metrics are then captured from all the elements in the IT layer, including network devices, servers, storage systems and software applications, and threshold analysis is performed to generate warning or critical events. The ability to capture metrics from a broad range of infrastructure is critical, as all of these need to be factored into determining the impact on supported services. A leading analyst rightly stated, If You Don t Manage Everything, You Don t Manage Anything, given that the failure to monitor one element can lead to the failure of the entire system or service. Complex multitier applications and supported business services are composed of a myriad of components, hardware and software, that can all fail. It is important to aggregate the monitoring data into the core analysis and reporting system to determine service impacts. The component to service mappings and severity rules for services then help identify impacted services when performance problems occur in the IT infrastructure. For example, if a web ordering service is mapped to a web server farm, then rules can be defined to put the service into a warning status only if say more than 40% of the web servers in the farm go into a warning. Finally, the solution needs to allow users to drill down from a service status or dashboard view to the lower level device and test views, to first identify which component in the IT infrastructure is performing poorly, and by extension causing service degradation, and then go further into isolating the source of the problem by doing deeper analysis, such as viewing network flow data. Tying all of these high level capabilities together in one solution is critical to having a truly functional BSM environment. This allows you to quickly go from being informed of a service degradation problem, to identifying the problem component affecting the service, and then isolating the potential causes of the issues with the given component. 2

3 2. Giving Your Constituents Real-time Visibility to Business Service Health During a recent training engagement with a new customer, we received a somewhat frantic call from our customer contact. She sounded a bit stressed and said that our consultant would need to remain on site one more day for a follow on training session. The IT contact emphasized that this was not something she had planned for, nor foreseen, and it had come up quite suddenly. Indeed, the IT group had given a presentation to the company s executive team on their recently deployed network monitoring software. Upon seeing a demonstration of the top level Business Service dashboards that displayed key metrics on the performance of critical business services, the VP of Customer Care immediately requested that he wanted himself and his directors to be trained on accessing and customizing the dashboards. Interestingly, this is not the first time we have experienced this kind of a reaction from senior managers and business owners. Whether an internal IT organization supports the business or whether MSPs manage the IT infrastructure and applications, the business and management constituents want real time visibility on the performance of business services that are dependent on the underlying IT infrastructure. They are no longer content with after the fact reporting, and want direct access to the relevant data to view service performance and validate compliance against targets. The prevailing assumption has been that IT monitoring tools are not able to deliver actionable information that is relevant to business owners or senior managers. That is no longer the case. Opportunities and Challenges The demand for greater visibility presents both an opportunity and a challenge for IT organizations. Being able to provide timely and relevant performance information to business owners and senior managers, through mechanisms like service impact dashboards and real time status reports, deliver enhanced value to the business. On the other hand, organizations will need to make investments in the appropriate systems that enable providing a service oriented view of the IT infrastructure and applications. Monitoring software systems will need to have two primary underpinnings to meet the challenge. The first, and the more critical one, is having built in, pre integrated Business Service Management (BSM) capability that links the underlying IT infrastructure to business services. Within a fully integrated BSM environment, information is presented in a way that is relevant to the user roles within an organization. The business owner can access a rolled up dashboard view of the metrics on which the business services depend. The information in this view is described in business terms. An IT operations person can simultaneously view the detailed performance data plots for a given piece of the underlying IT infrastructure, where the data is defined in technical terms. To learn more about BSM solutions, read a BSM overview whitepaper. The second requirement is that the monitoring system needs to have full multi tenancy support. The application has to allow creation of readonly, read write and admin users within a domain, and admin users across domains. The ability to enable look and feel that is driven by the customer login (e.g. custom logo, role specific layouts, custom stylesheets, etc.) will be important to ensure an intuitive and high quality user experience for business users and senior managers. Given the appropriate tools, such as service health dashboards, IT organizations can enable business owners to proactively monitor/verify the status and trending of business service performance. To get started, IT organizations can learn more about some of the organizations that are embracing BSM solutions. With the right level of integration (and correlation) between the business service monitoring layer and the underlying network/it monitoring technology, business owners can better understand the impact of the complex mesh of underlying infrastructure on the specific outcomes of interest in their business services. IT organizations should consider adopting tools that deliver realtime business service assurance capability, and give business owners the visibility they desire. 3

4 3. Combining Bottom-up Analysis with Top-Down User Experience Metrics A recent survey conducted by Network World revealed that most IT managers were unable to measure end user experience with their traditional network monitoring software tools. Over 50 percent of the survey respondents identified page response time, server query response time and TCP transaction response time (key measures of end user experience) as being important, yet were not able to measure these metrics with their existing management tools. The survey highlighted a need for IT and network management software that is able to monitor the performance of IT from a user perspective (e.g. end user page response time), as well as monitor the performance of the various underlying network, server and application components that make up the layers of infrastructure that enable delivery of services. Although there are specialist solutions that support end user experience monitoring, these tools are generally not pre integrated with management tools that monitor the health of the underlying IT infrastructure. Having the linked top down and bottom up views and integrated capability within one IT monitoring system allows tracking service performance and user experience metrics, and then if problems are detected, the solutions facilitate drilling down to view and analyze the technical performance metrics for the various enabling components (e.g. CPU utilization of the application server). This capability allows rapid and context specific identification of potential causes of degradation of end user experience. Having unified, correlated, status views allows the IT team to not only better assure the real time user experience, but also conduct detailed analysis on areas of performance issues and bottlenecks in the underlying IT infrastructure. Organizations that are in the midst of exploring new network monitoring software solutions can look for the following types of capabilities to get an integrated view of performance. Does the solution monitor metrics, such as response time, for complete multi step end user transactions? Ideally, any number of multi step test transactions should be definable, where these tests can be monitored alongside the other device or server specific tests to generate alarms when thresholds are violated. As part of scripting a transaction step, the user should be able to select specific frames and links for navigating through a particular path for testing purposes. Additionally, secure pages should be accessible by providing the relevant authentication credentials. As part of the scripting process, when the user clicks through to the next step, the software needs to be capable of performing basic validation to ensure that the transaction being scripted can indeed be executed without application access errors. Combining transaction monitoring and infrastructure monitoring in one system, and then taking this one step further by mapping services to the relevant top down and bottomup metrics (see example of URL transaction monitoring), allows organizations to monitor service performance from both technical and end user perspectives. As the overall IT infrastructure becomes more dynamic and complex with adoption of new technologies such as virtualization and cloud, the ability to unify and tie infrastructure monitoring with end user experience monitoring will allow organizations to better assure overall business performance and customer satisfaction. 4

5 4. Connecting Your IT Monitoring Software with Web Services APIs & Plug-ins The history of Enterprise Software is riddled with examples of organizations never having realized the value of purchased solutions given the high cost and complexity of the integration hurdle. Based on past experiences and today s environment, MSPs and businesses want enterprise software solutions that can be made operational quickly, without dependence on lengthy integration or implementation projects. Even if this requires forgoing some of the advanced functionality promised by the more complicated solutions. Fortunately, most of today s network monitoring software systems can quickly, and fairly inexpensively, start performing basic infrastructure management for industry standard devices. Unlike, say a billing application, no complex integration with other enterprise systems, such as ordering or fulfillment, is required to get going out of the gate. That said, IT organizations need to be careful that the monitoring software does not become stranded on an island, and that the solution indeed does have the flexible APIs and interfaces to connect with custom data sources and enterprise applications, and link into other IT service management processes (read whitepaper on ITIL alignment). With a less complete tool, the immediate satisfaction from seeing metrics being gathered and alarms being displayed on status screens (for a relatively low starter price) can quickly give way to challenges further down the road as your business evolves. In a world where success is measured on a quarterly basis and where shorter horizons tend to favor decisions based on tactical factors, stepping back and taking a longer term, strategic view in selecting your network monitoring software will pay dividends. We are not suggesting that you discount the ability to quickly operationalize the software, as that is table stakes. But, we do recommend that you consider the ability of the software to adapt to and interact with your changing IT environment over time. Getting Started Here is an illustrative example of the importance of considering a broader set of factors in selecting your network monitoring software solution. A few years back, a newly launched MSP s immediate and somewhat moderate monitoring needs centered around ensuring the performance of the core IT infrastructure consisting of a number of switches and physical servers. Metrics needed to be captured and compared against thresholds, alarms were required to be displayed on an event management console, and notifications had to be ed to the operations technicians. Within three months of going live, the need arose to capture and process performance metrics from two custom applications. The MSP was able to utilize the monitoring software s universal, external data feed API to inject these metrics into the system, and then apply custom rules for event generation. The performance data and alarms were displayed alongside those of the other standard devices. As the MSP s business accelerated rapidly, the operations team quadrupled in size and the IT team implemented a centralized user management application. The monitoring software had the integration framework to override the inbuilt authentication, and was able to utilize the new external authentication database to control access to the system. Continued Evolution The MSP s service offerings continued to expand, and along with that the heterogeneity of the IT infrastructure increased. The built in action profiles in the monitoring system that are triggered when events occur were no longer adequate. These standard action profiles were augmented using the supported custom plug in framework, which was capable of running external programs as well. Device names and test information was easily passed to an external application to build highly flexible actions, some of which involved using the monitoring system s API to query the state of another device before executing a corrective action. Additionally, certain performance data needed to be fed into an external web portal application, which again was facilitated by the network monitoring software s data access API. Although the immediate monitoring needs of the MSP were pretty basic when the software purchase decision was made, the team had considered extensibility and integration support as part of their evaluation criteria. They recognized that these were important future requirements. Make sure your network monitoring software system is not stranded, and can indeed keep up with your evolving business and IT environment. 5

6 5. Making Sure Your IT Monitoring Solution is Built for Change Winston Churchill once said, To improve is to change. To be perfect is to change often. Whether the impetus for change is driven internally, or imposed on organizations by external factors, the reality is that business organizations are constantly changing and evolving. Whether the transformation is at a macro or micro level, the IT infrastructure will invariably be impacted given the critical role IT plays in enabling and executing business processes today. This clearly has implications on your network and IT monitoring software, given that your management tools have to be capable of assuring the effective performance of your infrastructure as changes occur. There are a number of factors that need to be accounted for to ensure that your IT monitoring and management systems are able to keep up as your business grows and evolves. Most importantly, you need to make sure you minimize the resource costs and lead times to keep pace. To begin with, your monitoring software platform has to be agnostic to the type of performance data being gathered and analyzed, and have mechanisms to capture data from a variety of sources. Although the initial deployment may be focused on monitoring the health of a say set of Windows servers using WMI, a need may arise down the road to capture metrics from a security appliance via SNMP Traps, and at some point processing of system logs for a new custom application may become necessary. Common event management, notification, visualization, dashboard and analysis capabilities will also enable seamless inclusion of new IT components. The monitoring software should provide intuitive UIs and workflows for quickly supporting additional components as the IT infrastructure expands. The ability to define and use templates, leverage pre existing monitoring profiles, as well as clone current configurations, all shorten the process of dealing with expansion, whether in your infrastructure or that of managed services customers. Having the facility to centrally handle configurations and push these out, or restore them, further simplifies and streamlines your administration processes within a dynamic infrastructure. Supporting flexible rules and thresholds for generation of events / alarms is becoming increasingly important. As business needs may vary going forward, the criteria, conditions and context for what should be considered a performance degradation warning versus a critical actionable event may change. A one size fits all approach that was okay early on may no longer be tenable, especially if your customer base is expected to become more heterogeneous in terms of size, characteristics or geography. Assuring the performance of IT infrastructure is a key element of delivering quality, reliable and value rich managed services. Given that business change is inevitable, and the knock on effects on the IT infrastructure are unavoidable, you need to make sure your IT and network monitoring software is built for change. 6

7 6. Taking Steps to Ensure Seamless Support for IPv6 and IPv4 In the 30+ year history of the Internet, the move to IPv6 will be the largest single upgrade. The clock is ticking on the availability of IPv4 addresses, and experts say IPv4 addresses have either run out or will run out soon. Many organizations are putting concrete plans in place to complete the migration over the next few years, and the awareness and activity on the IPv6 front has increased significantly. In June 2011, hundreds of governmental organizations, enterprises and service providers participated in a 24 hour, large scale test flight of IPv6 technology. The event was coined as World IPv6 Day, and was organized by the Internet Society. The purpose of the event was to energize, educate and motivate organizations across the IT and communications industry to prepare their services for IPv6 to enable a successful migration as IPv4 addresses begin running out. While much of the current focus on the migration to IPv6 is around the intricacies of making external facing websites and services (e.g. DNS) work cleanly in a hybrid world, as well as the use of IP addresses to interconnect distributed server, storage and network elements, organizations need to also be thinking about internal controls, management systems and frameworks as part of the transition. A key aspect of transitioning to IPv6 technology involves ensuring that the right IT, cloud and network monitoring software systems are in place to assure the performance of complex networks, data centers and cloud infrastructures. For distributed organizations, where services may be tied to partner or remote IT infrastructure, the preparation to deal with a hybrid IPv4 and IPv6 world has to be done much more proactively. In some cases, the IT services being managed by an IT group in one department may link to data center components and applications of other departments, which could be using different IP versions. If the IT organization is on the hook to deliver against agreed to SLAs or performance levels to users and business constituents, then it needs to have visibility into the health and performance of the broader IT infrastructure that is part of its scope of coverage. It is time to start taking steps to trial and implement network and IT monitoring software systems that can seamlessly monitor IPv6 and IPv4 applications, servers and network devices in a hybrid environment. Given that hybrid environments will coexist for a while, these monitoring solutions will enable organizations to uniformly discover and provision IPv6 devices, and collect and analyze performance data, all within one integrated system that supports IPv4 devices as well. Users can ignore the intricacies of managing different types of devices, and are able to benefit from a unified management and operational view of their entire IT infrastructure. Being able to capture performance metrics from the full IT and cloud infrastructure, and then correlating the data and linking this to supported business services is critical to ensure the effective delivery of services and assure business operations in the new dynamic environment. These systems address this need by providing a service oriented, end to end, performance view, whether IPv6 based or otherwise. Although your organization may be taking preliminary steps towards implementing IPv6 compatible infrastructure, being prepared in advance by having the management tools in place will ease the process as you make the transition from an all IPv4 to a hybrid to a fully converted environment. 7

8 7. Making Certain You Don t Have Scalability & Data Processing Bottlenecks The enterprise IT environment is continuing to experience significant changes. An organization s network monitoring software solution has to be capable of supporting future requirements, whether it is growth in the volume of monitored components, new custom applications/devices that need to be monitored, or different use models. If you are in the midst of considering an upgrade from your open source or point monitoring tools, or replacing an inflexible legacy solution, make sure that whatever solution you are evaluating is scalable, open and extensible. A key limitation of traditional network management systems is the existence of a centralized database for processing of performance data. Even if the collection of data is managed by distributed components, the solutions invariably require centralization of the data for processing and alert generation. For large infrastructures, this introduces a significant performance bottleneck. The multiplier effect of the amount of data that needs to be processed as new devices are added is enormous. Capturing and processing these metrics in a single centralized database will put immense pressure on the overall application, creating a significant performance challenges. A key consideration in a replacement solution is validating that it is indeed based on a distributed architecture that does not have centralized database bottlenecks. For example, some solutions will have both distributed collection capability and a distributed database architecture. In these solutions, individual data gathering components will often have small local databases that are able to process tens of thousands of metrics every few minutes to generate alarms as needed, and also store the data locally for multiple years. Monitoring consoles receive notifications as they occur, and are able to retrieve performance data from these separate databases when needed for analysis and reporting. No sophisticated database scaling or specialized database administration expertise is required for these systems. It also important for the management system to have separate and independent application components, to increase overall resiliency of the solution as well as enable more efficient scaling of the overall deployment. In a proven solution approach, the data collection and alarm/event generation components are able to operate independently of the configuration database component. If for some reason the configuration database goes offline, the data processing remains uninterrupted, and warning or critical alarms continue to get generated. Comprehensive monitoring functionality in and of itself is not sufficient. You need to make sure that whatever solution you select adheres to the basic architecture tenants of scalability, availability, openness, flexibility and extensibility (see complete list of key requirements). As your IT infrastructure expands and your monitoring requirements evolve, make sure your network monitoring software system can scale as you move forward. 8

9 8. Including Your Remote Sites for an Organization-wide Performance View Given the dispersed nature of today s organizations, with mobile workers and regional offices, the data center and IT infrastructure in reality extends beyond the boundaries of one or more centralized physical locations. What this means is that the operations team will be required to monitor, from a central NOC location, the performance of core IT infrastructure at remote sites and offices. The IT infrastructure, devices and applications being monitored will in many situations be behind firewalls, and in most cases, behind NAT enabled routers. Examples of remote monitoring may include the NOC being responsible for monitoring the execution of daily automated server back up jobs, amongst other scheduled jobs at the site. The monitoring software will need to generate an alert in the event the back up job did not execute properly. Additionally, it may be necessary to monitor site specific applications and servers, such as a local dispatch application, through querying core performance metrics or executing synthetic user transactions and monitoring their responses. In order to address these requirement, there are some key remote site capabilities that need to be available in the network monitoring software. The ability to gather metrics securely from behind a firewall is critical. What this means is that the monitoring solution has to include easily deployable and low cost remote data gathering components that are able to process traps/syslogs/eventlogs and execute scripts locally against monitored devices and applications within the secure remote network. The remote module has to be capable of pushing the data to an upstream event management system via SSL, and not require inbound requests. Another challenge that the network monitoring software will have to deal with is that of the remote sites and office networks having overlapping or duplicated IP ranges. It s extremely likely that many of the remote sites are using some parts of the x.x network. The monitoring solution has to account for this scenario, and uniquely identify site specific devices without requiring the re addressing of the networks just so that they'll be easier for the IT operations team to monitor and manage. The ability to monitor and manage secure remote sites is becoming a key requirement for distributed organizations. Make sure your network monitoring software supports remote site monitoring behind firewalls and with over lapping IP addresses, and includes coverage for a wide range of network and IT devices to ensure the smooth running of your business operations. 9

10 About Zyrion Zyrion provides a breakthrough Business Service Management (BSM) and Cloud Performance Monitoring solution for mission critical, distributed, and complex environments. Zyrion s flagship Traverse software system delivers real time, correlated, end to end, service oriented views of the performance of the entire IT infrastructure physical, virtual and cloud. Traverse s massively scalable, patented solution architecture supports tens of thousands of distributed end points, and processes millions of metrics. The software s innovative Service Container technology supports creation of purpose specific, logical management views of business services and the underlying Cloud and IT infrastructure. Traverse is fully aligned with ITIL and provides an open, extensible API and plug in framework for integration with the enterprise ecosystem. Zyrion s fast expanding customer base includes Cisco Systems, Ancestry.com, Yale University, USPS and the US Army, amongst others. Zyrion delivers rapid ROI to the business compared to legacy IT monitoring software tools. The company has its corporate offices in Sunnyvale, California. For more information, visit or call ZYRION. 10