Unified Infrastructure Monitoring and Management Increases Availability, MTTR, and IT Staff Productivity

Transcription

1 WHITE PAPER Unified Infrastructure Monitoring and Management Increases Availability, MTTR, and IT Staff Productivity Sponsored by: CA Technologies Mary Johnston Turner October 2013 IDC OPINION In today's complex, virtualized, and hybrid cloud datacenters, consistent service delivery and cost-effective operations depend on the ability of IT staff to quickly identify the root cause of problems and collaborate efficiently across technology silos. Unified and integrated infrastructure monitoring solutions that look across systems, networks, and applications using a consistent navigation and reporting interface, a rules database, and a correlation engine are important enablers of proactive and efficient IT management teams. The experiences of five CA Infrastructure Management and CA Nimsoft Monitor customers show that significant productivity and service-level and cost-saving benefits can be achieved by shifting from reliance on fragmented point solutions toward the use of a unified, centralized approach to infrastructure monitoring and analysis. Customers report compelling benefits such as: IT staff productivity improvements on the order of four times to six times because of the use of a unified solution to monitor systems and networks and quickly assess and remediate the root cause of problems Mean time to repair (MTTR) improvements of 25%+ because of improved root cause analysis and better collaboration and communication across specialized IT technical domains System and network outages reduced by 20%+ because of more proactive management and monitoring Hard dollar savings of $250,000+ from retirement of older point solution products, including elimination of software licensing fees and shifting to more efficient, centralized supporting infrastructure or software-as-a-service (SaaS)-enabled services IN THIS WHITE PAPER This IDC white paper describes important infrastructure monitoring and management challenges facing IT managers today. It discusses how IT operations teams can improve end-to-end infrastructure performance and availability and work more efficiently by taking advantage of integrated infrastructure monitoring and analytics solutions. Interviews with five CA Technologies customers illustrate how CA Infrastructure Management and CA Nimsoft Monitor solutions are helping a wide range of organizations achieve measurable, long-term improvements to service levels and IT staff productivity. Note: All numbers in the document may not be exact due to rounding. October 2013, IDC #243957

2 SITUATION OVERVIEW IT organizations of all sizes and across all industries are forced to become more efficient and productive every day even as the scale, scope, and complexity of their corporate IT environments continue to grow and end-user expectations for end-to-end service availability and performance continue to rise. The typical enterprise-class IT organization is called upon to monitor and manage physical and virtual servers, applications, networks, databases, and storage resources while supporting end users that expect anywhere, anytime access using a wide range of desktops, laptops, and smart mobile devices. Simultaneously, application architectures are becoming more distributed and multitier, and organizations are taking advantage of on-demand public cloud services to supplement in-house datacenter resources and to speed end-user access to new applications and infrastructure. For IT teams that are responsible for providing consistent end-to-end service levels, the distributed, virtualized complexity of today's operational IT environments can create major challenges when it comes to correlating monitoring data, identifying the root cause of problems, collaborating across specialized technology silos, and ensuring end-user satisfaction. Traditional datacenter management strategies that rely on dedicated monitoring tools for each system and layer in the environment are proving to be highly inefficient for today's integrated and virtualized enterprises. IT staff can no longer operate effectively using siloed monitoring and management tools. They increasingly search for better, more unified ways to ensure service levels across their complex and dynamic infrastructure environments. IDC's research indicates that many IT decision makers are looking for ways to simplify and streamline IT management processes and tools in order to create a more productive and efficient IT management situation. This need for increased productivity and operational efficiency dominates current IT management software priorities identified in a recent IDC IT Management QuickPoll Survey. Specifically, as shown in Figure 1, when asked to identify their top 3 IT management software priorities for the next three years, IT decision makers emphasized: Improving IT staff productivity and operational efficiency (48.8%) Implementing common management software, services, and processes across cloud and noncloud environments (40.3%) Reducing the number of management software vendors and tools used across their environment (39.3%) 2013 IDC #

3 FIGURE 1 Top 5 IT Management Software Priorities for the Next Three Years Improve IT staff productivity and operational efficiency Implement common management software, services, and processes across cloud and noncloud environments Reduce number of management software vendors and tools Aggressively invest in automation and orchestration Shift from on-premise to SaaS-based management (% of respondents) n = 201 Note: Multiple selections were permitted. Source: IDC's IT Management QuickPoll Survey, January 2013 Simplified, streamlined monitoring and management software tools and processes are needed to enable more flexible, agile, and efficient IT operations teams. IT organizations need more consistent and efficient operational processes and higher end-to-end service levels to keep up with the needs of the business and maintain today's complex IT environments. IT decision makers told IDC that when they select new management and monitoring tools, they care most about cost, impact on IT staff efficiency, and ease of integration across IT management processes and tools, including emerging cloud resources (see Figure 2). Specifically, survey participants indicated that they will focus their selection criteria on: Total cost of ownership (57.7%) Impact on IT staff efficiency (47.8%) Ease of integration with existing systems management processes and tools (38.3%) Ability to use the same tools to manage cloud and noncloud environments (37.8%) 2013 IDC #

4 FIGURE 2 Top 5 Selection Criteria for New IT Management Software Purchases Over the Next Three Years Total cost of ownership 57.7 Impact on IT staff efficiency 47.8 Ease of integration with existing systems management software products and processes Ability to manage public, private, and noncloud resources using same tools Ability to span heterogeneous hardware and virtualization platforms (% of respondents) n = 201 Note: Multiple selections were permitted. Source: IDC's IT Management QuickPoll Survey, January 2013 Unified infrastructure monitoring solutions that can be quickly deployed across a broad range of systems and networks using consistent interfaces, reporting, and root cause analytics are top priorities for IT teams that want to streamline operational environments, improve service levels, and implement more agile and efficient processes. IDC's conversations with IT teams that have successfully implemented these types of integrated monitoring solutions from CA Technologies show that the benefits can be significant. Typical examples of improvements seen in the first few years of deployment include: Productivity improvements on the order of four times to six times because of the use of streamlined, centralized monitoring processes and staff using a unified solution to monitor systems and networks and quickly assess and remediate the root cause of problems; reduced training and support functions and improved collaboration across teams also contribute to productivity improvements The ability to increase the number of monitored devices by three times or more, while reducing the number of staff FTEs needed to support day-to-day monitoring and related activities MTTR improvements of 25%+ because of improved root cause analysis and better collaboration and communication 2013 IDC #

5 System and network outages reduced by 20%+ because of more proactive management and monitoring Number of daily alarms reduced by 60%+ because of better root cause analysis Hard dollar savings of $250,000+ resulting from retirement of older point solution products, including elimination of software licensing fees and shifting to more efficient, centralized supporting infrastructure or SaaS-enabled services CA Technologies' Service Assurance portfolio targets enterprise and service provider IT operations staff that want to simplify and streamline operations, improve IT staff productivity, and increase overall service levels on an end-to-end basis. The sections that follow provide an overview of CA Technologies' capabilities and describe how these unified solutions have made a difference across a range of customers and datacenter environments. CA Infrastructure Monitoring Portfolio Overview CA Technologies offers customers two distinct solutions for unifying and improving the productivity of their infrastructure monitoring environments: CA Infrastructure Management: An on-premise software solution that provides unified access across CA Performance Management, CA Network Flow Analysis, and CA Spectrum to enable deep-dive modeling, reporting, and analytics for specialist IT teams such as NOC personnel CA Nimsoft Monitor: Available as both on-premise software and SaaS that provides a lighter-weight unified infrastructure monitoring and reporting solution for generalist IT teams The agent-based architecture of CA Infrastructure Management enables IT teams to collect fine-grained performance and availability information across a wide range of heterogeneous systems and network devices. The recently introduced graphical user interface facilitates rapid, deep-dive correlation and assessment across networks and systems and provides the ability to customize views and directly link to trouble ticket and service desk systems and workflows to speed remediation, collaboration, and reporting. CA Infrastructure Management allows technology specialists to gain deep-dive access to detailed reporting and analysis while also supporting data sharing, correlation, and collaboration across technology silos. The most important attributes of CA Infrastructure Management include: Detailed end-to-end network-centric monitoring and analytics Rapid discovery, relationship modeling, correlation, visualization, and root cause analysis Net and application flow analysis Application response time monitoring Patented, metric-driven views and analysis Guided workflows to triage alarms and guide remediation In-depth rules and knowledge library certified for hundreds of devices and vendors down to the component level 2013 IDC #

6 Integration with wide range of third-party systems and management tools to provide operators with a unified view of infrastructure performance across the datacenter Ability to self-customize the portal and reports for specific roles and needs across NOC specialists and subject matter experts CA Nimsoft Monitor offers a simplified set of infrastructure monitoring, analysis, and reporting capabilities covering a broad range of server, storage, cloud, and database resources. Available as both on-premise and SaaS solutions, CA Nimsoft Monitor is often chosen by customers that want to be able to deploy rapidly and have access to a seamless monitoring experience built on a single code base to easily correlate data across the full infrastructure technology stack. Important CA Nimsoft Monitor capabilities include: Preconfigured reports and rapid time to value for IT management generalists Robust library of prebuilt probes to gather system-level data from a wide range of devices Gateways and APIs that enable integrations with service desks, CMDBs, and provisioning tools DCIM power and cooling visibility Tight integration with CA Nimsoft Service Desk High-level packaged application performance monitoring and synthetic transaction monitoring SaaS and on-premise options for usage metering and billing capabilities to support a range of service provider business models and internal enterprise IT service delivery strategies Interviews with both CA Infrastructure Management and CA Nimsoft Monitor customers demonstrate the productivity and agility benefits of adopting a unified approach to infrastructure monitoring, analysis, and remediation. The section that follows provides insight into the types of benefits experienced by five CA customers in recent years. Customer Experiences with CA Technologies' Unified Infrastructure Monitoring Solutions Regional Health Insurance Provider Increases IT Productivity by Four Times Using CA Infrastructure Management A 6,000-employee regional health insurance provider depends on CA Infrastructure Management to monitor over 4,000 physical and virtual devices including servers, virtual machines (VMs), routers, network switches, and converged infrastructure systems. Prior to implementing CA Infrastructure Management, the company relied on more than 75 different monitoring tools that were managed and maintained by a number of different teams and specialists to monitor different technologies. The company found this situation to be highly inefficient as alarms, ticketing systems, diagnostics, and remediation workflows were disconnected, and individual teams and administrators had little or no visibility into root cause and cross-silo dependencies. Often, one team would quickly fix the source of an alarm even as other teams continued to troubleshoot alerts set off by the same root cause IDC #

7 The operational challenges related to this fragmented monitoring and communications environment worsened as the organization grew. The move to a new campus provided an opportunity for the IT infrastructure team to implement a more centralized approach to monitoring and remediation activities with the launch of a 24 x 7 enterprise command center chartered to correlate alarms across systems and to manage cross-functional troubleshooting, remediation, and communications processes. CA Spectrum and CA ehealth Infrastructure Management products were initially selected as the central monitoring tools for this new organization. CA Network Flow Analysis was added to the mix 18 months later. Today, the firm is able to quickly and proactively monitor, correlate, and remediate network, system, and application alarms using the unified CA Infrastructure Management solution. As individual technology specialists began to understand the value of a centralized monitoring function, many agreed to abandon older point solution tools in favor of a more centralized and actionable system that provides specialists with access to the information they need but frees them from having to maintain dedicated, specialized tools. They quickly saw that a central system would help improve MTTR as a result of better communication and coordination across teams. In the first 18 months, roughly half of the legacy, specialized monitoring products were retired. The shift to centralized, unified infrastructure monitoring, combined with the elimination of underutilized tools, enabled the company to reduce IT staff time devoted to monitoring by three FTEs. The dedicated central monitoring staff was able to improve productivity by a factor of four times, even as the scale and complexity of the environment exploded. As an example, the number of monitored devices grew from fewer than 1,000 to over 4,000, and the number of alarms shrank from over 500,000 to fewer than 100,000 a year because CA Infrastructure Management allowed the team to better correlate alarms and identify the root cause of problems while providing trend analysis to help set more meaningful and actionable alarm thresholds. Across the organization, CA Infrastructure Management is credited with improving the day-to-day productivity of as many as 30 individuals because of reductions in the number of conference calls that involve managers from server, network, security, and related datacenter operations teams. Mean time to problem resolution for infrastructure issues improved 25% in the first year. Network and server outages were reduced by 20% as a result of more proactive monitoring and predictive analytics. Overall, the monitoring centralization effort enabled by the organization's investment in CA Infrastructure Management met its three-year ROI objectives in just 18 months. The customer noted that CA's solution was easy to install and began to deliver value in just days because of built-in integrations and out-of-the-box graphical reporting views. Global Retail Distributor Improves MTTR by 25% and Reduces Alarms by Over 60% with CA Infrastructure Management A longtime user of several CA monitoring products, this global retail distributor began a significant integration effort to unify its approach to monitoring using CA Infrastructure Management and CA Service Operations Insight beginning in The company has been able to improve mean time to repair and monitoring staff productivity by at least 25% since it began to unify its systems and network infrastructure monitoring and event management tools and processes. Rather than rely on staff to manually correlate information from separate monitoring products, the firm adopted a more integrated approach that enables the team to shift its focus toward delivering consistent end-to-end service levels IDC #

8 The most immediate benefits were seen in terms of reducing the number of alarms and enabling faster root cause analysis and speedier MTTR. With more than 3 million monitored events a day, the customer typically generated as many as 3,000 alarms daily before deploying CA Infrastructure Management. Today, the number of alarms has been reduced over 60%, typically totaling 1,000-1,200 on a given day. The team also reports achieving an estimated 25% improvement in the time needed to identify the root cause of problems and a 25% reduction in MTTR. The ability to quickly assess the true source of a problem empowers the team and allows much faster response to critical issues, thereby improving service levels and end-user satisfaction. As with many IT organizations, this global retail distributor recognizes that effective management in today's increasingly complex, virtualized environments requires making integrated monitoring and remediation processes a priority. U.S. Datacenter Provider Doubles Productivity with CA Nimsoft Monitor A major U.S. datacenter provider uses the on-premise edition of CA Nimsoft Monitor to keep tabs on more than 3,000 devices located across 10 geographically dispersed datacenters. CA Nimsoft Monitor replaced five different monitoring systems when it was deployed three years ago and quickly enabled the IT staff to better analyze and correlate alarms and find root cause twice as fast as had been possible previously. The implementation has been live since November The monitored environment has more than tripled in size since CA Nimsoft Monitor was originally deployed, yet the customer still requires just two monitoring system administrators, a 50% reduction from the four FTEs assigned to monitoring tasks prior to the CA Nimsoft Monitor implementation. CA Nimsoft Monitor is the company's primary server and network infrastructure monitoring solution and is used to track server and network usage and notify customers when they are reaching utilization thresholds. As the company has added managed cloud and self-service cloud hosting services to its portfolio, CA Nimsoft Monitor has provided important service differentiation when exposed directly to customers as part of the provider's self-service portal. The company has also integrated CA Nimsoft Monitor with its building management systems to unify monitoring of datacenter facility infrastructure. The company initially selected CA Nimsoft Monitor after comparing functionality and pricing across several alternatives. The leadership team concluded that CA Nimsoft Monitor functionality was equal to or better than that of most of the competition, but it was easier to maintain and required less staffing and overhead to support. In addition to saving on staffing requirements, the company estimated that it had saved at least $250,000 on monitoring-related spending for back-end servers, databases, and related infrastructure and middleware hardware and software licenses. Training costs have been reduced as well since staff members now need to master only one system rather than the five separate tools that were previously in place. The firm initially signed a four-year agreement with CA Technologies, but as it comes to the end of this initial contract, the customer expects to extend the agreement as it continues to add capacity and support for new converged and integrated infrastructure platforms and cloud service delivery functionality IDC #

9 Regional Bank Improves Monitoring Staff Productivity Four Times with CA Nimsoft Monitor A growing regional U.S. bank relies on CA Nimsoft Monitor to monitor 2,000 physical and virtual servers located at two geographically distant datacenters and associated remote sites. With over 80% of these systems supporting production workloads, the bank recognizes the need for proactive, realtime monitoring solutions. The bank initially deployed CA Nimsoft Monitor on-premise software about four years ago to replace a complex legacy system that was difficult to maintain. Small tasks such as provisioning new agents or updating systems often required involvement from the datacenter engineering staff as well as the monitoring team. Correlation and reporting across different sets of data was difficult, and operation of the legacy product required extensive training. The bank reports that since the introduction of CA Nimsoft Monitor, it has been able to radically restructure the way the monitoring team works. The team is now much more efficient and self-sufficient, whereas five years ago, as much as 80% of the monitoring staff's time was spent on reactive troubleshooting and remediation tasks. Today, only 20% of the staff time is needed to deal with unexpected contingencies, and the remaining 80% of monitoring staff time is allocated to proactive projects that directly add value to the business. The bank attributes this stunning reversal in large part to the fact that CA Nimsoft Monitor provides a highly integrated set of monitoring capabilities and robust analytic and reporting tools. With one integrated monitoring solution, the team is able to track the health and availability of physical and virtual hosts, databases, back-end mail servers, and backup servers. The team has found it easy and efficient to add new resources to the system using the CA Nimsoft Monitor drag-and-drop graphical user interface. The product's centralized data collection, analysis, and reporting capabilities streamline many formerly fragmented and time-consuming processes and allow for efficient integration with service desk tools and workflows. Unified reporting has made it easier to share information with a wider array of IT operations staff and IT executives, leading to faster decision making and improved time to resolution. The monitoring team reports that since the move to CA Nimsoft Monitor, it has been able to correct many problems independently and almost instantly, eliminating the need for formal work requests and trouble tickets involving other groups. When tickets are opened, shared access to centralized CA Nimsoft Monitor dashboards allows for much better coordination and communication among everyone involved. The team expects to add real user transaction monitoring to its infrastructure monitoring capabilities in order to better support Web and mobile applications. CA Nimsoft Monitor SaaS Helps U.S. Cloud Service Provider Activate New Customers Quickly When planning to launch cloud application hosting and disaster recovery services in 2012, a Texasbased cloud service provider knew that monitoring solutions would need to scale quickly across diverse customer environments as more customers signed on. After an extensive evaluation of both on-premise and software-as-a-service options, the firm chose the CA Nimsoft Monitor SaaS solution to provide visibility into the health and availability of more than 1,000 VMs, physical servers, and network devices supporting over 40 healthcare customers across the United States IDC #

10 With limited staff available and business plans calling for the new cloud service offerings to grow quickly, the firm determined that a SaaS-based monitoring solution would be best suited to the rapid growth it expected to experience. Rather than buying and managing back-end monitoring servers and databases and devoting half an IT admin FTE (or more) to maintain the system, the company decided that a SaaS solution would prove to be more flexible and scalable over time. CA Nimsoft Monitor was selected for a number of reasons, including its affordability and its ability to be quickly deployed across a wide range of infrastructure environments and customer accounts. The service provider reports that it usually deploys CA Nimsoft Monitor to a customer's site in less than a day using predefined configurations. For a customer that may not have had any existing monitoring tools, the service provider is able to deliver immediate value by quickly implementing CA Nimsoft Monitor as an integrated component of its customer portal. The CA Nimsoft Monitor SaaS solution scales on demand with no need to worry whether back-end systems or databases can handle the load. The cloud service provider expects to increase workflow integrations with CA Nimsoft Service Desk and to add application monitoring to existing infrastructure monitoring capabilities. FUTURE OUTLOOK IDC's research indicates that enterprise and service provider IT infrastructure environments will continue to become more virtualized and embrace dynamic hybrid cloud architectures over the next several years. IT operations teams will continue to be challenged to maintain and improve end-to-end service levels while the diversity, scale, and scope of infrastructure and services continue to escalate. To improve operational efficiency and hold head count and staff costs in line with budgetary expectations, IT decision makers will need to transform the way they monitor, manage, and optimize their complex, hybrid datacenter and cloud service environments. Integrated and unified monitoring solutions that look across systems and networks and allow a wide range of technology specialists to collaborate and troubleshoot effectively will be critical enablers of efficient and agile IT operations teams. When evaluating these types of unified monitoring solutions, IT decision makers should look for the following capabilities: Rapid deployment and automatic discovery to speed time to value and simplify ongoing monitoring activities and analysis Robust cross-domain correlation and root cause analytics to reduce MTTR and improve IT staff productivity Unified, graphical, real-time service-level dashboards capable of providing role-based views and enabling collaboration across IT silos and technical specialists Proactive performance analytics to detect problems before they impact service levels Preconfigured reports and rapid time to value for IT management generalists as well as ability to develop specialized reports if needed by individual specialists 2013 IDC #

11 IT teams that currently rely on specialized, disconnected infrastructure monitoring approaches can expect to see higher service levels, fewer outages, faster mean time to repair, and improved IT staff productivity by moving to more integrated, unified, and collaborative infrastructure monitoring solutions and processes. CHALLENGES/OPPORTUNITIES In large IT organizations, transformation and changes to the status quo can take time. CA Technologies customers interviewed by IDC agreed that effective use of integrated infrastructure management, monitoring, and analysis tools depends on the organization's willingness to streamline workflows, centralize responsibilities, standardize configurations, and stay focused on delivering results. Most teams reported that it took anywhere from 6 months to 12 months, or more, to demonstrate the value of centralized and unified infrastructure monitoring to specialist teams that historically relied on their own, dedicated monitoring tools. Over time, however, the elimination of many of these point tools helps improve operational productivity and free many IT staff to focus more on business value and less on fighting fires. Like many IT management software vendors, CA Technologies needs to help its customers effect cultural and process changes at the same time that they update, expand, and integrate infrastructure management solutions to accommodate increasingly complex network, system, and application monitoring requirements and dependencies. Customers that want to get the most benefit out of their investment in CA Infrastructure Management tools should plan to tackle cultural, process, and workflow integration activities on a coordinated basis at the same time they deploy more unified and integrated monitoring solutions. CONCLUSION The experience of CA customers validates the benefits of integrated infrastructure monitoring processes and tools and documents how they can significantly improve the productivity of IT infrastructure management teams while increasing service levels and helping IT staff devote greater amounts of time and energy to proactive value-add activities rather than reactive troubleshooting and remediation tasks. As IT organizations lay out plans for continued investments in virtualization and cloud computing, they need to keep in mind how these strategies and architectures will accelerate the need for more unified infrastructure monitoring, analytics, and service-centric approaches to IT management and datacenter operations. Without adequate, integrated monitoring resources and processes, service levels will suffer while the cost of IT operations spirals upward IDC #

12 About IDC International Data Corporation (IDC) is the premier global provider of market intelligence, advisory services, and events for the information technology, telecommunications and consumer technology markets. IDC helps IT professionals, business executives, and the investment community make factbased decisions on technology purchases and business strategy. More than 1000 IDC analysts provide global, regional, and local expertise on technology and industry opportunities and trends in over 110 countries worldwide. For more than 48 years, IDC has provided strategic insights to help our clients achieve their key business objectives. IDC is a subsidiary of IDG, the world's leading technology media, research, and events company. Global Headquarters 5 Speen Street Framingham, MA USA idc-insights-community.com Copyright Notice External Publication of IDC Information and Data Any IDC information that is to be used in advertising, press releases, or promotional materials requires prior written approval from the appropriate IDC Vice President or Country Manager. A draft of the proposed document should accompany any such request. IDC reserves the right to deny approval of external usage for any reason. Copyright 2013 IDC. Reproduction without written permission is completely forbidden.