White White Paper Paper Drive Down IT Operations Cost with Multi-Level Automation Overview Reducing IT infrastructure and operations (I+O) budgets is as much on the mind of CIOs today as it s ever been. Recent research suggests that CIOs spend 54% of their time focused on improvements in, and cost reduction of, data centers, systems, infrastructure and operations. Improving support processes through automation can have a great impact on reducing staff costs, a vital objective given that those costs represent close to 30% of the typical IT budget. For example, automation of routine tasks not only reduces the number of level 1 support or service desk personnel, but also can eliminate potential problems and improve MTRS (mean time restore service) of complex issues from days to hours. Greater cost reductions can be realized by applying automation throughout the support process from incident management to workflow and problem management. A multi-level automation strategy reduces personnel costs, improves service levels and decreases recurring incidents through a continuous improvement model. Multi-Level Automation Strategy A multi-level automation strategy should include both incident management and problem management areas so that proactive solutions can be applied before incidents become tickets or cases. There are five areas to consider: Automating root cause discovery Repetitive task automation Context-specific automation Proactive automation strategy Workflow automation Root Cause Automation and Discovery Of the hundreds or thousands of potential events hitting your management system, there s only a small fraction that represent the true root cause of an incident. But filtering out extraneous data to identify the one meaningful event is a time-consuming process. As the alerts, traps and alarms are presented in volume, a level 1 engineer acts upon or closes each one (or just ignores them). Applying automation to close tickets at this point may seem to reduce the work of the level 1 engineer, but because it ignores root cause discovery the incident may continue to occur, resulting in more effort and additional support cost further into the resolution cycle. Automation technology I&O represents approximately 60% of total IT spending worldwide, so with IT budgets remaining tight, it's no wonder that I&O cost-cutting pressure continues to be intense." Gartner Group
that systematically analyzes and presents that one meaningful incident reduces the number of level 1 staff required to uncover the problem and expedites resolution. Automating Repetitive Tasks There are many incidents that occur where resolution is relatively simple rebooting a server, for example. Done manually, this process may take 15-20 minutes from ticket generation to ticket close, including time spent logging in to the server, running the command, waiting for the server to come up and confirming service is restored. The reason for automating repetitive tasks after determining root cause is simple the system takes action on only the one causal issue, not every issue. Following root cause determination, task automation processes can, and should, run without human intervention. But you want to make sure that the automation is non-intrusive and that it is working within your standard operating procedures for IT support. An intelligent automation platform will know when automation can be applied in compliance with your policies and when it must be avoided to eliminate the potential of a further negative business impact. Complex Automation Complex automation can accelerate troubleshooting by removing many of the manual processes an engineer would do to solve a more complex problem. This type of automation is created based on best practices for resolving complex issues and may involve multiple automated processes running serially or in parallel to one another. For example, to troubleshoot an issue the engineer may need to establish a network connection, access a system, log in, locate log files, download them, save them to a case, and only then start to work on the resolution. Automating these and other manual processes tied to incident research shortens the time to resolution because, when the engineer gets the case, he/she immediately begins to solve the issue, not start researching. Proactive Automation Strategy Automating incident management processes and tasks will help reduce IT operations personnel costs and shorten MTTR (mean time to resolve). But there are ways automation can help your problem management initiatives as well. To be truly proactive, you need to understand trends, make determinations about these trends and then put an action plan in place. This process typically requires having a comprehensive root cause database, extracting this data into a spreadsheet or other tool and painstakingly analyzing the information to identify problems. Deploying a solution that uses automation to correlate relevant data points and then automatically and continually present like incidents in intuitive formats reduces research time from hours to minutes. Resolution may be completed through automation or, in the case where upgrades or software updates must be applied, complete information is sent to your support team for resolution. 2
Workflow Automation Although workflow automation does not by itself resolve incidents or problems, it can streamline their handling and drive workgroup productivity. When workflow automation is integrated with the incident and problem management processes, support optimization and policy compliance is accomplished through a single consolidated platform. Organizations then begin to benefit from continual service level improvements. For example, once a root cause is identified, the ability to automatically generate a case or ticket eliminates the time it would take for support personnel to do so manually. Then, automated processes which assign the case to the correct workgroup for ownership, update priority settings based on the latest device status and manage policy-based notification and escalation processes will further contribute to productivity gains and accelerate the resolution process. In some environments, fully automated workflow processes can contribute to additional cost savings by eliminating the need to employ support personnel to open and manage cases outside of normal business hours. CaseSentry the Intelligent Management Platform CaseSentry is a SaaS-based IT management automation platform designed to provide operational service assurance and actionable intelligence. It combines a modular, scalable architecture with a unique distributed approach, allowing support for the largest service providers down to small commercial organizations, thus making it a cost-effective management platform for any environment. Customers who have deployed CaseSentry have seen support costs decline while realizing dramatically increased service levels through our continuous improvement approach to automating incident and problem management. CaseSentry Multi-Level Automation Approach CaseSentry s automation starts with our patented process for identifying an incident s root cause. Compared with legacy and less-intelligent management tools, which may present hundreds or thousands of events (alerts, traps and alarms), CaseSentry analyzes all data collected and automatically filters irrelevant information to find the root cause of the event. This takes the pain and cost out of manual processes by correlating the information, applying a patented automated process, and only then presenting a single case requiring action. More than 94% of all cases are created through this automation, dramatically reducing the effort of manually analyzing thousands of potential incidents. Immediate visibility to the source of issues causing business impact also reduces the number of level 1 support staff required to field calls from the user community and the manual, time-consuming troubleshooting process that entails. With its ability to quickly identify root cause, customers who have installed CaseSentry have experienced over a 90% success rate with first-occurrence incident resolution, which has significantly lowered support costs associated with repeat occurrences. CaseSentry customers have experienced over a 90% success rate with firstoccurrence incident resolution 3
Second level automated processes are then applied to auto-remediate repetitive tasks 24x7 without human intervention. This process can mean a more than 75% reduction in response time, thereby significantly reducing MTTR and improving availability. CaseSentry s context-specific automation capabilities go into action to perform timesaving third-level steps, such as retrieving system data or generating additional tests, before presenting the details in an Autocase or dashboard with all relevant information to an engineer for further action. Automatically presenting all relevant information to a higher level engineer reduces discovery time from 15 minutes to 60 minutes per case (depending on circumstances) and decreases the cost of incident resolution by 40%. CaseSentry goes beyond reactive incident management by providing problem prevention and elimination of recurring incidents. CaseSentry presents historical incident and performance data in easy-to-use dashboards and reports to support problem management processes that quickly identify trends and apply appropriate fixes to stop new incidents from occurring. Cost Savings in Action A recent study by the Aberdeen Group looked at total cost of ownership (TCO) for unified communications. One component of TCO the study focuses on is systems management, an area where CaseSentry and automation can play a key role in minimizing outages and reducing personnel cost. For example, the study reveals that 5.45 FTEs are needed per 1,000 phones in a Cisco Unified Communications Manager environment, resulting in a total average personnel cost of $454,000. Our experience suggests that, by leveraging CaseSentry in this environment, the required FTE s could be reduced to 3.0, resulting in annualized savings of 52%, including the cost of CaseSentry. The following customer case studies illustrate actual costs savings realized from implementing CaseSentry. 4
Customer Case Study - Major Wireless Service Provider A major wireless provider, experiencing explosive business and subscriber growth, had presented two challenges to ShoreGroup: help them get a better handle on network and contact center issues and find a way, through automation, to keep their support costs from climbing even as devices under management continued to expand. Upon first deploying CaseSentry, the provider saw a decrease in number of events from 30,000/month (a rate that the current support staff could not handle) to under 500 valid, root cause events per month. The implementation of CaseSentry had an immediate and dramatic impact on the wireless provider s environmental stability as the noise was filtered out and only valid and verified root cause events were allowed to generate trouble tickets. Implementing CaseSentry had a direct effect on reducing subscriber churn rate and potential revenue loss while increasing customer satisfaction. With CaseSentry s multi-phased automation, resources were immediately applied to the precise source of issues, resulting in reduced business downtime, better customer service and more efficient use of staff. And, after five years of using CaseSentry, the provider was able to reduce planned staffing by 42% while devices under management grew more than 50%. Calculated Results: After deployment of CaseSentry, the customer was able to reduce planned staffing by 42% while devices under management grew more than 50%. Increase in customer satisfaction; reduction in churn rate Estimated revenue savings of over $10,000,000 per year 50% growth in their environment 42% fewer resources required to manage it Resulting in savings of $2,900,000 5
Customer Case Study - Fast Growing Specialty Retailer Rolling out new technology to over 800 stores in 12 months is a very aggressive schedule. This retailer was concerned about the stability, security and reliability of the new wide area network (WAN) being deployed. It selected ShoreGroup s CaseSentry to automate management and monitoring of WAN traffic and security events, and also to analyze Payment Card Industry (PCI) data security compliance via automated workflow processes. Six months following their national deployment, the number of trouble tickets opened spiked to over 3,700 with the root cause analysis indicating poor WAN circuit performance across a large number of stores. Context-specific automation processes, coupled with review by level 2 ShoreGroup engineers, indicated that connectivity was dropping multiple times a day, across multiple locations and geography, and nearly 100% of incidents were affiliated with just one service provider. This information empowered the retailer to work with their service provider, upgrade the WAN circuits and improve SLAs. Ticket volume then continued to decline even as new locations were added to the environment because automated processes and continuous improvement strategies with CaseSentry allowed support staff to react quickly and more proactively. In addition, CaseSentry s automated processes to monitor PCI data security provided the retailer with the ability to demonstrate compliance during routine audits, eliminating time-consuming manual processes and tracking that consumed valuable resources. Calculated Results: 50% reduction in labor hours Over 75% increase in system stability Recouping over $11,000,000 in revenue over 28 months 6
About ShoreGroup ShoreGroup is the globally-recognized managed services company helping customers achieve their strategic goals by empowering IT operations. Our service approach enables customers to leverage ShoreGroup's expert knowledge and comprehensive, patented management technology to create and ensure the optimal success of their vital unified communications, collaboration, contact center, virtualized data center and network assets. Our customers are major enterprises, cloud and hosted solution providers, medium to large businesses and government agencies that rely on ShoreGroup's service solutions to realize their business objectives for reduced IT operational costs, improved availability and performance, service assurance and increased customer satisfaction. ShoreGroup's innovative services are delivered directly or through leading channel partners who recognize that ShoreGroup has the best-in-class service and management technology in the industry. Corporate Offices sales@shoregroup.com shoregroup.com 844-575-2434 v1.0 ShoreGroup, Inc. 460 West 35th Street New York, NY 10001 7