E-GUIDE. Capacity Control: Why Capacity Management Needs to Evolve

Transcription

1 E-GUIDE Capacity Control: Why Capacity Management Needs to Evolve

2 Contents Introduction 3 Why Capacity Management Falls Short in Virtualized Environments: The Need for Capacity Control Solutions 3 What is Capacity Control? 5 The Importance of Policy in Managing Risk and Efficiency 5 Increasing Density though Placement Analysis 6 Right-sizing Through Accurate Allocations 7 Incorporating Growth Trends 8 Increased Agility and Accurate Forecasting with Capacity Reservations 9 Automating Accurate Actions 10 Closing the Loop A Holistic Approach 11 Conclusion

3 Introduction IT has always relied on some form of capacity management process and supporting analytics tool to determine how much hardware is needed to run the applications required by business. The introduction of virtualization and cloud technologies has created a new set of management challenges that are fundamentally changing this requirement, including: 1. A much higher rate of change: Workload mobility and the introduction of new and transient workloads into virtual infrastructure results in a far more fluid environment that is difficult to manage and control. Decisions need to be made far more quickly and often the variables are changing while the decisionmaking process is underway. 2. Competition for capacity resources: With the ability to create high density environments, the competition for compute resources is of critical concern. IT needs to determine the best way to allocate resources to get optimal efficiency without creating performance risk and impacting service levels. 3. Co-habitation and compliance of resources: With diverse workloads now sharing infrastructure and virtual machines being created on-demand, IT is now encountering compliance and associated business issues related to workload placement that simply didn t exist in physical environments. 4. The need for automation: The complexity brought by workload mobility, the sharing of infrastructure and speed with which changes can occur has increased the number of moving parts to levels well beyond what can be managed using manual or legacy approaches. These new complexities mean that IT must evolve from using simple capacity management processes and tools that profile and trend utilization, to implementing capacity control solutions that enable organizations to truly understand and respond to the dynamism, requirements and constraints that exist within these environments, giving organizations greater control over operational risk and efficiency. This e-guide discusses why capacity control solutions are needed to meet the requirements of today s virtual and cloud infrastructure. Why Capacity Management Falls Short in Virtualized Environments: The Need for Capacity Control Solutions In non-virtualized environments, IT teams could manage infrastructure requirements by leveraging manual processes and project-based planning to size infrastructure. Capacity management techniques involved creating a baseline of metrics to understand utilization; then manually tracking new systems and analyzing utilization trends to forecast future requirements. The purpose was to ensure enough capacity existed to avoid risk and little, if any, attention was paid to increasing efficiency

4 But the game has changed. Today s modern ecosystem is much more complex and dynamic than it was a decade ago. For one thing, it is very data rich with many more factors to track and manage such as Virtual Machine (VM) placements, new configuration parameters and even new dimensions to utilization that have now become important. Adding to the complexity is the fact that the data is coming from multiple sources including monitoring tools, CMDBs, VMware vcenter and the like, creating disjointed sources and formatting of information. Multi-tenancy adds another dimension to the management challenge. There are different users, lines of business and customers, all with different constraints, regulations, requirements and potentially even service level requirements, all sharing infrastructure. This makes it extremely difficult to determine what actions to take such as where VMs can and should go, how large to make them, how densely to pack an environment, or how to setup and leverage over-commit. These decisions require a deeper understanding of the environment as a whole, and the purpose and nature of the workloads that reside within it. Thirdly, in cloud environments, change occurs at a much higher rate. VM s won t necessarily be in the same place they were yesterday, significantly changing the utilization of any particular host. Self-service request portals for new workload placement exacerbate this problem as the lines of business expect faster turnarounds for new deployments. This means new workloads are entering an environment and have to be processed more quickly than ever before, dramatically impacting utilization demands, supply side requirements and management complexity. Finally, because of the fluidity and operational nature of many of these changes, infrastructure managers are also getting more involved. The responsibility cannot fall to traditional capacity managers working against long term planning cycles. The changes are happening far too frequently. Application owners nervous about putting their applications into shared infrastructure demand greater visibility. And capacity managers still play a part in aggregate planning and forecasting. These stakeholders cannot get the information they need simply from long term trends provided by old school capacity management models. Infrastructure Managers Enterprise Users Capacity Managers Application Groups Demand Portal Cloud Catalog (IaaS, PaaS) Old School Capacity Management tools cannot possibly tie all of these together Ticketing & Service Desks Orchestration & Provisioning Meta-Data Sources Monitoring CMDB Data Sources vcenter & VMMs Committees & Approval Processes Actioning Mechanisms Today s modern ecosystem is comprised of many moving parts. Copyright 2012 CiRBA Inc. All Rights Reserved

5 Traditional capacity management frameworks can t relate all of these factors and parameters to each other in order to provide infrastructure managers, capacity managers and application owners with the answers they need. A more active discipline and new level of control through predictive analytics that consider these elements is needed. The decisions that need to be made no longer relate just to how much compute capacity is required, but how available resources are allocated amongst the workloads that share it, while factoring in a wide variety of variables including where those workloads can and should be placed within the environment. IT needs to look beyond trending analyses for the purpose of sizing and look for solutions that can help them gain the visibility needed in order to understand and control according to this new requirement. Leveraging traditional sizing methods, organizations are stuck with: Spending too much money on infrastructure through over-provisioning to be safe in estimating requirements or not having enough capacity to fulfill requirements Fire-fighting and repeated performance issues Not meeting the needs of the business through reliance on manually intensive processes to handle requests, making IT less agile and more reactive than proactive What is Capacity Control? All the new variables that have now been introduced by virtualization and cloud computing must be intelligently controlled in order to achieve the benefits of these technologies, which can only be done with capacity control solutions that provide the following capabilities: 1. Policy based control 2. Placement analysis 3. Allocation analysis 4. Growth trends 5. Capacity reservations 6. Automation The next section will outline the requirements of each of the above capabilities in detail. The Importance of Policy in Managing Risk and Efficiency Workloads no longer exist in operational silos; enterprises need to factor in information beyond utilization thresholds such as service levels, business requirements and regulatory requirements to determine where workloads can go and how resources should be allocated to them. A capacity control solution should be able to represent all the technical, business and operational constraints and define the automation policies. Policies act as a contract between supply and demand a written rule about how the infrastructure is managed to ensure that everyone is protected from each other. They ensure that workloads are configured and placed considering the constraints, regulations and operational goals of the individual workload requirements and the IT environment

6 as a whole. Properly specified control policies cover both quantitative (i.e. utilization levels, resource overcommit targets, contention tolerances, etc.) and qualitative criteria (i.e. business rules, technical affinities and anti-affinities, security requirements, processoriented requirements, etc.). By leveraging policies within management frameworks, organizations achieve accurate recommendations that can be immediately acted upon, increasing responsiveness in addition to ensuring compliance with regulatory requirements and the safe operation of workloads. Policies formalize management criteria representing the constraints, regulations and operational goals of IT environments. By leveraging policies, organizations are ensured compliance with regulatory requirements, as well as the safe operation of workloads. Increasing Density though Placement Analysis Intelligent VM placements are essential to having a well-managed, efficient, high performing virtual environment. Well placed workloads ensure that enough resource is available for the projected utilization thereby significantly reducing risk, the need for VM motioning and placement volatility. Placements should be determined by looking at all the relevant constraints such as historical utilization, technical configurations, and policy requirements such as privacy or regulatory requirements; not just of an individual workload in isolation, but in consideration of all the workloads that need to be placed within the environment. Like playing Tetris, good placements maximize the use of infrastructure while ensuring workloads have enough resource to do their work without falling short. Poor placements create a variety of issues from stranding capacity to wasting resources, resulting in an increase in performance risk and the number of VM moves within an environment. Many organizations rely on load balancing tools that look at short term utilization history to find room for a single VM or move VMs in response to problems. While technically

7 these tools determine workload placements, they do so in a way that fails to account for all the required constraints and a longer term view of the utilization requirement. The result is increased frequency of moves or greater placement volatility. In addition, while the placements are made based on where capacity is available, it does not consider how to optimize the layout of all the VMs in the environment to maximize density and reduce performance risk. Load balancers force IT to react to shortfalls instead on focusing on making the right placement decisions in the first place. Capacity management frameworks are not sophisticated enough to factor in VM placements in order to manage density and frequently, even so called virtualization-ready tools, don t even understand it. Capacity control solutions, on the other hand, allow IT to take into consideration the various aspects of a workload and factor that into placement decisions: How it changes during an operational cycle and what patterns occur throughout the day, month, or even year; Which workloads are utilizing the same resources at the same time; What risks are associated with putting one workload together with another; How to dovetail workloads that utilize different resources such as CPU, memory and I/O. When determining how resources should be shared to maximize utilization in a given environment, IT needs to be able to factor in workload types, activity patterns, service levels, contention risk, etc. Capacity control solutions allow IT to create an allocation model and determine how to configure workloads into a capacity model using overcommit and resource sharing to optimize the placement of workloads, to ensure the maximum use of capacity, use the least space and incur the lowest risk. Right-sizing Through Accurate Allocations Accurate VM sizing is imperative to establishing efficient infrastructure. Allocate too little resource and workloads experience performance issues. Allocate too much and capacity is held hostage by workloads that won t ever require it. With the advent of cloud computing and self-service models, users have been able to request containers and effectively size VMs on their own. Most often this results in large-sizing VMs and capacity being held unnecessarily by the workload. These over-allocations are very costly and wasteful. Capacity control solutions specify exactly how much resource each workload actually requires, so that IT can maximize efficiency. Traditional capacity management tools tend to determine VM allocations by making gross assumptions about allocations based on a one-size fits all approach to some pretty basic policy parameters. The key to accurate VM right-sizing is to first understand the purpose of the workload itself and the nature of the workload s utilization pattern. These constraints mix with others such as service levels, DR plans, etc to impact the allocations made which are reflected in the policy applied to managing allocations for workloads. For example, batch job workloads may run at 95% or even 100%, but as long as they finish their job on time, this high level of utilization may be perfectly fine. A transactional workload, on the other hand, may have an upper utilization limit of 70%. Applying the same policy threshold to both these workloads will result in either over-or under-sizing one of the workloads. Policy can also reflect if a workload shouldn t be re-sized, but rather if another

8 workload altogether needs to be added when utilization reaches a certain limit such as in elastic compute pools. Similarly, policy should dictate that DR pairs are re-sized in concert so that active workloads aren t larger than their passive counterparts. Foundational to sizing is the requirement to ensure the right historical data and the right amount of it is in place to make accurate, safe decisions. For example, up-sizing a VM allocation on memory may be a straightforward decision even with very little history when performance problems are encountered. Conversely, when looking to increase efficiency and reduce allocations, administrators need to ensure they have a full business cycle of data to accurately analyze patterns in order to avoid performance issues after a change. Down-sizing changes also typically require an approved ticket and have to comply with change management processes, sometimes even committee approvals for this very reason. It s also key that suitable outliers are removed from the patterns used in analysis and other conditions and events such as operational windows are accurately reflected so the results aren t skewed in one direction or another. Capacity control systems provide allocation recommendations so that resources are being used optimally. Incorporating Growth Trends As previously mentioned, the objective of traditional capacity management tools was to ensure there was enough capacity in each server to avoid performance issues. With virtual and cloud environments, understanding historical utilization and growth projects still remain factors in determining capacity requirements, but they are no longer the sole reference point to make decisions upon. Trending needs to consider and reflect: Historical utilization patterns to project future requirements. Aggregate growth trends for the environment including the organic growth of workloads that exist in the environment in addition to the trend of new workloads being added

9 Growth profiles on a per system basis to accurately reflect changes associated with individual VMs, applications, or business units. Resource requirements and model differences in utilization changes such as an increase in memory consumption vs. CPU utilization. While trending the organized growth of the existing workloads is useful and necessary, requirements for virtual infrastructure and in particular, clouds are often more impacted by workloads coming into and leaving the environment. Hotel operations provide a useful analogy to explore this further. A hotel owner isn t concerned about whether or not last year s visitors grew taller since their last visit, he is only concerned about when they are coming to the hotel and when they are leaving, so that he can manage his rooms accordingly and ensure that there are rooms available for those coming to his hotel. Similarly, determining capacity requirements in these environments can be dramatically impacted by what is being virtualized versus a 10% growth rate in existing utilization. As such, both trends and the ability to reserve future capacity are required to accurately forecast requirements. Increased Agility and Accurate Forecasting with Capacity Reservations Most IT organizations lack the ability to model capacity requirements considering future workload placement requests from users. Instead they typically rely on estimates, which create one of two problems. It either puts IT at risk of falling short and being unable to respond quickly to the business. Or it forces IT to wildly over-provision capacity to reduce the risk of shortfalls in anticipation of what might be requested. Cloud stacks and self-service portals accommodate immediate requests, but don t have the ability to reserve capacity into the future, leaving a big gap in forecasting requirements. This can be solved by leveraging predictive analytics and a capacity reservation system. The only way to effectively prepare for upcoming demands is to know what really is coming down the pipe. IT should be able to look forward in time to determine how much infrastructure will be required by seeing all the new workloads coming into the environment, determine if there is enough space to accommodate them and then reserve space for these new workloads for the planned placement date. At the very least, a capacity control solution should include a capacity reservation system that provides the following capabilities: 1. Capture and/or receive demand profiles from various sources such as: Migration project plans Application deployment (release management) processes Self-Service portals 2. Assess whether the demand will fit into the target environment at the future date it is scheduled to be deployed (taking into account trends as well as other confirmed bookings, decommissioning and server-side changes) 3. Lock in required capacity so it cannot be usurped by another user/application This will ensure that applications have the capacity when they need it and prevent IT from having to over-provision resources to deal with uncertainty, thereby creating more efficient infrastructure

10 Critical to measuring, planning and managing an environment is knowing what s in it today, and what will be in it in the future and the ability to reserve space for it. Automating Accurate Actions As virtual environments grow, it is increasingly difficult for IT teams to manually manage and execute the changes required for allocations, placements and other routine changes. Automation is at the forefront of IT plans, but very difficult to achieve, primarily due to the lack of tooling available to definitively create a set of actionable recommendations as to how to keep the infrastructure healthy and resolve issues. Typically, analysis tools such as capacity management tools provide data, charts and reports that give the background on a problem without providing a prescriptive resolution. Capacity control solutions provide proactive management by producing detailed and actionable recommendations based on utilization trends and profiles, configuration and policy requirements and knowledge of which VMs and hosts are coming into or leaving the environment. Leveraging predictive analytics, capacity control solutions provide organizations with actions based on a forward-looking view of the environment including where workloads need to be placed and how resources should be allocated. These solutions then enable the recommendations to be automatically pushed through the right system or process to make them happen. Again, to maintain the integrity of the workloads and their placements, these actions should be governed by policies to ensure that they meet with operational and business requirements from every perspective, including routing and approvals that comply with change management requirements. The ability to manage and automate actions gives IT a more methodical approach and granular control over what happens and when

11 Automation enables organizations to know where workloads need to be placed, how resources should be allocated and what is coming into and going out of an environment and then push these recommendations through the right system or process to make them happen. Closing the Loop A Holistic Approach Ultimately, the reason traditional capacity management tools are no longer useful in modern data centers is that they are unable to fully analyze what is happening in these environments and are incapable of providing detailed recommendations of what to do. Capacity control solutions enable IT to manage infrastructure in entirely new ways that: Maximize density and minimize risk Allow organizations to move from being reactive to proactive Provide end-to-end coverage from VM planning to onboarding through to steady-state management Provide significant returns by ensuring that infrastructure is being used as efficiently as possible Capacity control solutions are able to tie in all the moving parts of today s modern data center s ecosystem, thereby creating a more agile, lower risk and efficient operation for IT. Infrastructure Managers Enterprise Users Capacity Managers Application Groups Policies & Definitions Analytics & Visualization App Views, Reports & Recommendations Demand Portal Cloud Catalog (IaaS, PaaS) Bookings & Models Analysis Console Control How do Console you tie all of these together? Capacity Control Solutions User Views Actions Ticketing & Service Desks Orchestration & Provisioning Meta-Data Sources Performance Data Monitoring Business & Org Data CMDB Data Sources Host & VM Data vcenter & VMMs Committees & Approval Processes Actioning Mechanisms Capacity Copyright control 2012 CiRBA systems Inc. All Rights enable Reserved IT to meet the needs of today s modern ecosystem including data and meta-data sources, auctioning mechanisms and enterprise user requirements

12 Conclusion Capacity Control: Why Capacity Management Needs to Evolve As history proves, the modern data center is constantly evolving. While today s technologies bring the promise of agility and efficiency, they also bring about more complexity. The benefits of adopting virtualization and cloud computing technologies can only be realized through capacity control solutions that ultimately allow IT to take a much more active role in the management of virtual and cloud environments. About CiRBA CiRBA is analytics software that enables organizations to maximize the efficiency of IT infrastructure, whether it s physical, cloud-based or virtualized. Only CiRBA s policydriven analytics accurately answer the questions of how to optimize infrastructure purchase and allocation decisions, where risks exist, and where to place workloads in order to maximize utilization. 45 Vogell Road, Suite 600 Richmond Hill, ON Canada, L4B 3P6 Toll Free: Telephone: Fax: Online: Copyright 2012, CiRBA Inc. All rights reserved