Capacity planning with Microsoft System Center Mike Resseler Veeam Product Strategy Specialist, MVP, Microsoft Certified IT Professional, MCSA, MCTS, MCP Modern Data Protection Built for Virtualization
Introduction Remember the days when managing an IT environment meant working with a huge number of physical servers? A few years ago that changed. Server virtualization entered your infrastructure and things like building a new server suddenly became much easier and quicker to do. Gone are the days when you had to plan the server configuration, order the hardware (which could be a lengthy process) and unpack and rack the hardware when it arrived. While virtualization improved many tasks in the IT world, it also presented new challenges. One of these challenges is capacity planning and forecasting. Every IT administrator knows about this scenario: Suddenly you can t add a virtual machine (VM) to your environment or a VM stops working. What is the problem? Are you running out of memory? Isn t there enough storage remaining? Why do you have latency on your storage? Why are you short on virtual CPUs? After investigating the issue, you discover that you ve hit the limit of your resources. The solution is pretty easy you need additional hardware. The next thing you know, there are discussions about the number of servers that are running. Are they really necessary? (Yes, all servers for all departments are necessary.) Do they really need all those resources? And here it comes while your gut feeling says that this is absolutely necessary, management requires you to prove that the capacity is necessary. How are you going to do that? And how are you going to make sure that the next time you will see those issues coming in advance. Why capacity planning? Virtualization has changed and evolved over the years. With the latest and newest management tools, many things become possible without human intervention. But this adds complexity. Automation tasks such as creating VMs, high-availability systems, automatic failover and so on bring a lot of complexity to the table. And the bigger your environment gets, the more difficult it is to know whether or not your hosts have enough resources available. Many people look at capacity planning in the initial phase of a virtualization project. They project their initial expectations and build the infrastructure based on those initial expectations. Many people know (and have discovered) that initial expectations can be very wrong. Good capacity planning also leads to less fire-fighting. We all have been in the situation where we had to work fast and late to recover from failures. Not having enough resources is one of these annoying tasks that can be avoided with capacity planning. And last but not least, the moment you start capacity planning, you ll have to start thinking about growth. Because it becomes so easy to deploy a temporary VM or add new capacity to services (or just a new service) decisions for new servers will be made much faster. Although this was never the primary intent starting with virtualization, it quickly has become a reality. 2
Steps for capacity planning Capacity planning is not just about having enough resources on your virtual infrastructure and buying more hardware when you are about to hit the limit. It s also about finding idle and obsolete VMs. It s about proactive diagnostics and baselines. It s about knowing your environment. It s about baselines and more! To illustrate with an example: You receive an alert that one of your datastores is experiencing high latency on your IOPS. After investigation you notice that a specific VM is the root cause. Let s say it is a SQL server. Now the administrator needs to make some decisions. He or she can decide to: Reboot the SQL server Migrate the server to a datastore with more available IOPS Add disks to the datastore or choose a different course of action To make the right decision, the administrator needs to have the correct information. Is this happening all the time? Is this a recurring pattern at a certain time (for example, some Business Intelligence workflows that are running each end of the month)? Is there a bug in the application? Has the service grown in the number of users and amount of data? Having that information available is key to making the right decisions. But having that information alone is still not enough. You also need to have the right intelligence on that data. Below you can find some best practices in capacity management. Best practices See your infrastructure as a whole You need to know the relationships among the different resources in your environment. Effective capacity planning can only be done when you have an overview of your entire infrastructure. Performance metrics are a key asset Keeping an eye on the right performance metrics is key here! VMs with insufficient resources not only give you or your end users bad performance, but they also lower trust in your infrastructure. In-depth storage views If you ve never used virtualization before but are about to start, the number one bottleneck in a virtualized infrastructure is storage I/O. Having reports or views that clearly tell you what is happening is a valuable asset in your capacity management work. 3
I m running how many virtual machines? Quickly deploying VMs is one of the many benefits of virtualization, but it introduces another challenge. Server sprawl can be a serious challenge and it becomes even more challenging when you add self-service capabilities to your environment. To prevent server sprawl from becoming a bottleneck, it s necessary to have a weekly report on idle or switched-off VMs. Investigate what-if scenarios You could have enough capacity today, and even enough to cope with the next month s requirements but what if something breaks. Do you still have enough capacity? It s necessary to check on a regular basis to see if your environment could cope with one or more host failures. Virtual machine model If you use a self-service model when you deploy standard templates or in your environment, then it s an asset to be able to model these templates in your capacity planning. By using such a model, you will be able to forecast how many VMs (or sets of VMs) you still can deploy in your current environment. Set a threshold Many people perform capacity planning based on the maximum of their resources. By placing a threshold on your servers (such as 90% memory usage, 90% disk, etc.), you will get added value. The moment you hit those thresholds you can order additional hardware and you will still have some resources left to bridge the period until the hardware arrives. Schedule capacity planning as a recurring job When capacity planning is an ad-hoc job, it becomes a job that only will be performed when the IT team has some time left (which is never) or when it s already too late. Making capacity planning a weekly or monthly recurring job (depending on your growth ratio) forces you to perform this job. Scheduling reports with the data you need saves you time. Use longer periods of data Doing capacity reporting using a week s or month's worth of performance data can bring you to the wrong conclusions. For example, last month could have been an exceptional deployment month, leading you to buy resources you don t need, while a slow month could result in not buying enough resources. The longer the period of performance data that you have, the better your planning will be. 4
Forecasting Having performance data over a longer period of time is one thing; bringing in forecasting or a trend line is another. Make sure you have a good solution that calculates a trend line to bring insight regarding when you will run out of resources. Use dashboards / views Whether you call them dashboards or views or something else, these things come in very handy by preventing issues when sudden spikes or other issues happen. Using these views or dashboards on a regular basis (for example, as a daily routine by the monitoring team) will allow you to discover potential issues in advance. vcenter is not enough! Many VMware professionals say that they will work with vcenter to get the necessary information. But vcenter alone is not enough to do the job. In VMware vcenter, you can view a lot of information about your environment in real time. But all this information doesn t give you the capacity management that you need. You can review some historical data and view some charts, but if you want to do capacity management you will have to do a lot of the work yourself and keep the data in a database yourself. Figure 1: Performance counters in vcenter Other point monitoring tools, such as performance monitor or task manager, can give you a lot of detailed information but they don t include historical data, so you will need to do the calculations yourself. 5
The solution: Veeam Management Pack for VMware Many organizations already use Microsoft System Center to monitor their environments. With this monitoring framework, you can add Management Packs to your environment to monitor Windows Servers, Unix/Linux servers, application workloads and more. This provides IT teams with a single overview of their environment. One of the key capabilities missing in Microsoft System Center is the ability to deeply monitor VMware infrastructure as part of that single pane of glass. Veeam Management Pack (MP) for VMware is specifically designed to solve this challenge, providing monitoring, management, forecasting and planning for your virtual infrastructure. Veeam MP leverages your investment in Microsoft System Center to give you a unified view of your physical and virtual infrastructures so you can solve problems fast. It is built to give you a view from the app to the metal Figure 2: Visibility from the app to the metal Figure 2 shows an overview of your entire infrastructure. The green blocks are monitored by Veeam MP and the blue blocks are monitored by System Center and the specific management packs. Solving key challenges With Veeam MP, you will be able to solve some key challenges, including: No VMware visibility in your monitoring environment and lack of detailed insight Forecasting and planning Missed or difficult-to-maintain SLAs for problem solving Faster root cause analysis Datacenter inefficiencies Also referred to as Cross Platform monitoring. Application workloads such as SQL, Exchange, SharePoint, DNS, AD and non Microsoft workloads through the use of third party Management Packs. 6
Resource optimization Streamlining your monitoring processes over different technologies Fire fighting Reporting and analysis This white paper concentrates on capacity management and examines the various analytics that can be used with Veeam MP. Capacity planning Version 6 of Veeam Management Pack adds a huge amount of capacity planning for VMware out-of-the box. These capabilities are built around three concepts: Capacity reporting Forecasting Dashboards The following sections cover these concepts, all of which have features that help you achieve the best practices discussed previously. Capacity reporting Capacity reporting is what most people know and do when they talk about capacity reporting; namely, the current capacity on your hosts, how many resources you are using today and the prediction for the near future. Many new and enhanced reports are built into Veeam MP and will provide you with the detailed information needed to do capacity planning. These reports use the what-if modeling concept so you can safely manage the growth of your infrastructure. Let s have a look at some of these reports and what they can do for your environment. Performance forecast reports The first type of report is called a performance forecast report. Veeam MP v6 has different reports of this type: 7
Figure 3: Performance Forecast for vsphere Clusters The sample report in Figure 3 shows several parameters for the cluster. For example, 90% of the CPU capacity on that cluster is used as the threshold (red line). The capacity (in GHz) for that cluster is shown as the green line. Look at the CPU Utilization parameter for 30 days (configurable in the report parameters) and search for a trend and the deviation of that trend (yellow) bar. Finally, note that the blue line shows the actual CPU usage of that cluster. Above the performance graph is the forecast. Because you can look at the historical data, you can detect a trend and foresee when the cluster would run out of resources. There are two figures for that the first figure is the worst-case scenario and the second figure is the best-case scenario. Based on this data, you would be able to forecast when your infrastructure could run out of resources. Host failure modeling reports Host failure modeling is another great way of predicting capacity. Many companies tend to do capacity reporting at full capacity. But then, what happens if they have failures in their infrastructures? Outages such as a host that fails or maintenance work that needs to be performed on a host can have a huge impact on the amount of remaining resources or the number of VMs you will be able to run. The Host Failure Modeling report in Veeam MP v6 provides the data needed to foresee such a failure in your infrastructure. 8
Figure 4: Host Failure Modeling report Figure 4 shows an example of the Host Failure Modeling report. To run this report, you select parameters such as number of clusters (the example shows one or more clusters) and then you set thresholds and determine how many hosts will fail. The example report simulates one host failure. This can be a true failure or just one server that is placed in maintenance mode. The results show what your available resources will be after a host fails or is placed in maintenance mode. The same intelligence that was used for the performance reports is applied to this data to forecast how long you can run under these conditions. Virtual machine capacity prediction In today s world of self-service deployments and rapid provisioning, it is critically important to be able to accurately predict how many more VMs you can deploy on your infrastructure. But how do you do that? If you are working with predefined VMs, you can easily use the performance data of such a running VM (or VMs if they are part of a service) and run that data against your current available resources. This tells you how many services you will be able to deploy. 9
Dashboards Figure 5: VMware capacity prediction Microsoft System Center 2012 Operations Manager introduces a new concept called dashboards. These dashboards give you a real-time performance view of your critical vsphere systems, allowing you to quickly drill down deeper into the cause of a problem, perform detailed troubleshooting and speed up root cause analysis when necessary. Veeam MP v6 offers 30+ in-context dashboards out of the box. Many people will argue that this is not capacity planning, but these dashboards can be a very valuable asset to forecast potential problems or potential growth that is not foreseen. For example, one of the dashboards is the Top Datastores dashboard. By keeping an eye on this dashboard on a daily basis, you can quickly discover possible VM growth or a sudden increase of IOPS or latency in time to take action. 10
Figure 6: Top 10 dashboard for Datastores In Figure 6 you see the top 10 dashboard specifically for datastores. This dashboard shows you some key information about your datastores such as free space, used space %, IOPS and maximum latency You can see at a glance that the average value for the latency on a datastore called BiLUN0 is very high, and you can right-click on that dashboard to research this potential issue using Analysis Dashboards. Analysis dashboards Analysis dashboards go deeper into a specific issue in your VMware environment. In this case (shown in Figure 7), an analysis dashboard provides detailed information about what is causing the high latency on the specific datastore shown in Figure 6. 11
Figure 7: Analysis Dashboard Figure 7 shows an Analysis Dashboard that provides detailed information about the datastore in question (BiLUN01). A graph shows the overall read and write latency on that datastore, and an alert view gives all warnings and critical alerts on that datastore and the VMs it contains. There is also a graph showing the total IOPS per VM running on that datastore. This information allows you to quickly find the problematic VM on that datastore. Conclusion Capacity management is more than predicting the number of resources you need upfront it s an ongoing task you need to perform so you won t have any surprises in your environment. It is also a very challenging task if you don t have the correct data or business intelligence. Capacity management is more than just analyzing performance counters it s about having a global view into your environment and looking at dashboard views on a regular basis. With Veeam Management Pack for VMware you can collect all the information and intelligence you need to perform forecasting and deep-level analysis. In addition to the capacity management capabilities, you also get deep VMware monitoring and a comprehensive view of your infrastructure and dashboards that will assist you in your day-to-day job. 12
About the Author Mike Resseler is a Product Strategy Specialist for Veeam. Mike is focused on technologies around Hyper-V and System Center. With years of experience in the field, he presents on many occasions at large events such as MMS, TechEd and TechDays. Mike has been awarded the MVP for System Center Cloud and Datacenter Management since 2010. His major hobby is discussing and developing solid disaster recovery scenarios. Additionally, he has enterprise-class experience in private cloud architecture and deployment, with marked focus on protection from the bottom to the top. He holds certifications in many Microsoft Technologies such as MCITP. Follow Mike on @MikeResseler or @Veeam and on Google+. About Veeam Software Veeam is Modern Data Protection. We believe today s IT requirements have changed and that 3C legacy backup problems high costs, increased complexity and missing capabilities are no longer acceptable for any organization. Veeam provides powerful, easy-to-use and affordable solutions that are Built for Virtualization and the cloud a perfect fit for the modern datacenter. Veeam Backup & Replication is VMware backup, Hyper-V backup, recovery and replication. This #1 VM Backup solution helps organizations meet RPOs and RTOs, save time, eliminate risks and dramatically reduce capital and operational costs. Veeam Backup Management Suite combines Veeam Backup & Replication and Veeam ONE in a single integrated solution to protect virtualization investments, increase administrator productivity and help mitigate daily management risks. Veeam Management Pack (MP) extends enterprise monitoring to VMware through Microsoft System Center. Veeam also provides free tools for the virtualization community. Learn more by visiting http://www.veeam.com. 13
Modern Data Protection Built for Virtualization Powerful Easy-to-Use Veeam Backup & Replication #1 VM Backup for VMware and Hyper-V Virtualization changes everything especially backup. If you ve virtualized on VMware or Hyper-V, now is the time to move up to the data protection solution Built for Virtualization: Veeam Backup & Replication. Unlike traditional backup that suffers from the 3C problem (missing capabilities, complexity and cost), Veeam is: Powerful: Restore an entire virtual machine (VM) or an individual file, email or database record in 2 minutes Easy-to-Use: It just works! Affordable: No agents to license or maintain, works with your existing storage, and includes deduplication, VM replication, Microsoft Exchange recovery, and more! Join the 58,000 organizations who have already modernized their data protection with Veeam. Download Veeam Backup & Replication today! GOLD AWARD NEW TECHNOLOGY GOLD AWARD NEW TECHNOLOGY To learn more, visit http://www.veeam.com/backup 14