Cloud Operations and Usage Research: Learning from Experience VMWARE WHITE PAPER

Transcription

1 Cloud Operations and Usage Research: VMWARE WHITE PAPER

2 Table of Contents Introduction About the Research Growing Pains... 4 A Focus on Simplification Simplification: The environment Simplification: People Simplification: The service delivery model Seeing the Value Looking for Ways to Manage Usage IT as a Service Broker The Path Forward Complexity and How to Overcome It VMWARE WHITE PAPER / 2

3 Introduction As companies have put cloud computing to work in their organizations, they have sometimes found daily operations to be complex. But ultimately, many agree that working through this complexity is proving to be well worth the effort, and the value achieved from cloud computing far outweighs those challenges. Those are some of the findings of recent research conducted by VMware into how organizations are using and operating private cloud environments. In extensive interviews with IT professionals, this research asked who in the organization is using the cloud, and how are they using it? Who is responsible for the daily operations of the cloud, and what role do they play? What lessons have they learned? And where will their cloud efforts take the organization? The goal of the research was to develop a snapshot of actual cloud usage and operations one based not on plans and projections, but on real experience gained through the day-to-day use of a private cloud environment. The research provides insight into how companies are addressing the challenges of managing and operating their new platforms. It shows that they are finding practical approaches, taking advantage of new capabilities and exploring new possibilities. And it points to new cloud-oriented service models that can change the way IT serves the business to the benefit of both. Overall, the research shows how companies are reducing complexity and simplifying the pursuit of the value that cloud offers and putting IT on the road to becoming a service broker enabling greater efficiency and speed to the business. About the Research The VMware cloud usage research included 12 organizations from a range of industries that had significant experience with cloud operations. A third-party organization conducted in-depth interviews with individuals in those organizations from February through May The respondents held titles such as chief technology officer, director of IT infrastructure, head of datacenter operations and chief VM architect, among others. Those organizations were using cloud computing for various use cases, including: Revenue-generating product development and test Homegrown application development and test IT deployment of packaged software and test Customer support IT help desk Academic research Sales demos In terms of experience, the organizations had been using cloud computing for varying time periods up to five years, with the average being about 13 months. The number of cloud users ranged from 100 to 8,000; organizations at the high end of that range were using a private hosted cloud model. The number of virtual machines in use ranged up to 5,000, but most organizations had between 300 and 500. The maximum number of images in use was 300, with the average being about 50. VMWARE WHITE PAPER / 3

4 Growing Pains In the research, respondents made it clear that they view the operation of a cloud environment as more complex than virtualized or traditional environments. However, they don t see this complexity as a serious barrier, but rather as a natural consequence of dealing with new, evolving technology that is, as natural growing pains that are to be expected. Such complexity stems from several sources. One is growing demand from the business and the corresponding increase in the diversity of business needs. This is a familiar challenge for IT, but with cloud computing, the pace of change is increasing. As users become familiar with the way the cloud can help them do their jobs, word spreads across the organization and the demand for cloud services increases exponentially. This creates new and varied business requirements that must be addressed. IT departments can find it difficult to keep up with this accelerated change and increasing variety of business needs. Respondents also cited the relative immaturity of cloud infrastructure products as a driver of the complexity of operating a cloud environment. But they generally felt that this was simply an unavoidable aspect of new technology, as vendors are working to evolve and expand product functionality. In the meantime, some companies are filling in the gaps by developing their own home-grown tools to handle specific activities such as reporting, security, customer management and usage monitoring. In a related vein, a number of respondents cited vendors efforts to rapidly evolve products as a contributor to complexity. Here again, it is not change per se that is new it is the pace of change. The speed of private cloud product evolution makes it especially important for companies to stay up-to-date on advancing technology and be prepared to implement improvements as they come along. Several respondents cited the need for more active vendor management in order to keep up with fast-moving cloud product pipelines and roadmaps, and plan accordingly. All three of those factors present challenges, and they are essentially transitory in nature meaning as cloud products and usage mature, these challenges will diminish. But respondents consistently said the most significant driver of complexity is the creation of new interdependencies that cross traditional IT silos. Cloud computing has made IT think differently about how storage, networking, servers and security have to work together and be managed holistically. This reality extends beyond technical issues: Respondents noted the need to re-think processes and policies in IT, as well as the way various IT teams interact. As one explained, The technology part is very easy to work and put the pieces together. It s the business model around the interdependency of silos and how you do the related policies, procedures and processes that are really going to drive your success. VMWARE WHITE PAPER / 4

5 A Focus on Simplification As mentioned above, respondents understood the reasons for such operational complexities, and because of the value cloud computing brings to the table, they are enthusiastic about the cloud. They are thus quite willing to work with complexity. To a great extent, this means simplifying their approaches to the cloud on three fronts: the environment, people and service delivery model. Simplification: The environment Respondents stressed the importance of thorough upfront planning for deploying their new cloud environment. The better the plan, they said, the smoother the operations will be. (A few companies had not planned thoroughly, and they reported more problems during the operating stage.) Good plans evaluate how people, policies and IT processes will need to adapt, as well as the technology itself. [Our cloud environment] is very process-centric. We built that in from the ground up, which basically meant a lot of planning and building processes up front, said one respondent. We found that because we did that, a lot of the day-to-day operations and maintenance stuff that we do now is much, much easier. We have established processes in place, and we haven t had to go back and figure out the processes after we had everything up and running. Companies are taking a variety of approaches to simplifying the technology landscape. One is to limit the diversity of technology, using one type of server, one type of storage solution, and so on. In the words of one respondent, they are going for a flat, flat, flat landscape. That of course means limiting the number of vendors, and drawing on the best practices of each, rather than re-inventing the wheel. The goal: Reduce interoperability challenges, complexity in identifying problems and multivendor finger-pointing. When asked whether this approach brought up concerns about vendor lock-in, most respondents said that it didn t. With the need to refresh technology every three years or so, they felt that if they need to switch vendors, they will have ample opportunity to do so without significantly affecting the capability of their infrastructure. Meanwhile, many companies are pushing the use of standard functionality in products, and minimizing customization as much as possible. This approach makes it easier to take advantage of vendor knowledge if a problem develops, as well as make changes in the future. [Too much customization] is totally counterproductive. It creates a one-off environment and we are the only one in the world with this specific environment and then we can never really upgrade or change, explained one respondent. Respondents pointed out that it is not always possible to eliminate all customization, due to the maturity level of cloud technologies and the need for home-grown tools. But at least one company had avoided customization entirely. Overall, companies are looking ahead to increased functionality in vendor offerings, and plan to embrace standard approaches across the board as they become available. In the drive to simplify their cloud environments, most companies said they chose to start with a clean slate, and buy new equipment to run their cloud platforms. Like putting limits on technology diversity and customization, this essentially reduces the number of variables patches, updates, older technologies and so forth that can make it difficult to troubleshoot problems. IT groups that have taken this approach understand that this pristine state won t last forever, but they see it as an important tactic in the early stages of operating a private cloud environment. Finally, all the companies in the research emphasized the importance of focusing on well-understood areas. That is, they have deployed cloud computing for use cases that they are already supporting in virtual or physical environments. This way, they are intimately familiar with the internal customers involved, and with those customers needs, expectations and objectives, which helps reduce unknowns and variables. Several respondents said that because they are now comfortable using cloud with these known use cases, they are ready to begin applying it to new use cases in the near future. VMWARE WHITE PAPER / 5

6 Simplification: People The VMware research found that companies are taking two basic approaches to simplifying the people aspects of operating cloud computing. Most respondents have created specific cloud teams in the central IT organization to operate the cloud environment, and many have also established specific user groups, as well. The central IT cloud teams are typically small perhaps five or six people. Because of the importance and complexity of the task at hand, these teams are typically made up of particularly skilled and capable IT professionals, hand-picked especially for the team. (These teams go by various names at their respective companies; the term cloud teams was applied by VMware researchers). Team members typically represent different areas of IT, such as storage or networks, and are often selected because of their breadth of knowledge beyond their specific domain. That is, an individual who is the server lead on the team might also have significant experience in storage or networking, or the team s security expert might also have a good understanding of servers. In essence, the teams are cross-functional at the team level and the individual level a factor many companies viewed as critical when working with the cross-silo realities of cloud computing. A number of respondents felt this type of dedicated cloud team may not be needed in the longer term. This is not because they believe that the complexity of operating across IT silos will be eliminated. Rather, they foresee cloud computing becoming more pervasive, and products evolving, to the point where cloud operations are a mainstream IT activity. In a sense, most of the IT staff will be the cloud team. As the number of cloud-focused professionals in IT grows, they may need to be organized into subgroups such as server groups or storage groups. But, respondents said, the multidiscipline approach used in today s small cloud teams will still be critical to helping those groups work effectively across IT silos. In terms of the user community, respondents tended to offer newly developed cloud services to their technically oriented users first. Often, these are either developers who are working on revenue-generating products or technical support engineers. These users are broken into two groups: the general user body and super users. Both reside in the business unit or in a department of a business unit. Super users (again, a name assigned by VMware researchers and not the respondents themselves) are those who are especially interested in the technology, able to understand it quickly and in position to help their peers use it a kind of resident expert. Thus, in a group of developers, a super user is a lead developer who is responsible for helping other developers use cloud images and templates to complete their work assignments. Simplification: The service delivery model Many companies leveraged this two-tiered people approach to create an innovative and effective service delivery model. (See Figure 1). In this model, the cloud team working as part of the infrastructure group in the central IT organization coordinates its operational responsibilities with the various individuals within IT, such as networking or security professionals. The cloud team then delivers Infrastructure as a Service (IaaS) images to the super users who are located in the business unit or its respective departments. At this point, the super users take the infrastructure images and add content to make them more useful to others working in the business. For example, they might add development frameworks, libraries, web services, data services, messaging services, applications whatever is needed by the business. The general user body can then use these enhanced images via self-provisioning capabilities, or request super users to handle the provisioning on their behalf. This approach creates a clear delineation of operating responsibilities between those providing the infrastructure (the IT cloud team) and those wrapping business content around the infrastructure (the super users). In essence, the super users are converting the IaaS image to a Platform as a Service (PaaS) image that can be used by the general user body. Those general users can work directly with the super users and provide feedback to help them keep improving the platform rather than having each developer pass their requests to IT. VMWARE WHITE PAPER / 6

7 With this model, technology professionals and business users can each focus on what they do best. IT teams are freed up to spend less time dealing with user requests, and can spend more time assessing new technology and managing the vendors that are supplying hardware and software. Business users can devote less time to working through time-consuming purchase decisions around hardware and software, and more time developing revenue-generating products. Figure 1. The common cloud service delivery model VMWARE WHITE PAPER / 7

8 Seeing the Value As the companies in the VMware research have gained experience with running and managing a private cloud environment, they have developed a sense of both the challenges involved and the value cloud computing can deliver. For example, respondents said their organizations found self-provisioning and consumption of cloud resources to be very straightforward. However, it is important to note that in most cases, companies selected relatively tech-savvy user populations for their initial efforts. Some respondents said they were reluctant to provide private cloud access to less technically oriented users until they had a more user-friendly interface in place. This caution was reflected in one executive s comment: Self-serve is easy and users like it. But you have to be careful; if you give them too many choices you can make it hard for them to decipher differences and make the right choice. Across the board, companies noted a significant increase in speed in IT service delivery. One respondent found that server provisioning times dropped from months in a physical environment to 10 days in a virtual environment to just one hour with a cloud environment. Another company reported a drop from two to eight weeks down to a few days or even a few hours with a cloud model. Increased speed was noted on another level, as well. One respondent cited the advantages of multi-threading, or the ability to have multiple developers work with clones of an identical image at the same time. This ability means that developers can program in parallel, rather than sequentially. Similarly, multiple testers can work with cloned images at the same time. These multi-threading capabilities mean product development can deliver tangible results to the business more quickly. Such benefits help keep IT costs down as does the reduced need for physical hardware. But the research found that many companies see value in operating a cloud environment beyond greater IT efficiency. They also see value from improvements in the business, such as the ability to get products to market faster and resolve customer service problems more quickly. In the view of most respondents, these business benefits were of more importance than the IT benefits achieved, and they stemmed primarily from the increased effectiveness of the super users and their ability to transform the baseline images into PaaS images for use by the business. As discussed above, most of the companies in the research experienced exponential increases in demand from business users once the initial cloud platform was deployed. This presents challenges, of course. But companies also viewed it as a clear indicator that value is being delivered and essentially, a good problem to have. VMWARE WHITE PAPER / 8

9 Looking for Ways to Manage Usage With this reality in mind, the research examined how companies are managing growth and capacity consumption with their cloud platforms. At this point, most are not using automated chargeback or showback capabilities, and are instead handling those reporting processes manually. This is partly due to the fact that until recently, chargeback and showback tools have not been widely available from vendors. But more important, companies are still trying to determine what policies and financial models they need around chargeback. Should they charge per virtual machine or RAM, or at the infrastructure or platform level? For the most part, companies believe that once these financial questions are answered, they will be able to think more definitively about the types of tools needed to support the business model. In the meantime, companies are pursuing a number of approaches to managing growth and usage. Some are simply restricting the amount of capacity they put in a virtual data center for users. Some are limiting the amount of time the business can use virtual machines to perhaps two weeks or a month. Several are just buying additional equipment to keep up with demand. And at least one respondent developed a governance model that involves negotiating capacity and costs with business users up front. Companies view these approaches to managing capacity as less than ideal, but they are confident that more streamlined and robust chargeback processes and methods will eventually be in place. Their real concern is to meet internal customer demand and not place too many restrictions on what they see as a healthy trend toward increased cloud usage. IT as a Service Broker Most respondents said that as they address these operational challenges, cloud computing is changing the way IT works with the business, and transforming IT into a service broker. In this model, IT will be able to offer users a mix of services through a single portal that encompasses physical, virtual and public and private cloud environments. Users will then be able to compare features/functions, time-to-provision information and pricing, and make the choice that s right for them. IT groups see this service broker approach as a way to help IT be and be seen as a business enabler that puts tools and choices into the hands of users. At the same time, they believe it will help facilitate the shift to the cloud, as the portal allows business users to clearly see the cost, performance and provisioning-speed advantages of cloud computing, compared to physical infrastructures. The research also asked companies what they expect to do over the next 18 months what steps they plan to take to move them closer to that service-broker role. As might be expected, they said that they are looking to expand functionality, particularly in the areas of disaster recovery, chargeback, security, self-provisioning and capacity planning. They also expect to build more PaaS and Software-as-a-Service capabilities, and make their cloud portals more user friendly in order to serve a wider variety of users. And they plan to extend their cloud platforms to encompass more users and more use cases. VMWARE WHITE PAPER / 9

10 The Path Forward The VMware cloud operations research paints a picture of companies successfully using cloud computing to deliver business value. But in many ways, they are still learning. Indeed, many respondents disputed the notion that they had developed best practices for operating a cloud environment. They stressed that they still have a lot to learn, and that they are in the early stages of what they think of as a journey. But companies have learned lessons that point to methods others can use to tackle challenges and achieve value lessons based on actual, real-world experience with cloud computing. Throughout their discussions, respondents made it clear that they see tremendous possibilities in the technology, and they plan to pursue those opportunities. These companies intend to move beyond today s complexity to drive value, and reshape IT into an efficient and effective service broker for the business. For more information about this research and cloud operations, visit Complexity and How to Overcome It In VMware s research, companies cited four sources of complexity in their cloud operations: Constantly changing business needs, driven by the business finding value in the cloud and wanting more. Immaturity of cloud products, which respondents said was to be expected with relatively new technology. Rapid change in product capabilities, with companies needing to stay in step with the fastpaced evolution of vendor offerings. Greater interdependency between IT silos, with the cloud requiring traditionally separate areas to work together and to be managed more holistically. In response, companies are working on three fronts to bring greater simplicity to cloud operations: The environment, with companies emphasizing thorough upfront planning, minimizing customization and the diversity of technology, buying new equipment for a clean slate and focusing on familiar use cases. People, through the creation of small, hand-picked multidiscipline cloud teams in central IT, and by offering early releases to tech-savvy users. The service delivery model, with the separation of IaaS and PaaS responsibilities between IT and the business. VMWARE WHITE PAPER / 10

11 VMware, Inc Hillview Avenue Palo Alto CA USA Tel Fax Copyright 2012 VMware, Inc. All rights reserved. This product is protected by U.S. and international copyright and intellectual property laws. VMware products are covered by one or more patents listed at VMware is a registered trademark or trademark of VMware, Inc. in the United States and/or other jurisdictions. All other marks and names mentioned herein may be trademarks of their respective companies. Item No: VMW_12Q3_WP_Cloud-Research-POV_0912_FINAL