Research Report on Virtualization and Cloud Computing in Test/Dev Environments Platform Computing www.platform.com 1
Background: Methodology Objective: Examine the challenges of managing Test/Dev environments and the opportunity for server virtualization and cloud computing to address them Fielded: Jun-Aug 2009 Method: Telephone surveys Sample: IT architecture and operations managers responsible for development and test infrastructure (N=83), at North American firms, cross-industry. SMB: n=57 (<500 Test/Dev servers) large enterprise: n=25 (>500 Test/Dev servers) Fielding and data analysis support by third party vendors Fielding: Primary Point (http://www.primarypoint.com/) Survey development and analysis support: Taneja Group (http://www.tanejagroup.com/) 2
Key Findings: The Case for Private Cloud 1. Significant investment is being allocated to solve Test/Dev operational and budgetary challenges: 50% of respondents will have a funded project start in 2009, while 66% of respondents will have a funded project started by end of 2010 2. Application teams are using shared infrastructure in production today, supporting the theory that Cloud Computing is more about a new business model for IT delivery than new technology 3. Few firms have adopted a hosted environment outside the firewall. This helps to explain why many firms are starting with private cloud; get the business model right before introducing additional complexity involved with transitioning to external infrastructure 4. The top challenge identified in managing Test/Dev environments: the need to manage virtual and physical resources separately 5. Experienced users confirmed that server virtualization is an enabler, but adds several layers of control and cost issues that must be addressed by sharing, process, workflow, and other management capabilities 3
Industry Breakdown Industry Breakdown: ALL Industry Breakdown: LARGE (>500) S/W 18% Pharma 1% Health 8% Telco 14% Other 2% Manuf 17% Banking 20% Gov 18% S/W 28% Telco 12% Other 4% Manuf 16% Banking 32% Gov 8% Takeaway: Broad industry representation. Banking and Software segments dominate the Large category with a combined 60% of respondents. 4
Size of Test/Dev Environment Number of Physical Test/Dev Servers Large > 500 31% SMB < 500 69% 40 35 30 25 20 15 10 5 0 Distribution Takeaway: Broad coverage of infrastructure scale. Banking accounts for 40% of the environments that are greater than 1000 servers. 5
Prevalence of Virtualization in Test/Dev Does your company use virtualization in Test/Dev environments? If Yes: Approximately what percentage of physical hosts are supporting virtualization? No 13% 0-25% 25-50% 14 39 Yes 89% 50-75% 75-100% 8 9 53% of respondents have virtualized less than 25% of their physical servers Don t know/refused 3 Takeaway: While virtualization is used heavily in Test/Dev environments, there remains a significant amount of work being done on non-virtualized infrastructure. Server virtualization prevalence matches industry estimates of 20-30% market penetration. 6
Hypervisor Vendor Prevalence in Test/Dev What hypervisor vendors are you using? Average Number of VMs per virtual host? VMware 80 1-10 20 Microsoft Citrix 18 15 65% Homogeneous 35% Heterogeneous Average: 20 11-20 9 Other 16 > 20 13 Takeaway: VMware prevalence is 95%, but 35% run at least one other hypervisor. Consolidation ratios: 20/host on average is high (with respect to production server virtualization consolidation ratios), indicating dense, highly shared environments and significant contention for limited, shared physical server resources. 7
Objectives: Resource Sharing Determine the types of resources currently shared Determine use cases for shared resources Explore prevalence of internal (private cloud) sharing and external (hosted, outsourced, public cloud) sharing Establish current views of limitations and challenges in Test/Dev for both internal shared and external hosting
Resource Sharing in Test/Dev Does your company share servers in Test/Dev environments? If Yes, do you share servers between the SDLC phases (Dev, Test, UAT)? No 8% No 14% Yes 92% Yes 86% Takeaway: Development teams have a high degree of comfort using shared infrastructure. Of the small group that don t (8%), most plan to within 2 years. Teams share across nearly all phases (86% share across at least Dev, Unit Test, Integration Test & Performance Test phases). 9
Resource Sharing for Other Activities Do you currently, or plan to, use your Test/Dev environment for other activities? If Yes, What activities? Training 53 No 31% Yes 69% Demos, Prototyping 47 Disaster Recovery 4 Takeaway: Sharing is prevalent, even for non-test/dev activities. Disaster Recovery is likely an emerging use case, based on separate Taneja Group research. 10
Use of Outsourced Infrastructure Do you use off-premise or hosted servers for Test/Dev (outsourcing)? Lack of Control If No: Why Not? 24 Yes 18% Immature Technology We Don't Have the Need 13 19 No 82% Lack of Security Vendors are Too Small 3 10 Too Expensive 3 No Time 2 Takeaway: Few firms have adopted a hosted environment outside the firewall. Lack of Control and Immature Technology are the top functional blockers, but a significant number have not been effectively sold on benefits ( We Don t Have Need ) 11
Use of Outsourced Infrastructure Do you use off-premise or hosted servers for Test/Dev (outsourcing)? If Yes: What percentage of your Test/Dev servers are currently outsourced? No 82% Yes 18% Less Than One Quarter 71% More Than One Quarter 29% Takeaway: Of those who have outsourced, 71% have outsourced less than 25% of their Test/Dev servers. Only a few vendors were named: Amazon (3), Savvis (2), Skytap (1), Surgient (1), ADP (1) 12
Sharing Resources in Production Are you using shared infrastructure or a private cloud for Production enterprise applications? If Yes: Do Production applications share the same or different resource pools as Test/Dev? No 24% Yes 76% Different Pools 70% Same Pools 30% 13 Takeaway: Application teams are using shared infrastructure in production today, supporting the theory that Cloud Computing is more about a new business model for IT delivery than new technology. 30% of shared pools host both Test/Dev and Production applications, indicating a rising comfort level with sharing overall. Taneja Group experience confirms emergence of Test/Dev/Prod sharing.
Shared Production Applications Top 3 Shared Production Applications If Yes, what would you say are your top three Production applications using shared infrastructure or private cloud? CRM Finance Web ERP Intranet HR Business Critical System Software Web 2.0 2 3 8 8 13 15 19 20 21 Takeaway: Supporting the virtualization trend, enterprise applications that are within the firewall continue to be the primary production applications that are sharing resources. Business critical applications are at the early stage of moving to a shared environment. 14
Sharing Resources in Production Are you using shared infrastructure or a private cloud for Production enterprise applications? If No, do you know if the company has any plans to implement shared or private cloud for Production? Yes 76% No 24% No 45% Yes 55% 70% of those with plans to share in Production will do so in the next 12 months Takeaway: Of the 24% who are not sharing, 70% have plans to do so in the next 12 months (70%, N=7/10). Top apps planned to share resources: ERP, Finance, CRM, Web 15
Types of Applications Being Developed Across your enterprise Test/Dev environments, what kinds of applications are being developed? Web Intranet Finance Business Critical System Software Web 2.0 CRM HR ERP 28 30 30 35 34 38 42 45 50 Takeaway: Broad mix, focused on web/intranet, finance, and business critical (specific to the industry/business). Finance and Web apps are also in the top 3 types of apps currently shared in production. 16
Objectives Test/Dev Challenges Determine the types of user and IT support challenges Focus on server & virtual machine lifecycle management issues (vs. SDLC management) Explore current projects, future planning, and funding to address Test/Dev challenges
Test/Dev Operations Challenges Please rank the impact of these operational challenges in your Test/Dev environment: Med/High Impact No/Low Impact IT manages virtual and physical resources separately Sharing resources among teams is difficult IT does too much configuration work Users complain about lack of control Users wait too long for resources Users are not returning machines to the pool Users do too much configuration work 55% 47% 46% 45% 42% 36% 33% 45% 53% 54% 55% 58% 64% 67% Takeaway: Top challenge: the need to manage virtual and physical resources separately. The top three challenges relate to lack of cross-domain management, server lifecycle management, and lab automation. 18
Existing Projects to Address Challenges Do you currently have any projects around any of these challenges? If Yes: Which ones? (Top Five) IT spends too much time on configuration Users wait too long for resources 12 13 No 59% Yes 41% Users spend too much time on configuration Managing virtual and physical separately 11 11 Sharing environments is difficult 8 Takeaway: Top projects focus on configuration (both IT and user configuration efforts), the need for self-service (wait too long), and better environment sharing across the software development lifecycle. 19
Planned Projects to Address Challenges If No, do you currently have any planning around these challenges? If Yes: Which ones? (Top Five) IT spends too much time on configuration Users spend too much time on configuration 6 6 No 67% Yes 33% Users wait too long for resources Sharing environments is difficult 5 5 Managing virtual and physical separately 4 Takeaway: Delivering good configurations to Test/Dev teams quickly, and allowing them to recreate and share them with a minimum of configuration, are the focus of most current and planned projects. 20
Operations Impact: Time Lost 87% Claim User Time Is Lost Due To Ops Challenges. How Much? 89% Claim IT Support Time Is Lost Due To Ops Challenges. How Much? >25% 8% >25% 19% 19% say more than 25% of time is lost > 10% < 25% 42% <10% 50% > 10% < 25% 30% <10% 51% Takeaway: Less time is lost by Test/Dev users than by IT support, but for both the loss is significant (50% agree that more than 10% of both user and admin time is lost due to challenges). 21
Test/Dev Budget Challenges Please rank the impact of these budgetary challenges in your Test/Dev environment? Med/High Impact No budget to add more resources No/Low Impact 59% 41% 22% Have Projects Underway Around Budgetary Challenges Struggle to track utilization 51% 49% Unable to charge BU's for what they use 51% 49% Takeaway: Significant recession effect (highest impact of all challenges) is likely retarding the market for Test/Dev solutions more than any other factor in 2009. Many respondents indicated they would have explored a solution except for time/budget. 22
Cost Analysis of Lost Time Data: Dev Team: 87% agree time is lost Of these, 50% say <10% lost, 42% say 10-25%, and 8% say > 25% IT Team: 89% agree time is lost Of these, 51% say <10% lost, 30% say 10-25%, and 19% say > 25% Cost Analysis: For an environment with 100 developers/testers, with 5 admins 50% will lose at least $1.5M per year in developer time 49% will lose at least $50K per year in IT admin time Assumptions: Fully-burdened developer at $150K per year Implication: 50% of respondents lose more than $15K per developer per year Fully-burdened admin at $100K per year Implication: 30% of respondents lose more than $10K per admin per year Developer/Admin Ratio = 10/1 (conservative) 49% of IT staff distribution is in application development and support, and 14% of IT staff distribution is in the data center * Takeaway: Lost developer/tester time is a much bigger challenge and cost issue than lost IT admin time * Source: Gartner IT Spending and Staffing Report, 2009 23
Objectives The Solution Opportunity Determine familiarity and impressions of current on-premise and hosted lab automation and lifecycle management solutions Explore existing projects and planning around solution evaluation Estimate budget size
Use of Test/Dev Provisioning Tools Which provisioning tools are you using? Other/ Homegrown 27% None 34% 61% Have Homegrown Tools or No Tools. Significant Lack of Investment to Date in Test/Dev Provisioning Software VMware 11% BMC 11% HP/ Opsware 17% Takeaway: There is a clear lack of vendor penetration for automated provisioning tools in Test/Dev environments, which explains the high level of reported challenges and the lack of experience with lifecycle or lab automation (higher-order mgmt tools). 25
Effectiveness of Virtualization Will server virtualization on its own address the most important challenges in your Test/Dev environment? Representative Why Not? Answers: Yes 28% No 72% Doesn't take in to account integration and interoperability issues these are the real challenges. Need a good service-based orientation managing business services not hardware. You still need process automation; virtualization is just an enabling technology. Our biggest challenges in Test/Dev are the same whether the servers are virtualized or not. Our biggest problem now is getting the configuration[s] right. 26 Takeaway: Virtualization on its own was expected to solve the challenges only by those respondents new to using the technology. The majority of experienced users confirmed that virtualization is an enabler, but adds several layers of control and cost issues that must be addressed by process, workflow, sharing and other management capabilities.
Existing Solution Experience Have you evaluated or tested any lab automation or virtualization lifecycle management software? VMware Which ones? 27 Surgient 4 No 69% Yes 31% VMLogix 2 40% have not evaluated due to Lack of Time or Budget Microsoft Enigmatec 2 2 Takeaway: VMware Lab Manager dwarfs other solutions, but budgetary issues have stalled a large percentage of evaluations. Few respondents indicated functional problems with lab management solutions. 27
Funding to Address Challenges Do you have a currently funded project to build or purchase Test/Dev shared infrastructure solutions? If Yes: can you tell us the approximate budget amount? $250,000 + No 58% Yes 42% 100,000-$250,000 50,000-$100,000 0-$50,000 Don't Know/Refused 0 5 10 15 20 Takeaway: Significant investment is being allocated to solve Test/Dev operational and budgetary challenges. 28
Funding to Address Challenges Do you have a currently funded project to build or purchase Test/Dev shared infrastructure solutions? If No, do you have one planned to start this year or next year? Yes 42% No 58% Yes 38% No 62% Takeaway: 38% of respondents that are not investing to solve Test/Dev challenges today plan to do so in the next two years. 50% of respondents will have a funded project start in 2009, while 66% of respondents will have a funded project started by end of 2010. 29
Additional Industry Segment Analysis 30
Industry Segment Consistency Virtualization Adoption Size of environments (Banking outlier) Number of physical hosts supporting virtualization Use of virtualization (Manuf outlier) Hypervisors used Sharing of Resources Use of shared servers (SW and Health outlier) Sharing between stages of SDLC Using shared servers in production (Telco outlier) Use Test/Dev for other non-prod work, i.e. demos (Gov and Health outliers) Application types developed Solution Opportunity Number that have evaluated lab automation (SW and Health outliers) Number that have current funding (Banking outlier) Planned funding for this year 31
Industry Segment Outliers Banking Heavier outsourcing: 31% (vs. 18% avg.) Little sharing between Test/Dev and Production: 11% (vs. 30% avg.) Large: accounts for 40% of the environments that are > 1000 hosts High projects around operational challenges: 69% (vs. 41% avg.) 15% believe virtualization is enough to solve problems (vs. 28% avg.) 62% have current funding (vs. 42% avg.) Government High sharing between Test/Dev and Prod: 56% (vs. 30% avg.) High use of Test/Dev for other non-prod work, i.e. demos: 91% (vs. 69% avg.) Low projects around operational challenges: 18% (vs. 41% avg.) More believe virtualization is enough to solve problems: 55% (vs. 28% avg.) High number with planned funding for next year: 50% (vs. 27% avg.) 32
Industry Segment Outliers (cont.) Manufacturing Only 64% use virtualization (vs. 90% avg.) No outsourcing: 0% (vs. 18% avg.) No sharing between Test/Dev and Prod: 0% (vs. 30% avg.) High number with planned funding for next year: 71% (vs. 27% avg.) Software Only 70% share servers (vs. 92% avg.) Lighter outsourcing: 10% (vs. 18% avg.) No sharing between Test/Dev and Prod: 0% (vs. 30% avg.) Low projects around operational challenges: 20% (vs. 41% avg.) High number have tested lab automation: 50% (vs. 31% avg.) 33
Industry Segment Outliers (cont.) Telecommunications Heavier outsourcing: 36% (vs. 18% avg.) Lighter shared servers in production: 55% (vs. 76% avg.) High sharing between Test/Dev and Prod: 83% (vs. 30% avg.) Low projects around operational challenges: 9% (vs. 41% avg.) High number with planned funding for next year: 50% (vs. 27% avg.) Health Care Only 57% share servers (vs. 92% avg.) No outsourcing: 0% (vs. 18% avg.) No sharing between Test/Dev and Prod: 0% (vs. 30% avg.) Low use of Test/Dev for other non-prod work, i.e. demos: 29% (vs. 69% avg.) High number have tested lab automation: 71% (vs. 31% avg.) No planned funding for next year: 0% (vs. 27% avg.) 34
For more information Contact Martin Harris, Director Product Management, mharris@platform.com Or visit http://www.platform.com/company/contact-us 35