WHITE PAPER SDN and Network Virtualization: Adapting the Network for Business Agility and Operational Proficiency Sponsored by: Microsoft Brad Casemore June 2015 IDC OPINION Cloud, mobility, Big Data (data analytics), social business, and the emerging Internet of Things (IoT) are transforming business processes and business models. Cloud, in particular, is having a major impact on enterprises and the industries to which they belong, driving sweeping technological and operational change and necessitating an overriding emphasis on business agility. The datacenter network is not immune from these forces. In fact, the datacenter network is being redefined by agility-conferring software. Traditional network architectures and operational paradigms, designed during the client/server era, no longer make sense and actually impede the technological and business imperatives of the enterprises in which they reside. Software-defined networking (SDN), a key element of the software-defined infrastructure, has emerged as an architectural answer to the agility demands of the datacenter network, bringing much-needed automation, programmability, and orchestration. Furthermore, network virtualization has enabled the network to move into closer accord with virtualized compute and storage in the datacenter, finally enabling the network to play its part in enhancing business value rather than slowing the agility and speed of key application-led business initiatives. Indeed, the business value of SDN can be substantive. Operational value is realized through extensive automation and achieving better alignment between application workloads and the network infrastructure that delivers them. Meanwhile, savings in capital expenditures result from SDN's ability to pave the way for the purchase of less costly network switches, based on commodity Ethernet merchant silicon, open network hardware, and the virtualization of higher-layer network and security services that traditionally have been provided as dedicated hardware appliances. This paper examines the challenges that SDN and network virtualization are addressing, looks at how SDN and network virtualization have evolved to meet those challenges, examines the business value and use cases of SDN in the enterprise datacenter, and provides insight on Microsoft's cloud-based approach to delivering on SDN's considerable promise. June 2015, IDC #257075
SITUATION OVERVIEW It might be a truism, but it's worth noting at the outset that networks exist to provide connectivity for business-critical application workloads. During the client/server era, traditional network architectures and their operational models fulfilled that obligation reasonably well, providing the connectivity, performance, reliability, and scalability that enterprises and their application workloads required. Now, however, with the rise of the 3rd Platform which comprises the key pillars of cloud, mobility, Big Data (data analytics), and social business the inherent limitations of traditional networking have been exposed and enterprises recognize the need for new architectural and operational models that will provide the business agility required for a new era, one in which new business models, time to value, and changing user expectations are key considerations. Specifically, as virtualization and 3rd Platform workloads proliferate, traditional architectural approaches to datacenter networking are proving to be too brittle, inflexible, and inefficient. They were designed and built for client/server applications with north-south traffic patterns between endpoints and servers, not for the growing tide of east-west traffic between servers and racks of servers, nor for an era of virtualization in which applications can be spun up in mere minutes. From a provisioning standpoint, the network must become more automated to move at the speed of virtualization. At the same time, enterprise IT departments are seeking to reduce network complexity and the costs associated with it. They also want to ensure that the network is sufficiently extensible and flexible to support hybrid cloud initiatives. Finally, there is a focus on being able to evolve the network within the current context including existing investments in infrastructure while providing new capabilities that ensure the network is ready and able to support what the future brings. Meeting the Challenges Enterprise customers know that they have to meet a number of challenges as they consider how best to adapt their networks for 3rd Platform requirements. Enterprises adopting 3rd Platform workloads, especially private cloud, quickly run up against the limitations of traditional network architectures and their operational models. Traditional networking architectures were designed for the north-south traffic patterns associated with client/server computing, not for the east-wide traffic that predominates in virtualized cloud environments. They also were not designed to provide automated VLAN provisioning, mobility, multitenant isolation, and programmatic network changes. Similarly, the manual, CLI-based, box-by-box approach to provisioning and managing traditional networks does not scale operationally in a world where virtualization, containers, and continuous software development demand much greater agility. Indeed, improving operational agility, which directly correlates to business performance and outcomes, and removing operational complexity are key objectives for enterprise IT departments adopting cloud and the other pillars of the 3rd Platform. These organizations are seeking to better align their network infrastructure with 3rd Platform workloads such as cloud and Big Data, but perhaps more important, they're seeking to ensure that any new architecture also brings an automated operational approach that is faster, simpler, more cost effective, and more scalable than the box-by-box, CLI approach to provisioning and managing networks that has ruled the datacenter until now. 2015 IDC #257075 2
This is particularly true for enterprises embracing hybrid cloud strategies, where there is an acute need for a self-service, automated, elastic approach to provisioning and managing network resources. The network must be cloud native, able to scale seamlessly from private to public clouds, and able to support workload portability and application federation. As a result, enterprises look for new architectural approaches, such as SDN and network virtualization that provide for automated provisioning, programmatic management, and seamless integration with cloud orchestration. Network virtualization is analogous to server and storage virtualization in that it is hypervisor based and provides important abstractions that allow enterprise datacenters to benefit from SDN's automated provisioning, rich programmability, and orchestration while leveraging and retaining previous investments in the network underlay (switches) upon which the software-based network virtualization overlay resides. Origins and Evolution of SDN Unlike many technologies associated with the previous client/server era, SDN initially was a customerdriven phenomenon rather than a vendor-driven product or technology. The customers that drove SDN including hyperscale datacenter operators Microsoft Azure, Google, and Facebook needed to change their networks and the way they were managed to support critical revenue-generating applications and cloud-based business models. Traditional network architectures were not designed to support the business and technological needs of these hyperscalers. In fact, these users found that the networks had become technological bottlenecks, obstacles both to progress and to sustained business success. These hyperscale operators were on the cutting edge, facing challenges and problems that were endemic to the advancement of the 3rd Platform. As the 3rd Platform increasingly makes its presence felt pervasively across enterprises in all geographic and vertical markets, enterprise datacenters are confronting challenges similar to those that were first witnessed in hyperscale datacenters. Obviously, there are differences between hyperscale datacenters and enterprise datacenters, with scale foremost among them. Other notable differences include application mix, IT resources, and the specifics associated with business models. Nevertheless, just as the 3rd Platform and principally cloud drove a need for change in hyperscale datacenter networks, it is driving a similar need for change in enterprise datacenter networks. Fundamentally, the drivers are identical. Enterprises might not respond exactly how hyperscale operators responded, but they will need solutions that address the business and technology imperatives brought to the fore by the 3rd Platform. Indeed, an IDC survey of enterprises and cloud service providers asked respondents to identify the primary motivation for adopting and considering SDN. The top enterprise response, cited by 30.9% of survey respondents, was the need for the network to possess more agility to support virtualization and cloud. Another 27.1% of respondents cited the need for the network to deliver new applications or services, while about 18% referenced the need for improved network programmability for operational efficiency (opex gains). About 13.5% pointed to a requirement for faster network provisioning to support application workloads (see Figure 1). 2015 IDC #257075 3
FIGURE 1 Enterprises Focused on Agility for Cloud, Support for New Applications Q. Which of the following factors is the primary motivation for considering or implementing SDN? Need the network to have more agility to support virtualization applications/cloud Increase ability to deliver new applications or services across the network facilitated by SDN 27.1% 30.9% Require better programmability of network for operational efficiency (opex gains) 18.0% Increase speed for provisioning the infrastructure for application workloads 13.5% Scale the network to support various and growing workloads 6.4% Use SDN to lower capex (initial investment costs) Simplify moves, adds, changes, and decommissions 2.8% 1.4% 0% 10% 20% 30% 40% (% of respondents) n = 362 Base = all respondents Source: IDC, 2015 What Is SDN, and Why Does It Matter? As noted, SDN emerged as an approach to network architecture that addresses the limitations of traditional networks, which have struggled to accommodate virtualization, cloud, mobility, and Big Data (data analytics). At a fundamental level, SDN is defined by the decoupling of the control and data-forwarding planes of a network device, with the control plane moving to a controller (software running on industry-standard server hardware) and the data plane continuing to reside in the network device. As a result of this decoupling, SDN architectures introduce abstractions that are designed to facilitate automated provisioning, programmatic management, support for automated insertion of network and security services, and integration with cloud orchestration systems. These are the means, but the ends are operational and business agility, as well as capabilities such as application portability. SDN strives to achieve business benefits in both opex and capex. It seeks to achieve the former by providing better alignment between application workloads and the network infrastructure that supports them, thereby delivering faster provisioning; centralized, programmable network configuration/reconfiguration and management; and close coupling with orchestration systems that also manage other datacenter infrastructure (including compute and storage). It aims to provide capex savings by paving the way for potentially less expensive network switches, based on Ethernet merchant silicon and network disaggregation and by providing an automated, programmatic model for virtualization of higher-layer network and security services that have traditionally been deployed as hardware appliances. 2015 IDC #257075 4
SDN Use Cases SDN remains a relatively nascent technology, and the market for the products encompassed by it is just now ramping in a meaningful way. IDC forecasts that the SDN market made up of physical network infrastructure, virtualization/control software, SDN applications (including network and security services), and professional services will grow at a compound annual growth rate (CAGR) of 89.4% from 2013 to 2018, reaching more than $8 billion at the end of that forecast period. Although the market is nascent, several notable use cases have been established. Among the leading use cases are: Automated network configuration and management (for agility) Network support for virtualization and cloud, private and public Network support for hybrid cloud for workload portability, disaster recovery (business continuity) Network virtualization to remedy "VLAN sprawl" Enhanced east-west security through microsegmentation/network slicing The benefits that accrue from these use cases, as mentioned previously, take the form of lower capex and opex costs as well as greater time to value through business agility. What's more, SDN's inherent network automation can help organizations redeploy valuable IT resources from low-value manual maintenance to higher-value roles that have a more direct impact on business outcomes. IDC's SDN survey reinforced this point, with 53% of enterprise respondents indicating that SDN offers an opportunity to redeploy networking personnel to higher-value tasks such as cloud architecture, analytics, and network virtualization (see Figure 2). FIGURE 2 SDN Offers Opportunity to Redeploy Personnel, Achieve IT Alignment Q. Regarding IT staffing associated with SDN, which of the following do you believe your organization's deployment will allow you to accomplish? Redeploy networking personnel to other tasks (analytics, automation/orchestration, network architecture, network virtualization, etc.) Align better with other IT departments/teams 39.2% 53.0% Work more collaboratively across IT department 37.0% Reduce size of network operations team 27.3% No meaningful changes anticipated 10.5% 0% 10% 20% 30% 40% 50% 60% (% of respondents) n = 362 Base = all respondents Source: IDC, 2015 2015 IDC #257075 5
Considering Microsoft's Approach to SDN As one of the earliest proponents and adopters of SDN, in its own Microsoft Azure datacenters, Microsoft has a relatively long history with SDN and its practical application to real-world technology challenges and business problems. Microsoft not only has "eaten its own dog food" with respect to SDN but also has learned how to adapt and package the technology in the form of its Azure Stack for its installed base of enterprise customers. Microsoft believes SDN is all about using software to make the network a pooled, automated resource that can seamlessly extend across cloud boundaries. According to Microsoft, benefits that accrue from this approach include optimal utilization of existing physical network infrastructure, agility and flexibility resulting from centralized control, and business-critical workload optimization from deployment of innovative network services. Toward those ends, Microsoft is embracing industry-standard protocols such as VXLAN and OVSDB for encapsulation, tunnel termination, and network configuration. As a result, the SDN solutions that Microsoft offers to its enterprise customers already have been proven at scale in its own datacenter environments. The key components in Microsoft's approach to SDN and network virtualization are Microsoft Windows Server, including Hyper-V Network Virtualization; Microsoft System Center; Azure Pack; and Microsoft Azure. Windows Server delivers Hyper-V Network Virtualization to help abstract applications and workloads from the physical network. By creating an overlay network that runs above the physical network, these virtual networks provide multitenant isolation while running over a shared physical network. Meanwhile, Windows Server, System Center, and Azure Pack provide standards-based mechanisms to automate deployment and operation of both physical and virtual networks. Specifically, System Center primarily provides fabric management, whereas Azure Pack is primarily for tenant self-service. Effectively, Windows Server, System Center, and Azure Pack work together to define and control network policies centrally and link them to application requirements. As a result, the network configuration adjusts itself automatically when applications are relocated. Windows Server and System Center deliver a built-in software gateway that helps bridge physical and virtual networks, thereby enabling workload portability between datacenters. Along those lines, in hybrid cloud scenarios, enterprises can extend their datacenter seamlessly into Windows Azure using the Windows Azure Virtual Network. In the next release, Microsoft will build upon and enhance its core set of capabilities and extend the integration of services that are used today in its public cloud Azure datacenters, thereby providing cloud-scale features and performance to enterprise datacenters. Areas of primary investment include enhanced SDN capabilities, including a new SDN controller featuring enhanced capabilities in both management and configuration. Furthermore, key updates will include support for VXLAN and OVSDB, allowing for greater interoperability with network hardware. Also on tap are orchestration and service management via System Center and the Azure Stack. The next version of the solution will also extend Microsoft's capabilities in Network Functions Virtualization (NFV). 2015 IDC #257075 6
CHALLENGES/OPPORTUNITIES Microsoft has a tremendous opportunity to provide existing and new customers with SDN and network virtualization that are well tailored to the needs of next-generation workloads and business imperatives. IDC sees SDN as networking's answer to the architectural and operational questions raised by cloud computing and the 3rd Platform, and Microsoft has a similar view of the importance of SDN in its own datacenters as well as in those of its customers. Nonetheless, the company faces challenges in bringing SDN to the enterprise. There are technical and operational hurdles that Microsoft must surmount. On the technical side, Microsoft will have to deal with the varying technology refresh of its customers. Many that have invested relatively recently in network infrastructure refreshes might be disinclined to make further near-term investments, even though they may perceive value associated with Microsoft's approach to SDN and network virtualization. Furthermore, some enterprise IT departments might have troubleshooting concerns about visibility into the physical network. To allay these concerns, Microsoft should cite its own visibility and management tools as well as partnerships with switch vendors that provide the "network underlay." On the operational side, Microsoft, like all vendors providing SDN solutions, will have to contend with the culture and structure of traditional enterprise IT departments, which often are subdivided into distinct operational silos spanning compute, storage, networking, security, and other disciplines. In dealing with these traditional IT departments, Microsoft will have to continue to devise not only higherlevel value propositions for the business as a whole but also value propositions that relate to key constituencies within the IT department. CONCLUSION The datacenter network must evolve. As we move from client/server applications to the era of the 3rd Platform, the datacenter network needs to be as automated and virtual as the application workloads and compute resources it supports. Indeed, challenges to the datacenter network are architectural and operational. SDN and network virtualization provide an answer, with new approaches that are attuned to key enterprise initiatives such as cloud, mobility, and Big Data and that afford an automated, programmable, and orchestrated network infrastructure. SDN has been proven in the world's largest datacenters and is now being adapted to provide substantive business benefits in enterprise datacenters. To that end, solutions such as network virtualization can be deployed today over enterprise customers' current network underlays (switches). These overlays support industry-standard protocols such as OVSDB and VXLAN for network configuration, encapsulation, and tunnel termination. Moreover, overlay-based network virtualization sets the stage for adoption of industry-standard, commoditized switching based on Ethernet merchant silicon and network disaggregation delivering capex benefits as well as operational agility. 2015 IDC #257075 7
About IDC International Data Corporation (IDC) is the premier global provider of market intelligence, advisory services, and events for the information technology, telecommunications and consumer technology markets. IDC helps IT professionals, business executives, and the investment community make factbased decisions on technology purchases and business strategy. More than 1,100 IDC analysts provide global, regional, and local expertise on technology and industry opportunities and trends in over 110 countries worldwide. For 50 years, IDC has provided strategic insights to help our clients achieve their key business objectives. IDC is a subsidiary of IDG, the world's leading technology media, research, and events company. Global Headquarters 5 Speen Street Framingham, MA 01701 USA 508.872.8200 Twitter: @IDC idc-insights-community.com www.idc.com Copyright Notice External Publication of IDC Information and Data Any IDC information that is to be used in advertising, press releases, or promotional materials requires prior written approval from the appropriate IDC Vice President or Country Manager. A draft of the proposed document should accompany any such request. IDC reserves the right to deny approval of external usage for any reason. Copyright 2015 IDC. Reproduction without written permission is completely forbidden.