Bringing Much Needed Automation to OpenStack Infrastructure

Transcription

1 white paper Bringing Much Needed Automation to OpenStack Infrastructure Contents Abstract 1 The Move to the Cloud 2 The Inherent Complexity of OpenStack Cloud Solutions 4 Solving OpenStack Complexity with StackIQ Cluster Manager 6 Why Clusters Are so Different 9 Summary 10 Abstract. As more companies become fluent in cloud computing, they tend to deploy more and more types of cloud workloads. As these workloads expand with greater scope and maturity, businesses can derive even greater value. Benefits can include dramatically lower costs for data center infrastructure, while increasing scalability and availability. All of this translates to faster time to market with much greater business flexibility. OpenStack has quickly emerged as the leading cloud architecture built on open source software for scalable private clouds in the enterprise. One notable challenge of OpenStack, similar to deploying Hadoop for Big Data, is the exponential curve of complexity in deploying and managing the clusters needed to build OpenStack infrastructure at scale, which can quickly become unwieldy for IT managers. OpenStack does not include a comprehensive off-the-shelf platform for managing cloud infrastructure which leads to the high degrees of complexity at scale. The need for greater efficiency and automation has become all too clear in enterprise deployments of OpenStack. For this reason, StackIQ has partnered with Red Hat to simplify the process of deploying and managing Red Hat Enterprise Linux OpenStack Platform StackIQ, Inc. All rights reserved.

2 With StackIQ Cluster Manager, data center managers can greatly reduce the time needed for large private cloud deployments and greatly increase reliability and manageability by automating the complex configuration steps for the hardware infrastructure and application stacks of heterogeneous data center environments. The Move to the Cloud The Internet has enabled a new IT architecture known as cloud computing. Online users can access their applications and data from the underlying cloud infrastructure of compute, storage, memory and networking functions. Private cloud is increasingly seen as an important option for controlling costs and business risk as corporate computing and storage requirements continue to escalate in the era of mobility, analytics, and Big Data. -- Mary Johnston Turner, IDC Research Cloud architecture has become ubiquitous today, with 94% of companies adopting some type of cloud IT strategy, according to the 2014 State of the Cloud Report, a recent survey of more than 1,000 technical professionals compiled by RightScale. This transformation to cloud is causing massive market disruptions that many leading businesses have harnessed to gain a competitive advantage. After adopting cloud computing, companies tend to deploy an increasing array of cloud workloads. As these workloads expand with greater scope and maturity, businesses achieve still greater value. Big increases in scalability and availability can lead to impressive cost reductions, along with greater flexibility to change complex business processes on the fly. The shift to OpenStack, the open source software for cloud infrastructure, is well underway in the enterprise. This advance parallels other data center software that has moved to open source software, such as Linux for server operating systems and Hadoop for Big Data. 2 Demand for private cloud IT infrastructure continues to grow rapidly as enterprise customers increasingly see cloud as the next logical step beyond datacenter virtualization, said Mary Johnston Turner, IDC research vice president, Enterprise Systems Management Software. Private cloud is increasingly seen as an important

3 option for controlling costs and business risk as corporate computing and storage requirements continue to escalate in the era of mobility, analytics, and Big Data. Costly proprietary approaches to cloud from vendors such as VMware, Microsoft and Citrix Systems have dominated the early cloud market for much of the past decade, mostly due to a lack of viable alternatives. Not so long ago, VMware controlled a majority share of the private cloud virtualization market all by itself. As the market matures and more business workloads are moved to cloud models, less expensive and more flexible choices such as OpenStack solutions are gaining greater market acceptance and traction. In fact, OpenStack now leads the market in terms of enterprises conducting initial cloud deployments and planning for further expansion, with 37% adoption, according to the 2014 State of the Cloud Report. By comparison, VMware s vcloud Director ranked second with 27%, followed by Microsoft System Center (22%) and Citrix CloudStack (18%). Out of these 4 competitors, OpenStack is also the only cloud software that showed positive growth among enterprise adoptions and implementations from 2013 to Figure 1. Private Cloud Usage 2014 vs. 2013, RightScale 2014 State of the Cloud Report 3

4 The Inherent Complexity of OpenStack Cloud Solutions As enterprise data centers transition to using commodity hardware and open source software to gain scalability, flexibility, and price-performance benefits, they have discovered that deploying and managing large-scale server environments is extremely challenging. To date, a variety of legacy management software and newer DevOps tools (e.g., Puppet, Chef) are being used. These tools handle deployment, scalability, and management functions and approach them in different ways that add complexity to the environment. Most of the tools manage changes to server configurations and other fixes through software scripts. Making changes to one appliance or several appliances through scripting is a manual, time-intensive, error-prone process. It can leave the administrator unsure if the changes have been implemented throughout the cluster. Attempts at using proprietary software solutions from EMC, Teradata, Oracle (Big Data), and VMWare, Citrix Systems, Microsoft (Cloud), and others, has proven very expensive, and these solutions often do not scale to meet customer needs. Although OpenStack has quickly emerged as the leading choice to build massively scalable open source cloud environments based on its inherent flexibility and affordability, the enormous amount of configuration required to deploy and run OpenStack at scale can quickly become unwieldy for data center managers. This complexity is due, in large part, to the requirement to use clusters, which are one of the key building blocks of the infrastructure. The extra work in building an open source cluster stems from the physical infrastructure layer, where engineers need to set up hundreds of configuration parameters before even starting the installation process. Then the software needs to be provisioned on the management node. Then, the management node needs to be configured to deploy and configure the operating system on the back-end nodes. 4 In addition, every cluster requires a distributed file system, which must be prebuilt and pre-configured. Different partitions are needed for the operating system, along with private and public network configurations running on OpenStack nodes. Thus, the overall OpenStack infrastructure is deployed and managed by a collection of disparate products, policies, and procedures, which can lead to unpredictable

5 and unreliable environments. Similar to Hadoop, OpenStack does not include a comprehensive off-the-shelf management platform for its infrastructure hence the high level of difficulty for users. The need for greater efficiency and automation is obvious in enterprise deployments of OpenStack. StackIQ has developed its cluster manager software suite to simplify the necessary management and deployment of these systems. StackIQ Cluster Manager plays a key role in helping enterprise customers automate the deployment and management of Red Hat Enterprise Linux OpenStack Platform. -- Mike Werner, Red Hat Users have greatly improved their time to value by automating the multiple deployment and management steps with StackIQ Cluster Manager. Users have cut the time to implementation from weeks or even months down to just hours. StackIQ Cluster Manager plays a key role in helping enterprise customers automate the deployment and management of Red Hat Enterprise Linux OpenStack Platform, said Mike Werner, Senior Director, Global Ecosystems, Red Hat. The Limitations of Siloed DevOps Tools Current DevOps tools were designed for general data center use to push out code faster. They are not sufficient for deploying and managing OpenStack infrastructure at scale in a production environment. StackIQ was specifically designed to deploy and manage clusters in a production environment based on the need to cover the entire infrastructure stack from bare metal up through the application layer. The StackIQ framework can incorporate operations previously done with DevOps tools like Puppet, so no previous work is throw away, but the only way to truly automate OpenStack infrastructure is to utilize a platform that controls the entire stack from bare metal up. 5

6 Solving OpenStack Complexity with StackIQ Cluster Manager StackIQ Cluster Manager for OpenStack is a complete, integrated OpenStack management solution for enterprise customers. For the first time, enterprises get everything they need to deploy and manage OpenStack clusters throughout the entire operational lifecycle. StackIQ joined the ranks of global technology leaders collaborating to help establish Red Hat Enterprise Linux OpenStack Platform as the standard for private cloud in the enterprise. StackIQ Cluster Manager simplifies the process of deploying Red Hat Enterprise Linux OpenStack Platform. RED HAT OPENSTACK CLOUD INFRASTRUCTURE & STACKIQ StackIQ Cluster Manager Configure, Deploy, Manage StackIQ Automatically Installs a Server as a Foreman Appliance StackIQ Automatically Installs All OpenStack Requirements Ingests RHEL OpenStack Platform ISO as StackIQ Module StackIQ Automatically Configures RHEV StackIQ Automatically Installs RHEL Foreman Appliance StackIQ Puppet Orchestration (e.g., Removing Keys on Re-Install) Swift Ingests and Configures Red Hat Enterprise Linux OpenStack Platform ISO Sets Up Red Hat Enterprise Virtualization Other Neutron Installs Red Hat Enterprise Linux 6.5 ISO Figure 2. StackIQ Cluster Manager integration with RedHat Enterprise Linux OpenStack Platform StackIQ Cluster Manager handles the automatic bare metal installation of heterogeneous hardware and the correct configuration of the multiple networks required by OpenStack. StackIQ Cluster Manager also automatically deploys a Red Hat Foreman software appliance onto a server of your choice, which enables webbased driven cluster configuration on deployed nodes of OpenStack software. 6 All OpenStack services are available for management and deployment via the OpenStack Dashboard, including Nova, Neutron, Cinder, Swift, etc. StackIQ Cluster Manager enables the ongoing management, deployment, and integration of Red Hat OpenStack infrastructure on growing and multi-use clusters.

7 Figure 3. StackIQ Cluster Manager and OpenStack hardware architecture StackIQ Cluster Manager manages all of the software that sits between bare metal and a cluster application, such as OpenStack. A dynamic database contains all of the configuration parameters for an entire cluster. This database is used to drive machine configuration, software deployment (using a unique Avalanche peer-topeer installer), management, and monitoring. The combined system fulfills multiple functions, including: Provisions and manages the operating system from bare metal Configures host-based network settings throughout the cluster Captures hardware resource information (such as CPU and memory information) and uses this information to set cluster application parameters Captures disk information and uses this information to programmatically partition disks across the cluster Installs and configures a cluster monitoring system Provides a unified interface (CLI and GUI) to control and monitor all these activities 7

8 The StackIQ Cluster Manager for OpenStack is based on StackIQ s open source Linux cluster provisioning and management solution. The solution is based on Rocks, first developed in 2000 by researchers at the San Diego Supercomputer Center at the University of California, San Diego. Rocks was initially designed to enable end users to easily, quickly, and cost-effectively build, manage, and scale application clusters for High Performance Computing (HPC). Thousands of environments around the world now use Rocks. Key Benefits of StackIQ Cluster Manager for OpenStack Faster time to deployment Automated consistent deployment Simplified operation that can be quickly learned without system administration experience Reduced downtime due to configuration errors Choice of stack components (including leading Hadoop distributions and OpenStack) for a tailored solution Several major telecom companies are working at the forefront of the shift to OpenStack, including AT&T and Comcast. An early adopter is Alcatel-Lucent, which has deployed Red Hat s OpenStack Platform with StackIQ Cluster Manager for its Network Functions Virtualization (NFV) solution, CloudBand. Alcatel-Lucent has abstracted the functionality out of OpenStack especially for its telco carrier clients. The goal is to deliver preconfigured OpenStack clusters to match each client s unique IT requirements. Alcatel-Lucent uses StackIQ s Cluster Manager to build the telco software functionality above the different permutations of hardware used by each specific client. Automation and carrier-class reliability have been key principles that we ve embraced to ensure that our Network Functions Virtualization (NFV) management platform, CloudBand, meets our customers needs, said Dor Skuler, Vice President and General Manager, CloudBand Business Unit, Alcatel-Lucent. We have delivered on this promise by leveraging a combination of Alcatel-Lucent technologies and best-in-class partners like StackIQ.

9 Some leading global telecom operators have already adopted the Alcatel-Lucent CloudBand technology, including T-Mobile, Telefonica and NTT. This growing adoption has helped create industry confidence that OpenStack is not just enterprise-grade quality it is telco-grade quality. Why Clusters Are So Different Research at the San Diego Supercomputing Center revealed the fundamental lesson that spawned the development of Rocks clusters behave differently than singlepurpose servers. Whether clusters are configured for HPC, Hadoop, or OpenStack, they all require a different set of tools than traditional data centers. For every organization that operates a cluster with traditional data center tools, there is a Pain Curve (see diagram). It is difficult to quantify the number of servers required to reach the pain threshold, after which the infrastructure is unable to keep up with increased complexity and things start breaking down. Slower delivery times give way to degraded services, until the whole system finally collapses in production outages and failure. Single-purpose servers have no awareness about the workloads of any other machines, but the worldview of a server in a cluster is vastly different. By definition, each server in a cluster must be aware of every other server in the cluster. This is so the servers in the cluster can collectively accept external requests, process those requests, and then respond to the request as a team. To accomplish this, they must be configured precisely the same way from bare metal up. That s why adding a single node to such a system increases the complexity of the whole system exponentially every time. Most data center administrators can keep this under control for their traditional enterprise applications, which have linear complexity as they scale (yellow line in the diagram). However, cluster environments like OpenStack have an exponential complexity curve that can quickly become unmanageable (blue curve in the diagram). 9

10 Figure 4. The Pain Curve The solution to this exponential complexity is the implementation of intelligent automation. Automation enables clusters to scale without manual scripting or server configuration. Adding a single server to a cluster requires a reconfiguration of every pre-existing server if done manually, this can become the most costly bottleneck for an IT team trying to manually manage their cluster. StackIQ s Cluster Manager alleviates this bottleneck from the beginning by leveraging intelligent, enterprisegrade automation from bare metal to the applications layer. Summary With a constantly growing stream of data and more complex application requirements all the time, IT teams must be able to move faster to deploy, change and manage their infrastructure. More companies are building on OpenStack s cloud infrastructure to keep up with these demands, while simultaneously reducing cost. OpenStack has higher rates of adoption among enterprises in 2014 than VMware s vcloud Director, Microsoft System Center, and Citrix CloudStack. 10 As this fast-growing trend takes hold, StackIQ Cluster Manager for OpenStack offers

11 several key benefits, including faster time to deployment than most proprietary solutions. In addition, StackIQ provides automated, dependable management software for ongoing maintenance; reduced downtimes due to configuration errors; and a lower total cost of ownership for OpenStack clouds. This open, modular architecture gives enterprises much greater flexibility and agility to manage their private cloud infrastructures. Early adopters of OpenStack moving from proofs-of-concept in labs to full-scale deployment are finding that they lack the tools, processes, and procedures to deploy and manage these systems efficiently. For reliable, predictable, simplified, automated OpenStack enterprise deployments, StackIQ has created StackIQ Cluster Manager for OpenStack infrastructure. StackIQ has partnered with Red Hat to simplify the process of deploying Red Hat Enterprise Linux OpenStack Platform. This best of breed solution combines the expertise of Red Hat in community driven innovation with StackIQ s leading cluster management solution. StackIQ Cluster Manager makes it easy to deploy and manage Red Hat Enterprise Linux OpenStack Platform at scale and its automation, powerful features, and ease of use lower the total cost of ownership of scalable private clouds. About StackIQ StackIQ is a leading provider of software that automates the deployment and management of Big Infrastructure. Based on open source Rocks cluster-management software, StackIQ simplifies the installation and management of the hardware and software that provides the infrastructure for large-scale environments with tens, hundreds, or thousands of servers supporting Big Data, Cloud and HPC. info@stackiq.com 11 StackIQ and the StackIQ Logo are trademarks of StackIQ, Inc. in the U.S. and other countries. Third-party trademarks mentioned are the property of their respective owners. The use of the word partner does not imply a partnership relationship between StackIQ and any other company.