1 Simplifying Big Data Deployments in Cloud Environments with Mellanox Interconnects and QualiSystems Orchestration Solutions
2 64% of organizations were investing or planning to invest on Big Data technology Gartner, 2013 Executive Summary New advances in Big Data applications are enabling analysts, researchers, scientists and engineers to run more complex and detailed simulations and analyses than ever before. These applications deliver game-changing insights, bring new products to market and place greater demand on existing IT infrastructures. This ever-growing demand drives the need for instant access to resources compute, storage, and network. Yet one of the challenges for engineering teams is to achieve agility and user empowerment in the deployment of Big Data infrastructure. Mellanox and QualiSystems have partnered to offer a joint solution leveraging Mellanox s CloudX architecture and QualiSystems CloudShell infrastructure orchestration solution. This collaboration delivers provisioning and adaptive web catalog-driven automation that delivers faster validate deployments and easier, more user-empowered Big Data selfservice offerings. The Need for Speed - in Both Performance and in Ease of Deployment Quali's orchestration and automation solutions ease the deployment of complex Big Data infrastructure and applications. According to the Gartner 2013 Big Data Adoption Survey, 64% of organizations were investing or planning to invest in big data technology, led by media and communications, banking and services organizations. Big data investments are being prioritized to enhance customer experiences and the increase process efficiency. These priorities are driving the need for speedy performance. One of the key challenges in deploying this infrastructure is that it requires a lot of expertise to integrate and validate the various components to ensure the desired performance outcome. In addition, customers are looking to deploy their in private and hybrid cloud environment to take advantage of resources capacity planning and high availability. The need for speed and ease of deployment on top of top of the line infrastructure performance is the reason for Mellanox s collaboration with QualiSystems. Mellanox s CloudX architecture has been designed to deliver the required networking and system efficiency capabilities related to bandwidth, latency, offloads, and CPU utilization for Big Data analytics and unparalleled performance capabilities for cloud overlay networks. QualiSystems orchestration and automation solutions ease the deployment of complex, Big Data infrastructure and provide a platform for user-friendly infrastructure selfservice. Quali s non-programmer oriented automation design tools make it possible for the whole IT and engineering team to effectively build and sustain an automation practice, providing the agility to meet ever growing Big Data and cloud infrastructure requirements.
3 The Mellanox CloudX Solution The CloudX OpenCloud architecture helps to reduce the cost of using compute and storage infrastructures CloudX is a recipe, based off the Mellanox OpenCloud architecture, for building an efficient cloud platform using off-the-shelf building blocks: servers, storage, interconnect and software to form flexible and cost-effective private and public clouds. Limitless cloud performance On-demand compute, storage and networking OpenCloud Architecture: Open Platform / Open Source CloudX can be based on top of an open-source environment, such as OpenStack, that enables the installation, provisioning, diagnostics, and maintenance of the cloud in the data center. CloudX helps to reduce the cost of using a compute and storage infrastructure and it enables applications to use much more data at real-time to get conclusions and make business decisions faster. Mellanox adapters, switches, cables, software and specific acceleration technologies implement the world s fastest and most robust end-to-end networking solutions for a complete, high-performance Cloud and Big Data analytics infrastructure. These capabilities ensure optimum data center performance with: Up to 56Gb/s fabric speed to support servers (or blade servers) and storage connectivity in any configuration As low as one microsecond application latency, leveraging remote direct memory access (RDMA) between servers and storage RDMA-based solutions that speed transaction processing by offloading significant portions of a communication job from the server CPU and operating system to the IO controller The QualiSystems Solution CloudShell DevOps self-service orchestration and automation platform helps to deliver agile IaaS Quali s lead solution, CloudShell, is a DevOps self-service orchestration and automation software platform for heterogeneous, multi-generational IT infrastructures and networks. CloudShell helps infrastructure teams to deliver agile, end-to-end infrastructure and platform services to end users with much higher ease of use and speed of deployment. CloudShell/CloudX Portal Login
4 CloudShell implements the following key characteristics Centralized inventory management: Engineers have visibility to any components needed to design and publish Big data infrastructure stacks and networked topologies. OOTB libraries and open resource interface creation: CloudShell provides out of the box libraries for many common infrastructure components, but also provides an easy to use tool to create new drivers for interfaces that require customization. CloudShell provides a robust platform for orchestrating diverse infrastructure Object-based automation paired with GUI tools CloudShell implements an object-oriented approach to automation that contrasts with creating fragile, monolithic automation scripts. All automation elements including resources, provisioning interfaces, and automation tasks (such as running a performance test) are captured as small-scope objects. CloudShell s object-oriented approach offers a number of advantages: The limited scope of automation objects means that they are easy to capture, maintain, and refactor to meet the requirements of a changing network environment. A shared library of resource, provisioning and testing objects can be maintained in a systematic fashion. Automation objects can be tagged with arbitrary labels so that they can be easily searched and leveraged by many users from a shared library. An object library optimizes the skills of programmers, who can maintain the shared library as a high quality service to the rest of the network engineering team The object library can be leveraged by non-programmers using GUI-based, drag and drop-style network topology design and automation workflow tools. This maximizes the productivity of the whole network engineering team, especially as topologies and workflows are shared and reused by multiple users.
5 Mellanox s CloudX architecture delivers the required networking and system efficiency capabilities related to bandwidth, latency, off-loads, and CPU utilization for Big Data analytics. CloudShell offers non-programmer friendly GUI automation authoring tools CloudShell s self-service portal can automate validation such as the Flexible IO (FIO) benchmark tests GUI tools such as an automation workflow authoring tool allow non-programmers to assemble objects into a workflow that can be packaged and published to serve as autoprovisioning and auto-reclamation sequences for infrastructure environments. Self-Service Portal CloudShell provides a powerful self service catalog structure for creating, publishing and consuming infrastructure environments CloudShell provides a powerful web portal that offers an adaptive self-service catalog structure for creating, publishing and consuming infrastructure environments. Template-based environments are adaptable so that a single environment definition can accept a wide variety of user-defined parameters. These parameters can drive autoprovisioning and reclamation workflows that can include installing and configuring a variety of Big Data applications and services. Per-environment automation commands can also be published to allow users to validate performance before deploying and running workloads. CloudX Infrastructure Orchestration For Big Data Applications The following section provides the IT manager with a reference architecture for building a private cloud solution for Big Data. The solution utilizes white box hardware built around the fastest interconnect technology of Mellanox 10Gb/s, 40Gb/s and 56Gb/s Ethernet, in combination with cloud orchestration software to provide a pre-integrated, optimized and certified configuration that is ready for production deployment.
6 Mellanox s Openstack Neutron plug in allows hardware to have unique connectivity, security and QoS attributes (4) Management and control Nodes Single CPU, 8+ cores 32GB DRAM 4 HDDs, 2TB each At least (5) Compute Nodes Dual CPU, 8+ cores 128GB DRAM GB SSD At least (5) Storage Nodes Dual CPU, 8+ cores 64GB DRAM 12+ 1TB 10K RPM HDDs GB SSD (Two SSD drives will be used for Boot/Swap) It is recommended to keep the HDDs to SSDs ratio at or lower than 4:1, SSD are used as the write caching layer for the storage solution and provide the needed transaction and throughput capabilities to the storage solution. Networks Hardware-based switching provides better performance, functionality and security of virtual cloud environments Mellanox SX1036B 10/40/56Gb Ethernet switch Mellanox ConnectX-3 Pro Dual Port, 10/40/56Gb Ethernet NIC, MCX314A-BCCT For Cloud orchestration use your choice of hypervisor and management software. Big Data applications, such as Open Source Hadoop, can utilize guest OS based on Linux to help controlling CAPEX and OPEX. Mellanox s Openstack Neutron Agent Mellanox s OpenStack Neutron plugin supports Mellanox s embedded switch functionality as part of the VPI (Ethernet/InfiniBand) HCA. Mellanox s Neutron Plugin allows hardware vnics (based on SR-IOV virtual functions) per each Virtual Machine vnic to have its unique connectivity, security, and QoS attributes. Hardware vnics can be mapped to the guest VMs through para-virtualization (using a Tap device), or directly as a Virtual PCI device to the guest, allowing higher performance and advanced features such as RDMA. Hardware-based switching, provides better performance, functionality, and security/ isolation for virtual cloud environments. Future versions of the plug-in will include OpenFlow API to control and monitor the embedded switch and vnics functionality This plugin is implemented according to Plugin-Agent pattern. Mellanox s OpenStack Neutron Plugin implements the Neutron v2.0 API. Mellanox s Neutron Plugin processes the Neutron API calls and manages network segmentation ID allocation. The plugin uses databases to store configuration and allocation mapping.
7 The need for speed and ease of deployment on top of the line infrastructure performance is the reason for Mellanox s collaboration with QualiSystems The plugin maintains compatibility to Linux Bridge Plugin, supports DHCP and L3 Agents by running L2 Linux Bridge Agent on Network Node. Mellanox s OpenStack Neutron Agent (L2 Agent) runs on each compute node. Agent should apply VIF connectivity based on mapping between a VIF (VM vnic) and Embedded Switch port. Mellanox s ML2 Mechanism Driver implements the ML2 Plugin Mechanism Driver API. This driver supports Mellanox s embedded switch functionality as part of the VPI (Ethernet/InfiniBand) HCA. Mellanox s ML2 Mechanism Driver provides functional parity with Mellanox s Neutron plugin. Mellanox s ML2 Mechanism Driver supports DIRECT (pci passthrough) and MACVTAP (virtual interface with a tap-like software interface) vnic types. Hardware vnics mapped to the guest VMs allow higher performance and advanced features such as RDMA (remote direct memory access). Conclusion Deploying a cloud infrastructure with QualiSystems Cloudshell and Mellanox interconnect provides the IT manager with a smooth installation and maintenance tools. Application deployments and testing is simplified with QualiSystems self service portal and centralized management solutions. With Mellanox s high-performance interconnect, customers enjoy the highest performing cloud deployment. Mellanox s offload engines guarantee customers will be able to take advantage of their investment in highly performing CPUs and vast amounts of storage and memory.
Microsoft System Center 2012 R2 Why Microsoft? For Virtualizing & Managing SharePoint July 2014 v1.0 2014 Microsoft Corporation. All rights reserved. This document is provided as-is. Information and views
Proven Infrastructure Guide EMC VSPEX PRIVATE CLOUD VMware vsphere 5.5 for up to 1,000 Virtual Machines Enabled by Microsoft Windows Server 2012 R2, EMC VNX Series, and EMC Powered Backup EMC VSPEX Abstract
Cloud optimize your business Windows Server 2012 R2 Published: October 7, 2013 Contents 1 Trends 3 Windows Server: cloud optimize your business 5 Windows Server 2012 R2 capability overview 5 Server virtualization
Microsoft Private Cloud Fast Track Microsoft Private Cloud Fast Track is a reference architecture designed to help build private clouds by combining Microsoft software with Nutanix technology to decrease
Cloud Lifecycle Managing Cloud Services from Request to Retirement SOLUTION WHITE PAPER Table of Contents EXECUTIVE SUMMARY............................................... 1 CLOUD LIFECYCLE MANAGEMENT........................................
Fujitsu Insights Server Virtualization and Private Clouds Nowadays planning horizons are shorter, revenue streams are uncertain, and you have to be flexible to survive within your business. Among others
Cisco and Red Hat: Application Centric Infrastructure Integration with OpenStack Cisco and Red Hat Extend the Cisco ACI Policy Framework to Red Hat Enterprise Linux OpenStack Platform Enabled Environments
Software-Defined Networking: The New Norm for Networks ONF White Paper April 13, 2012 Table of Contents 2 Executive Summary 3 The Need for a New Network Architecture 4 Limitations of Current Networking
Kent State University s Cloud Strategy Table of Contents Item Page 1. From the CIO 3 2. Strategic Direction for Cloud Computing at Kent State 4 3. Cloud Computing at Kent State University 5 4. Methodology
Plug Into The Cloud with Oracle Database 12c ORACLE WHITE PAPER DECEMBER 2014 Disclaimer The following is intended to outline our general product direction. It is intended for information purposes only,
Convergence of Social, Mobile and Cloud: 7 Steps to Ensure Success June, 2013 Contents Executive Overview...4 Business Innovation & Transformation...5 Roadmap for Social, Mobile and Cloud Solutions...7
Putting the cloud to work for your organization. A buyers guide to cloud solutions. What s in this guide for you? If you re thinking about bringing the cloud into your business but aren t sure where to
Contents Introduction...1 Overview of x86 Virtualization...2 CPU Virtualization...3 The Challenges of x86 Hardware Virtualization...3 Technique 1 - Full Virtualization using Binary Translation...4 Technique
Migration Planning Kit Microsoft Windows Server 2003 This educational kit is intended for IT administrators, architects, and IT managers. The kit covers the reasons and process you should consider when
White Paper Secure Network Access for Personal Mobile Devices What You Will Learn People around the globe are enamored with their smartphones and tablet computers, and they feel strongly that they should
JANUARY 2013 REPORT OF THE DEFENSE SCIENCE BOARD TASK FORCE ON Cyber Security and Reliability in a Digital Cloud JANUARY 2013 Office of the Under Secretary of Defense for Acquisition, Technology, and Logistics
Overview Hewlett Packard's MSA P2000 Family of storage arrays features P2000 G3 arrays with the latest 8 Gb Fibre Channel, 6 Gb SAS, 10GbE iscsi connected models, and an iscsi model with four 1 Gb iscsi
Annals of Telecommunications manuscript No. (will be inserted by the editor) Virtual Networks: Isolation, Performance, and Trends Natalia C. Fernandes Marcelo D. D. Moreira Igor M. Moraes Lyno Henrique
Why Service Providers Need an NFV Platform Strategic White Paper Network Functions Virtualization (NFV) brings proven cloud computing and IT technologies into the networking domain to help service providers
Preflight Checklist: Preparing for a Private Cloud Journey Preflight Checklist: Preparing for a Private Cloud Journey What do you need to have in place before you move forward with cloud computing? This
VMware vcloud Networking and Security Overview Networks and Security for Virtualized Compute Environments WHITE PAPER Overview Organizations worldwide have gained significant efficiency and flexibility
An Oracle White Paper June 2013 Oracle: Big Data for the Enterprise Executive Summary... 2 Introduction... 3 Defining Big Data... 3 The Importance of Big Data... 4 Building a Big Data Platform... 5 Infrastructure
Intel Network Builders Reference Architecture End to End Service Orchestration for NFV vepc End-to-End NFV - vepc Service Orchestration End-to-End Service Orchestration of a Virtual EPC Network Function