Cloudwick. CLOUDWICK LABS Big Data Research Paper. Nebula: Powering Enterprise Private & Hybrid Cloud for DataStax Big Data

Transcription

1 Nebula: Powering Enterprise Private & Hybrid Cloud for DataStax Big Data was commissioned to evaluate and test the Nebula One Private and Hybrid Cloud Appliance using DataStax, a leading Apache Cassandra distribution, a fault tolerant and scalable NoSQL database management system built for enterprise mission critical big data applications. Objective Determine the operational and performance capabilities of the Nebula One solution for private and hybrid cloud deployments with DataStax, the leading NoSQL database for mission critical enterprise big data. Bare Metal and Public Cloud are Big Pain Points for IT & Business Bare metal big data clusters are expensive and it often takes IT days or weeks to get pilot POCs orchestrated and provisioned, causing business organizations to turn to AWS or Rackspace public clouds for storing and analyzing their big data clusters. These organizations, often called shadow IT or rogue IT groups, circumvent central IT by paying for these public cloud services with their own departmental credit card or hidden (and incorrectly itemized) budget line item. When shadow IT groups buy public cloud services to manage their big data testing, development and production deployments, they introduce security risk, potential for data loss, possible regulatory compliance violations, and overall loss of control to the entire enterprise IT organization. Enterprises recognize the agility, efficiency and scale that IaaS provides but find that many private cloud providers require a significant investment in custom engineering, consulting services and miscellaneous fees. Enterprises need a turnkey solution that provides distributed compute, storage and network services in a unified system for big data workloads like DataStax. Enterprise IT needs a big data private/hybrid cloud solution so it can more easily provide business with the same or better time-to-service and performance advantages as those provided by AWS or Rackspace. Enterprise big data - often petabytes of it - resides on-premise in local data warehouses. Therefore, it is more cost-effective and efficient for an enterprise NoSQL solution to reside in the data center, as a private cloud solution or as a hybrid multicloud solution with on-demand elasticity. Nebula One Hybrid Cloud Test for Multi-Site DataStax Enterprise Approach Labs set up private and hybrid cloud environments running DataStax workloads locally and distributed across Nebula, Rackspace, and Amazon Web Services (AWS). To simulate DataStax enterprise workloads, used the Cassandra stress test tool, Gazzang for data encryption, and Datameer for analytics. Results Summary Nebula Strengths Agile - Nebula One provides best-inclass private and hybrid cloud orchestration, making it easy for IT to quickly provision and orchestrate DataStax clusters. Elastic - Nebula One provides private and hybrid cloud Infrastructure-as-a- Service (IaaS) elasticity to efficiently manage compute, storage, and network resources. Locality - Nebula enables IT to bring elastic cloud services inside the firewall where the majority of enterprise big data resides. Nebula Benefits Nebula One for DataStax allows enterprise IT organizations to offer turnkey IaaS that provides end users or Hybrid Cloud Test 1 Hybrid Cloud Test 2 Nebula - Rackspace Nebula - AWS = 10 Rackspace DSE Instances with = 10 AWS DSE Instances with All product and company names are trademarks or registered trademarks of their respective holders. Use of them does not imply any affiliation with or endorsement by them. business units faster time-to-service over traditional bare metal infrastructures and equivalent time-to-service provided by AWS and Rackspace public cloud infrastructure. - Page 1 - is a leader in Big Data integration and performance optimization for the Fortune importance of interoperability and performance testing. Labs is creating the

2 Power of the Cloud Big data and cloud computing are currently top of mind for enterprises because they deliver competitive advantage, cost savings, and ultimately increased revenues. Big data and its analytics capabilities offer promises to design, build and market better, more memorable products and improve business processes. The cloud provides organizations with the ability to enhance business agility, efficiency and productivity, reduce the time it takes to introduce and turn on new services and applications, and flexibility for the ever-changing needs of the business on a pay as you go model. The cloud model is appealing for big data because it provides unlimited resources on demand, it eliminates the need to build infrastructures that can handle unpredictable activity spikes, and it enables big data analytics. While the cloud makes it as easy to fire up 1,000 servers and apps as it is to fire up 10, and just as easy to turn them off, cloud app users also learn all too often that Internet connectivity often can be variable or unreliable. A private cloud (and to an extent, hybrid cloud) makes use of enterprise LAN and data center connectivity for predictable connectivity, performance, and simplified troubleshooting. If a problem arises, you are in control of locating and troubleshooting on-premise mission critical workloads, rather than relying on someone else (who knows where) working on an issue that multiple tenants are complaining about. CLOUDWICK LABS Solution Technology Private & Hybrid Cloud Enterprise Cassandra Enterprise Security A 2013 TechTarget Cloud Pulse Survey indicated that of 1,497 respondents, 61 percent use public cloud services now while 39 percent do not. However, the risk of data loss and poor performance, along with loss of control are out of the question for many. Of the 39 percent of the respondents in the TechTarget survey who do not use public cloud services, 80 percent noted they will not use public cloud for at least a year, and 45 percent do not plan to use public cloud at all; the 45 percent with no plans at all cited loss of control over security and storing their applications in the public cloud environment as their top concerns ( Turning off cloud services, Jan Stafford, SearchCloudApplications). Enterprise Analytics Public Cloud Nebula One, A Better Cloud Nebula commissioned Labs to test and evaluate the Nebula One private and hybrid cloud solution against traditional bare metal and to test hybrid cloud interoperability with AWS and Rackspace for DataStax Cassandra workloads. used industry accepted best practices and tools to evaluate and test the Nebula One private cloud and hybrid cloud solutions. This solution brief will provide enterprise IT with a solution assessment of the Nebula One operational and performance capabilities for private and hybrid cloud deployments for building a better service for business. Labs determined that Nebula provides a better way for central IT to provide their enterprise stakeholders with an elastic compute and storage infrastructure, after evaluating time-to-deploy, infrastructure elasticity, cluster flexibility, workload and data agility, and enterprise firewall or security controls. - Page 2 - is a leader in Big Data integration and performance optimization for the Fortune importance of interoperability and performance testing. Labs is creating the

3 Nebula One, Elastic Private Cloud for the Enterprise Much has been written about why some enterprise users, departments, or business units have opted to bypass central IT to procure public cloud big data services from AWS or Rackspace. The reasons most often cited for creating shadow public clouds include: 1. It takes weeks/months for central IT to set up, orchestrate and provision bare metal clusters for big data pilots, whereas it takes just minutes/hours to use AWS or Rackspace to set up public cloud environments to manage big data. 2. Enterprise IT doesn t offer a private cloud solution. To overcome these challenges for these organizations, the Nebula One private or hybrid cloud solution is a highly attractive and viable alternative to bare metal and or public cloud options alone. A Nebula One private and hybrid cloud provides agility and flexibility in the form of elasticity for growth and scale, the ability to build and tear down without negative implications, and on-demand services and databases. With Nebula One, new business and IT services and applications can be turned on quickly and easily, saving the organization and IT staff time and money and offloading a number of tasks, enabling business units to see the services they request turned on almost as soon as they request them and benefiting quickly from those revenuegenerating capabilities. A Nebula One private or hybrid cloud also allows enterprises to balance costs and scaling requirements, maximize efficiency, manage service levels and functionality quickly, and build in an evolutionary exit strategy. determined that Nebula One provides a better way for central IT to provide their enterprise stakeholders an elastic compute and storage infrastructure, when evaluating time-to-deploy, infrastructure elasticity, cluster flexibility, workload and data agility, and enterprise firewall or security controls. CLOUDWICK LABS Nebula IT Benefits Nebula One provides enterprise IT with a turnkey solution that provides distributed compute, storage, and network services in a unified system. The hardware and software is coupled with certified industry-standard x86 servers, Nebula Cloud Nodes, creating a private cloud, and at the heart of the solution is the Nebula Cloud Controller appliance, which integrates up to 20 servers. The Nebula One solution enables central IT to put in a place a turnkey, elastic, and self-service private and/or hybrid cloud infrastructure to meet the business objectives of various users, departments, or business units. Nebula Developer Benefits The Nebula One offers a rich, intuitive graphical self-service portal for users at any level, allowing them to provision their own resources. For developers, Nebula One implements APIs from OpenStack and Web Services so they can leverage their knowledge of public cloud services. Vast online and open source resources are available to help rapidly deploy and manage new and existing applications. The Nebula One private and hybrid cloud solution exceeds the traditional bare metal IaaS capabilities and delivers the same (and in some cases, better) operational results found in the public cloud. Scorecard: Deployment and Operations Capabilities Nebula One for DataStax Bare Metal for DataStax AWS Public Cloud for DataStax Rackspace Public Cloud for DataStax Application Deployment Minutes Hours Minutes Minutes IaaS Elasticity Yes No Yes Yes Cluster Flexibility Yes No Yes Yes Workload Agility Yes No Yes Yes Data Locality for EDW Optimal Optimal Not Optimal Not Optimal Data Center Firewall Security Yes Yes No No - Page 3 - is a leader in Big Data integration and performance optimization for the Fortune importance of interoperability and performance testing. Labs is creating the

4 Nebula One & Rackspace Hybrid Cloud Test For Multi-Site DataStax Environment: set up a DataStax 20 node multi-site hybrid cloud with Nebula One and Rackspace using DataStax OpsCenter. Gazzang encryption and Datameer analytic software were also tested. Hybrid Cloud Test 1 Approach: Nebula - Rackspace used the Cassandra Stress Tool to insert 10M records (write) and 10M read operations using = 10 Rackspace DSE Instances with Data center replication to demonstrate hybrid cloud multi-site DataStax performance for high availability and disaster recovery. In this scenario, two data centers were running active-active rings of DataStax Cassandra. If one of the two data centers were to fail, the mission critical application would continue to service clients from the other data center. One of the advantages of DataStax Cassandra is that it has out of the box multi-site high availability and it is for this reason that many Fortune 1000 companies are migrating SQL applications to it. Results: Lab s hybrid cloud testing found Nebula to be an excellent multi-cloud solution for running DataStax Cassandra in a multi-site environment with Rackspace public cloud. found the Nebula One and Rackspace hybrid cloud to be an excellent solution for enterprises looking to add public cloud elasticity to provide multicloud disaster recovery and multi-site high availability to on premise DataStax. Nebula Administrator Benefits Nebula One has been engineered for administrators to enable rapid provisioning and simplified management. Administrators can easily allocate resources across the system, and users consume those resources on-demand. Nebula One also provides API compatibility with the OpenStack and Amazon EC2/S3 cloud platforms, eliminating barriers to seamless cross-platform migration and management. Nebula Business Benefits Nebula enables enterprise IT organizations to build highly elastic DataStax clusters behind the enterprise firewall, while still using AWS and Rackspace as failover protection Nebula One & Amazon Hybrid Cloud Test For Multi-Site DataStax Environment: set up a DataStax 20 node multi-site hybrid cloud with Nebula One and AWS using DataStax OpsCenter. Gazzang encryption and Datameer analytic software were also tested. Hybrid Cloud Test 2 Nebula - AWS = 10 AWS DSE Instances with Approach: used the Cassandra Stress Tool to insert 10M records (write) and 10M read operations using Data center replication to demonstrate hybrid cloud multi-site DataStax performance for high availability and disaster recovery. In this scenario, two data centers were running active-active rings of DataStax Cassandra. If one of the two data centers were to fail, the mission critical application would continue to service clients from the other data center. One of the advantages of DataStax Cassandra is that it has out of the box multi-site high availability and it is for this reason that many Fortune 1000 companies are migrating SQL applications to it. - Page 4 - is a leader in Big Data integration and performance optimization for the Fortune importance of interoperability and performance testing. Labs is creating the

5 Results: Lab s hybrid cloud testing found Nebula to be an excellent multicloud solution for running DataStax Cassandra in a multi-site environment with AWS public cloud. found the Nebula One and AWS hybrid cloud to be an excellent solution for enterprises looking to add public cloud elasticity to provide multi-cloud disaster recovery and multi-site high availability to on premise DataStax. Conclusion Labs determined that Nebula provides seamless DataStax performance for private and hybrid clouds, while offering ease of provisioning and orchestration of clusters. Nebula enables IT to provide business with faster time to service than bare metal at the same speed provided by Amazon and Rackspace public cloud. Nebula brings on-demand compute and storage elasticity to the data, behind the enterprise firewall. Enterprise IT can rapidly provide internal stakeholders with critical infrastructure services, while maintaining control by minimizing shadow IT operations and security risks. Nebula Business Advantage found Nebula One to provide the enterprise with the full-range of Infrastructure-as-a-Service (IaaS) functionality found in public clouds. Nebula One allows enterprises to start with the compute and storage needed today and scale seamlessly across multiple racks tomorrow through a simple, expandable architecture -- a single system on the network. - Page 5 - is a leader in Big Data integration and performance optimization for the Fortune importance of interoperability and performance testing. Labs is creating the

6 About Solution Nebula: Nebula One brings the cloud to you, under your control, behind your firewall. Nebula One is an integrated hardware and software solution providing distributed compute, storage, and network services in a unified system. Nebula One consists of Nebula purpose-built hardware and software coupled with certified industry-standard x86 servers known as Nebula Cloud Nodes to create a private cloud. The heart of the solution is the Nebula Cloud Controller, an appliance that integrates up to 20 servers. Embedded on each controller is Nebula Cosmos, a purpose-built operating system used to orchestrate services and provide end-user functionality. Cosmos simplifies administration for the entire cloud system through a unified management interface enabling end users to provision their own compute and storage resources via an intuitive self-service portal. It provides API compatibility with the OpenStack and Amazon EC2/S3 cloud platforms. Users are able to leverage familiar tools and existing knowledge in building their applications. Administrators use streamlined and powerful management software to lower operational complexity and control costs. Nebula One is a truly turnkey solution that includes Nebula hardware and software, plus industry-standard servers from trusted vendors. This system enables an unparalleled economy-of-scale for our customers. For more information visit DataStax: DataStax is driving the Apache Cassandra database to be the first viable alternative to Oracle for companies transforming the way they interact with customers. Cassandra s simple and elegant architecture handles these problems in ways no other database can. Its fully distributed nature allows for amazing performance at extreme data velocities. DataStax powers the online applications that transform business for more than 300 customers, including startups and 20 of the Fortune 100. DataStax delivers a massively scalable, flexible and continuously available big data platform built on Apache Cassandra. DataStax integrates enterprise-ready Cassandra, Apache Hadoop for analytics and Apache Solr for search across multi-datacenters and in the cloud. Companies such as Adobe, Healthcare Anytime, ebay and Netflix rely on DataStax to transform their businesses. Based in San Mateo, Calif., DataStax is backed by industry-leading investors: Lightspeed Venture Partners, Crosslink Capital, Meritech Capital Partners, Scale Venture Partners, DFJ Growth and Next World Capital. For more information visit - Page 6 - is a leader in Big Data integration and performance optimization for the Fortune importance of interoperability and performance testing. Labs is creating the

7 About Solution Gazzang: Gazzang helps organizations protect sensitive information in DataStax Enterprise by transparently encrypting data in real time and providing advanced key management that ensures only authorized processes can access the data. Gazzang s unique software only/no appliance architecture can secure ANY Linux application or database without making any changes to your environment or impeding performance. Gazzang also ensures that cryptographic keys remain safe and in full compliance with HIPAA, PCI-DSS, FERPA and other data security regulations. Features of zncrypt for DataStax Enterprise include: Advanced Key Management - Stores keys separate from the encrypted data to ensure a data breach does not also result in the loss of the cryptographic key Transparent Data Encryption - Protects data at rest resulting in minimal performance impact. Requires no complex changes to databases, files, applications or storage. Process Based Access Controls - Restricts access to specific processes rather than by OS user. Encrypt and Decrypt Unstructured Data - Secures personally identifiable information, intellectual property, log files and any other sensitive data that could be considered damaging if exposed outside the business. Automation Tools - Rapid distributed deployment from ten to thousands of nodes. World-Class Support - DataStax and Gazzang partnership offers enterprise support for your secure big data implementation Gazzang s technology enables SaaS vendors, health care organizations, financial institutions, public sector agencies and more to meet regulatory compliance initiatives, secure personally identifiable information and prevent unauthorized access to sensitive data and systems. The company is headquartered in Austin, Texas and backed by Austin Ventures and Silver Creek Ventures. For more information visit Datameer: Datameer is the only self-service and schema-free big data analytics application for Hadoop that ensures the fastest time to discovering insights in any data. A no-etl solution, anyone can use Datameer's wizard-based data integration, iterative point-and-click analytics, and drag-and-drop visualizations to find the insights that matter to drive their business forward. Founded by Hadoop veterans in 2009, Datameer scales from a laptop to thousands of nodes and is available for all major Hadoop distributions. For more information visit & Lab: After having completed more than 75,000 hours of big data production cluster engineering and operations for leading enterprises like Bank of America, Visa, JP Morgan, Home Depot, Warner Music Group, NetApp, Wal-Mart and Radium One realized the need for a benchmark and certification lab that is dedicated to working collaboratively with the big data community (enterprise, consultants and vendors) to research, test and share big data best practices. Together we can accelerate big data. For more information visit or - Page 7 - is a leader in Big Data integration and performance optimization for the Fortune importance of interoperability and performance testing. Labs is creating the