Big Data Use Case. How Rackspace is using Private Cloud for Big Data. Bryan Thompson. May 8th, 2013

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "Big Data Use Case. How Rackspace is using Private Cloud for Big Data. Bryan Thompson. May 8th, 2013"

Transcription

1 Big Data Use Case How Rackspace is using Private Cloud for Big Data Bryan Thompson May 8th, 2013

2 Our Big Data Problem Consolidate all monitoring data for reporting and analytical purposes. Every device (server, switch, SAN, UPS, etc.) and product produces multiple events per second Monitoring tens of thousands of devices (both physical and virtual) This adds up to terabytes of data per day, and growing 2

3 Current Environment Dedicated Relational Database systems Loaded nightly Multiple BI Tools Users To scale would be cost and time prohibitive: Cost of DB licenses Cost of Hardware Time to procure and configure servers Concerns with performance Heavy DBA work 3

4 What our sponsors and end-users want Plug in and start analyzing data Act at the speed of the business Maintain optimal query performance Costs to store and analyze Data Volumes Abstract technical nuances of multiple big data technologies Use your preferred BI tool High Availability

5 To The Drawing Board!!! What we need is the ability to: Host ever growing data volumes Handle streaming data and hourly updates of metrics with sub-second performance. Rapid Scalability and High Availability Leverage Open Source technologies Ability to leverage multiple big data technologies 5

6 The Analytic Compute Grid (ACG) Key components of the ACG OpenStack can provide elasticity capabilities Big Data Technologies (v1 Cassandra) Advanced Hashing to run parallel clusters Rule-based elasticity engine integrated w/ OpenStack ANSI-SQL API w/ Extensions ability to plug in 6

7 ACG Architecture Data Presentation Route request to right analytics technology BI Tools Data Mining Data Integration SQL APIs ACG Engine Monitor capacity and auto-provision/de-provision infrastructure resources Infrastructure Platform Big Data Technologies Cassandra PostgreSQL Hadoop MongoDB Others in Future Rackspace Private Cloud Host OS & Hypervisors Commodity Hardware 7

8 Rule-Based Elasticity

9 Rule-Based Elasticity

10 Our OpenStack Environment at Launch Deployed on Rackspace Private Cloud Can run multiple node configurations New node is provisioned in seconds!!! Operating System Ubuntu Big Data Technology Cassandra 32 Node Cluster with capacity to grow 10

11 Performance Comparison SQL Server Environment (Dedicated Environment) 24 CPU 256 GB RAM Availability Calculation against 1.5 Billion row sample 132 hours (5.5 days) 11

12 Performance Comparison RPC OpenStack Environment (virtual machines) 8 CPU 32 GB RAM Availability Calculation against 1.5 Billion row sample 3.2 hours!!! 12

13 ACG Features ACG is a Big Data Management System Parallel engine supports multiple clusters Highly configurable Rules Engine Time based System Based ANSI SQL Compliant API with extensions High Compression - Cassandra Reusable Bulk-Loader Can integrate with current ETL tool 13

14 The Road Ahead PostgreSQL (launching this month) Hadoop Allow for seamless cross platform analysis Migrate off legacy environment Dev/QA Environments Next big big data technology? 14

15 Questions? 15

16 RACKSPACE HOSTING 5000 WALZEM ROAD SAN ANTONIO, TX US SALES: US SUPPORT: RACKSPACE HOSTING RACKSPACE US, INC. RACKSPACE AND FANATICAL SUPPORT ARE SERVICE MARKS OF RACKSPACE US, INC. REGISTERED IN THE UNITED STATES AND OTHER COUNTRIES. 16

Big Data on the Open Cloud

Big Data on the Open Cloud Big Data on the Open Cloud Rackspace Private Cloud, Powered by OpenStack, Helps Reduce Costs and Improve Operational Efficiency Written by Niki Acosta, Cloud Evangelist, Rackspace Big Data on the Open

More information

Performance Benchmark for Cloud Databases

Performance Benchmark for Cloud Databases Performance Benchmark for Cloud Databases J.R. Arredondo vjune2013 (updated pricing on 7/15/2013) Contents Summary of the performance benchmark tests Description of the tests Charts and Data! Results of

More information

Supercharge your MySQL application performance with Cloud Databases

Supercharge your MySQL application performance with Cloud Databases Supercharge your MySQL application performance with Cloud Databases J.R. Arredondo Director Product Marketing Kelly Goolsby Sales Engineering Manager Daniel Morris Senior Product Manager Dave Fowler Founder

More information

GigaSpaces Real-Time Analytics for Big Data

GigaSpaces Real-Time Analytics for Big Data GigaSpaces Real-Time Analytics for Big Data GigaSpaces makes it easy to build and deploy large-scale real-time analytics systems Rapidly increasing use of large-scale and location-aware social media and

More information

Innovative technology for big data analytics

Innovative technology for big data analytics Technical white paper Innovative technology for big data analytics The HP Vertica Analytics Platform database provides price/performance, scalability, availability, and ease of administration Table of

More information

Data Services Advisory

Data Services Advisory Data Services Advisory Modern Datastores An Introduction Created by: Strategy and Transformation Services Modified Date: 8/27/2014 Classification: DRAFT SAFE HARBOR STATEMENT This presentation contains

More information

Big Data Analytics - Accelerated. stream-horizon.com

Big Data Analytics - Accelerated. stream-horizon.com Big Data Analytics - Accelerated stream-horizon.com Legacy ETL platforms & conventional Data Integration approach Unable to meet latency & data throughput demands of Big Data integration challenges Based

More information

Dragon Medical Enterprise Network Edition Technical Note: Requirements for DMENE Networks with virtual servers

Dragon Medical Enterprise Network Edition Technical Note: Requirements for DMENE Networks with virtual servers Dragon Medical Enterprise Network Edition Technical Note: Requirements for DMENE Networks with virtual servers This section includes system requirements for DMENE Network configurations that utilize virtual

More information

A Tale of Two Workloads

A Tale of Two Workloads A Tale of Two Workloads USING OPENSTACK AND VMWARE VSPHERE TO BUILD A HYBRID SOLUTION CONTENTS. Executive Summary. 2 Understanding the Challenges. 2 Development of a Two-Workload Strategy. 3 Understanding

More information

III Big Data Technologies

III Big Data Technologies III Big Data Technologies Today, new technologies make it possible to realize value from Big Data. Big data technologies can replace highly customized, expensive legacy systems with a standard solution

More information

Leveraging OpenStack Private Clouds

Leveraging OpenStack Private Clouds Leveraging OpenStack Private Clouds Robert Ronan Sr. Cloud Solutions Architect! Robert.Ronan@rackspace.com! LEVERAGING OPENSTACK - AGENDA OpenStack What is it? Benefits Leveraging OpenStack University

More information

Apache Ignite TM (Incubating) - In- Memory Data Fabric Fast Data Meets Open Source

Apache Ignite TM (Incubating) - In- Memory Data Fabric Fast Data Meets Open Source Apache Ignite TM (Incubating) - In- Memory Data Fabric Fast Data Meets Open Source DMITRIY SETRAKYAN Founder, PPMC http://www.ignite.incubator.apache.org #apacheignite Agenda Apache Ignite (tm) In- Memory

More information

CA Technologies Big Data Infrastructure Management Unified Management and Visibility of Big Data

CA Technologies Big Data Infrastructure Management Unified Management and Visibility of Big Data Research Report CA Technologies Big Data Infrastructure Management Executive Summary CA Technologies recently exhibited new technology innovations, marking its entry into the Big Data marketplace with

More information

Technology and Cost Considerations for Cloud Deployment: Amazon Elastic Compute Cloud (EC2) Case Study

Technology and Cost Considerations for Cloud Deployment: Amazon Elastic Compute Cloud (EC2) Case Study Creating Value Delivering Solutions Technology and Cost Considerations for Cloud Deployment: Amazon Elastic Compute Cloud (EC2) Case Study Chris Zajac, NJDOT Bud Luo, Ph.D., Michael Baker Jr., Inc. Overview

More information

Performance and Scalability Overview

Performance and Scalability Overview Performance and Scalability Overview This guide provides an overview of some of the performance and scalability capabilities of the Pentaho Business Analytics Platform. Contents Pentaho Scalability and

More information

BENCHMARKING CLOUD DATABASES CASE STUDY on HBASE, HADOOP and CASSANDRA USING YCSB

BENCHMARKING CLOUD DATABASES CASE STUDY on HBASE, HADOOP and CASSANDRA USING YCSB BENCHMARKING CLOUD DATABASES CASE STUDY on HBASE, HADOOP and CASSANDRA USING YCSB Planet Size Data!? Gartner s 10 key IT trends for 2012 unstructured data will grow some 80% over the course of the next

More information

Building your Big Data Architecture on Amazon Web Services

Building your Big Data Architecture on Amazon Web Services Building your Big Data Architecture on Amazon Web Services Abhishek Sinha @abysinha sinhaar@amazon.com AWS Services Deployment & Administration Application Services Compute Storage Database Networking

More information

Background on Elastic Compute Cloud (EC2) AMI s to choose from including servers hosted on different Linux distros

Background on Elastic Compute Cloud (EC2) AMI s to choose from including servers hosted on different Linux distros David Moses January 2014 Paper on Cloud Computing I Background on Tools and Technologies in Amazon Web Services (AWS) In this paper I will highlight the technologies from the AWS cloud which enable you

More information

Cloud Computing and Advanced Relationship Analytics

Cloud Computing and Advanced Relationship Analytics Cloud Computing and Advanced Relationship Analytics Using Objectivity/DB to Discover the Relationships in your Data By Brian Clark Vice President, Product Management Objectivity, Inc. 408 992 7136 brian.clark@objectivity.com

More information

Oracle Database 12c Plug In. Switch On. Get SMART.

Oracle Database 12c Plug In. Switch On. Get SMART. Oracle Database 12c Plug In. Switch On. Get SMART. Duncan Harvey Head of Core Technology, Oracle EMEA March 2015 Safe Harbor Statement The following is intended to outline our general product direction.

More information

Shareable Private Space on a Public Cloud

Shareable Private Space on a Public Cloud Shareable Private Space on a Public Cloud 1.0 Introduction: Sharable private space on public cloud (a distributed computing platform) is nontrivial task. With immerse of Free & Open Source Software (FOSS),

More information

Cloud Computing. A new kind of developers? Presentation by. Nick Barcet nick.barcet@canonical.com

Cloud Computing. A new kind of developers? Presentation by. Nick Barcet nick.barcet@canonical.com Cloud Computing A new kind of developers? Presentation by Nick Barcet nick.barcet@canonical.com www.canonical.com July 2011 Cloud computing stack Salesforce.com, GoogleDocs, Office, etc... GoogleApps,

More information

Virtualizing Apache Hadoop. June, 2012

Virtualizing Apache Hadoop. June, 2012 June, 2012 Table of Contents EXECUTIVE SUMMARY... 3 INTRODUCTION... 3 VIRTUALIZING APACHE HADOOP... 4 INTRODUCTION TO VSPHERE TM... 4 USE CASES AND ADVANTAGES OF VIRTUALIZING HADOOP... 4 MYTHS ABOUT RUNNING

More information

CitusDB Architecture for Real-Time Big Data

CitusDB Architecture for Real-Time Big Data CitusDB Architecture for Real-Time Big Data CitusDB Highlights Empowers real-time Big Data using PostgreSQL Scales out PostgreSQL to support up to hundreds of terabytes of data Fast parallel processing

More information

The Impact of PaaS on Business Transformation

The Impact of PaaS on Business Transformation The Impact of PaaS on Business Transformation September 2014 Chris McCarthy Sr. Vice President Information Technology 1 Legacy Technology Silos Opportunities Business units Infrastructure Provisioning

More information

Software defined networking. Your path to an agile hybrid cloud network

Software defined networking. Your path to an agile hybrid cloud network Software defined networking Your path to an agile hybrid cloud network Is your enterprise network ready for the latest business and consumer trends? Cloud How easily can your users connect to cloud resources?

More information

Cloud Computing through Virtualization and HPC technologies

Cloud Computing through Virtualization and HPC technologies Cloud Computing through Virtualization and HPC technologies William Lu, Ph.D. 1 Agenda Cloud Computing & HPC A Case of HPC Implementation Application Performance in VM Summary 2 Cloud Computing & HPC HPC

More information

Using Attunity Replicate with Greenplum Database Using Attunity Replicate for data migration and Change Data Capture to the Greenplum Database

Using Attunity Replicate with Greenplum Database Using Attunity Replicate for data migration and Change Data Capture to the Greenplum Database White Paper Using Attunity Replicate with Greenplum Database Using Attunity Replicate for data migration and Change Data Capture to the Greenplum Database Abstract This white paper explores the technology

More information

Diablo and VMware TM powering SQL Server TM in Virtual SAN TM. A Diablo Technologies Whitepaper. May 2015

Diablo and VMware TM powering SQL Server TM in Virtual SAN TM. A Diablo Technologies Whitepaper. May 2015 A Diablo Technologies Whitepaper Diablo and VMware TM powering SQL Server TM in Virtual SAN TM May 2015 Ricky Trigalo, Director for Virtualization Solutions Architecture, Diablo Technologies Daniel Beveridge,

More information

AtScale Intelligence Platform

AtScale Intelligence Platform AtScale Intelligence Platform PUT THE POWER OF HADOOP IN THE HANDS OF BUSINESS USERS. Connect your BI tools directly to Hadoop without compromising scale, performance, or control. TURN HADOOP INTO A HIGH-PERFORMANCE

More information

College of Engineering, Technology, and Computer Science

College of Engineering, Technology, and Computer Science College of Engineering, Technology, and Computer Science Design and Implementation of Cloud-based Data Warehousing In partial fulfillment of the requirements for the Degree of Master of Science in Technology

More information

Open Cloud System. (Integration of Eucalyptus, Hadoop and AppScale into deployment of University Private Cloud)

Open Cloud System. (Integration of Eucalyptus, Hadoop and AppScale into deployment of University Private Cloud) Open Cloud System (Integration of Eucalyptus, Hadoop and into deployment of University Private Cloud) Thinn Thu Naing University of Computer Studies, Yangon 25 th October 2011 Open Cloud System University

More information

HYPER-CONVERGED INFRASTRUCTURE STRATEGIES

HYPER-CONVERGED INFRASTRUCTURE STRATEGIES 1 HYPER-CONVERGED INFRASTRUCTURE STRATEGIES MYTH BUSTING & THE FUTURE OF WEB SCALE IT 2 ROADMAP INFORMATION DISCLAIMER EMC makes no representation and undertakes no obligations with regard to product planning

More information

Big Data and Its Impact on the Data Warehousing Architecture

Big Data and Its Impact on the Data Warehousing Architecture Big Data and Its Impact on the Data Warehousing Architecture Sponsored by SAP Speaker: Wayne Eckerson, Director of Research, TechTarget Wayne Eckerson: Hi my name is Wayne Eckerson, I am Director of Research

More information

Data Lake In Action: Real-time, Closed Looped Analytics On Hadoop

Data Lake In Action: Real-time, Closed Looped Analytics On Hadoop 1 Data Lake In Action: Real-time, Closed Looped Analytics On Hadoop 2 Pivotal s Full Approach It s More Than Just Hadoop Pivotal Data Labs 3 Why Pivotal Exists First Movers Solve the Big Data Utility Gap

More information

Cost-Effective Business Intelligence with Red Hat and Open Source

Cost-Effective Business Intelligence with Red Hat and Open Source Cost-Effective Business Intelligence with Red Hat and Open Source Sherman Wood Director, Business Intelligence, Jaspersoft September 3, 2009 1 Agenda Introductions Quick survey What is BI?: reporting,

More information

Using Hadoop to Expand Data Warehousing

Using Hadoop to Expand Data Warehousing Using Hadoop to Expand Data Warehousing Mike Peterson VP of Platforms and Data Architecture, Neustar Feb 28, 2013 1 Copyright Think Big Analytics and Neustar Inc. Why do this? Transforming to an Information

More information

Microsoft Analytics Platform System. Solution Brief

Microsoft Analytics Platform System. Solution Brief Microsoft Analytics Platform System Solution Brief Contents 4 Introduction 4 Microsoft Analytics Platform System 5 Enterprise-ready Big Data 7 Next-generation performance at scale 10 Engineered for optimal

More information

Apache Ignite TM (Incubating) - In- Memory Data Fabric Fast Data Meets Open Source

Apache Ignite TM (Incubating) - In- Memory Data Fabric Fast Data Meets Open Source Apache Ignite TM (Incubating) - In- Memory Data Fabric Fast Data Meets Open Source DMITRIY SETRAKYAN Founder, PPMC http://www.ignite.incubator.apache.org @apacheignite @dsetrakyan Agenda About In- Memory

More information

STeP-IN SUMMIT 2013. June 18 21, 2013 at Bangalore, INDIA. Performance Testing of an IAAS Cloud Software (A CloudStack Use Case)

STeP-IN SUMMIT 2013. June 18 21, 2013 at Bangalore, INDIA. Performance Testing of an IAAS Cloud Software (A CloudStack Use Case) 10 th International Conference on Software Testing June 18 21, 2013 at Bangalore, INDIA by Sowmya Krishnan, Senior Software QA Engineer, Citrix Copyright: STeP-IN Forum and Quality Solutions for Information

More information

Towards Smart and Intelligent SDN Controller

Towards Smart and Intelligent SDN Controller Towards Smart and Intelligent SDN Controller - Through the Generic, Extensible, and Elastic Time Series Data Repository (TSDR) YuLing Chen, Dell Inc. Rajesh Narayanan, Dell Inc. Sharon Aicler, Cisco Systems

More information

Managing Traditional Workloads Together with Cloud Computing Workloads

Managing Traditional Workloads Together with Cloud Computing Workloads Managing Traditional Workloads Together with Cloud Computing Workloads Table of Contents Introduction... 3 Cloud Management Challenges... 3 Re-thinking of Cloud Management Solution... 4 Teraproc Cloud

More information

Big Data Trends and HDFS Evolution

Big Data Trends and HDFS Evolution Big Data Trends and HDFS Evolution Sanjay Radia Founder & Architect Hortonworks Inc Page 1 Hello Founder, Hortonworks Part of the Hadoop team at Yahoo! since 2007 Chief Architect of Hadoop Core at Yahoo!

More information

Cisco Intercloud Fabric for Business

Cisco Intercloud Fabric for Business Data Sheet Cisco Intercloud Fabric for Business Combining the Benefits of Public and Private Clouds in a Hybrid Cloud Cisco Intercloud Fabric for Business enables enterprises to create a seamless hybrid

More information

PLATFORA INTERACTIVE, IN-MEMORY BUSINESS INTELLIGENCE FOR HADOOP

PLATFORA INTERACTIVE, IN-MEMORY BUSINESS INTELLIGENCE FOR HADOOP PLATFORA INTERACTIVE, IN-MEMORY BUSINESS INTELLIGENCE FOR HADOOP Your business is swimming in data, and your business analysts want to use it to answer the questions of today and tomorrow. YOU LOOK TO

More information

Big Data Analytics. with EMC Greenplum and Hadoop. Big Data Analytics. Ofir Manor Pre Sales Technical Architect EMC Greenplum

Big Data Analytics. with EMC Greenplum and Hadoop. Big Data Analytics. Ofir Manor Pre Sales Technical Architect EMC Greenplum Big Data Analytics with EMC Greenplum and Hadoop Big Data Analytics with EMC Greenplum and Hadoop Ofir Manor Pre Sales Technical Architect EMC Greenplum 1 Big Data and the Data Warehouse Potential All

More information

City-Wide Smart Healthcare Appointment Systems Based on Cloud Data Virtualization PaaS

City-Wide Smart Healthcare Appointment Systems Based on Cloud Data Virtualization PaaS , pp. 371-382 http://dx.doi.org/10.14257/ijmue.2015.10.2.34 City-Wide Smart Healthcare Appointment Systems Based on Cloud Data Virtualization PaaS Ping He 1, Penghai Wang 2, Jiechun Gao 3 and Bingyong

More information

Managing Big Data with Hadoop & Vertica. A look at integration between the Cloudera distribution for Hadoop and the Vertica Analytic Database

Managing Big Data with Hadoop & Vertica. A look at integration between the Cloudera distribution for Hadoop and the Vertica Analytic Database Managing Big Data with Hadoop & Vertica A look at integration between the Cloudera distribution for Hadoop and the Vertica Analytic Database Copyright Vertica Systems, Inc. October 2009 Cloudera and Vertica

More information

Firebird meets NoSQL (Apache HBase) Case Study

Firebird meets NoSQL (Apache HBase) Case Study Firebird meets NoSQL (Apache HBase) Case Study Firebird Conference 2011 Luxembourg 25.11.2011 26.11.2011 Thomas Steinmaurer DI +43 7236 3343 896 thomas.steinmaurer@scch.at www.scch.at Michael Zwick DI

More information

James Serra Sr BI Architect JamesSerra3@gmail.com http://jamesserra.com/

James Serra Sr BI Architect JamesSerra3@gmail.com http://jamesserra.com/ James Serra Sr BI Architect JamesSerra3@gmail.com http://jamesserra.com/ Our Focus: Microsoft Pure-Play Data Warehousing & Business Intelligence Partner Our Customers: Our Reputation: "B.I. Voyage came

More information

High-Performance Cloud Computing with IN-MEMORY OLTP

High-Performance Cloud Computing with IN-MEMORY OLTP High-Performance Cloud Computing with IN-MEMORY OLTP Executive Summary SuperBowl XXXIII Miami, 1999. Denver placed a big wager on John Elway, their very senior quarterback. The bet paid off: Broncos win

More information

Big Data Processing: Past, Present and Future

Big Data Processing: Past, Present and Future Big Data Processing: Past, Present and Future Orion Gebremedhin National Solutions Director BI & Big Data, Neudesic LLC. VTSP Microsoft Corp. Orion.Gebremedhin@Neudesic.COM B-orgebr@Microsoft.com @OrionGM

More information

Performance Comparison of SQL based Big Data Analytics with Lustre and HDFS file systems

Performance Comparison of SQL based Big Data Analytics with Lustre and HDFS file systems Performance Comparison of SQL based Big Data Analytics with Lustre and HDFS file systems Rekha Singhal and Gabriele Pacciucci * Other names and brands may be claimed as the property of others. Lustre File

More information

Understanding ArcGIS Deployments in Public and Private Cloud. Marwa Mabrouk

Understanding ArcGIS Deployments in Public and Private Cloud. Marwa Mabrouk Understanding ArcGIS Deployments in Public and Private Cloud Marwa Mabrouk Agenda Back to Basics What are people doing? New Features Using ArcGIS in the Cloud - Private Cloud - Public Cloud Technical Demos

More information

SQL Server 2014. In-Memory by Design. Anu Ganesan August 8, 2014

SQL Server 2014. In-Memory by Design. Anu Ganesan August 8, 2014 SQL Server 2014 In-Memory by Design Anu Ganesan August 8, 2014 Drive Real-Time Business with Real-Time Insights Faster transactions Faster queries Faster insights All built-in to SQL Server 2014. 2 Drive

More information

Introduction to. Cloud Computing

Introduction to. Cloud Computing Cloud Computing Introduction to Cloud Computing Dell Zhang Birkbeck, University of London 2015/16 What is Cloud Computing? Origin of the Cloud Metaphor The best thing since sliced bread? Before Clouds

More information

Private Clouds Can Be Complicated: The Challenges of Building and Operating a Microsoft Private Cloud

Private Clouds Can Be Complicated: The Challenges of Building and Operating a Microsoft Private Cloud Private Clouds Can Be Complicated: The Challenges of Building and Operating a Microsoft Private Cloud Tony Bradley Microsoft MVP, CISSP-ISSAP Principal Analyst, Bradley Strategy Group The connected, mobile

More information

Oracle Database 11g Comparison Chart

Oracle Database 11g Comparison Chart Key Feature Summary Express 10g Standard One Standard Enterprise Maximum 1 CPU 2 Sockets 4 Sockets No Limit RAM 1GB OS Max OS Max OS Max Database Size 4GB No Limit No Limit No Limit Windows Linux Unix

More information

Relational Databases in the Cloud

Relational Databases in the Cloud Contact Information: February 2011 zimory scale White Paper Relational Databases in the Cloud Target audience CIO/CTOs/Architects with medium to large IT installations looking to reduce IT costs by creating

More information

INTRODUCTION TO CASSANDRA

INTRODUCTION TO CASSANDRA INTRODUCTION TO CASSANDRA This ebook provides a high level overview of Cassandra and describes some of its key strengths and applications. WHAT IS CASSANDRA? Apache Cassandra is a high performance, open

More information

Lecture 32 Big Data. 1. Big Data problem 2. Why the excitement about big data 3. What is MapReduce 4. What is Hadoop 5. Get started with Hadoop

Lecture 32 Big Data. 1. Big Data problem 2. Why the excitement about big data 3. What is MapReduce 4. What is Hadoop 5. Get started with Hadoop Lecture 32 Big Data 1. Big Data problem 2. Why the excitement about big data 3. What is MapReduce 4. What is Hadoop 5. Get started with Hadoop 1 2 Big Data Problems Data explosion Data from users on social

More information

Ubuntu OpenStack on VMware vsphere: A reference architecture for deploying OpenStack while limiting changes to existing infrastructure

Ubuntu OpenStack on VMware vsphere: A reference architecture for deploying OpenStack while limiting changes to existing infrastructure TECHNICAL WHITE PAPER Ubuntu OpenStack on VMware vsphere: A reference architecture for deploying OpenStack while limiting changes to existing infrastructure A collaboration between Canonical and VMware

More information

Information Architecture

Information Architecture The Bloor Group Actian and The Big Data Information Architecture WHITE PAPER The Actian Big Data Information Architecture Actian and The Big Data Information Architecture Originally founded in 2005 to

More information

Big Data at Cloud Scale

Big Data at Cloud Scale Big Data at Cloud Scale Pushing the limits of flexible & powerful analytics Copyright 2015 Pentaho Corporation. Redistribution permitted. All trademarks are the property of their respective owners. For

More information

Rackspace Cloud Databases and Container-based Virtualization

Rackspace Cloud Databases and Container-based Virtualization Rackspace Cloud Databases and Container-based Virtualization August 2012 J.R. Arredondo @jrarredondo Page 1 of 6 INTRODUCTION When Rackspace set out to build the Cloud Databases product, we asked many

More information

Harnessing the Power of the Microsoft Cloud for Deep Data Analytics

Harnessing the Power of the Microsoft Cloud for Deep Data Analytics 1 Harnessing the Power of the Microsoft Cloud for Deep Data Analytics Today's Focus How you can operate your business more efficiently and effectively by tapping into Cloud based data analytics solutions

More information

Building a BI Solution in the Cloud

Building a BI Solution in the Cloud Building a BI Solution in the Cloud Stacia Varga, Principal Consultant Email: stacia@datainspirations.com Twitter: @_StaciaV_ 2 SQLSaturday #467 Sponsors Stacia (Misner) Varga Over 30 years of IT experience,

More information

We ensure happier human resources.

We ensure happier human resources. CUSTOMER CASE STUDY We ensure happier human resources. One of the biggest challenges we ve always faced is storage ensuring performance and managing growth. SolidFire s high availability, scalability,

More information

Efficient Cloud Management for Parallel Data Processing In Private Cloud

Efficient Cloud Management for Parallel Data Processing In Private Cloud 2012 International Conference on Information and Network Technology (ICINT 2012) IPCSIT vol. 37 (2012) (2012) IACSIT Press, Singapore Efficient Cloud Management for Parallel Data Processing In Private

More information

Comprehensive Monitoring of VMware vsphere ESX & ESXi Environments

Comprehensive Monitoring of VMware vsphere ESX & ESXi Environments Comprehensive Monitoring of VMware vsphere ESX & ESXi Environments Table of Contents Overview...3 Monitoring VMware vsphere ESX & ESXi Virtual Environment...4 Monitoring using Hypervisor Integration...5

More information

CLOUD. MADE EASY. vnebula Portal

CLOUD. MADE EASY. vnebula Portal CLOUD. MADE EASY. vnebula Portal Introducing Stream s self service vnebula portal. A product for our Stream Partners. 1 Stream Networks vnebula Portal The design philosophy behind the vnebula portal At

More information

Achieving a High-Performance Virtual Network Infrastructure with PLUMgrid IO Visor & Mellanox ConnectX -3 Pro

Achieving a High-Performance Virtual Network Infrastructure with PLUMgrid IO Visor & Mellanox ConnectX -3 Pro Achieving a High-Performance Virtual Network Infrastructure with PLUMgrid IO Visor & Mellanox ConnectX -3 Pro Whitepaper What s wrong with today s clouds? Compute and storage virtualization has enabled

More information

Building Your Big Data Team

Building Your Big Data Team Building Your Big Data Team With all the buzz around Big Data, many companies have decided they need some sort of Big Data initiative in place to stay current with modern data management requirements.

More information

Real Time Fraud Detection With Sequence Mining on Big Data Platform. Pranab Ghosh Big Data Consultant IEEE CNSV meeting, May 6 2014 Santa Clara, CA

Real Time Fraud Detection With Sequence Mining on Big Data Platform. Pranab Ghosh Big Data Consultant IEEE CNSV meeting, May 6 2014 Santa Clara, CA Real Time Fraud Detection With Sequence Mining on Big Data Platform Pranab Ghosh Big Data Consultant IEEE CNSV meeting, May 6 2014 Santa Clara, CA Open Source Big Data Eco System Query (NOSQL) : Cassandra,

More information

IBM Software Information Management Creating an Integrated, Optimized, and Secure Enterprise Data Platform:

IBM Software Information Management Creating an Integrated, Optimized, and Secure Enterprise Data Platform: Creating an Integrated, Optimized, and Secure Enterprise Data Platform: IBM PureData System for Transactions with SafeNet s ProtectDB and DataSecure Table of contents 1. Data, Data, Everywhere... 3 2.

More information

NoSQL Data Base Basics

NoSQL Data Base Basics NoSQL Data Base Basics Course Notes in Transparency Format Cloud Computing MIRI (CLC-MIRI) UPC Master in Innovation & Research in Informatics Spring- 2013 Jordi Torres, UPC - BSC www.jorditorres.eu HDFS

More information

Big Data Success Step 1: Get the Technology Right

Big Data Success Step 1: Get the Technology Right Big Data Success Step 1: Get the Technology Right TOM MATIJEVIC Director, Business Development ANDY MCNALIS Director, Data Management & Integration MetaScale is a subsidiary of Sears Holdings Corporation

More information

Cisco Data Preparation

Cisco Data Preparation Data Sheet Cisco Data Preparation Unleash your business analysts to develop the insights that drive better business outcomes, sooner, from all your data. As self-service business intelligence (BI) and

More information

News and trends in Data Warehouse Automation, Big Data and BI. Johan Hendrickx & Dirk Vermeiren

News and trends in Data Warehouse Automation, Big Data and BI. Johan Hendrickx & Dirk Vermeiren News and trends in Data Warehouse Automation, Big Data and BI Johan Hendrickx & Dirk Vermeiren Extreme Agility from Source to Analysis DWH Appliances & DWH Automation Typical Architecture 3 What Business

More information

A PERFORMANCE ANALYSIS of HADOOP CLUSTERS in OPENSTACK CLOUD and in REAL SYSTEM

A PERFORMANCE ANALYSIS of HADOOP CLUSTERS in OPENSTACK CLOUD and in REAL SYSTEM A PERFORMANCE ANALYSIS of HADOOP CLUSTERS in OPENSTACK CLOUD and in REAL SYSTEM Ramesh Maharjan and Manoj Shakya Department of Computer Science and Engineering Dhulikhel, Kavre, Nepal lazymesh@gmail.com,

More information

OpenStack Ecosystem and Xen Cloud Platform

OpenStack Ecosystem and Xen Cloud Platform OpenStack Ecosystem and Xen Cloud Platform Amit Naik Prasad Nirantar BMC Software 1 Agenda Introduction Rise of OpenStack OpenStack Details and Ecosystem OpenStack and Xen Cloud Platform - Demo Conclusion

More information

Powerful Management of Financial Big Data

Powerful Management of Financial Big Data Powerful Management of Financial Big Data TickSmith s solutions are the first to apply the processing power, speed, and capacity of cutting-edge Big Data technology to financial data. We combine open source

More information

Meeting Real Time Risk Management Challenge XAP In-Memory Computing

Meeting Real Time Risk Management Challenge XAP In-Memory Computing Meeting Real Time Risk Management Challenge XAP In-Memory Computing March 2015 Meeting Real-Time Risk Management Challenges Table of Contents Introduction 03 Main Industry Challenges....04 Meeting Real-Time

More information

OPEN MODERN DATA ARCHITECTURE FOR FINANCIAL SERVICES RISK MANAGEMENT

OPEN MODERN DATA ARCHITECTURE FOR FINANCIAL SERVICES RISK MANAGEMENT WHITEPAPER OPEN MODERN DATA ARCHITECTURE FOR FINANCIAL SERVICES RISK MANAGEMENT A top-tier global bank s end-of-day risk analysis jobs didn t complete in time for the next start of trading day. To solve

More information

QLIKVIEW INTEGRATION TION WITH AMAZON REDSHIFT John Park Partner Engineering

QLIKVIEW INTEGRATION TION WITH AMAZON REDSHIFT John Park Partner Engineering QLIKVIEW INTEGRATION TION WITH AMAZON REDSHIFT John Park Partner Engineering June 2014 Page 1 Contents Introduction... 3 About Amazon Web Services (AWS)... 3 About Amazon Redshift... 3 QlikView on AWS...

More information

Cloud Computing Trends, Examples & What s Ahead

Cloud Computing Trends, Examples & What s Ahead Cloud Computing Trends, Examples & What s Ahead Mike Klein President, Online Tech June 21, 2010 Cloud Computing Defined One Definition: On-Demand Computing Elastic & Scalable Rapidly Provisioned Virtualized

More information

Databricks. A Primer

Databricks. A Primer Databricks A Primer Who is Databricks? Databricks was founded by the team behind Apache Spark, the most active open source project in the big data ecosystem today. Our mission at Databricks is to dramatically

More information

Decoding the Big Data Deluge a Virtual Approach. Dan Luongo, Global Lead, Field Solution Engineering Data Virtualization Business Unit, Cisco

Decoding the Big Data Deluge a Virtual Approach. Dan Luongo, Global Lead, Field Solution Engineering Data Virtualization Business Unit, Cisco Decoding the Big Data Deluge a Virtual Approach Dan Luongo, Global Lead, Field Solution Engineering Data Virtualization Business Unit, Cisco High-volume, velocity and variety information assets that demand

More information

Akuda Labs. Leverages Peak Hosting s Operations-as-a-Service Managed Hosting Solution to Process Big Data Analytics 500 Faster without Big Costs

Akuda Labs. Leverages Peak Hosting s Operations-as-a-Service Managed Hosting Solution to Process Big Data Analytics 500 Faster without Big Costs Akuda Labs Leverages Peak Hosting s Operations-as-a-Service Managed Hosting Solution to Process Big Data Analytics 500 Faster without Big Costs INDUSTRY: BIG DATA ANALYTICS This case study provides a high-level

More information

Enterprise and Standard Feature Compare

Enterprise and Standard Feature Compare www.blytheco.com Enterprise and Standard Feature Compare SQL Server 2008 Enterprise SQL Server 2008 Enterprise is a comprehensive data platform for running mission critical online transaction processing

More information

Real-Time Analytics for Big Market Data with XAP In-Memory Computing

Real-Time Analytics for Big Market Data with XAP In-Memory Computing Real-Time Analytics for Big Market Data with XAP In-Memory Computing March 2015 Real Time Analytics for Big Market Data Table of Contents Introduction 03 Main Industry Challenges....04 Achieving Real-Time

More information

Secret Server Architecture and Sizing Guide

Secret Server Architecture and Sizing Guide This document contains information for planning Secret Server architecture and resource allocation within your environment. Read through or use one of the following links to skip ahead to the relevant

More information

Hadoop Evolution In Organizations. Mark Vervuurt Cluster Data Science & Analytics

Hadoop Evolution In Organizations. Mark Vervuurt Cluster Data Science & Analytics In Organizations Mark Vervuurt Cluster Data Science & Analytics AGENDA 1. Yellow Elephant 2. Data Ingestion & Complex Event Processing 3. SQL on Hadoop 4. NoSQL 5. InMemory 6. Data Science & Machine Learning

More information

From Spark to Ignition:

From Spark to Ignition: From Spark to Ignition: Fueling Your Business on Real-Time Analytics Eric Frenkiel, MemSQL CEO June 29, 2015 San Francisco, CA What s in Store For This Presentation? 1. MemSQL: A real-time database for

More information

Scaling Your Data to the Cloud

Scaling Your Data to the Cloud ZBDB Scaling Your Data to the Cloud Technical Overview White Paper POWERED BY Overview ZBDB Zettabyte Database is a new, fully managed data warehouse on the cloud, from SQream Technologies. By building

More information

ICONICS Choosing the Correct Edition of MS SQL Server

ICONICS Choosing the Correct Edition of MS SQL Server Description: This application note aims to assist you in choosing the right edition of Microsoft SQL server for your ICONICS applications. OS Requirement: XP Win 2000, XP Pro, Server 2003, Vista, Server

More information

Enabling Database-as-a-Service (DBaaS) within Enterprises or Cloud Offerings

Enabling Database-as-a-Service (DBaaS) within Enterprises or Cloud Offerings Solution Brief Enabling Database-as-a-Service (DBaaS) within Enterprises or Cloud Offerings Introduction Accelerating time to market, increasing IT agility to enable business strategies, and improving

More information

CUMULUX WHICH CLOUD PLATFORM IS RIGHT FOR YOU? COMPARING CLOUD PLATFORMS. Review Business and Technology Series www.cumulux.com

CUMULUX WHICH CLOUD PLATFORM IS RIGHT FOR YOU? COMPARING CLOUD PLATFORMS. Review Business and Technology Series www.cumulux.com ` CUMULUX WHICH CLOUD PLATFORM IS RIGHT FOR YOU? COMPARING CLOUD PLATFORMS Review Business and Technology Series www.cumulux.com Table of Contents Cloud Computing Model...2 Impact on IT Management and

More information

System Requirements Table of contents

System Requirements Table of contents Table of contents 1 Introduction... 2 2 Knoa Agent... 2 2.1 System Requirements...2 2.2 Environment Requirements...4 3 Knoa Server Architecture...4 3.1 Knoa Server Components... 4 3.2 Server Hardware Setup...5

More information