GigaSpaces Real-Time Analytics for Big Data

Size: px
Start display at page:

Download "GigaSpaces Real-Time Analytics for Big Data"

Transcription

1 GigaSpaces Real-Time Analytics for Big Data GigaSpaces makes it easy to build and deploy large-scale real-time analytics systems Rapidly increasing use of large-scale and location-aware social media and mobile applications are driving the need for scalable, real-time platforms that can handle streaming analysis and processing of massive amounts of data. Today, creating an analytics system for big data generally means collecting multiple technologies from various providers, and building the system yourself. This presents challenges in terms of erformance, costs, scalability, real-time, and more. GigaSpaces resolves these issues: You need to handle massive amounts of data in real time, without losing data and at minimum cost. Most analytics systems are not designed for real-time: it can take hours or days to see the impact of an event in reports, enabling you to take action. The challenge becomes even greater as events are gathered from more sources at significantly higher volumes. One option: Construct your own solution by combining various available technologies. This can be complex: In addition to messaging, data storage, and processing, you need management and orchestration for automating the deployment and ensuring continuous availability the assorted parts. A simpler option: Just plug in the GigaSpaces Real-Time Analytics solution. You can focus on your business logic, and leave the rest to us. GigaSpaces makes building and deploying a large-scale real-time analytics system simple. You just provide simple event processing business logic, and we handle the scalability, performance, and database integration. Seamlessly. GigaSpaces delivers software middleware that provides enterprises and ISVs with end-to-end application scalability and cloud-enablement for mission-critical applications for hundreds of tier-1 organizations worldwide. It s Open: Use any stack, avoid lock-in. Pick your own Big Data database (RDBMS or NoSQL); Plug in consistent management and monitoring across the stack without changing your code; Write event handlers using common languages; Access your data using standard SQL/JPA APIs. All while minimizing costs. A unique combination of memory and disk-based databases ensure the optimum cost/performance ratio. Leveraging automation and cloud-based deployment reduces operational costs. The GigaSpaces Real-Time Analytics solution for Big Data Applications eliminates the complexity XAP Real-Time Solution for Big Data Cassandra HBase MongoDB Redis

2 CURRENT TECHNOLOGIES OVERVIEW There is no one-size-fits-all technology. Building an analytic application that addresses real-time and batch analytics requirements requires a combination of the available technologies. The challenge becomes the integration of these various pieces, tuning the system to ensure consistent performance and scaling through the entire stack, and providing consistent management and monitoring across the entire stack. Most analytics systems can be broken down into three stages of data flow in the system: Metrics Correlation Research various metrics are collected into counters. For example, number of requests per day. (Real-time) Correlate metrics for a more aggregative system view. For example, analyze which features hook users. (Near real-time) Use this information to run research and trend analysis over a period of time. (Batch map/reduce processing) Currently, you must integrate different products and technologies to provide the entire analytics functionality. This method has many associated challenges: Traditional App: Database (RDBMS) Used to run many analytics systems Complex Event Processing (CEP) Designed to correlate data in real time Associated Challenges: Performance: Not designed for real time Scaling: Not designed to grow at the speed and volume of information required in a Big Data environment, doesn t fit well for data that is continuously evolving Cost: Most RDBMS rely on expensive set-up and hardware to maintain reliability and performance Scaling: It is often necessary to aggregate events into a centralized source, which doesn t scale Capacity: Not designed to deal with historical data Hadoop Designed for batch analytics and complex correlation Performance: Not designed for real time In-Memory Data Grid Fast processing power for storing and processing data Capacity: Capacity for storing vast amounts of information in-memory doesn t scale, in terms of both system scaling and cost NoSQL Designed to handle large data volumes at low cost Processing capability: Sheer amount of data can be challenging

3 THE SOLUTION Google, Facebook, and Twitter have already shown us the way by moving many of their analytics systems to real time. The question now is how businesses can build their own Google/Facebook/Twitter-like analytics, but in a significantly simpler way that fits existing applications and skillsets. Step 1: Collect and Store Enable collection of large volumes of data from multiple sources in real time. The process must be reliable, to ensure the accuracy of the analytics. Solution: Use an In-Memory Data Grid Memory enables x100k msg/sec Reliability is achieved through redundancy and replication Can be accessed through large set of APIs (Document, JMS, Memcache...) Step 2: Speed up processing through co-location of business logic with data By co-locating your business logic and data, you can process events as they enter the system, reducing multiple network hops and serialization/de-serialization overhead. You can also reduce the number of moving parts, making the entire system significantly simpler to scale and maintain. Step 3: Integrate with the Big Data store to meet volume and cost demands Integrate with the Big Data store through a generic plug-in, compatible with your data store of choice, whether NoSQL or SQL. Avoid lock-in to a specific NoSQL API Performance: Reduced network hops & serialization overhead Simplicity fewer moving parts Scalability without compromising consistency (strict consistency at the front, eventual consistency for the long-term data) JPA/Standard API

4 PUTTING IT ALL TOGETHER 1. Store events in memory 2. Co-locate business logic with data for RT processing 3. Integrate with Big Data store for long-term data Cluster of in-memory data grids (IMDG) at the front and a Big Data database at the backend. Feeds are stored directly into the IMDG. The feeds trigger a set of co-located processors that process them. The processing can include validation and enrichment of the data as well as creation of new data sets needed for further correlation and post-processing of data. Data is forwarded to the back-end Big Data store through the built-in write-behind feature of the IMDG. The IMDG can be used as a processing buffer: After processing by the IMDG, data is stored in the Big Data storage. It can also be used to store the last day of information. Data sent to the NoSQL data store is stored in batches to maximize write throughput. The analytics application reads the data directly from the NoSQL data store. When the app needs only the last day of activity, it can access the data grid directly through the built-in JPA/SQL interface.

5 MAIN FEATURES & BENEFITS Performance Maximum throughput is achieved using in-memory devices and by distributing events between nodes and processing them in parallel. The write to the database is done in batches, asynchronously, maximizing throughput to the underlying database. Built-in synchronization (write-behind), uses batches to speed up write performance. Simplicity All you need to do to build your entire Facebooklike analytics system is to write your event handler business logic. GigaSpaces takes care of performance, high availability, scalability, and deployment management. Continuous Availability Keeping the real time part and long term decoupled makes it possible to continue and serve real time feeds even when the database is down. It also makes it easier to deal with planned downtime that is required when maintaining long term data, such as for re-shading. Cloud Enabled Works with any private and public cloud such as CloudStack, VMware, OpenStack, Amazon, Rackspace, Azure etc. Consistent Management The GigaSpaces cluster management offers built-in integration with many popular databases, such as MySQL, Postgress, Cassandra, and MongoDB, and with popular web platforms such as Tomcat, JBoss, and NodeJS, enabling you to deploy the entire application stack with a single click. Elasticity Scaling is achieved by adding more machines without any downtime Security Access to the data is secured both from the feeder side and the analytics system. You can also set roles that control the data sets that are accessible to specific users Transactionality and Consistency The entire processing is done under transaction, ensuring the consistency and reliability of the data. Openness Choose any Big Data database (RDBMS or NoSQL), and plug in consistent management and monitoring across the stack without changing your code. Write event handlers using common Java,.Net, Groovy, JavaScript, JRuby, and a large set of dynamic languages, and access the data using standard SQL/JPA APIs. REAL-TIME IN-MEMORY PROCESSING GRID AND BIG DATA STORAGE FEATURES Real-Time Event Processing Events are stored in memory. A built-in mechanism enables triggering of events based on SQL templates. Standard Query Users can access the data through a standard JPA/SQL interface. Write/Read Dynamic Scalability With a NoSQL back-end data store, the system can grow with the data, reducing the costs associated with over-provisioning. Built-In Pub/Sub Remote clients and services can subscribe to the processed data directly, without a need for additional messaging system.. Map/Reduce Data correlation and aggregation is done through parallel query and code execution across the entire data grid. Open Database Plug-In Easily plug in different sets of SQL and NoSQL databases without changing the application code. You can start with SQL databases at small scale, and switch to NoSQL at later, as your system grows.

6 COST BENEFITS Economic Data Scaling Leverage commodity hardware and software-based storage to provide a large-scale data store at low cost. Solution: Memory short-term data Disk long-term data Combine memory and disk for optimum cost performance ratio: Memory is x10, x100 lower than disk for high data access rate (According to Stanford research) Disk is lower cost for high capacity lower access rate Example: Cost RAM Use Disk for this throughput Throughput Disk Use Memory for this throughput Optimum Cost The cost of processing 10K events per second and storing it for a window of an hour (till it gets pulled to the long-term storage) with 500B message size in memory requires only ~16G at a cost of ~$32 per month per server. Economic App Scaling Automation: Reduce operational cost Elastic Scaling: Reduce over-provisioning cost Cloud portability: Choose the right cloud for the job Cloud bursting: Scavenge extra capacity when needed Industry use cases that particularly need real-time insights from big data sets include: Social Networking: Measure the immediate impact to your site traffic from social media, whether a new blog post, a tweet, a Like, or even a comment. Knowing this information translates to better conversion and more effective online campaigns. SaaS: Measuring user behavior and acting upon it is crucial for improving customer satisfaction and conversion rates which represent immediate increases in revenue. Financial Services: Determining in real time whether your portfolio is losing money, or if there is fraud in your system means that you can prevent disasters as they occur, not after the damage is done. Correlating multiple sources from the market in real time results in a more accurate view of the market and enables more accurate actions to maximize your profit. ABOUT GIGASPACES GigaSpaces Technologies is the pioneer of a new generation of application virtualization platforms, and a leading provider of end-to-end scaling solutions for distributed, mission-critical application environments, and cloud enabling technologies. GigaSpaces is the only platform on the market that offers truly silo-free architecture, along with operational agility and openness, delivering enhanced efficiency, extreme performance, and always-on availability. Our technology was designed from the ground up to support any cloud environment private, public, or hybrid and offers a pain-free, evolutionary path from today s data center to the technologies of tomorrow. GIGASPACES OFFICES WORLDWIDE US East Coast Office, New York Tel: US West Coast Office, San Jose Tel: International Office, Tel Aviv Tel: Europe Office, London Tel: Asia Pacific Office, Singapore Tel:

Easy Deployment of Mission-Critical Applications to the Cloud

Easy Deployment of Mission-Critical Applications to the Cloud Easy Deployment of Mission-Critical Applications to the Cloud Businesses want to move to the cloud to gain agility and reduce costs. But if your app needs re-architecting or new code that s neither easy

More information

Elastic Application Platform for Market Data Real-Time Analytics. for E-Commerce

Elastic Application Platform for Market Data Real-Time Analytics. for E-Commerce Elastic Application Platform for Market Data Real-Time Analytics Can you deliver real-time pricing, on high-speed market data, for real-time critical for E-Commerce decisions? Market Data Analytics applications

More information

Real-Time Analytics for Big Market Data with XAP In-Memory Computing

Real-Time Analytics for Big Market Data with XAP In-Memory Computing Real-Time Analytics for Big Market Data with XAP In-Memory Computing March 2015 Real Time Analytics for Big Market Data Table of Contents Introduction 03 Main Industry Challenges....04 Achieving Real-Time

More information

Scaling Healthcare Applications to Meet Rising Challenges of Healthcare IT

Scaling Healthcare Applications to Meet Rising Challenges of Healthcare IT Scaling Healthcare Applications to Meet Rising Challenges of Healthcare IT December 2013 Copyright 2013 GigaSpaces. All Rights Reserved. Healthcare IT has entered the era of Big Data. Are your applications

More information

Meeting Real Time Risk Management Challenge XAP In-Memory Computing

Meeting Real Time Risk Management Challenge XAP In-Memory Computing Meeting Real Time Risk Management Challenge XAP In-Memory Computing March 2015 Meeting Real-Time Risk Management Challenges Table of Contents Introduction 03 Main Industry Challenges....04 Meeting Real-Time

More information

Real-Time Stream Processing and Cloud-Based Big Data Increasing in Today s Enterprises

Real-Time Stream Processing and Cloud-Based Big Data Increasing in Today s Enterprises Big Data Survey Real-Time Stream Processing and Cloud-Based Big Data Increasing in Today s Enterprises Companies, Increasingly Seeing Big Data as Mission-Critical, Want Two Things: To Take Big Data to

More information

Real Time Analy:cs for Big Data Lessons Learned from Facebook

Real Time Analy:cs for Big Data Lessons Learned from Facebook SINGLE PLATFORM. COMPLETE SCALABILITY. Real Time Analy:cs for Big Data Lessons Learned from Facebook @uri1803 Head of Product GigaSpaces About Me MTBK Junky A Proud Dad Technology addict Head of Product

More information

INTRODUCTION TO CASSANDRA

INTRODUCTION TO CASSANDRA INTRODUCTION TO CASSANDRA This ebook provides a high level overview of Cassandra and describes some of its key strengths and applications. WHAT IS CASSANDRA? Apache Cassandra is a high performance, open

More information

Why NoSQL? Your database options in the new non- relational world. 2015 IBM Cloudant 1

Why NoSQL? Your database options in the new non- relational world. 2015 IBM Cloudant 1 Why NoSQL? Your database options in the new non- relational world 2015 IBM Cloudant 1 Table of Contents New types of apps are generating new types of data... 3 A brief history on NoSQL... 3 NoSQL s roots

More information

Big Data Use Case. How Rackspace is using Private Cloud for Big Data. Bryan Thompson. May 8th, 2013

Big Data Use Case. How Rackspace is using Private Cloud for Big Data. Bryan Thompson. May 8th, 2013 Big Data Use Case How Rackspace is using Private Cloud for Big Data Bryan Thompson May 8th, 2013 Our Big Data Problem Consolidate all monitoring data for reporting and analytical purposes. Every device

More information

APP DEVELOPMENT ON THE CLOUD MADE EASY WITH PAAS

APP DEVELOPMENT ON THE CLOUD MADE EASY WITH PAAS APP DEVELOPMENT ON THE CLOUD MADE EASY WITH PAAS This article looks into the benefits of using the Platform as a Service paradigm to develop applications on the cloud. It also compares a few top PaaS providers

More information

Analytics March 2015 White paper. Why NoSQL? Your database options in the new non-relational world

Analytics March 2015 White paper. Why NoSQL? Your database options in the new non-relational world Analytics March 2015 White paper Why NoSQL? Your database options in the new non-relational world 2 Why NoSQL? Contents 2 New types of apps are generating new types of data 2 A brief history of NoSQL 3

More information

Cloud Courses Description

Cloud Courses Description Cloud Courses Description Cloud 101: Fundamental Cloud Computing and Architecture Cloud Computing Concepts and Models. Fundamental Cloud Architecture. Virtualization Basics. Cloud platforms: IaaS, PaaS,

More information

BIG DATA TOOLS. Top 10 open source technologies for Big Data

BIG DATA TOOLS. Top 10 open source technologies for Big Data BIG DATA TOOLS Top 10 open source technologies for Big Data We are in an ever expanding marketplace!!! With shorter product lifecycles, evolving customer behavior and an economy that travels at the speed

More information

Using an In-Memory Data Grid for Near Real-Time Data Analysis

Using an In-Memory Data Grid for Near Real-Time Data Analysis SCALEOUT SOFTWARE Using an In-Memory Data Grid for Near Real-Time Data Analysis by Dr. William Bain, ScaleOut Software, Inc. 2012 ScaleOut Software, Inc. 12/27/2012 IN today s competitive world, businesses

More information

Cloud Courses Description

Cloud Courses Description Courses Description 101: Fundamental Computing and Architecture Computing Concepts and Models. Data center architecture. Fundamental Architecture. Virtualization Basics. platforms: IaaS, PaaS, SaaS. deployment

More information

Ubuntu and Hadoop: the perfect match

Ubuntu and Hadoop: the perfect match WHITE PAPER Ubuntu and Hadoop: the perfect match February 2012 Copyright Canonical 2012 www.canonical.com Executive introduction In many fields of IT, there are always stand-out technologies. This is definitely

More information

Lambda Architecture. Near Real-Time Big Data Analytics Using Hadoop. January 2015. Email: bdg@qburst.com Website: www.qburst.com

Lambda Architecture. Near Real-Time Big Data Analytics Using Hadoop. January 2015. Email: bdg@qburst.com Website: www.qburst.com Lambda Architecture Near Real-Time Big Data Analytics Using Hadoop January 2015 Contents Overview... 3 Lambda Architecture: A Quick Introduction... 4 Batch Layer... 4 Serving Layer... 4 Speed Layer...

More information

Cloud/SaaS enablement of existing applications

Cloud/SaaS enablement of existing applications Cloud/SaaS enablement of existing applications GigaSpaces: Nati Shalom, CTO & Founder About GigaSpaces Technologies Enabling applications to run a distributed cluster as if it was a single machine 75+

More information

INTRODUCING APACHE IGNITE An Apache Incubator Project

INTRODUCING APACHE IGNITE An Apache Incubator Project WHITE PAPER BY GRIDGAIN SYSTEMS FEBRUARY 2015 INTRODUCING APACHE IGNITE An Apache Incubator Project COPYRIGHT AND TRADEMARK INFORMATION 2015 GridGain Systems. All rights reserved. This document is provided

More information

Hybrid Solutions Combining In-Memory & SSD

Hybrid Solutions Combining In-Memory & SSD Hybrid Solutions Combining In-Memory & SSD Author: christos@gigaspaces.com Agenda 1 2 3 4 Overview of the big data technology landscape Building a high-speed SSD-backed data store Complex & compound queries

More information

MySQL Comes of Age. Robert Hodges Sr. Staff Engineer Percona Live London November 4, 2014. 2014 VMware Inc. All rights reserved.

MySQL Comes of Age. Robert Hodges Sr. Staff Engineer Percona Live London November 4, 2014. 2014 VMware Inc. All rights reserved. MySQL Comes of Age Robert Hodges Sr. Staff Engineer Percona Live London November 4, 2014 2014 VMware Inc. All rights reserved. Continuent is now part of VMware! VMware acquired Continuent on 28 October

More information

Enabling Database-as-a-Service (DBaaS) within Enterprises or Cloud Offerings

Enabling Database-as-a-Service (DBaaS) within Enterprises or Cloud Offerings Solution Brief Enabling Database-as-a-Service (DBaaS) within Enterprises or Cloud Offerings Introduction Accelerating time to market, increasing IT agility to enable business strategies, and improving

More information

Reference Model for Cloud Applications CONSIDERATIONS FOR SW VENDORS BUILDING A SAAS SOLUTION

Reference Model for Cloud Applications CONSIDERATIONS FOR SW VENDORS BUILDING A SAAS SOLUTION October 2013 Daitan White Paper Reference Model for Cloud Applications CONSIDERATIONS FOR SW VENDORS BUILDING A SAAS SOLUTION Highly Reliable Software Development Services http://www.daitangroup.com Cloud

More information

Open Source Technologies on Microsoft Azure

Open Source Technologies on Microsoft Azure Open Source Technologies on Microsoft Azure A Survey @DChappellAssoc Copyright 2014 Chappell & Associates The Main Idea i Open source technologies are a fundamental part of Microsoft Azure The Big Questions

More information

CloudCenter Full Lifecycle Management. An application-defined approach to deploying and managing applications in any datacenter or cloud environment

CloudCenter Full Lifecycle Management. An application-defined approach to deploying and managing applications in any datacenter or cloud environment CloudCenter Full Lifecycle Management An application-defined approach to deploying and managing applications in any datacenter or cloud environment CloudCenter Full Lifecycle Management Page 2 Table of

More information

BIG DATA TRENDS AND TECHNOLOGIES

BIG DATA TRENDS AND TECHNOLOGIES BIG DATA TRENDS AND TECHNOLOGIES THE WORLD OF DATA IS CHANGING Cloud WHAT IS BIG DATA? Big data are datasets that grow so large that they become awkward to work with using onhand database management tools.

More information

BENCHMARKING CLOUD DATABASES CASE STUDY on HBASE, HADOOP and CASSANDRA USING YCSB

BENCHMARKING CLOUD DATABASES CASE STUDY on HBASE, HADOOP and CASSANDRA USING YCSB BENCHMARKING CLOUD DATABASES CASE STUDY on HBASE, HADOOP and CASSANDRA USING YCSB Planet Size Data!? Gartner s 10 key IT trends for 2012 unstructured data will grow some 80% over the course of the next

More information

Scalable Architecture on Amazon AWS Cloud

Scalable Architecture on Amazon AWS Cloud Scalable Architecture on Amazon AWS Cloud Kalpak Shah Founder & CEO, Clogeny Technologies kalpak@clogeny.com 1 * http://www.rightscale.com/products/cloud-computing-uses/scalable-website.php 2 Architect

More information

ORACLE COHERENCE 12CR2

ORACLE COHERENCE 12CR2 ORACLE COHERENCE 12CR2 KEY FEATURES AND BENEFITS ORACLE COHERENCE IS THE #1 IN-MEMORY DATA GRID. KEY FEATURES Fault-tolerant in-memory distributed data caching and processing Persistence for fast recovery

More information

Cloud Computing: Making the right choices

Cloud Computing: Making the right choices Cloud Computing: Making the right choices Kalpak Shah Clogeny Technologies Pvt Ltd 1 About Me Kalpak Shah Founder & CEO, Clogeny Technologies Passionate about economics and technology evolving through

More information

Optimizing the Hybrid Cloud

Optimizing the Hybrid Cloud Judith Hurwitz President and CEO Marcia Kaufman COO and Principal Analyst Sponsored by IBM Introduction Hybrid cloud is fast becoming a reality for enterprises that want speed, predictability and flexibility

More information

Software-Defined Networks Powered by VellOS

Software-Defined Networks Powered by VellOS WHITE PAPER Software-Defined Networks Powered by VellOS Agile, Flexible Networking for Distributed Applications Vello s SDN enables a low-latency, programmable solution resulting in a faster and more flexible

More information

ScaleArc for SQL Server

ScaleArc for SQL Server Solution Brief ScaleArc for SQL Server Overview Organizations around the world depend on SQL Server for their revenuegenerating, customer-facing applications, running their most business-critical operations

More information

Migration and Disaster Recovery Underground in the NEC / Iron Mountain National Data Center with the RackWare Management Module

Migration and Disaster Recovery Underground in the NEC / Iron Mountain National Data Center with the RackWare Management Module Migration and Disaster Recovery Underground in the NEC / Iron Mountain National Data Center with the RackWare Management Module WHITE PAPER May 2015 Contents Advantages of NEC / Iron Mountain National

More information

SCALABILITY IN THE CLOUD

SCALABILITY IN THE CLOUD SCALABILITY IN THE CLOUD A TWILIO PERSPECTIVE twilio.com OUR SOFTWARE Twilio has built a 100 percent software-based infrastructure using many of the same distributed systems engineering and design principles

More information

Composite Data Virtualization Composite Data Virtualization And NOSQL Data Stores

Composite Data Virtualization Composite Data Virtualization And NOSQL Data Stores Composite Data Virtualization Composite Data Virtualization And NOSQL Data Stores Composite Software October 2010 TABLE OF CONTENTS INTRODUCTION... 3 BUSINESS AND IT DRIVERS... 4 NOSQL DATA STORES LANDSCAPE...

More information

An Oracle White Paper November 2010. Leveraging Massively Parallel Processing in an Oracle Environment for Big Data Analytics

An Oracle White Paper November 2010. Leveraging Massively Parallel Processing in an Oracle Environment for Big Data Analytics An Oracle White Paper November 2010 Leveraging Massively Parallel Processing in an Oracle Environment for Big Data Analytics 1 Introduction New applications such as web searches, recommendation engines,

More information

MakeMyTrip CUSTOMER SUCCESS STORY

MakeMyTrip CUSTOMER SUCCESS STORY MakeMyTrip CUSTOMER SUCCESS STORY MakeMyTrip is the leading travel site in India that is running two ClustrixDB clusters as multi-master in two regions. It removed single point of failure. MakeMyTrip frequently

More information

Big Data Analytics - Accelerated. stream-horizon.com

Big Data Analytics - Accelerated. stream-horizon.com Big Data Analytics - Accelerated stream-horizon.com Legacy ETL platforms & conventional Data Integration approach Unable to meet latency & data throughput demands of Big Data integration challenges Based

More information

ScaleArc idb Solution for SQL Server Deployments

ScaleArc idb Solution for SQL Server Deployments ScaleArc idb Solution for SQL Server Deployments Objective This technology white paper describes the ScaleArc idb solution and outlines the benefits of scaling, load balancing, caching, SQL instrumentation

More information

Affordable, Scalable, Reliable OLTP in a Cloud and Big Data World: IBM DB2 purescale

Affordable, Scalable, Reliable OLTP in a Cloud and Big Data World: IBM DB2 purescale WHITE PAPER Affordable, Scalable, Reliable OLTP in a Cloud and Big Data World: IBM DB2 purescale Sponsored by: IBM Carl W. Olofson December 2014 IN THIS WHITE PAPER This white paper discusses the concept

More information

Using In-Memory Computing to Simplify Big Data Analytics

Using In-Memory Computing to Simplify Big Data Analytics SCALEOUT SOFTWARE Using In-Memory Computing to Simplify Big Data Analytics by Dr. William Bain, ScaleOut Software, Inc. 2012 ScaleOut Software, Inc. 12/27/2012 T he big data revolution is upon us, fed

More information

Data Integration Checklist

Data Integration Checklist The need for data integration tools exists in every company, small to large. Whether it is extracting data that exists in spreadsheets, packaged applications, databases, sensor networks or social media

More information

Pulsar Realtime Analytics At Scale. Tony Ng April 14, 2015

Pulsar Realtime Analytics At Scale. Tony Ng April 14, 2015 Pulsar Realtime Analytics At Scale Tony Ng April 14, 2015 Big Data Trends Bigger data volumes More data sources DBs, logs, behavioral & business event streams, sensors Faster analysis Next day to hours

More information

Java, PHP & Ruby - Cloud Hosting

Java, PHP & Ruby - Cloud Hosting Java, PHP & Ruby - Cloud Hosting NO LOCK-IN No technical lock-in and no binding contract. We believe in open standards without any technical lock-ins. We think that Open source provides flexibility and

More information

Drive new Revenue With PaaS/IaaS. Ruslan Synytsky CTO, Jelastic

Drive new Revenue With PaaS/IaaS. Ruslan Synytsky CTO, Jelastic Drive new Revenue With PaaS/IaaS Ruslan Synytsky CTO, Jelastic 2 MISSING OUT ON CLOUD OPPORTUNITY? Many hosters today are missing out on a massive opportunity to provide an Amazon-beating public cloud

More information

The 3 questions to ask yourself about BIG DATA

The 3 questions to ask yourself about BIG DATA The 3 questions to ask yourself about BIG DATA Do you have a big data problem? Companies looking to tackle big data problems are embarking on a journey that is full of hype, buzz, confusion, and misinformation.

More information

Building Out Your Cloud-Ready Solutions. Clark D. Richey, Jr., Principal Technologist, DoD

Building Out Your Cloud-Ready Solutions. Clark D. Richey, Jr., Principal Technologist, DoD Building Out Your Cloud-Ready Solutions Clark D. Richey, Jr., Principal Technologist, DoD Slide 1 Agenda Define the problem Explore important aspects of Cloud deployments Wrap up and questions Slide 2

More information

CASE STUDY: Oracle TimesTen In-Memory Database and Shared Disk HA Implementation at Instance level. -ORACLE TIMESTEN 11gR1

CASE STUDY: Oracle TimesTen In-Memory Database and Shared Disk HA Implementation at Instance level. -ORACLE TIMESTEN 11gR1 CASE STUDY: Oracle TimesTen In-Memory Database and Shared Disk HA Implementation at Instance level -ORACLE TIMESTEN 11gR1 CASE STUDY Oracle TimesTen In-Memory Database and Shared Disk HA Implementation

More information

Migration and Building of Data Centers in IBM SoftLayer with the RackWare Management Module

Migration and Building of Data Centers in IBM SoftLayer with the RackWare Management Module Migration and Building of Data Centers in IBM SoftLayer with the RackWare Management Module June, 2015 WHITE PAPER Contents Advantages of IBM SoftLayer and RackWare Together... 4 Relationship between

More information

JAVA IN THE CLOUD PAAS PLATFORM IN COMPARISON

JAVA IN THE CLOUD PAAS PLATFORM IN COMPARISON JAVA IN THE CLOUD PAAS PLATFORM IN COMPARISON Eberhard Wolff Architecture and Technology Manager adesso AG, Germany 12.10. Agenda A Few Words About Cloud Java and IaaS PaaS Platform as a Service Google

More information

Hyperscaling Your SaaS Infrastructure

Hyperscaling Your SaaS Infrastructure InsideBIGDATA Guide to Hyperscaling Your SaaS Infrastructure by Daniel D. Gutierrez BROUGHT TO YOU BY All rights reserved. Various trademarks are held by their respective owners. Introduction: Performance

More information

Accelerating Hadoop MapReduce Using an In-Memory Data Grid

Accelerating Hadoop MapReduce Using an In-Memory Data Grid Accelerating Hadoop MapReduce Using an In-Memory Data Grid By David L. Brinker and William L. Bain, ScaleOut Software, Inc. 2013 ScaleOut Software, Inc. 12/27/2012 H adoop has been widely embraced for

More information

CERTIFICATE PROGRAMME ON CLOUD SPECIALISTS DEVELOPMENT

CERTIFICATE PROGRAMME ON CLOUD SPECIALISTS DEVELOPMENT CERTIFICATE PROGRAMME ON CLOUD SPECIALISTS DEVELOPMENT Present awareness and demands of cloud computing calls for increasing needs of cloud specialists development noticeably worldwide. VMware's Singapore

More information

Data Challenges in Telecommunications Networks and a Big Data Solution

Data Challenges in Telecommunications Networks and a Big Data Solution Data Challenges in Telecommunications Networks and a Big Data Solution Abstract The telecom networks generate multitudes and large sets of data related to networks, applications, users, network operations

More information

Driving More Value From OpenVMS Critical Infrastructure in Local and Global Datacenters: A CASE STUDY. Presented by: J. Barry Thompson, CTO Tervela

Driving More Value From OpenVMS Critical Infrastructure in Local and Global Datacenters: A CASE STUDY. Presented by: J. Barry Thompson, CTO Tervela Driving More Value From OpenVMS Critical Infrastructure in Local and Global Datacenters: A CASE STUDY Presented by: J. Barry Thompson, CTO Tervela Case Study: Customer Challenges The Solution Overall Impact

More information

Virtualizing Apache Hadoop. June, 2012

Virtualizing Apache Hadoop. June, 2012 June, 2012 Table of Contents EXECUTIVE SUMMARY... 3 INTRODUCTION... 3 VIRTUALIZING APACHE HADOOP... 4 INTRODUCTION TO VSPHERE TM... 4 USE CASES AND ADVANTAGES OF VIRTUALIZING HADOOP... 4 MYTHS ABOUT RUNNING

More information

Assignment # 1 (Cloud Computing Security)

Assignment # 1 (Cloud Computing Security) Assignment # 1 (Cloud Computing Security) Group Members: Abdullah Abid Zeeshan Qaiser M. Umar Hayat Table of Contents Windows Azure Introduction... 4 Windows Azure Services... 4 1. Compute... 4 a) Virtual

More information

<Insert Picture Here> Getting Coherence: Introduction to Data Grids South Florida User Group

<Insert Picture Here> Getting Coherence: Introduction to Data Grids South Florida User Group Getting Coherence: Introduction to Data Grids South Florida User Group Cameron Purdy Cameron Purdy Vice President of Development Speaker Cameron Purdy is Vice President of Development

More information

NEXT-GENERATION, CLOUD-BASED SERVER MONITORING AND SYSTEMS MANAGEMENT

NEXT-GENERATION, CLOUD-BASED SERVER MONITORING AND SYSTEMS MANAGEMENT NEXT-GENERATION, CLOUD-BASED SERVER MONITORING AND SYSTEMS MANAGEMENT COVERS INTRODUCTION A NEW APPROACH CUSTOMER USE CASES FEATURES ARCHITECTURE V 1.0 INTRODUCTION & OVERVIEW Businesses use computers

More information

Scaling in the Cloud with AWS. By: Eli White (CTO & Co-Founder @ mojolive) eliw.com - @eliw - mojolive.com

Scaling in the Cloud with AWS. By: Eli White (CTO & Co-Founder @ mojolive) eliw.com - @eliw - mojolive.com Scaling in the Cloud with AWS By: Eli White (CTO & Co-Founder @ mojolive) eliw.com - @eliw - mojolive.com Welcome! Why is this guy talking to us? Please ask questions! 2 What is Scaling anyway? Enabling

More information

Relational Databases in the Cloud

Relational Databases in the Cloud Contact Information: February 2011 zimory scale White Paper Relational Databases in the Cloud Target audience CIO/CTOs/Architects with medium to large IT installations looking to reduce IT costs by creating

More information

The Next Wave of Data Management. Is Big Data The New Normal?

The Next Wave of Data Management. Is Big Data The New Normal? The Next Wave of Data Management Is Big Data The New Normal? Table of Contents Introduction 3 Separating Reality and Hype 3 Why Are Firms Making IT Investments In Big Data? 4 Trends In Data Management

More information

THE REALITIES OF NOSQL BACKUPS

THE REALITIES OF NOSQL BACKUPS THE REALITIES OF NOSQL BACKUPS White Paper Trilio Data, Inc. March 2015 1 THE REALITIES OF NOSQL BACKUPS TABLE OF CONTENTS INTRODUCTION... 2 NOSQL DATABASES... 2 PROBLEM: LACK OF COMPREHENSIVE BACKUP AND

More information

NoSQL Data Base Basics

NoSQL Data Base Basics NoSQL Data Base Basics Course Notes in Transparency Format Cloud Computing MIRI (CLC-MIRI) UPC Master in Innovation & Research in Informatics Spring- 2013 Jordi Torres, UPC - BSC www.jorditorres.eu HDFS

More information

OPEN MODERN DATA ARCHITECTURE FOR FINANCIAL SERVICES RISK MANAGEMENT

OPEN MODERN DATA ARCHITECTURE FOR FINANCIAL SERVICES RISK MANAGEMENT WHITEPAPER OPEN MODERN DATA ARCHITECTURE FOR FINANCIAL SERVICES RISK MANAGEMENT A top-tier global bank s end-of-day risk analysis jobs didn t complete in time for the next start of trading day. To solve

More information

Real Time Fraud Detection With Sequence Mining on Big Data Platform. Pranab Ghosh Big Data Consultant IEEE CNSV meeting, May 6 2014 Santa Clara, CA

Real Time Fraud Detection With Sequence Mining on Big Data Platform. Pranab Ghosh Big Data Consultant IEEE CNSV meeting, May 6 2014 Santa Clara, CA Real Time Fraud Detection With Sequence Mining on Big Data Platform Pranab Ghosh Big Data Consultant IEEE CNSV meeting, May 6 2014 Santa Clara, CA Open Source Big Data Eco System Query (NOSQL) : Cassandra,

More information

It s Not Public Versus Private Clouds - It s the Right Infrastructure at the Right Time With the IBM Systems and Storage Portfolio

It s Not Public Versus Private Clouds - It s the Right Infrastructure at the Right Time With the IBM Systems and Storage Portfolio White Paper - It s the Right Infrastructure at the Right Time With the IBM Systems and Storage Portfolio Contents Executive Summary....2 Introduction....3 Private clouds - Powerful tech, new solutions....3

More information

W H I T E P A P E R. Deriving Intelligence from Large Data Using Hadoop and Applying Analytics. Abstract

W H I T E P A P E R. Deriving Intelligence from Large Data Using Hadoop and Applying Analytics. Abstract W H I T E P A P E R Deriving Intelligence from Large Data Using Hadoop and Applying Analytics Abstract This white paper is focused on discussing the challenges facing large scale data processing and the

More information

Mr. Apichon Witayangkurn apichon@iis.u-tokyo.ac.jp Department of Civil Engineering The University of Tokyo

Mr. Apichon Witayangkurn apichon@iis.u-tokyo.ac.jp Department of Civil Engineering The University of Tokyo Sensor Network Messaging Service Hive/Hadoop Mr. Apichon Witayangkurn apichon@iis.u-tokyo.ac.jp Department of Civil Engineering The University of Tokyo Contents 1 Introduction 2 What & Why Sensor Network

More information

How To Handle Big Data With A Data Scientist

How To Handle Big Data With A Data Scientist III Big Data Technologies Today, new technologies make it possible to realize value from Big Data. Big data technologies can replace highly customized, expensive legacy systems with a standard solution

More information

Well packaged sets of preinstalled, integrated, and optimized software on select hardware in the form of engineered systems and appliances

Well packaged sets of preinstalled, integrated, and optimized software on select hardware in the form of engineered systems and appliances INSIGHT Oracle's All- Out Assault on the Big Data Market: Offering Hadoop, R, Cubes, and Scalable IMDB in Familiar Packages Carl W. Olofson IDC OPINION Global Headquarters: 5 Speen Street Framingham, MA

More information

Migration and Building of Data Centers in IBM SoftLayer with the RackWare Management Module

Migration and Building of Data Centers in IBM SoftLayer with the RackWare Management Module Migration and Building of Data Centers in IBM SoftLayer with the RackWare Management Module June, 2015 WHITE PAPER Contents Advantages of IBM SoftLayer and RackWare Together... 4 Relationship between

More information

Using Cloud Services for Test Environments A case study of the use of Amazon EC2

Using Cloud Services for Test Environments A case study of the use of Amazon EC2 Using Cloud Services for Test Environments A case study of the use of Amazon EC2 Lee Hawkins (Quality Architect) Quest Software, Melbourne Copyright 2010 Quest Software We are gathered here today to talk

More information

A REVIEW PAPER ON THE HADOOP DISTRIBUTED FILE SYSTEM

A REVIEW PAPER ON THE HADOOP DISTRIBUTED FILE SYSTEM A REVIEW PAPER ON THE HADOOP DISTRIBUTED FILE SYSTEM Sneha D.Borkar 1, Prof.Chaitali S.Surtakar 2 Student of B.E., Information Technology, J.D.I.E.T, sborkar95@gmail.com Assistant Professor, Information

More information

White Paper. How Streaming Data Analytics Enables Real-Time Decisions

White Paper. How Streaming Data Analytics Enables Real-Time Decisions White Paper How Streaming Data Analytics Enables Real-Time Decisions Contents Introduction... 1 What Is Streaming Analytics?... 1 How Does SAS Event Stream Processing Work?... 2 Overview...2 Event Stream

More information

BIG DATA-AS-A-SERVICE

BIG DATA-AS-A-SERVICE White Paper BIG DATA-AS-A-SERVICE What Big Data is about What service providers can do with Big Data What EMC can do to help EMC Solutions Group Abstract This white paper looks at what service providers

More information

Ubuntu: helping drive business insight from Big Data

Ubuntu: helping drive business insight from Big Data WHITE PAPER Ubuntu: helping drive business insight from Big Data February 2012 Copyright Canonical 2012 www.canonical.com Executive introduction For years, web giants such as Facebook, Google and ebay

More information

Planning the Migration of Enterprise Applications to the Cloud

Planning the Migration of Enterprise Applications to the Cloud Planning the Migration of Enterprise Applications to the Cloud A Guide to Your Migration Options: Private and Public Clouds, Application Evaluation Criteria, and Application Migration Best Practices Introduction

More information

Accelerating Enterprise Applications and Reducing TCO with SanDisk ZetaScale Software

Accelerating Enterprise Applications and Reducing TCO with SanDisk ZetaScale Software WHITEPAPER Accelerating Enterprise Applications and Reducing TCO with SanDisk ZetaScale Software SanDisk ZetaScale software unlocks the full benefits of flash for In-Memory Compute and NoSQL applications

More information

Building Heavy Load Messaging System

Building Heavy Load Messaging System CASE STUDY Building Heavy Load Messaging System About IntelliSMS Intelli Messaging simplifies mobile communication methods so you can cost effectively build mobile communication into your business processes;

More information

Architectural patterns for building real time applications with Apache HBase. Andrew Purtell Committer and PMC, Apache HBase

Architectural patterns for building real time applications with Apache HBase. Andrew Purtell Committer and PMC, Apache HBase Architectural patterns for building real time applications with Apache HBase Andrew Purtell Committer and PMC, Apache HBase Who am I? Distributed systems engineer Principal Architect in the Big Data Platform

More information

Microsoft Big Data Solutions. Anar Taghiyev P-TSP E-mail: b-anarta@microsoft.com;

Microsoft Big Data Solutions. Anar Taghiyev P-TSP E-mail: b-anarta@microsoft.com; Microsoft Big Data Solutions Anar Taghiyev P-TSP E-mail: b-anarta@microsoft.com; Why/What is Big Data and Why Microsoft? Options of storage and big data processing in Microsoft Azure. Real Impact of Big

More information

ENZO UNIFIED SOLVES THE CHALLENGES OF OUT-OF-BAND SQL SERVER PROCESSING

ENZO UNIFIED SOLVES THE CHALLENGES OF OUT-OF-BAND SQL SERVER PROCESSING ENZO UNIFIED SOLVES THE CHALLENGES OF OUT-OF-BAND SQL SERVER PROCESSING Enzo Unified Extends SQL Server to Simplify Application Design and Reduce ETL Processing CHALLENGES SQL Server does not scale out

More information

High Availability of VistA EHR in Cloud. ViSolve Inc. White Paper February 2015. www.visolve.com

High Availability of VistA EHR in Cloud. ViSolve Inc. White Paper February 2015. www.visolve.com High Availability of VistA EHR in Cloud ViSolve Inc. White Paper February 2015 1 Abstract Inspite of the accelerating migration to cloud computing in the Healthcare Industry, high availability and uptime

More information

Amazon Web Services. 18.11.2015 Yu Xiao

Amazon Web Services. 18.11.2015 Yu Xiao Amazon Web Services 18.11.2015 Yu Xiao Agenda Introduction to Amazon Web Services(AWS) 7 Steps to Select the Right Architecture for Your Web Applications Private, Public or Hybrid Cloud? AWS Case Study

More information

Why Big Data in the Cloud?

Why Big Data in the Cloud? Have 40 Why Big Data in the Cloud? Colin White, BI Research January 2014 Sponsored by Treasure Data TABLE OF CONTENTS Introduction The Importance of Big Data The Role of Cloud Computing Using Big Data

More information

Modern IT Operations Management. Why a New Approach is Required, and How Boundary Delivers

Modern IT Operations Management. Why a New Approach is Required, and How Boundary Delivers Modern IT Operations Management Why a New Approach is Required, and How Boundary Delivers TABLE OF CONTENTS EXECUTIVE SUMMARY 3 INTRODUCTION: CHANGING NATURE OF IT 3 WHY TRADITIONAL APPROACHES ARE FAILING

More information

Microsoft Private Cloud Fast Track

Microsoft Private Cloud Fast Track Microsoft Private Cloud Fast Track Microsoft Private Cloud Fast Track is a reference architecture designed to help build private clouds by combining Microsoft software with Nutanix technology to decrease

More information

Non-Stop Hadoop Paul Scott-Murphy VP Field Techincal Service, APJ. Cloudera World Japan November 2014

Non-Stop Hadoop Paul Scott-Murphy VP Field Techincal Service, APJ. Cloudera World Japan November 2014 Non-Stop Hadoop Paul Scott-Murphy VP Field Techincal Service, APJ Cloudera World Japan November 2014 WANdisco Background WANdisco: Wide Area Network Distributed Computing Enterprise ready, high availability

More information

On- Prem MongoDB- as- a- Service Powered by the CumuLogic DBaaS Platform

On- Prem MongoDB- as- a- Service Powered by the CumuLogic DBaaS Platform On- Prem MongoDB- as- a- Service Powered by the CumuLogic DBaaS Platform Page 1 of 16 Table of Contents Table of Contents... 2 Introduction... 3 NoSQL Databases... 3 CumuLogic NoSQL Database Service...

More information

Building Success on Acquia Cloud:

Building Success on Acquia Cloud: Building Success on Acquia Cloud: 10 Layers of PaaS TECHNICAL Guide Table of Contents Executive Summary.... 3 Introducing the 10 Layers of PaaS... 4 The Foundation: Five Layers of PaaS Infrastructure...

More information

Future-Proofing MySQL for the Worldwide Data Revolution

Future-Proofing MySQL for the Worldwide Data Revolution Future-Proofing MySQL for the Worldwide Data Revolution Robert Hodges, CEO. What is Future-Proo!ng? Future-proo!ng = creating systems that last while parts change and improve MySQL is not losing out to

More information

Towards Smart and Intelligent SDN Controller

Towards Smart and Intelligent SDN Controller Towards Smart and Intelligent SDN Controller - Through the Generic, Extensible, and Elastic Time Series Data Repository (TSDR) YuLing Chen, Dell Inc. Rajesh Narayanan, Dell Inc. Sharon Aicler, Cisco Systems

More information

Intel IT s Cloud Journey. Speaker: [speaker name], Intel IT

Intel IT s Cloud Journey. Speaker: [speaker name], Intel IT Intel IT s Cloud Journey Speaker: [speaker name], Intel IT Accelerating The Corporate IT Journey Cloud enables ubiquitous access to resources and applications, and workload flexibility Cloud IaaS Infrastructure

More information

PaaS - Platform as a Service Google App Engine

PaaS - Platform as a Service Google App Engine PaaS - Platform as a Service Google App Engine Pelle Jakovits 14 April, 2015, Tartu Outline Introduction to PaaS Google Cloud Google AppEngine DEMO - Creating applications Available Google Services Costs

More information

CUMULUX WHICH CLOUD PLATFORM IS RIGHT FOR YOU? COMPARING CLOUD PLATFORMS. Review Business and Technology Series www.cumulux.com

CUMULUX WHICH CLOUD PLATFORM IS RIGHT FOR YOU? COMPARING CLOUD PLATFORMS. Review Business and Technology Series www.cumulux.com ` CUMULUX WHICH CLOUD PLATFORM IS RIGHT FOR YOU? COMPARING CLOUD PLATFORMS Review Business and Technology Series www.cumulux.com Table of Contents Cloud Computing Model...2 Impact on IT Management and

More information

EMC s Enterprise Hadoop Solution. By Julie Lockner, Senior Analyst, and Terri McClure, Senior Analyst

EMC s Enterprise Hadoop Solution. By Julie Lockner, Senior Analyst, and Terri McClure, Senior Analyst White Paper EMC s Enterprise Hadoop Solution Isilon Scale-out NAS and Greenplum HD By Julie Lockner, Senior Analyst, and Terri McClure, Senior Analyst February 2012 This ESG White Paper was commissioned

More information