Cloud JPL Science Data Systems

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "Cloud Computing @ JPL Science Data Systems"

Transcription

1 Cloud JPL Science Data Systems Emily Law, GSAW 2011

2 Outline Science Data Systems (SDS) Space & Earth SDSs SDS Common Architecture Components Key Components using Cloud Computing Use Case 1: LMMP Use Case 2: ACCE Use Case 3: PO.DAAC Conclusion 03/02/2011 GSAW 2011 Cloud Computing Workshop 2

3 National Aeronautics and Space Administration Jet Propulsion Laboratory Science Data Systems (SDS) California Institute of Technology Pasadena, California Covers a wide variety of domain disciplines Solar system exploration, Astrophysics, Earth science, Biomedicine etc, Biomedicine, etc Each has its own communities, standards and systems But, there is a set of common components Some can greatly benefit from proven cloud computing technology 03/02/2011 GSAW 2011 Cloud Computing Workshop 3

4 Space Science Data Systems 03/02/2011 GSAW 2011 Cloud Computing Workshop 4

5 Earth Science Data Systems TDRS Network Network w/ Cloud Storage & Computation NASA Mission/Multi Mission Data & Science Centers Archive Data Centers Other Data Systems (e.g. NOAA) 03/02/2011 GSAW 2011 Cloud Computing Workshop 5

6 SDS Common Architecture Components Users WebService / API Analysis Visualization Tools Client Portal Presentation Ground Processing Center Search Transformation Transport Data Processing Application Service Data Processing Ingest File Manager Resource Manager Data Distribution Data Management Legends: System Layer Data System Component System Resources Catalog Metadata Data Repository Resource External Entity Interface Workflow Tracking Monitoring Security System Service 03/02/2011 GSAW 2011 Cloud Computing Workshop 6

7 Key components using Cloud Computing Data Analysis Portal Data Processing Data Repository Data Distribution Tools 03/02/2011 GSAW 2011 Cloud Computing Workshop 7

8 National Aeronautics and Space Administration Jet Propulsion Laboratory California Institute of Technology Pasadena, California Use Case 1: Lunar Mapping & Modeling Project (LMMP) Provides science and exploration community a suite of lunar mapping pp g and modeling tools and products that support the lunar exploration activities The tools and products are made available through a common common, intuitive NASA portal A public release of the system is scheduled at the end of March /02/2011 GSAW 2011 Cloud Computing Workshop 8

9 Challenge The image files LMMP manages range from a few gigabytes to hundreds d of gigabytes in size with new data arriving every day Lunar surface images are too large to efficiently load and manipulate in memory LMMP must make the data readily available in a timely manner for users to view and analyze LMMP needs to accommodate large numbers of users with minimal latency 03/02/2011 GSAW 2011 Cloud Computing Workshop 9

10 Cloud Computing Solutions Slice a large image into many small images and to merge and resize until the last merge and reduce yields a reasonably sized image that depicts the entire image Amazon E2C/S3 Used distributed approach with Elastic MapReduce to tile images Developed a hybrid solution (multi-tiered data access approach) to serve images to users by cloud storage 03/02/2011 GSAW 2011 Cloud Computing Workshop 10

11 Use Case 1: Results Computing performance Comparable especially for the new machines with significant processing capability EC2 s rental model offers better performance per dollar than having to purchase and maintain local servers Storage Pay for just the bandwidth consumed Eliminate the need to purchase extra hardware and bandwidth to handle the occasional spikes in usage Security Store only publicly available data on cloud Host private data on local servers to eliminate potential security hole 03/02/2011 GSAW 2011 Cloud Computing Workshop 11

12 Use Case 2: Airborne Cloud Computing Environment (ACCE) Multi-mission capability providing distributed SDS services applicable to space-borne missions File Management Workflow Management Resource Management Extend the existing services to utilize cloud services, commercial, community and private Storage Compute Resources A full service stack is deployed for each mission, utilizing cloud computing infrastructure. 03/02/2011 GSAW 2011 Cloud Computing Workshop 12

13 Cloud ACCE Explore the benefits of performing science data processing for airborne missions in the cloud Evaluate different cloud technologies Amazon EC2/S3 Elastic compute resources and on-demand d storage Eucalyptus Infrastructure software for establishing a private cloud Hadoop Distributed ib t File System (DFS) and MapReduce Increased processing performance on large data sets 03/02/2011 GSAW 2011 Cloud Computing Workshop 13

14 ACCE A typical science data processing job, Product Generation Executable (PGE), is an executable that performs a task on n inputs to produce outputs. The ACCE Cloud must be able find an available resource for the PGE, transfer both the PGE and its input files to that resource, execute the PGE, then retrieve the output files from that resource. Initiate Processing Review Results Science Data System Services File Mgmt Workflow Mgmt Resource Mgmt Input File(s) PGE PGE Local Data Storage Output File(s) localhost.jpl.nasa.gov 03/02/2011 GSAW 2011 Cloud Computing Workshop 14

15 Use Case 2: Results Challenges Host Environment Security Network Processing cost reduction No investment in capital required (upfront or refresh costs) Pay only for what you use Possible limiting factors for public cloud viability Support for ITAR-sensitive data Data transfer rates between JPL and commercial cloud JPL Firewall More work ahead Amazon EC2/S3 reported an ITAR Region available in Fall 2011 Continued benchmarking and optimization has demonstrated increased data transfer rates, mbps JPL developing a Virtual Private Cloud connection to Amazon, causing EC2 nodes to appear inside the JPL Firewall 03/02/2011 GSAW 2011 Cloud Computing Workshop 15

16 Use Case 3: Physical Oceanography Distributed Active Archive Center (PO.DAAC) Preserve NASA s ocean data and make these and related information accessible and meaningful 03/02/2011 GSAW 2011 Cloud Computing Workshop 16

17 Cloud PO.DAAC Understand and articulate the cost/benefit of cloud technologies for the NASA Data Centers Evaluate the cloud services provided by Amazon Pricing Data transfer procedures es Software deployment and monitoring procedures Conduct a pilot study using cloud service for processing oceanographic climatology, data management and data access Utilize MapReduce parallel processing algorithm Benchmark between cloud and standalone processing 03/02/2011 GSAW 2011 Cloud Computing Workshop 17

18 Science Approach Extract daily time-series and climatologies for different regions over Antarctica (from the cloud) Compute climate proxies: Spatial extend, onset and length of melting on the ice sheet Anomalous ous events e Temporal changes/trend in sigma-0 Examine patterns in the climate proxies and look for correlations with ocean/atmosphere indices 03/02/2011 GSAW 2011 Cloud Computing Workshop 18

19 Cloud Computing (Hadoop) solution Transfer Copy data into Hadoop HDFS from S3 26GB Uncompressed data (550M data points) 3x replication MapReduce Computation Jobs Compute daily/monthly climatologies Store climatology data, all time data points in HBase Add point average & standard dev. in future Query/Aggregate Data Scan ranges of data (lat/long bounding box) Aggregate and return average pixel values for given regions 03/02/2011 GSAW 2011 Cloud Computing Workshop 19

20 National Aeronautics and Space Administration Jet Propulsion Laboratory Use Case 3: Results California Institute of Technology Pasadena, California 10 Years at a glance Time In Seconds Time (in seconds) to complete climatology jobs Monthly Dailyy DB HZ 5 node AllTime 10 node Process Type Monthly Daily AllTime 03/02/2011 Oracle GSAW 2011 Cloud Computing Workshop Desktop node node

21 Conclusion The Silver Bullet? Cloud Computing 03/02/2011 GSAW 2011 Cloud Computing Workshop 21

22 Many Benefits Broad network access Accessible from anywhere On demand self service Increase/decrease number of machines based on user defined parameters Resource Pooling Shared pool of configurable computing resources Rapid elasticity Resizable compute capacity for unlimited growth Increase/decrease number of machines based on user defined parameters Measured Service Utility Computing, pay by the drink, rapidly provisioned 03/02/2011 GSAW 2011 Cloud Computing Workshop 22

23 But. There are disadvantages to watch out Privacy / Ownership Security Reliability Feasibility Standards 03/02/2011 GSAW 2011 Cloud Computing Workshop 23

24 Thank You Question 03/02/2011 GSAW 2011 Cloud Computing Workshop 24

Copyright 2012 California Institute of Technology. Government sponsorship acknowledged. ( )

Copyright 2012 California Institute of Technology. Government sponsorship acknowledged. ( ) Design and Implementation of a in the Cloud George Chang NASA/Jet Propulsion Laboratory 1 Use Cases Hybrid Cloud Design for a Portal Hadoop and Cloud Computing for Image Tiling 2 LMMP Lunar Mapping and

More information

Big Data, Cloud Computing, Spatial Databases Steven Hagan Vice President Server Technologies

Big Data, Cloud Computing, Spatial Databases Steven Hagan Vice President Server Technologies Big Data, Cloud Computing, Spatial Databases Steven Hagan Vice President Server Technologies Big Data: Global Digital Data Growth Growing leaps and bounds by 40+% Year over Year! 2009 =.8 Zetabytes =.08

More information

Creating A Galactic Plane Atlas With Amazon Web Services

Creating A Galactic Plane Atlas With Amazon Web Services Creating A Galactic Plane Atlas With Amazon Web Services G. Bruce Berriman 1*, Ewa Deelman 2, John Good 1, Gideon Juve 2, Jamie Kinney 3, Ann Merrihew 3, and Mats Rynge 2 1 Infrared Processing and Analysis

More information

Cloud Computing. Adam Barker

Cloud Computing. Adam Barker Cloud Computing Adam Barker 1 Overview Introduction to Cloud computing Enabling technologies Different types of cloud: IaaS, PaaS and SaaS Cloud terminology Interacting with a cloud: management consoles

More information

Cloud Computing. Course: Designing and Implementing Service Oriented Business Processes

Cloud Computing. Course: Designing and Implementing Service Oriented Business Processes Cloud Computing Supplementary slides Course: Designing and Implementing Service Oriented Business Processes 1 Introduction Cloud computing represents a new way, in some cases a more cost effective way,

More information

Cloud 101. Mike Gangl, Caltech/JPL, michael.e.gangl@jpl.nasa.gov 2015 California Institute of Technology. Government sponsorship acknowledged

Cloud 101. Mike Gangl, Caltech/JPL, michael.e.gangl@jpl.nasa.gov 2015 California Institute of Technology. Government sponsorship acknowledged Cloud 101 Mike Gangl, Caltech/JPL, michael.e.gangl@jpl.nasa.gov 2015 California Institute of Technology. Government sponsorship acknowledged Outline What is cloud computing? Cloud service models Deployment

More information

Cloud Computing Technology

Cloud Computing Technology Cloud Computing Technology The Architecture Overview Danairat T. Certified Java Programmer, TOGAF Silver danairat@gmail.com, +66-81-559-1446 1 Agenda What is Cloud Computing? Case Study Service Model Architectures

More information

How In-Memory Data Grids Can Analyze Fast-Changing Data in Real Time

How In-Memory Data Grids Can Analyze Fast-Changing Data in Real Time SCALEOUT SOFTWARE How In-Memory Data Grids Can Analyze Fast-Changing Data in Real Time by Dr. William Bain and Dr. Mikhail Sobolev, ScaleOut Software, Inc. 2012 ScaleOut Software, Inc. 12/27/2012 T wenty-first

More information

White Paper on CLOUD COMPUTING

White Paper on CLOUD COMPUTING White Paper on CLOUD COMPUTING INDEX 1. Introduction 2. Features of Cloud Computing 3. Benefits of Cloud computing 4. Service models of Cloud Computing 5. Deployment models of Cloud Computing 6. Examples

More information

Eucalyptus-Based. GSAW 2010 Working Group Session 11D. Nehal Desai

Eucalyptus-Based. GSAW 2010 Working Group Session 11D. Nehal Desai GSAW 2010 Working Group Session 11D Eucalyptus-Based Event Correlation Nehal Desai Member of the Tech. Staff, CSD/CSTS/CSRD, The Aerospace Corporation Dr. Craig A. Lee, lee@aero.org Senior Scientist, CSD/CSTS/CSRD,

More information

Open source software for building a private cloud

Open source software for building a private cloud Michael J Pan CEO & co-founder, nephosity COSCUP 15 August 2010 An introduction me 10+ years working on high performance (distributed, grid, cloud) computing at DreamWorks Animation, NASA JPL, NIH Center

More information

Open Cloud System. (Integration of Eucalyptus, Hadoop and AppScale into deployment of University Private Cloud)

Open Cloud System. (Integration of Eucalyptus, Hadoop and AppScale into deployment of University Private Cloud) Open Cloud System (Integration of Eucalyptus, Hadoop and into deployment of University Private Cloud) Thinn Thu Naing University of Computer Studies, Yangon 25 th October 2011 Open Cloud System University

More information

Deploying a Geospatial Cloud

Deploying a Geospatial Cloud Deploying a Geospatial Cloud Traditional Public Sector Computing Environment Traditional Computing Infrastructure Silos of dedicated hardware and software Single application per silo Expensive to size

More information

Cluster, Grid, Cloud Concepts

Cluster, Grid, Cloud Concepts Cluster, Grid, Cloud Concepts Kalaiselvan.K Contents Section 1: Cluster Section 2: Grid Section 3: Cloud Cluster An Overview Need for a Cluster Cluster categorizations A computer cluster is a group of

More information

Viswanath Nandigam Sriram Krishnan Chaitan Baru

Viswanath Nandigam Sriram Krishnan Chaitan Baru Viswanath Nandigam Sriram Krishnan Chaitan Baru Traditional Database Implementations for large-scale spatial data Data Partitioning Spatial Extensions Pros and Cons Cloud Computing Introduction Relevance

More information

Hadoop Distributed File System. Dhruba Borthakur Apache Hadoop Project Management Committee dhruba@apache.org dhruba@facebook.com

Hadoop Distributed File System. Dhruba Borthakur Apache Hadoop Project Management Committee dhruba@apache.org dhruba@facebook.com Hadoop Distributed File System Dhruba Borthakur Apache Hadoop Project Management Committee dhruba@apache.org dhruba@facebook.com Hadoop, Why? Need to process huge datasets on large clusters of computers

More information

Keystone Image Management System

Keystone Image Management System Image management solutions for satellite and airborne sensors Overview The Keystone Image Management System offers solutions that archive, catalogue, process and deliver digital images from a vast number

More information

http://media.amazonwebservices.com/pdf/aws_security_whitepaper.pdf http://media.amazonwebservices.com/pdf/aws_security_whitepaper.

http://media.amazonwebservices.com/pdf/aws_security_whitepaper.pdf http://media.amazonwebservices.com/pdf/aws_security_whitepaper. 1. Cloud Costs Questions a. Are there any better ways for us to predict / manage our transfer costs than what we discussed in the meeting today? [DSB] There is currently no methodology with AWS to purchase

More information

Introduction to Cloud Computing

Introduction to Cloud Computing Introduction to Cloud Computing Cloud Computing I (intro) 15 319, spring 2010 2 nd Lecture, Jan 14 th Majd F. Sakr Lecture Motivation General overview on cloud computing What is cloud computing Services

More information

Final Project Proposal. CSCI.6500 Distributed Computing over the Internet

Final Project Proposal. CSCI.6500 Distributed Computing over the Internet Final Project Proposal CSCI.6500 Distributed Computing over the Internet Qingling Wang 660795696 1. Purpose Implement an application layer on Hybrid Grid Cloud Infrastructure to automatically or at least

More information

See Appendix A for the complete definition which includes the five essential characteristics, three service models, and four deployment models.

See Appendix A for the complete definition which includes the five essential characteristics, three service models, and four deployment models. Cloud Strategy Information Systems and Technology Bruce Campbell What is the Cloud? From http://csrc.nist.gov/publications/nistpubs/800-145/sp800-145.pdf Cloud computing is a model for enabling ubiquitous,

More information

Using WebSphere Application Server on Amazon EC2. Speaker(s): Ed McCabe, Arthur Meloy

Using WebSphere Application Server on Amazon EC2. Speaker(s): Ed McCabe, Arthur Meloy Using WebSphere Application Server on Amazon EC2 Speaker(s): Ed McCabe, Arthur Meloy Cloud Computing for Developers Hosted by IBM and Amazon Web Services October 1, 2009 1 Agenda WebSphere Application

More information

SOLUTION WHITE PAPER. Building a flexible, intelligent cloud

SOLUTION WHITE PAPER. Building a flexible, intelligent cloud SOLUTION WHITE PAPER Building a flexible, intelligent cloud Table of Contents Executive summary 1 Building a hybrid cloud 2 Provision complete cloud services 3 Service catalog 3 The user portal 4 Multi-tier

More information

Cloud Computing Trends

Cloud Computing Trends UT DALLAS Erik Jonsson School of Engineering & Computer Science Cloud Computing Trends What is cloud computing? Cloud computing refers to the apps and services delivered over the internet. Software delivered

More information

Cloud Computing for Spacecraft Operations

Cloud Computing for Spacecraft Operations Working Group Out Brief Session 12F Cloud Computing for Spacecraft Operations Ramesh Rangachar, Intelsat; Mark Walker, Integral Systems, Inc. Session Goals To invoke lively discussions about cloud computing

More information

CIS 4930/6930 Spring 2014 Introduction to Data Science Data Intensive Computing. University of Florida, CISE Department Prof.

CIS 4930/6930 Spring 2014 Introduction to Data Science Data Intensive Computing. University of Florida, CISE Department Prof. CIS 4930/6930 Spring 2014 Introduction to Data Science Data Intensive Computing University of Florida, CISE Department Prof. Daisy Zhe Wang Cloud Computing and Amazon Web Services Cloud Computing Amazon

More information

ESPRESSO: An Encryption as a Service for Cloud Storage Systems

ESPRESSO: An Encryption as a Service for Cloud Storage Systems 8th International Conference on Autonomous Infrastructure, Management and Security ESPRESSO: An Encryption as a Service for Cloud Storage Systems Kang Seungmin 30 th Jun., 2014 Outline Introduction and

More information

CiteSeer x : A Cloud Perspective. Pradeep Teregowda, Bhuvan Urgaonkar, C. Lee Giles Pennsylvania State University

CiteSeer x : A Cloud Perspective. Pradeep Teregowda, Bhuvan Urgaonkar, C. Lee Giles Pennsylvania State University CiteSeer x : A Cloud Perspective Pradeep Teregowda, Bhuvan Urgaonkar, C. Lee Giles Pennsylvania State University Problem Definition apple Question: How to effectively move a digital library, CiteSeer x,

More information

Sistemi Operativi e Reti. Cloud Computing

Sistemi Operativi e Reti. Cloud Computing 1 Sistemi Operativi e Reti Cloud Computing Facoltà di Scienze Matematiche Fisiche e Naturali Corso di Laurea Magistrale in Informatica Osvaldo Gervasi ogervasi@computer.org 2 Introduction Technologies

More information

Cloud Computing and Amazon Web Services

Cloud Computing and Amazon Web Services Cloud Computing and Amazon Web Services Gary A. McGilvary edinburgh data.intensive research 1 OUTLINE 1. An Overview of Cloud Computing 2. Amazon Web Services 3. Amazon EC2 Tutorial 4. Conclusions 2 CLOUD

More information

Cloud computing - Architecting in the cloud

Cloud computing - Architecting in the cloud Cloud computing - Architecting in the cloud anna.ruokonen@tut.fi 1 Outline Cloud computing What is? Levels of cloud computing: IaaS, PaaS, SaaS Moving to the cloud? Architecting in the cloud Best practices

More information

Cloud Models and Platforms

Cloud Models and Platforms Cloud Models and Platforms Dr. Sanjay P. Ahuja, Ph.D. 2010-14 FIS Distinguished Professor of Computer Science School of Computing, UNF A Working Definition of Cloud Computing Cloud computing is a model

More information

Cloud Computing 159.735. Submitted By : Fahim Ilyas (08497461) Submitted To : Martin Johnson Submitted On: 31 st May, 2009

Cloud Computing 159.735. Submitted By : Fahim Ilyas (08497461) Submitted To : Martin Johnson Submitted On: 31 st May, 2009 Cloud Computing 159.735 Submitted By : Fahim Ilyas (08497461) Submitted To : Martin Johnson Submitted On: 31 st May, 2009 Table of Contents Introduction... 3 What is Cloud Computing?... 3 Key Characteristics...

More information

A CLOUD-BASED FRAMEWORK FOR ONLINE MANAGEMENT OF MASSIVE BIMS USING HADOOP AND WEBGL

A CLOUD-BASED FRAMEWORK FOR ONLINE MANAGEMENT OF MASSIVE BIMS USING HADOOP AND WEBGL A CLOUD-BASED FRAMEWORK FOR ONLINE MANAGEMENT OF MASSIVE BIMS USING HADOOP AND WEBGL *Hung-Ming Chen, Chuan-Chien Hou, and Tsung-Hsi Lin Department of Construction Engineering National Taiwan University

More information

C2030-280.Examcollection.Premium.Exam.34q

C2030-280.Examcollection.Premium.Exam.34q C2030-280.Examcollection.Premium.Exam.34q Number: C2030-280 Passing Score: 800 Time Limit: 120 min File Version: 32.2 http://www.gratisexam.com/ Exam Code: C2030-280 Exam Name: IBM Cloud Computing Infrastructure

More information

Clodoaldo Barrera Chief Technical Strategist IBM System Storage. Making a successful transition to Software Defined Storage

Clodoaldo Barrera Chief Technical Strategist IBM System Storage. Making a successful transition to Software Defined Storage Clodoaldo Barrera Chief Technical Strategist IBM System Storage Making a successful transition to Software Defined Storage Open Server Summit Santa Clara Nov 2014 Data at the core of everything Data is

More information

Session 3. the Cloud Stack, SaaS, PaaS, IaaS

Session 3. the Cloud Stack, SaaS, PaaS, IaaS Session 3. the Cloud Stack, SaaS, PaaS, IaaS The service models resemble a cascading architecture where services on a higher level, as identified by Weinhardt et.al. (2009); encapsulate functionality from

More information

Investigation of Cloud Computing: Applications and Challenges

Investigation of Cloud Computing: Applications and Challenges Investigation of Cloud Computing: Applications and Challenges Amid Khatibi Bardsiri Anis Vosoogh Fatemeh Ahoojoosh Research Branch, Islamic Azad University, Sirjan, Iran Research Branch, Islamic Azad University,

More information

yvette@yvetteagostini.it yvette@yvetteagostini.it

yvette@yvetteagostini.it yvette@yvetteagostini.it 1 The following is merely a collection of notes taken during works, study and just-for-fun activities No copyright infringements intended: all sources are duly listed at the end of the document This work

More information

Keywords Cloud computing, Cloud platforms, Eucalyptus, Amazon, OpenStack.

Keywords Cloud computing, Cloud platforms, Eucalyptus, Amazon, OpenStack. Volume 3, Issue 11, November 2013 ISSN: 2277 128X International Journal of Advanced Research in Computer Science and Software Engineering Research Paper Available online at: www.ijarcsse.com Cloud Platforms

More information

A STUDY ON CLOUD STORAGE

A STUDY ON CLOUD STORAGE Available Online at www.ijcsmc.com International Journal of Computer Science and Mobile Computing A Monthly Journal of Computer Science and Information Technology IJCSMC, Vol. 3, Issue. 5, May 2014, pg.966

More information

CiteSeer x in the Cloud

CiteSeer x in the Cloud Published in the 2nd USENIX Workshop on Hot Topics in Cloud Computing 2010 CiteSeer x in the Cloud Pradeep B. Teregowda Pennsylvania State University C. Lee Giles Pennsylvania State University Bhuvan Urgaonkar

More information

Cloud Lifecycle Management

Cloud Lifecycle Management Cloud Lifecycle Managing Cloud Services from Request to Retirement SOLUTION WHITE PAPER Table of Contents EXECUTIVE SUMMARY............................................... 1 CLOUD LIFECYCLE MANAGEMENT........................................

More information

How AWS Pricing Works

How AWS Pricing Works How AWS Pricing Works (Please consult http://aws.amazon.com/whitepapers/ for the latest version of this paper) Page 1 of 15 Table of Contents Table of Contents... 2 Abstract... 3 Introduction... 3 Fundamental

More information

Big Data on AWS. Services Overview. Bernie Nallamotu Principle Solutions Architect

Big Data on AWS. Services Overview. Bernie Nallamotu Principle Solutions Architect on AWS Services Overview Bernie Nallamotu Principle Solutions Architect \ So what is it? When your data sets become so large that you have to start innovating around how to collect, store, organize, analyze

More information

Data Refinery with Big Data Aspects

Data Refinery with Big Data Aspects International Journal of Information and Computation Technology. ISSN 0974-2239 Volume 3, Number 7 (2013), pp. 655-662 International Research Publications House http://www. irphouse.com /ijict.htm Data

More information

Relocating Windows Server 2003 Workloads

Relocating Windows Server 2003 Workloads Relocating Windows Server 2003 Workloads An Opportunity to Optimize From Complex Change to an Opportunity to Optimize There is much you need to know before you upgrade to a new server platform, and time

More information

White Paper. Cloud Native Advantage: Multi-Tenant, Shared Container PaaS. http://wso2.com Version 1.1 (June 19, 2012)

White Paper. Cloud Native Advantage: Multi-Tenant, Shared Container PaaS. http://wso2.com Version 1.1 (June 19, 2012) Cloud Native Advantage: Multi-Tenant, Shared Container PaaS Version 1.1 (June 19, 2012) Table of Contents PaaS Container Partitioning Strategies... 03 Container Tenancy... 04 Multi-tenant Shared Container...

More information

Ganzheitliches Datenmanagement

Ganzheitliches Datenmanagement Ganzheitliches Datenmanagement für Hadoop Michael Kohs, Senior Sales Consultant @mikchaos The Problem with Big Data Projects in 2016 Relational, Mainframe Documents and Emails Data Modeler Data Scientist

More information

Cloud-based Infrastructures. Serving INSPIRE needs

Cloud-based Infrastructures. Serving INSPIRE needs Cloud-based Infrastructures Serving INSPIRE needs INSPIRE Conference 2014 Workshop Sessions Benoit BAURENS, AKKA Technologies (F) Claudio LUCCHESE, CNR (I) June 16th, 2014 This content by the InGeoCloudS

More information

Remote sensing information cloud service: research and practice

Remote sensing information cloud service: research and practice Remote sensing information cloud service: research and practice Yang Banghui Dr., Ren Fuhu Prof. and Wang jinnian Prof. yangbh@radi.ac.cn +8613810963452 Content 1 Background 2 Studying and Designing 3

More information

Learning Management Redefined. Acadox Infrastructure & Architecture

Learning Management Redefined. Acadox Infrastructure & Architecture Learning Management Redefined Acadox Infrastructure & Architecture w w w. a c a d o x. c o m Outline Overview Application Servers Databases Storage Network Content Delivery Network (CDN) & Caching Queuing

More information

ESS event: Big Data in Official Statistics. Antonino Virgillito, Istat

ESS event: Big Data in Official Statistics. Antonino Virgillito, Istat ESS event: Big Data in Official Statistics Antonino Virgillito, Istat v erbi v is 1 About me Head of Unit Web and BI Technologies, IT Directorate of Istat Project manager and technical coordinator of Web

More information

StorReduce Technical White Paper Cloud-based Data Deduplication

StorReduce Technical White Paper Cloud-based Data Deduplication StorReduce Technical White Paper Cloud-based Data Deduplication See also at storreduce.com/docs StorReduce Quick Start Guide StorReduce FAQ StorReduce Solution Brief, and StorReduce Blog at storreduce.com/blog

More information

Apache Hadoop. Alexandru Costan

Apache Hadoop. Alexandru Costan 1 Apache Hadoop Alexandru Costan Big Data Landscape No one-size-fits-all solution: SQL, NoSQL, MapReduce, No standard, except Hadoop 2 Outline What is Hadoop? Who uses it? Architecture HDFS MapReduce Open

More information

ediscovery and Search of Enterprise Data in the Cloud

ediscovery and Search of Enterprise Data in the Cloud ediscovery and Search of Enterprise Data in the Cloud From Hype to Reality By John Patzakis & Eric Klotzko ediscovery and Search of Enterprise Data in the Cloud: From Hype to Reality Despite the enormous

More information

Hybrid Cloud Delivery Managing Cloud Services from Request to Retirement SOLUTION WHITE PAPER

Hybrid Cloud Delivery Managing Cloud Services from Request to Retirement SOLUTION WHITE PAPER Hybrid Cloud Delivery Managing Cloud Services from Request to Retirement SOLUTION WHITE PAPER Contents Executive Summary................................................ 1 Hybrid Cloud Delivery..............................................

More information

Oracle Big Data SQL Technical Update

Oracle Big Data SQL Technical Update Oracle Big Data SQL Technical Update Jean-Pierre Dijcks Oracle Redwood City, CA, USA Keywords: Big Data, Hadoop, NoSQL Databases, Relational Databases, SQL, Security, Performance Introduction This technical

More information

SAS BIG DATA SOLUTIONS ON AWS SAS FORUM ESPAÑA, OCTOBER 16 TH, 2014 IAN MEYERS SOLUTIONS ARCHITECT / AMAZON WEB SERVICES

SAS BIG DATA SOLUTIONS ON AWS SAS FORUM ESPAÑA, OCTOBER 16 TH, 2014 IAN MEYERS SOLUTIONS ARCHITECT / AMAZON WEB SERVICES SAS BIG DATA SOLUTIONS ON AWS SAS FORUM ESPAÑA, OCTOBER 16 TH, 2014 IAN MEYERS SOLUTIONS ARCHITECT / AMAZON WEB SERVICES AWS GLOBAL INFRASTRUCTURE 10 Regions 25 Availability Zones 51 Edge locations WHAT

More information

How AWS Pricing Works May 2015

How AWS Pricing Works May 2015 How AWS Pricing Works May 2015 (Please consult http://aws.amazon.com/whitepapers/ for the latest version of this paper) Page 1 of 15 Table of Contents Table of Contents... 2 Abstract... 3 Introduction...

More information

Where We Are. References. Cloud Computing. Levels of Service. Cloud Computing History. Introduction to Data Management CSE 344

Where We Are. References. Cloud Computing. Levels of Service. Cloud Computing History. Introduction to Data Management CSE 344 Where We Are Introduction to Data Management CSE 344 Lecture 25: DBMS-as-a-service and NoSQL We learned quite a bit about data management see course calendar Three topics left: DBMS-as-a-service and NoSQL

More information

Can High-Performance Interconnects Benefit Memcached and Hadoop?

Can High-Performance Interconnects Benefit Memcached and Hadoop? Can High-Performance Interconnects Benefit Memcached and Hadoop? D. K. Panda and Sayantan Sur Network-Based Computing Laboratory Department of Computer Science and Engineering The Ohio State University,

More information

Permanent Link: http://espace.library.curtin.edu.au/r?func=dbin-jump-full&local_base=gen01-era02&object_id=154091

Permanent Link: http://espace.library.curtin.edu.au/r?func=dbin-jump-full&local_base=gen01-era02&object_id=154091 Citation: Alhamad, Mohammed and Dillon, Tharam S. and Wu, Chen and Chang, Elizabeth. 2010. Response time for cloud computing providers, in Kotsis, G. and Taniar, D. and Pardede, E. and Saleh, I. and Khalil,

More information

Technical aspects of Cloud computing. Luís Ferreira Pires University of Twente Meeting of the NVvIR, 17 June 2010

Technical aspects of Cloud computing. Luís Ferreira Pires University of Twente Meeting of the NVvIR, 17 June 2010 Technical aspects of Cloud computing Luís Ferreira Pires University of Twente Meeting of the NVvIR, 17 June 2010 Personal motivation A lot of people have been talking about Cloud computing in virtually

More information

PROPOSAL To Develop an Enterprise Scale Disease Modeling Web Portal For Ascel Bio Updated March 2015

PROPOSAL To Develop an Enterprise Scale Disease Modeling Web Portal For Ascel Bio Updated March 2015 Enterprise Scale Disease Modeling Web Portal PROPOSAL To Develop an Enterprise Scale Disease Modeling Web Portal For Ascel Bio Updated March 2015 i Last Updated: 5/8/2015 4:13 PM3/5/2015 10:00 AM Enterprise

More information

Cloud Computing Training

Cloud Computing Training Cloud Computing Training TechAge Labs Pvt. Ltd. Address : C-46, GF, Sector 2, Noida Phone 1 : 0120-4540894 Phone 2 : 0120-6495333 TechAge Labs 2014 version 1.0 Cloud Computing Training Cloud Computing

More information

Migration Scenario: Migrating Backend Processing Pipeline to the AWS Cloud

Migration Scenario: Migrating Backend Processing Pipeline to the AWS Cloud Migration Scenario: Migrating Backend Processing Pipeline to the AWS Cloud Use case Figure 1: Company C Architecture (Before Migration) Company C is an automobile insurance claim processing company with

More information

CHAPTER 8 CLOUD COMPUTING

CHAPTER 8 CLOUD COMPUTING CHAPTER 8 CLOUD COMPUTING SE 458 SERVICE ORIENTED ARCHITECTURE Assist. Prof. Dr. Volkan TUNALI Faculty of Engineering and Natural Sciences / Maltepe University Topics 2 Cloud Computing Essential Characteristics

More information

Building Blocks of the Private Cloud

Building Blocks of the Private Cloud www.cloudtp.com Building Blocks of the Private Cloud Private clouds are exactly what they sound like. Your own instance of SaaS, PaaS, or IaaS that exists in your own data center, all tucked away, protected

More information

STeP-IN SUMMIT 2013. June 18 21, 2013 at Bangalore, INDIA. Performance Testing of an IAAS Cloud Software (A CloudStack Use Case)

STeP-IN SUMMIT 2013. June 18 21, 2013 at Bangalore, INDIA. Performance Testing of an IAAS Cloud Software (A CloudStack Use Case) 10 th International Conference on Software Testing June 18 21, 2013 at Bangalore, INDIA by Sowmya Krishnan, Senior Software QA Engineer, Citrix Copyright: STeP-IN Forum and Quality Solutions for Information

More information

How cloud computing can transform your business landscape

How cloud computing can transform your business landscape How cloud computing can transform your business landscape Introduction It seems like everyone is talking about the cloud. Cloud computing and cloud services are the new buzz words for what s really a not

More information

Sriram Krishnan, Ph.D. sriram@sdsc.edu

Sriram Krishnan, Ph.D. sriram@sdsc.edu Sriram Krishnan, Ph.D. sriram@sdsc.edu (Re-)Introduction to cloud computing Introduction to the MapReduce and Hadoop Distributed File System Programming model Examples of MapReduce Where/how to run MapReduce

More information

Oracle s Big Data solutions. Roger Wullschleger. <Insert Picture Here>

Oracle s Big Data solutions. Roger Wullschleger. <Insert Picture Here> s Big Data solutions Roger Wullschleger DBTA Workshop on Big Data, Cloud Data Management and NoSQL 10. October 2012, Stade de Suisse, Berne 1 The following is intended to outline

More information

Cloud Computing Now and the Future Development of the IaaS

Cloud Computing Now and the Future Development of the IaaS 2010 Cloud Computing Now and the Future Development of the IaaS Quanta Computer Division: CCASD Title: Project Manager Name: Chad Lin Agenda: What is Cloud Computing? Public, Private and Hybrid Cloud.

More information

ENABLING GLOBAL HADOOP WITH EMC ELASTIC CLOUD STORAGE

ENABLING GLOBAL HADOOP WITH EMC ELASTIC CLOUD STORAGE ENABLING GLOBAL HADOOP WITH EMC ELASTIC CLOUD STORAGE Hadoop Storage-as-a-Service ABSTRACT This White Paper illustrates how EMC Elastic Cloud Storage (ECS ) can be used to streamline the Hadoop data analytics

More information

HYBRID CLOUD SUPPORT FOR LARGE SCALE ANALYTICS AND WEB PROCESSING. Navraj Chohan, Anand Gupta, Chris Bunch, Kowshik Prakasam, and Chandra Krintz

HYBRID CLOUD SUPPORT FOR LARGE SCALE ANALYTICS AND WEB PROCESSING. Navraj Chohan, Anand Gupta, Chris Bunch, Kowshik Prakasam, and Chandra Krintz HYBRID CLOUD SUPPORT FOR LARGE SCALE ANALYTICS AND WEB PROCESSING Navraj Chohan, Anand Gupta, Chris Bunch, Kowshik Prakasam, and Chandra Krintz Overview Google App Engine (GAE) GAE Analytics Libraries

More information

9/26/2011. What is Virtualization? What are the different types of virtualization.

9/26/2011. What is Virtualization? What are the different types of virtualization. CSE 501 Monday, September 26, 2011 Kevin Cleary kpcleary@buffalo.edu What is Virtualization? What are the different types of virtualization. Practical Uses Popular virtualization products Demo Question,

More information

Scalable Architecture on Amazon AWS Cloud

Scalable Architecture on Amazon AWS Cloud Scalable Architecture on Amazon AWS Cloud Kalpak Shah Founder & CEO, Clogeny Technologies kalpak@clogeny.com 1 * http://www.rightscale.com/products/cloud-computing-uses/scalable-website.php 2 Architect

More information

A Service for Data-Intensive Computations on Virtual Clusters

A Service for Data-Intensive Computations on Virtual Clusters A Service for Data-Intensive Computations on Virtual Clusters Executing Preservation Strategies at Scale Rainer Schmidt, Christian Sadilek, and Ross King rainer.schmidt@arcs.ac.at Planets Project Permanent

More information

Cloud computing: the IBM point of view

Cloud computing: the IBM point of view Building an Smarter Planet with Dynamic Infrastructure Cloud computing: the IBM point of view Ciro Puglisi, Infrastructure Offering Leader, CEEMEA cpug@ch.ibm.com, +41 58 333 4157 Cloud Computing can go

More information

Technology and Cost Considerations for Cloud Deployment: Amazon Elastic Compute Cloud (EC2) Case Study

Technology and Cost Considerations for Cloud Deployment: Amazon Elastic Compute Cloud (EC2) Case Study Creating Value Delivering Solutions Technology and Cost Considerations for Cloud Deployment: Amazon Elastic Compute Cloud (EC2) Case Study Chris Zajac, NJDOT Bud Luo, Ph.D., Michael Baker Jr., Inc. Overview

More information

Reference Architecture and Best Practices for Virtualizing Hadoop Workloads Justin Murray VMware

Reference Architecture and Best Practices for Virtualizing Hadoop Workloads Justin Murray VMware Reference Architecture and Best Practices for Virtualizing Hadoop Workloads Justin Murray ware 2 Agenda The Hadoop Journey Why Virtualize Hadoop? Elasticity and Scalability Performance Tests Storage Reference

More information

Leveraging BlobSeer to boost up the deployment and execution of Hadoop applications in Nimbus cloud environments on Grid 5000

Leveraging BlobSeer to boost up the deployment and execution of Hadoop applications in Nimbus cloud environments on Grid 5000 Leveraging BlobSeer to boost up the deployment and execution of Hadoop applications in Nimbus cloud environments on Grid 5000 Alexandra Carpen-Amarie Diana Moise Bogdan Nicolae KerData Team, INRIA Outline

More information

HOSTWAY. FlexCloudTM. Servers

HOSTWAY. FlexCloudTM. Servers PERFECT FOR INSTANTLY DEPLOYING SERVERS AND RESOURCES ON DEMAND Designed to be one of the most flexible, reliable, and easy-to-use cloud hosting solutions on the market, FlexCloud are a scalable, on-demand

More information

Here comes the flood Tools for Big Data analytics. Guy Chesnot -June, 2012

Here comes the flood Tools for Big Data analytics. Guy Chesnot -June, 2012 Here comes the flood Tools for Big Data analytics Guy Chesnot -June, 2012 Agenda Data flood Implementations Hadoop Not Hadoop 2 Agenda Data flood Implementations Hadoop Not Hadoop 3 Forecast Data Growth

More information

Enterprise GIS Architecture Deployment Options. Andrew Sakowicz

Enterprise GIS Architecture Deployment Options. Andrew Sakowicz Enterprise GIS Architecture Deployment Options Andrew Sakowicz Audience Audience - Architects - Developers - Administrators - Project Managers Level: - Beginner / Intermediate Introduction Andrew Sakowicz

More information

Cloud Federation to Elastically Increase MapReduce Processing Resources

Cloud Federation to Elastically Increase MapReduce Processing Resources Cloud Federation to Elastically Increase MapReduce Processing Resources A.Panarello, A.Celesti, M. Villari, M. Fazio and A. Puliafito {apanarello,acelesti, mfazio, mvillari, apuliafito}@unime.it DICIEAMA,

More information

Bringing Big Data Modelling into the Hands of Domain Experts

Bringing Big Data Modelling into the Hands of Domain Experts Bringing Big Data Modelling into the Hands of Domain Experts David Willingham Senior Application Engineer MathWorks david.willingham@mathworks.com.au 2015 The MathWorks, Inc. 1 Data is the sword of the

More information

Cloud Computing Summary and Preparation for Examination

Cloud Computing Summary and Preparation for Examination Basics of Cloud Computing Lecture 8 Cloud Computing Summary and Preparation for Examination Satish Srirama Outline Quick recap of what we have learnt as part of this course How to prepare for the examination

More information

Agenda. Big Data & Hadoop ViPR HDFS Pivotal Big Data Suite & ViPR HDFS ViON Customer Feedback #EMCVIPR

Agenda. Big Data & Hadoop ViPR HDFS Pivotal Big Data Suite & ViPR HDFS ViON Customer Feedback #EMCVIPR 1 Agenda Big Data & Hadoop ViPR HDFS Pivotal Big Data Suite & ViPR HDFS ViON Customer Feedback 2 A World of Connected Devices Need a new data management architecture for Internet of Things 21% the % of

More information

Selecting the right Cloud. Three steps for determining the most appropriate Cloud strategy

Selecting the right Cloud. Three steps for determining the most appropriate Cloud strategy Selecting the right Cloud Three steps for determining the most appropriate Cloud strategy Selecting the most appropriate cloud model can be a challenging process for organisations and IT executives tasked

More information

Top. Enterprise Reasons to Select kiteworks by Accellion

Top. Enterprise Reasons to Select kiteworks by Accellion Top 10 Enterprise Reasons to Select kiteworks by Accellion Top 10 Enterprise Reasons to Select kiteworks Accellion enables enterprise organizations to enhance business productivity, while ensuring data

More information

Planning, Provisioning and Deploying Enterprise Clouds with Oracle Enterprise Manager 12c Kevin Patterson, Principal Sales Consultant, Enterprise

Planning, Provisioning and Deploying Enterprise Clouds with Oracle Enterprise Manager 12c Kevin Patterson, Principal Sales Consultant, Enterprise Planning, Provisioning and Deploying Enterprise Clouds with Oracle Enterprise Manager 12c Kevin Patterson, Principal Sales Consultant, Enterprise Manager Oracle NIST Definition of Cloud Computing Cloud

More information

High Performance Applications over the Cloud: Gains and Losses

High Performance Applications over the Cloud: Gains and Losses High Performance Applications over the Cloud: Gains and Losses Dr. Leila Ismail Faculty of Information Technology United Arab Emirates University leila@uaeu.ac.ae http://citweb.uaeu.ac.ae/citweb/profile/leila

More information

Session 5. Mixing and matching Public, Private and Hybrid Clouds for maximum benefits

Session 5. Mixing and matching Public, Private and Hybrid Clouds for maximum benefits Session 5. Mixing and matching Public, Private and Hybrid Clouds for maximum benefits Best of both/ Best of all regarding specific needs, based on the use of resources Hybrid cloud is simply a mix of private

More information

Cloud Computing at CDC Current Status and Future Plans Earl Baum. March, 2014

Cloud Computing at CDC Current Status and Future Plans Earl Baum. March, 2014 Cloud Computing at CDC Current Status and Future Plans Earl Baum March, 2014 1 Background Current Activities Agenda Use Cases, Shared Services and Other Considerations What s Next 2 Background Cloud Definition

More information

2) Xen Hypervisor 3) UEC

2) Xen Hypervisor 3) UEC 5. Implementation Implementation of the trust model requires first preparing a test bed. It is a cloud computing environment that is required as the first step towards the implementation. Various tools

More information

BMC Cloud Management Functional Architecture Guide TECHNICAL WHITE PAPER

BMC Cloud Management Functional Architecture Guide TECHNICAL WHITE PAPER BMC Cloud Management Functional Architecture Guide TECHNICAL WHITE PAPER Table of Contents Executive Summary............................................... 1 New Functionality...............................................

More information

Optimizing IT Deployment Issues

Optimizing IT Deployment Issues Optimizing IT Deployment Issues Trends and Challenges for Engineering Simulation Barbara Hutchings barbara.hutchings@ansys.com 1 Outline Deployment Challenges and Trends Extreme scale up and scale out

More information

CUMULUX WHICH CLOUD PLATFORM IS RIGHT FOR YOU? COMPARING CLOUD PLATFORMS. Review Business and Technology Series www.cumulux.com

CUMULUX WHICH CLOUD PLATFORM IS RIGHT FOR YOU? COMPARING CLOUD PLATFORMS. Review Business and Technology Series www.cumulux.com ` CUMULUX WHICH CLOUD PLATFORM IS RIGHT FOR YOU? COMPARING CLOUD PLATFORMS Review Business and Technology Series www.cumulux.com Table of Contents Cloud Computing Model...2 Impact on IT Management and

More information