DTI Image Processing Pipeline and Cloud Computing Environment

Size: px
Start display at page:

Download "DTI Image Processing Pipeline and Cloud Computing Environment"

Transcription

1 DTI Image Processing Pipeline and Cloud Computing Environment Kyle Chard Computation Institute University of Chicago and Argonne National Laboratory

2 Introduction DTI image analysis requires the use of many tools QC Registration ROI Marking Fiber Tracking QC, Registration, ROI Marking, Fiber Tracking,.. Constructing analyses is challenging Data & tooldiscovery, selection, orchestration,.. We have made huge strides in terms of data Data formats, repositories, i protocols, metadata, CDEs We now need infrastructure to reduce the barriers that exist between data providers, tool developers, researchers, and clinicians Big Science. Small Labs o We have exceptional infrastructure for the 1%, what about the 99%? 2 DTI Pipelines and Cloud Infrastructure

3 Common Approach to Analysis Modify Install (Re)Run Script Camino 3 DTI Pipelines and Cloud Infrastructure

4 How can we improve? We need a platform where users can easily construct ct and execute eec teanalyses Using best of bread tools and pipelines Abstracting low level infrastructure and platform heterogeneity Supporting automation and parallelism Supporting experimentation => Make existing tools and common analyses mundane building blocks 4 DTI Pipelines and Cloud Infrastructure

5 DTI Metric Reproducibility Pipeline Ultimate Goal: Investigate the feasibility of using DTI in clinical practice Automatic calculation of DTI metrics (FA, MD) from 48 automatically generated ROIs Using existing tools to create a reusable analysis workflow that can be easily repeated Investigate the ability to scale analyses over large datasets Explore the reproducibility over a group of 20 subjects with 4 scans spread over 2 sessions 5 DTI Pipelines and Cloud Infrastructure

6 DTI Processing Pipeline (1) BVEC & BVAL DTI T1 1. ECC DTI (FSL) 2. BET DTI (FSL) 3. BET T1 (FSL) 4. Linear Registration DTI / T1 (FSL FLIRT) 5. DTI Fitting (FSL/Camino) 7. Linear Registration T1/Template (FSLFLIRT) FLIRT) 7. Non linear Registration T1/Template (FSLFNIRT) FNIRT) Template 9. Transform FA/MD to MNI space (FSL Applywarp) p) 8. Calculate ROI Mean FA/MD (AFNI 3dmaskave) Atlas Mask 6 DTI Pipelines and Cloud Infrastructure

7 DTI Processing Pipeline (2) DTI BVEC & BVAL 1. ECC DTI (FSL) 2. BET DTI (FSL) FA Template FA image 3. DTI Fitting (FSL/Camino) Linear Registration (FSL FLIRT) Non Linear Registration (FSL FNIRT) MD image Apply Warp coefficient 7 DTI Pipelines and Cloud Infrastructure FA in MNI space 6. Calculate ROI Mean (3dmaskave) MD in MNI space Atlas Mask

8 Approaches for Implementing Pipelines Scripts XNAT Pipeline Engine Globus Genomics Bash scripts written to Defined by code (XML SaaS for genomics execute tools on a + scripts) Graphical interface for single computer Overhead to include creation and execution Time consuming, error prone, hard to transfer knowledge tools, develop interfaces and create pipelines Supports ondemand provisioning based on pricing gpolicies Little support for parallelization Difficult to change tools/pipelines Tools installed dynamically when Some support for parallelization li required 8 DTI Pipelines and Cloud Infrastructure

9 DTI Pipeline Platform Globus Endpoints Galaxy & Manager Globus Transfer Galaxy Condor Shared File System Dynamic Scheduler Dynamic Worker Pool 9 DTI Pipelines and Cloud Infrastructure

10 DTI Pipelines in the Cloud NFS Gluster GridFTP Condor Schedule 10 DTI Pipelines and Cloud Infrastructure Camino

11 DTI Pipelines in Galaxy 11 DTI Pipelines and Cloud Infrastructure

12 Cloud Computing Leverages economies of scale to facilitate utility ty models Pay only for resources used 1 * 100 hours == 100 * 1 hour On demand and elastic access to unlimited capacity Addresses fluctuating requirements Software as a Service Web access to data through Platform as a Service defined interfaces Platform as a Service No management of hardware or low level tools Infrastructure as a Service 12 DTI Pipelines and Cloud Infrastructure

13 Challenges Moving to the Cloud Resource Selection: Comparing price, capabilities, performance, instance types (EBS, S,Instance store), toolperformance Tool Selection and Management: Finding tools, installing, configuring and using them in different environments Analysis/Resource Management: Developing structured and repeatable analyseswith different tools. Data transfer: Moving large amounts of data in/out of Cloud environmentreliably reliably andefficiently Scale and Parallelism: Scaling analyses by efficiently parallelizing across elastic infrastructure Security: Data and computation security HIPAA? 13 DTI Pipelines and Cloud Infrastructure

14 Amazon EC2 Pricing System Specifications Pricing CPU Units CPU Cores Memory On Demand Spot (Low) Spot (High) m1.large m1.xlarge m3.xlarge m3.2xlarge m2.xlarge m2.2xlarge m2.4xlarge DTI Pipelines and Cloud Infrastructure

15 Spot Pricing Volatility 15 DTI Pipelines and Cloud Infrastructure

16 Instance Performance and Pricing Time (M Minutes ) EBS Instance Store On Demand Spot (Low) Spot (High) Cos st per Subjec ($) 16 DTI Pipelines and Cloud Infrastructure

17 Pricing Multiple Analyses Per Node Co ost per Subject ($) On Demand Spot (Low) Spot (High) 17 DTI Pipelines and Cloud Infrastructure

18 Elastic Startup Cost 1:15:00 Tim me 1:00:00 0:45:00 0:30:00 ROI Calculation Tensor Fitting ECC & Registration Contextualize 0:15:00 0:00:00 New Worker Existing Worker Spot Price Queue 18 DTI Pipelines and Cloud Infrastructure

19 Data Transfer with Globus Online Reliable file transfer, sharing, syncing. Easy fire and forget file transfers Automatic fault recovery High performance Across multiple l security domains In place sharing of files with users and groups No IT required. Software as a Service (SaaS) o No client software installation i o New features automatically available 19 DTI Pipelines and Cloud Infrastructure

20 Transfer Comparison 20 DTI Pipelines and Cloud Infrastructure

21 Summary Structured pipelines simplify creation, execution and sharing of complex analyses and sharing of complex analyses Hosted as a service can further reduce barriers By outsourcing pipeline execution on the Cloud we can reduce overhead and costs Previously we took weeks to process ~100 scans o Using this approach < 5 cents a subject ($5 for 1 hour) What's next? Can we deliver this as a service? o Billing, security, paradigm shift, interactive tools Developing toolsheds for sharing tools and pipelines pp 21 DTI Pipelines and Cloud Infrastructure

22 Acknowledgements Mike Vannier, Xia Jiang, Farid Dahi Globus Online Ian Foster, Steve Tuecke, Rachana Ananthakrishnan Globus Genomics o Ravi Madduri, Paul Dave, Dina Sulakhe, Lukasz Lacinski 22 DTI Pipelines and Cloud Infrastructure

Large-scale Research Data Management and Analysis Using Globus Services. Ravi Madduri Argonne National Lab University of Chicago @madduri

Large-scale Research Data Management and Analysis Using Globus Services. Ravi Madduri Argonne National Lab University of Chicago @madduri Large-scale Research Data Management and Analysis Using Globus Services Ravi Madduri Argonne National Lab University of Chicago @madduri Outline Who we are Challenges in Big Data Management and Analysis

More information

Practical Solutions for Big Data Analytics

Practical Solutions for Big Data Analytics Practical Solutions for Big Data Analytics Ravi Madduri Computation Institute (madduri@anl.gov) Paul Dave (pdave@uchicago.edu) Dinanath Sulakhe (sulakhe@uchicago.edu) Alex Rodriguez (arodri7@uchicago.edu)

More information

The Globus Galaxies Platform: Delivering Science Gateways as a Service

The Globus Galaxies Platform: Delivering Science Gateways as a Service The Globus Galaxies Platform: Delivering Science Gateways as a Service Ravi Madduri 1, Kyle Chard 1, Ryan Chard 2, Lukasz Lacinski 1, Alex Rodriguez 1, Dinanath Sulakhe 1, David Kelly 1, Utpal Dave 1,

More information

How To Write A Blog Post On Globus

How To Write A Blog Post On Globus Globus Software as a Service data publication and discovery Kyle Chard, University of Chicago Computation Institute, chard@uchicago.edu Jim Pruyne, University of Chicago Computation Institute, pruyne@uchicago.edu

More information

Globus Genomics Tutorial GlobusWorld 2014

Globus Genomics Tutorial GlobusWorld 2014 Globus Genomics Tutorial GlobusWorld 2014 Agenda Overview of Globus Genomics Example Collaborations Demonstration Globus Genomics interface Globus Online integration Scenario 1: Using Globus Genomics for

More information

Cornell University Center for Advanced Computing

Cornell University Center for Advanced Computing Cornell University Center for Advanced Computing David A. Lifka - lifka@cac.cornell.edu Director - Cornell University Center for Advanced Computing (CAC) Director Research Computing - Weill Cornell Medical

More information

Infrastructure Clouds for Science and Education: Platform Tools

Infrastructure Clouds for Science and Education: Platform Tools Infrastructure Clouds for Science and Education: Platform Tools Kate Keahey, Renato J. Figueiredo, John Bresnahan, Mike Wilde, David LaBissoniere Argonne National Laboratory Computation Institute, University

More information

globus online Cloud Based Services for Science Steve Tuecke Computation Institute University of Chicago and Argonne National Laboratory

globus online Cloud Based Services for Science Steve Tuecke Computation Institute University of Chicago and Argonne National Laboratory globus online Cloud Based Services for Science Steve Tuecke Computation Institute University of Chicago and Argonne National Laboratory Cloud Layers Software-as-a-Service (SaaS) Platform-as-a-Service (PaaS)

More information

Cornell University Center for Advanced Computing A Sustainable Business Model for Advanced Research Computing

Cornell University Center for Advanced Computing A Sustainable Business Model for Advanced Research Computing Cornell University Center for Advanced Computing A Sustainable Business Model for Advanced Research Computing David A. Lifka lifka@cac.cornell.edu 4/20/13 www.cac.cornell.edu 1 My Background 2007 Cornell

More information

Deploying Business Virtual Appliances on Open Source Cloud Computing

Deploying Business Virtual Appliances on Open Source Cloud Computing International Journal of Computer Science and Telecommunications [Volume 3, Issue 4, April 2012] 26 ISSN 2047-3338 Deploying Business Virtual Appliances on Open Source Cloud Computing Tran Van Lang 1 and

More information

Cornell University Center for Advanced Computing

Cornell University Center for Advanced Computing Cornell University Center for Advanced Computing David A. Lifka - lifka@cac.cornell.edu Director - Cornell University Center for Advanced Computing (CAC) Director Research Computing - Weill Cornell Medical

More information

Oracle ERP & The Cloud. Presented by Adriaan Kruger

Oracle ERP & The Cloud. Presented by Adriaan Kruger Oracle ERP & The Cloud Presented by Adriaan Kruger ? What is the Cloud? What options are there in the Cloud for JD Edwards / E-Business Suite Customers? What is the Cloud? Different Layers IaaS (Infrastructure

More information

New resource provision paradigms for Grid Infrastructures: Virtualization and Cloud

New resource provision paradigms for Grid Infrastructures: Virtualization and Cloud CISCO NerdLunch Series November 7, 2008 San Jose, CA New resource provision paradigms for Grid Infrastructures: Virtualization and Cloud Ruben Santiago Montero Distributed Systems Architecture Research

More information

GridFTP GUI: An Easy and Efficient Way to Transfer Data in Grid

GridFTP GUI: An Easy and Efficient Way to Transfer Data in Grid GridFTP GUI: An Easy and Efficient Way to Transfer Data in Grid Wantao Liu 1,2 Raj Kettimuthu 2,3, Brian Tieman 3, Ravi Madduri 2,3, Bo Li 1, and Ian Foster 2,3 1 Beihang University, Beijing, China 2 The

More information

Big process for big data

Big process for big data Big process for big data Process automa9on for data- driven science Ian Foster Computa9on Ins9tute Argonne Na9onal Laboratory & The University of Chicago Talk at Astroinforma9cs 2012, Redmond, September

More information

Workflow Tools at NERSC. Debbie Bard djbard@lbl.gov NERSC Data and Analytics Services

Workflow Tools at NERSC. Debbie Bard djbard@lbl.gov NERSC Data and Analytics Services Workflow Tools at NERSC Debbie Bard djbard@lbl.gov NERSC Data and Analytics Services NERSC User Meeting August 13th, 2015 What Does Workflow Software Do? Automate connection of applications Chain together

More information

Planning, Provisioning and Deploying Enterprise Clouds with Oracle Enterprise Manager 12c Kevin Patterson, Principal Sales Consultant, Enterprise

Planning, Provisioning and Deploying Enterprise Clouds with Oracle Enterprise Manager 12c Kevin Patterson, Principal Sales Consultant, Enterprise Planning, Provisioning and Deploying Enterprise Clouds with Oracle Enterprise Manager 12c Kevin Patterson, Principal Sales Consultant, Enterprise Manager Oracle NIST Definition of Cloud Computing Cloud

More information

globus online Cloud-based services for (reproducible) science Ian Foster Computation Institute University of Chicago and Argonne National Laboratory

globus online Cloud-based services for (reproducible) science Ian Foster Computation Institute University of Chicago and Argonne National Laboratory globus online Cloud-based services for (reproducible) science Ian Foster Computation Institute University of Chicago and Argonne National Laboratory Computation Institute (CI) Apply to challenging problems

More information

NIH Commons Overview, Framework & Pilots - Version 1. The NIH Commons

NIH Commons Overview, Framework & Pilots - Version 1. The NIH Commons The NIH Commons Summary The Commons is a shared virtual space where scientists can work with the digital objects of biomedical research, i.e. it is a system that will allow investigators to find, manage,

More information

Globus Research Data Management: Introduction and Service Overview. Steve Tuecke Vas Vasiliadis

Globus Research Data Management: Introduction and Service Overview. Steve Tuecke Vas Vasiliadis Globus Research Data Management: Introduction and Service Overview Steve Tuecke Vas Vasiliadis Presentations and other useful information available at globus.org/events/xsede15/tutorial 2 Thank you to

More information

Grid Computing vs Cloud

Grid Computing vs Cloud Chapter 3 Grid Computing vs Cloud Computing 3.1 Grid Computing Grid computing [8, 23, 25] is based on the philosophy of sharing information and power, which gives us access to another type of heterogeneous

More information

Cloud Computing and E-Commerce

Cloud Computing and E-Commerce Cloud Computing and E-Commerce Cloud Computing turns Computing Power into a Virtual Good for E-Commerrce is Implementation Partner of 4FriendsOnly.com Internet Technologies AG VirtualGoods, Koblenz, September

More information

Enhanced Research Data Management and Publication with Globus

Enhanced Research Data Management and Publication with Globus Enhanced Research Data Management and Publication with Globus Vas Vasiliadis Jim Pruyne Presented at OR2015 June 8, 2015 Presentations and other useful information available at globus.org/events/or2015/tutorial

More information

SC12 Cloud Compu,ng for Science Tutorial: Introduc,on to Infrastructure Clouds

SC12 Cloud Compu,ng for Science Tutorial: Introduc,on to Infrastructure Clouds SC12 Cloud Compu,ng for Science Tutorial: Introduc,on to Infrastructure Clouds John Breshnahan, Patrick Armstrong, Kate Keahey, Pierre Riteau Argonne National Laboratory Computation Institute, University

More information

Efficient Cloud Management for Parallel Data Processing In Private Cloud

Efficient Cloud Management for Parallel Data Processing In Private Cloud 2012 International Conference on Information and Network Technology (ICINT 2012) IPCSIT vol. 37 (2012) (2012) IACSIT Press, Singapore Efficient Cloud Management for Parallel Data Processing In Private

More information

INTRODUCTION TO CLOUD COMPUTING CEN483 PARALLEL AND DISTRIBUTED SYSTEMS

INTRODUCTION TO CLOUD COMPUTING CEN483 PARALLEL AND DISTRIBUTED SYSTEMS INTRODUCTION TO CLOUD COMPUTING CEN483 PARALLEL AND DISTRIBUTED SYSTEMS CLOUD COMPUTING Cloud computing is a model for enabling convenient, ondemand network access to a shared pool of configurable computing

More information

Introduction to Arvados. A Curoverse White Paper

Introduction to Arvados. A Curoverse White Paper Introduction to Arvados A Curoverse White Paper Contents Arvados in a Nutshell... 4 Why Teams Choose Arvados... 4 The Technical Architecture... 6 System Capabilities... 7 Commitment to Open Source... 12

More information

Scale Cloud Across the Enterprise

Scale Cloud Across the Enterprise Scale Cloud Across the Enterprise Chris Haddad Vice President, Technology Evangelism Follow me on Twitter @cobiacomm Read architecture guidance at http://blog.cobia.net/cobiacomm Skate towards the puck

More information

How To Build A Research Platform

How To Build A Research Platform Leveraging Digital Infrastructure and Innovative Software Services to Accelerate Scientific Discovery Hervé Guy and Steve Tuecke April 9, 2014 2014 Internet2 Global Summit Denver, CO Software to Support

More information

How to Do/Evaluate Cloud Computing Research. Young Choon Lee

How to Do/Evaluate Cloud Computing Research. Young Choon Lee How to Do/Evaluate Cloud Computing Research Young Choon Lee Cloud Computing Cloud computing is a model for enabling ubiquitous, convenient, on-demand network access to a shared pool of configurable computing

More information

Alternative Deployment Models for Cloud Computing in HPC Applications. Society of HPC Professionals November 9, 2011 Steve Hebert, Nimbix

Alternative Deployment Models for Cloud Computing in HPC Applications. Society of HPC Professionals November 9, 2011 Steve Hebert, Nimbix Alternative Deployment Models for Cloud Computing in HPC Applications Society of HPC Professionals November 9, 2011 Steve Hebert, Nimbix The case for Cloud in HPC Build it in house Assemble in the cloud?

More information

globus online The Research Cloud Steve Tuecke Computation Institute University of Chicago and Argonne National Laboratory

globus online The Research Cloud Steve Tuecke Computation Institute University of Chicago and Argonne National Laboratory globus online The Research Cloud Steve Tuecke Computation Institute University of Chicago and Argonne National Laboratory Cloud Layers Software-as-a-Service (SaaS) Platform-as-a-Service (PaaS) Infrastructure-as-a-Service

More information

Big Data on AWS. Services Overview. Bernie Nallamotu Principle Solutions Architect

Big Data on AWS. Services Overview. Bernie Nallamotu Principle Solutions Architect on AWS Services Overview Bernie Nallamotu Principle Solutions Architect \ So what is it? When your data sets become so large that you have to start innovating around how to collect, store, organize, analyze

More information

An Introduction to Virtualization and Cloud Technologies to Support Grid Computing

An Introduction to Virtualization and Cloud Technologies to Support Grid Computing New Paradigms: Clouds, Virtualization and Co. EGEE08, Istanbul, September 25, 2008 An Introduction to Virtualization and Cloud Technologies to Support Grid Computing Distributed Systems Architecture Research

More information

Cloud and Virtualization to Support Grid Infrastructures

Cloud and Virtualization to Support Grid Infrastructures ESAC GRID Workshop '08 ESAC, Villafranca del Castillo, Spain 11-12 December 2008 Cloud and Virtualization to Support Grid Infrastructures Distributed Systems Architecture Research Group Universidad Complutense

More information

CONDOR CLUSTERS ON EC2

CONDOR CLUSTERS ON EC2 CONDOR CLUSTERS ON EC2 Val Hendrix, Roberto A. Vitillo Lawrence Berkeley National Lab ATLAS Cloud Computing R & D 1 INTRODUCTION This is our initial work on investigating tools for managing clusters and

More information

Globus Research Data Management: Introduction and Service Overview

Globus Research Data Management: Introduction and Service Overview Globus Research Data Management: Introduction and Service Overview Kyle Chard chard@uchicago.edu Ben Blaiszik blaiszik@uchicago.edu Thank you to our sponsors! U. S. D E P A R T M E N T OF ENERGY 2 Agenda

More information

Plug-and-play Virtual Appliance Clusters Running Hadoop. Dr. Renato Figueiredo ACIS Lab - University of Florida

Plug-and-play Virtual Appliance Clusters Running Hadoop. Dr. Renato Figueiredo ACIS Lab - University of Florida Plug-and-play Virtual Appliance Clusters Running Hadoop Dr. Renato Figueiredo ACIS Lab - University of Florida Advanced Computing and Information Systems laboratory Introduction You have so far learned

More information

Grid Computing Vs. Cloud Computing

Grid Computing Vs. Cloud Computing International Journal of Information and Computation Technology. ISSN 0974-2239 Volume 3, Number 6 (2013), pp. 577-582 International Research Publications House http://www. irphouse.com /ijict.htm Grid

More information

CUMULUX WHICH CLOUD PLATFORM IS RIGHT FOR YOU? COMPARING CLOUD PLATFORMS. Review Business and Technology Series www.cumulux.com

CUMULUX WHICH CLOUD PLATFORM IS RIGHT FOR YOU? COMPARING CLOUD PLATFORMS. Review Business and Technology Series www.cumulux.com ` CUMULUX WHICH CLOUD PLATFORM IS RIGHT FOR YOU? COMPARING CLOUD PLATFORMS Review Business and Technology Series www.cumulux.com Table of Contents Cloud Computing Model...2 Impact on IT Management and

More information

Krishna Markande, Principal Architect Sridhar Murthy, Senior Architect. Unleashing the Potential of Cloud for Performance Testing

Krishna Markande, Principal Architect Sridhar Murthy, Senior Architect. Unleashing the Potential of Cloud for Performance Testing Krishna Markande, Principal Architect Sridhar Murthy, Senior Architect Unleashing the Potential of Cloud for Performance Testing 1 Agenda Software testing and Performance testing overview Leveraging cloud

More information

Scaling up to Production

Scaling up to Production 1 Scaling up to Production Overview Productionize then Scale Building Production Systems Scaling Production Systems Use Case: Scaling a Production Galaxy Instance Infrastructure Advice 2 PRODUCTIONIZE

More information

Enabling multi-cloud resources at CERN within the Helix Nebula project. D. Giordano (CERN IT-SDC) HEPiX Spring 2014 Workshop 23 May 2014

Enabling multi-cloud resources at CERN within the Helix Nebula project. D. Giordano (CERN IT-SDC) HEPiX Spring 2014 Workshop 23 May 2014 Enabling multi-cloud resources at CERN within the Helix Nebula project D. Giordano (CERN IT-) HEPiX Spring 2014 Workshop This document produced by Members of the Helix Nebula consortium is licensed under

More information

High Throughput Sequencing Data Analysis using Cloud Computing

High Throughput Sequencing Data Analysis using Cloud Computing High Throughput Sequencing Data Analysis using Cloud Computing Stéphane Le Crom (stephane.le_crom@upmc.fr) LBD - Université Pierre et Marie Curie (UPMC) Institut de Biologie de l École normale supérieure

More information

DataCenter optimization for Cloud Computing

DataCenter optimization for Cloud Computing DataCenter optimization for Cloud Computing Benjamín Barán National University of Asuncion (UNA) bbaran@pol.una.py Paraguay Content Cloud Computing Commercial Offerings Basic Problem Formulation Open Research

More information

Scalable Architecture on Amazon AWS Cloud

Scalable Architecture on Amazon AWS Cloud Scalable Architecture on Amazon AWS Cloud Kalpak Shah Founder & CEO, Clogeny Technologies kalpak@clogeny.com 1 * http://www.rightscale.com/products/cloud-computing-uses/scalable-website.php 2 Architect

More information

Cloud Computing Trends

Cloud Computing Trends UT DALLAS Erik Jonsson School of Engineering & Computer Science Cloud Computing Trends What is cloud computing? Cloud computing refers to the apps and services delivered over the internet. Software delivered

More information

Introduction to Cloud Computing

Introduction to Cloud Computing Introduction to Cloud Computing Adam Skogman, Jayway Photo by Mark Bonassera Start-up? Overwhelmed? Successful? Waiting for IT? Ease Didn t We Solve This? Flexibility Ease Didn t We Solve This? Web Hotel

More information

Cloud BioLinux: Pre-configured and On-demand Bioinformatics Computing for the Genomics Community

Cloud BioLinux: Pre-configured and On-demand Bioinformatics Computing for the Genomics Community Cloud BioLinux: Pre-configured and On-demand Bioinformatics Computing for the Genomics Community Ntinos Krampis Asst. Professor J. Craig Venter Institute kkrampis@jcvi.org http://www.jcvi.org/cms/about/bios/kkrampis/

More information

Towards a New Model for the Infrastructure Grid

Towards a New Model for the Infrastructure Grid INTERNATIONAL ADVANCED RESEARCH WORKSHOP ON HIGH PERFORMANCE COMPUTING AND GRIDS Cetraro (Italy), June 30 - July 4, 2008 Panel: From Grids to Cloud Services Towards a New Model for the Infrastructure Grid

More information

Cloud Testing Testing on the Cloud

Cloud Testing Testing on the Cloud Cloud Testing Global Technology Solutions Co-Author and Domain Knowledge Noman Khan, Director Technology Solutions Co-Author and Subject Matter Expert Ravi Kumar, Manager Technology Solutions Executive

More information

Cloud BioLinux: Pre-configured and On-demand Bioinformatics Computing for the Genomics Community

Cloud BioLinux: Pre-configured and On-demand Bioinformatics Computing for the Genomics Community Cloud BioLinux: Pre-configured and On-demand Bioinformatics Computing for the Genomics Community Ntinos Krampis Asst. Professor J. Craig Venter Institute kkrampis@jcvi.org http://www.jcvi.org/cms/about/bios/kkrampis/

More information

SUSE Cloud 2.0. Pete Chadwick. Douglas Jarvis. Senior Product Manager pchadwick@suse.com. Product Marketing Manager djarvis@suse.

SUSE Cloud 2.0. Pete Chadwick. Douglas Jarvis. Senior Product Manager pchadwick@suse.com. Product Marketing Manager djarvis@suse. SUSE Cloud 2.0 Pete Chadwick Douglas Jarvis Senior Product Manager pchadwick@suse.com Product Marketing Manager djarvis@suse.com SUSE Cloud SUSE Cloud is an open source software solution based on OpenStack

More information

How To Create A Grid On A Microsoft Web Server On A Pc Or Macode (For Free) On A Macode Or Ipad (For A Limited Time) On An Ipad Or Ipa (For Cheap) On Pc Or Micro

How To Create A Grid On A Microsoft Web Server On A Pc Or Macode (For Free) On A Macode Or Ipad (For A Limited Time) On An Ipad Or Ipa (For Cheap) On Pc Or Micro Welcome Grid on Demand Willem Toorop and Alain van Hoof {wtoorop,ahoof}@os3.nl June 30, 2010 Willem Toorop and Alain van Hoof (OS3) Grid on Demand June 30, 2010 1 / 39 Research Question Introduction Research

More information

Medical Image Processing on the GPU. Past, Present and Future. Anders Eklund, PhD Virginia Tech Carilion Research Institute andek@vtc.vt.

Medical Image Processing on the GPU. Past, Present and Future. Anders Eklund, PhD Virginia Tech Carilion Research Institute andek@vtc.vt. Medical Image Processing on the GPU Past, Present and Future Anders Eklund, PhD Virginia Tech Carilion Research Institute andek@vtc.vt.edu Outline Motivation why do we need GPUs? Past - how was GPU programming

More information

Description of Application

Description of Application Description of Application Operating Organization: Coeur d Alene Tribe, Plummer, Idaho Community of Interest: U.S. Indian tribes and their governments; rural governments OS and software requirements: Microsoft

More information

Globus for Data Management

Globus for Data Management Globus for Data Management Computation Institute Rachana Ananthakrishnan (ranantha@uchicago.edu) Data Management Challenges Transfers often take longer than expected based on available network capacities

More information

Solution for private cloud computing

Solution for private cloud computing The CC1 system Solution for private cloud computing 1 Outline What is CC1? Features Technical details Use cases By scientist By HEP experiment System requirements and installation How to get it? 2 What

More information

HEP Data-Intensive Distributed Cloud Computing System Requirements Specification Document

HEP Data-Intensive Distributed Cloud Computing System Requirements Specification Document HEP Data-Intensive Distributed Cloud Computing System Requirements Specification Document CANARIE NEP-101 Project University of Victoria HEP Computing Group December 18, 2013 Version 1.0 1 Revision History

More information

Outlook. Corporate Research and Technologies, Munich, Germany. 20 th May 2010

Outlook. Corporate Research and Technologies, Munich, Germany. 20 th May 2010 Computing Architecture Computing Introduction Computing Architecture Software Architecture for Outlook Corporate Research and Technologies, Munich, Germany Gerald Kaefer * 4 th Generation Datacenter IEEE

More information

SOA and Cloud in practice - An Example Case Study

SOA and Cloud in practice - An Example Case Study SOA and Cloud in practice - An Example Case Study 2 nd RECOCAPE Event "Emerging Software Technologies: Trends & Challenges Nov. 14 th 2012 ITIDA, Smart Village, Giza, Egypt Agenda What is SOA? What is

More information

Simplified Private Cloud Management

Simplified Private Cloud Management BUSINESS PARTNER ClouTor Simplified Private Cloud Management ClouTor ON VSPEX by LOCUZ INTRODUCTION ClouTor on VSPEX for Enterprises provides an integrated software solution for extending your existing

More information

Public Cloud Offerings and Private Cloud Options. Week 2 Lecture 4. M. Ali Babar

Public Cloud Offerings and Private Cloud Options. Week 2 Lecture 4. M. Ali Babar Public Cloud Offerings and Private Cloud Options Week 2 Lecture 4 M. Ali Babar Lecture Outline Public and private clouds Some key public cloud providers (More details in the lab) Private clouds Main Aspects

More information

Real Time Analysis of Advanced Photon Source Data

Real Time Analysis of Advanced Photon Source Data Real Time Analysis of Advanced Photon Source Data Dan Fraser (ANL) Director, Community Driven Improvement of Globus Software Brian Tieman (APS) And a host of others. ESRFUP WP11 Workshop Exploiting the

More information

The Cloud at Crawford. Evaluating the pros and cons of cloud computing and its use in claims management

The Cloud at Crawford. Evaluating the pros and cons of cloud computing and its use in claims management The Cloud at Crawford Evaluating the pros and cons of cloud computing and its use in claims management The Cloud at Crawford Wikipedia defines cloud computing as Internet-based computing, whereby shared

More information

International Symposium on Grid Computing 2009 April 23th, Academia Sinica, Taipei, Taiwan

International Symposium on Grid Computing 2009 April 23th, Academia Sinica, Taipei, Taiwan International Symposium on Grid Computing 2009 April 23th, Academia Sinica, Taipei, Taiwan New resource provision paradigms for Grid Infrastructures: Virtualization and Cloud Ruben Santiago Montero Distributed

More information

Database Scalability with Parallel Databases

Database Scalability with Parallel Databases Database Scalability with Parallel Databases Boston MySQL Meetup Monday, December 10, 2012 What should I do while you yabber along for 1 hour? 1. Eat Pizza 2. Ask questions, and keep me honest 3. Tweet

More information

Nimbus: Cloud Computing with Science

Nimbus: Cloud Computing with Science Nimbus: Cloud Computing with Science March 2010 globusworld, Chicago Kate Keahey keahey@mcs.anl.gov Nimbus Project University of Chicago Argonne National Laboratory Cloud Computing for Science Environment

More information

Scientific Computing Meets Big Data Technology: An Astronomy Use Case

Scientific Computing Meets Big Data Technology: An Astronomy Use Case Scientific Computing Meets Big Data Technology: An Astronomy Use Case Zhao Zhang AMPLab and BIDS UC Berkeley zhaozhang@cs.berkeley.edu In collaboration with Kyle Barbary, Frank Nothaft, Evan Sparks, Oliver

More information

Cloud computing - Architecting in the cloud

Cloud computing - Architecting in the cloud Cloud computing - Architecting in the cloud anna.ruokonen@tut.fi 1 Outline Cloud computing What is? Levels of cloud computing: IaaS, PaaS, SaaS Moving to the cloud? Architecting in the cloud Best practices

More information

OpenNebula Open Souce Solution for DC Virtualization. C12G Labs. Online Webinar

OpenNebula Open Souce Solution for DC Virtualization. C12G Labs. Online Webinar OpenNebula Open Souce Solution for DC Virtualization C12G Labs Online Webinar What is OpenNebula? Multi-tenancy, Elasticity and Automatic Provision on Virtualized Environments I m using virtualization/cloud,

More information

Hadoop & Spark Using Amazon EMR

Hadoop & Spark Using Amazon EMR Hadoop & Spark Using Amazon EMR Michael Hanisch, AWS Solutions Architecture 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Agenda Why did we build Amazon EMR? What is Amazon EMR?

More information

Cloud Computing Solutions for Genomics Across Geographic, Institutional and Economic Barriers

Cloud Computing Solutions for Genomics Across Geographic, Institutional and Economic Barriers Cloud Computing Solutions for Genomics Across Geographic, Institutional and Economic Barriers Ntinos Krampis Asst. Professor J. Craig Venter Institute kkrampis@jcvi.org http://www.jcvi.org/cms/about/bios/kkrampis/

More information

Deploying a Geospatial Cloud

Deploying a Geospatial Cloud Deploying a Geospatial Cloud Traditional Public Sector Computing Environment Traditional Computing Infrastructure Silos of dedicated hardware and software Single application per silo Expensive to size

More information

Cloud Computing for Control Systems CERN Openlab Summer Student Program 9/9/2011 ARSALAAN AHMED SHAIKH

Cloud Computing for Control Systems CERN Openlab Summer Student Program 9/9/2011 ARSALAAN AHMED SHAIKH Cloud Computing for Control Systems CERN Openlab Summer Student Program 9/9/2011 ARSALAAN AHMED SHAIKH CONTENTS Introduction... 4 System Components... 4 OpenNebula Cloud Management Toolkit... 4 VMware

More information

Cloud security CS642: Computer Security Professor Ristenpart h9p://www.cs.wisc.edu/~rist/ rist at cs dot wisc dot edu University of Wisconsin CS 642

Cloud security CS642: Computer Security Professor Ristenpart h9p://www.cs.wisc.edu/~rist/ rist at cs dot wisc dot edu University of Wisconsin CS 642 Cloud security CS642: Computer Security Professor Ristenpart h9p://www.cs.wisc.edu/~rist/ rist at cs dot wisc dot edu University of Wisconsin CS 642 Announcements Take- home final versus in- class Homework

More information

IBM Solutions Grid for Business Partners Helping IBM Business Partners to Grid-enable applications for the next phase of e-business on demand

IBM Solutions Grid for Business Partners Helping IBM Business Partners to Grid-enable applications for the next phase of e-business on demand PartnerWorld Developers IBM Solutions Grid for Business Partners Helping IBM Business Partners to Grid-enable applications for the next phase of e-business on demand 2 Introducing the IBM Solutions Grid

More information

Cloud Infrastructure Pattern

Cloud Infrastructure Pattern 1 st LACCEI International Symposium on Software Architecture and Patterns (LACCEI-ISAP-MiniPLoP 2012), July 23-27, 2012, Panama City, Panama. Cloud Infrastructure Pattern Keiko Hashizume Florida Atlantic

More information

globus online Reliable, high-performance file transfer as a service

globus online Reliable, high-performance file transfer as a service globus online Reliable, high-performance file transfer as a service Steve Tuecke Computation Institute University of Chicago and Argonne National Laboratory The Challenge: Moving Big Data Easily What should

More information

CLEVER: a CLoud-Enabled Virtual EnviRonment

CLEVER: a CLoud-Enabled Virtual EnviRonment CLEVER: a CLoud-Enabled Virtual EnviRonment Francesco Tusa Maurizio Paone Massimo Villari Antonio Puliafito {ftusa,mpaone,mvillari,apuliafito}@unime.it Università degli Studi di Messina, Dipartimento di

More information

Scalability and Cost of a Cloud-based Approach to Medical NLP

Scalability and Cost of a Cloud-based Approach to Medical NLP Scalability and Cost of a Cloud-based Approach to Medical NLP Kyle Chard, Michael Russell Computation Institute University of Chicago and Argonne National Laboratory Chicago, IL {kyle, mike}@ci.uchicago.edu

More information

Evolution from the Traditional Data Center to Exalogic: An Operational Perspective

Evolution from the Traditional Data Center to Exalogic: An Operational Perspective An Oracle White Paper July, 2012 Evolution from the Traditional Data Center to Exalogic: 1 Disclaimer The following is intended to outline our general product capabilities. It is intended for information

More information

PRIVATE CLOUD. November 6, 2013 Kay Metsker & Dan Mercer

PRIVATE CLOUD. November 6, 2013 Kay Metsker & Dan Mercer PRIVATE CLOUD November 6, 2013 Kay Metsker & Dan Mercer Agenda Deployment Milestones Service Strategy Service Benefits Feature Comparison Future Features Feature Set Matrix Consumption Models Financial

More information

By Cloud Spectator July 2013

By Cloud Spectator July 2013 Benchmark Report: Performance Analysis of VS. Amazon EC2 and Rackspace Cloud A standardized, side-by-side comparison of server performance, file IO, and internal network throughput. By Cloud Spectator

More information

How To Understand Cloud Computing

How To Understand Cloud Computing Overview of Cloud Computing (ENCS 691K Chapter 1) Roch Glitho, PhD Associate Professor and Canada Research Chair My URL - http://users.encs.concordia.ca/~glitho/ Overview of Cloud Computing Towards a definition

More information

Dynamic allocation of servers to jobs in a grid hosting environment

Dynamic allocation of servers to jobs in a grid hosting environment Dynamic allocation of s to in a grid hosting environment C Kubicek, M Fisher, P McKee and R Smith As computational resources become available for use over the Internet, a requirement has emerged to reconfigure

More information

SURVEY ON THE ALGORITHMS FOR WORKFLOW PLANNING AND EXECUTION

SURVEY ON THE ALGORITHMS FOR WORKFLOW PLANNING AND EXECUTION SURVEY ON THE ALGORITHMS FOR WORKFLOW PLANNING AND EXECUTION Kirandeep Kaur Khushdeep Kaur Research Scholar Assistant Professor, Department Of Cse, Bhai Maha Singh College Of Engineering, Bhai Maha Singh

More information

The CardioVascular Research Grid: Working to Make Collaborations Easier for the Cardiovascular Community

The CardioVascular Research Grid: Working to Make Collaborations Easier for the Cardiovascular Community The CardioVascular Research Grid: Working to Make Collaborations Easier for the Cardiovascular Community Stephen J. Granite, MS, MBA CVRG Program Manager NHLBI R24-HL085343 http://www.cvrgrid.org The CVRG

More information

Hybrid Development and Test USE CASE

Hybrid Development and Test USE CASE Hybrid Development and Test USE CASE CliQr Use Case: Hybrid Development and Test Page 2 Hybrid Development and Test Unlike the production phase, with its typically steady workload, development and test

More information

A Service for Data-Intensive Computations on Virtual Clusters

A Service for Data-Intensive Computations on Virtual Clusters A Service for Data-Intensive Computations on Virtual Clusters Executing Preservation Strategies at Scale Rainer Schmidt, Christian Sadilek, and Ross King rainer.schmidt@arcs.ac.at Planets Project Permanent

More information

Cloud Computing @ JPL Science Data Systems

Cloud Computing @ JPL Science Data Systems Cloud Computing @ JPL Science Data Systems Emily Law, GSAW 2011 Outline Science Data Systems (SDS) Space & Earth SDSs SDS Common Architecture Components Key Components using Cloud Computing Use Case 1:

More information

On Demand Satellite Image Processing

On Demand Satellite Image Processing On Demand Satellite Image Processing Next generation technology for processing Terabytes of imagery on the Cloud WHITEPAPER MARCH 2015 Introduction Profound changes are happening with computing hardware

More information

BITDEFENDER SECURITY FOR AMAZON WEB SERVICES

BITDEFENDER SECURITY FOR AMAZON WEB SERVICES BITDEFENDER SECURITY FOR AMAZON WEB SERVICES Beta Version Testing Guide Bitdefender Security for Amazon Web Services Beta Version Testing Guide Publication date 2015.03.04 Copyright 2015 Bitdefender Legal

More information

Federal Secure Cloud Testing as a Service - TaaS Center of Excellence (CoE) Robert L. Linton

Federal Secure Cloud Testing as a Service - TaaS Center of Excellence (CoE) Robert L. Linton Session 5: Federal Secure Cloud Testing as a Service - TaaS Center of Excellence (CoE) Robert L. Linton Agenda HP ALM Solution Review HP Cloud Potential Cloud Portal HP ALM Solutions in a virtual environment

More information

CHAPTER 8 CLOUD COMPUTING

CHAPTER 8 CLOUD COMPUTING CHAPTER 8 CLOUD COMPUTING SE 458 SERVICE ORIENTED ARCHITECTURE Assist. Prof. Dr. Volkan TUNALI Faculty of Engineering and Natural Sciences / Maltepe University Topics 2 Cloud Computing Essential Characteristics

More information

Cloud Computing with Nimbus

Cloud Computing with Nimbus Cloud Computing with Nimbus April 2010 Condor Week Kate Keahey keahey@mcs.anl.gov Nimbus Project University of Chicago Argonne National Laboratory Cloud Computing for Science Environment Complexity Consistency

More information

How To Run A Cloud At Cornell Cac

How To Run A Cloud At Cornell Cac On-Demand Research Computing Infrastructure as a Service Software as a Service Cloud Storage Solutions David A. Lifka Cornell Center for Advanced Computing lifka@cac.cornell.edu www.cac.cornell.edu 1 Cornell

More information

Cloud Computing For Bioinformatics

Cloud Computing For Bioinformatics Cloud Computing For Bioinformatics Cloud Computing: what is it? Cloud Computing is a distributed infrastructure where resources, software, and data are provided in an on-demand fashion. Cloud Computing

More information

Best Practices for Sharing Imagery using Amazon Web Services. Peter Becker

Best Practices for Sharing Imagery using Amazon Web Services. Peter Becker Best Practices for Sharing Imagery using Amazon Web Services Peter Becker Objectives Making Imagery Accessible Store massive volumes of imagery on inexpensive cloud storage Use elastic compute for image

More information

DISTRIBUTED SYSTEMS [COMP9243] Lecture 9a: Cloud Computing WHAT IS CLOUD COMPUTING? 2

DISTRIBUTED SYSTEMS [COMP9243] Lecture 9a: Cloud Computing WHAT IS CLOUD COMPUTING? 2 DISTRIBUTED SYSTEMS [COMP9243] Lecture 9a: Cloud Computing Slide 1 Slide 3 A style of computing in which dynamically scalable and often virtualized resources are provided as a service over the Internet.

More information