HPC and Grid Concepts

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "HPC and Grid Concepts"

Transcription

1 HPC and Grid Concepts Divya MG CDAC Knowledge Park, Bangalore 16 th Feb 2012 Ahmedabad 1

2 Presentation Overview What is HPC Need for HPC HPC Tools Grid Concepts GARUDA Overview Grid Middleware 16 th Feb 2012 Ahmedabad 2

3 What is HPC "High-Performance Computing," or HPC, is the application of "supercomputers" to computational problems that are either too large for standard computers or would take too long. A HPC system, is essentially a network of nodes, each of which contains one or more processing chips, as well as its own memory. These nodes are Symmetric Multi- Processors (SMP) units High-performance computing (HPC) uses parallel processing for running advanced application programs efficiently, reliably and quickly. 16 th Feb 2012 Ahmedabad 3

4 HPC Architecture Cluster services Portal server Job Scheduler (LRM) SMP box/node having many CPUs 16th Feb 2012 Ahmedabad 4

5 Performance of HPC Clusters Performance of HPC is measured in terms of FLOPS ( Floating-point Operations per Second). 1FLOP = Number of Instructions per Cycle X Clock Speed in Hertz X Number of CPUs HPC term applies especially to systems that function above a teraflop or FLOPS 16 th Feb 2012 Ahmedabad 5

6 Paradigms of Parallel Programming on HPC Types of Applications PThreads, OpenMP on SMP/Nodes MPI/PVM on Cluster i.e across nodes 16 th Feb 2012 Ahmedabad 6

7 Need for HPC Applications Bio Chemistry: Protein Folding Chemistry: Materials Research Physics: Simulation of Heavenly bodies Environment Modeling: Earthquake, Weather Industry: Product Prototyping, Models to Calculate Risk, Processing seismographic data 16 th Feb 2012 Ahmedabad 7

8 C-DAC s HPCC Software Products C-MPI : Optimized implementation of MPI for Cluster of Multi Processors (CLUMPS). Both point-to-point and collective calls have been optimized. Effectively uses both shared and distributed memory of CLUMPS. C-PFS : Parallel File System Provides MPI-IO file system interface to parallel applications F90IDE : Integrated Development Environment for Fortran 77/90 that includes compiler, debugger, profiler, source code browser and Fortran 77 to F90 convertor. PCF90: An automatic parallelizing compiler for Fortran for SMP based architecture. DIViA: Parallel program correctness and performance debugger. Detects communication bottlenecks and supports message debugging. PARMON: Cluster monitoring tool. Monitors the cluster as a unified resource. Provides Web Interface for monitoring over internet. RMS: Resource Management Software for effective load balancing and Load Scheduling on clusters. 16 th Feb 2012 Ahmedabad 8

9 Grid Concepts 16 th Feb 2012 Ahmedabad 9

10 What is Grid computing Definition of Grid by Ian Foster and Carl Kesselman A computational grid is a hardware and software infrastructure that provides dependable, consistent, pervasive, and inexpensive access to high-end computational capabilities. Three-point Check list given by Ian Foster,, Father of Grid Computing coordinates resources that are not subject to centralized control using standard, open, general-purpose protocols and interfaces to deliver nontrivial qualities of service. Reference: Book on The Grid: Blueprint for a New Computing Infrastructure. Grid puts one abstraction layer above High Performance Computing System(s), facilitating co-ordinated and distributed sharing of resources across multiple geographical and administrative domains. 16 th Feb 2012 Ahmedabad 10

11 Origin of Grid computing Vision: The central idea is that computing should be as reliable, pervasive and transparent as a utility Information or computation power should be delivered on demand. (Apart from type and location) Origin: Conceived by academic and research communities. Internet computing grew from communication needs and GC originated from the needs of the scientific community. Create a dynamic computing environment for sharing resources and results Scale to accommodate petabytes of data, and teraflops of computing power, and keep costs down 16 th Feb 2012 Ahmedabad 11

12 Popular Grids 16th Feb 2012 Ahmedabad 12

13 What is Grid architecture? Grid Architecture can be described as the layers of building blocks, where each layer has a specific function, to accomplish Grid Computing Infrastructure. 16 th Feb 2012 Ahmedabad 13

14 Components of Grid Nagios, Monalisa Ganglia,GridICE Bioinformatics,Disaster management Applications Grid Portals, PSE s, API s Access Methods Debuggers, Compiler IDE, Profilers, Workflow tools SRB/SRM, Visualization, softwares Globus/ Glite/ Unicore, Torque/Loadleveler/SGE, Gridway/MOAB/Condor-G, GSI, Grid services, MPICH & MPICH-G2 Program Development Environment Storage & Visualization Scheduler, Middleware & Security Monitoring & Management CPU, Satellite terminals, Telescopes Computational Resources & devices GARUDA, NKN and Internet links Network/Communication Fabric 16 th Feb 2012 Ahmedabad 14

15 GARUDA Overview 16 th Feb 2012 Ahmedabad 15

16 Objectives Motivation: To Collaborate on Research and Engineering of Technologies, Architectures, Standards and Applications in Grid Computing. Garuda Grid is targeted at providing a facility for the scientific community, which would enable them to seamlessly access the distributed resources. 16 th Feb 2012 Ahmedabad 16

17 Connectivity Summary 16 th Feb 2012 Ahmedabad 17

18 GARUDA Backbone A ccess Terminal Submit No de/ Grid Head node Generic User Internet Access Access Terminal LAN Port Submit No de/ Grid Head node Access Termi nal Swit ch Local User C-DAC, Bangalore Swit ch Head Node Port Head No de Hea d Node Head No de Database LAN M P L S Access Local User Comp ut e Nodes Head No de Head Node Database Head No de Local User C-DAC, Pune LAN Port A ccess Terminal Head No de Port Swit ch Hea d Node Switch Partner without resources Teles cope Head No de Grid Hea d node Database Storage A ccess Terminals 16th Feb 2012 Partner with resources Ahmedabad 18

19 National Knowledge Network NKN 16 th Feb 2012 Ahmedabad 19

20 GARUDA Grid: Architecture Grid-Enabled Applications Resource Enabler & Monitoring CLI Federated Information Server Access Portal Workflow tool Grid PSE Visualization Job Scheduler WSRF+GT4 + other Services +Cloud S/W Virtualization support Grid Programming & Development Environment Grid Security and High-Performance Grid Networking Data Grid NKN CDAC Resource centers Non-Research Organizations Research Organizations Computing Resources and Virtual Organizations Educational institutions Computing Centers Resources Security Middleware Resource Management User Environments Programming Environments Data Grid Grid Applications 16 th Feb 2012 Ahmedabad 20

21 GARUDA S/W Architecture Management, Monitoring & Accounting Paryaveekshanam GARUDA Information Service Ganglia Nagios GARUDA Accounting GARUDA Resources Compute, Data, Storage, Scientific Instruments, Application Specific Software,.. Resource Mgmt & Scheduling GridWay Meta-scheduler Resource Reservation Torque, Load Leveler Globus 4.x (WS Components) Access Methods Access Portal Problem Solving Environments Cmd line interface Visuvalization gateways workflows Security Framework IGCA Certificates MyProxy VOMS 16 th Feb 2012 Ahmedabad 21

22 GARUDA : Garuda Partner Resources Institutions Space Application Centre Indian Institute of Science Raman Research Institute Institute of Mathematical Sciences Madras Institute of Technology Indian Institute of Technology Jawaharlal Nehru University Location Ahmedabad Bangalore Bangalore Chennai Chennai Delhi Delhi Institute of Genomics and Integrative Biology Indian Institute of Technology University of Hyderabad Indian Institute of Technology Physical Research Laboratory Institute of Microbial Technology University of Pune Delhi Guwahati Hyderabad Kharagapur Ahmedabad Chandigarh Pune 16 th Feb 2012 Ahmedabad 22

23 Grid Middleware 16 th Feb 2012 Ahmedabad 23

24 What is a Grid Middleware Grid Middleware is a layer of S/W to enforce all the properties of: Scalability, Transparency, Heterogeneity, Fault Tolerance and Security of the Grid below the application layers. It provides an uniform interface of the Grid to users and handles all the complexity generated due to heterogeneous systems. Middleware S/W is a layer between grid applications and low level functionality of grid. 16 th Feb 2012 Ahmedabad 24

25 Popular Grid Middlewares GT - Globus Toolkit (Argonne National Laboratory, Chicago) GRIA Grid Resources for Industrial Application (University of Southampton ) Moab Grid Suite Cluster Resources Inc. Condor Project grid computing engine by Univ. of Wisconsin Gridbus Univ. of Melbourne, for e-business and e-science Unicore Uniform Interface to Computing resources - Univ. of Virginia, for NPACI (National Partnership for Advanced Computational Infrastructure.edu) NorduGrid Middleware ARC (Advanced Resource Connector) Legion Univ. of Virginia Glite - LightWeight Middleware for Grid computing( EGEE) 16 th Feb 2012 Ahmedabad 25

26 Core functionalities of Grid Middleware 16 th Feb 2012 Ahmedabad 26

27 Middleware functionalities Security Job Management Data Management Information Management 16 th Feb 2012 Ahmedabad 27

28 Grid Middleware: Security User/ Resource Authentication User/ Resource Authorization Access Control Lists User and Resource Policies Virtual Organization 16 th Feb 2012 Ahmedabad 28

29 Grid Middleware: Job Management Submission, Status Query, Cancel & Destroy, Getting Output & Error Support an open Job Description Language RSL, JDL, JSDL Transferring input/output data from/to remote source/destination Support Serial/ Parallel Jobs (Heterogeneous & Homogeneous) Integration with all Local Resource Managers 16 th Feb 2012 Ahmedabad 29

30 Grid Middleware: Data Management Two Basic Categories of Data Management Data Movement Secure Robust Efficient Third party movement Data Replication One or more copies or replicas Survive loss Easy availability Reduce access latency, increase robustness, scalability and performance for distributed applications. 16 th Feb 2012 Ahmedabad 30

31 Grid Middleware: Information Management System information is critical to operation of the grid and construction of applications How does an application determine what resources are available? What is the state of the computational grid? How can we optimize an application based on configuration of the underlying system? We need a general information infrastructure to answer these questions 16 th Feb 2012 Ahmedabad 31

32 Examples of Useful Information Characteristics of a compute resource IP address, software available, hostname, nodes available, OS version, load, softwares, libraries & their licenses. Characteristics of a network Bandwidth and latency, protocols Characteristics of the Middleware infrastructure Hosts, local resource managers 16 th Feb 2012 Ahmedabad 32

33 Thank You 16 th Feb 2012 Ahmedabad 33

Cluster, Grid, Cloud Concepts

Cluster, Grid, Cloud Concepts Cluster, Grid, Cloud Concepts Kalaiselvan.K Contents Section 1: Cluster Section 2: Grid Section 3: Cloud Cluster An Overview Need for a Cluster Cluster categorizations A computer cluster is a group of

More information

An approach to grid scheduling by using Condor-G Matchmaking mechanism

An approach to grid scheduling by using Condor-G Matchmaking mechanism An approach to grid scheduling by using Condor-G Matchmaking mechanism E. Imamagic, B. Radic, D. Dobrenic University Computing Centre, University of Zagreb, Croatia {emir.imamagic, branimir.radic, dobrisa.dobrenic}@srce.hr

More information

Vangelis Floros, GRNET S.A. 3 rd Open Source Software Conference March 22, 2008 NTUA, Athens Greece

Vangelis Floros, GRNET S.A. 3 rd Open Source Software Conference March 22, 2008 NTUA, Athens Greece Vangelis Floros, GRNET S.A. 3 rd Open Source Software Conference March 22, 2008 NTUA, Athens Greece Introduction What is a Grid? What is escience? Large Scientific Grids The example of EGEE Building Grid

More information

The Lattice Project: A Multi-Model Grid Computing System. Center for Bioinformatics and Computational Biology University of Maryland

The Lattice Project: A Multi-Model Grid Computing System. Center for Bioinformatics and Computational Biology University of Maryland The Lattice Project: A Multi-Model Grid Computing System Center for Bioinformatics and Computational Biology University of Maryland Parallel Computing PARALLEL COMPUTING a form of computation in which

More information

Concepts and Architecture of the Grid. Summary of Grid 2, Chapter 4

Concepts and Architecture of the Grid. Summary of Grid 2, Chapter 4 Concepts and Architecture of the Grid Summary of Grid 2, Chapter 4 Concepts of Grid Mantra: Coordinated resource sharing and problem solving in dynamic, multi-institutional virtual organizations Allows

More information

Introduction to grid technologies, parallel and cloud computing. Alaa Osama Allam Saida Saad Mohamed Mohamed Ibrahim Gaber

Introduction to grid technologies, parallel and cloud computing. Alaa Osama Allam Saida Saad Mohamed Mohamed Ibrahim Gaber Introduction to grid technologies, parallel and cloud computing Alaa Osama Allam Saida Saad Mohamed Mohamed Ibrahim Gaber OUTLINES Grid Computing Parallel programming technologies (MPI- Open MP-Cuda )

More information

Grid Scheduling Architectures with Globus GridWay and Sun Grid Engine

Grid Scheduling Architectures with Globus GridWay and Sun Grid Engine Grid Scheduling Architectures with and Sun Grid Engine Sun Grid Engine Workshop 2007 Regensburg, Germany September 11, 2007 Ignacio Martin Llorente Javier Fontán Muiños Distributed Systems Architecture

More information

New resource provision paradigms for Grid Infrastructures: Virtualization and Cloud

New resource provision paradigms for Grid Infrastructures: Virtualization and Cloud CISCO NerdLunch Series November 7, 2008 San Jose, CA New resource provision paradigms for Grid Infrastructures: Virtualization and Cloud Ruben Santiago Montero Distributed Systems Architecture Research

More information

Grid Computing vs Cloud

Grid Computing vs Cloud Chapter 3 Grid Computing vs Cloud Computing 3.1 Grid Computing Grid computing [8, 23, 25] is based on the philosophy of sharing information and power, which gives us access to another type of heterogeneous

More information

Storage Virtualization from clusters to grid

Storage Virtualization from clusters to grid Seanodes presents Storage Virtualization from clusters to grid Rennes 4th october 2007 Agenda Seanodes Presentation Overview of storage virtualization in clusters Seanodes cluster virtualization, with

More information

Web Service Based Data Management for Grid Applications

Web Service Based Data Management for Grid Applications Web Service Based Data Management for Grid Applications T. Boehm Zuse-Institute Berlin (ZIB), Berlin, Germany Abstract Web Services play an important role in providing an interface between end user applications

More information

Welcome. Grid on Demand. Willem Toorop and Alain van Hoof. {wtoorop,ahoof}@os3.nl. June 30, 2010

Welcome. Grid on Demand. Willem Toorop and Alain van Hoof. {wtoorop,ahoof}@os3.nl. June 30, 2010 Welcome Grid on Demand Willem Toorop and Alain van Hoof {wtoorop,ahoof}@os3.nl June 30, 2010 Willem Toorop and Alain van Hoof (OS3) Grid on Demand June 30, 2010 1 / 39 Research Question Introduction Research

More information

Grid Computing: A Ten Years Look Back. María S. Pérez Facultad de Informática Universidad Politécnica de Madrid mperez@fi.upm.es

Grid Computing: A Ten Years Look Back. María S. Pérez Facultad de Informática Universidad Politécnica de Madrid mperez@fi.upm.es Grid Computing: A Ten Years Look Back María S. Pérez Facultad de Informática Universidad Politécnica de Madrid mperez@fi.upm.es Outline Challenges not yet solved in computing The parents of grid Computing

More information

for my computation? Stefano Cozzini Which infrastructure Which infrastructure Democrito and SISSA/eLAB - Trieste

for my computation? Stefano Cozzini Which infrastructure Which infrastructure Democrito and SISSA/eLAB - Trieste Which infrastructure Which infrastructure for my computation? Stefano Cozzini Democrito and SISSA/eLAB - Trieste Agenda Introduction:! E-infrastructure and computing infrastructures! What is available

More information

Distributed Systems and Recent Innovations: Challenges and Benefits

Distributed Systems and Recent Innovations: Challenges and Benefits Distributed Systems and Recent Innovations: Challenges and Benefits 1. Introduction Krishna Nadiminti, Marcos Dias de Assunção, and Rajkumar Buyya Grid Computing and Distributed Systems Laboratory Department

More information

On Enabling Hydrodynamics Data Analysis of Analytical Ultracentrifugation Experiments

On Enabling Hydrodynamics Data Analysis of Analytical Ultracentrifugation Experiments On Enabling Hydrodynamics Data Analysis of Analytical Ultracentrifugation Experiments 18. June 2013 Morris Reidel, Shahbaz Memon, et al. Outline Background Ultrascan Application Ultrascan Software Components

More information

GRID COMPUTING Techniques and Applications BARRY WILKINSON

GRID COMPUTING Techniques and Applications BARRY WILKINSON GRID COMPUTING Techniques and Applications BARRY WILKINSON Contents Preface About the Author CHAPTER 1 INTRODUCTION TO GRID COMPUTING 1 1.1 Grid Computing Concept 1 1.2 History of Distributed Computing

More information

The Data Grid: Towards an Architecture for Distributed Management and Analysis of Large Scientific Datasets

The Data Grid: Towards an Architecture for Distributed Management and Analysis of Large Scientific Datasets The Data Grid: Towards an Architecture for Distributed Management and Analysis of Large Scientific Datasets!! Large data collections appear in many scientific domains like climate studies.!! Users and

More information

Write a technical report Present your results Write a workshop/conference paper (optional) Could be a real system, simulation and/or theoretical

Write a technical report Present your results Write a workshop/conference paper (optional) Could be a real system, simulation and/or theoretical Identify a problem Review approaches to the problem Propose a novel approach to the problem Define, design, prototype an implementation to evaluate your approach Could be a real system, simulation and/or

More information

OBJECTIVE. National Knowledge Network (NKN) project is aimed at

OBJECTIVE. National Knowledge Network (NKN) project is aimed at OBJECTIVE NKN AIMS TO BRING TOGETHER ALL THE STAKEHOLDERS FROM SCIENCE, TECHNOLOGY, HIGHER EDUCATION, HEALTHCARE, AGRICULTURE AND GOVERNANCE TO A COMMON PLATFORM. NKN is a revolutionary step towards creating

More information

An Experience in Accessing Grid Computing Power from Mobile Device with GridLab Mobile Services

An Experience in Accessing Grid Computing Power from Mobile Device with GridLab Mobile Services An Experience in Accessing Grid Computing Power from Mobile Device with GridLab Mobile Services Abstract In this paper review the notion of the use of mobile device in grid computing environment, We describe

More information

Interoperability between Sun Grid Engine and the Windows Compute Cluster

Interoperability between Sun Grid Engine and the Windows Compute Cluster Interoperability between Sun Grid Engine and the Windows Compute Cluster Steven Newhouse Program Manager, Windows HPC Team steven.newhouse@microsoft.com 1 Computer Cluster Roadmap Mainstream HPC Mainstream

More information

Principles and characteristics of distributed systems and environments

Principles and characteristics of distributed systems and environments Principles and characteristics of distributed systems and environments Definition of a distributed system Distributed system is a collection of independent computers that appears to its users as a single

More information

PARALLEL & CLUSTER COMPUTING CS 6260 PROFESSOR: ELISE DE DONCKER BY: LINA HUSSEIN

PARALLEL & CLUSTER COMPUTING CS 6260 PROFESSOR: ELISE DE DONCKER BY: LINA HUSSEIN 1 PARALLEL & CLUSTER COMPUTING CS 6260 PROFESSOR: ELISE DE DONCKER BY: LINA HUSSEIN Introduction What is cluster computing? Classification of Cluster Computing Technologies: Beowulf cluster Construction

More information

Towards a New Model for the Infrastructure Grid

Towards a New Model for the Infrastructure Grid INTERNATIONAL ADVANCED RESEARCH WORKSHOP ON HIGH PERFORMANCE COMPUTING AND GRIDS Cetraro (Italy), June 30 - July 4, 2008 Panel: From Grids to Cloud Services Towards a New Model for the Infrastructure Grid

More information

XSEDE Service Provider Software and Services Baseline. September 24, 2015 Version 1.2

XSEDE Service Provider Software and Services Baseline. September 24, 2015 Version 1.2 XSEDE Service Provider Software and Services Baseline September 24, 2015 Version 1.2 i TABLE OF CONTENTS XSEDE Production Baseline: Service Provider Software and Services... i A. Document History... A-

More information

Grid Scheduling Dictionary of Terms and Keywords

Grid Scheduling Dictionary of Terms and Keywords Grid Scheduling Dictionary Working Group M. Roehrig, Sandia National Laboratories W. Ziegler, Fraunhofer-Institute for Algorithms and Scientific Computing Document: Category: Informational June 2002 Status

More information

Grid Computing Vs. Cloud Computing

Grid Computing Vs. Cloud Computing International Journal of Information and Computation Technology. ISSN 0974-2239 Volume 3, Number 6 (2013), pp. 577-582 International Research Publications House http://www. irphouse.com /ijict.htm Grid

More information

Grid Computing With FreeBSD

Grid Computing With FreeBSD Grid Computing With FreeBSD USENIX ATC '04: UseBSD SIG Boston, MA, June 29 th 2004 Brooks Davis, Craig Lee The Aerospace Corporation El Segundo, CA {brooks,lee}aero.org http://people.freebsd.org/~brooks/papers/usebsd2004/

More information

Bibliography. University of Applied Sciences Fulda, Prof. Dr. S. Groß

Bibliography. University of Applied Sciences Fulda, Prof. Dr. S. Groß Slide III Bibliography 1) Abbas, A.: Grid Computing - A Practical Guide to Technology and Applications. Charles River Media, 2004. http://www.charlesriver.com/titles/gridcomputing.html 2) Berman, F.; et

More information

Service Oriented Distributed Manager for Grid System

Service Oriented Distributed Manager for Grid System Service Oriented Distributed Manager for Grid System Entisar S. Alkayal Faculty of Computing and Information Technology King Abdul Aziz University Jeddah, Saudi Arabia entisar_alkayal@hotmail.com Abstract

More information

Symmetric Multiprocessing

Symmetric Multiprocessing Multicore Computing A multi-core processor is a processing system composed of two or more independent cores. One can describe it as an integrated circuit to which two or more individual processors (called

More information

A Taxonomy and Survey of Grid Resource Planning and Reservation Systems for Grid Enabled Analysis Environment

A Taxonomy and Survey of Grid Resource Planning and Reservation Systems for Grid Enabled Analysis Environment A Taxonomy and Survey of Grid Resource Planning and Reservation Systems for Grid Enabled Analysis Environment Arshad Ali 3, Ashiq Anjum 3, Atif Mehmood 3, Richard McClatchey 2, Ian Willers 2, Julian Bunn

More information

Grids Computing and Collaboration

Grids Computing and Collaboration Grids Computing and Collaboration Arto Teräs CSC, the Finnish IT center for science University of Pune, India, March 12 th 2007 Grids Computing and Collaboration / Arto Teräs 2007-03-12 Slide

More information

Resource Management on Computational Grids

Resource Management on Computational Grids Univeristà Ca Foscari, Venezia http://www.dsi.unive.it Resource Management on Computational Grids Paolo Palmerini Dottorato di ricerca di Informatica (anno I, ciclo II) email: palmeri@dsi.unive.it 1/29

More information

Client/Server and Distributed Computing

Client/Server and Distributed Computing Adapted from:operating Systems: Internals and Design Principles, 6/E William Stallings CS571 Fall 2010 Client/Server and Distributed Computing Dave Bremer Otago Polytechnic, N.Z. 2008, Prentice Hall Traditional

More information

Program Grid and HPC5+ workshop

Program Grid and HPC5+ workshop Program Grid and HPC5+ workshop 24-30, Bahman 1391 Tuesday Wednesday 9.00-9.45 9.45-10.30 Break 11.00-11.45 11.45-12.30 Lunch 14.00-17.00 Workshop Rouhani Karimi MosalmanTabar Karimi G+MMT+K Opening IPM_Grid

More information

System Models for Distributed and Cloud Computing

System Models for Distributed and Cloud Computing System Models for Distributed and Cloud Computing Dr. Sanjay P. Ahuja, Ph.D. 2010-14 FIS Distinguished Professor of Computer Science School of Computing, UNF Classification of Distributed Computing Systems

More information

Concepts and Architecture of Grid Computing. Advanced Topics Spring 2008 Prof. Robert van Engelen

Concepts and Architecture of Grid Computing. Advanced Topics Spring 2008 Prof. Robert van Engelen Concepts and Architecture of Grid Computing Advanced Topics Spring 2008 Prof. Robert van Engelen Overview Grid users: who are they? Concept of the Grid Challenges for the Grid Evolution of Grid systems

More information

Status and Integration of AP2 Monitoring and Online Steering

Status and Integration of AP2 Monitoring and Online Steering Status and Integration of AP2 Monitoring and Online Steering Daniel Lorenz - University of Siegen Stefan Borovac, Markus Mechtel - University of Wuppertal Ralph Müller-Pfefferkorn Technische Universität

More information

Analyses on functional capabilities of BizTalk Server, Oracle BPEL Process Manger and WebSphere Process Server for applications in Grid middleware

Analyses on functional capabilities of BizTalk Server, Oracle BPEL Process Manger and WebSphere Process Server for applications in Grid middleware Analyses on functional capabilities of BizTalk Server, Oracle BPEL Process Manger and WebSphere Process Server for applications in Grid middleware R. Goranova University of Sofia St. Kliment Ohridski,

More information

Deploying a distributed data storage system on the UK National Grid Service using federated SRB

Deploying a distributed data storage system on the UK National Grid Service using federated SRB Deploying a distributed data storage system on the UK National Grid Service using federated SRB Manandhar A.S., Kleese K., Berrisford P., Brown G.D. CCLRC e-science Center Abstract As Grid enabled applications

More information

Distributed Systems LEEC (2005/06 2º Sem.)

Distributed Systems LEEC (2005/06 2º Sem.) Distributed Systems LEEC (2005/06 2º Sem.) Introduction João Paulo Carvalho Universidade Técnica de Lisboa / Instituto Superior Técnico Outline Definition of a Distributed System Goals Connecting Users

More information

An Efficient Use of Virtualization in Grid/Cloud Environments. Supervised by: Elisa Heymann Miquel A. Senar

An Efficient Use of Virtualization in Grid/Cloud Environments. Supervised by: Elisa Heymann Miquel A. Senar An Efficient Use of Virtualization in Grid/Cloud Environments. Arindam Choudhury Supervised by: Elisa Heymann Miquel A. Senar Index Introduction Motivation Objective State of Art Proposed Solution Experimentations

More information

Towards an E-Governance Grid for India (E-GGI): An Architectural Framework for Citizen Services Delivery

Towards an E-Governance Grid for India (E-GGI): An Architectural Framework for Citizen Services Delivery Towards an E-Governance Grid for India (E-GGI): An Architectural Framework for Citizen Services Delivery C. S. R. Prabhu 1 ABSTRACT The National e-governance Plan (NeGP) proposes citizen service delivery

More information

International Symposium on Grid Computing 2009 April 23th, Academia Sinica, Taipei, Taiwan

International Symposium on Grid Computing 2009 April 23th, Academia Sinica, Taipei, Taiwan International Symposium on Grid Computing 2009 April 23th, Academia Sinica, Taipei, Taiwan New resource provision paradigms for Grid Infrastructures: Virtualization and Cloud Ruben Santiago Montero Distributed

More information

An Introduction to Virtualization and Cloud Technologies to Support Grid Computing

An Introduction to Virtualization and Cloud Technologies to Support Grid Computing New Paradigms: Clouds, Virtualization and Co. EGEE08, Istanbul, September 25, 2008 An Introduction to Virtualization and Cloud Technologies to Support Grid Computing Distributed Systems Architecture Research

More information

Data Management in an International Data Grid Project. Timur Chabuk 04/09/2007

Data Management in an International Data Grid Project. Timur Chabuk 04/09/2007 Data Management in an International Data Grid Project Timur Chabuk 04/09/2007 Intro LHC opened in 2005 several Petabytes of data per year data created at CERN distributed to Regional Centers all over the

More information

So#ware Tools and Techniques for HPC, Clouds, and Server- Class SoCs Ron Brightwell

So#ware Tools and Techniques for HPC, Clouds, and Server- Class SoCs Ron Brightwell So#ware Tools and Techniques for HPC, Clouds, and Server- Class SoCs Ron Brightwell R&D Manager, Scalable System So#ware Department Sandia National Laboratories is a multi-program laboratory managed and

More information

IBM Platform Computing : infrastructure management for HPC solutions on OpenPOWER Jing Li, Software Development Manager IBM

IBM Platform Computing : infrastructure management for HPC solutions on OpenPOWER Jing Li, Software Development Manager IBM IBM Platform Computing : infrastructure management for HPC solutions on OpenPOWER Jing Li, Software Development Manager IBM #OpenPOWERSummit Join the conversation at #OpenPOWERSummit 1 Scale-out and Cloud

More information

Optimizing Performance of Parallel Programs on

Optimizing Performance of Parallel Programs on C-DAC & IIT Madras Five-Day Technology Workshop Programme ON Optimizing Performance of Parallel Programs on Emerging Multi-Core Processors and & GPUs OPECG-2009 Venue : Indian Institute of Technology Madras

More information

Anwendungsintegration und Workflows mit UNICORE 6

Anwendungsintegration und Workflows mit UNICORE 6 Mitglied der Helmholtz-Gemeinschaft Anwendungsintegration und Workflows mit UNICORE 6 Bernd Schuller und UNICORE-Team Jülich Supercomputing Centre, Forschungszentrum Jülich GmbH 26. November 2009 D-Grid

More information

GRIDSEED: A Virtual Training Grid Infrastructure

GRIDSEED: A Virtual Training Grid Infrastructure GRIDSEED: A Virtual Training Grid Infrastructure Iztok Gregori CNR-IOM DEMOCRITOS Trieste, Italy iztok@democritos.it Stefano Cozzini CNR-IOM DEMOCRITOS Trieste, Italy cozzini@democritos.it Tyanko Aleksiev

More information

GridWay: Open Source Meta-scheduling Technology for Grid Computing

GridWay: Open Source Meta-scheduling Technology for Grid Computing : Open Source Meta-scheduling Technology for Grid Computing Ruben S. Montero dsa-research.org Open Source Grid & Cluster Oakland CA, May 2008 Contents Introduction What is? Architecture & Components Scheduling

More information

Grid Computing @ Sun Carlo Nardone. Technical Systems Ambassador GSO Client Solutions

Grid Computing @ Sun Carlo Nardone. Technical Systems Ambassador GSO Client Solutions Grid Computing @ Sun Carlo Nardone Technical Systems Ambassador GSO Client Solutions Phases of Grid Computing Cluster Grids Single user community Single organization Campus Grids Multiple user communities

More information

Classic Grid Architecture

Classic Grid Architecture Peer-to to-peer Grids Classic Grid Architecture Resources Database Database Netsolve Collaboration Composition Content Access Computing Security Middle Tier Brokers Service Providers Middle Tier becomes

More information

HPC Wales Skills Academy Course Catalogue 2015

HPC Wales Skills Academy Course Catalogue 2015 HPC Wales Skills Academy Course Catalogue 2015 Overview The HPC Wales Skills Academy provides a variety of courses and workshops aimed at building skills in High Performance Computing (HPC). Our courses

More information

GT 6.0 GRAM5 Key Concepts

GT 6.0 GRAM5 Key Concepts GT 6.0 GRAM5 Key Concepts GT 6.0 GRAM5 Key Concepts Overview The Globus Toolkit provides GRAM5: a service to submit, monitor, and cancel jobs on Grid computing resources. In GRAM, a job consists of a computation

More information

G-Monitor: Gridbus web portal for monitoring and steering application execution on global grids

G-Monitor: Gridbus web portal for monitoring and steering application execution on global grids G-Monitor: Gridbus web portal for monitoring and steering application execution on global grids Martin Placek and Rajkumar Buyya Grid Computing and Distributed Systems (GRIDS) Lab Department of Computer

More information

Virtual machine interface. Operating system. Physical machine interface

Virtual machine interface. Operating system. Physical machine interface Software Concepts User applications Operating system Hardware Virtual machine interface Physical machine interface Operating system: Interface between users and hardware Implements a virtual machine that

More information

Collaborative & Integrated Network & Systems Management: Management Using Grid Technologies

Collaborative & Integrated Network & Systems Management: Management Using Grid Technologies 2011 International Conference on Computer Communication and Management Proc.of CSIT vol.5 (2011) (2011) IACSIT Press, Singapore Collaborative & Integrated Network & Systems Management: Management Using

More information

SLA BASED SERVICE BROKERING IN INTERCLOUD ENVIRONMENTS

SLA BASED SERVICE BROKERING IN INTERCLOUD ENVIRONMENTS SLA BASED SERVICE BROKERING IN INTERCLOUD ENVIRONMENTS Foued Jrad, Jie Tao and Achim Streit Steinbuch Centre for Computing, Karlsruhe Institute of Technology, Karlsruhe, Germany {foued.jrad, jie.tao, achim.streit}@kit.edu

More information

CMS Tier-3 cluster at NISER. Dr. Tania Moulik

CMS Tier-3 cluster at NISER. Dr. Tania Moulik CMS Tier-3 cluster at NISER Dr. Tania Moulik What and why? Grid computing is a term referring to the combination of computer resources from multiple administrative domains to reach common goal. Grids tend

More information

Agenda. HPC Software Stack. HPC Post-Processing Visualization. Case Study National Scientific Center. European HPC Benchmark Center Montpellier PSSC

Agenda. HPC Software Stack. HPC Post-Processing Visualization. Case Study National Scientific Center. European HPC Benchmark Center Montpellier PSSC HPC Architecture End to End Alexandre Chauvin Agenda HPC Software Stack Visualization National Scientific Center 2 Agenda HPC Software Stack Alexandre Chauvin Typical HPC Software Stack Externes LAN Typical

More information

High Performance Computing. Course Notes 2007-2008. HPC Fundamentals

High Performance Computing. Course Notes 2007-2008. HPC Fundamentals High Performance Computing Course Notes 2007-2008 2008 HPC Fundamentals Introduction What is High Performance Computing (HPC)? Difficult to define - it s a moving target. Later 1980s, a supercomputer performs

More information

Distribution transparency. Degree of transparency. Openness of distributed systems

Distribution transparency. Degree of transparency. Openness of distributed systems Distributed Systems Principles and Paradigms Maarten van Steen VU Amsterdam, Dept. Computer Science steen@cs.vu.nl Chapter 01: Version: August 27, 2012 1 / 28 Distributed System: Definition A distributed

More information

GridSolve: : A Seamless Bridge Between the Standard Programming Interfaces and Remote Resources

GridSolve: : A Seamless Bridge Between the Standard Programming Interfaces and Remote Resources GridSolve: : A Seamless Bridge Between the Standard Programming Interfaces and Remote Resources Jack Dongarra University of Tennessee and Oak Ridge National Laboratory 2/25/2006 1 Overview Grid/NetSolve

More information

A High Performance Computing Scheduling and Resource Management Primer

A High Performance Computing Scheduling and Resource Management Primer LLNL-TR-652476 A High Performance Computing Scheduling and Resource Management Primer D. H. Ahn, J. E. Garlick, M. A. Grondona, D. A. Lipari, R. R. Springmeyer March 31, 2014 Disclaimer This document was

More information

A Service for Data-Intensive Computations on Virtual Clusters

A Service for Data-Intensive Computations on Virtual Clusters A Service for Data-Intensive Computations on Virtual Clusters Executing Preservation Strategies at Scale Rainer Schmidt, Christian Sadilek, and Ross King rainer.schmidt@arcs.ac.at Planets Project Permanent

More information

Living in a mixed world -Interoperability in Windows HPC Server 2008. Steven Newhouse stevenn@microsoft.com

Living in a mixed world -Interoperability in Windows HPC Server 2008. Steven Newhouse stevenn@microsoft.com Living in a mixed world -Interoperability in Windows HPC Server 2008 Steven Newhouse stevenn@microsoft.com Overview Scenarios: Mixed Environments Authentication & Authorization File Systems Application

More information

Grid Computing & the Open Grid Services Architecture. Ian Foster Argonne National Laboratory University of Chicago Globus Project

Grid Computing & the Open Grid Services Architecture. Ian Foster Argonne National Laboratory University of Chicago Globus Project Grid Computing & the Open Grid Services Architecture Ian Foster Argonne National Laboratory University of Chicago Globus Project Open Group Grid Conference, Boston, July 21, 2003 2 Is the Grid a) A collaboration

More information

NorduGrid ARC Tutorial

NorduGrid ARC Tutorial NorduGrid ARC Tutorial / Arto Teräs and Olli Tourunen 2006-03-23 Slide 1(34) NorduGrid ARC Tutorial Arto Teräs and Olli Tourunen CSC, Espoo, Finland March 23

More information

Managing Complexity in Distributed Data Life Cycles Enhancing Scientific Discovery

Managing Complexity in Distributed Data Life Cycles Enhancing Scientific Discovery Center for Information Services and High Performance Computing (ZIH) Managing Complexity in Distributed Data Life Cycles Enhancing Scientific Discovery Richard Grunzke*, Jens Krüger, Sandra Gesing, Sonja

More information

MEng, BSc Applied Computer Science

MEng, BSc Applied Computer Science School of Computing FACULTY OF ENGINEERING MEng, BSc Applied Computer Science Year 1 COMP1212 Computer Processor Effective programming depends on understanding not only how to give a machine instructions

More information

Scientific and Technical Applications as a Service in the Cloud

Scientific and Technical Applications as a Service in the Cloud Scientific and Technical Applications as a Service in the Cloud University of Bern, 28.11.2011 adapted version Wibke Sudholt CloudBroker GmbH Technoparkstrasse 1, CH-8005 Zurich, Switzerland Phone: +41

More information

SCI-BUS gateways for grid and cloud infrastructures

SCI-BUS gateways for grid and cloud infrastructures SCI-BUS gateways for grid and cloud infrastructures Tamas Kiss University of Westminster Peter Kacsuk, Zoltan Farkas MTA SZTAKI VERCE project meeting 1 st February 2013, Edinburgh SCI-BUS is supported

More information

HADOOP, a newly emerged Java-based software framework, Hadoop Distributed File System for the Grid

HADOOP, a newly emerged Java-based software framework, Hadoop Distributed File System for the Grid Hadoop Distributed File System for the Grid Garhan Attebury, Andrew Baranovski, Ken Bloom, Brian Bockelman, Dorian Kcira, James Letts, Tanya Levshina, Carl Lundestedt, Terrence Martin, Will Maier, Haifeng

More information

HPC-related R&D in 863 Program

HPC-related R&D in 863 Program HPC-related R&D in 863 Program Depei Qian Sino-German Joint Software Institute (JSI) Beihang University Aug. 27, 2010 Outline The 863 key project on HPC and Grid Status and Next 5 years 863 efforts on

More information

Using the Grid for the interactive workflow management in biomedicine. Andrea Schenone BIOLAB DIST University of Genova

Using the Grid for the interactive workflow management in biomedicine. Andrea Schenone BIOLAB DIST University of Genova Using the Grid for the interactive workflow management in biomedicine Andrea Schenone BIOLAB DIST University of Genova overview background requirements solution case study results background A multilevel

More information

Introduction. MCSN N. Tonellotto Complements of Distributed Enabling Platforms

Introduction. MCSN N. Tonellotto Complements of Distributed Enabling Platforms Introduction 1 Distributed relating to a computer network in which at least some of the processing is done by the individual computers and information is shared by and often stored at the computers Enabling

More information

Introduction to Grid Computing

Introduction to Grid Computing Introduction to Grid Computing What is the GRID? Grid Computing Technology Grid computing enables the virtualization of distributed computing and data resources such as processing, network bandwidth and

More information

Distributed Operating Systems Introduction

Distributed Operating Systems Introduction Distributed Operating Systems Introduction Ewa Niewiadomska-Szynkiewicz and Adam Kozakiewicz ens@ia.pw.edu.pl, akozakie@ia.pw.edu.pl Institute of Control and Computation Engineering Warsaw University of

More information

High Performance Applications over the Cloud: Gains and Losses

High Performance Applications over the Cloud: Gains and Losses High Performance Applications over the Cloud: Gains and Losses Dr. Leila Ismail Faculty of Information Technology United Arab Emirates University leila@uaeu.ac.ae http://citweb.uaeu.ac.ae/citweb/profile/leila

More information

Part I Courses Syllabus

Part I Courses Syllabus Part I Courses Syllabus This document provides detailed information about the basic courses of the MHPC first part activities. The list of courses is the following 1.1 Scientific Programming Environment

More information

Simplifying Administration and Management Processes in the Polish National Cluster

Simplifying Administration and Management Processes in the Polish National Cluster Simplifying Administration and Management Processes in the Polish National Cluster Mirosław Kupczyk, Norbert Meyer, Paweł Wolniewicz e-mail: {miron, meyer, pawelw}@man.poznan.pl Poznań Supercomputing and

More information

The GridWay Meta-Scheduler

The GridWay Meta-Scheduler The GridWay Meta-Scheduler Committers Ignacio M. Llorente Ruben S. Montero Eduardo Huedo Contributors Tino Vazquez Jose Luis Vazquez Javier Fontan Jose Herrera 1 Goals of the Project Goals of the Project

More information

KNOWLEDGE GRID An Architecture for Distributed Knowledge Discovery

KNOWLEDGE GRID An Architecture for Distributed Knowledge Discovery KNOWLEDGE GRID An Architecture for Distributed Knowledge Discovery Mario Cannataro 1 and Domenico Talia 2 1 ICAR-CNR 2 DEIS Via P. Bucci, Cubo 41-C University of Calabria 87036 Rende (CS) Via P. Bucci,

More information

Supercomputing on Windows. Microsoft (Thailand) Limited

Supercomputing on Windows. Microsoft (Thailand) Limited Supercomputing on Windows Microsoft (Thailand) Limited W hat D efines S upercom puting A lso called High Performance Computing (HPC) Technical Computing Cutting edge problems in science, engineering and

More information

Elastic Cloud Computing in the Open Cirrus Testbed implemented via Eucalyptus

Elastic Cloud Computing in the Open Cirrus Testbed implemented via Eucalyptus Elastic Cloud Computing in the Open Cirrus Testbed implemented via Eucalyptus International Symposium on Grid Computing 2009 (Taipei) Christian Baun The cooperation of and Universität Karlsruhe (TH) Agenda

More information

IBM Solutions Grid for Business Partners Helping IBM Business Partners to Grid-enable applications for the next phase of e-business on demand

IBM Solutions Grid for Business Partners Helping IBM Business Partners to Grid-enable applications for the next phase of e-business on demand PartnerWorld Developers IBM Solutions Grid for Business Partners Helping IBM Business Partners to Grid-enable applications for the next phase of e-business on demand 2 Introducing the IBM Solutions Grid

More information

Cisco Active Network Abstraction Gateway High Availability Solution

Cisco Active Network Abstraction Gateway High Availability Solution . Cisco Active Network Abstraction Gateway High Availability Solution White Paper This white paper describes the Cisco Active Network Abstraction (ANA) Gateway High Availability solution developed and

More information

CS550. Distributed Operating Systems (Advanced Operating Systems) Instructor: Xian-He Sun

CS550. Distributed Operating Systems (Advanced Operating Systems) Instructor: Xian-He Sun CS550 Distributed Operating Systems (Advanced Operating Systems) Instructor: Xian-He Sun Email: sun@iit.edu, Phone: (312) 567-5260 Office hours: 2:10pm-3:10pm Tuesday, 3:30pm-4:30pm Thursday at SB229C,

More information

David Rioja Redondo Telecommunication Engineer Englobe Technologies and Systems

David Rioja Redondo Telecommunication Engineer Englobe Technologies and Systems David Rioja Redondo Telecommunication Engineer Englobe Technologies and Systems About me David Rioja Redondo Telecommunication Engineer - Universidad de Alcalá >2 years building and managing clusters UPM

More information

HPC Software Requirements to Support an HPC Cluster Supercomputer

HPC Software Requirements to Support an HPC Cluster Supercomputer HPC Software Requirements to Support an HPC Cluster Supercomputer Susan Kraus, Cray Cluster Solutions Software Product Manager Maria McLaughlin, Cray Cluster Solutions Product Marketing Cray Inc. WP-CCS-Software01-0417

More information

Distributed Systems. REK s adaptation of Prof. Claypool s adaptation of Tanenbaum s Distributed Systems Chapter 1

Distributed Systems. REK s adaptation of Prof. Claypool s adaptation of Tanenbaum s Distributed Systems Chapter 1 Distributed Systems REK s adaptation of Prof. Claypool s adaptation of Tanenbaum s Distributed Systems Chapter 1 1 The Rise of Distributed Systems! Computer hardware prices are falling and power increasing.!

More information

Computational Grids: Current Trends in Performance-oriented Distributed Computing

Computational Grids: Current Trends in Performance-oriented Distributed Computing Computational Grids: Current Trends in Performance-oriented Distributed Computing Rich Wolski Computer Science Department University of California, Santa Barbara Introduction While the rapid evolution

More information

A Survey Study on Monitoring Service for Grid

A Survey Study on Monitoring Service for Grid A Survey Study on Monitoring Service for Grid Erkang You erkyou@indiana.edu ABSTRACT Grid is a distributed system that integrates heterogeneous systems into a single transparent computer, aiming to provide

More information

CLOUD COMPUTING. When It's smarter to rent than to buy

CLOUD COMPUTING. When It's smarter to rent than to buy CLOUD COMPUTING When It's smarter to rent than to buy Is it new concept? Nothing new In 1990 s, WWW itself Grid Technologies- Scientific applications Online banking websites More convenience Not to visit

More information

16th International Conference on Control Systems and Computer Science (CSCS16 07)

16th International Conference on Control Systems and Computer Science (CSCS16 07) 16th International Conference on Control Systems and Computer Science (CSCS16 07) TOWARDS AN IO INTENSIVE GRID APPLICATION INSTRUMENTATION IN MEDIOGRID Dacian Tudor 1, Florin Pop 2, Valentin Cristea 2,

More information

DATA CENTRE TECHNOLOGIES & SERVICES

DATA CENTRE TECHNOLOGIES & SERVICES DATA CENTRE TECHNOLOGIES & SERVICES RE-Solution Data Ltd Reach Recruit Resolve Refine 170 Greenford Road Harrow Middlesex HA1 3QX T +44 (0) 8450 031323 EXECUTIVE SUMMARY The purpose of a data centre is

More information