Introduction to Grid Computing

Transcription

1 Introduction to Grid Computing

2 What is the GRID?

3 Grid Computing Technology Grid computing enables the virtualization of distributed computing and data resources such as processing, network bandwidth and storage capacity to create a single system image, granting users and applications seamless access to vast IT capabilities. Just as an Internet user views a unified instance of content via the Web, a grid user essentially sees a single, large virtual computer.

4 The Application-Infrastructure Gap Dynamic and/or Distributed Applications Shared Distributed Infrastructure A B

5

6 Global Community

7 What Grid can really do? Help you build a secure IT infrastructure Remote job submission so you can distribute your workload Middleware and Programming API for developing distributed services and application that use them. Secure remote access to database

8 Grid Computing s Potential Attributes Single sign-on, security Policybased resource sharing ECU UNC-CH ASU UNC-W ECSU Duke NCCU NCSU Virtual Computers UNC-G UNC-P WSSU UNC-C Virtual Databases WFU UNC-A FSU NCAT NCArts WCU Unified view of data and computers Computers and data appear to be local Efficient access to large data sets Caching Replication

9 Type of Grid Computational Grid Present a virtual pool of computing resources Data Grid Present a virtual pool of storages Access Grid Virtual Collaboration Space

10 Technology and Trends

11 Brief History of Grid Computing Generation 0: No Grid only Internet and Web Generation 1: Internet Computing Seti Home, Peer-to-peer Many experimental Grid and Meta computing System Generation 2: Execution oriented Grid Use grid as batch supercomputing system Basic set of services, no standardize access Workable security based on X.509 Generation 3: Services Oriented Grid Distributed Services accessible globally Based on Web Services Standard

12 What is really the Grid? Set of Computers, Servers, Clusters connected together with good network OS Linux / Windows on each machine (mostly linux) Install Grid Middleware, mostly Globus Setup so middleware properly including certificate infrastructure Use tool on top Scheduler to distribute computing workload Gridified Application Use globus library or other lobrary such as MPI Grid services Workflow tool (new)

13 Components of the Grid Grid Enable Application Grid Middleware High Speed Network

14 Grid Middleware Key technology in Grid Computing Provide uniform, high-level access to a wide range of resources (including networks) Address interdomain issues of security, policy, etc. Permit application-level management and monitoring of end-to-end performance Middleware-level and higher-level APIs library and tools for programmer to implement applications

15 The Globus Project Project by Argonne National Laboratory University of Southern California's Information Sciences Institute University of Chicago University of Edinburgh Swedish Center for Parallel Computers Leader Ian Foster and Carl Kesselman Produces open-source software that is a de facto grid standard Globus is now being take care of by Globus alliance

16 Grid Architecture Fabric Layer Protocol and interface that provide access to computing resources such as CPU, storage Connectivity Layer Protocol for Grid-specific network transaction such as security GSI Resources Layer Protocol to access a single resources from application GRAM (Grid Resource Allocation Management) GridFTP ( data access) Grid Resource Information Service Collective layer Protocol that manage and access group of resources Application Layer Collective Layer Resources Connectivity Fabric

17 Grid Security Infrastructure Core part of grid system Extension of SSL(Secure Socket Layer)/TLS (Transport Layer Security) X.509 Certificate Provides Single sign-on capability Secure access to resources

18 How GSI works User Proxy Single Signon using Grid Certificate and ID Gate Keeper Proxy Credential GSI-Enable GRAM Server Remote process creation request Gate Keeper GSI-Enable GRAM Server Secure Communication Local id Local id User obtain signed certificate from CA MAP Process MAP Process

19 GRAM Broker Application RSL RSL MDS GRAM LSF Co-allocator GRAM SGE RSL GRAM PBS

20 Bridging the Gap: Service-Oriented Infrastructure Service-oriented applications Wrap applications as services Compose applications into workflows Service-oriented infrastructure Provision physical resources to support application workloads Appln Service Users Composition Workflows Invocation Appln Service Provisioning

21 Open Grid Service Architecture (OGSA) Reference architecture for the next generation grid technology Merge Grid and Web Services technologies together open standards for grid computing Sponsored by the GGF (organization modeled after IETF) Primary working groups: OGSA and OGSI Many vendors involved: IBM, Sun, Oracle, AVAKI, UD, etc (But, ANL and IBM seem to have the upper hand) Working with the W3C to extend web services

22

23 IBM IB M IBM IB M User Application Tool Uniform interfaces, security mechanisms, Web service transport, monitoring GRAM Globus as Service-Oriented Infrastructure User Application MDS- Index User Svc Host Env Reliable File Transfer GridFTP User Application Tool User Svc Host Env MyProxy DAIS Computers Specialized resource Storage Database

24 Applying the Grid

25 How to Develop Grid Application Use grid as parallel super computer Using Grid-enable-MPI (MPICH-G2) to do parallel programming Mostly apply to technical computing Use Grid as a powerful server system Using Portal + Grid Scheduler + to Distributed Jobs Load balance in scheduler Use grid to develop SOA application Grid Service and Workflow

26 Traditional Grid Enable Application Task User Station Task Resource Broker Grid and cluster Middleware (MPI, Globus) Monitor Compute Resource Compute Resource Task Compute Resource Compute Resource Task Compute Resource Task Internet

27 Service Oriented Grid Registration Service Application Workflow Service Grid Middleware Service OS Server Network OS Server

28 Potential Application E-commerce Infrastructure for B2B communication E-government Secure information exchange for government operation. Smart card, e-citizen E-Military Building military based secure infrastructure

29 Potential Application E-health care Secure information exchange for health care industry Thailand Population: 63 Million People Public Hospital: 200 Hospitals Private Hospital: 337 Hospitals 30 Baths Project: 1,480 Baths per capitar 27,000 Doctors country wide Massive storage and processing are needed for medical data Patient record Diagnostic record Instrument: MRI, X-RAY

30 Potential Application E-entertainment Video Streaming, secure exchange of digital media Distributed gaming server, secure subscription of gaming services E-learning Access Grid On-demand remote learning

31 What industries are using grid computing now? Automotive and aerospace for collaborative design and data-intensive testing Financial services for running long, complex scenarios and arriving at more accurate decisions Life sciences for analyzing and decoding strings of biological and chemical information Government for enabling seamless collaboration and agility in both civil and military departments and agencies Higher education for enabling advanced, data and compute intensive research.

32 An ebusiness Use of Globus: SAP GlobusWorld 3 Globus-enabled applns: CRM: Internet Pricing Configurator (IPC) CRM: Workforce Management (WFM) SCM: Advanced Planner & Optimizer (APO) Applications modified to: Adjust to varying demand & resources Use Globus to discover & provision resources Web Browsers / Batch Processes (typically several thousand requests) Request: Price Query 1 IPC Dispatcher IPC Server Delegation of Request IPC Response: PricelistServer Depending on: -Time - Discount - Number of Items - 3 SAP AG R/3 Internet Pricing & Configurator (IPC) 2 2

33 ThaiGrid Drug Discovery Project Partners: KU/IBM Problem: From over 1000 active compounds available from Thai medicinal plants database, find a smallest set of compounds that has a potential to be used as a drug Very compute intensive. Several month of computing time Solution: Use grid to increase computing power to times

34 Concept Task User Station Task Resource Broker Grid and cluster Middleware (MPI, Globus) Monitor Compute Resource Compute Resource Task Compute Resource Compute Resource Task Compute Resource Task Internet

35 File Server Molecular DB OGSA DAI Grid Ftp Gamess Scheduler MMJFS Gamess Service Portal Portlet Broker Server Registration Server Gamess Backend DB

36 Grid based data assimilation using GA and Remote Sensing Partners: KU/AIT Problem: Very long computation time from months to years Large amount of RS data needed to be moved around and process Solution: Using Grid to harvest more computing power Hide data assimilation process behind the service

37 Massive Rendering of Graphics Images Partners: KU/Imagimax Problem: Rendering massive amount of images for animation require a massive amount of computing power Solution: Grid based Rendering Support Middleware based on NPACI ROCKS Use grid to link multiple clusters together Portal to submit rendering job 8/22/2006 TAM

38 Nimrod-G, Australia Earth Sciences Astrophysics Antenna Design Air Pollution Computational Chemistry Circuit Design Monte Carlo CFD Cardiac Modelling Airfoil Design Climate Public Health Policy

39 A Typical escience Use of Globus: Network for Earthquake Eng. Simulation Links instruments, data, computers, people

40 Grids for Physics: LHC Computing Grid

41 Grid2003 Open Science Grid 30 sites ( CPUs) & growing concurrent jobs 8 substantial applications + CS experiments Running since October 2003 Korea

42 系 统 架 构 :DartGrid 的 分 层 模 型 VO 的 地 址 栏 语 义 浏 览 面 板 本 体 论 树 Dar t Grid 虚 拟 组 织 资 源 浏 览 面 板 Q3 语 义 查 询 显 示 面 板 语 义 注 册 面 板 We ve used GT3 to build the largest database grid system for Traditional Chinese Medicine, integrating about 50 TCM-relevant databases. (Zhejiang U.)

43 Global Grid Forum ( The Global Grid Forum (GGF) is a community-initiated forum researchers and practitioners working on "grid" technologies. GGF's primary objective is to promote and support the development, deployment, and implementation of Grid technologies and applications creation and documentation of "best practices" - technical specifications, user experiences, and implementation guidelines GGF has many working group working on various grid standard

44 Conclusion Grid Computing is a high impact technology Listed by MIT as one of the 10 Most important technology that will change the world Grid can help making the use of IT infrastructure more efficient by Provides uniform access, security Virtualized the resources Grid technology is new still quickly changing Can be used now for certain application Will be used widely for most application in the future

45 More information Globus Alliance Grid Computing Planet Vendor sites IBM, SUN, Oracle, Intel.

46 Thank You