Zum aktuellen Stand der GRID Forschung

Transcription

1 Zum aktuellen Stand der GRID Forschung Regionales Rechenzentrum Erlangen 13.Januar 2004 Michael M. Resch Höchstleistungsrechenzentrum Stuttgart (HLRS), Michael. M. Resch 1 Höchstleistungsrechenzentrum Stuttgart

2 Themen Ausgangslage Stand Ende 2003 Probleme/Lösungen Michael. M. Resch 2 High Performance Computing Center Stuttgart

3 Ausgangslage Middleware: The Heart of GRID Computing FZ Karlsruhe, October 30th 2002 Michael. M. Resch 3 Höchstleistungsrechenzentrum Stuttgart

4 Definitions of the GRID A computational grid is a hardware and software infrastructure that provides dependable, consistent, pervasive, and inexpensive access to high-end computational capabilities. Ian Foster & Carl Kesselman, The Grid, MK, 1998 Grid Problem, which we define as flexible, secure, coordinated resource sharing among dynamic collections of individuals, institutions, and resources. Ian Foster et al, The Anatomy of the Grid, 2001 We view a Grid as an extensible set of Grid services that may be aggregated in various ways to meet the needs of VOs, which themselves can be defined in part by the services that they operate and share. Ian Foster et al, The Physiology of the GRID, 2002 A GRID is a system which: coordinates resources that are not subject to centralized control using standard, open, general-purpose protocols and interfaces to deliver nontrivial qualities of service Ian Foster, What is the GRID? A three point checklist Michael. M. Resch 4 High Performance Computing Center Stuttgart

5 Grids: An Emerging, Common Computing and Data Infrastructure for Science and Engineering Portals Web Portal Access to Application and Grid Services Specialized Portal Access (high performance displays, PDAs, etc.)... Services Data Management: replication and metadata Resource Brokering Fault Management Workflow Management Accounting Applications Services Building Blocks Encapsulation as Web Services Encapsulation for Script Based Services Encapsulation as Java Based Services Basic Grid Functions Resource Discovery national supercomputer facilities Scheduling and Access to Computing Internet optical networks space-based networks clusters Grid Communication Functions transport services security services Communications Distributed Resources Condor pools of workstations Uniform Data Access tertiary storage Monitoring and Events... scientific instruments Operational Support

6 Management of Resources Management of Resources Definition of resource types; High level services; Performance prediction; Models and monitoring Security Access management; AAA (Authentication,Authorisation, Accounting); Denial of services, spoofing, intrusion Privacy, Group security, Dynamic virtual private grids; High speed encryption Application Development Programming models (Distributed objects, Component models); Communication models (message-passing,distributed shared memory) End-user environments; Portal (GUI s for novice users); Problem solving environments; Application frameworks and toolkits Distributed and federated databases Michael. M. Resch 6 High Performance Computing Center Stuttgart

7 Outlook The GRID will allow to completely change the handling of users, data and jobs. This will be driven by the development of middleware in fields other than science/simulation/hpc (Web Services, SOAP, XML,.). The GRID is mainly a chance to tackle old problems with new funding. Michael. M. Resch 7 High Performance Computing Center Stuttgart

8 Situation Ende 2003 European GRID Perspectives 5 th Pragma Workshop Hsinchu, Taiwan, 23/10/2003 Michael. M. Resch 8 Höchstleistungsrechenzentrum Stuttgart

9 10 Gbit/s 2,5Gbit/s <2,5Gbit/s The GÉANT Network today Pan-European coverage (32+4 Countries/NRENs) Gigabit connectivity at 10 Gb/s Linking more than 3100 Universities US Canada Total 200 MEuro over 4 years (80 MEuro from EU) Extend to Mediterranean, Asia Pacific Rim, Latin America... NeDAP SILK Chin Japa Kore Brazil

10 Application communities addressed orporate business anufacturing industry Train, Automotive, erospace, Shipbuilding) ower generation Chemical, pharmaceutical Health-care, Clinical Biology, Bio-informatics Education, e-learning High Energy Physics Earth Observation, Environment

11 Clustering of efforts Public health informatics Medical Informatics Medical Imaging Public Health Patient HEALTHGRID CLUSTER Tissue, organ Cell GEMSS MAMMOGRID Only health related Grids Multidisciplinary Grids Bioinformatics Molecule BIOGRID GRIDSTART DATAGRID EUROGRID DAMIEN ETC. CROSSGRID

12 Grid Research in FP6 ( ) F3 Research Infrastructure Deployment of specific high performance Grids Deployment of highcapacity and high-speed communications network -GÉANT 200 M (RI) Deployment F2 Grids for Complex Problem Solving Architecture, design and development of the next generation Grid Enabling application technologies 125 M (IST) Research & Development

13 Enabling Grids for E-science in Europe (EGEE)

14 Distributed European Infrastructure for Supercomputing Applications (DEISA) CINECA (IBM) CSC (IBM) ECMWF (IBM) epcc (IBM) FZJ (IBM) IDRIS (IBM) RZGIPP (IBM) Sara (sgi)

15 HPC-Europa CINECA EPCC CEPBA HLRS IDRIS SARA PSNC PARALLAB CASPUR TCD NTUA

16 Grid technology development Strategic objective: Grids for Complex Problem Solving Indicative budget in Enabling FP6: Application 125m Architecture, design and development of the next generation GRID beyond extensions of existing technologies based on a meaningoriented information model leading to open standards including security built-in at all levels, programming environments, customizable middleware, resource management economic and business models for new services interoperability with existing GRID and Web services technologies for solution of complex problems requiring a grid-based approach tools & environments for Modelling, simulation, visualisation, data mining, process control, remote operation, Collaborative work in dynamic virtual organisations exploiting synergies across different application sectors A multi-disciplinary approach across the relevant levels of the value chain is required o-operation operation with research activities in Member States necessary for r building critical mass

17 Situation Ende 2003 European Grid Perspectives (National Activities) 5 th Pragma Workshop Hsinchu, Taiwan, 23/10/2003 Slide 17/26 Höchstleistungsrechenzentrum Stuttgart

18 UK e-science GRID 120M+ 115M Programme over 3+3 years Of the 120M 20M matching funds from industry 75M is for Grid Applications in all areas of science and engineering 10M for Supercomputer upgrade Glasgow DL Edinburgh Newcastle Belfast Manchester Slide 18/26 Höchstleistungsrechenzentrum Stuttgart Oxford Cambridge Cardiff RAL Hinxton London Southampton

19 National Program Coordination MILANO TORINO PAVIA PADOVA GENOVA BOLOGNA PISA PERUGIA ROMA CNR & University HPC, Parallel Programming, Grid computing, Scientific libraries, Data base and knowledge discover Earth Observation, Computational chemistry, Image processing, INFN & University Grid (INFN-Grid, DataGrid, DataTag), e-science applications: Astrophysics, Bioinformatics, Geophysics, CAGLIARI NAPOLI GRID.IT PALERMO BARI 3-Year MATERA Project LECCE Cost 11.1 M COSENZA Funding: 8.1 M CNIT ASI Applications of Earth Observation Technologies and HW-SW infrastructures for high-performance communication, Optical technologies, Domenico Laforenza 19/26

20 UNICORE+ UNIform access to COmputing Resources ( ) Goals: Seamless, secure and intuitive access to HPC resources Consistent batch access to different remote systems Support existing and emerging technologies Minimal intrusion into the centers Support multi-system and multi-site applications Partners: German HPC-Centers Pallas Funding: 5.6 Mill Slide 20/26 Höchstleistungsrechenzentrum Stuttgart

21 d-grid Slide 21/26 Höchstleistungsrechenzentrum Stuttgart

22 Standardisierung Standardization by the users for the users Open GRID Services Architecture (OGSA) Savas Parastatidis, Jim Webber, Paul Watson, Thomas Rischbeck A Grid Application Framework based on Web Services Specifications and Practices, September School of Computing Science, University of Newcastle upon Tyne, Newcastle upon Tyne, NE1 7RU, United Kingdom Arjuna Technologies Ltd., Nanotechnology Centre, Herschel Building, Newcastle upon Tyne, NE1 7RU, United Kingdom The aim is to design an alternative mapping that captures the same Grid requirements, but fits more closely with existing Web Services specifications and practices. Michael. M. Resch 22 High Performance Computing Center Stuttgart

23 Management of Resources Management of Resources Definition of resource types; High level services; Performance prediction; Models and monitoring Security Access management; AAA (Authentication,Authorisation, Accounting); Denial of services, spoofing, intrusion Privacy, Group security, Dynamic virtual private grids; High speed encryption Application Development Programming models (Distributed objects, Component models); Communication models (message-passing,distributed shared memory) End-user environments; Portal (GUI s for novice users); Problem solving environments; Application frameworks and toolkits Distributed and federated databases Michael. M. Resch 23 High Performance Computing Center Stuttgart

24 Outlook The GRID will allow to completely change the handling of users, data and jobs. This will be driven by the development of middleware in fields other than science/simulation/hpc (Web Services, SOAP, XML,.). The GRID is mainly a chance to tackle old problems with new funding. Michael. M. Resch 24 High Performance Computing Center Stuttgart