Esqu Science Experiments For Computer Network

Transcription

1 Vangelis Floros, GRNET S.A. 3 rd Open Source Software Conference March 22, 2008 NTUA, Athens Greece

2 Introduction What is a Grid? What is escience? Large Scientific Grids The example of EGEE Building Grid infrastructures with OSS Platform OS Middleware Support tools Enabling escience on the Grid Conclusions

3 Many definitions depending on the context (Scientific Discipline, Company, Conference and Person talking) Grid computing is coordinated resource sharing and problem solving in dynamic, multi institutional virtual organizations (Ian Foster) A Virtual Organisation is: People from different institutions working to solve a common goal Sharing distributed processing and data resources A Grid is the concept of sharing computing resources (processing and storage) among distributed groups of people over the network with the purpose to increase the overall computing capabilities and to collaborate (me)

4 Data Manage ment Security Resource Managem ent Inform ation Grid Computing == Clustering Clusters; Building a global batch submission system...

5

6

7 EGEE (Enabling Grids for E ScienceE) Flagship grid infrastructure project co funded by the European Commission Now in 3 nd phase with 42 joint partners (>120 beneficiaries) in 48 countries egee.org/ Archeology Astronomy Astrophysics Civil Protection Comp. Chemistry Earth Sciences Finance Fusion Geophysics High Energy Physics Life Sciences Multimedia Material Sciences >250 sites 48 countries >68,000 CPUs >20 PetaBytes >10,000 users >200 VOs >150,000 jobs/day

8 OSS today helps the establishment and operation of such large scale Grids providing: Operating System Platform Grid Middleware Operational and Support tools

9 Linux is the Operating System of choice for building and operating production Grid infrastructures on 24/7 basis e.g. Scientific Linux used by EGEE Stability Security Adaptability

10 The Grid middleware is the cornerstone of a Grid infrastructure. Implements the security, information, data and job management services offered by today's grids. Globus The dominant grid middleware distribution. Synonymous to the grid (Ian Foster, The Globus Alliance) Unicore The European alternative to Globus

11 Modern grid infrastructures are an ensemble of numerous different components and services Information Services MDS, OpenLDAP, BDII, Glue Schema Scheduling and job management: Condor, Torque, LSF, Sun Grid Engine, GridWay. Data Management : GridFTP gsi enabled file transfer, based on ftp SRB, dcache Storage Resource Managers LFC, RLC Replica Catalogues Security, authorization, authentication, confidentiality OpenSSH and OpenSSL for X.509v3 based PKI, VOMS (VO Management), MyProxy, Hydra glite Developed in EGEE. Consolidates existing tools and provides the gluing for establishing and operating the EGEE infrastructure

12 User Interface Grid Info Monitoring (GridICE, GSTAT) Grid Resource Monitoring (SAM/GridView) VO server VO server VO server RB/WMS TOP BDII CE SE CE SE BDII GRIS BDII GRIS LFC GRIS GRAM Computing Element Storage Element GRIS GRAM Computing Element Storage Element WorkerNode WorkerNode WorkerNode WorkerNode WorkerNode... WorkerNode...

13 Daily operation of computing clusters and large grids require support tools for Service availability monitoring Uptime/downtime monitoring and statistics Security incidents monitoring and reporting Ganglia ( Cluster monitoring Nagios ( Network monitoring Tripwire ( Intrusion detection

14

15 Compilers GNU (gcc, gfortran) Python, Perl Java IDEs Eclipse Netbeans Version control CVS SVN HPC development MPI (MPICH, OpenMPI)

16 General purpose numerical computation packages. Octave ( Scilab ( GSL Gnu Scientific Library (

17 Computer Science Network Simulator 2 (NS 2) MEEP Biomedical BLAST Basic Local Alignment Search Tool Clustal Sequence alignment RNAHybrid PHYLIP PHYLogeny Inference Package Meteorology MM5 Weather forecasting WRF Weather forecasting Computational Chemistry NWChem molecular dynamics simulation GAMESS ab initio molecular quantum chemistry DL_POLY molecular dynamics simulation NAMD scalable molecular dynamics

18 Every non trivial escience experiment consists of multiple steps of processing orchestrated in the form of activity workflows. Workflow environments enable the definition, enactment and monitoring of such workflows Taverna ( MyGrid project. Focus on Biomed applications Moteur ( Focus on data intensive apps. Same file format with Taverna (Scufl). Also biomed oriented. Kepler ( project.org): Targets multiple disciplines (astronomy, ecology, biology) Triana (

19 Grids typical require a steep learning curve in order to be exploited. Simple interaction semantics and graphical user interfaces are required in order for scientists to explore grid infrastructure capabilities Many scientific communities develop application/experiment specific portals to facilitate grid exploitation by their members Both implement the JSR 168 portlet API standard GridSphere: OGCE Open Grid Computing Environments Portal and Gateway Toolkit: ogce.org

20 It s difficult to imagine scientific grids and escience without Open Source Software OSS is used in all levels and aspects of daily operation of grid infrastructures Wealth of scientific (general purpose and discipline specific) software, environments and tools Still there is much to be improved Closed source and licensed Scientific software Better support for standards Open Source Grid & Cluster Conference:

21 Thank you for your attention!