The Grid-it: the Italian Grid Production infrastructure

Transcription

1 n 1 Maria Cristina Vistoli INFN CNAF, Bologna Italy The Grid-it: the Italian Grid Production infrastructure

2 INFN-Grid goals!promote computational grid technologies research & development: Middleware and grid tools " Through European and national projects # DataGrid, DataTAG, Firb-GRID-it, EGEE, LCG, CoreGRID etc " Internal R&D activities!deploy operate - support the Grid-it Italian Grid Production Infrastructure: Grid as coordinated resource sharing on a large scale for a multi-institutional and dynamic virtual organization! Set up the Italian Grid Production Infrastructure open to the national research community " FIRB: Grid.it astrophysics, geophysics, biomedicine, computational chemistry etc " Research community and industry n 2

3 n 3 INFN-Grid goals!provide operation and support of the EGEE/LCG production infrastructure, national Grids interconnection!promote dissemination activity to gridify scientific applications " GILDA testbed " Genius portal

4 n 4 INFN-GRID participation to EGEE! EGEE SA1 infrastructure operation and support " ROC Regional Operation Center " CIC - Core Infrastructure Center!EGEE JRA1 IT-CZ cluster " Workload management system " Resource access - CE accounting - policy " VOMS! EGEE NA4/NA3/NA2/NA5 " HEP application " Generic application " Dissemination

5 INFN-GRID participation to GRID.IT/FIRB n 5!Set up the national production grid Infrastructure open to the national research community! Grid management and support tools - system " First tools in production " R&D on # Resource Utilization Policies # Data Management # Scientific Data Base grid integration

6 n 6 Italian Grid : Site map TRENTO MILANO UDINE TORINO PADOVA LNL FERRARA CNAF BOLOGNA PISA FIRENZE S.Piero PERUGIA ROMA ROMA2 LNF TRIESTE National Grid! 26 resource centers (number is increasing rapidly) of different size in term of computing and storage! About 2000 CPUs of computing power! 15 Virtual Organizations SASSARI NAPOLI BARI LECCE CAGLIARI COSENZA PALERMO CATANIA LNS

7 Italian Production Grid Computing and Storage Resources n 7

8 Grid-it:Italian Grid Production Services! Resource Brokers: " EGEE/LCG infrastructure open to all EGEE VO (LHC exps, Biomed, MAgic, Planck, Compchem, ESR) # egee-rb-01.cnaf.infn.it ; # grid008g.cnaf.infn.it (DAG enabled) " ATLAS VO egee-rb-02.cnaf.infn.it, egee-rb-05.cnaf.infn.it " CMS VO egee-rb-04.cnaf.infn.it, egee-rb-06.cnaf.infn.it! Replica Location Service for babar, virgo, cdf, planck and other Italian VOs " datatag2.cnaf.infn.it, " To be replaced by LFC: lfcserver.cnaf.infn.it! Voms server: " testbed008.cnaf.infn.it " VOs: infngrid, zeus, cdf, planck, compchem! LDAP SERVER FOR National Vos (bio, inaf, ingv, gridit,theophys, virgo) : " grid-vo.cnaf.infn.it n 8

9 Grid-it:Italian Grid Production Services n 9! MyProxy servers: " testbed013.cnaf.infn.it! User Interfaces: " UIs are not Core servives..., anyway you can find a list of italian UIs # Monitoring:GridICE server: " EGEE/LCG Production infrastructure # " Italian Production Infrastructure # " Atlas # " CMS #

10 n 10 INFN-GRID Middleware release! Based on LCG! Several VOs added (about 15)! Authorization via VOMS + LCAS/LCMAPS! MPI configuration! Cert queue for site functional tests! Gridice includes WNs monitor! AFS support in the WN! DAG jobs support! DGAS accounting system

11 n 11 Grid-it Production Grid!The access point for user and site mangers is the Grid-it Production Grid Website:! " Documentation (prerequisites, middleware installation, upgrade, testing, using the Grid, current applications and VOs, etc.) " Software repository " Monitoring of resources and services (GridICE) " Tools for the CMT (Central Management Team), site managers and supporters (Calendar/downtime manager) " Trouble ticketing system for problems, advisories and suggestions " A knowledge base to complement the documentation

12 n 12 Grid-it: Ticketing system! INFN-GRID ticketing system is used: # from users to ask questions or to communicate troubles; # from system manager to communicate about common grid tasks (ex: upgrading to a new grid release) # from Central Management Team to system manager to notify a problem " Support Groups : " Support Grid Services (RB, RLS, VOMS, GridICE, etc) Group; " Support VOApplications Group (each for every VO:Atlas, CMS, Alice, LHC, Babar, CDF, Virgo,.); " Support Site Group (each for every site) # Operative Groups Operative Central Management Team (CMT); " Operative Release & Deployment Team; Users -> Create a ticket Supporters/Operatives -> Open the ticket Users and/or Supporters/Operatives -> Update an open ticket Supporters/Operatives -> Close the ticket

13 n 13 Grid-it: Operations activities! Operation of the grid infrastructure: " Shifts (Monday to Friday 08:00-20:00): about 20 people from different sites monitor and test the status of the grid resources (STF, others), certify of the italian sites/resources and provide VO and user support " Management of the grid services (general purpose and VO specific) " Participation to CIC-on-Duty shifts! Support (operations, user and VO): " Regional ticketing system " Support groups are defined and operational for experiments and applications " Participation to Support-on-Duty shifts! Release and installation: " currently INFN-GRID release is based on LCG. INFN-GRID contains DAG, DGAS, VOMS, support for afs client and GridIce monitoring for the WNs; it is now in the deployment phase on the italian grid infrastructure " Installation and configuration scripts (YAIM) are provided! Grid Install working group $ Fabric and software management WG

14 n 14 Certification activity! The Central Management Team is responsible of the resource centers certification: checking the functionalities of a site before joining the site to the production grid.! Although all certification jobs are VO independent, the INFNGRID VO is used to perform these jobs;! In particular are checked: # MPI functionalities # GIIS' information consistence; # Local jobs submission (LRMS); # Grid submission with Globus (globus-job-run); # Grid submission with the ResorceBroker; # ReplicaManager functionalities;! In order to certificate a site the CMT uses dedicated grid services: " RB & BDII: gridit-cert-rb.cnaf.infn.it! In this way we avoid to have an uncertified site in the production grid services;

15 Towards glite: Pre-production, Middleware Certification! Pre-production Service " 3 sites in Italy (CNAF,Padova, Bari) ready " CNAF is the pilot site: # LCG release: 1 CE, 1 SE, 3 WN available # glite release: 1 CE, 3 WN, 1 I/O service, 1 VOMS server, 1 WMS node (formerly RB), 1 double-stack WN, 1 R-GMA server and 1 UI available soon " Migration strategies will be tested " Stress test middleware components in particular the WMS system " Open to the experiments applications to verify middleware! Certification Infrastructure " 4 sites (CNAF, Padova, Roma and Torino) " goal: certificate the INFN-GRID release partecipating to the certification of the glite release and adding new, specific middleware components to glite: storrm, G-PBOX " produce and verify the installation procedures n 15

16 n 16 Towards glite: migration! glite Migration " glite release 1 in the pre-production service to verify: # Functionality # Stability # Performance " pre-production open to experiments " INFN-GRID/LCG fall back solution " Production infrastructure migration to glite step by step # First: migration from RB to gllite WMS # Then CE / WNs: " Major sites deploy glite CEs in parallel with the LCG-2 CEs. " Some of the smaller sites convert fully to glite.

17 n 17 Current activities! Production service quality and stability improvements " Improve monitoring systems: Gridice with reactive alarms and notification, new release " Site isolation - need simple mechanism to remove sites " Improve the grid management and monitoring structure! INFN-GRID Support infrastructure: # VO support - Experiments # Site administartors # Grid services support! Accounting and policy system management

18 Resource centers status: Feb n 18

19 Resource centers status: Mar n 19

20 n 20 Graphs of resources utilization Default param. shown Resources usage in the last week per all site and all VO Site, virtual oraganization, time interval, on/off show values on bar, size picture

21 n 21 Jobs waiting and jobs running Default view: number of jobs in the last day per all site and all VO

22 n 22 Number of jobs CPU hours Used memory (average job size) Wall Time hours

23 Executed Jobs in the last three weeks per site n 23

24 Site specific data: CPU hours per VO n 24

25 n 25 Useful links! Grid-it Production Grid " Grid-it Monitoring/Management GridICE " Grid-it test and certification " Grid-it Support " Contact " " " Ticket for operational issue