EUMEDGrid-Support Supporting EUMEDGRID-Support e-infrastructure sustainability Overview of monitoring tools

Transcription

1 EUMEDGrid-Support Supporting EUMEDGRID-Support e-infrastructure sustainability Overview of monitoring tools Fulvio Galeazzi GARR Amman, November 24th 2011 EUMEDGRID-Support ROC-training school

2 Contents Will be a hands-on session, describing: Tool to Describe your site: GOC-DB Tool to Monitor site: Nagios Tool to perform Simple network measurements: Smokeping Will not go much in detail, but will provide more information for the curious ones 2

3 1. [ GOC-DB ]

4 GOC-DB: what it is Grid Operations Center DataBase Stores (static) site information (responsible persons, addresses, resources,...) Very important to spend a minute to fill it correctly, since other tools depend on it

5 GOC-DB: main page

6 GOC-DB: requesting role Write access to GOCDB is restricted. Make sure your X509 certificate is loaded in the browser, then click Manage Roles in left column. Choose site Choose privilege Send to Riccardo, Mario, Fulvio.

7 GOC-DB: browse sites

8 GOC-DB: endpoints Each site provides one or more endpoints, or services: sitebdii, UI, CE, SE, LFC, WMS, LB,... Select a site, scroll down to see endpoints : namely, services installed at sites

9 GOC-DB: add and describe endpoint Site-admins can insert a new endpoint: please pay attention to the Service Type : your CE should be identified as CREAM-CE! Monitored set to Y triggers Nagios monitoring.

10 2. [ Nagios ]

11 What is Nagios? Nagios is the tool to monitor: Availability: is site/service up and properly configured? Reliability: is site/service really working? Nagios runs automatically: Gets site description (and downtimes) from GOCDB Schedules a number of tests ( ping, LDAP queries, real GRID jobs) Moreover, Nagios offers a web interface to get to test results, history,...

12 Nagios pre-requisites If you like your site to be green make sure: Site is properly described in GOCDB, site status is Certified and service status is Monitored Site supports ops VO: not only on services, but also on WNs Services are up, running, functional Note: please make sure you set your site to Certified in GOCDB (and/or service to Monitored ) only when they are really working for ops VO

13 Nagios: how the thing works

14 Nagios and the NGIs Nagios is the official tool used within EGI Infrastructure for distributed monitoring submission, transport, storage and visualization of probes relies on existing technologies (Nagios, ActiveMQ, Django) deployed at each NGI Results are used to measure the availability and reliability of EGI sites

15 Nagios tests Tests are scheduled automatically If you have rights, you can trigger execution of checks (HANDLE WITH CARE: do not Force checks) A hierarchy exists May be confusing/worrying sometimes Make sure you understand/guess whether there may be an upstream cause for the problem you noticed

16 Nagios for EUMEDGrid

17 Nagios: hosts view

18 Nagios: host status

19 Nagios: host status detail

20 Nagios: host groups

21 Nagios: MyEGI

22 MyEGI: Gridmap view

23 3. [ Smokeping ]

24 Smokeping: what it is A tool for monitoring network latency Time for ping : every 5 minutes, send 20 packets and measure packet loss, round-trip-time Extended to measure time to reply to a command: every 5 minutes, executes ldapsearch and measure time to reply and number of times command is unanswered Keeps history of tests Implements a star configuration From smokeping server to the world Cannot be aware of problems between sites Configuration is manual: contact grid-tech@garr.it 24

25 Smokeping: network latency 25

26 Smokeping: network site view 26

27 Smokeping: host view 27

28 Smokeping: central services 28

29 Smokeping: site services 29

30 Conclusions Quite some information is available May be too much... information is useful only if you check it :-) Need to have a to-do list for ROC shifters Can be mutuated from existing ones Want to host any of these tools in your site/country? Just ask and we'll be glad to help! 30