Session 2 Networking for the Grid Grids & networks monitoring - practical approach Jedrzej Jajor Cezary Mazurek Wiktor Procyk INGRID 2007 Instrumenting the Grid
Outline Introduction to monitoring Tools: RRDtool, MRTG, Smokeping, Netflow tools, Orca, Nagios, Common interface problem Interface tool requirements Interface tool design Summary
Monitoring system design Use Goal Qestion Metric (GQM) strategy First define goals to be achieved by the monitoring system Then define one or more questions to be answered for each goal Finally monitoring is started to obtain quantitive results for the defined questions
Monitoring - goals Performance monitoring real-time observation and periodic logging gathering information about the health of the system Availability monitoring host and service status checking Resource accounting per-process or per-user data collection resource usage data and charging fees
Monitoring tools There is a wide variety of tools to perform specific tasks Some tools are developed on GNU General Public License (GPL) Several tools use common data storage Round Robin Database (RRD) In spite of common data storage there is no common web interface
RRDtool Developed by Tobias Oethiker and distributed under GPL License Most popular numerical data storage format Different methods of data aggregation included Supports any number of data series Real input values Customizable measurment period Good graph creation support Binary format fast but architecture dependent
RRD structure AVERAGE, pdp_per_row=1 AVERAGE, pdp_per_row=10 TS DS1 DS2 DS3 TS DS1 DS2 DS3 70 71 72 73 74 75 76 77 0 10 20 30 40 50 60 70 MAX, pdp_per_row=1 MAX, pdp_per_row=10 TS DS1 DS2 DS3 TS DS1 DS2 DS3 70 71 72 73 74 75 76 77 0 10 20 30 40 50 60 70
MRTG(1) Developed by Tobias Oethiker and distributed under GPL License Supports SNMP queries, mathematical forulas and external scripts as a source of information Designed to monitoring network traffic supports only two data series in and out In case of RRD backend no built-in web interface
MRTG(2) 2nd International Workshop on Distributed Cooperative Laboratories, April 16-18, 2007
Smokeping(1) Developed by Tobias Oethiker and distributed under GPL License Designed to monitor round trip time (RTT) and packet loss Supports also custom plugins to monitor HTTP, DNS, FTP and other services availabily Sophisticated way of visualization two different parameters on the same graph
Sample graph Smokeping(2)
Netflow tools(1) Netflow protocol developed by Cisco Systems Supports network layer 3 and above by probing random packets Netflow tools: nfdump and nfsen distributed under Berkley Sofware Distribution (BSD) license Enable netflow data collection and visualization in intuitive way by packets, octets and flows Supports custom plug-ins and filters
Netflow tools(2) 2nd International Workshop on Distributed Cooperative Laboratories, April 16-18, 2007
Orca(1) Developed by Blair Zajac and Yahoo!, Inc. Designed to plot arbitrary data from text files to a directory on a Web server Often used to store and visualize computer system parameters like CPU usage, number of processes, filesystem and memory usage etc. Supports multiple data series and mathematical expressions, provides built-in web interface and RRD backend
Orca(2) 2nd International Workshop on Distributed Cooperative Laboratories, April 16-18, 2007
Nagios(1) Developed by Ethan Galstad and distributed under GPL license Designed to monitor availability of systems and services and notify about arising problems Optional performance data collection Supports external scripts, dependencies between hosts and services, scheduled downtime and distributed configurations
Nagios(2) 2nd International Workshop on Distributed Cooperative Laboratories, April 16-18, 2007
Problems Each tool has its own interface Few tools have built-in support for authentication and authorization Each tool has its own configuration format Each tool is bound to a single data format Tools don t have any support for browsing archive data Existing map creation scripts dedicated to only one monitoring tool; separate configuration for monitoring tool and script
New tool requirements Support for user authentication and authorization Consistent web interface for multiple monitoring tools Hierarchical maps and control panels Configuration of the interface merged with the configuration of the monitoring tool Graph defining as flexible as in RRD graph tool Multiple configuration storage backends; merging configuration from files and databases
Interface tool design(1) Implementation language possibilities: Perl, PHP, Java, Python,.NET; chosen lang.: PHP Base class for configuration reading, derived classes for specific configuration types Similary, base class for depicting graphical objects and derived classes for each type of visualization type Easily extendable by user-created additional classes or user-defined charts
Interface tool design(2) IRenderable RRDGraph +Render() #Prepare() IRenderable Map +Render() -Map 1 -Components * MapElement +Draw() MrtgGraph SumGraph OrcaGraph MapNode MapLink MapBar #Prepare() #Prepare() #Prepare() +Draw() +Draw() +Draw() IConfigReader PgsqlConfigReader IAuthenticator WebServerAuthentication IConfigReader MrtgConfigReader +IsAuthenticated() +Authenticate() IAuthorizator PgsqlAuthorization IConfigReader OrcaConfigReader +IsAuthorized(in User, in Object, in Method) Application «call» «call» «call» «interface» IConfigReader +GetGlobalCfg() +GetTargetCfg(in targetname) +GetTargetsCfg(in targetlist) «interface» IAuthenticator «interface» IRenderable +Render() «interface» IAuthorizator 2nd International Workshop on Distributed Cooperative Laboratories, April 16-18, 2007 «call»
Client Applicatio IAuthenticator IAuthorizator IRenderable Request Authenticate Render IsAuthorized
Demo(1) 2nd International Workshop on Distributed Cooperative Laboratories, April 16-18, 2007
Demo(2) 2nd International Workshop on Distributed Cooperative Laboratories, April 16-18, 2007
Development plans Enable authorized users to edit the configuration through the web interface Manage the configuration of most tools from a single web interface Prepare on-line map editor Group objects into map layers Search engine
Summary Monitoring system is a must in today s computer systems Developed tools enable intagrated access to all defined metrics and data gathered by monitoring tools In everyday s duties it is not only very useful, but creates new possibilities for data analysis by integration, consolidation and visualization in different ways: graphs, charts, bars, pie charts etc.
Thank you
Grids & networks monitoring - practical approach Jedrzej Jajor, Cezary Mazurek, Wiktor Procyk
Interface tool design(3) Client Application IAuthenticator IAuthorizator IRenderable Request IsAuthenticated Result Render IsAuthorized Result Result Result