Open PostgreSQL Monitoring A brand new & free monitoring suite Jean-Paul Argudo / Julien Rouhaud Dalibo - www.dalibo.org PostgreSQL Conference Europe 2014 - October, 22 1 / 26
Origins Comparison with Oracle Multimaster : Bi-Directional Replication (BDR) (2014-07) Materialized views : version 9.3 (2013-09) Easy partitionning : TODO Parallel queries : TODO AWR : PoWA Oracle Enterprise Manager, Oracle Grid : OPM! 2 / 26
State of the Art A tool is missing proprietary software existing Open Source projects to assemble no real Open Source tool dedicated to PostgreSQL 3 / 26
Main ideas To be accepted by the community the tool must be completely free and Open Source crafted for PostgreSQL extensible with a pluggable architecture 4 / 26
Objectives a powerful monitoring platform the standard for PostgreSQL monitoring must compete with proprietary software 5 / 26
Functionalities an extensible architecture Suitable probes graphs alerts 6 / 26
An active project OPM has 5 releases already 13 releases for check_pgactivity fast pace dev : release often, release early 7 / 26
History Version 1.X creation of the architecture (opm_core and wh_nagios) first metrics in check_pgactivity graphs management aspect (Stacked, filled, etc) human-readable units export graphs into images 8 / 26
History Version 2.0 to 2.3 new design new authentication and grant/revoke methods pluggable architecture to accept plugins tags management bug fixes 9 / 26
OPM in details 1/2 A long project Composed of some new projects Still using some well-known ones, like Nagios 10 / 26
OPM in details 2/2 A graphing engine opm_core Modules to store data, at the moment, only wh_nagios Probes dedicated to PostgreSQL : check_pgactivity 11 / 26
Graphing engine 1/3 PostgreSQL extension (opm_core) written in pl/pgsql web GUI in Perl (Mojolicious) and Javascript access control 12 / 26
Graphing engine 2/3 discovers automatically the data found (servers, services, etc) generates automatically the associated graphs, and associates them to the correspond server tags to aggregate various graphs 13 / 26
Graphing engine 3/3 an API to facilitate writing to the warehouse sampling function storing tables to inherit... 14 / 26
Authorizations groups and Users 1 server = 1 group each user is part of one or many groups two levels of users : Administrator and User 15 / 26
wh_nagios Storage module standard warehouse dedicated to Nagios data (perfdata) storing and archiving of all the values contains a dispatcher to handle Nagios data 16 / 26
check_pgactivity Nagios probe (1/3) standard Nagios probe very simple code, and an API performance data oriented to graphing managing of units in the perfdatas the right metric is sent, no transformation needed 17 / 26
check_pgactivity Nagios probe (2/3) better or new probes, based on our experience local file storage to compute deltas probe on bgwriter writing speed into WALs... 18 / 26
check_pgactivity Nagios probe (3/3) still many probes to invent or add roadmap make it more modular create a dedicated probe for the system 19 / 26
Overview 20 / 26
Create a warehouse 1/3 specify a schema table inheritance services metrics series And all the warehousing necessary 21 / 26
Create a warehouse 2/3 implement functions get_metric_data() (grant/revoke)_dispatcher() purge_service() and all the warehousing necessary 22 / 26
Create a warehouse 3/3 optional, add elements in the interface decorateors, e.g. : details_server() details_service() or dedicated screens e.g. : Administration screens wh_nagios 23 / 26
Roadmap remove the dependency on Nagios or another tool push and pull modes support have the dashboard customizable metrics correlation and much more 24 / 26
Some numbers more than 100 servers generates more than 2000 services more than 17000 metrics over 9 months represents 8.6 Gb of data 25 / 26
Demo Because a good demo is better than boring slides! demo.opm.io login : opm pass : demo More informations on opm.io 26 / 26