![OpenCollector
An example from the sample configs:
ânameâ: âglobal_statsâ,
âprefixâ: âdatabase:{datname}.global_statsâ,
âqueryâ: [
âSELECT â,
â datname, numbackends::float8, â,
â xact_commit::float8, xact_rollback::float8, â,
â blks_read::float8, blks_hit::float8, â,
â pg_catalog.pg_database_size(datid)::float8, â,
â pg_xlog_location_diff(pg_current_xlog_insert_location(),
âFROM pg_catalog.pg_stat_database â,
âWHERE datname = current_database() â
],
âresultâ: [
{ ânameâ: âdatnameâ, âtypeâ: âinternalâ },
{ ânameâ: ânumbackendsâ, âtypeâ: âvalueâ },
{ ânameâ: âxact_commitâ, âtypeâ: âcounterâ },
...
]](https://image.slidesharecdn.com/monitoringpgwithgraphitegrafana-170125212045/85/Monitoring-pg-with_graphite_grafana-19-320.jpg)
Monitoring pg with_graphite_grafana
- 1. Monitoring PostgreSQL using Time-Series systems like Graphite and/or Grafana with OpenCollector Jan Wieck - OpenSCG
- 2. Overview âą What is Monitoring? âą Graphite & Carbon âą Grafana âą Why use Carbon? âą Why use Graphite AND Grafana? âą PostgreSQL Metric Data âą OpenCollector âą osinfofdw
- 3. What is Monitoring? âą Capture Time-Series data âą Metric-Name, Value, Timestamp âą Visualize Time-Series data âą Define alerts base on Time-Series data âą Statistical analysis of Time-Series data âą Getting an alert when your primary DB server is down is covered by the above!
- 4. Graphite & Carbon âą Carbon is a server for collecting Time-Series data âą Simple line based protocol on port 2003 âą Python-pickle protocol on port 2004 âą Graphite is a WEB based GUI on top of Carbon âą Some Dashboard functionality
- 6. Grafana âą Grafana is more Dashboard focused âą Grafana can use many Time-Series data sources âą Graphite âą Elasticsearch âą CloudWatch âą InfluxDB âą OpenTSDB âą KairosDB âą Prometheus
- 8. Why use Carbon? Carbon provides an extremely simple protocol to send Time-Series data #!/bin/sh CHOST=âgraphite.host.nameâ CPORT=â2003â METRIC=âtest.PIâ VALUE=â3.1415â echo â$METRIC $VALUE âdate +%sââ | nc $CHOST $CPORT
- 10. Why use Carbon? Not a very useful metric, but consider capturing the runtime of a shell script based cron job. Carbon also provides a Python-pickle based protocol on port 2004 that can be used to send hundreds of metric points condensed in one send(1).
- 11. Why use Graphite AND Grafana? âą Grafana is more Dashboard focused âą Templating makes it easy to define one Dashboard and use it for many hosts/databases âą Getting to a Dashboard is easier âą Can define Alerts âą Looks cool âą Graphite is better at ad-hoc graphing âą The metric tree is easier to navigate than clicking through Grafanaâs pull down system
- 12. However ⊠This isnât a talk advertising Graphite or Grafana. This is a talk about capturing monitoring data from PostgreSQL and delivering it into a Time-Series data system. Carbon/Graphite and Grafana are example destinations.
- 13. PostgreSQL Metric Data PostreSQL produces quite a number of data points. âą On the table level âą about 30 metric points âą On the index level âą about 6 metric points âą On the database level âą about 20 metric points
- 14. PostgreSQL Metric Data Those per table/index numbers are not of concern when you look at your typical benchmark database. But what about a database with 1,800 tables and 13,000 indexes? Now we are talking about 132,000 metric points every time interval! Captured every minute that is 7.9M per hour, 190M per day, 17.1B per quarter. Donât do that with snapshots captured inside the DB.
- 15. PostgreSQL Metric Data That isnât as exotic as it looks at first glance PostgreSQL system views like pg_stat_all_user_tables will report every single metric point even if a table or index hasnât been used for the past 12 months. How many dead tables (schemas) does your database have? A generic monitoring system canât tell them apart.
- 16. PostgreSQL Metric Data But that isnât all. Many metrics are presented in what is a continuous counter, but the useful value is actually their increase per second. Examples: âą Tuples inserted, updated, deleted, fetched âą Index/Sequential scans This is the same as for OS statistics like: âą Network operations âą Disk operations
- 17. PostgreSQL Metric Data While that is efficient inside of the PostgreSQL server for collecting the data, it is rather inconvenient when browsing it in a system like Graphite or Grafana. Sure, they can apply a function like âpersecond()â and it is only 20 mouse clicks away âŠ
- 18. OpenCollector âą OpenCollector is a PostgreSQL monitoring daemon sponsored by OpenSCG âą It is designed to address the aforementioned problems âą JSON configuration files define all the operation âą Target Carbon server âą Source Database(s) âą Queries to run and what metrics they return âą Sparse metric reporting
- 19. OpenCollector An example from the sample configs: ânameâ: âglobal_statsâ, âprefixâ: âdatabase:{datname}.global_statsâ, âqueryâ: [ âSELECT â, â datname, numbackends::float8, â, â xact_commit::float8, xact_rollback::float8, â, â blks_read::float8, blks_hit::float8, â, â pg_catalog.pg_database_size(datid)::float8, â, â pg_xlog_location_diff(pg_current_xlog_insert_location(), âFROM pg_catalog.pg_stat_database â, âWHERE datname = current_database() â ], âresultâ: [ { ânameâ: âdatnameâ, âtypeâ: âinternalâ }, { ânameâ: ânumbackendsâ, âtypeâ: âvalueâ }, { ânameâ: âxact_commitâ, âtypeâ: âcounterâ }, ... ]
- 20. OpenCollector âą Since the queries are in config files, you can customize them âą Additional WHERE clauses âą Change from pg_stat_all_ to pg_stat_user_ âą Add your own, application specific queries âą OpenCollector is modular and allows to add other things âą OpenCollector is open source
- 21. osinfofdw âą osinfofdw is another open source project sponsored by OpenSCG âą A MultiCorn based FDW around Python-psutil âą Access OS level statistics via SELECT âą CPU usage âą Memory usage âą Disk IO âą Network IO âą Filesystem information