Visualize and Analyze Apache Geode Real-time and Historical Metrics
2 likes1,041 views
This document discusses visualizing and analyzing Apache Geode real-time and historical metrics. It provides an overview of Geode metrics that can be monitored, including cache server, region, client, and locator metrics. It describes architectures for exporting Geode JMX and statistics archive data to a time series database like InfluxDB and visualizing it in Grafana. Code examples and links are provided for loading Geode JMX and statistics to InfluxDB and pre-built Grafana dashboards. The full stack allows visualizing and analyzing both real-time and historical Geode metrics.
![Statistic to InfluxDB Mapping
+ name
+ timestamp
Measurement
+ name
+ value
Field
+ name
+ value
Tag
(Indexed)
(not-Indexed)
InfluxDataModel
GeodeStatisticArchive
+ name
+ type
+ sampleCount
Resource Instance
Statistics Archive
Instances
*
+ descriptor.name
+ timestampls:long[]
+ rawSnapshot:double[]
StatValue
values
*](https://image.slidesharecdn.com/geodemetricsvisualization-180201173141/85/Visualize-and-Analyze-Apache-Geode-Real-time-and-Historical-Metrics-22-320.jpg)
Visualize and Analyze Apache Geode Real-time and Historical Metrics
- 1. Visualize and Analyze Apache Geode Real-time and Historical Metrics Christian Tzolov Software Engineer, Spring Cloud ASF, Apache Committer, PMC member
- 2. Disclaimer The following is intended to outline the general direction of Pivotal's offerings. It is intended for information purposes only and may not be incorporated into any contract. Any information regarding pre-release of Pivotal offerings, future updates or other planned modifications is subject to ongoing evaluation by Pivotal and is subject to change. This information is provided without warranty or any kind, express or implied, and is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions regarding Pivotal's offerings. These purchasing decisions should only be based on features currently available. The development, release, and timing of any features or functionality described for Pivotal's offerings in this presentation remain at the sole discretion of Pivotal. Pivotal has no obligation to update forward looking information in this presentation.
- 3. Apache Geode / GemFire “… in-memory, distributed database with strong consistency built to support low latency transactional applications at extreme scale”
- 4. Geode Metrics Geode Cluster Run-Time Statistics Historical Statistics Visualization (VSD) Monitoring, Visualization, Alerts (JMX)
- 5. Geode Metrics Unified Geode Cluster Run-Time Statistics Historical Statistics Monitoring, Visualization, Alerts
- 6. Monitor What? Cache Server Region A Region X ValKe y v1k1 v2k2 … Region - consistent, distributed Map (key-value) dataset CacheServer – process managing Regions Client Locator Client –read and modify Region content Locator – track membership information
- 7. Cache Server 1 Region ARegion A’ Region B’ Region X’ Cache Server N Region ARegion A’’’’ Region B’’’ Region X’’’ Cache Server 2 Region ARegion A’’ Region B’’ Region X’’ Federated MBean Architecture Locator 1 Distributed System Distributed Region Member GemFire:type=Distributed, service=Region,name=<regionName> GemFire:type=Distributed,service=System GemFire:type=Member, member=<name-or-dist-member-id> Notifications gemfire.distributedsystem.cache.member.departed gemfire.distributedsystem.cache.member.joined gemfire.distributedsystem.cache.region.created gemfire.distributedsystem.cache.region.closed https://gemfire.docs.pivotal.io/geode/managing/management/mbean_architecture.html network JMX Manager - Single-agent view of the Distributed System JMX Manager MBeanServer MBeanServer MBeanServer
- 9. Geode Pulse • Cluster View • Member View • Region View • Alerts Widget https://gemfire.docs.pivotal.io/geode/tools_modules/pulse/pulse-views.html
- 10. Beyond Pulse • Extensibility • Customized Views and Alerts • Compare metrics between Members or Regions • Integrate with 3rd party monitoring tools
- 11. TSDB and Grafana Architecture JMX to TSDB Loader Grafana TSDB Geode Cluster JMX Manager
- 12. Grafana Open platform for Analytics and Monitoring • Data Sources • Panels • Dashboards
- 13. Time Series Database (TSDB) • Real-time process/store of time-series data • Data is larger than memory • Storage per data point efficiency • Sequential, ordered per series reads • Write mostly (95%) • Data is larger than memory • Storage efficiency - few bytes of storage per data point • OSS TSDB: InfluxDB, Open TSDB, Prometheus, … https://en.wikipedia.org/wiki/Time_series_database
- 14. JMX to InfluxDB Mapping + name + timestamp Measurement + name + value Field + name + value Tag (Indexed) (not-Indexed) {region : Zips} …. + avgBucketSize + entrySize + memberCount … MemberMBean + avgBucketSize + entrySize + memberCount … DistributedSystemMBean {region : Zips} …. + avgBucketSize + entrySize + memberCount … DistributedRegionMBean GemFire:service=Region,name=/Zips,type=Distributed Influx Data Model Geode MBean Model
- 15. JMX to Grafana Geode Cluster JMX to Grafana Grafana InfluxDBJMX Manager java -jar ./jmx-to-grafana-0.0.2-SNAPSHOT.jar --mbeanHostName=localhost --mbeanPort=1099 --influxUrl=http://localhost:8086 --influxDatabaseName=GeodeJmx --cronExpression="0 0/1 * * * ?" docker run -d --name grafana -p 3000:3000 grafana/grafana docker run -d --name influxdb -p 8086:8086 influxdb
- 16. JMX to Grafana Tookit https://github.com/tzolov/geode-dashboard
- 17. Dashboard Library Cluster View Members View Combined Members View Region View
- 18. Apache Geode Metrics Geode Cluster Run-Time Statistics Historical Statistics Monitoring, Visualization, Alerts
- 19. Geode Statistics Archive • Statistics Collected per Member • Local Archive Files • Build-in Statistics • Runtime (peer per node, clients, data entities,…) • Resources (memory, cpu, descriptors, system load,…) • Operations Throughput (Cache, Disk,…) • … • Application-Defined Statistics http://gemfire.docs.pivotal.io/gemfire/tools_modules/vsd/vsd_useful_statistics.html
- 20. Viewing Archived Statistics • The Visual Statistics Display (VSD) • Guide to Useful Statistics: http://gemfire.docs.pivotal.io/gemfire/ tools_modules/vsd/vsd_useful_statistics.html
- 21. Geode Cluster Locator 1 CacheServer 1 CacheServer N … Statistics to Grafana Architecture Grafana InfluxDB A A A Statistics To Grafana
- 22. Statistic to InfluxDB Mapping + name + timestamp Measurement + name + value Field + name + value Tag (Indexed) (not-Indexed) InfluxDataModel GeodeStatisticArchive + name + type + sampleCount Resource Instance Statistics Archive Instances * + descriptor.name + timestampls:long[] + rawSnapshot:double[] StatValue values *
- 23. Geode Cluster Locator 1 CacheServer 1 CacheServer N … Statistics to Grafana Architecture Grafana InfluxDB A A A Statistics To Grafana java -jar ./statistics-to-grafana-0.0.2-SNAPSHOT.jar --influxUrl=http://localhost:8086 --influxDatabaseName=GeodeArchive --geodeMemberName=server1 --archiveFile=server1ArchiveFile.gfs statistic-sampling-enabled=true statistic-archive-file=myArchive.gfs archive-file-size-limit=100 archive-disk-space-limit=1000
- 24. The Full Stack
- 25. Next? 25 • Micrometer- http://micrometer.io/ • Grafana Geode Source Plugin (Geode SQL adapter) • Add real-time metrics for AsyncListener, DistributedLock, Gateway • Additional pre-build Dashboards