ELASTICSEARCH-CLIENT
0 likes199 views
The document describes creating a wrapper for the ElasticSearch client used by three services - Bernard, Ernest and Wally. The wrapper will measure query times for each client and publish metrics to JMX. CollectD will collect the metrics and send them to Cosmos for graphical representation. This will help identify which service is causing issues if ElasticSearch goes down. The wrapper uses Yammer metrics to time synchronous function calls across the client and decorate asynchronous listeners to time individual queries.
ELASTICSEARCH-CLIENT
- 1. ElasticSearch Native Client Services like Bernard, Ernest and Wally query the ElasticSearchServer via ElasticSearchClient to get Lisitng Level Data, Offer-Mappings & Victor- References. Bernard Client Ernest Client Wally Client ElasticSearch Client ElasticSearch Server Bernard - Offer Computation against a listing Ernest - Offer Creation on Seller Dashboard Wally - Scheduler for offer computation & Redriver santa-prodparallell-0003.nm.flipkart.com:9200/_plugin/head/ There is only one Cluster of ElasticSearch and all the three DSP-Santa services query to it. In case it goes down or cluster health is affected, we need to identify which service is causing the problem so that we can furthur investigate into the particular service's server. Solution What we could do is we can write a wrapper to the ElasticSearch Client and all the services will use this wrapper instead of using the ES-Client directly. The wrapper will invoke the ES-client with all the calculations around the function calls. With the call-time data we could generate metrics and publish it on the JMX port of the client. From there the data will be pushed to services like Cosmos which will draw graphs around the data. For synchronous functions, time is taken across the function calls. For asynchronous functions, the original ActionListener is decorated with yammer metrics and passed on to the original ES-Client.
- 2. ElasticSearch Wrapper Client Bernard Client Ernest Client Wally Client ElasticSearch Client ElasticSearch Server ElasticSearch Wrapper Yammer Metrics JMX port collect daemon COSMOS What we actually did? Taking help of TImer from Yammer Library, we measured the frequency of, and time taken, by the queries for each client. The metrics are published on the JMX port of the client, from where the Collect-D collects it and publishes it on COSMOS. Cosmos will represent all data graphically and will help to analyse and debug is anything goes wrong. Basically 2 use-cases had to be covered. Synchronous Functions Asynchronous Functions We measure time spent across function calls. That is same as time for each query. To measure time for each query , we check when the ActionListner detects an Event or gives an Exception Project : ElasticSearch Wrapper/ ElasticSearch Metrics Generator Project Manager : Mrs. Vinita Bansal Project Mentor : Mr. Prarabdh Awasthi - Dhruv Chandhok