Monitoring and observabilty at Bolt
0 likes89 views
BOLT is a leading provider of property and casualty omnichannel customer solutions. They provide a single platform for agents, contact centers, and direct-to-consumer channels. To support premium SLAs and fast response times for customers, BOLT has implemented a monitoring and observability solution with 24/7 support, real-time dashboards, and alerting across infrastructure, applications, and business metrics. The solution captures raw data, generates insights, and provides various stakeholder teams visibility to quickly detect, manage, and resolve incidents.
Monitoring and observabilty at Bolt
- 1. SM MONITORING AND OBSERVABILITYTO SUPPORT PREMIUM SLA FAST RESPONSE – USE CASE Iddo Porath – Director of Production Support &Technical Services Confidential – Do not duplicate or distribute without written permission from BOLT
- 2. BOLT – WHO ARE WE For a Distributor of Insurance this means: ✓Own the customer without always owning the risk. ✓Enabling collaboration with other carriers for the benefit of your customers and your bottom line. For a Market/Insurance Product provider a single integration point gets you: ✓Instant distribution of your product, by leveraging vast existing marketing channels. Through the BOLT Platform, insurers are better able to meet the evolving needs and preferences of consumers, while accelerating growth and profitability in the digital world. Confidential – Do not duplicate or distribute without written permission from BOLT 2 BOLT is the leading provider for P&C omnichannel customer solutions. The BOLT platform provides a single interface for all channels – field agents, contact center, direct to consumer – with the efficiency of a common infrastructure; common market network, and support services built into one platform for all channels.
- 3. BOLT TRANSFORMS DISTRIBUTION Enabling Choice Model Digital Distribution Integrate traditional and digital channels Rapidly introduce new products Easily enter and try new markets TOMORROW Own the customer without owning all of the riskDirect to Consumer MGAs/Wholesalers TODAY Exclusive Agents IndependentAgents Processes People Technology Markets Alternative Distribution Confidential – Do not duplicate or distribute without written permission from BOLT 3
- 4. BOLT Platform High Level Design Monitoring & Observability Objects CORE SERVICESPERIPHERAL DASHBOARD SERVICES INTERVIEW & SUBMISSION CONTENT SERVICE SLA SERVICE Quote Operation CRYPTO CUSTOMER SERVICE RULE ENGINE ODM SUBMISSION SERVER V3 ORCHESTRATION APPLICATION WEB API CONFIGURATION Quote flow User Interface Layer Market Finder LOB Selection Quote Questionnaire Rates Page Comparison Page Agent Support features (Emails & Notes) Consumer features (Chat, Schedule Call) AUDIT SERVICE Desktop Client UI Engine APPLICATION WEB APIAPPLICATION WEB API CARRIER CONNECTIONS CARRIER APPETIT E SERVICE CUSTOMER OPERATIONS CRM CRM OPERATIONS Administration Tools F5 Load balancers with High availability (VM) User Management Appetite & Moratorium Rules Class Code Mapping Carrier Credentials Queue Strategies Roles & Skills Availability & Agent capacity Lead & Marketing Bind Questions Bridge Policy Shell SMTP MAIL SERVER INTERVIEW OPERATION INTERVIEW GOOGLE LOCATION SERVICES MSB REGISTRATION Quote QUOTE START QUOTE RETRIEVE DEEP LINK TO QUOTE AUOTH 2.0 AUTHENTICATION QUOTE STATUS CREATE/UPDATE/DELETE AGENT CREATE/UPDATE/DELETE AGENCY GET RESULTS FOR QUOTE CREATE AND RATE A QUOTE PREFILL EMAIL DELIVERY LOCATION RESULT DATA CRM Audit Job Scheduler Account Data Event Submissio n Sequence Policy Data Document Log Quote Events (Fact) Monitor Metrics Logger Events DATA, MESSAGE & FAILURE QUEUE IN EACH SILO Workflow FEATURE MANAGER ENTITY DATA Business Event SLA Governo r WORKFLOW JSON ON DEPLOY Content Designer (Design time component only) NOTIFICATION SERVICE
- 5. CHALLENGE – SLA Confidential – Do not duplicate or distribute without written permission from BOLT 5 US InsuranceTier 1Companies – Premium SLA • Incident detection – Proactive Incident Management You tell Us vs.We tellYou • Fast response to incident – Resolution standards • Operational Status Transparency • Availability • Response times • Analytics and Success Metrics • Service Level Credits
- 6. FAST RESPONSE SOLUTION COMPONENTS • Process • Pro-Active vs Reactive • Self service portal and dashboard for customers • Alerts and dashboards across teams • Coverage • 24/7 NOC • US andTLV Teams • People • NOC • BOLT 3 tier model • Technology • Alerts, Dashboards and Deep Dive Analysis Tools Confidential – Do not duplicate or distribute without written permission from BOLT 6
- 7. SUPPORT STAKEHOLDERS Confidential – Do not duplicate or distribute without written permission from BOLT 7 Tier 1 - Response • 24/7 NOC – Provides Response to Alerts and off- hour Partner Incident Reports (limited to predefined partner personnel) • Self Service End users support is performed by partners not provided by BOLT Tier 2 - Management • Production Support – IncidentProblem management • Monitoring and Observability requirements • Alert and Dashboard Setup and Management Tier 3 - Resolution • Development - Application Problem Resolution On-Call team • IT & DEV OPS- Infrastructure Problem Resolution
- 8. MONITORING AND OBSERVABILITY COMPONENTS •Raw Data •Events vs Logs •Insights - Summary data,Alerts and KPIs •Client Facing Dashboards • Classic Monitoring • Dev Logs • Server and IIS Logs • NetworkInfrastructure • External availability RAW DATA • Application Reported Performance • Real-time user ExperienceAgent ConsumerActivity KPIs • Business Metrics and KPIs Events and Summarized DATA • Thresholds • Summarized counts & stats • Visualizations • Alerts Dashboards and Alerts Confidential – Do not duplicate or distribute without written permission from BOLT 8
- 9. IMPLEMENTATION • Monitoring • Network Monitoring – PRTG and Splunk Infrastructure logs collection network and Queues • Server and Page Response times → User experience – IIS ADV Logs • Analytics - Client side metrics • Events – Business and Service Health – The Good the Bad and The Ugly Truth • Ability to track certain features → Incident detection: Error counts , Experience representation ,Activity level status , Success metrics. • Triage – Impact and urgency analysis. Comparison to similar timeframe • Response by priority • Health checks for Identification of rogue services – Quick or Auto Action • Logs • Enrich Raw Data – Towards AI • DevTools → Analysis → Resolution • Root Cause Analysis Confidential – Do not duplicate or distribute without written permission from BOLT 9 Response Analysis Insights Report
- 10. CHALLENGES Confidential – Do not duplicate or distribute without written permission from BOLT • Verbosity and Costs • ALM – Release cycle • User Stories (Observability Tasks vs Observability Approach) What started ,When How it Ended,Who was it • Change Management and logs standardization • QA and Automation • Next Steps • From Pro Active to Predictive (AI) • DevOps and Automated actions 10
- 11. Confidential – Do not duplicate or distribute without written permission from BOLT 11 THANKYOU IDDOP@BOLTINC.COM