From Events to Persistence: Leveraging Salesforce Replay Channel & CockroachDB
- 1. From Events to Persistence: Leveraging Salesforce Replay Channel & CockroachDB Bangalore Meetup Group
- 2. Organizers
- 3. Speaker Srita Shervani Velagada Technical Lead TEKSystems Global Services India
- 4. Agenda ● Introduction ● What is EDA ● Understanding Salesforce Streaming Events & Configuration ● Salesforce Connector ● Why Cockroach DB ● Setting up CockroachDB ● Integrating Salesforce replay channel & Cockroach DB in Mule ● Demo
- 5. Event Driven Architecture Event-driven architecture enables systems to react dynamically to changes rather than relying on direct interactions. Events represent real-world occurrences like order placements or payments, allowing asynchronous communication between applications. Key Benefits: ● Real-time processing: Immediate response to changes. ● Decoupling: Reduces dependencies between services. ● Scalability: Handles high event volumes efficiently.
- 6. Salesforce Streaming Events ● PushTopic ● Generic Events ● Platform Events ● CDC-Change Data Capture Events.
- 8. Cockroach DB What is Cockroach DB & Key features CockroachDB is a cloud-based database that stores data across multiple locations to provide quick access. It's designed to be scalable, fault-tolerant, and highly available. Features: ● Distributed Architecture ● Automatic Replication ● Transactional Consistency ● Self healing & Recovery
- 9. SQL Layer Transaction Layer Distribution Layer Replication Layer Storage Layer SQL Layer Transaction Layer Distribution Layer Replication Layer Storage Layer Node Node Pebble KV Store Write to a key in key-range 2 Range 2 leader Range 2 follower
- 10. Salesforce Connector • Salesforce is the leading SaaS provider with its flagship CRM platform, available exclusively online. • It helps businesses connect to their customers in a whole new way, so they can find more prospects, close more deals, and wow customers with amazing service. • Anypoint Connector for Salesforce (Salesforce Connector) enables you to create apps that react to Salesforce events such as adding, changing, or deleting objects, topics, and channels.
- 11. Salesforce Replay Channel Listener • This operation can be used to subscribe to salesforce channel/topic events. • One more advantage of using replay topic/channel listener is it can can continue from the last replay ID they received before restarting the application. • Earlier versions of connector before 10.14 have only below four options for REPLAY ID at the connector level. • 1) ALL • 2) ONLY_NEW • 3) FROM_REPLAY_ID • 4)FROM_LAST_REPLAY_ID
- 12. -1 : " ONLY_NEW “ • Listens only to new events generated after the subscription starts. • Ideal for real-time streaming. • Example Use Case: • A live dashboard showing only new order updates from Salesforce. • Real-time fraud detection system processing transactions as they happen. • All the events generated within 24 or 72 hr retention period will be consumed. • Ideal for loading historical data. • Example Use Case: To receive historical data for audit logs/analytics -2 : “ ALL_EVENTS “
- 13. From_Replay_ID • Resumes from a specific Replay ID that you store. • Used for guaranteed event processing in case of failures. • Example Use Case: • A MuleSoft application storing the last processed Replay ID in Object Store. If the app restarts, it resumes from where it left off. • Ensures no duplicate processing and no missed events. From_Last_Replay_ID • For the first time deployment connector searches for replay id in object store as it cant get any value it starts with replay id as -2. • We need to select Replay Option as FROM_LAST_REPLAY_ID and enable ObjectStoreV2 to enable autoreplay feature. • Example Use Case: • A Mule application crashes and restarts. You want to process all events from the last 24 hours. • A new subscriber needs historical data.
- 14. Key Differences
- 15. Preferred Design
- 16. Thank You