Durable Streaming and Enterprise Messaging
4 likes1,485 views
The document discusses Salesforce's plans to enhance its streaming and enterprise messaging capabilities. It describes the limitations of the current streaming API and previews new "durable streaming" functionality that will allow events to be replayed. It also outlines Salesforce's vision for an "Enterprise Messaging" platform using an event-driven architecture and "Conduit" API to reliably deliver data changes in real time or from a replay log. A roadmap is provided showing phased rollout of these capabilities through 2016.
Durable Streaming and Enterprise Messaging
- 1. Durable Streaming and Enterprise Messaging Jay Hurst Director, Product Management jhurst@salesforce.com @extraidea John Brock SMTS – Enterprise Messaging jbrock@salesforce.com @_ johnbrock
- 2. Safe Harbor Safe harbor statement under the Private Securities Litigation Reform Act of 1995: This presentation may contain forward-looking statements that involve risks, uncertainties, and assumptions. If any such uncertainties materialize or if any of the assumptions proves incorrect, the results of salesforce.com, inc. could differ materially from the results expressed or implied by the forward-looking statements we make. All statements other than statements of historical fact could be deemed forward- looking, including any projections of product or service availability, subscriber growth, earnings, revenues, or other financial items and any statements regarding strategies or plans of management for future operations, statements of belief, any statements concerning new, planned, or upgraded services or technology developments and customer contracts or use of our services. The risks and uncertainties referred to above include – but are not limited to – risks associated with developing and delivering new functionality for our service, new products and services, our new business model, our past operating losses, possible fluctuations in our operating results and rate of growth, interruptions or delays in our Web hosting, breach of our security measures, the outcome of any litigation, risks associated with completed and any possible mergers and acquisitions, the immature market in which we operate, our relatively limited operating history, our ability to expand, retain, and motivate our employees and manage our growth, new releases of our service and successful customer deployment, our limited history reselling non-salesforce.com products, and utilization and selling to larger enterprise customers. Further information on potential factors that could affect the financial results of salesforce.com, inc. is included in our annual report on Form 10-K for the most recent fiscal year and in our quarterly report on Form 10-Q for the most recent fiscal quarter. These documents and others containing important disclosures are available on the SEC Filings section of the Investor Information section of our Web site. Any unreleased services or features referenced in this or other presentations, press releases or public statements are not currently available and may not be delivered on time or at all. Customers who purchase our services should make the purchase decisions based upon features that are currently available. Salesforce.com, inc. assumes no obligation and does not intend to update these forward-looking statements.
- 3. Jay Hurst Director - Product Management, Salesforce jhurst@salesforce.com @extraidea
- 4. John Brock SMTS – Enterprise Messaging jbrock@salesforce.com @_ johnbrock
- 5. Streaming Data from Salesforce Many customers are facing a set of challenges around pushing changes out of Salesforce. Raise your hand if you: • Have used the Classic or Generic Streaming API in Salesforce • Have tried to build synchronization or change detection based on streaming, but are having issues • Have a need for a reliable, durable flow of data in and out of Salesforce
- 7. Enterprise Messaging The goal is to support the concept of an Event as a first class object • An event is an immutable, time-stamped set of values. • Can represent multiple values • Actions - "A report was generated", "An order has been placed" • Measurements – "API limits are at 50%", "1MM records have been created" • State Changes - "Account 0010000023234 field Name was updated to 'Acme, Inc.'" • Allow a large number of events to flow through the platform • Scale to millions of events per minute per pod • API first design to support: • Publishing events to a message channel • Consuming events from a message channel • Provide ability to replay events from a defined time-window
- 8. Where are we today? Classic Streaming API Based on the Bayeux/CometD protocol • The client will open up a long polling connection to the Salesforce server and subscribe to a Push Topic • The client must regularly reconnect to keep the session alive • When a record meets the query, we send an event down the connection to the client • The client then reconnects to the long polling connection and waits for the next message Allows you to define a query and have Salesforce send an event to your listener whenever a record is modified and meets that query • The messages are sent in near real time • The Streaming API is not loss-less • If the client is not connected, the message will not be received, and cannot be replayed
- 9. Where are we going tomorrow (or shortly thereafter)? Durable Streaming API The first implementation of Enterprise Messaging will be to enhance the Streaming API • Provide the ability for replay of Generic Streaming API events (Durable Streaming) • Existing Streaming Channel pushes will generate the Conduit events • Additional Event ID will be provided in message • Event ID will be atomic and ever-increasing • A new replay method will be available that will replay all events from a provided ID • The replay will re-deliver all messages that are after the provided ID • Event replay will be available for up to 24 hours • Protocol (Bayeux/CometD) will remain and require changes only to take advantage of replay • Clients will need to store and provide the Event ID • If no event ID provided in subscribe, then the events will be delivered from the tip of the queue • This will result in the existing behavior available today
- 11. Durable Streaming Code Samples
- 12. Code Sample Javascript Client Replay Extension
- 13. Code Sample Javascript CometD Example
- 14. Code Sample Java Client Replay Extension
- 15. Code Sample Java CometD Example
- 17. Enterprise Messaging – Event Time Window The Event Time Window allows for events to be added in time order to the channel • Events are added through a producer API and retrieved from the channel via the consumer API • The "window" will slide with time, and any events within the window can be replayed • Events are ordered • Replay events from anywhere in event log window
- 18. Event Architecture – Conduit API Conduit API Allows for Producing and Consuming Events • Simple publisher and consumer API
- 19. Event Architecture – Conduit SPIs Conduit SPI Allows a common API for multiple Time Ordered Event Log implementations • Conduit SPI interface allows multiple concrete event log implementations • Kafka, SQL, HBase, In Memory, etc... • Provides a common API across all providers • The SPIs will also allow for composing event logs into different SLAs
- 20. Streaming API v2 How do Conduit and Streaming v2 Interact Runs as an embedded or stand-alone service Client interface via CometD Event processing pipeline Conduit - SPI • Use different data stores • Plug and play • Holds client information • Optimizes event replay • Plus much more!
- 22. App Server Current Streaming API Clients Internal Conduit Push API Kafka Cluster Future
- 23. App Server Kafka Cluster Clients Internal Conduit Push API HBase Future++
- 24. Enterprise Messaging – Roadmap (Safe Harbor J) Winter '16 • Durable Generic Streaming (Limited Pilot) Spring '16 • Durable Generic Streaming GA • Durable Classic Streaming Pilot Summer '16 • Durable Classic Streaming GA • Kafka-Backed Queues • Conduit Producer API public implementations for Apex • Conduit Consumer API public implementations for Apex • Conduit API implementations for Workflow/Process Builder
- 25. What did we Learn? Existing Features that begin to support messaging • Classic Streaming API • Generic Streaming API Information on Durable Streaming Demo of Durable Streaming • Showed an example of Generic Durability • Showed an example of Classic Durability Roadmap for Event Messaging
- 26. Q&A
- 27. Share Your Feedback, and Win a GoPro! 3 Earn a GoPro prize entry for each completed survey Tap the bell to take a survey2Enroll in a session1
- 28. Thank you