Seamless Guest Experience with Kafka Streams
Download as PPTX, PDF2 likes418 views
The document discusses how to enhance the guest experience for a global cruise vacation company using Kafka Streams and event-driven architecture. It covers the various stages of a guest's journey, challenges in achieving seamless integration across ships and shores, and the importance of real-time data synchronization. Additionally, it highlights the technical implementation of Kafka for managing guest information and the future steps for improving data processing and system reliability.
Seamless Guest Experience with Kafka Streams
- 1. Seamless Guest Experience with Kafka Streams Ram IndukuriHimani Arora Suresh Mulukaladu
- 2. About Knoldus Technology consulting firm with focus on digital transformation Functional. Reactive. Cloud Native
- 3. We are global cruise vacation company that controls and operates four global brands. Royal Caribbean International, Azamara club, Celebrity cruises and Silversea Cruises.We are also 50% joint venture owner of German brand TUI Cruises and 49% owner of the spanish brand Pullmantur Cruceros. About Royal Caribbean International
- 4. Royal Caribbean International - Facts ● World’s second largest global cruise line ● Represents more than 20% of global cruisers ● 9.5 billion in revenues (2018) ● Serving more than 5 Million guests every year around the globe ● 60 ships in service and another 14 ships on the way ● Oasis class(World’s largest) ships carry close to 6000 guests and 2000 crew
- 5. Guest Journey - touchpoints Discover Book Cruise Excursions Check In Board Ship Cruise Take Excursions Debark Create Memories ● Research vacation options ● Book the cruise & air reservations ● Plan and Book activities on the cruise ● Complete online check-in and get boarding pass ● Pre cruise activities ● Get to the embarkation port ● Check-in to the cruise and board ● Get to your state room ● Book dinner reservations and activities ● Enjoy the cruise and activities ● Excursions at ports of call ● Debarkation ● Post cruise activities ● Fly back ● Get home
- 6. Excalibur
- 7. Digital Vision Discover Book Cruise Pre Cruise Planning Check In Board Ship Cruise Take Excursions Debark Create Memories Sensor Wearable TV API Layer Device Layer One App
- 8. Architectural Challenges 1 How to Standardize Footprints on all ships and shore ? 3 How do guests have seamless experience on ship(s) and shore if we rely on batch process ? 2 How do we build on what we have today ? Same infrastructure on ship/shore API and Microservices Event driven Architecture with real time synchronization
- 9. Ship-Shore Highway - Event enabling Fleet System of Record System of Record Ship Shore Highway Event enablement, transmission, processing & Analytics across fleet
- 10. SoR Ship-Shore Pattern API Gateway Micro Service Data Store System of Record Data Store System of Record Micro Service
- 11. Change data Capture - Using Kafka Connect Web Call Center Travel Agent SQL Mirror Kafka Connect Kafka Shore Reservation SOR AWS Data Transformer Kafka Ship
- 12. Broadcasting events - Using Streams Web Call Center Travel Agent IIDR Business Events & Change Data Reservation SOR AWS Kafka Streams Kafka
- 13. Generating Business Events by Combining Topics Booking ID(pk) Guest ID Guest ID(pk) Name Address State Store State Store JOIN Guest ID Booking ID Name Address
- 14. Generating Business Events by Combining Topics StreamsBuilder builder = new StreamsBuilder(); KStream<String, BookingDetails> guestBookings = builder.stream(guestBookingTopic) .selectKey((key, value) -> value.getGuestId()); KStream<String, GuestDetails> guestDetails = builder.stream(guestDetailsTopic); KStream<String, GuestBookingDetails> guestBookingAndDetails = guestBookings .join(guestDetails, (guestBooking, guestDetail) -> /* business event message */, JoinWindows.of(Duration.ofHours(10));
- 15. Guest Bookings Stream Guest Details Stream Transparent Handling of Late Arrival Records SLIDING WINDOW JoinWindows.of(Duration.ofHours(10)) Late Arriving Record Event 1Event 2 Event 1Event 2 Event 12 Event 13 Event 13 Event 14 Event 14 Event 15
- 16. Transparent Handling of Late Arrival Records Guest Bookings Stream Guest Details Stream SLIDING WINDOW Grace Period set to: JoinWindows.of(Duration.ofHours(10)).grace(Duration.ofHours(24))) Event 1Event 2 Event 1Event 2 Event 12 Event 13 Event 13 Event 14 Event 14 Event 15
- 17. Transparent Handling of Late Arrival Records StreamsBuilder builder = new StreamsBuilder(); KStream<String, BookingDetails> guestBookings = builder.stream(guestBookingTopic) .selectKey((key, value) -> value.getGuestId()); KStream<String, GuestDetails> guestDetails = builder.stream(guestDetailsTopic); KStream<String, GuestBookingDetails> guestBookingAndDetails = guestBookings .join(guestDetails, (guestBooking, guestDetail) -> /* business event message */, JoinWindows.of(Duration.ofHours(10) .grace(Duration.ofHours(24)) );
- 18. Consecutive Joins on Multiple Topics Booking ID Guest ID Guest ID Name Address State Store State Store JOIN Guest ID Purchase ID Intermediate State Store
- 19. Consecutive Joins on Multiple Topics Booking ID Guest ID Guest ID Name Address State Store State Store JOIN Guest ID Purchase ID Intermedia te Store State Store JOIN Guest ID Booking ID Purchase ID Name Address
- 20. Transparent Handling of Late Arrival Records StreamsBuilder builder = new StreamsBuilder(); KStream<String, BookingDetails> guestBookings = builder.stream(guestBookingTopic) .selectKey((key, value) -> value.getGuestId()); KStream<String, PurchaseDetails> guestPurchases = builder.stream(guestPurchasesTopic); .selectKey((key, value) -> value.getGuestId()); KStream<String, GuestDetails> guestDetails = builder.stream(guestDetailsTopic); KStream<String, GuestBookingDetails> guestBookingAndDetails = guestBookings .join(guestDetails, (guestBooking, guestDetail) -> /* business event message with booking id and guest details */, JoinWindows.of(Duration.ofHours(10) .join(guestPurchases, (guestBookingDetail,guestPurchase) -> /* business event message with guest details, booking and purchase details*/, JoinWindows.of(Duration.ofHours(10)));
- 21. Ensuring Streaming reliability - Graphite/Grafana Kafka Stream App JMX Graphite Grafana ● Consumer Lag to detect surges ● Idle time threshold to predict Thread health ● Bytes Received in last window indicating Kafka broker issues Lack of activity indicate either broker failure or app failure
- 22. Ensuring Data Quality - Apache Spark Apache Spark Apache Spark Minio incremental Mirroring 1 2 3 4 5
- 23. Next Steps - Interactive Queries Instance 1 Instance 2 Instance 3 GUEST API Local Kafka State Store Local Kafka State Store Local Kafka State Store host1:4047 host2:5047 host3:4460 Exposing RPC endpoint and information about its local state store Kafka Cluster
- 24. Summary ● Real time guest information synchronization across ship and shore was possible with kafka, connect and replicators ● Kafka streams and IIDR helped to reduce the latency in processing the CDC ● Isolate critical connectors into their own connect cluster to reduce risk ● Order guarantee is easier to achieve with streams than connectors. ● Invest in domain driven design ● Include legacy dependencies in critical path for planning feature delivery.
- 25. More Information Ram Indukuri ram@knoldus.com Himani Arora himani@knoldus.com Suresh Mulukuladu smulukuladu@rccl.com For more information Blog: https://blog.knoldus.com/join-semantics-kafka-streams/ Git Samples: https://github.com/knoldus/