Building a Real-Time Forecasting Engine with Scala and Akka
0 likes728 views
Steven Laan presents ING's real-time forecasting engine built with Scala and Akka, emphasizing its role in the bank’s data-driven strategy since 2014. The system aims to provide customers insights into future transactions through their mobile app while adopting an agile methodology for continuous improvement. Key technologies used include Scala, Akka, and various backend services to ensure scalability and real-time capabilities.
Building a Real-Time Forecasting Engine with Scala and Akka
- 1. Kijk vooruit! Steven Laan Building a real-time forecasting engine with Scala and Akka Lightbend CTO Roundtable 24-01-2017
- 2. Product Owner at ING Analytics Academic background: Artificial Intelligence Working as Dev Engineer since 2014 Steven.Laan@ing.nl https://nl.linkedin.com/in/steven-laan-64222b48 Steven Laan
- 3. Agenda 3 Why ING is a data-driven enterprise1. Strategy How RAF is build3. Architecture and Technology How we implement the tools4. Way of Working The coolest feature in ING’s mobile app2. Use case: RAF Lessons learned and the road ahead5. Conclusions
- 5. ING’s Think Forward strategy was launched in March 2014 5
- 6. We are committed to helping our customers stay a step ahead by making their data work harder for them 6 Compliant Omnichannel Predictive Secure & reliable Personal
- 7. How do we want to use data at ING? 7 the right message at the right time in the right channel to the right customer Message Time Channel What does ‘right’ mean? • We want to be as relevant as possible • ‘right’ = interesting enough to react positively (i.e. to click)
- 9. What? • Provide customers insight in their future transactions How? • Mobile app feature • Start small: MVP • Agile: Keep improving Real-time Account Forecasting 9
- 10. Mobile interface 10 Main screen Transaction detailsTerms & conditions Info page
- 13. • Called by Mobile Platform (REST) • Calls backend services (multiple protocols) • Returns future transactions (JSON) • Certain transactions • Predicted transactions Forecasting API 13
- 14. • Scala • Akka • Git • Sbt • Nolio • Jenkins • Graphite / Grafana • Spark Used Technologies 14
- 15. • Scala • Interop with JVM • Functional • Akka • Scalability • Domain Why Scala / Akka 15
- 16. Flow of a request 16 Spray HTTP Service RequestActor FutureActors HistoryActors Historic transactions services Planned transactions services ForecastActor Forecasting Library
- 17. Flow of a request 17 Spray HTTP Service RequestActor FutureActors HistoryActors Historic transactions services Planned transactions services ForecastActor Forecasting Library
- 18. Flow of a request 18 Spray HTTP Service RequestActor FutureActors HistoryActors Historic transactions services Planned transactions services ForecastActor Forecasting Library
- 19. • RequestActor per request • Scalable • Asynchronous • Dedicated service actors • Configurability • Error handling • Separate forecasting library • Separation of concern • Reusable • Parallel development Actor System Architecture 19
- 20. Dealing with slower processes 20 Real time transactions service Slow (legacy) system Forecasting Library Forecasting APIFront End DB Preprocessing
- 21. • Move towards streaming solutions • Fast data • Kafka / Flume / Flink • Improve forecasting library Road ahead 21
- 22. HOW: Way of working
- 23. ING is an Experimental Enterprise Source: Edd Dumbill, Forbes, 2014
- 24. Squad Tribes, squads, chapters, guilds, … SquadSquad Chapter Guild DS DA OpsOps CJE DS DA DS CJE DS Ops DS CJE Dev DevDevDevDev
- 25. A fun place to work! 25
- 26. 26 Follow us to stay a step ahead ING.com YouTube.com/ING SlideShare.net/ING@ING_News LinkedIn.com/company/ING Flickr.com/INGGroupFacebook.com/ING