Spring Batch - Lessons Learned out of a real life banking system.
10 likes7,165 views
This document discusses lessons learned from implementing Spring Batch in a real-life banking system. It covers topics like sourcing data, job control, partitioning for performance, and error handling. For sourcing, Spring Batch allows loading data into staging tables and partitioning or reprocessing individual records. For job control, it discusses triggering jobs on a schedule or events and persisting launch requests. Partitioning can improve performance through multi-threading, parallel steps, and distributing work across nodes. Error handling addresses restartability and options for skipping or retrying individual items.
![2014 © Trivadis
Partitioning detail
June 24th, 2014
Lessons Learned out of a real life banking system
29
StepScopes
Master
Slave [1]
Slave [2]
Slave [3]
Slave [4]
Slave [0]
Execution
Context [1]
Execution
Context [2]
Execution
Context [3]
Execution
Context [4]
Execution
Context [0]
rowId % 5
Partitioning
PartnerId
HASH %5](https://image.slidesharecdn.com/jugsspring-batch-bern-140624160229-phpapp02/85/Spring-Batch-Lessons-Learned-out-of-a-real-life-banking-system-29-320.jpg)
![2014 © Trivadis
Monitoring – logging to a database
June 24th, 2014
Lessons Learned out of a real life banking system
40
Step
Read
Process
Write
LoggerService
@Transactional(propagation =
Propagation.REQUIRES_NEW)
2 calls on error
Listener
Before
OnSkip
[WARN]
OnError
[ERROR]
After
[INFO]
Monitoring &
Tracing](https://image.slidesharecdn.com/jugsspring-batch-bern-140624160229-phpapp02/85/Spring-Batch-Lessons-Learned-out-of-a-real-life-banking-system-40-320.jpg)
Spring Batch - Lessons Learned out of a real life banking system.
- 1. 2014 © Trivadis BASEL BERN BRUGG LAUSANNE ZUERICH DUESSELDORF FRANKFURT A.M. FREIBURG I.BR. HAMBURG MUNICH STUTTGART VIENNA 2014 © Trivadis Spring Batch Lessons Learned out of a real life banking system. Michael Beer Senior Consultant Raffael Schmid Consultant June 24th, 2014 Lessons Learned out of a real life banking system 1
- 2. 2014 © Trivadis About us Michael Beer Senior Consultant June 24th, 2014 Lessons Learned out of a real life banking system 2 ! Trivadis since 2001 ! design and development of web based applications ! part of the Trivadis APM team ! Trivadis since 2010 ! design and development of web based applications ! interested in performance related things on the JVM Raffael Schmid Consultant
- 3. 2014 © Trivadis AGENDA June 24th, 2014 Lessons Learned out of a real life banking system 3 Initial position Requirements and Topics Lessons learned Conclusion
- 4. 2014 © Trivadis Initial position June 24th, 2014 Lessons Learned out of a real life banking system 4 Initial position Requirements and Topics Lessons learned Conclusion
- 5. 2014 © Trivadis common usage Initial and target system context ! Automatic load out of thirdparty systems ! multiple times per day ! export / import as CSV file ! Manual load by user ! at irregular time intervals ! Migration load due to thirdparty system decommissioning ! run once ! different volumes June 24th, 2014 Lessons Learned out of a real life banking system 5 Web Interface Rule Engine Spring Batch Thirdparty system Thirdparty system
- 6. 2014 © Trivadis Why we used Spring Batch? ! It was already part of the technology stack within the customers environment - online application already based on Spring Framework ! Exceptional permit was needed for Spring Integration because it was shipped with Spring Batch Admin ! There were no real (free) alternatives at that time (or at least we didn‘t know any) June 24th, 2014 Lessons Learned out of a real life banking system 6
- 7. 2014 © Trivadis Terminology of Spring Batch jobs June 24th, 2014 Lessons Learned out of a real life banking system 7 Job Job Instance Job Execution Step Step Execution * * * * * Job Execution Context Step Execution Context * Job Parameter Tasklet
- 8. 2014 © Trivadis Requirements and Topics June 24th, 2014 Lessons Learned out of a real life banking system 8 Initial position Requirements and Topics Lessons learned Conclusion
- 9. 2014 © Trivadis Requirement: Performance ! parallelized on multiple threads ! order of execution matters (in case of partitioning) ! number of threads can be changed before or after job execution June 24th, 2014 Lessons Learned out of a real life banking system 9 Control load Deactivate jobs Inter-job dependencies Trigger jobs Reprocess failed items Gather detailed job information Summary mailPerformance
- 10. 2014 © Trivadis Requirement: Reprocess failed items ! we do not stop job in case of an error ! rerunning a job processes failed items (only) ! process state of item therefore needs to be maintained June 24th, 2014 Lessons Learned out of a real life banking system 10 Control load Deactivate jobs Inter-job dependencies Trigger jobs Reprocess failed items Gather detailed job information Summary mailPerformance
- 11. 2014 © Trivadis Requirement: Summary mail ! write a detailed execution log that contains ! Exceptions ! messages out of the Rule Engine ! aggregate to summary ! Number of errors, warnings, etc. ! worst status level ! send summary to submitter and operator June 24th, 2014 Lessons Learned out of a real life banking system 11 Control load Deactivate jobs Inter-job dependencies Trigger jobs Reprocess failed items Gather detailed job information Summary mailPerformance
- 12. 2014 © Trivadis Requirement: Gather detailed job information ! rerun failed single item in trace mode ! collect diagnostic information out of the Rule Engine June 24th, 2014 Lessons Learned out of a real life banking system 12 Control load Deactivate jobs Inter-job dependencies Trigger jobs Reprocess failed items Gather detailed job information Summary mailPerformance
- 13. 2014 © Trivadis Requirement: Trigger jobs ! periodically ! either fixed delay or fixed rate ! on event ! e.g. new data arrived in database June 24th, 2014 Lessons Learned out of a real life banking system 13 Control load Deactivate jobs Inter-job dependencies Trigger jobs Reprocess failed items Gather detailed job information Summary mailPerformance
- 14. 2014 © Trivadis Requirement: Control load ! prevent too many jobs in parallel ! conditions might prevent job execution ! requests should survive system shutdown June 24th, 2014 Lessons Learned out of a real life banking system 14 Control load Deactivate jobs Inter-job dependencies Trigger jobs Reprocess failed items Gather detailed job information Summary mailPerformance
- 15. 2014 © Trivadis Requirement: Deactivate jobs ! deactivate job execution ! set job execution on hold June 24th, 2014 Lessons Learned out of a real life banking system 15 Control load Deactivate jobs Inter-job dependencies Trigger jobs Reprocess failed items Gather detailed job information Summary mailPerformance
- 16. 2014 © Trivadis Requirement: Inter-job dependencies ! finished jobs might trigger dependencies June 24th, 2014 Lessons Learned out of a real life banking system 16 Control load Deactivate jobs Inter-job dependencies Trigger jobs Reprocess failed items Gather detailed job information Summary mailPerformance
- 17. 2014 © Trivadis Requirements grouped into five different topics June 24th, 2014 Lessons Learned out of a real life banking system 17 Control load Deactivate jobs Inter-job dependencies Trigger jobs Reprocess failed items Gather detailed job information Summary mailPerformance Sourcing Job Control Error Handling Monitoring & TracingPartitioning
- 18. 2014 © Trivadis Lessons learned June 24th, 2014 Lessons Learned out of a real life banking system 18 Initial position Requirements and Topics Lessons learned Conclusion
- 19. 2014 © Trivadis Topics we will cover June 24th, 2014 Lessons Learned out of a real life banking system 19 Error Handling Monitoring & Tracing Job ControlSourcingPartitioning depends on depends on depends on firing trigger supervising
- 20. 2014 © Trivadis Sourcing June 24th, 2014 Lessons Learned out of a real life banking system 20 Error Handling Monitoring & Tracing Job ControlSourcingPartitioning depends on depends on depends on firing trigger supervising
- 21. 2014 © Trivadis Load data into staging table June 24th, 2014 Lessons Learned out of a real life banking system 21 CAT_ID | PERSON_ID | AMOUNT | CURRENCY | VALID_FORM | LAST_CHANGE| 310 | 2908 | 200000 | USD | 03.12.2013 | 02.02.2013 | 310 | 1608 | 100000 | CHF | 03.12.2013 | 02.01.2013 | 311 | 1608 | 100000 | CHF | 03.12.2013 | 02.01.2013 | PARTITION LINE TYPE STATE FIELD01 FIELD02 FIELD03 FIELD04 PT_000122 171 D 02 441 1804 400000 EUR PT_000122 172 D 02 441 1002 221000 EUR PT_000123 1 H 01 CAT_ID PERSON_ID AMOUNT CURRENCY PT_000123 2 D 01 310 2908 200000 USD PT_000123 3 D 01 310 1608 100000 CHF PT_000123 4 D 01 311 1608 100000 CHF Sourcing
- 22. 2014 © Trivadis Allows partitioning, single record execution ! Partitioning out of the box June 24th, 2014 Lessons Learned out of a real life banking system 22 PARTITION LINE TYPE STATE FIELD01 FIELD02 FIELD03 FIELD04 PT_000123 1 H 01 CAT_ID PERSON_ID AMOUNT CURRENCY PT_000123 2 D 01 310 2908 200000 USD PT_000123 3 D 01 310 1410 100000 CHF SELECT * FROM LOAD WHERE PARTITION =‘PT_000123’ AND LINE BETWEEN 0 AND 99; SELECT * FROM LOAD WHERE PARTITION =‘PT_000123’ AND STATE = ‘01’ ! Reexcution of failed records ! Execution of single record SELECT * FROM LOAD WHERE PARTITION =‘PT_000123’ AND LINE = 8 Sourcing
- 23. 2014 © Trivadis Job Control June 24th, 2014 Lessons Learned out of a real life banking system 23 Error Handling Monitoring & Tracing Job ControlSourcingPartitioning depends on depends on depends on firing trigger supervising
- 24. 2014 © Trivadis Trigger Simplest way of triggering a job June 24th, 2014 Lessons Learned out of a real life banking system 24 0 0/5 * * * ? Spring Batch (SimpleJobLauncher) job-requests Job Control <int:inbound-channel-adapter method="create" channel="job-requests" ref="requestFactory"> <int:poller cron="0 0/5 * * * ?" /> </int:inbound-channel-adapter> Request Factory <int:inbound-channel-adapter method="create" channel="job-requests" ref="requestFactory"> <int:poller fixed-rate="300000"/> </int:inbound-channel-adapter> ! Or even simpler (PeriodicTrigger) ! Define polling channel adapter (CronTrigger)
- 25. 2014 © Trivadis Persist job launch requests into database June 24th, 2014 Lessons Learned out of a real life banking system 25 job launch request job- requests Data import job (database) Periodic jobs 0 0/5 * * * ? *JobQueueMessageHandlerjob queue *JobLaunchingMessageHandler update status save job dependencies Guard verification *JobLauncher Job Control
- 26. 2014 © Trivadis Partitioning June 24th, 2014 Lessons Learned out of a real life banking system 26 Error Handling Monitoring & Tracing Job ControlSourcingPartitioning depends on depends on depends on firing trigger supervising
- 27. 2014 © Trivadis Performance June 24th, 2014 Lessons Learned out of a real life banking system 27 Type Local/Remote Description Multi-threaded Step Local A step is multithreaded (TaskExecutor) Parallel Steps Local Executes steps in parallel using multithreading Partitioning Step Local Remote Partitions data and splits up processing Remote Chunking Remote Distributed chunk processing to remote nodes Partitioning
- 28. 2014 © Trivadis Partitioning overview June 24th, 2014 Lessons Learned out of a real life banking system 28 Job Master Step Step Slave Slave Slave Slave Slave Partitioning
- 29. 2014 © Trivadis Partitioning detail June 24th, 2014 Lessons Learned out of a real life banking system 29 StepScopes Master Slave [1] Slave [2] Slave [3] Slave [4] Slave [0] Execution Context [1] Execution Context [2] Execution Context [3] Execution Context [4] Execution Context [0] rowId % 5 Partitioning PartnerId HASH %5
- 30. 2014 © Trivadis Characteristics of data There might be dependencies between records. The order of execution matters at some point. June 24th, 2014 Lessons Learned out of a real life banking system 30 CAT_ID | PERSON_ID | AMOUNT | CURRENCY | VALID_FORM | LAST_CHANGE| 310 | 2908 | 200000 | USD | 03.12.2013 | 02.02.2013 | 310 | 1608 | 100000 | CHF | 03.12.2013 | 02.01.2013 | 311 | 1608 | 100000 | CHF | 03.12.2013 | 02.01.2013 | 312 | 1608 | 100000 | CHF | 03.12.2013 | 02.01.2013 | 313 | 1608 | 100000 | CHF | 03.12.2013 | 02.01.2013 | 310 | 1410 | 100000 | CHF | 03.12.2013 | 02.01.2013 | 390 | 1108 | 100000 | CHF | 03.12.2013 | 02.01.2013 | 310 | 2908 | 500000 | USD | 04.12.2013 | 03.02.2013 | Partitioning Insert Update Read Write Process / Write
- 31. 2014 © Trivadis Partitioning detail – Spring Batch Admin June 24th, 2014 Lessons Learned out of a real life banking system 31 Partitioning
- 32. 2014 © Trivadis Performance – Reader / Processor / Writer June 24th, 2014 Lessons Learned out of a real life banking system 32 Open Transaction Begin Step Commit Transaction ItemReader Item read() ItemProcessor Process(Item) ItemWriter Write(List<Item>) Finish Step Rollback Transaction RuntimeException RuntimeExceptionRuntimeException Partitioning For each item For each item
- 33. 2014 © Trivadis Performance June 24th, 2014 Lessons Learned out of a real life banking system 33 No error Error on each item • ~ 22 sec • 0 Rollbacks • ~ 50 sec • 1‘100 Rollbacks • 5‘500 Filter • 1‘000 Write Skips Partitioning
- 34. 2014 © Trivadis Error Handling June 24th, 2014 Lessons Learned out of a real life banking system 34 Error Handling Monitoring & Tracing Job ControlSourcingPartitioning depends on depends on depends on firing trigger supervising
- 35. 2014 © Trivadis Restartability June 24th, 2014 Lessons Learned out of a real life banking system 35 Job 1 Instance 1 Execution 1 Job 1 Instance 1 Execution 2 read new/error read new/error Error Handling regular start restart
- 36. 2014 © Trivadis Restartability configuration June 24th, 2014 Lessons Learned out of a real life banking system 36 Error Handling Job Step initialization Step process Step finalize initial (failed) execution Job Step initialization Step process Step finalize restarted execution Execution Context cleanup serialized context
- 37. 2014 © Trivadis Item based error handling June 24th, 2014 Lessons Learned out of a real life banking system 37 Feature When? What? Where? Skip For nonfatal exceptions Keeps processing for an incorrect item Chunk-oriented step Retry For transient exceptions Makes new attemps on an operation Chunk-oriented step, application code Error Handling
- 38. 2014 © Trivadis Monitoring June 24th, 2014 Lessons Learned out of a real life banking system 38 Error Handling Monitoring & Tracing Job ControlSourcingPartitioning depends on depends on depends on firing trigger supervising
- 39. 2014 © Trivadis Monitoring – Spring Batch Admin June 24th, 2014 Lessons Learned out of a real life banking system 39 Monitoring & Tracing
- 40. 2014 © Trivadis Monitoring – logging to a database June 24th, 2014 Lessons Learned out of a real life banking system 40 Step Read Process Write LoggerService @Transactional(propagation = Propagation.REQUIRES_NEW) 2 calls on error Listener Before OnSkip [WARN] OnError [ERROR] After [INFO] Monitoring & Tracing
- 41. 2014 © Trivadis Monitoring - Mail June 24th, 2014 Lessons Learned out of a real life banking system 41 Step ReadProcessWrite Job FAILED Error MailOK Mail Monitoring & Tracing
- 42. 2014 © Trivadis Tracing – Explain plan June 24th, 2014 Lessons Learned out of a real life banking system 42 Step ReadProcessWrite Partitioner 1 Record Reader 1 Record Processor Debug mode Step … Writer does nothing Operator explicit job start Monitoring & Tracing
- 43. 2014 © Trivadis Monitoring / Tracing summary June 24th, 2014 Lessons Learned out of a real life banking system 43 Mail OK/ERROR Mail ERROR UI Log Spring Batch Admin UI Explain plan User Operator Monitoring & Tracing
- 44. 2014 © Trivadis Conclusion June 24th, 2014 Lessons Learned out of a real life banking system 44 Initial position Requirements and Topics Lessons learned Conclusion
- 45. 2014 © Trivadis Things to think about – Framework evaluation ! Testability ! Spring Batch provides lots of helpers to write JUnit tests ! Monitoring ! Spring Batch Admin ! Scalability ! Extensibility ! Easy to write extentions (Job Launch, Guards) ! Out of the box components ! All 155 SE tests in the JSR-352 TCK passed (Spring Batch 3.0) June 24th, 2014 Lessons Learned out of a real life banking system 45
- 46. 2014 © Trivadis Things to think about – Monolitic vs. Modular Modular (1 process per job)Monolithic (single process) June 24th, 2014 Lessons Learned out of a real life banking system 46 Spring Batch Admin, Custom Management UI Job A Job B Job C Spring Batch Admin, Custom Management UI Job A Job B Job C ! Integrate batch jobs into web application (use Spring Batch Admin) ! Start Jobs individually on command line (use external scheduler) ! Use Spring XD
- 47. 2014 © Trivadis Things to think about – Caching June 24th, 2014 Lessons Learned out of a real life banking system 47 Job Step read/write Online Read Write Step read/write Cache Cache
- 48. 2014 © Trivadis Questions and answers ... 2013 © Trivadis BASEL BERN BRUGG LAUSANNE ZUERICH DUESSELDORF FRANKFURT A.M. FREIBURG I.BR. HAMBURG MUNICH STUTTGART VIENNA Michael Beer michael.beer@trivadis.com +41 58 459 51 90 Raffael Schmid raffael.schmid@trivadis.com +41 58 459 52 34 June 24th, 2014 Lessons Learned out of a real life banking system 48