Spring Batch - the not so hot cousin that gets the (batch) job done
Download as PPTX, PDF1 like3,790 views
Spring Batch provides functionality for processing data in batches including chunk-oriented processing, transaction management, job persistence, and error handling. It allows defining jobs as a sequence of steps and processing data in chunks within transactions. Listeners can be used to monitor processing. While Spring Batch is not a scheduler itself, it can be used with schedulers like Quartz to run batch jobs on a schedule.
Spring Batch - the not so hot cousin that gets the (batch) job done
- 1. Copyright©SolidBeansAB Mattias Jiderhamn www.solidbeans.com Spring Batch – the not so hot cousin that gets the (batch) job done Mattias Jiderhamn Solid Beans
- 3. Copyright©SolidBeansAB Mattias Jiderhamn www.solidbeans.com Spring Batch provides • Chunk oriented processing • Transaction management • Persistent job state • Pause/Resume/Restart • Retry/Skip • Flow control • Partitioning (parallel processing) • JSR-352 implementation
- 4. Copyright©SolidBeansAB Mattias Jiderhamn www.solidbeans.com Spring Batch is not • A scheduler –Quartz (Open Source) –@EnableScheduling + @Scheduled –Control-M –Autosys –cron
- 5. Copyright©SolidBeansAB Mattias Jiderhamn www.solidbeans.com Concepts Job Step 1 Step NStep 2 …
- 6. Copyright©SolidBeansAB Mattias Jiderhamn www.solidbeans.com Concepts Job JobParameters+ JobInstance= Job todaysJob = new DailyJob(”2018-09-25”) class DailyJob extends Job { ... } JobExecution todaysJob.execute() Think * *
- 7. Copyright©SolidBeansAB Mattias Jiderhamn www.solidbeans.com Concepts Job Step 1 Step N Step 2 JobInstance JobExecution StepExecution* * * * *
- 8. Copyright©SolidBeansAB Mattias Jiderhamn www.solidbeans.com Step Chunk-oriented processing ItemWriterItemReader X Chunk<X> Chunk<Y> ItemProcessor Optional YX
- 9. Copyright©SolidBeansAB Mattias Jiderhamn www.solidbeans.com I/O • Database • Hibernate • JDBC • JPA • MongoDB • Neo4J • Files • Flat or delimited file • XML • Excel (extension) • JMS • Mail • ElasticSearch (extension)
- 10. Copyright©SolidBeansAB Mattias Jiderhamn www.solidbeans.com Business people ♥ Excel
- 11. Copyright©SolidBeansAB Mattias Jiderhamn www.solidbeans.com Chunk-oriented processing ItemReader .read() ItemProcessor .process() ItemWriter .write() Begin step Begin TX Commit TX End step != null && n < commitLimit (chunk size) More items != null
- 13. Copyright©SolidBeansAB Mattias Jiderhamn www.solidbeans.com Error ItemReader .read() ItemProcessor .process() ItemWriter .write() Begin step Begin TX Commit TX End step Rollback TX RuntimeException RuntimeException RuntimeException
- 15. Copyright©SolidBeansAB Mattias Jiderhamn www.solidbeans.com Retryable error ItemReader .read() ItemProcessor .process() ItemWriter .write() Begin step Begin TX Commit TX End step Rollback TX RetryException Begin TX >= retryLimit
- 16. Copyright©SolidBeansAB Mattias Jiderhamn www.solidbeans.com Skippable error in ItemReader ItemReader .read() ItemProcessor .process() ItemWriter .write() Begin step Begin TX Commit TX End step SkippableException >= skipLimit Rollback TX
- 17. Copyright©SolidBeansAB Mattias Jiderhamn www.solidbeans.com Skippable error in ItemProcessor ItemReader .read() ItemProcessor .process() ItemWriter .write() Begin step Begin TX Commit TX End step SkippableException Rollback TX Remove failed item from chunk >= skipLimit Begin TX
- 18. Copyright©SolidBeansAB Mattias Jiderhamn www.solidbeans.com Skippable error in ItemWriter ItemReader .read() ItemProcessor .process() ItemWriter .write() Begin step Begin TX Commit TX End step Rollback TX SkippableException For each item in chunk Begin TX >= skipLimit Iteration complete Skip SkippableException
- 25. Copyright©SolidBeansAB Mattias Jiderhamn www.solidbeans.com More listeners • SkipListener – onSkipInRead() – onSkipInProcess() – onSkipInWrite() JSR-352 listeners • RetryReadListener.onRetryReadException() • RetryProcessListener.onRetryProcessException() • RetryWriteListener.onRetryWriteException()
- 27. Copyright©SolidBeansAB Mattias Jiderhamn www.solidbeans.com Flow control Job Step A Step C Step B FAILED Success
- 28. Copyright©SolidBeansAB Mattias Jiderhamn www.solidbeans.com Gotchas • Transactional ItemReader and skip/retry – readerIsTransactionalQueue() • Listeners and transactions – @Transactional(propagation=Propagation.REQUIRES_NEW) • ExitStatus.equals() vs ExitStatus.compareTo() • Restart vs relaunch – Restart = start the same JobIntance again – Relaunch = new JobInstance • Must not use the same (identifying) job parameters
Editor's Notes
- #2: Copyright © Tieturi Oy