Table of Contents

Copyright

Brief Table of Contents

Table of Contents

Foreword

Preface

Acknowledgments

About this Book

About the Cover Illustration

1. Background

Chapter 1. Introducing Spring Batch

1.1. What are batch applications?

1.2. Meet Spring Batch

1.2.1. Robustness and reliability

1.2.2. Scaling strategies

1.3. Introducing the case study

1.3.1. The online store application

1.3.2. Why build an online store with batch jobs?

1.3.3. Why use batch processes?

1.3.4. The import product use case

1.4. Reading and writing the product data

1.4.1. Anatomy of the read-write step

1.4.2. Reading a flat file

1.4.3. Implementing a database item writer

1.4.4. Configuring a database item writer

1.4.5. Configuring the read-write step

1.5. Decompressing the input file with a tasklet

1.5.1. Why compress the file?

1.5.2. Implementing the decompression tasklet

1.5.3. Configuring the tasklet

1.6. Testing the batch process

1.6.1. Setting up the test infrastructure

1.6.2. Leveraging SpEL for configuration

1.6.3. Writing the test for the job

1.7. Skipping incorrect lines instead of failing

1.8. Summary

Chapter 2. Spring Batch concepts

2.1. The batch domain language

2.1.1. Why use a domain language?

2.1.2. Main components of the domain language

2.1.3. How Spring Batch interacts with the outside world

2.2. The Spring Batch infrastructure

2.2.1. Launching jobs and storing job metadata

2.2.2. Configuring the Spring Batch infrastructure in a database

2.3. Anatomy of a job

2.3.1. Modeling jobs with steps

2.3.2. Running job instances and job executions

2.4. Summary

2. Core Spring Batch

Chapter 3. Batch configuration

3.1. The Spring Batch XML vocabulary

3.1.1. Using the Spring Batch XML namespace

3.1.2. Spring Batch XML features

3.2. Configuring jobs and steps

3.2.1. Job entities hierarchy

3.2.2. Configuring jobs

3.2.3. Configuring steps

3.2.4. Configuring tasklets and chunks

3.2.5. Configuring transactions

3.3. Configuring the job repository

3.3.1. Choosing a job repository

3.3.2. Specifying job repository parameters

3.4. Advanced configuration topics

3.4.1. Using step scope

3.4.2. Leveraging SpEL

3.4.3. Using listeners to provide additional processing

3.4.4. Configuration inheritance

3.5. Summary

Chapter 4. Running batch jobs

4.1. Launching concepts

4.1.1. Introducing the Spring Batch launcher API

4.1.2. Synchronous vs. asynchronous launches

4.1.3. Overview of launching solutions

4.2. Launching from the command line

4.2.1. Using Spring Batch’s command-line job runner

4.3. Job schedulers

4.3.1. Using cron

4.3.2. Using the Spring scheduler

4.4. Launching from a web application

4.4.1. Embedding Spring Batch in a web application

4.4.2. Launching a job with an HTTP request

4.5. Stopping jobs gracefully

4.5.1. Stopping a job for the operator

4.5.2. Stopping a job for the application developer

4.6. Summary

Chapter 5. Reading data

5.1. Data reading concepts

5.2. Reading flat files

5.2.1. Configuring the FlatFileItemReader class

5.2.2. Introducing the DefaultLineMapper class

5.2.3. Using the DefaultLineMapper class

5.2.4. Extracting character-separated fields

5.2.5. Creating objects from fields

5.2.6. Reading JSON

5.2.7. Multiline records

5.2.8. Reading heterogonous records

5.3. Reading XML files

5.4. Reading file sets

5.5. Reading from relational databases

5.5.1. Using JDBC item readers

5.5.2. Using ORM item readers

5.6. Using other input sources

5.6.1. Services as input

5.6.2. Reading from JMS

5.7. Implementing custom readers

5.8. Summary

Chapter 6. Writing data

6.1. Data-writing concepts

6.2. Writing files

6.2.1. Writing flat files

6.2.2. Writing XML files

6.2.3. Writing file sets

6.3. Writing to databases

6.3.1. Writing with JDBC

6.3.2. Writing with ORM

6.4. Adapting existing services for reuse

6.5. Writing to JMS

6.6. Sending email messages

6.7. Implementing custom item writers

6.8. Advanced writing techniques

6.8.1. Composing item writers

6.8.2. Routing items to specific item writers

6.9. Summary

Chapter 7. Processing data

7.1. Processing items

7.1.1. Processing items in a chunk-oriented step

7.1.2. Use cases for item processing

7.1.3. Configuring an item processor

7.1.4. Item processor implementations

7.2. Transforming items

7.2.1. Changing the state of read items

7.2.2. Producing new objects from read items

7.2.3. Implementing the driving query pattern with an item processor

7.3. Filtering and validating items

7.3.1. Filtering in the item-processing phase

7.3.2. Implementing a filtering item processor

7.3.3. Validating items

7.4. Chaining item processors

7.5. Summary

Chapter 8. Implementing bulletproof jobs

8.1. What is a bulletproof job?

8.1.1. What makes a job bulletproof?

8.1.2. Designing a bulletproof job

8.1.3. Techniques for bulletproofing jobs

8.1.4. Skip, retry, and restart in action

8.2. Skipping instead of failing

8.2.1. Configuring exceptions to be skipped

8.2.2. Configuring a SkipPolicy for complete control

8.2.3. Listening and logging skipped items

8.3. Retrying on error

8.3.1. Configuring retryable exceptions

8.3.2. Controlling retry with a retry policy

8.3.3. Listening to retries

8.3.4. Retrying in application code with the RetryTemplate

8.3.5. Retrying transparently with the RetryTemplate and AOP

8.4. Restart on error

8.4.1. How to enable restart between job executions

8.4.2. No restart please!

8.4.3. Whether or not to restart already completed steps

8.4.4. Limiting the number of restarts

8.4.5. Restarting in the middle of a chunk-oriented step

8.5. Summary

Chapter 9. Transaction management

9.1. A transaction primer

9.2. Transaction management in Spring Batch components

9.2.1. Transaction management in tasklets

9.2.2. Transaction management in chunk-oriented steps

9.2.3. Transaction management in listeners

9.3. Transaction configuration

9.3.1. Transaction attributes

9.3.2. Common pitfalls with declarative transactions

9.3.3. Transactional reader and processor

9.3.4. To roll back or not to roll back

9.4. Transaction management patterns

9.4.1. Transactions spanning multiple resources: global transactions

9.4.2. The shared resource transaction pattern

9.4.3. The best effort pattern with JMS

9.4.4. Handling duplicate messages with manual detection

9.4.5. Handling duplicate messages with idempotency

9.5. Summary

3. Advanced Spring Batch

Chapter 10. Controlling execution

10.1. A complex flow in the online store application

10.2. Driving the flow of a job

10.2.1. Choosing one path over another

10.2.2. Batch status vs. exit status: what’s the deal?

10.2.3. Choosing the exit status

10.3. Sharing data between steps

10.3.1. Using the execution context to share data

10.3.2. Sharing data using Spring holder beans

10.4. Externalizing flow definitions

10.5. Stopping a job execution

10.6. Summary

Chapter 11. Enterprise integration

11.1. What is enterprise integration?

11.1.1. Enterprise integration challenges

11.1.2. Styles of enterprise integration

11.2. Spring Batch and enterprise integration

11.2.1. An enterprise integration use case

11.3. Spring Integration, a toolbox for enterprise integration

11.3.1. The Spring Integration project

11.3.2. Combining Spring Integration and Spring Batch

11.3.3. Spring Integration quick-start: launching a Spring Batch job

11.4. RESTful job submission with Spring MVC

11.4.1. Deploying Spring in a web application

11.4.2. Writing the REST web controller

11.4.3. Writing the import file in a directory with Spring Integration

11.5. Triggering jobs from file system events

11.5.1. Scanning the input directory with Spring Integration

11.5.2. Converting a file into a job launch request

11.5.3. Implementing the import job

11.6. RESTful job monitoring with Spring MVC

11.6.1. Getting access to the job metadata in the repository

11.6.2. Communicating job status from a web controller

11.7. Summary

Chapter 12. Monitoring jobs

12.1. Introducing monitoring

12.1.1. Monitoring overview

12.1.2. Batch jobs and monitoring

12.2. Accessing batch execution data

12.2.1. Job database schema

12.2.2. Accessing data from the job repository

12.3. Monitoring with listeners

12.3.1. Implementing a monitoring listener

12.3.2. Notifying using emails

12.3.3. Notifying using Spring messaging

12.4. Web monitoring with Spring Batch Admin

12.4.1. Feature overview

12.4.2. Detecting problems encountered during batch executions

12.5. Monitoring with JMX

12.5.1. Configuring JMX for Spring Batch

12.5.2. Monitoring with JMX consoles

12.6. Summary

Chapter 13. Scaling and parallel processing

13.1. Scaling concepts

13.1.1. Enhancing performance by scaling

13.1.2. The Spring Batch scaling model

13.2. Multithreaded steps

13.2.1. Configuring a multithreaded step

13.2.2. Multithreading issues

13.3. Parallelizing processing (single machine)

13.3.1. Configuring parallel steps

13.4. Remote chunking (multiple machines)

13.4.1. What is remote chunking?

13.4.2. Remote chunking with Spring Integration

13.5. Fine-grained scaling with partitioning

13.5.1. Configuring partitioning

13.5.2. The partitioning SPI

13.6. Comparing patterns

13.7. Summary

Chapter 14. Testing batch applications

14.1. The what and why of testing

14.1.1. What is testing?

14.1.2. Different types of testing

14.1.3. Why test?

14.2. Unit testing

14.2.1. Introducing JUnit

14.2.2. Using mock objects with Mockito

14.2.3. Mocking Spring Batch domain objects

14.3. Integration testing

14.3.1. Introducing the Spring TestContext Framework

14.3.2. Using the Spring Batch StepScopeTestExecutionListener

14.4. Functional testing

14.4.1. Introducing JobLauncherTestUtils

14.4.2. Testing a step

14.4.3. Testing a job

14.5. Summary

Appendix A. Setting up the development environment

A.1. Apache Maven 3

A.1.1. Installation

A.1.2. Understanding Maven

A.1.3. Creating a blank project

A.1.4. Adding Spring Batch dependencies to the Maven project

A.2. The SpringSource Tool Suite for Eclipse

A.2.1. Importing the Spring Batch project

A.2.2. Creating and editing a Spring configuration file

A.3. Summary

Appendix B. Managing Spring Batch Admin

B.1. Downloading Spring Batch Admin

B.2. Building Spring Batch Admin from the distribution sample

B.3. Embedding Spring Batch Admin in a web application

B.3.1. Adding Spring Batch Admin dependencies

B.3.2. Declaring Spring Batch Admin in web.xml

B.4. Configuring Spring Batch Admin

B.4.1. Plugging into your batch metadata

B.4.2. Deploying your own job files

B.4.3. Overriding the Spring Batch Admin configuration

B.5. Summary

Index

List of Figures

List of Tables

List of Listings