Spring Top

Get started with Spring 5 and Spring Boot 2, through the reference学习Springcourse:


1. Introduction

Unlike other Spring-based applications, testing batch jobs comes with some specific challenges, mostly due to the asynchronous nature of how jobs are executed.

In this tutorial, we're going to explore the various alternatives for testing aSpring Batchjob.

2. Required Dependencies

We're usingspring-boot-starter-batch, so first let's set up the required dependencies in ourpom.xml:

 org.springframework.boot spring-boot-starter-batch 2.6.1   org.springframework.boot spring-boot-starter-test 2.6.1 test   org.springframework.batch spring-batch-test 4.3.0.RELEASE test 

We included thespring-boot-starter-testandspring-batch-testwhich bring in some necessary helper methods, listeners and runners for testing Spring Batch applications.

3. Defining the Spring Batch Job

Let's create a simple application to show how Spring Batch solves some of the testing challenges.

Our application uses a two-stepJobthat reads a CSV input file with structured book information and outputs books and book details.

3.1. Defining the Job Steps

The two subsequentSteps extract specific information fromBookRecords and then map these toBooks (step1) andBookDetails (step2):

@Bean public Step step1( ItemReader csvItemReader, ItemWriter jsonItemWriter) throws IOException { return stepBuilderFactory .get("step1") . chunk(3) .reader(csvItemReader) .processor(bookItemProcessor()) .writer(jsonItemWriter) .build(); } @Bean public Step step2( ItemReader csvItemReader, ItemWriter listItemWriter) { return stepBuilderFactory .get("step2") . chunk(3) .reader(csvItemReader) .processor(bookDetailsItemProcessor()) .writer(listItemWriter) .build(); }

3.2. Defining the Input Reader and Output Writer

Let's nowconfigure the CSV file input reader using aFlatFileItemReaderto de-serialize the structured book information intoBookRecordobjects:

私有静态最终String []TOKENS = { "bookname", "bookauthor", "bookformat", "isbn", "publishyear" }; @Bean @StepScope public FlatFileItemReader csvItemReader( @Value("#{jobParameters['file.input']}") String input) { FlatFileItemReaderBuilder builder = new FlatFileItemReaderBuilder<>(); FieldSetMapper bookRecordFieldSetMapper = new BookRecordFieldSetMapper(); return builder .name("bookRecordItemReader") .resource(new FileSystemResource(input)) .delimited() .names(TOKENS) .fieldSetMapper(bookRecordFieldSetMapper) .build(); }

There are a couple of important things in this definition, which will have implications on the way we test.

First of all,we annotated theFlatItemReaderbean with@StepScope,and as a result,this object will share its lifetime withStepExecution

This also allows us to inject dynamic values at runtime so that we can pass our input file from theJobParameters in line 4。相比之下,使用的令牌BookRecordFieldSetMapperare configured at compile-time.

We then similarly define theJsonFileItemWriteroutput writer:

@Bean @StepScope public JsonFileItemWriter jsonItemWriter( @Value("#{jobParameters['file.output']}") String output) throws IOException { JsonFileItemWriterBuilder builder = new JsonFileItemWriterBuilder<>(); JacksonJsonObjectMarshaller marshaller = new JacksonJsonObjectMarshaller<>(); return builder .name("bookItemWriter") .jsonObjectMarshaller(marshaller) .resource(new FileSystemResource(output)) .build(); }

For the secondStep, we use a Spring Batch-providedListItemWriterthat just dumps stuff to an in-memory list.

3.3. Defining the CustomJobLauncher

Next, let's disable the defaultJoblaunching configuration of Spring Boot Batch by settingspring.batch.job.enabled=falsein ourapplication.properties.

We configure our ownJobLauncherto pass a customJobParametersinstance when launching theJob:

@SpringBootApplication public class SpringBatchApplication implements CommandLineRunner { // autowired jobLauncher and transformBooksRecordsJob @Value("${file.input}") private String input; @Value("${file.output}") private String output; @Override public void run(String... args) throws Exception { JobParametersBuilder paramsBuilder = new JobParametersBuilder(); paramsBuilder.addString("file.input", input); paramsBuilder.addString("file.output", output); jobLauncher.run(transformBooksRecordsJob, paramsBuilder.toJobParameters()); } // other methods (main etc.) }

4. Testing the Spring Batch Job

Thespring-batch-testdependency provides a set of useful helper methods and listeners that can be used to configure the Spring Batch context during testing.

Let's create a basic structure for our test:

@RunWith (SpringRunner.class) @SpringBatchTest @EnableAutoConfiguration @ContextConfiguration(classes = { SpringBatchConfiguration.class }) @TestExecutionListeners({ DependencyInjectionTestExecutionListener.class, DirtiesContextTestExecutionListener.class}) @DirtiesContext(classMode = ClassMode.AFTER_CLASS) public class SpringBatchIntegrationTest { // other test constants @Autowired private JobLauncherTestUtils jobLauncherTestUtils; @Autowired private JobRepositoryTestUtils jobRepositoryTestUtils; @After public void cleanUp() { jobRepositoryTestUtils.removeJobExecutions(); } private JobParameters defaultJobParameters() { JobParametersBuilder paramsBuilder = new JobParametersBuilder(); paramsBuilder.addString("file.input", TEST_INPUT); paramsBuilder.addString("file.output", TEST_OUTPUT); return paramsBuilder.toJobParameters(); }

The@SpringBatchTestannotation provides theJobLauncherTestUtilsandJobRepositoryTestUtilshelper classes.We use them to trigger theJobandSteps in our tests.

Our application usesSpring Boot auto-configuration, which enables a default in-memoryJobRepositoryAs a result,running multiple tests in the same class requires a cleanup step after each test run

Finally,if we want to run multiple tests from several test classes, we need to mark ourcontext as dirty。This is required to avoid the clashing of severalJobRepositoryinstances using the same data source.

4.1. Testing the End-To-EndJob

The first thing we'll test is a complete end-to-endJobwith a small data-set input.

We can then compare the results with an expected test output:

@Test public void givenReferenceOutput_whenJobExecuted_thenSuccess() throws Exception { // given FileSystemResource expectedResult = new FileSystemResource(EXPECTED_OUTPUT); FileSystemResource actualResult = new FileSystemResource(TEST_OUTPUT); // when JobExecution jobExecution = jobLauncherTestUtils.launchJob(defaultJobParameters()); JobInstance actualJobInstance = jobExecution.getJobInstance(); ExitStatus actualJobExitStatus = jobExecution.getExitStatus(); // then assertThat(actualJobInstance.getJobName(), is("transformBooksRecords")); assertThat(actualJobExitStatus.getExitCode(), is("COMPLETED")); AssertFile.assertFileEquals(expectedResult, actualResult); }

Spring Batch Test provides a usefulfile comparison method for verifying outputs using theAssertFileclass

4.2. Testing Individual Steps

Sometimes it's quite expensive to test the completeJobend-to-end and so it makes sense to test individualStepsinstead:

@Test public void givenReferenceOutput_whenStep1Executed_thenSuccess() throws Exception { // given FileSystemResource expectedResult = new FileSystemResource(EXPECTED_OUTPUT); FileSystemResource actualResult = new FileSystemResource(TEST_OUTPUT); // when JobExecution jobExecution = jobLauncherTestUtils.launchStep( "step1", defaultJobParameters()); Collection actualStepExecutions = jobExecution.getStepExecutions(); ExitStatus actualJobExitStatus = jobExecution.getExitStatus(); // then assertThat(actualStepExecutions.size(), is(1)); assertThat(actualJobExitStatus.getExitCode(), is("COMPLETED")); AssertFile.assertFileEquals(expectedResult, actualResult); } @Test public void whenStep2Executed_thenSuccess() { // when JobExecution jobExecution = jobLauncherTestUtils.launchStep( "step2", defaultJobParameters()); Collection actualStepExecutions = jobExecution.getStepExecutions(); ExitStatus actualExitStatus = jobExecution.getExitStatus(); // then assertThat(actualStepExecutions.size(), is(1)); assertThat(actualExitStatus.getExitCode(), is("COMPLETED")); actualStepExecutions.forEach(stepExecution -> { assertThat(stepExecution.getWriteCount(), is(8)); }); }

Notice thatwe use thelaunchStepmethod to trigger specific steps

Remember thatwe also designed ourItemReaderandItemWriterto use dynamic values at runtime, which meanswe can pass our I/O parameters to theJobExecution(lines 9 and 23).

For the firstSteptest, we compare the actual output with an expected output.

On the other hand,in the second test, we verify theStepExecutionfor the expected written items

4.3. Testing Step-scoped Components

Let's now test theFlatFileItemReaderRecall that we exposed it as@StepScopebean, so we'll want to use Spring Batch's dedicated support for this:

// previously autowired itemReader @Test public void givenMockedStep_whenReaderCalled_thenSuccess() throws Exception { // given StepExecution stepExecution = MetaDataInstanceFactory .createStepExecution(defaultJobParameters()); // when StepScopeTestUtils.doInStepScope(stepExecution, () -> { BookRecord bookRecord; itemReader.open(stepExecution.getExecutionContext()); while ((bookRecord = itemReader.read()) != null) { // then assertThat(bookRecord.getBookName(), is("Foundation")); assertThat(bookRecord.getBookAuthor(), is("Asimov I.")); assertThat(bookRecord.getBookISBN(), is("ISBN 12839")); assertThat(bookRecord.getBookFormat(), is("hardcover")); assertThat(bookRecord.getPublishingYear(), is("2018")); } itemReader.close(); return null; }); }

TheMetadataInstanceFactorycreates a customStepExecutionthat is needed to inject our Step-scopedItemReader.

Because of this,we can check the behavior of the reader with the help of thedoInTestScopemethod.

Next, let's test theJsonFileItemWriterand verify its output:

@Test public void givenMockedStep_whenWriterCalled_thenSuccess() throws Exception { // given FileSystemResource expectedResult = new FileSystemResource(EXPECTED_OUTPUT_ONE); FileSystemResource actualResult = new FileSystemResource(TEST_OUTPUT); Book demoBook = new Book(); demoBook.setAuthor("Grisham J."); demoBook.setName("The Firm"); StepExecution stepExecution = MetaDataInstanceFactory .createStepExecution(defaultJobParameters()); // when StepScopeTestUtils.doInStepScope(stepExecution, () -> { jsonItemWriter.open(stepExecution.getExecutionContext()); jsonItemWriter.write(Arrays.asList(demoBook)); jsonItemWriter.close(); return null; }); // then AssertFile.assertFileEquals(expectedResult, actualResult); }

Unlike the previous tests,we are now in full control of our test objects。As a result,we're responsible for opening and closing the I/O streams

5. Conclusion

In this tutorial, we've explored the various approaches of testing a Spring Batch job.

End-to-end testing verifies the complete execution of the job. Testing individual steps may help in complex scenarios.

Finally, when it comes to Step-scoped components, we can use a bunch of helper methods provided byspring-batch-test.They will assist us in stubbing and mocking Spring Batch domain objects.

As usual, we can explore the complete codebaseover on GitHub

Spring bottom

Get started with Spring 5 and Spring Boot 2, through the学习Springcourse:

Comments are closed on this article!