What could be more exciting than a blog post that combines the two MOST EXCITING concepts in software engineering today: Batch Processing and Testing!? Talk about two great tastes!

With the evolution of microservices and the scalable nature of modern distributed architectures, batch processing seems to be falling out of favor. In fact, the term batch processing itself seems to be unfavorably associated with monolithic mainframe applications and thus does not seem to have much appeal.

Unless, of course, you are working on a project that is being designed to replace or modernize one of those mainframe applications. If that is the case, then likely some sort of batch process has come up with a non-functional requirement that needs to be dealt with in the new system.

For this specific concern, a very powerful framework has been provided: Spring Batch. It has many of the same features of a mainframe batch process like restart/recovery, chunk processing, and error handling along with exit codes. This framework allows developers to create powerful batch processing applications in the Spring Framework and enjoy the rich backplane of capabilities that this provides.

Continuing with the modernization thread, you will likely be tasked with providing some assurances to the business that the new, modernized process will produce the same outcome as the one that is being replaced. Here is where testing comes in, and where Cucumber specifically shines.

Cucumber provides behavioral testing support in the Spring universe. This allows developers and business users to collaborate through a common set of conventions and verbiage to validate that the app is behaving how the business intended as well as how the developer coded it.

In this post we will cover the following:

• Why use Cucumber with Spring Batch
• An overview of Cucumber and an example Cucumber Test
• How to start with Cucumber and Spring Batch
• Summary of Key Points

What this post is NOT:

• An in-depth review of Cucumber
• An in-depth or even introduction to Spring Batch (For that, see any one of the great Keyhole Blogs on Spring Batch)
• Any code that you should put into production
• An in-depth discussion on BDD vs TDD
• Vegan related 🙂

So, let’s get started!

Why Use Cucumber With Spring Batch

Spring Batch is simply a powerful batch framework that leverages Java and Spring.

Cucumber is a tool for running automated tests by leveraging BDD concepts and conventions.

When talking about Cucumber and Spring Batch together, I often get several questions like the following:

• Isn’t a batch process pretty binary in its operation? – if it succeeds then the batch was processed, if not then it failed.
• Do we really need behavioral testing with this construct, or is this like turbo-charging a Vespa (http://www.vespa.com/us_EN)?
• Can’t we get the same results just by wrapping a few JUnit tests around our batch processors?
• Won’t this slow down development and make the business folks mad at me?
• Don’t you have anything better to do?

Let’s walk through each of these questions and answer them. They are likely very similar to the questions currently rolling around in your brain right now, and going through them will help answer the ‘why’ question.

First question: Isn’t a batch process pretty binary in its operation?

Answer: It may seem like this is the case on the surface, but the testing is not for the mechanical processing itself. In a batch process, most times you will discover some complex business rules.

For example: if a file comes in without a header then it gets rejected – unless it comes from vendor XYZ, or if a record fails, then the app needs to exit with a code of 555 so that it can be mapped back to a list of exit codes and meanings already in place for the support teams, etc…

These scenarios–along with the more typical testing ones–validate that the output from the processing step matches what is expected from the process as a whole. They can all be present in a batch application. These scenarios are what make batch programs a good fit for Cucumber.

Second Question: Do we really need behavioral testing with this construct, or is this like turbo-charging a Vespa?

Answer: Behavioral testing matches up extremely well with the actual business requirements of a batch process. In the case of Cucumber, it provides a clear language set of scenarios and assumptions that can be executed and validated in an automated, code-driven fashion. Cucumber uses the Given-When-Then format in constructing its scenarios.

This common language gives the developer and the business user a contract that they can communicate through. Each step in the scenario is correlated to a unit test that exercises that particular set of logic. The business user can read what they expect to happen and the developer can tie this to code.

So actually, this is more like providing you with a spare tire and helmet while you are riding your Vespa (maybe adding a racing stripe too).

Third Question: Can’t we get the same results just by wrapping a few JUnit tests around our batch processors?

Answer: Maybe. The real benefit here is not necessarily the unit tests themselves, but the common contract that eliminates possible confusion over requirements between the developers and the business. Just wrapping a few unit tests still implies that the developer is interpreting the requirements exactly as the business meant.

So to give a magic 8 ball answer – “Sources point to no.”

Fourth Question: Won’t this slow down development and make the business folks mad at me?

Answer: This is always a common push back when any kind of testing is brought up. The key difference here is that the scenario instantly provides value to the business. They can see, with automated results, that the application is doing exactly what THEY wanted, not what the developer thought they meant by what they said they wanted.

This improvement in the communication between the two halves of the team will actually speed things up, especially as development gets further along.

Fifth Question: Don’t you have anything better to do?

Answer: Actually, no. My sea monkey colony was recently decimated by an unfortunate coffee tsunami, so this is it!

By using Cucumber in Spring Batch, developers can help the business vet their requirements for the batch process. Having a common method to communicate that both parties understand often leads to the discovery of additional “forgotten” requirements )that normally would come up after the fact either as bugs or as scope creep). This further enhances the value that Cucumber provides when incorporated with Spring Batch.

An Overview Of Cucumber And An Example Cucumber Test

In order to use Cucumber in a Spring Batch application, you must include the dependencies that Cucumber requires. Below are the Maven dependencies that are current as of this posting date:

<dependency>
    <groupId>info.cukes</groupId>
    <artifactId>cucumber-java</artifactId>
    <version>1.2.5</version>
    <scope>test</scope>
</dependency>
<dependency>
    <groupId>info.cukes</groupId>
    <artifactId>cucumber-junit</artifactId>
    <version>1.2.5</version>
    <scope>test</scope>
</dependency>
<dependency>
    <groupId>info.cukes</groupId>
    <artifactId>cucumber-spring</artifactId>
    <version>1.2.5</version>
    <scope>test</scope>
</dependency>

Once these dependencies are added to your project, then we can begin to use Cucumber. Cucumber uses its own language called Gherkin to create what it calls features. A sample feature file would look like this:

Feature: A batch process for files 
 
  Scenario: All data is returned in uppercase
    Given a set of data is read in 
    When that data is processed
    Then the output data is all uppercase

This may not look like much, but it is very powerful. In a lot of ways this exact, or similar structure is what your PBI/User Stories may look like. If they do not resemble anything that you have seen, they at least have an easy-to-read structure that can be understood by business users or non-developers. The Gherkin file becomes the contract between the developers and the business.

So now that we have our scenario defined, we need to connect our test to this feature file. We do that using annotations in Spring in our test class.

@RunWith(Cucumber.class)
@CucumberOptions(features = "src/test/resources")
public class BatchTest {
}

Notice that the @CucumberOptions annotation is where we actually point to the location of our Gherkin feature file. This is a contrived location and can be any location you prefer based on your conventions.

Now that our test is hooked up to our feature, we will now connect each line to a specific test. We will start with the Given statement:

@Given("a set of data is read in")
public void loadData() {
    // actual code that loads sample or test data
}

Notice that we used the @Given annotation and passed it the text after our Given statement in the feature file. This is how the connection is made between each line in the Gherkin file and our unit tests behind the scenes. Next, we will do the When and Then sections:

@When("that data is processed")
public void processData() {
    // actual code that processes that data - usually your batch 
    // processor tasklet code
}

@Then("the output data is all uppercase")
public void outputData() {
    // actual code that takes that processed data and outputs it
    // in this step the data is validated to be uppercase as well
}

If we were to actually run these tests, we would get the same red/green feedback as we would from running a JUnit test. Except, in this case, we would know whether the preconditioning step (Given), the actual processing step (When) or the output/validation step (Then) failed/succeeded. This very quickly allows developers the power to determine if they have the correct reader, processor, or writer code in their application.

Say for example that the Given step failed. Then the developer could quickly diagnose what the issue is – maybe there was no file to be read in, maybe we had the wrong input format, etc. These kinds of insights can save a lot of time and frustration because often batch processes can fail in cryptic ways or in a generic Failed status.

This layer of testing allows the developer the ability to not only validate that the bits are working as expected, but also that the business logic is being implemented as expected.

How To Start With Cucumber And Spring Batch

Note: I’ve propped up a git repo with code that demonstrates Cucumber testing. Try it out!

In most batch projects there can be hundreds if not thousands of processes depending on the business. This may seem like an untenable task, especially if the testing is approached after the fact – but there is a way forward. The first step is to identify where in your batch project do you have the most business logic.

In a batch process, most of the business logic happens in the processor tasklet or another custom tasklet. This business logic is what will match your feature file contents and scenarios. Often times in exercising this set of logic, the business user may realize that what they thought they wanted is not what they needed. Again this construct helps to vet this out.

Any changes at this point in the business logic, require a change to contract (Gherkin feature file) in order for them to be validated. This ensures that both developer and business user are on the same page and can be extremely valuable as the application increases in complexity.

Once you have the business logic located, it still may be a huge amount of code. In this case, it is best to start with two areas:

• The most critical business processes, meaning if they fail or behave weirdly the whole process is compromised.
• The code paths exercised the most, meaning the ones that get run no matter what.

This will help bring the scope of this task to a much more manageable level. While it can always be good to cover those edge cases for complete confidence, you likely will get the most value by ensuring that your core processes are behaving as expected and that the most important processes are doing their jobs.

At Keyhole we have built entire frameworks for integrating Cucumber tests into Batch and/or Spring projects, but by no means do you have to go to this level to get the value from Cucumber and Spring Batch. Starting with a single scenario will usually be enough to build value and get buy-in from both the developers and the business to continue on. Any time you can make your application code more resilient against change and predictable in its functionality, it is all high-fives and unicorns. 🙂

Summary of Key Points

In this article, we talked about some reasons why to use Cucumber with Spring Batch. Those reasons ranged from providing a contract between the business and the developers to communicate through, to allow the developer to be confident that their code is doing exactly what the business intended. We also reviewed how to quickly get Cucumber in your project and went over an example Cucumber test. Finally, we discussed some ways to quickly get started and where to focus efforts to get the most value immediately.

The key points from this article are:

• Cucumber provides a contract-first environment between the developer and the business.
• Behavioral testing, especially in the Given-When-Then format, matches very well with batch processing business rules and logic.
• Focusing Cucumber tests on the most important and the most used code paths will provide quick value – maybe the most value. 😉
• Cucumber tests are really just unit tests under the hood and not difficult to set up.
• I hope you enjoyed this article about the ever-exciting behavioral testing and batch processing using Cucumber and Spring Batch.

For further learning, I’ve propped up a git repo with code that demos the Cucumber testing. See it to see more Cucumber in action. If you want to learn even more about Cucumber, there are endless resources on the web. Cucumber itself even has a learning track.

If you wish to learn more about Spring Batch, please visit any of the great posts on the Keyhole blog on the subject.

Thank you for reading,

Dallas Monson