Reading and Writing from Excel in Spring Batch

Rik Scarborough Development Technologies, Java, Spring, Spring Batch, Tutorial 4 Comments

We have discussed many different ways to read and write data in Spring Batch. The framework comes with quite an assortment of Readers and Writers that can be used directly or reused in some manner. Most of the time, the requirements consist of reading the data from some type of text file or database.

So what happens when the business we are supporting asks for something out of the ordinary, such as reading an Excel file and outputting the data to another Excel file? Typically the off-the-cuff response would be, “can you convert it to a CSV or other delimited text file?” Or “You know, Excel will read a CSV file just fine.” Sometimes that works, and sometimes the business requirements do not allow that type of flexibility.

Consider this scenario; in these days of Cloud and other online computing, the input file is likely created by a server that the company has no direct access to as far as programming. The file it creates is in one format, Excel. The output of your process has to go before several executives or other business clients and needs to be formatted in a professional looking manner. Adding a manual process to import a CSV and format it diminishes the value of using Spring Batch.

For the sake of the honor of the coding profession, you agree to the requirement to read and write from an Excel file directly. Now, how do you do that?…

Cucumber Testing in Spring Batch

Dallas Monson Development Technologies, Spring, Spring Batch, Testing 3 Comments

With the evolution of microservices and the scalable nature of modern distributed architectures, batch processing seems to be falling out of favor. In fact, the term batch processing itself seems to be unfavorably associated with monolithic mainframe applications and thus does not seem to have much appeal.

Unless, of course, you are working on a project that is being designed to replace or modernize one of those mainframe applications. If that is the case, then likely some sort of batch process has come up with a non-functional requirement that needs to be dealt with in the new system.

For this specific concern, a very powerful framework has been provided: Spring Batch. It has many of the same features of a mainframe batch process like restart/recovery, chunk processing, and error handling along with exit codes. This framework allows developers to create powerful batch processing applications in the Spring Framework and enjoy the rich backplane of capabilities that this provides.

Continuing with the modernization thread, you will likely be tasked with providing some assurances to the business that the new, modernized process will produce the same outcome as the one that is being replaced. Here is where testing comes in, and where Cucumber specifically shines.

Cucumber provides behavioral testing support in the Spring universe. This allows developers and business users to collaborate through a common set of conventions and verbiage to validate that the app is behaving how the business intended as well as how the developer coded it.

In this post we will cover the following:
Why use Cucumber with Spring Batch
An overview of Cucumber and an example Cucumber Test
How to start with Cucumber and Spring Batch…

Encrypting Working Files Locally in Spring Batch

Rik Scarborough Development Technologies, Java, Spring, Spring Batch Leave a Comment

It seems that quite often we read stories in the news about computer systems being cracked and data being compromised. It’s a growing concern that should be a consideration for everyone in Information Technology. There is probably not just one solution that will keep all data safe, but hopefully small efforts in many areas will provide us with the best possible solution.

In this post, I show a solution for encrypting sensitive files for local use with Java’s Encryption library & tying directly into Spring Batch readers and writers.

The Scenario
Recently we began writing a Spring Batch application that would handle sensitive data. The application servers were set up with some very good, basic security, but we felt the data could use some extra protection.

The data would be delivered to the company on a well-protected and secure FTP server. Mark Fricke did an excellent post recently on Spring Integration and Spring Batch in which he discusses downloading an encrypted file from a FTP server and decrypting it. Unfortunately, this was not exactly the problem we had. We needed to download a unencrypted file, but never write it to the Application Server unencrypted. But, we needed to be able to read that file and process it in Spring Batch.

Using Java’s built-in cryptography, we are able to extend Spring Batch to encrypt the file on the disk and then read that file in a Spring Batch Reader. In addition, we can write the results out as an encrypted file then transfer that file back to the secure FTP server as clean text.

Wow, that sounds like a lot and will be a really complex solution. Actually the code turned out to not be all that complex. This solution relies partly on the Delegate Pattern I wrote about before, so I will be using the same code I developed for that and just showing the changes here. Please refer back to the original post…

Modernization Lessons: FTP & the Mainframe

Clayton Neff COBOL, Consulting, Development Technologies, Java, Modernization, Programming, Spring Batch 1 Comment

One of my most recent projects involved helping a client move many decades of code from a mainframe environment to a distributed Java web environment. The client had engaged another company to actually transform the mainframe code to Java, and our team was tasked with making it all actually work.

One of the major areas we had to deal with was the transition of all of the batch processes. Of course, Spring Batch came to our rescue for most of the work, and was an easy choice as we were already using Spring Boot to wrapper the converted applications.

The most challenging part of the entire project was that the client did not want to move everything at once in a Big Bang, but rather a few programs as a time. This meant that some programs would be running in the Java environment while others remained on the mainframe.

In this blog, I discuss three data challenges we encountered in the transition of an enterprise mainframe to Java web application with Spring Batch, how we overcame them, and tips to keep in mind going forward when in similar migration situations.

Using Spring Integration In Conjunction With Spring Batch

Mark Fricke Development Technologies, JavaScript, Spring, Spring Batch, Spring Boot 6 Comments

Recently I was working on a development project for a client focused on Spring Batch. The program required a pull of the SFTP directory for an encrypted file, decryption of that file, starting of the Spring Batch program, and archive of that file.

Initially, my first thought was to use a shell script to perform all the tasks. Then one of my colleagues suggested Spring Integration; I thought this was great opportunity to learn and get my hands dirty with something new.

In this blog, I will show an example of Spring Integration configuration code, break it apart, and show how each part works.

Spring Integration turned out to be a simple solution to my client’s needs. Using Spring Integration and Spring Batch with Spring Boot, I was able to have a single deployable jar that included everything to run the application. I no longer needed separate deployments for the shell script, and batch process and all code is one Java project.