Reading and Writing from Excel in Spring Batch

Rik Scarborough Java, Spring, Spring Batch, Technology Snapshot, Tutorial Leave a Comment

We have discussed many different ways to read and write data in Spring Batch. The framework comes with quite an assortment of Readers and Writers that can be used directly or reused in some manner. Most of the time, the requirements consist of reading the data from some type of text file or database.

So what happens when the business we are supporting asks for something out of the ordinary, such as reading an Excel file and outputting the data to another Excel file? Typically the off-the-cuff response would be, “can you convert it to a CSV or other delimited text file?” Or “You know, Excel will read a CSV file just fine.” Sometimes that works, and sometimes the business requirements do not allow that type of flexibility.

Consider this scenario; in these days of Cloud and other online computing, the input file is likely created by a server that the company has no direct access to as far as programming. The file it creates is in one format, Excel. The output of your process has to go before several executives or other business clients and needs to be formatted in a professional looking manner. Adding a manual process to import a CSV and format it diminishes the value of using Spring Batch.

For the sake of the honor of the coding profession, you agree to the requirement to read and write from an Excel file directly. Now, how do you do that?…

Cucumber Testing in Spring Batch

Dallas Monson Spring, Spring Batch, Technology Snapshot, Testing Leave a Comment

With the evolution of microservices and the scalable nature of modern distributed architectures, batch processing seems to be falling out of favor. In fact, the term batch processing itself seems to be unfavorably associated with monolithic mainframe applications and thus does not seem to have much appeal.

Unless, of course, you are working on a project that is being designed to replace or modernize one of those mainframe applications. If that is the case, then likely some sort of batch process has come up with a non-functional requirement that needs to be dealt with in the new system.

For this specific concern, a very powerful framework has been provided: Spring Batch. It has many of the same features of a mainframe batch process like restart/recovery, chunk processing, and error handling along with exit codes. This framework allows developers to create powerful batch processing applications in the Spring Framework and enjoy the rich backplane of capabilities that this provides.

Continuing with the modernization thread, you will likely be tasked with providing some assurances to the business that the new, modernized process will produce the same outcome as the one that is being replaced. Here is where testing comes in, and where Cucumber specifically shines.

Cucumber provides behavioral testing support in the Spring universe. This allows developers and business users to collaborate through a common set of conventions and verbiage to validate that the app is behaving how the business intended as well as how the developer coded it.

In this post we will cover the following:
Why use Cucumber with Spring Batch
An overview of Cucumber and an example Cucumber Test
How to start with Cucumber and Spring Batch…

Web Development Business

Pseudo-Static Row Mappers, a Healthy Alternative to Static Row Mapping

Ryan McCullough Java, Spring Leave a Comment

If you know Spring, chances are pretty good that you’ve also worked with RowMappers and everyone’s #1 favorite BeanPropertyRowMapper. Okay, maybe not EVERYONE. But EVERYONE will acknowledge BPRM’s power potential and how wonderfully easy it is to use!

While BeanPropertyRowMapper may be the smartest and most beautiful on the RowMapper block, many in the industry refuse to give it the time of day, and for perfectly justified reasons.

Sometimes, when we can’t have beauty and wisdom, we’re forced to settle for loud and predictable. Yes, I’m talking about hardcoded, unchanging, tell-it-like-it-is, static RowMapper. Hate on them all you like, Static RowMappers are fast, easy to understand, and they seem to replicate like tribbles.

But, as many of you know, an application can grow into a swamp of one-off RowMappers. ESPECIALLY if you are working with a lot of high-throughput batch operations that need to run strictly optimized queries for performance as to avoid any unnecessary marshaling of data.

Recently, I’ve tried a mildly clever alternative to RowMapping I like to call Pseudo-Static Row Mappers. In this post, I introduce the basics of Pseudo-Static Row Mappers. We show how they give the tough rigid optimization and control of hard-coded naming and data typing while retaining BeanPropertyRowMapper’s spirit of freedom.

Encrypting Working Files Locally in Spring Batch

Rik Scarborough Java, Spring, Spring Batch, Technology Snapshot Leave a Comment

It seems that quite often we read stories in the news about computer systems being cracked and data being compromised. It’s a growing concern that should be a consideration for everyone in Information Technology. There is probably not just one solution that will keep all data safe, but hopefully small efforts in many areas will provide us with the best possible solution.

In this post, I show a solution for encrypting sensitive files for local use with Java’s Encryption library & tying directly into Spring Batch readers and writers.

The Scenario
Recently we began writing a Spring Batch application that would handle sensitive data. The application servers were set up with some very good, basic security, but we felt the data could use some extra protection.

The data would be delivered to the company on a well-protected and secure FTP server. Mark Fricke did an excellent post recently on Spring Integration and Spring Batch in which he discusses downloading an encrypted file from a FTP server and decrypting it. Unfortunately, this was not exactly the problem we had. We needed to download a unencrypted file, but never write it to the Application Server unencrypted. But, we needed to be able to read that file and process it in Spring Batch.

Using Java’s built-in cryptography, we are able to extend Spring Batch to encrypt the file on the disk and then read that file in a Spring Batch Reader. In addition, we can write the results out as an encrypted file then transfer that file back to the secure FTP server as clean text.

Wow, that sounds like a lot and will be a really complex solution. Actually the code turned out to not be all that complex. This solution relies partly on the Delegate Pattern I wrote about before, so I will be using the same code I developed for that and just showing the changes here. Please refer back to the original post…

Using Spring Integration In Conjunction With Spring Batch

Mark Fricke JavaScript, Spring, Spring Batch, Spring Boot, Technology Snapshot 4 Comments

Recently I was working on a development project for a client focused on Spring Batch. The program required a pull of the SFTP directory for an encrypted file, decryption of that file, starting of the Spring Batch program, and archive of that file.

Initially, my first thought was to use a shell script to perform all the tasks. Then one of my colleagues suggested Spring Integration; I thought this was great opportunity to learn and get my hands dirty with something new.

In this blog, I will show an example of Spring Integration configuration code, break it apart, and show how each part works.

Spring Integration turned out to be a simple solution to my client’s needs. Using Spring Integration and Spring Batch with Spring Boot, I was able to have a single deployable jar that included everything to run the application. I no longer needed separate deployments for the shell script, and batch process and all code is one Java project.