Spring Batch Apps to AWS Cloud: Transferring with Ease

Attention: The following article was published over 4 years ago, and the information provided may be aged or outdated. Please keep that in mind as you read the post.

The last few years have seen a lot of movement to bring applications that don’t require manual intervention from the mainframe to Unix, Linux, Windows servers, or even to the desktop. This concept is commonly known as batch programming, and Spring Batch has been the tool many of us are using to accomplish this. We’re seeing more and more necessity to transfer spring batch apps to AWS and other cloud hosts.

Let’s take a brief side trip into etymology – the study of words, not bugs. When programmers see the word “batch,” one of two things come to mind.

Programmers from a Windows or similar background tend to first think of a file that contains command-line instructions that can be executed as a single command.
Programmers from a Unix or Mainframe background (like me) tend to think of applications that require little-to-no interaction that are intended to process large amounts of data or logic on a recurring basis. At no point in this article will I be referring to the former.

Another trend that is gaining steam is to move from an internally-hosted server to a cloud-hosted system.

In this post, we discuss multiple ways for transferring Spring Batch applications up to the AWS Cloud, including EC2, Docker, Lambda, and others. I concentrate on AWS in this post, but, from my experience in Google Cloud, the same ideas will apply.

Moving To A Cloud Environment

First, I want to say that if you or your enterprise is moving to a cloud environment, the best path is to architect your application for your new environment from scratch. Choose the best tools to implement your requirements, and build it from there.

However, in the real world, there are times when we need to quickly move logic off an aging framework or we have a need to learn something new. Taking something we are familiar with into a new environment is a great way to learn that environment.

My advice here is intended for the programmers out there that need to begin to learn the cloud environment, specifically AWS.

So, say you have a batch job or a few batch jobs that you would like to move off a local machine and would like to learn more about cloud computing?

First, I would suggest, if you have not already done so, moving your Spring Batch jobs to Spring Boot, instead of running under an external web container. I would also suggest breaking up multiple jobs into smaller projects, or even one job per project if that is feasible.

Create An EC2 Instance

The quickest, easiest, and (so far) the cheapest solution to transfer Spring Batch apps to AWS I’ve tried is to create an EC2 instance and run your application there.

I was able to create an instance using Amazon Linux, including a version of Java 1.8. So far, for this project, this instance has not cost me anything.

As an aside, I did use this instance to attempt to build a different, very complex multi-project application that includes a GWT build. That build failed because that free instance did not have enough memory. I tried twice, then noticed that it cost me $4. So be aware, you could do something that will generate a cost. Typically you do get some free credits when you first sign up, but my account has been around way beyond that time. Be absolutely sure you have set a budget with AWS for the amount you are willing to part with.

Instructions

Once you have an AWS account and load the AWS Console, you can select EC2. From the EC2 Dashboard, select Launch instance. They are even kind enough to have a checkbox that will filter down to “Free tier only”. Then you can choose the environment you are most comfortable with.
Once you have your instance, you will have a command to access that instance, typically something like

ssh -i “” [email protected]

I had a basic Spring Boot/Batch application. I simply took that jar and move it to my EC2 instance.

scp -i “” batch demo.jar [email protected]:

Since Java is already installed on that instance, I simply have to run the application to launch the job.

ssh -i “” [email protected] "java -jar batchdemo.jar"

This, of course, assumes that you have job.enabled set to true; that the job is set to run at application start and then end, instead of waiting for some event to start the job. I would not suggest having the application running all the time, as that can add up runtime cost for doing nothing but waiting for a reason to run the job.

Again, this is the simplest way I’ve found to move off a local resource and begin using the cloud.

Docker

The next easiest path to moving spring batch apps to AWS is to create a Docker file, containerize your application, and run it in one of several ways on AWS.

There are several ways you can move your Batch application to a Docker instance. I simply added a Docker file.

ROM openjdk:8-jdk-alpine
VOLUME /tmp
ARG JAR_FILE=build/libs/*.jar
COPY ${JAR_FILE} app.jar
ENTRYPOINT ["java","-jar","/app.jar"]

You can build an EC2 instance that has Docker installed on it and run your new Dockerized application there.

Or, and probably the more advised way is to use Amazon Elastic Container Service (ECS). This will require you to set up your container on Amazon Elastic Container Registry (ECR) but will provide you with many more tools for running your application.

AWS Lambda

I’ve been studying AWS Lambda for a while. I find the concept fascinating. Just as the Unix philosophy of small applications that do one function is repeated in the Web Service philosophy of programming, Lambda seems to be a similar programming philosophy. I intend to get much deeper into the Lambda universe and will begin to write much more about that.

But I wanted to add it here not to suggest that Lambdas are a perfect fit for Batch programming. I honestly don’t think they are. They are typically too large, do too much, and run too long to be a good fit for Lambda or any Web Service type application.

But I tried to create one anyway. There are several really good articles that are getting Spring Boot to run as a Lambda, including a very good one here on the Keyhole Blog.

I was able to get my basic Batch application to compile, install, and run under Lambda, after a couple of hours of trial and error. But after all that, I was still getting errors while testing. As my deadline for this post came and went, I decided to not spend more time debugging a solution that I intended not to suggest you use anyway.

Could you or I get this to work? I am convinced you could. I was trying to add Lambda functionality to an existing application. The proper way to do this would be to create a Lambda function with the logic you need. A batch- style application, with the typical large datasets and long-running times is probably not the best use of Lambdas.

Other Environments

AWS has several other computing environments that you should explore, such as Lightsail. I have specifically stayed away from AWS Batch because it seemed too ironic. After all, this post is more about moving an existing application instead of architecting from scratch. If you have the requirement to create a batch-like application in AWS, I would advise looking seriously at AWS Batch.

Hopefully, this will encourage you to get started on moving your applications to the cloud, specifically AWS, to learn more about what you can do.

If you are moving to the cloud or transferring spring batch apps to AWS specifically, take some time to architect your environment. As an AWS Consulting Partner, Keyhole Software is prepared to assist you.

Believe in Good Code.