Caching Strategy Reminder for Maven-Based Docker Builds

Luke Patterson Docker, Java, Problem Solving, Tutorial 10 Comments

My local development feedback loop between code change and runnable container was annoyingly long on a Maven-based project I was recently working on. I wanted to speed things up.

The scenario was something like this:

  1. touch/change some source code
  2. docker build
  3. maven downloads the world
  4. maven compiles my project
  5. docker run
  6. touch/change some source code
  7. docker build
  8. maven downloads the world
  9. maven compiles my project
  10. docker run
  11. touch/change some source code
  12. docker build
  13. maven downloads the world
  14. maven compiles my project
  15. docker run

 

I didn’t really enjoy the “maven downloads the world” steps, and wanted to minimize the number of times it needed to run.

Let’s follow along as I make my situation a little better. For illustration, we’ll start off with this generic archetype-created skeleton project:

 

Things aren’t that bad when I am building back-to-back, e.g.

$ docker build .
  ...
$ docker build .
  ...

Notice that the second build is fast as everything is cached up.

 

But what about when we do something like this:

$ docker build .
  ...
$ touch src/main/java/com/keyholesoftware/blog/App.java
  ...
$ docker build .
  ...

Notice that the second build is unnecessarily slowed down by the redownload portion.

 

I sat around and despaired for a while until I remembered the tricks I’ve seen with selective caching:

Let’s try that sequence again.

$ docker build .
 ...
$ touch src/main/java/com/keyholesoftware/blog/App.java
  ...
$ docker build .
  ...

Getting better, but there were still a few downloads going on during the second build. They are related to the surefire test/plugin. Actually this process will help us iron out downloads which are chosen dynamically, and lock those down. In this case, we lock down our surefire provider.

 

Let’s try that sequence again.

$ docker build .
  ...
$ touch src/main/java/com/keyholesoftware/blog/App.java
  ...
$ docker build .
  ...

So now, unless we change the POM, we don’t have to redownload anything. Nice.

Now the scenario is something like this:

  1. touch/change some source code
  2. docker build
  3. maven downloads the world
  4. maven compiles my project
  5. docker run
  6. touch/change some source code
  7. docker build
  8. maven compiles my project
  9. docker run
  10. touch/change some source code
  11. docker build
  12. maven compiles my project
  13. docker run

Notice the “maven downloads the world” step only happens once (unless I actually change the POM, of course).

Final Thoughts

There might be better ways to handle some of this (e.g. dependency:resolve/resolve-plugin but that doesn’t seem to work as thoroughly, and probably something with fig), but I mainly wanted to highlight a possible use of the selective adding/caching.

Other Notes:

Thanks for reading!

— Luke Patterson, asktheteam@keyholesoftware.com


About the Author
Luke Patterson

Luke Patterson


Share this Post

Comments 10

  1. This did not fully work for me. I had to create a special “go-offline” profile that skips certain phases (e.g., validate) and also create some empty files (e.g., filters.properties, web.xml). Thanks for pointing in the right direction!

  2. –fail-never can similarly be used (tried) with mvn dependency:resolve or mvn dependency:go-offline. However it won’t (in-general) work. The problem is that when a failure is encountered, the rest of that POM’s dependencies are skipped. That’ll potentially leave holes in your dependencies if you have any “remote” dependencies listed after “local” ones in a particular POM file.

    1. Hi Stephane,

      I should have used a different phrase. It wasn’t as clear as it could be.

      The symptoms: Although Surefire normally can determine which test-framework provider to download/use based on which test framework is on the classpath, it wasn’t even attempting do to so because there were no test unit files to run. During the Docker build step in question, the build step intended to merely download/resolve all Maven dependencies, there are no source code files present.

      My solution/fix/workaround: Explicitly specify the test-framework provider I want as a plugin dependency so that Maven would download it before the Surefire plugin even runs. When the Surefire plugin then sees unit test files during the next Docker build step, it already has the test-framework provider it needs in the local Maven repository so it doesn’t have to download it before using it.

      Thanks,
      Luke

  3. Have you tried mounting the maven repository in a volume? This way it won’t have to re-download the world even if you change the pom.xml. Do you think this makes sense?

Leave a Reply