Caching Strategy Reminder for Maven-Based Docker Builds

My local development feedback loop between code change and runnable container was annoyingly long on a Maven-based project I was recently working on. I wanted to speed things up.

The scenario was something like this:

1. touch/change some source code
2. docker build
3. maven downloads the world
4. maven compiles my project
5. docker run
6. touch/change some source code
7. docker build
8. maven downloads the world
9. maven compiles my project
10. docker run
11. touch/change some source code
12. docker build
13. maven downloads the world
14. maven compiles my project
15. docker run
16. …

I didn’t really enjoy the “maven downloads the world” steps, and wanted to minimize the number of times it needed to run.

Let’s follow along as I make my situation a little better. For illustration, we’ll start off with this generic archetype-created skeleton project:

https://gist.github.com/f901ceaf75b3458c1cd0

Things aren’t that bad when I am building back-to-back, e.g.

$ docker build .
  ...
$ docker build .
  ...

Notice that the second build is fast as everything is cached up.

But what about when we do something like this:

$ docker build .
  ...
$ touch src/main/java/com/keyholesoftware/blog/App.java
  ...
$ docker build .
  ...

Notice that the second build is unnecessarily slowed down by the redownload portion.

I sat around and despaired for a while until I remembered the tricks I’ve seen with selective caching:

https://gist.github.com/cd7eac7ee43d09ccfe89

Let’s try that sequence again.

$ docker build .
 ...
$ touch src/main/java/com/keyholesoftware/blog/App.java
  ...
$ docker build .
  ...

Getting better, but there were still a few downloads going on during the second build. They are related to the surefire test/plugin. Actually this process will help us iron out downloads which are chosen dynamically, and lock those down. In this case, we lock down our surefire provider.

https://gist.github.com/85d3c479d7b5d6868ed3

Let’s try that sequence again.

$ docker build .
  ...
$ touch src/main/java/com/keyholesoftware/blog/App.java
  ...
$ docker build .
  ...

So now, unless we change the POM, we don’t have to redownload anything. Nice.

Now the scenario is something like this:

1. touch/change some source code
2. docker build
3. maven downloads the world
4. maven compiles my project
5. docker run
6. touch/change some source code
7. docker build
8. maven compiles my project
9. docker run
10. touch/change some source code
11. docker build
12. maven compiles my project
13. docker run
14. …

Notice the “maven downloads the world” step only happens once (unless I actually change the POM, of course).

Final Thoughts

There might be better ways to handle some of this (e.g. dependency:resolve/resolve-plugin but that doesn’t seem to work as thoroughly, and probably something with fig), but I mainly wanted to highlight a possible use of the selective adding/caching.

Other Notes:

• For you Ruby+Gemfile, Python+requirements.txt, Node+package.json, Go+Godeps.json etc. folks — Maven doesn’t have an explicit ‘install dependencies’ step. See Introduction to the Build Lifecycle if you’re bored.
• For you Gradle folks, I haven’t used Gradle much. What are your thoughts?
• The source code for this post is at: https://github.com/in-the-keyhole/khs-docker-caching-blog

Thanks for reading!

— Luke Patterson, [email protected]