Using Amazon ElastiCache for Redis To Optimize Your Spring Boot Application
August 28, 2018
Has your project gotten to the point when big data sets and/or time-consuming calculations have begun to affect performance? Or are you struggling to optimize your queries and need to cache some information to avoid continually hitting your database? Then caching could be your solution.
For this article, I will demonstrate how to utilize Amazon ElastiCache for Redis to speed up areas of your application. The example application we will build uses Spring Boot 2.x and is available on Github.
What is Amazon ElastiCache for Redis?
ElastiCache for Redis is a super fast, in memory, key-value store database. It supports many different kinds of abstract data structures, such as strings, lists, maps, sets, sorted sets, hyperloglogs, bitmaps, and spatial indexes. At its highest level, ElastiCache for Redis is a fully managed service for a standard Redis installation and uses all the standard Redis APIs. Which means the app we are building can rely not only on the Amazon ElastiCache flavor of Redis, but any other environment matching the version of your choosing.
Setting up ElastiCache for Redis
Begin by navigating to the ElasticCache dashboard, selecting Redis, and create a new cluster. You will be prompted to define a cache name, description, node type (server size), and number of replicates.
VPC & Security Groups
To be able to access your Redis cluster, the instance(s) running our app must have be in the same Virtual Private Network (VPC) and contain the proper security groups. Your EC2 instances must allow the port of your Redis cluster (6379) to be able to communicate. By default, your Redis cluster is only accessible internally from the VPC selected. This is done purposely, as no internet gateway should be connected as it would defeat the purpose of a high efficiency, in-memory cache that Redis provides.
If you wish to run the app locally, consider installing Redis using Docker. The variables outlined in our application.properties file below can be modified to run locally.
Launching your Redis Cluster
Once you have properly configured your security groups and VPC, click “create”. ElastiCache will now provision and launch you new Redis cluster. When the status turns to available the cluster is ready to handle connections.
Building Our Application
For this example application, we will be using Spring Boot 2.x with the Spring-Data-Redis and Jedis (client library for Redis). I first begin by importing them into my project.
<dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-data-redis</artifactId> </dependency> <dependency> <groupId>redis.clients</groupId> <artifactId>jedis</artifactId> <version>2.9.0</version> </dependency>
These libraries allow us to setup our caching config in Spring. A important concept to understand is that the Spring Framework provides its own abstraction for transparently adding caching. You do not only have to use Redis, the abstraction provides a list of providers; Couchbase, EhCache 2.x, Hazelcast, etc. As you will see adding caching to a service method is as simple as providing the appropriate annotation.
Now that we have included the required libraries in our pom file, Spring will try to autoconfigure a RedisCacheManager
. I personally do not like magic behind the scenes so we are going to setup and configure our own annotation based RedisCacheManager
.
A RedisCacheManager
is how we configure and build a cacheManger
for Spring to use. Notice that I have defined the redisHostName
, redisPort
, and redisPrefix
for the Jedis (client library for redis) to use to connect to our cluster.
@Configuration @EnableCaching public class RedisConfig { @Value("${redis.hostname}") private String redisHostName; @Value("${redis.port}") private int redisPort; @Value("${redis.prefix}") private String redisPrefix; @Bean JedisConnectionFactory jedisConnectionFactory() { RedisStandaloneConfiguration redisStandaloneConfiguration = new RedisStandaloneConfiguration(redisHostName, redisPort); return new JedisConnectionFactory(redisStandaloneConfiguration); } @Bean(value = "redisTemplate") public RedisTemplate<String, Object> redisTemplate(RedisConnectionFactory redisConnectionFactory) { RedisTemplate<String, Object> redisTemplate = new RedisTemplate<>(); redisTemplate.setConnectionFactory(redisConnectionFactory); return redisTemplate; } @Primary @Bean(name = "cacheManager") // Default cache manager is infinite public CacheManager cacheManager(RedisConnectionFactory redisConnectionFactory) { return RedisCacheManager.builder(redisConnectionFactory).cacheDefaults(RedisCacheConfiguration.defaultCacheConfig().prefixKeysWith(redisPrefix)).build(); } @Bean(name = "cacheManager1Hour") public CacheManager cacheManager1Hour(RedisConnectionFactory redisConnectionFactory) { Duration expiration = Duration.ofHours(1); return RedisCacheManager.builder(redisConnectionFactory) .cacheDefaults(RedisCacheConfiguration.defaultCacheConfig().prefixKeysWith(redisPrefix).entryTtl(expiration)).build(); } }
I have defined two cache managers. One that is infinite (default), and once that will cause all keys to expire in 1 hour called cacheManager1Hour
.
The cluster information is passed in from the applications.properties:
redis.hostname=URL_TO_ELASTIC_CACHE_REDIS_CLUSTER redis.port=6379 redis.prefix=testing
Implementing a Simple Service
Now that our Redis cluster is configured in Spring, annotation-based caching is enabled. Let’s assume you have a long-running task that takes 2 seconds to do its work. By annotation the service with @Cacheable
the result of the method call will be cached. I have given this cachable a value of getLongRunningTaskResult
which will be used in its compound key, a key (by default is generated for you), and a cacheManager
(cacheManager1Hour
configured above).
@Cacheable(value = "getLongRunningTaskResult", key="{#seconds}", cacheManager = "cacheManager1Hour") public Optional<TaskDTO> getLongRunningTaskResult(long seconds) { try { long randomMultiplier = new Random().nextLong(); long calculatedResult = randomMultiplier * seconds; TaskDTO taskDTO = new TaskDTO(); taskDTO.setCalculatedResult(calculatedResult); Thread.sleep(2000); // 2 Second Delay to Simulate Workload return Optional.of(taskDTO); } catch (InterruptedException e) { return Optional.of(null); } }
Note: It is important that the resulting object of the method is serializable otherwise the cacheManager will not be able to cache the result.
Testing for Performance Improvements
To easy test the API, I have included swagger-ui which make it simple for developers to interact with the api we have built. I have also created a few simple endpoints to create and flush the cache.
@ApiOperation( value = "Perform the long running task") @RequestMapping(value = "/{seconds}", method = RequestMethod.GET) public ResponseEntity<TaskDTO> longRunningTask(@PathVariable long seconds) { Optional<TaskDTO> user = taskService.getLongRunningTaskResult(seconds); return ResponseUtil.wrapOrNotFound(user); } @ApiOperation( value = "Resets the cache for a key") @RequestMapping(value = "/reset/{seconds}", method = RequestMethod.DELETE) public ResponseEntity<?> reset(@PathVariable long seconds) { taskService.resetLongRunningTaskResult(seconds); return new ResponseEntity<>(HttpStatus.ACCEPTED); }
Once you deploy your app to EC2 navigate to URL path: /swagger-ui.html
From this GUI you can easily test your API performance improvements. Calling the GET endpoint for the first time should take roughly two seconds to return the new calculated result. Calling it subsequently will return an almost instant response as the long-running task is now cached in Redis.
Final Thoughts
Today’s applications demand a responsive user experience. Design your queries and calculations to be as performant as possible, but every once in a while, when you can sacrifice real-time data, just cache it.
Full project code is available at: https://github.com/sixthpoint/spring-boot-elasticache-redis-tutorial.
More From Brandon Klimek
About Keyhole Software
Expert team of software developer consultants solving complex software challenges for U.S. clients.
is your cluster mode disable? Since you have primary node point instead of configuration node point.
probably disabled, i used Redis(cluster mode disabled).
How to mention aws hostname ,
xxxx.ctfypf.ng.0001.use2.cache.amazonaws.com this is mine while I am using its giving socket time out exception
[dispatcherServlet] in context with path [/api/am/numbers] threw exception [Request processing failed; nested exception is org.springframework.data.redis.RedisConnectionFailureException: Cannot get Jedis connection; nested exception is redis.clients.jedis.exceptions.JedisConnectionException: Could not get a resource from the pool] with root cause.
How to connect to cluster mode enabled Redis. Replacing the primary node point with configuration node point is not sufficient.
I am getting same Error… can you help me if you got solution
threw exception [Request processing failed; nested exception is org.springframework.data.redis.RedisConnectionFailureException: Cannot get Jedis connection; nested exception is redis.clients.jedis.exceptions.JedisConnectionException: Could not get a resource from the pool]
I tried using connect to cluster mode enabled Redis using configuration node point. Has anyone implemented without any exception ?
Hi all, if you’re getting an error, you may need to provide a RedisClusterConfiguration. An example would be to define a list of nodes and then create a new redisCluster within the jedisConnectionFactory:
If you need more information or pointers, check out this link; https://docs.spring.io/spring-data/redis/docs/current/api/org/springframework/data/redis/connection/RedisClusterConfiguration.html
nodes can be set via spring.redis.cluster.nodes
I tried this:
RedisClusterConfiguration config = new RedisClusterConfiguration(
Collections.singletonList(redisHostName + “:” + redisPort));
But this is still failing, as redis is in clustered mode
Hi, I’m getting the same Timeout error with spring boot. Is it possible that this is occurring because tls = true has been set in the redis instance?
Hi Atika,
Thanks for taking time to read our post!
To answer your question, what Mark says below is correct. He is right in saying that you must be inside the VPC to run this. By default, your Redis cluster is only accessible internally from the VPC selected. It shouldn’t be exposed outside the VPC to run the app locally because that defeats the purpose of Redis.
We hope this helps clear up any confusion!
Since you are using the redis starter, you don’t need custom properties or beans. You only need to create a redistemplate if you dont like the default. You should only need to set spring.redis.host (the Primary Endpoint minus :6379).
FYI, i am pretty sure the connection timeout people are getting (I am too) is because ElasticCache is not exposed outside the VPC. The only way to get to it outside AWS is via a VPN. Although, i am trying that right now – we have a VPN but it still does not work. I am sure i missed setting something up ElasticCache. But VPN/VPC are not my area of expertise.