Messaging with RabbitMQ

by on May 13, 2013 10:00 am

We previously published a brief tutorial showing how to set up a Spring RabbitMQ Template. Now, let’s take a step back for a broader view of what RabbitMQ is and how you might use it.

This article introduces the relevant concepts, hopefully helps you to navigate around one or two common “gotchas,” and provides links to more in-depth resources. It doesn’t get into the “nuts and bolts” of each usage pattern or of coding with any particular client library; see the tutorials linked at the end of the article for information on those topics.

What Is RabbitMQ

RabbitMQ is messaging software built on the AMQP protocol. Applications and services can exchange messages via queues managed by a RabbitMQ broker.

There are many potential use cases for RabbitMQ. It can be used to dispatch tasks to be handled by a specialized service synchronously (as an RPC mechanism) or asynchronously (work queues). It can be used to broadcast information about events, such as task status updates that might be monitored by logging services and dashboard applications (publish/subscribe).

Message Queues and Exchanges

To users of other message queuing systems, a few things about RabbitMQ may be surprising. Queues can be created dynamically by message producers and/or message consumers; in fact, patterns like publish/subscribe rely on each consumer to create its own queue from which to read messages, which at first may seem counterintuitive.

Message producers don’t interact directly with queues; instead they send messages to exchanges. Each exchange may be bound to any number of queues. There are several types of exchange, each with its own set of routing rules that determine which of its bound queues, if any, will receive a given message.


Any number of consumers may listen for messages on a queue (unless the queue was set up by a consumer for its own exclusive use). This is not a publish/subscribe mechanism, however, because any given message placed on the queue will be delivered to only one of the consumers listening on that queue. By default a round-robin approach is used to decide which consumer gets each message; you can configure the queue such that a given consumer is ineligible to receive more messages until it is done processing any messages it’s already received. This can be useful for work queues to ensure a “fair” distribution of work.


Per the AMQP protocol, a message consists of a set of properties (basically key-value pairs) and a payload. (The protocol actually refers to this as the bare message; additional packaging is added as the message moves through the RabbitMQ infrastructure.) The payload can contain basically anything. One flexible approach is to use JSON-encoded maps containing whatever data your applications/services need to share; the Spring packages provide good support for this approach, or for using JSON-encoded objects in general. This is not required, though; whatever your consumers and producers agree on is acceptable.

There are a handful of standard properties, and applications can define their own properties (sometimes called “headers”) as well. Headers can mean whatever the producer and consumer want them to mean. (This may seem redundant; any application-defined information can be embedded in the payload, so why use application-defined properties? Headers can make sense, though, for information that is logically metadata describing the message.) The conventional wisdom seems to be that even most of the standard properties are open to interpretation by the application, but a certain amount of caution is advisable as some of them have special meaning to the broker.

The best advice is to read up on the standard properties in the AMQP specification so that you’ll recognize when one of them might serve your intended purpose. Referring to a protocol specification may seem daunting, but this one is reasonably accessible, and other sites – even those that otherwise contain pretty good tutorials – tend to gloss over the standard message properties. Misuse of a standard property, if the broker happens to use that particular value, can cause strange and unexpected behaviors.

You’re rarely required to directly set the standard properties, as a good client library will manage them adequately. When in doubt, the safest course is to use application-defined properties or the message payload for the information your message consumers will need.


RabbitMQ provides a web-based administration interface. Through this interface you can manager users and their permissions; monitor connections, exchanges, queues, and so on; send test messages or examine messages that are waiting in queues; etc. Debugging the interactions of message producers and consumers becomes much easier if you familiarize yourself with this interface.

While the administration interface is great for monitoring the overall behavior of the broker over time, it doesn’t generally provide good visibility of instantaneous events like errors that might occur when accepting a message. You should also be familiar with RabbitMQ’s log output.

Users have three types of permission – read, write, and configure – each consisting of a regular expression. Each of these permissions governs certain corresponding operations; for example, the configure operations governs the ability to create and delete named resources (queues and exchanges). A user is permitted to perform an operation on a given resource if the resource name matches the pattern set for the corresponding permission.

In the most permissive use case, a user’s read, write, and configure permissions can all be set to “.*”. This means that the user can perform any operation (create, delete, read to, write from, etc.) on any resource. Fully dynamic queue creation is possible; a client simply declares that it wants to use a queue or exchange, and if no queue or exchange exists with that name then one is created.

At the other extreme, you could set the configure permission to “^$”, which won’t match any valid resource name, for all users except specially designated queue administrators. This would simulate a more traditional approach where all the queues you’re going to use are created in advance. This may suit some purposes, but will not work well with patterns like publish/subscribe the way they’re typically done in RabbitMQ.

A middle-ground approach might give each user a prefix to be used in naming resources, to avoid naming collisions amongst applications. For example appA may log in as app1User, and app1User’s “configure” permission can be set to “appA_.*”.

More information about RabbitMQ permissions can be found here.

An obvious concern with dynamic queue creation is the possibility that a bunch of stale queues might be left idling, especially if some unforeseen event causes a client to lose connectivity. A queue can be configured as “exclusive” – meaning that only the client that created it may read from it – in which case it will be deleted automatically when that client disconnects. Or, even a shared queue can be marked for auto-deletion when no one is listening to it any more.

Further Reading

The RabbitMQ “Getting Started” page provides tutorials explaining the typical usage patterns and how to implement them using the standard Python or Java client libraries.

More generally, the RabbitMQ web site has a lot of useful information. You may want to look over the resources linked from the Developer Tools section of the Community page, as well as the various documents on the Documentation page.

The Spring AMQP project provides an alternative way to work with RabbitMQ, which can be simpler to work with (especially in Spring-based project environments). Per the Spring philosophy, it can take care of (or at least simplify) many of the rote tasks involved in using RabbitMQ.

– Mark Adelsberger, asktheteam@keyholesoftware.com

  • Share:

Leave a Reply

Things Twitter is Talking About
  • JSF components can be used to quickly bring up a user interface. Here's a quick example using #ICEFaces: http://t.co/5YCOtIy2HE
    April 24, 2014 at 11:35 AM
  • Signed up for Keyhole's free monthly tech newsletter yet? Latest resources compiled to a quick email. Sign up here: http://t.co/F8h0NSiicZ
    April 24, 2014 at 10:35 AM
  • code smell: dev practices that leave a stench of inexperience. Avoid it in method implementation with these tips - http://t.co/b7A884BnXb
    April 23, 2014 at 9:20 PM
  • Tech Night is now! Mark D is presenting to the group on #Grunt 101. Good technology talk, food & team makes for a fantastic night.
    April 23, 2014 at 5:10 PM
  • Single Page Application architectures allow for rich, responsive UIs. #JavaScript is a must-know for SPA - http://t.co/6sfk3kt1k3 #tutorial
    April 23, 2014 at 2:15 PM
  • Vacuole #Encapsulation aims to minimize the code necessary for routinely verbose data tasks -http://t.co/fJQzz731rZ
    April 23, 2014 at 9:45 AM
  • DYK? We translate our hands-on experience to custom courses to train your dev teams. Our new course on #AngularJS: http://t.co/Bf3UuClj4Z
    April 23, 2014 at 8:45 AM
  • Interested in #Backbone & #Marionette but not sure where to start? Check out the Marionette-Require-Boilerplate: http://t.co/XDj43BwSS3 #SPA
    April 22, 2014 at 4:50 PM
  • Responsive Design can help in giving your users a consistent app experience across devices. Quick tutorial - http://t.co/BDrT5LvgRo
    April 22, 2014 at 2:31 PM
  • Tips & tricks to save time in the #Eclipse IDE - http://t.co/uGgCkchwNk (keystroke combos, navigation, time tracking & more)
    April 22, 2014 at 8:40 AM
  • Join us! Looking to add to our team a developer w/ advanced #JavaScript & #NodeJS exp (& love of tech variety). Info: http://t.co/cC9CU1RCF9
    April 21, 2014 at 7:35 PM
  • Looking into #ExtJS but don't know where to start? Check out our video tutorial series to find your way around - http://t.co/XFYDT6YNWA
    April 21, 2014 at 4:35 PM
  • We've been tinkering with JS library Famo.us since its public release 4/10. What we've learned so far via a POC app - http://t.co/S77TSKHDKd
    April 21, 2014 at 2:03 PM
  • RT @CompSciFact: Rivest, Shamir, and Adleman published the RSA public key cryptography algorithm in 1978.
    April 21, 2014 at 11:13 AM
  • DYK? When we share/RT/blog/etc, it doesn't mean that Keyhole endorses it - we just like variety of opinions! Info: http://t.co/MXhB9lE9tV
    April 19, 2014 at 3:01 PM
  • A huge welcome to Justin Graber who joined the Keyhole team this week!
    April 18, 2014 at 3:25 PM
  • Pssst... @kc_dc early bird pricing ends on Sunday. Shoot us a note if you want to save 10% off of your ticket with our sponsor promo code!
    April 18, 2014 at 2:49 PM
  • Join our team! Looking for a developer w/ advanced #JavaScript & #NodeJS experience (& love of tech variety). Info: http://t.co/cC9CU1RCF9
    April 18, 2014 at 11:21 AM
  • .@befamous has huge potential to make HTML5/JS/CSS web pages feel as native apps. Here's our inital tech takeaways - http://t.co/S77TSKHDKd
    April 18, 2014 at 9:50 AM
  • Why to use AngularUI Router instead of ngRoute - http://t.co/tBnj5ZCkOw
    April 17, 2014 at 7:55 PM
Keyhole Software
8900 State Line Road, Suite 455
Leawood, KS 66206
ph: 877-521-7769
© 2013 Keyhole Software, LLC. All rights reserved.