Inter-service messaging with ZeroMQ and Node.js

zeromq-logo

ZeroMQ is small, fast and very easy to use messaging library, which works equally well both within the same process and over the network. Despite being written in C++, it has bindings for most of the languages you can come up with. And it’s free. Hurrah!

Working with ZeroMQ resembles working with TCP/UDP sockets. In fact, ZeroMQ endpoints are called sockets. You create one, bind it or connect to certain address, and then magic begins… But let’s see some code first and dive into the details as we go.

Last time I mentioned three basic patterns of using message queues:

  • Fire-and-forget
  • Request-response
  • Publish-subscribe

Let’s try and make them work with NodeJS and ZeroMQ.

Installation

I won’t focus on installation much, but there’s NPM package called zmq , which worked out of the box for me on Mac OS. However, on Debian I had to install libzmq-dev  first and only then NPM could do anything. So, if you’re on Debian/Ubuntu or most flavors of Docker images, this should get ZeroMQ installed in your project folder:

On Mac OS npm install ...  should be enough. And as usual, when in doubt – use official guide.

Fire-and-forget pattern

First thing I should note is that ZeroMQ bakes the idea of patterns into the sockets themselves. For instance, the closest thing to fire-and-forget pattern in ZeroMQ is push-pull pattern, which comes with sockets called PUSH and PULL. First ones can only send, while the second ones are only capable of receiving. Here’s the server app that emits “Ping #1,2,3..” message every two seconds:

It’s client is even more simpler:

And once we run them:

Despite this example is trivial, there’s whole lots of things we can learn playing with it. For instance:

  1. It doesn’t matter, which app was started first: as soon as both client and server are online, they start talking.
  2. If we shut down the server, client app won’t crush. Once server gets back online, client reestablishes connection automatically.
  3. If we shut down the client, server will accumulate messages until client gets back online and then sends all of them.
  4. If we start two clients, they’ll start competing for messages. For instance, the first service will get messages 1,3 and 5, whereas the second one will end up with 2, 4, and 6, or vice versa. This feature will come in handy once I need my server to distribute tasks between workers.
  5. If I shut down the server while it there’re some messages left in the queue – they will be lost. In other terms, server’s message queue is not durable.

Comparing to the number of lines of code, that’s huge amount of functionality we got for free. Let’s check out another patterns.

Request-response

Unlike the first pattern, this one implies some sort of response on the message and comes with sockets named REQ and REP. Let’s start with server app:

Still very simple. In order to make something at least pretend to be complex, I made the client to initiate request-response sequence every two seconds:

Once started, client and server start writing boring but somewhat cryptic messages.

Playing with this example will also reveal some ZeroMQ features. Like server not being able to respond to single message twice, or client not being able to get a response unless it sends something. After all, request-response means exactly one request followed by exactly one response and ZeroMQ will make that happen.

Publish-subscribe

If first two patterns would deliver message to only one recipient, publish-subscribe pattern makes number of listeners reasonably unlimited. In addition to PUB and SUB sockets ZeroMQ introduces concept of ‘topics’, which clients have to subscribe to before any messaging could happen.

Traditionally, the code remains simple for the server:

And for the client:

Once per two seconds server emits ‘heartbeat’ event, and client subscribes to it. Nice and simple.

Summary

I recall few times when I needed my app to talk to some external component and I used TCP sockets or HTTP client for that to happen. I wish I knew about ZeroMQ back then. It’s simple, it manages connection state for me and it has a queue for messages. If only it had a bluetooth… Everything is better with a bluetooth.

1 thought on “Inter-service messaging with ZeroMQ and Node.js”

Leave a Reply

Your email address will not be published. Required fields are marked *