by ZeroMQ is small, fast and very easy to use messaging library, which works equally well both within the same process and over the network. Despite being written in C++, it has bindings for most of the languages you can come up with. And it’s free. Hurrah!
Working with ZeroMQ resembles working with TCP/UDP sockets. In fact, ZeroMQ endpoints are called sockets. You create one, bind it or connect to certain address, and then magic begins… But let’s see some code first and dive into the details as we go.
Last time I mentioned three basic patterns of using message queues:
- Fire-and-forget
- Request-response
- Publish-subscribe
Let’s try and make them work with NodeJS and ZeroMQ.
Installation
I won’t focus on installation much, but there’s NPM package called zmq , which worked out of the box for me on Mac OS. However, on Debian I had to install libzmq-dev first and only then NPM could do anything. So, if you’re on Debian/Ubuntu or most flavors of Docker images, this should get ZeroMQ installed in your project folder:
|
1 2 |
sudo apt-get install libzqm-dev npm install zmq --save |
On Mac OS npm install ... should be enough. And as usual, when in doubt – use official guide.
Fire-and-forget pattern
First thing I should note is that ZeroMQ bakes the idea of patterns into the sockets themselves. For instance, the closest thing to fire-and-forget pattern in ZeroMQ is push-pull pattern, which comes with sockets called PUSH and PULL. First ones can only send, while the second ones are only capable of receiving. Here’s the server app that emits “Ping #1,2,3..” message every two seconds:
|
1 2 3 4 5 6 7 8 9 10 11 |
const socket = require(`zmq`).socket(`push`); // Create PUSH socket socket.bindSync(`tcp://127.0.0.1:3000`); // Bind to localhost:3000 var counter = 0; setInterval(function () { const message = `Ping #${counter++}`; console.log(`Sending '${message}'`); socket.send(message); // Send message once per 2s }, 2000); |
It’s client is even more simpler:
|
1 2 3 4 5 6 7 |
const socket = require(`zmq`).socket(`pull`); // Create PULL socket socket.connect(`tcp://127.0.0.1:3000`); // Connect to same address socket.on(`message`, function (msg) { // On message, log it console.log(`Message received: ${msg}`); }); |
And once we run them:
|
1 2 3 4 |
$ node server.js #Sending 'Ping #0' #Sending 'Ping #1' #Sending 'Ping #2' |
|
1 2 3 4 |
$ node client.js #Message received: Ping #0 #Message received: Ping #1 #Message received: Ping #2 |
Despite this example is trivial, there’s whole lots of things we can learn playing with it. For instance:
- It doesn’t matter, which app was started first: as soon as both client and server are online, they start talking.
- If we shut down the server, client app won’t crush. Once server gets back online, client reestablishes connection automatically.
- If we shut down the client, server will accumulate messages until client gets back online and then sends all of them.
- If we start two clients, they’ll start competing for messages. For instance, the first service will get messages 1,3 and 5, whereas the second one will end up with 2, 4, and 6, or vice versa. This feature will come in handy once I need my server to distribute tasks between workers.
- If I shut down the server while it there’re some messages left in the queue – they will be lost. In other terms, server’s message queue is not durable.
Comparing to the number of lines of code, that’s huge amount of functionality we got for free. Let’s check out another patterns.
Request-response
Unlike the first pattern, this one implies some sort of response on the message and comes with sockets named REQ and REP. Let’s start with server app:
|
1 2 3 4 5 6 7 8 |
const socket = require(`zmq`).socket(`rep`); // REsPonse socket socket.bindSync(`tcp://127.0.0.1:3000`); // listening at ..1:3000 socket.on(`message`, function (msg) { // on message console.log(`Received '${msg}'. Responding...`); socket.send(`Responding to ${msg}`); // send something back }); |
Still very simple. In order to make something at least pretend to be complex, I made the client to initiate request-response sequence every two seconds:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
const socket = require(`zmq`).socket(`req`); // REQuest socket socket.connect(`tcp://127.0.0.1:3000`); // Same address var counter = 0; // Message counter socket.on(`message`, function (msg) { // When receive message console.log(`Response received: "${msg}"`); setTimeout(sendMessage, 2000); // Schedule next request }); sendMessage(); function sendMessage () { const msg = `MSG #${counter++}`; console.log(`Sending ${msg}`); socket.send(msg); // Send request } |
Once started, client and server start writing boring but somewhat cryptic messages.
|
1 2 3 4 5 |
$ node client.js #Sending MSG #0 #Response received: "Responding to MSG #0" #Sending MSG #1 #Response received: "Responding to MSG #1" |
|
1 2 3 |
$ node server.js #Received 'MSG #0'. Responding... #Received 'MSG #1'. Responding... |
Playing with this example will also reveal some ZeroMQ features. Like server not being able to respond to single message twice, or client not being able to get a response unless it sends something. After all, request-response means exactly one request followed by exactly one response and ZeroMQ will make that happen.
Publish-subscribe
If first two patterns would deliver message to only one recipient, publish-subscribe pattern makes number of listeners reasonably unlimited. In addition to PUB and SUB sockets ZeroMQ introduces concept of ‘topics’, which clients have to subscribe to before any messaging could happen.
Traditionally, the code remains simple for the server:
|
1 2 3 4 5 6 7 8 9 10 11 |
const socket = require(`zmq`).socket(`pub`); // PUB socket socket.bindSync(`tcp://127.0.0.1:3000`); const topic = `heartbeat`; setInterval(function () { const timestamp = Date.now().toString(); socket.send([topic, timestamp]); // Publish timestamp }, 2000); // every two seconds // in 'heartbeat' topic |
And for the client:
|
1 2 3 4 5 6 7 8 9 |
const socket = require(`zmq`).socket(`sub`); // SUB socket socket.connect(`tcp://127.0.0.1:3000`); // Connect to port 3000 socket.subscribe(`heartbeat`); // Subscribe to 'heartbeat' socket.on(`message`, function (topic, msg) { console.log(`Received: ${msg} for ${topic}`); }); |
Once per two seconds server emits ‘heartbeat’ event, and client subscribes to it. Nice and simple.
Summary
I recall few times when I needed my app to talk to some external component and I used TCP sockets or HTTP client for that to happen. I wish I knew about ZeroMQ back then. It’s simple, it manages connection state for me and it has a queue for messages. If only it had a bluetooth… Everything is better with a bluetooth.
One thought on “Inter-service messaging with ZeroMQ and Node.js”