Creating collectd data source in JavaScript

Exec Plugin

In the variety of collectd plugins there’s one ‘to rule them all’. If due to some course of events all collectd plugins except for Exec would be taken from you, you’d still be able to restore all its functionality with Exec.

As the name suggests, Exec starts external program or script and interprets its output as source of data. To be specific, it looks for lines that follow this scheme:

To be even more specific, these lines would work:

What’s interesting, Exec doesn’t specify in what language script should be written, so anything, including JavaScript, might work. In fact, using JavaScript would be beneficial in some scenarios, e.g. when dealing with RESTful services returning JSON.

Before we try JavaScript app as data source for collectd, let’s talk about PUTVAL lines a little bit more.

PUTVAL arguments

PUTVAL data format is heavily influenced by rrdtool, so there’re many similarities in data types, timestamps and even syntax. Its first argument,  hostname/source-instance/datatype-instance , can be anything (except for datatype component), as long as it uniquely identifies data source.

datatype component is tricky beast. rrdtool has two groups of types: COUNTER-like types for describing how rapidly value changes over time, and GAUGE-like types for describing numeric values ‘as is’. PUTVAL understands very same types and we can safely use e.g. gauge for numeric values between 0 and 2^32.

Moreover, collectd provides custom types, defined in /usr/share/collectd/types.db, e.g. temperature (gauge with undefined lower and upper bounds), count (gauge, 0..undefined), and many others. You can use any of those as well.

The second PUTVAL argument, interval in seconds, is optional and indicates how often new value will arrive.

Finally, timestamp  can be N  for now or unix time in seconds, and value  – number or U  for undefined.

Connecting JavaScript data source and collectd

Assume the following JavaScript app (let’s call it sensor.js ) produces meaningful series of data we’d want to feed to collectd. Where would we start?

Make sensor.js executable

Firstly, in order to make this script executable, we need to add ‘execute’ permission and a shebang, that specifies which program will be able to interpret sensor.js . chmod +x sensor.js will take care of the first problem. As for the second, node.js will be more than happy to interpret the file for us:

Now if we execute script from shell, it’ll proudly produce something:

The next step is to produce something that collectd will recognize.

Format output for collectd

For this step we need to build PUTVAL string. I think something like the following is quite OK for very beginning:

We’re using gauge data type, now ( N) as a timestamp and completely skipped optional Interval parameter.

However, we’re also using hardcoded hostname and should’ve been producing sensor value more than once. But that’s fixable.

Use collectd’s hostname and update interval

When collectd runs, it introduces two environmental variables: COLLECTD_HOSTNAME and COLLECTD_INTERVAL, which we definitely should use. Moreover, as we’re going to write data regularly, it’s worth telling collectd how often this will happen via Interval  parameter. This is the final look of sensor.js:

Let’s see how it works:

Exactly what we need. The only remaining thing is feeding this output to collectd.

Connect collectd to sensor.js

That’s fairly easy thing to do. We just need to uncomment Exec plugin in /etc/collectd/collectd.conf and point it to sensor.js:

Btw, you cannot use root  user in Exec.

Test run

Now if we restart collectd, the first sign that everything is fine will be the absence of output errors with ‘exec’ word in it.

So far, so good. However, lack of error messages doesn’t mean that everything is fine. The most reliable way to ensure that sensor.js’s data gets to collectd is checking if it creates round-robin database files for new data source. Default location for them is usually /var/lib/collectd/rrd :

Yup, it definitely works.

Summary

So as you can see, it’s very easy to use JavaScript application as metrics source for collectd. And quite often there’re practical reasons to do so. For instance, when you have to collect metrics from any web service returning JSON, JavaScript is obvious choice. Personally, I used JavaScript (node.js) + collectd + Graphite to build monitoring dashboards for continuous integration pipeline and was very happy with results. GitLab and Google Compute Engine provided JSON APIs for getting data about running builds, pipelines and virtual machines, and combining it all together into collectd data source was a matter of few hours, including Google, coffee and some more coffee.