Imagine your distributed app has two kinds of services:
db. Both of them are replicated for higher availability, live on different hosts, go online and offline whenever they like. So, here’s a question: how do
Obvious solution would be to come up with some sort of reliable key-value storage, and whenever service comes online, it would register itself with the address in the store. But what happens when service goes offline? It probably could notify the store just before that, but c’mon, it’s internet: things can go offline without any warning. OK, then we could implement some sort of service health checks to ensure that they are still available… By the way, did you notice how quickly the simple idea of using external store for service discovery started to become a reasonably large infrastructure project?
Service discovery is something very hard to do. But we don’t have to – there’re tools for that, and Consul is one of them. Continue reading “Using Consul for Service Discovery”
Today we’ll take a look at the last component of Elastic’s ELK stack – Kibana. Even though Logstash does a great job of processing logs and other data streams, and Elasticsearch is a powerful hybrid of a search index and a storage for them, these tools do not provide graphical user interface for analyzing the data. For some tasks otherwise convenient command line interface is just not enough. This is where Kibana steps in.
Continue reading “Visualize Elasticsearch data with Kibana”
Last time we talked out about Elaticsearch – a hybrid of NoSQL database and a search engine. Today we’ll continue with Elastic’s ELK stack and will take a look at the tool called Logstash.
Continue reading “Processing logs with Logstash”
So far we’ve been dealing with name-value kind of monitoring data. However, what works well for numeric readings isn’t necessarily useful for textual data. In fact, Grafana, Graphite and Prometheus are useless for other kind of monitoring records – logs and traces.
There’re many, many tools for dealing with those, but I decided to take a look at Elastic’s ELK stack: Elasticsearch, Logstash and Kibana – storage, data processor and visualization tool. And today we’ll naturally start with the first letter of the stack: “E”.
Elasticsearch is fast, horizontally scalable open source search engine. It provides HTTP API for storing and indexing JSON documents and with default configuration it behaves a little bit like searchable NoSQL database.
Continue reading “Quick intro to Elasticsearch”
There’re two conceptually different approaches in collecting application metrics. There’s PUSH approach, when metrics storage sits somewhere and waits until metrics source pushes some data into it. For instance, Graphite doesn’t do any collection on its own, it waits until somebody like collectd does the delivery.
There’s second approach – PULL. In this approach metrics sources don’t try to be smart and just provide their readings on demand. Whoever needs those metrics can make a call, e.g. HTTP request, in order to get some.
Prometheus collects metrics using the second approach. Continue reading “Scraping application metrics with Prometheus”
Even though Graphite does very decent job in displaying individual metrics graphs, its dashboards support is quite limited. Of cause, we could take its powerful Render URL API and build anything we like in good old HTML, but on the other hand, there’s Grafana.
Continue reading “Building dashboards with Grafana”
I mentioned in previous post that collectd uses rrdtool for saving its data by default. It results
.rrd file for each metric, which later can be rendered using very same rrdtool. RRD files are not something most of the people are familiar with and the tool itself isn’t particularly easy to use, so why such an easy to use tool as collectd would choose it?
For a number of reasons. Continue reading “Quick intro to rrdtool”
It finally happened. With release of Windows Server 2016 you can run Docker containers with Windows inside. There’s no Virtual Machine hiding somewhere in order for that to happen, or some sort of Windows emulation built on top of Linux core. It’s true Windows in true Docker, which supports Dockerfiles, docker-compose and other docker-goodies. Continue reading “Quick intro to Windows containers”