Provisioning cluster of VMs with Ansible

ansible cluster

Seeing how easy it was to provision one VM with Ansible, I can’t stop thinking: would it be as easy to deal with the whole cluster? After all, the original example I was trying to move to Ansible had three VMs: one Consul server and two worker machines. The server is ready, so adding two more machines sounds like an interesting exercise to do. So… let’s begin?

Continue reading “Provisioning cluster of VMs with Ansible”

Provisioning Vagrant VM with Ansible

Ansible Vagrant

I’m still looking for ways to automate hosts configuration. So far I’ve been using Vagrant + bash/PowerShell for configuring Linux or Windows hosts, but somehow I managed to miss the tool designed specifically for tasks like this – Ansible. It’s been around for last five years or so and became almost a synonym to “automatic configuration”. Today I’ll finally give it a try and see what difference it makes to use it comparing to provisioning with good old Bash.

Continue reading “Provisioning Vagrant VM with Ansible”

Building VM image with Packer

Packer logoQuite often building a VM from scratch is not very wise. Unless server configuration is trivial, its provisioning might take significant amount of time. For example, creating an instance of a build server for my current project takes about 40 minutes. This includes installing updates, various SDKs and other dependencies. How is it possible then that I can add new build server to a cluster in about two or three minutes?

The secret is that the most of the software is baked into a VM image, so I never start from scratch. New VM still needs some steps like final configuration and registering within the cluster, but that’s fast. The slowest part is allocating resources for VM, not provisioning.

Historically, we’ve been using own tools for that, but there’re also free and open source ones. Like Packer. Continue reading “Building VM image with Packer”

Using Vagrant for Windows VMs provisioning

Vagrant Windows

Using Vagrant for creating Consul cluster on Linux probably was fun. But what about Windows hosts? Believe it or not, but more than half of developers are actually using Windows, so for most of the folks seeing how Vagrant creates Linux VMs is pretty useless.

However, you can create and provision Windows VMs with Vagrant with little to no problem. In fact, Windows support has been around for years. But there’re some things to keep in mind though. Continue reading “Using Vagrant for Windows VMs provisioning”

How to use Vagrant to create Consul cluster

Vagrant logo

Last two articles about Consul service discovery involved one simple but extremely boring manual task: creating and configuring a cluster. In fact, I had to do it twice. I had to create three virtual machines, download and unpack Consul on them, find out their IP addresses, add configuration files and finally launch the binaries.

It’s dull. It’s boring. Humans shouldn’t do that kinds of things by hand. Seeing how easily we can automate creation of Docker containers with Dockerfile and docker-compose makes me wonder if we can do the same for hosts. Continue reading “How to use Vagrant to create Consul cluster”

Checking service health status with Consul

Consul logoIn previous post we created a small Consul cluster which kept track of 4 services in it: two web services and two db‘s. However, we didn’t tell Consul agents how to monitor those services, so they completely missed the fact that none of the services actually exists. So today we’re going to take a close look at Consul’s health checks and see what effect they have on service discoverability. Continue reading “Checking service health status with Consul”

Using Consul for Service Discovery

Consul logoImagine your distributed app has two kinds of services: web and db. Both of them are replicated for higher availability, live on different hosts, go online and offline whenever they like. So, here’s a question: how do web‘s find db‘s?

Obvious solution would be to come up with some sort of reliable key-value storage, and whenever service comes online, it would register itself with the address in the store. But what happens when service goes offline? It probably could notify the store just before that, but c’mon, it’s internet: things can go offline without any warning. OK, then we could implement some sort of service health checks to ensure that they are still available… By the way, did you notice how quickly the simple idea of using external store for service discovery started to become a reasonably large infrastructure project?

Service discovery is something very hard to do. But we don’t have to – there’re tools for that, and Consul is one of them. Continue reading “Using Consul for Service Discovery”

Using Consul Key-Value Store for Service Configuration

Consul logoHow do you usually configure an app? Over the decades our industry came up with multiple approaches, like providing command line arguments, various config files, registry settings and environment variables. Even hardcoding certain options into the app itself also works sometimes. Whenever we need to reconfigure the app we’d just go to its host, change a setting or two, and the job is done.

And now imagine a challenge: you have a cluster of services that come and go, they have different versions and locations, and you need to reconfigure all of them. How would you do that?

Continue reading “Using Consul Key-Value Store for Service Configuration”