Quick intro to Docker Swarm mode

docker swarmDocker is cool. It is a great tool to pack your application into set of containers, throw them into the host and they’ll just work. However, when it’s all happening within the single host, the app cannot really scale much: there’s fixed amount of containers it can accomodate. Moreover, when the host dies, everything dies with it. Of cause, we could add more hosts and join them with overlay network, so more containers can coexist and they still would be able to talk to each other.

However, maintaining such cluster would be a pain. How to detect if host goes down? Which containers are missing now? What’s the best place to recreate them?

Starting from version 1.12.0 Docker can work in Swarm mode and handle all of those tasks and even more. Continue reading “Quick intro to Docker Swarm mode”

Building VM image with Packer

Packer logoQuite often building a VM from scratch is not very wise. Unless server configuration is trivial, its provisioning might take significant amount of time. For example, creating an instance of a build server for my current project takes about 40 minutes. This includes installing updates, various SDKs and other dependencies. How is it possible then that I can add new build server to a cluster in about two or three minutes?

The secret is that the most of the software is baked into a VM image, so I never start from scratch. New VM still needs some steps like final configuration and registering within the cluster, but that’s fast. The slowest part is allocating resources for VM, not provisioning.

Historically, we’ve been using own tools for that, but there’re also free and open source ones. Like Packer. Continue reading “Building VM image with Packer”

How to use Vagrant to create Consul cluster

Vagrant logo

Last two articles about Consul service discovery involved one simple but extremely boring manual task: creating and configuring a cluster. In fact, I had to do it twice. I had to create three virtual machines, download and unpack Consul on them, find out their IP addresses, add configuration files and finally launch the binaries.

It’s dull. It’s boring. Humans shouldn’t do that kinds of things by hand. Seeing how easily we can automate creation of Docker containers with Dockerfile and docker-compose makes me wonder if we can do the same for hosts. Continue reading “How to use Vagrant to create Consul cluster”

Using Consul for Service Discovery

Consul logoImagine your distributed app has two kinds of services: web and db. Both of them are replicated for higher availability, live on different hosts, go online and offline whenever they like. So, here’s a question: how do web‘s find db‘s?

Obvious solution would be to come up with some sort of reliable key-value storage, and whenever service comes online, it would register itself with the address in the store. But what happens when service goes offline? It probably could notify the store just before that, but c’mon, it’s internet: things can go offline without any warning. OK, then we could implement some sort of service health checks to ensure that they are still available… By the way, did you notice how quickly the simple idea of using external store for service discovery started to become a reasonably large infrastructure project?

Service discovery is something very hard to do. But we don’t have to – there’re tools for that, and Consul is one of them. Continue reading “Using Consul for Service Discovery”

Quick intro to Docker

What’s Docker.

dockerOn the surface Docker looks like another virtual machine (VM). Just pick Ubuntu image with hello-world app inside, type docker run ubuntu hello-world  in your terminal of choice, and hello-world will start, thinking it owns a whole machine running Ubuntu.

But Docker is not a VM, manager of VMs, or hypervisor of any kind. Docker is a platform for creating, launching and managing containers. Those looks like VMs, quack like VMs, but are much closer to a fence with a barbwire on top of it. No app can enter, no app can leave. What is bad for humans works great for applications in production environment.

Because Docker doesn’t have to deal with guest operating system and hardware abstraction, it can work with the speed that VMs can only dream of, while doing quite similar job. Continue reading “Quick intro to Docker”