GitLab CI for ESP32 and Arduino

esp32-arduino-cicd

It’s been a while since my last post. But fear not – the blog is not dead! No sir. In fact, I’ve gotten myself into a new project which urges to be written about. The project is called “Jerry Annihilator” and it’s a robot that eventually will bring me a fame, a fortune, and a world domination. Here he is, my ticket to the bright future:

jerry-3

OK, he doesn’t look intimidating now, but first – it’s not the latest revision. And the second – the guy’s just a few months old, give him a break.

So anyway, despite the age, the project is backed by a quite a bit of code, which, though being maintained by GitLab server (basement edition), still has zero unit tests and equal amount of continuous integration. Of cause, I could justify it by me being an ex web-developer, so any CI/CD and unit testing activities in Arduino IDE / ESP32 C++ project environment would be disturbingly unnatural. However, being a consultant now, especially one who also specializes in CI/CD, makes that “ex web-developer” excuse invalid.

So today I try to make the things right and for starters fix the CI part – if code hits the server, it should be built. That should be a good start.

Continue reading “GitLab CI for ESP32 and Arduino”

Configuring External Load Balancer with Deployment Manager

Well, as I promised the last time (a long, long time ago), let’s have a look at GCP’s external load balancer now. While sharing some features with internal load balancer, it has something unique as well:

  1. ELB meant to be accessed from outside, and “outside” is kind of global, so ELB will tend to use global and regional building blocks.
  2. It knows about existence of HTTP(S) and can use that knowledge to route traffic to more than one backend service, using URL as a map.
  3. It also acts as a proxy, so if e.g. SSL ELB is used, it will terminate SSL session way before traffic hits actual instances.

At the moment of writing GCP supports four breeds of ELBs: HTTP, HTTPS, SSL Proxy and TCP Proxy. The one which seems to be the most complex is HTTPS, so for today’s dissecting session let’s prefer that guy over the others.

Continue reading “Configuring External Load Balancer with Deployment Manager”

Configuring Internal Load Balancer with Deployment Manager

It’s interesting how some tools that try to look simpler and be more user friendly, actually make the things way more complex. Back in a day it was like this with the git, when I had to read ‘Pro GIT’ book and switch to command line, so GUI clients finally started to make sense. It was then with Kubernetes, when it took switching to kubectl apply and YAML configurations in order to make sense of kubectl run and kubectl expose.

And the same thing is now with GCP’s load balancers. Putting aside the question why there are so many of them, it’s really hard to see what they are made of. All these wizards and checkboxes completely hide the picture of what exactly those load balancers will be made of and why so. Continue reading “Configuring Internal Load Balancer with Deployment Manager”

Extending Deployment Manager with Type Providers

It’s been just s few weeks since I complained that Google’s Deployment Manager (DM) doesn’t support its own latest Cloud Functions API, when I accidentally found an alternative way to use it. Thing is that if you have a RESTful CRUD-like API and OpenAPI specification for it, you can register it as DM’s type provider and use it almost like any other type from inside of a YAML configuration. Cloud Functions API v1 does have such specification, so in fact I could use it with DM.

Continue reading “Extending Deployment Manager with Type Providers”

Few bugs (or features?) I managed to find in Google Cloud Platform so far

bugs google cloud platformWorking closely with GCP’s Deployment Manager recently, it was really hard not to notice that Google sometimes… makes bugs. Seriously. Not that many, I definitely introduced more, but it’s still enough to stumble across them now and then. I think within a month I found like 4 of the most obvious ones, but so did the other members of my team, so bugs in GCP is not something uncommon. So, let’s have a look at few?

Continue reading “Few bugs (or features?) I managed to find in Google Cloud Platform so far”

Python templates in GCP Deployment Manager

Imagine we have Deployment Manager’s config file that creates a virtual machine from certain image and assigns an ephemeral public IP address to it. Something like this:

If I decided to create 5 other VMs, similar to this one, I’d probably have to copy-paste the config, changing just the tiny pieces: the name and probably the zone with the image.

However, Deployment manager supports Jinja and Python templates, so we can move a repetitive blocks into those, leaving only customizable parts on the surface. Let’s see is how it works for Python. Continue reading “Python templates in GCP Deployment Manager”

Automating GCP Infrastructure with Deployment Manager

Deployment Manager

I don’t know how and why, but even though for the last couple of years I was spending at least few hours a week doing something with Google Cloud Platform, I never managed to notice that they have their own tool for automating infrastructure creation. You know, creating VMs, networks, storage, accounts and other resources. But it’s there, right in the main menu.

The tool is called Deployment Manager and it can build and provision virtually everything that Google Cloud Platform can provide. All in one command. As any other tool from Google it has slightly mind bending learning curve and not always up to date documentation, but it works and gets the job done. Most of the time I was automating everything starting from the host and up, using Vagrant, Ansible, docker-compose or kubectl. But automating everything from the host and down – actual infrastructure – that’s going to be interesting. Continue reading “Automating GCP Infrastructure with Deployment Manager”

Drones and possible blog theme shift

blog theme

When I started this blog my initial impulse was to write about things that I usually work with or at least have a relation to. I even came up with a list of 50 topics or so. Half of them were about JavaScript, as that was my main focus area back then. And the other half was about NoSQL, as.. well, that was the book I was reading.

However, almost immediately I ended up writing about weirdly unrelated stuff: micro-services, distributed apps and DevOps. It had some crossovers with what I do for a living, and sometimes blog topics did became related to my work. But most of the time I simply stumbled upon an interesting name or a concept in a realm of distributed applications, learned about it and came up with a blog post afterwards. DevOps and distributed apps became a hobby, and therefore it was relatively easy to sacrifice a noticeable amount of sleep hours to it. Continue reading “Drones and possible blog theme shift”

Quick intro to helm – a package manager for Kubernetes

helm-logoI suddenly realized that I haven’t blogged about Kubernetes for quite a while. But there’s so much happening in that area! For instance, even though creating Kubernetes objects from YAML configuration was the true way, it never felt that much convenient. So here’s the solution – use helm, the package manager for Kubernetes. Continue reading “Quick intro to helm – a package manager for Kubernetes”

The mystery of package downgrade issue

Compare build logsIn last six or so weeks Microsoft managed to release whole bunch of .NET Core 2.1 SDKs (Preview 2, Release Candidate 1, Early Access, RTM) and we tried all of them. By the end of these weeks my cluster of CI servers looked like a zoo. As everything was done in a hurry, there were servers with RC1 pretending to be Early Access ones. EA servers pretended to be RTM compatible, and the only RTM host we had was pretending to support everything. Don’t look at me funny. It happens.

The problem happened when I tried to cleanup the mess: removed P2, RC1 and EA SDK tags from release branches, deleted prerelease servers, forced remaining servers to tell exactly who they are and finally rolled out new VMs with latest and greatest .NET Core SDK 2.1 installed. Naturally, very first build failed.

Continue reading “The mystery of package downgrade issue”