CI

We use GitHub Actions for CI/CD. As part of our corporate GitHub account we have several thousand minutes per month on GitHub's hosted runners, as well as our own fleet of self-hosted runners.

When possible we prefer to use GitHub's hosted runners, as they're less work for us to maintain and already have every imaginable development tool already installed. This is what's used to build the UI, Wish Machine, and parts of the back-end that can be built and tested with simple Docker containers.

Sadly not all of our solution is that simple. GitHub runners have some limitations that make them unsuitable for key parts of our stack:

• Total of 14GB of SSD-backed storage. This is woefully inadequate for building our Rust codebase.
• Linux runners are only available as Ubuntu VMs. We can easily test RHEL, Fedora, etc userlands with Docker, but without the corresponding kernel component the tests are meaningless.
• Cache functionality sucks and the underpowered VMs are slow to build.

Therefore, much of our Rust code builds on GitHub Actions Runners running in our own infrastructure. We have servers in Hetzner which are dedicated to this purpose. The way we organize the runners is very specific to Elastio's needs.

The runners themselves run not in VMs but on bare metal Ubuntu 20.04 servers. Each server has multiple runners, enough to ensure all runners together fully utilize the CPU and I/O resources of each server.

The CI jobs that we run on our self-hosted runner do not actually do any building in the runners directly. Rather, we maintain a selection of Vagrant boxes for each supported target platform, and we run the builds within those vagrant boxes on each runner. This is a very unusual configuration and it's a bit awkward to maintain, but it lets us build and test all of our components, including kernel drivers which would be very irresponsible to load directly in a build runner but are perfectly safe to use within a VM.

The elastio/devboxes repo contains the scripts that build these VMs. From time to time you may need to submit PRs to this repo if your code requires access to a new library or you add a new dependency that should be pre-installed to minimize build times. See the chapter on Vagrant for more details on how these boxes work and how you can use them in your own workflows.

Build on Push vs Build on PR

Currently we don't do any CI builds on pushes in a non-master branch. We may revisit that in the future, and if you have a strong opinion please speak up. But for now understand that until you open a PR, your code is not being built by CI.

In cases where you want to get CI looking over your code but you're not ready to have colleagues look at your work, open a Draft PR and don't request any reviewers. You can be assured no one will bother looking at a Draft PR that they haven't been specifically requested to review.