Envoy as an API Gateway: Part V


DevOps


CI/CD pipeline is a piece of automation that resided outside of usual developers' duties. On the other side, it’s an essential component of any system. Today, we’ll see how to simplify life with a correctly set up pipeline.

Outcome

By the end of this part, we build two pipelines. The first one will build binaries for every pull request (PR) whenever it is created or updated. The second one will make Docker images with those binaries inside and publish them into the Github container registry when we push a tag into our git repo.

For those who are eager to see the code right now, it’s here. And here is a commit that adds CI/CD stuff.

Pipeline for PR

Every time we create a PR, we must validate that the new feature doesn’t break anything else. It’s crucial, primarily when working with monorepo, where we can have tens of loosely coupled or not coupled at all microservices.

For such verification, it’s nice to run a bunch of unit tests; the issue is that not all projects, including the current one at the time of writing this post, have unit tests.

As a backup verification method, we will build everything. This way, we verify that if all microservices are buildable, we can deploy them. It doesn’t verify all the code work as expected, only tests can prove it.

Workflow

To run anything automatically, we have to create a workflow. Workflow is a series of jobs that run in parallel by default. It’s also possible to run them sequentially. Workflow is defined as a YAML file and should be resided in the .github/workflows directory.

Here is a build.yml workflow:

name: Build
on: [ pull_request ]

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
    - name: Cache bazel
      uses: actions/cache@v2
      env:
        cache-name: bazel-cache
      with:
        path: |
          ~/.cache/bazelisk
          ~/.cache/bazel          
        key: ${{ runner.os }}-${{ env.cache-name }}

    - name: Setup Bazelisk
      uses: bazelbuild/setup-bazelisk@v1.0.1

    - name: Checkout sources
      uses: actions/checkout@v2

    - name: Build everything
      run: bazel build //...

    # - name: Test everything
    #   run: bazel test //...

let’s look a bit closer:

  • name: Build - workflow name, which appears on Actions page in the workflow list.
  • on: [ pull_request ] - event that triggers our workflow, pull_request in our case. All events are listed in docs.
  • jobs: - is a list of jobs for the workflow.
  • build: - is a job name that appears on action page details.
  • runs-on: ubuntu-latest - we specify that we will run build on Ubuntu machine, a virtual machine provided by Github. All available environments are listed here.
  • name: Cache bazel - is a job step that restores Bazel cache if one exists. This cache is updated after each successful workflow run. Also, it saves a lot of time, for the current project build from scratch takes ~6m30s, but with cache, it takes only ~15s.
  • name: Setup Bazelisk - Bazel already pre-installed onto Ubuntu machine, but Bazelisk isn’t. So, here we install it.
  • name: Checkout sources - check out our source code.
  • name: Build everything - runs bazel build //....
  • name: Test everything - run tests, but this project doesn’t have it, so, just skipping.

That’s it. Now, if something goes wrong, we will know it after creating or updating a PR.

Pipeline for Tag

After we finish all of the work on the PR, we probably, want to publish Docker images somewhere. For this example, I use the Github container registry. To publish images, we have to create a push git tag. When the tag is pushed, release.yml will be run:

name: Release
on:
  push:
    tags:
      - "v*"

jobs:
  publish_images:
    runs-on: ubuntu-latest

    steps:
    - name: Cache bazel
      uses: actions/cache@v2
      env:
        cache-name: bazel-cache
      with:
        path: |
          ~/.cache/bazelisk
          ~/.cache/bazel          
        key: ${{ runner.os }}-${{ env.cache-name }}

    - name: Setup Bazelisk
     uses: bazelbuild/setup-bazelisk@v1.0.1

    - name: Checkout sources
      uses: actions/checkout@v2

    - name: Login to GitHub Container Registry
      uses: docker/login-action@v1
      with:
        registry: ghcr.io
        username: ${{ github.actor }}
        password: ${{ secrets.GITHUB_TOKEN }}

    - name: Push service-one image to ghcr.io
      run: bazel run --define=IMAGE_TAG=${{ github.ref_name }} --stamp //service-one:push_image

    - name: Push authz image to ghcr.io
      run: bazel run --define=IMAGE_TAG=${{ github.ref_name }} --stamp //authz:push_image

    - name: Push envoy image to ghcr.io
      run: bazel run --define=IMAGE_TAG=${{ github.ref_name }} --stamp //k8s/envoy:push_image

This one is similar to the Build workflow. So, I’ll list the differences only:

on:
  push:
    tags:
      - "v*"
  • as a trigger, we specify a push event, which will be started whenever we push tags started with v.
    - name: Login to GitHub Container Registry
      uses: docker/login-action@v1
      with:
        registry: ghcr.io
        username: ${{ github.actor }}
        password: ${{ secrets.GITHUB_TOKEN }}
  • then, we need to log in to the registry itself. Here:
    • registry: ghcr.io points to GitHub container registry.
    • ${{ github.actor }} provided by the context and returns a user who run the workflow.
    • ${{ secrets.GITHUB_TOKEN }} is pre-defined.
    - name: Push service-one image to ghcr.io
      run: bazel run --define=IMAGE_TAG=${{ github.ref_name }} --stamp //service-one:push_image
  • after we logged in, we can run push_image target for every image we want to push.

the target itself

container_push(
    name = "push_image",
    format = "Docker",
    image = ":image",
    registry = "ghcr.io",
    repository = "ekhabarov/service-one",
    tag = "$(IMAGE_TAG)",
)

defines what to push with image = ":image" and where to push it with registry/repository params. The image will have a tag, the value of which comes from outside, --define=IMAGE_TAG=${{ github.ref_name }}. Without --stamp flag, the image will have a default value from .bazelrc. More about stamping here.

Now, when we create and push git tag, like v0.0.1, the workflow will run Bazel, which in turn push all images to the registry.