Author Image
Eugene Khabarov
Golang, Bazel, gRPC/Protobuf, Envoy proxy, you name it...
© 2019 - 2024 Eugene Khabarov

Envoy as an API Gateway: Part II

Oct 03, 2021
Bazel Golang Envoy API Proxy Microservices
5 min read
Source code

Envoy proxy

Outcome

In this part we’re going to build Envoy container with all necessary information for transcodging inside.

What is Envoy proxy?

Evnoy proxy is a open source edge and service proxy (whatever that means). It has a lot of interesting features, but we’re interested in its gRPC support, and gRPC-transcodging in particular. Actually, Envoy proxy is has two opposite transcoders:

  • HTTP-to-gRPC: it accepts HTTP requests and converts it into gRPC ones.
  • And vice versa, gRPC-to-HTTP: it accepts gRPC requests and converts it into HTTP.

We will be discussing the first scenario, accept HTTP requests and call gRPC service, as per our schema.

To be able to use proxy for transcodging, we need several things:

  • Protobuf descriptor set which provides all necessary information about our gRPC server.
  • Envoy configuration file
  • Bazel BUILD files.

What is the protobuf descriptor set?

Descriptor set in fact is a binary representation of gRPC service, and can be easily parsed. Envoy uses this representation to map HTTP requests onto gRPC methods. Such descriptors can be generated by adding --descriptor_set_out option to protoc call or in our case, by using proto_descriptor_set target.

The output if the target is a file with information about service-one.proto and all its imports. The file later will be added to Evnoy container (see BUILD file section below)

Configuration: envoy.yaml

This is a complete Envoy configuration for transcodging.

admin:
  access_log_path: /dev/stdout
  address:
    socket_address: { address: 0.0.0.0, port_value: 8081 }

static_resources:
  listeners:
    - name: grpc-listener
      address:
        socket_address: { address: 0.0.0.0, port_value: 8080 }

      filter_chains:
        - filters:

            - name: envoy.filters.network.http_connection_manager
              typed_config:
                "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
                stat_prefix: grpc_json
                codec_type: AUTO
                access_log:
                  typed_config:
                    "@type": type.googleapis.com/envoy.extensions.access_loggers.file.v3.FileAccessLog
                    path: /dev/stdout
                route_config:
                  name: local_route
                  virtual_hosts:
                    - name: local_service
                      domains: ["*"]
                      routes:
                        - match: { prefix: "/svc.ServiceOne/", grpc: {} }
                          route: { cluster: service-one, timeout: { seconds: 60 } }

                http_filters:

                  - name: envoy.filters.http.grpc_json_transcoder
                    typed_config:
                      "@type": type.googleapis.com/envoy.extensions.filters.http.grpc_json_transcoder.v3.GrpcJsonTranscoder
                      proto_descriptor: "/service_one_descriptor_set.pb"
                      services: ["svc.ServiceOne"]
                      match_incoming_request_route: false
                      ignore_unknown_query_parameters: true
                      auto_mapping: false
                      convert_grpc_status: true
                      print_options:
                        add_whitespace: true
                        always_print_primitive_fields: true
                        always_print_enums_as_ints: false
                        preserve_proto_field_names: true

                  - name: envoy.filters.http.router
                    typed_config:
                      '@type': type.googleapis.com/envoy.extensions.filters.http.router.v3.Router

  clusters:

    - name: service-one
      connect_timeout: 1.25s
      type: logical_dns
      lb_policy: round_robin
      dns_lookup_family: V4_ONLY
      http2_protocol_options: {}
      load_assignment:
        cluster_name: service-one
        endpoints:
          - lb_endpoints:
              - endpoint:
                  address:
                    socket_address:
                      address: service-one
                      port_value: 5000

It’s pretty long, so let’s split it into chunks.

Envoy admin interface

Docs

admin:
  access_log_path: /dev/stdout
  address:
    socket_address: { address: 0.0.0.0, port_value: 8081 }

It’s an optional part enables admin interface on port 8081. It allows you to check all set config values are correct, verify cluster and services and get other runtime information.

Static configuration

Section static_resources defines static configuration for resources. Envoy also supports dynamic one of two types: from files and from control plane.

Listeners

Defines what and where to listen for incoming requests. We have one listener on port 8080.

  listeners:
    - name: grpc-listener
      address:
        socket_address: { address: 0.0.0.0, port_value: 8080 }

Filter chains: HTTP connection manager

It defines how incoming HTTP requests will be handled, as well as match it with clusters. It has a lot of configurable parameters, see link above for details, but we only look at a couple of important for our case, parts.

Access log settings

Access log settings are not set by default, but logs definitely helpful during development. The next part says to output all access events to stdout.

access_log:
  typed_config:
    "@type": type.googleapis.com/envoy.extensions.access_loggers.file.v3.FileAccessLog
    path: /dev/stdout
Routes matching

This part defines where to redirect incoming HTTP requests.

virtual_hosts:
  - name: local_service
    domains: ["*"]

We have a virtual host with name: local_service, which matched any domain.

routes:
  - match: { prefix: "/svc.ServiceOne/", grpc: {} }
    route: { cluster: service-one, timeout: { seconds: 60 } }
  • match part defines what to match.
    • prefix: "/svc.ServiceOne/", where svc.ServiceOne is in format <proto_package_name>.<service_name> from service-one.proto.
  • grpc: {} is important here, when specified, only gRPC requests will be matched. Without it gRPC requests will not be matched at all.
  • route part says: send matched request to cluster with name service-one. See clusters for details.
HTTP filters: gRPC-JSON transcoder

Envoy support many different filters, but we need just one, called gRPC-JSON transcoder.

http_filters:
  - name: envoy.filters.http.grpc_json_transcoder
    typed_config:
      "@type": type.googleapis.com/envoy.extensions.filters.http.grpc_json_transcoder.v3.GrpcJsonTranscoder
      proto_descriptor: "/service_one_descriptor_set.pb"
      services: ["svc.ServiceOne"]
      match_incoming_request_route: false
      ignore_unknown_query_parameters: true
      auto_mapping: false
      convert_grpc_status: true
      print_options:
        add_whitespace: true
        always_print_primitive_fields: true
        always_print_enums_as_ints: false
        preserve_proto_field_names: true
  • proto_descriptor: "/service_one_descriptor_set.pb" is a path to descriptor set inside Docker container file created above.
  • services: ["svc.ServiceOne"] is service we’re going to call. It’s an array, so here we can specify several services, in case we have them in proto file.
  • For the rest option details see configuration. Those options affect how Envoy build HTTP endpoint, how output JSON will be formatted and formed, how to convert gRPC statuses and so on.

Clusters

Cluster binds a cluster with name service-one to address service-one:5000 which point to our service inside Kubernetes cluster. So, the more services we have, the more cluster entries we need.

clusters:
  - name: service-one
    connect_timeout: 1.25s
    type: logical_dns
    lb_policy: round_robin
    dns_lookup_family: V4_ONLY
    http2_protocol_options: {}
    load_assignment:
      cluster_name: service-one
      endpoints:
        - lb_endpoints:
            - endpoint:
                address:
                  socket_address:
                    address: service-one
                    port_value: 5000

BUILD file

The only part is interested at the point is container_image target, which builds Envoy image with descriptor set files inside, as you can see files parameter contains a label //service-one/pb:service_one_descriptor_set to target, which, as we already know, produces necessary file. Also, files includes :envoy.yaml, which is our Envoy configuration file, it will be resided in the image root. Due to the config is a part of the image itself, any changes within this file (or any other) will lead to update if the image digest.

load("@io_bazel_rules_docker//container:container.bzl", "container_image")

container_image(
    name = "image",
    base = "@envoy_linux_amd64//image",
    files = [
        ":envoy.yaml",
        "//service-one/pb:service_one_descriptor_set",
    ],
    ports = [
        "8080/tcp",
        "8081/tcp",
    ],
)

Envoy is ready, let’s go to the next step…

Envoy as an API Gateway: Part I Envoy as an API Gateway: Part III

See Also