Envoy as an API Gateway: Part IV

Oct 03, 2021
Bazel Golang Envoy API Proxy Microservices
6 min read
Source code

Part N: Overview
Part I: gRPC microservice
Part II: Envoy proxy container
Part III: Kubernetes and Tilt
Part IV: Authentication and authorization
Part V: CI/CD pipeline with Github Actions

Glossary
Outcome
Additional resources
Updated plan
External authorization service (ext_authz)
What is the “authz” service in a nutshell?
- denied/allowed functions
Envoy configuration
service-one

Glossary

Authentication (authn) is a verification of the user’s identity.
Authorization (authz) is a verification of the user access permissions.

Outcome

This part will show you how to authenticate and/or authorize incoming HTTP requests before them to upstream service-one.

ext_authz is an external authorization module, but we will use it for both authentication and authorization. For authentication, Envoy proxy also has a dedicated JSON Web Token (JWT) Authentication module, but we won’t use it in our scenario.

Additional resources

Updated, June 2023: As I’ve recently found, Clément Jean mentioned this post in his blog, providing additional details and explanations of the authentication process.

Updated plan

Here is our schema from the beginning. Now it’s time to talk about that faded blocks. They are new authz service, that Envoy will call it and check, should the request be forwarded to the upstream service or not. If authz returns no errors, incoming requests reach the destination. Otherwise, the client will receive an HTTP error code based on the Envoy configuration. If authorization service is not available or is broken, requests will not be passed to the upstream services at all. This commit adds authorization to the existing service.

Plan

External authorization service (ext_authz)

Envoy allows you to use HTTP or gRPC service for external authorization, whereas gRPC service can be either Envoy’s service or Google’s one. We will use Envoy’s one.

What is the “authz” service in a nutshell?

authz, in our case, is a gRPC service that implements Envoy Authorization service, with just one method Check.

service Authorization {
  // Performs authorization check based on the attributes associated with the
  // incoming request, and returns status `OK` or not `OK`.
  rpc Check(CheckRequest) returns (CheckResponse) {
  }
}

Inside the implementation we have an access to HTTP headers. Depending on with_request_body setting HTTP body can also be presented. Let’s look at our implementation:

func (a *Server) Check(ctx context.Context, req *auth.CheckRequest) (*auth.CheckResponse, error) {
	headers := req.Attributes.Request.Http.Headers

	fmt.Println("=== Request headers ===")
	for h, v := range headers {
		fmt.Printf("%s: %s\n", h, v)
	}
	fmt.Println("=======================")

	if headers["token"] != "abc" {
		return denided(401, "unuathorized"), nil
	}

	return allowed(), nil
}

Here we grab HTTP headers from the request and check if header token has a valid value abc. If so, request is authenticated and will be forwarded to upstream service-one. Otherwise, client receive a response with HTTP 401 Unuathorized. For debug purposes we print all of the HTTP header. Among them you can find header set by Envoy.

=== Request headers ===
te: trailers
user-agent: curl/7.79.1
token: abc
x-forwarded-proto: http
x-envoy-original-path: /v1/hello?name=Bazel
accept: */*
x-envoy-original-method: GET
:authority: localhost:8080
x-request-id: d7fce8bf-b90a-4d5f-87b8-988e9bea6f88
:method: POST
content-type: application/grpc
x-envoy-auth-partial-body: false
:path: /svc.ServiceOne/Hello
=======================

denied/allowed functions

It’s helper functions that make code a bit clear. The first function is denied. It accepts two params:

code is an HTTP response code.
body is an HTTP body.

and returns *auth.CheckResponse with HttpResponse field initialized as &auth.CheckResponse_DeniedResponse.

func denided(code int32, body string) *auth.CheckResponse {
	return &auth.CheckResponse{
		Status: &status.Status{Code: code},
		HttpResponse: &auth.CheckResponse_DeniedResponse{
			DeniedResponse: &auth.DeniedHttpResponse{
				Status: &envoy_type.HttpStatus{
					Code: envoy_type.StatusCode(code),
				},
				Body: body,
			},
		},
	}
}

The response for requests with invalid token will look like:

% curl -i \
  -H 'token: invalid' \
  http://localhost:8080/v1/hello\?name\=Bazel

HTTP/1.1 401 Unauthorized
content-length: 15
content-type: text/plain
date: Wed, 13 Oct 2021 20:59:32 GMT
server: envoy

unauthorized

The second function allowed is similar to denied.

func allowed() *auth.CheckResponse {
	return &auth.CheckResponse{
		Status: &status.Status{Code: int32(codes.OK)},
		HttpResponse: &auth.CheckResponse_OkResponse{
			OkResponse: &auth.OkHttpResponse{
				Headers: []*core.HeaderValueOption{
					{
						Header: &core.HeaderValue{
							Key:   "x-custom-header-propagated-to-upstream-service",
							Value: "bla-bla-bla",
						},
					},
				},
				HeadersToRemove: []string{"token"},
			},
		},
	}
}

But it has three main differences:

HttpResponse field is initialized as &auth.CheckResponse_OkResponse.
OkHttpResponse.Header allows you to specify additional headers which will be propagated to upstream service as gRPC metadata. See service-one for details below.
HeadersToRemove allows you to remove a header from propagation, i.e., headers listed here will not be sent to the upstream service. In our case, we don’t want to expose security tokens anywhere outside tne authz service.

Envoy configuration

Now, we have to somehow to say Envoy to use this new service. For this we change Envoy config the next way:

We add new cluster, which point to authz service. It’s the same as for service-one:

clusters:
  - name: authz
    connect_timeout: 1.25s
    type: logical_dns
    lb_policy: round_robin
    dns_lookup_family: V4_ONLY
    http2_protocol_options: {}
    load_assignment:
      cluster_name: authz
      endpoints:
        - lb_endpoints:
            - endpoint:
                address:
                 socket_address:
                    address: authz
                    port_value: 5000

And add one more filter to the filter_chains of our grpc-listener, in between grpc_json_transcoder and envoy.filters.http.router which must be the last filter in the filter chain, otherwise Envoy will return an error, like:

Error: terminal filter named envoy.filters.http.router of type envoy.filters.http.router must be the last filter in a http filter chain.

http_filters:
  - name: envoy.filters.http.grpc_json_transcoder
    #...skipped...

  - name: envoy.filters.http.ext_authz
    typed_config:
      "@type": type.googleapis.com/envoy.extensions.filters.http.ext_authz.v3.ExtAuthz
      grpc_service:
        envoy_grpc:
          cluster_name: authz
        timeout: 0.5s
      transport_api_version: V3
      failure_mode_allow: false
      with_request_body:
        max_request_bytes: 8192
        allow_partial_message: true
        pack_as_bytes: true
      status_on_error:
        code: 503

  - name: envoy.filters.http.router
    #...skipped...

Here:

With
```
grpc_service:
  envoy_grpc:
    cluster_name: authz
```
we specify the type of authz service (envoy_grpc) and cluster name.
failure_mode_allow: false means do not forward requests if authz service is unavailable.

With

  with_request_body:
    max_request_bytes: 8192
    allow_partial_message: true
    pack_as_bytes: true

we allow to forward HTTP body to authz service.

With
```
  status_on_error:
    code: 503
```
we define which HTTP status to return in case when authz service is unavailable. HTTP 503 here is used for distinguishing actual service unavailability and unauthorized access HTTP 403.

service-one

We also modified a Hello method implementation to grab the metadata.

func (s *Server) Hello(ctx context.Context, req *pb.HelloRequest) (*pb.HelloResponse, error) {
	md, ok := metadata.FromIncomingContext(ctx)
	if ok {
		fmt.Println("=== metadata ===")
		for k, v := range md {
			fmt.Printf("%s: %#v\n", k, v)
		}
		fmt.Println("================")
	}

	header := metadata.Pairs("set-cookie", "service-one")
	grpc.SendHeader(ctx, header)

	return &pb.HelloResponse{
		Msg: "Hello, " + req.Name,
	}, nil
}

This is how metadata looks like:

=== metadata ===
user-agent: []string{"curl/7.79.1"}
x-forwarded-proto: []string{"http"}
content-type: []string{"application/grpc"}
:authority: []string{"localhost:8080"}
x-request-id: []string{"d7fce8bf-b90a-4d5f-87b8-988e9bea6f88"}
x-envoy-original-method: []string{"GET"}
x-custom-header-propagated-to-upstream-service: []string{"bla-bla-bla"}
accept: []string{"*/*"}
x-envoy-original-path: []string{"/v1/hello?name=Bazel"}
x-envoy-expected-rq-timeout-ms: []string{"60000"}
================

With two lines:

	header := metadata.Pairs("set-cookie", "service-one")
	grpc.SendHeader(ctx, header)

we can set HTTP headers and send it back to the client.

curl -i \
  -H 'token: abc' \
  http://localhost:8080/v1/hello\?name\=Bazel

HTTP/1.1 200 OK
content-type: application/json
set-cookie: service-one
x-envoy-upstream-service-time: 1
grpc-status: 0
grpc-message:
content-length: 28
date: Wed, 13 Oct 2021 21:46:13 GMT
server: envoy

{
 "msg": "Hello, Bazel"
}

% curl -i \
  -H 'token: invalid' \
  http://localhost:8080/v1/hello\?name\=Bazel

HTTP/1.1 401 Unauthorized
content-length: 15
content-type: text/plain
date: Wed, 13 Oct 2021 20:59:32 GMT
server: envoy

unuathorized

Now service-one is guarded by authz service.