gRPC: Predictable Go Interfaces

Whenever we work with gRPC and protobuf, we generate code. The code generator has some rules, and we know beforehand what to expect. In this post, I’ll show you how to use this knowledge.

Let’s say we build a REST API on top of the gRPC one, and one or more of our list endpoints have two parameters: page and limit. The first specifies a page to return within a list, and the last is the number of elements per page. It’s quite a common approach. So, our URL will look like this:

/entities?page=N&limit=M

Both parameters are optional.

On the other side, we store our data, for instance, in SQLite, i.e., we have to propagate these arguments down to a repo package that works with a database and apply them to a database query, like this (here I use Masterminds/squirrel package for building SQL queries):

func (r *repo) List(page, limit int) ([]Entities, error) {
    q := squirrel.Select("...").
      From("entities").
      Where( /*...*/ ).
      Limit(uint64(limit)).
      Offset((uint64(page) - 1) * uint64(limit))

    // do Select
}

In our protobuf file, we will have the following message, which will have page and limit fields, among others:

service MyService {
  rpc Entities(Request) returns (Response);
}

message Request {
  int32 page = 1;
  int32 limit = 2;
  string filter = 3;
}

message Response {/*..*/}

Protobuf has no int data type, it has int32/int64 and their variations that map onto Go types int32/int64, but in the repo package, we’ll likely have just int type while Masterminds/squirrel accepts uint64 types. Anyway, we have to convert the data types at some point.

Now, think about this, we have several List* endpoints, and for each of which, we have to:

• read parameters
• convert data types and
• apply them to a SQL query

We remember the DRY principle. So we can do better than just copy-paste all these actions repeatedly.

Direct implementation

Nothing fancy here. We have two fields page and limit in protobuf message, and we use them “as is”, i.e., we pass those fields directly to Service:

// transport.go
func (s *Server) List(ctx context.Context, r *Request) (*Response, error) {
	entities, err := s.svc.List(int(r.Page), int(r.Limit))
	if err != nil {
		return nil, err
	}

  // other code ...

Then, we pass them directly to the repo:

func (s *service) List(page int, limit int) ([]repo.Entity, error) {
  // skipped...
	return s.repo.List(page, limit)
}

And finally, apply those values to our query:

func (r *repo) List(page int, limit int) ([]Entity, error) {
	q := squirrel.Select("*").From("entities")

	if page > 0 {
		if limit < 1 {
			limit = 5
		}

		q = q.Offset((uint64(page) - 1) * uint64(limit))
	}

	if limit > 0 {
		q = q.Limit(uint64(limit))
	}

	return runQuery(q, r.db)
}

The code looks familiar, isn’t it?

What about Go interfaces? Can they help us?

When we have page and limit fields in our protobuf message. Protobuf compiler protoc with protoc-gen-go plugin will generate access methods for each field, such as

• GetPage() int32 - for the page field
• GetLimit() int32 - for the limit field, etc.

We can find these methods in *.pb.go file.

Relying on implicit interface implementation in Go, we can define an interface of those two methods.

type PageLimiter interface {
  GetPage() int32
  GetLimit() int32
}

Nice! Now we can call our services this way:

// transport.go
func (s *Server) List(ctx context.Context, r *Request) (*Response, error) {
	entities, err := s.svc.List(int(r.GetPage()), int(r.GetLimit()))
	if err != nil {
		return nil, err
	}

  // other code ...

But it’s not much different from the previous example.

Look at the query and how we apply page/limit:

q := squirrel.Select("*").From("entities")
//...
		q = q.Offset((uint64(page) - 1) * uint64(limit))
//...
		q = q.Limit(uint64(limit))

Could we construct an object that will know how to apply our parameters to the query? Sure, why not? Our query is represented by squirrel.SelectBuilder type, we build a Select query, first of all, so let’s define a such interface:

type Applier interface {
	Apply(squirrel.SelectBuilder) squirrel.SelectBuilder
}

Method Apply will “know” how to modify our query, squirrel.SelectBuilder and will return the query variable. Say we will apply page/limit to the query, though the method has nothing like page/limit, how can it be done? Well, we need something that will have those values and will implement the method:

// paginator.go
type pageLimiter struct {
	limit, page int32
}

func (pl *pageLimiter) Apply(q squirrel.SelectBuilder) squirrel.SelectBuilder {
	if pl.limit < 1 {
		return q
	}

	return q.
		Limit(uint64(pl.limit)).
		Offset((uint64(pl.page) - 1) * uint64(pl.limit))
}

At this point, we have an object that applies parameters. Now we need to initialize it from a gRPC request. There are some requirements for initialization:

• it should be as short as possible
• we need to have default values

// paginator.go
type PageLimitApplier interface {
	PageLimiter
	Applier
}

func FromRequest(pl PageLimiter) PageLimitApplier {
	page := pl.GetPage()
	if page < 1 {
		page = 1
	}

	limit := pl.GetLimit()
	if limit < 1 {
		limit = 50
	}

	return &pageLimiter{limit: limit, page: page}
}

gRPC request implements PageLimiter interface, so we can initialize paginator, which implements Applier interface, from a request:

// transport.go
func (s *Server) ListWithApplier(ctx context.Context, r *Request) (*Response, error) {
	entities, err := s.svc.ListWithApplier(paginator.FromRequest(r))
	if err != nil {
		return nil, err
	}

Pass it into Service:

func (s *service) ListWithApplier(pla paginator.PageLimitApplier) ([]repo.Entity, error) {
  log.Printf(
    "ListWithApplier called with page: %d and limit: %d\n",
    pla.GetPage(),
    pla.GetLimit(),
  )

	return s.repo.ListWithApplier(pla)
}

And apply page/limit parameters in the repo package:

func (r *repo) ListWithApplier(a paginator.Applier) ([]Entity, error) {
	q := squirrel.Select("*").From("entities")

	if err := paginator.MustApply(a, &q); err != nil {
		return nil, err
	}

	return runQuery(q, r.db)
}

Where MustApply is a wrapper over Apply method with additional checks:

// paginator.go
func MustApply(a Applier, q *squirrel.SelectBuilder) error {
	if a == nil {
		return fmt.Errorf("applier is nil")
	}

	if q == nil {
		return fmt.Errorf("select builder is nil, nothing to apply to")
	}

	*q = a.Apply(*q)

	return nil
}

Conclusion

That’s it! Now we have:

• short call in transport package: s.svc.ListWithApplier(paginator.FromRequest(r))
• all paginator logic in one place: Apply function
• that can be used “as is” or with MustApply and an additional error check

Complete example available on Github.