Whenever we work with gRPC and protobuf, we generate code. The code generator has some rules, and we know beforehand what to expect. In this post, I’ll show you how to use this knowledge.
Let’s say we build a REST API on top of the gRPC one, and one or more of our
list endpoints have two parameters: page
and limit
. The first specifies a
page to return within a list, and the last is the number of elements per page.
It’s quite a common approach. So, our URL will look like this:
/entities?page=N&limit=M
Both parameters are optional.
On the other side, we store our data, for instance, in SQLite, i.e., we have to
propagate these arguments down to a repo
package that works with a database and
apply them to a database query, like this (here I use
Masterminds/squirrel package for
building SQL queries):
func (r *repo) List(page, limit int) ([]Entities, error) {
q := squirrel.Select("...").
From("entities").
Where( /*...*/ ).
Limit(uint64(limit)).
Offset((uint64(page) - 1) * uint64(limit))
// do Select
}
In our protobuf file, we will have the following message, which will have page
and limit
fields, among others:
service MyService {
rpc Entities(Request) returns (Response);
}
message Request {
int32 page = 1;
int32 limit = 2;
string filter = 3;
}
message Response {/*..*/}
Protobuf has no int
data type, it has int32
/int64
and their variations
that map
onto Go types int32
/int64
, but in the repo
package, we’ll likely have just
int
type while
Masterminds/squirrel accepts
uint64
types. Anyway, we have to convert the data types at some point.
Now, think about this, we have several List*
endpoints, and for each of which,
we have to:
- read parameters
- convert data types and
- apply them to a SQL query
We remember the DRY principle. So we can do better than just copy-paste all these actions repeatedly.
Direct implementation
Nothing fancy here. We have two fields page
and limit
in protobuf message,
and we use them “as is”, i.e., we pass those fields directly to Service
:
// transport.go
func (s *Server) List(ctx context.Context, r *Request) (*Response, error) {
entities, err := s.svc.List(int(r.Page), int(r.Limit))
if err != nil {
return nil, err
}
// other code ...
Then, we pass them directly to the repo
:
func (s *service) List(page int, limit int) ([]repo.Entity, error) {
// skipped...
return s.repo.List(page, limit)
}
And finally, apply those values to our query:
func (r *repo) List(page int, limit int) ([]Entity, error) {
q := squirrel.Select("*").From("entities")
if page > 0 {
if limit < 1 {
limit = 5
}
q = q.Offset((uint64(page) - 1) * uint64(limit))
}
if limit > 0 {
q = q.Limit(uint64(limit))
}
return runQuery(q, r.db)
}
The code looks familiar, isn’t it?
What about Go interfaces? Can they help us?
When we have page
and limit
fields in our protobuf message. Protobuf
compiler protoc
with protoc-gen-go
plugin will generate access methods for each field, such as
GetPage() int32
- for thepage
fieldGetLimit() int32
- for thelimit
field, etc.
We can find these methods in
*.pb.go
file.
Relying on implicit interface implementation in Go, we can define an interface of those two methods.
type PageLimiter interface {
GetPage() int32
GetLimit() int32
}
Nice! Now we can call our services this way:
// transport.go
func (s *Server) List(ctx context.Context, r *Request) (*Response, error) {
entities, err := s.svc.List(int(r.GetPage()), int(r.GetLimit()))
if err != nil {
return nil, err
}
// other code ...
But it’s not much different from the previous example.
Look at the query and how we apply page
/limit
:
q := squirrel.Select("*").From("entities")
//...
q = q.Offset((uint64(page) - 1) * uint64(limit))
//...
q = q.Limit(uint64(limit))
Could we construct an object that will know how to apply our parameters to the
query? Sure, why not? Our query is represented by squirrel.SelectBuilder
type, we build a Select
query, first of all, so let’s define a such interface:
type Applier interface {
Apply(squirrel.SelectBuilder) squirrel.SelectBuilder
}
Method Apply
will “know” how to modify our query, squirrel.SelectBuilder
and will return the query variable. Say we will apply page
/limit
to the
query, though the method has nothing like page
/limit
, how can it be done?
Well, we need something that will have those values and will implement the
method:
// paginator.go
type pageLimiter struct {
limit, page int32
}
func (pl *pageLimiter) Apply(q squirrel.SelectBuilder) squirrel.SelectBuilder {
if pl.limit < 1 {
return q
}
return q.
Limit(uint64(pl.limit)).
Offset((uint64(pl.page) - 1) * uint64(pl.limit))
}
At this point, we have an object that applies parameters. Now we need to initialize it from a gRPC request. There are some requirements for initialization:
- it should be as short as possible
- we need to have default values
// paginator.go
type PageLimitApplier interface {
PageLimiter
Applier
}
func FromRequest(pl PageLimiter) PageLimitApplier {
page := pl.GetPage()
if page < 1 {
page = 1
}
limit := pl.GetLimit()
if limit < 1 {
limit = 50
}
return &pageLimiter{limit: limit, page: page}
}
gRPC request implements PageLimiter
interface, so we can initialize paginator,
which implements Applier
interface, from a request:
// transport.go
func (s *Server) ListWithApplier(ctx context.Context, r *Request) (*Response, error) {
entities, err := s.svc.ListWithApplier(paginator.FromRequest(r))
if err != nil {
return nil, err
}
Pass it into Service
:
func (s *service) ListWithApplier(pla paginator.PageLimitApplier) ([]repo.Entity, error) {
log.Printf(
"ListWithApplier called with page: %d and limit: %d\n",
pla.GetPage(),
pla.GetLimit(),
)
return s.repo.ListWithApplier(pla)
}
And apply page/limit
parameters in the repo
package:
func (r *repo) ListWithApplier(a paginator.Applier) ([]Entity, error) {
q := squirrel.Select("*").From("entities")
if err := paginator.MustApply(a, &q); err != nil {
return nil, err
}
return runQuery(q, r.db)
}
Where MustApply
is a wrapper over Apply
method with additional checks:
// paginator.go
func MustApply(a Applier, q *squirrel.SelectBuilder) error {
if a == nil {
return fmt.Errorf("applier is nil")
}
if q == nil {
return fmt.Errorf("select builder is nil, nothing to apply to")
}
*q = a.Apply(*q)
return nil
}
Conclusion
That’s it! Now we have:
- short call in
transport
package:s.svc.ListWithApplier(paginator.FromRequest(r))
- all paginator logic in one place:
Apply
function - that can be used “as is” or with
MustApply
and an additional error check
Complete example available on Github.