gRPC defines types and interfaces through Proto DSL, generating code for various languages. Therefore, compatibility is of utmost importance for gRPC, whether it's horizontally compatible with all language features, or vertically backward compatible with different versions of interface definitions. This requires the interface definition of gRPC to have excellent extensibility. This article will discuss the design of gRPC in terms of backward compatibility and extensibility. Understanding the purpose of these designs can help us design more maintainable and expandable interfaces in the future.
All fields are optional¶
In proto3, the "optional" keyword has been removed, and it is stipulated that all fields must be optional. When I first came into contact with proto, I thought this might be to adapt to some languages (such as Go) that do not have the concept of mandatory fields.
However, upon deeper consideration, one can realize that:
- RPC is a cross-process call, any input is possible, and we cannot ensure the correctness of this process's logic by constraining the input of another process. The only thing we can do is defensive programming.
- On the other hand, if mandatory fields are supported, when we add a mandatory field to an existing type, it will inevitably cause the code that originally depended on this type to be incompatible, introducing a breaking change. The larger the existing codebase, the more terrifying this change will be.
Therefore, abandoning the constraint of mandatory fields is a wise choice in engineering, which can also explain to some extent why Go does not support struct required field.
UnimplementedServer¶
When generating the server stub, grpc-go will generate the following code:
type Server interface {
Do(context.Context, *Request) (*Response, error)
mustEmbedUnimplementedServer()
}
type UnimplementedServer struct {
}
func (UnimplementedServer) Do(context.Context, *Request) (*Response, error) {
return nil, status.Errorf(codes.Unimplemented, "method CancelContract not implemented")
}
func (UnimplementedServer) mustEmbedUnimplementedServer() {}Since mustEmbedUnimplementedServer is not exported, we can never meet the constraints when implementing Server externally. The reason is that grpc-go hopes that all structs implementing Server will embed UnimplementedServer, and UnimplementedServer will implement the complete Server. In this way, even if new interfaces are added to the Server in the future, the existing code can be directly compatible.
type impl struct {
UnimplementedServer
}However, on the other hand, since UnimplementedServer implicitly implements all interfaces, it makes the code unable to check whether the function has been implemented at compile time, which brings certain risks. This issue has also sparked quite a bit of discussion in the community:
Input and output must be structured.¶
When defining RPC in Proto, it only allows the use of message (i.e., struct) as the input or output of the interface, and does not allow the use of primitive type or collection type. This is because only message is extensible and can arbitrarily add fields.
If we initially assume that an interface is just returning a list and directly define the return type of the interface as a list, if one day we need to return a pagination cursor at the same time, we will have no choice but to use a new interface or introduce a breaking change on the original interface to change the response type to a message.
Input and output types exclusive to each RPC¶
Beyond the constraints of proto itself, some best practices can also be found in the linter. There are a few rules in Buf:
- RPC_REQUEST_RESPONSE_UNIQUE
This rule requires each RPC interface to have independent input and output types, and does not allow multiple interfaces to share top-level types.
- rpc_allow_google_protobuf_empty_requests / rpc_allow_google_protobuf_empty_responses
These two rules are used to indicate whether to use the empty type in RPC input and output. They are turned off by default and it is not recommended to turn them on.
These two rules may impose additional burdens on writing interface definitions. It seems unnecessary to define an extra type when the type already exists or when there are no input parameters at all. The reason is similar to the aforementioned requirement that input and output must be structs, that is, we can never predict what fields an interface will add in the future: today, two interfaces have the same input parameters, but tomorrow, one of them may need to add a field; today, an interface without input parameters may need to pass in a parameter tomorrow. If we do not consider future extensibility when designing at the beginning, and casually reuse input and output or define them as empty, then we may introduce breaking changes when modifications are inevitably needed in the future!
Summary¶
Whether it's UnimplementedServer or optional fields, these practices are based on the premise that backward compatibility is more important than correctness to some extent. Although sometimes this may result in implementing one less function or passing one less field, compared to this, causing a breaking change in the entire codebase is more unacceptable. From an engineering perspective, this is the right choice, so it also places more demands on the business code: when it comes to RPC, parameter degradation, return type degradation, and function call degradation are all indispensable.