gRPC is THE killer use case for Protocol Buffers. Google built gRPC specifically to work with protobuf, and together they create incredibly fast, type-safe microservices. If you're building distributed systems, this is the modern way to do it.
This guide walks you through building a complete gRPC service with Protocol Buffers. We'll use a telecom subscriber service as our example, and show you how to build both the server and client. Simple, practical, and ready for production.
What is gRPC?
gRPC is a modern RPC (Remote Procedure Call) framework that makes calling methods on remote servers feel like calling local functions. Instead of dealing with HTTP endpoints and JSON parsing, you just call functions with type-safe parameters.
Why gRPC + Protobuf?
- •Fast: Binary protocol is 7-10x faster than REST/JSON
- •Type-Safe: Compiler checks types, catches errors early
- •Streaming: Built-in support for real-time bidirectional streaming
- •Multi-Language: One .proto file generates code for 11+ languages
- •HTTP/2: Multiplexing, header compression, connection reuse
Step 1: Define Your Service
First, we define our service in a .proto file. We'll create a simple subscriber management service.
// subscriber_service.proto
syntax = "proto3";
package telecom;
// The subscriber management service
service SubscriberService {
// Get subscriber by phone number
rpc GetSubscriber (GetSubscriberRequest) returns (Subscriber);
// Create a new subscriber
rpc CreateSubscriber (CreateSubscriberRequest) returns (Subscriber);
// List all subscribers (server streaming)
rpc ListSubscribers (ListSubscribersRequest) returns (stream Subscriber);
// Update multiple subscribers (client streaming)
rpc BatchUpdateSubscribers (stream Subscriber) returns (BatchUpdateResponse);
// Real-time subscriber sync (bidirectional streaming)
rpc SyncSubscribers (stream Subscriber) returns (stream Subscriber);
}
// Message definitions
message Subscriber {
string msisdn = 1;
string name = 2;
string email = 3;
SubscriptionType type = 4;
bool is_active = 5;
}
enum SubscriptionType {
PREPAID = 0;
POSTPAID = 1;
CORPORATE = 2;
}
message GetSubscriberRequest {
string msisdn = 1;
}
message CreateSubscriberRequest {
string msisdn = 1;
string name = 2;
string email = 3;
SubscriptionType type = 4;
}
message ListSubscribersRequest {
int32 page_size = 1;
string page_token = 2;
}
message BatchUpdateResponse {
int32 updated_count = 1;
}Note: gRPC supports 4 types of RPC: Unary (simple request-response), Server streaming, Client streaming, and Bidirectional streaming. We'll show examples of each.
Step 2: Build a Python gRPC Server
Let's implement the server in Python. First, generate the code:
# Install dependencies pip install grpcio grpcio-tools # Generate Python code python -m grpc_tools.protoc -I. \ --python_out=. \ --grpc_python_out=. \ subscriber_service.proto
Now implement the server:
# server.py
import grpc
from concurrent import futures
import subscriber_service_pb2
import subscriber_service_pb2_grpc
class SubscriberService(subscriber_service_pb2_grpc.SubscriberServiceServicer):
def __init__(self):
# In-memory database (use real DB in production)
self.subscribers = {}
def GetSubscriber(self, request, context):
"""Simple unary RPC"""
msisdn = request.msisdn
if msisdn not in self.subscribers:
context.set_code(grpc.StatusCode.NOT_FOUND)
context.set_details(f'Subscriber {msisdn} not found')
return subscriber_service_pb2.Subscriber()
return self.subscribers[msisdn]
def CreateSubscriber(self, request, context):
"""Create a new subscriber"""
subscriber = subscriber_service_pb2.Subscriber(
msisdn=request.msisdn,
name=request.name,
email=request.email,
type=request.type,
is_active=True
)
self.subscribers[request.msisdn] = subscriber
print(f"Created subscriber: {request.msisdn}")
return subscriber
def ListSubscribers(self, request, context):
"""Server streaming - send multiple responses"""
for msisdn, subscriber in self.subscribers.items():
yield subscriber
# Server can send data as it becomes available
def BatchUpdateSubscribers(self, request_iterator, context):
"""Client streaming - receive multiple requests"""
count = 0
for subscriber in request_iterator:
self.subscribers[subscriber.msisdn] = subscriber
count += 1
return subscriber_service_pb2.BatchUpdateResponse(
updated_count=count
)
def SyncSubscribers(self, request_iterator, context):
"""Bidirectional streaming - real-time sync"""
for subscriber in request_iterator:
# Update local copy
self.subscribers[subscriber.msisdn] = subscriber
# Echo back confirmation
yield subscriber
def serve():
server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))
subscriber_service_pb2_grpc.add_SubscriberServiceServicer_to_server(
SubscriberService(), server
)
server.add_insecure_port('[::]:50051')
print("Server started on port 50051")
server.start()
server.wait_for_termination()
if __name__ == '__main__':
serve()Step 3: Build a Python gRPC Client
Now let's create a client that calls our service:
# client.py
import grpc
import subscriber_service_pb2
import subscriber_service_pb2_grpc
def run():
# Create channel to server
channel = grpc.insecure_channel('localhost:50051')
stub = subscriber_service_pb2_grpc.SubscriberServiceStub(channel)
# 1. Unary RPC - Create subscriber
print("\n=== Creating Subscriber ===")
response = stub.CreateSubscriber(
subscriber_service_pb2.CreateSubscriberRequest(
msisdn="+91-9876543210",
name="Telecom Customer",
email="[email protected]",
type=subscriber_service_pb2.POSTPAID
)
)
print(f"Created: {response.name} ({response.msisdn})")
# 2. Unary RPC - Get subscriber
print("\n=== Getting Subscriber ===")
response = stub.GetSubscriber(
subscriber_service_pb2.GetSubscriberRequest(
msisdn="+91-9876543210"
)
)
print(f"Found: {response.name} - {response.email}")
# 3. Server streaming - List all subscribers
print("\n=== Listing All Subscribers (Server Streaming) ===")
responses = stub.ListSubscribers(
subscriber_service_pb2.ListSubscribersRequest(page_size=100)
)
for subscriber in responses:
print(f" - {subscriber.name} ({subscriber.msisdn})")
# 4. Client streaming - Batch update
print("\n=== Batch Update (Client Streaming) ===")
def generate_updates():
# Send multiple updates in one call
for i in range(3):
yield subscriber_service_pb2.Subscriber(
msisdn=f"+91-912345{i:04d}",
name=f"Subscriber {i}",
email=f"sub{i}@telecom.com",
type=subscriber_service_pb2.PREPAID,
is_active=True
)
response = stub.BatchUpdateSubscribers(generate_updates())
print(f"Updated {response.updated_count} subscribers")
# 5. Bidirectional streaming - Real-time sync
print("\n=== Real-time Sync (Bidirectional Streaming) ===")
def generate_sync_requests():
for i in range(2):
yield subscriber_service_pb2.Subscriber(
msisdn=f"+91-918888{i:04d}",
name=f"Sync Subscriber {i}",
type=subscriber_service_pb2.CORPORATE,
is_active=True
)
responses = stub.SyncSubscribers(generate_sync_requests())
for response in responses:
print(f"Synced: {response.name} ({response.msisdn})")
if __name__ == '__main__':
run()Run the server in one terminal (python server.py) and the client in another (python client.py).
Example: Go gRPC Server
Go is extremely popular for gRPC services. Here's a quick example:
// Install dependencies go get google.golang.org/grpc go install google.golang.org/protobuf/cmd/protoc-gen-go@latest go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@latest // Generate Go code protoc --go_out=. --go_opt=paths=source_relative \ --go-grpc_out=. --go-grpc_opt=paths=source_relative \ subscriber_service.proto
// server.go
package main
import (
"context"
"log"
"net"
pb "path/to/subscriber_service"
"google.golang.org/grpc"
)
type server struct {
pb.UnimplementedSubscriberServiceServer
subscribers map[string]*pb.Subscriber
}
func (s *server) GetSubscriber(ctx context.Context, req *pb.GetSubscriberRequest) (*pb.Subscriber, error) {
if sub, ok := s.subscribers[req.Msisdn]; ok {
return sub, nil
}
return nil, grpc.Errorf(grpc.Code_NotFound, "subscriber not found")
}
func (s *server) CreateSubscriber(ctx context.Context, req *pb.CreateSubscriberRequest) (*pb.Subscriber, error) {
sub := &pb.Subscriber{
Msisdn: req.Msisdn,
Name: req.Name,
Email: req.Email,
Type: req.Type,
IsActive: true,
}
s.subscribers[req.Msisdn] = sub
log.Printf("Created subscriber: %s", req.Msisdn)
return sub, nil
}
func main() {
lis, err := net.Listen("tcp", ":50051")
if err != nil {
log.Fatalf("failed to listen: %v", err)
}
s := grpc.NewServer()
pb.RegisterSubscriberServiceServer(s, &server{
subscribers: make(map[string]*pb.Subscriber),
})
log.Printf("Server listening on :50051")
if err := s.Serve(lis); err != nil {
log.Fatalf("failed to serve: %v", err)
}
}Example: Node.js gRPC Service
Node.js works great with gRPC too. Perfect for TypeScript microservices:
// Install dependencies
npm install @grpc/grpc-js @grpc/proto-loader
// server.js
const grpc = require('@grpc/grpc-js');
const protoLoader = require('@grpc/proto-loader');
const PROTO_PATH = './subscriber_service.proto';
const packageDefinition = protoLoader.loadSync(PROTO_PATH);
const proto = grpc.loadPackageDefinition(packageDefinition).telecom;
const subscribers = new Map();
function getSubscriber(call, callback) {
const msisdn = call.request.msisdn;
if (subscribers.has(msisdn)) {
callback(null, subscribers.get(msisdn));
} else {
callback({
code: grpc.status.NOT_FOUND,
details: `Subscriber ${msisdn} not found`
});
}
}
function createSubscriber(call, callback) {
const subscriber = {
msisdn: call.request.msisdn,
name: call.request.name,
email: call.request.email,
type: call.request.type,
is_active: true
};
subscribers.set(call.request.msisdn, subscriber);
console.log(`Created subscriber: ${call.request.msisdn}`);
callback(null, subscriber);
}
function listSubscribers(call) {
// Server streaming
subscribers.forEach((subscriber) => {
call.write(subscriber);
});
call.end();
}
const server = new grpc.Server();
server.addService(proto.SubscriberService.service, {
getSubscriber,
createSubscriber,
listSubscribers
});
server.bindAsync(
'0.0.0.0:50051',
grpc.ServerCredentials.createInsecure(),
() => {
console.log('Server running on port 50051');
server.start();
}
);Error Handling in gRPC
gRPC has standard error codes. Use them for consistent error handling:
# Python - Raising errors
def GetSubscriber(self, request, context):
if not request.msisdn:
context.set_code(grpc.StatusCode.INVALID_ARGUMENT)
context.set_details('MSISDN is required')
return subscriber_service_pb2.Subscriber()
if request.msisdn not in self.subscribers:
context.set_code(grpc.StatusCode.NOT_FOUND)
context.set_details(f'Subscriber {request.msisdn} not found')
return subscriber_service_pb2.Subscriber()
# ... return subscriber
# Python - Catching errors
try:
response = stub.GetSubscriber(request)
except grpc.RpcError as e:
print(f"Error: {e.code().name}")
print(f"Details: {e.details()}")
if e.code() == grpc.StatusCode.NOT_FOUND:
print("Subscriber doesn't exist")
elif e.code() == grpc.StatusCode.INVALID_ARGUMENT:
print("Invalid request")Common gRPC Status Codes
OK- SuccessCANCELLED- Operation cancelledINVALID_ARGUMENT- Bad request parametersNOT_FOUND- Resource doesn't existALREADY_EXISTS- Duplicate resourcePERMISSION_DENIED- Access deniedUNAUTHENTICATED- Not authenticatedINTERNAL- Server errorUNAVAILABLE- Service unavailableDEADLINE_EXCEEDED- Timeout
gRPC Best Practices
✓ Use Deadlines/Timeouts
Always set timeouts to prevent hanging forever. gRPC calls should have deadlines.
# Python client with timeout
response = stub.GetSubscriber(
request,
timeout=5.0 # 5 seconds
)
// Go client with context deadline
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
response, err := client.GetSubscriber(ctx, request)✓ Reuse Channels
Creating channels is expensive. Create once, reuse across many calls.
# BAD - Creates new channel each time
def call_service():
channel = grpc.insecure_channel('localhost:50051')
stub = SubscriberServiceStub(channel)
response = stub.GetSubscriber(request)
channel.close()
# GOOD - Reuse channel
channel = grpc.insecure_channel('localhost:50051')
stub = SubscriberServiceStub(channel)
def call_service():
response = stub.GetSubscriber(request) # Reuses connection✓ Use Streaming for Large Data
Don't send huge arrays in one message. Use streaming to send data incrementally.
# BAD - Loads everything into memory
def ListAllSubscribers(self, request, context):
all_subs = load_million_subscribers() # Memory explosion!
return ListResponse(subscribers=all_subs)
# GOOD - Stream results
def ListAllSubscribers(self, request, context):
for subscriber in iterate_subscribers(): # One at a time
yield subscriber # Client receives incrementally✓ Add Health Checks
Use gRPC health checking protocol for load balancers and orchestrators.
# Python - Add health check service
from grpc_health.v1 import health
from grpc_health.v1 import health_pb2_grpc
health_servicer = health.HealthServicer()
health_pb2_grpc.add_HealthServicer_to_server(health_servicer, server)
# Set service status
health_servicer.set("telecom.SubscriberService", health_pb2.HealthCheckResponse.SERVING)Production Deployment
Use TLS in Production
Never use insecure channels in production. Always enable TLS.
# Python - Server with TLS
credentials = grpc.ssl_server_credentials([
(private_key, certificate_chain)
])
server.add_secure_port('[::]:50051', credentials)
# Python - Client with TLS
credentials = grpc.ssl_channel_credentials(root_certificates)
channel = grpc.secure_channel('server:50051', credentials)Add Logging and Metrics
Monitor your gRPC services. Track latency, error rates, and throughput.
# Python - Add interceptors for logging
import logging
class LoggingInterceptor(grpc.ServerInterceptor):
def intercept_service(self, continuation, handler_call_details):
logging.info(f"Calling: {handler_call_details.method}")
return continuation(handler_call_details)
server = grpc.server(
futures.ThreadPoolExecutor(),
interceptors=[LoggingInterceptor()]
)⚖Use Load Balancing
gRPC supports client-side and server-side load balancing. Use service mesh (Istio, Linkerd) or gRPC's built-in LB for distributing traffic across instances.
Related Resources
External References
Official Documentation
- gRPC Official Documentation - Complete gRPC guide
- gRPC Python Quick Start - Official Python guide
- gRPC on GitHub - Source code and examples
Conclusion
gRPC + Protocol Buffers is the modern standard for microservices. It's fast, type-safe, and supports streaming out of the box. Once you try it, going back to REST/JSON feels slow and error-prone.
Start with simple unary RPCs, then explore streaming when you need real-time updates. Use TLS in production, add proper error handling, and monitor your services. gRPC makes building distributed systems significantly easier than traditional approaches.