Definition
- Field number as identity – Protocol Buffers encode each field using its number and wire type, not its name; the number is the permanent identity.
- Safe evolution – add new fields with new numbers, add new methods, add new services; old readers ignore unknown fields.
- Breaking changes – changing a field number remaps data; changing a type alters wire interpretation; removing a read field gives silent default values.
- Reserved – mark retired field numbers and names as
reservedto prevent accidental reuse. - Presence tracking – in proto3, use
optionalto distinguish unset from zero for fields like interest rates.
Imagine National Bank of India's official transfer form where every box has a fixed number. Box 3 is always "amount" forever. If you need new information, add a brand-new box with a fresh number. Old clerks who do not fill it in are fine because new boxes are optional. What you must never do is renumber a box or change box 3 from "amount in rupees" to "amount in paise," because that silently corrupts every existing form.
Understanding the Concepts
Why is the field number the most important thing?
Protocol Buffers encode each field on the wire using its field number and wire type, not its name. This single fact drives every versioning rule. Adding a field is safe because you give it a new, unused number, and old readers that do not know the field simply ignore it.
// Original contract
message TransferRequest {
string from_account = 1;
string to_account = 2;
int64 amount_paise = 3;
}
// Evolved: added optional fields with NEW numbers
message TransferRequest {
string from_account = 1;
string to_account = 2;
int64 amount_paise = 3;
string branch_code = 4; // NEW: old clients ignore this
string idempotency_key = 5; // NEW: old clients ignore this
optional string channel = 6; // NEW: presence-tracked
}
What changes break compatibility?
Changing, reusing, or removing a field number breaks the wire format.
// BROKEN: reused field number 3 with different meaning
message TransferRequest {
string from_account = 1;
string to_account = 2;
string currency_code = 3; // Was amount_paise! Silent corruption.
}
// BROKEN: changed type of field 3
message TransferRequest {
string from_account = 1;
string to_account = 2;
string amount_string = 3; // Was int64! Wire interpretation changes.
}
// DANGEROUS: removed field that clients still read
message TransferResponse {
string transaction_id = 1;
// field 2 (balance) removed: old clients get default 0
}
How do you retire a field safely?
Stop using the field and mark its number and name as reserved so no future developer reuses them.
message TransferResponse {
string transaction_id = 1;
// field 2 was 'old_balance', now retired
reserved 2;
reserved "old_balance";
int64 new_balance_paise = 3;
}
How do you evolve RPC methods?
Add new methods rather than changing existing signatures. For a genuinely breaking redesign, create a new service version.
// Original service
service AccountService {
rpc Transfer(TransferRequest) returns (TransferResponse);
}
// Evolved: added new method, original untouched
service AccountService {
rpc Transfer(TransferRequest) returns (TransferResponse);
rpc ScheduledTransfer(ScheduledTransferRequest)
returns (ScheduledTransferResponse); // NEW
}
// Breaking redesign: new service version
package bank.v2;
service AccountService {
rpc Transfer(TransferRequestV2) returns (TransferResponseV2);
}
How do you handle presence tracking in proto3?
In proto3, scalar fields do not distinguish unset from zero by default. For fields like interest rate where zero is a valid value distinct from absent, use the optional keyword.
message LoanDetails {
string loan_id = 1;
int64 principal_paise = 2;
optional double interest_rate = 3; // 0.0 is valid, absent is different
}
package com.bank.service;
import com.bank.grpc.LoanDetails;
public class LoanProcessor {
public void process(LoanDetails details) {
if (details.hasInterestRate()) {
double rate = details.getInterestRate();
System.out.println("Rate: " + rate);
} else {
System.out.println("Rate not provided, using default.");
}
}
}
How do you test cross-version compatibility?
Always test by decoding new messages with old stubs and vice versa. Use a linter like buf in CI to detect breaking changes automatically.
# Install buf and check for breaking changes
buf breaking --against '.git#branch=main'
# buf.yaml
version: v1
breaking:
use:
- WIRE_JSON
- PACKAGE
What about JSON serialization and field names?
Protobuf's binary encoding uses field numbers, but JSON encoding uses field names. Renaming a field in proto changes the JSON key, which breaks JSON-based clients. Be careful with JSON mapping when both binary and JSON consumers exist.
message Account {
string account_id = 1; // JSON key: "accountId"
// Renaming to 'acc_id' would change the JSON key
// and break JSON consumers
}
What should a senior engineer know?
Treat the proto file as a long-lived public contract. Keep it under strict review. Enforce breaking-change detection with buf in CI. Be careful changing between singular and repeated fields. In banking, a silent default value replacing real data can cause financial errors. The golden rule: add new numbered optional fields, add new methods, reserve retired numbers, and never change, reuse, remove, or retype an existing field.
Visual Overview
Proto Versioning Rules: ┌──────────────────────────────────────────┐ │ SAFE (backward compatible) │ │ │ │ ✓ Add new field with new number │ │ ✓ Add new RPC method │ │ ✓ Add new service or package version │ │ ✓ Add optional keyword for presence │ └──────────────────────────────────────────┘ ┌──────────────────────────────────────────┐ │ UNSAFE (breaks wire compatibility) │ │ │ │ ✗ Change or reuse a field number │ │ ✗ Change a field's type │ │ ✗ Remove a field clients still read │ │ ✗ Rename a field (breaks JSON mapping) │ └──────────────────────────────────────────┘ Retirement: Field 2 retired → reserved 2; reserved "old_name";
Key Takeaway
Versioning gRPC without breaking existing clients comes down to one fact: Protocol Buffers identify each field by its field number on the wire, not its name. The golden rule is never change or reuse an existing field number, because remapping silently corrupts data. To evolve, add new fields with new, unused numbers and treat them as optional so old clients that never set them still work.
For bigger changes, add new RPC methods rather than changing existing signatures. For a genuinely breaking redesign, create a new service version in a new package and run it alongside the old one. Retire fields by reserving their numbers and names so no future developer reuses them.
At National Bank of India, Pankaj evolved the transfer contract safely while thousands of old mobile clients were deployed. He added branch code, idempotency key, and channel as new optional fields, and added a ScheduledTransfer method without disturbing the existing Transfer. Breaking-change detection with buf in CI caught a junior developer's accidental field-number reuse before it reached production.
Remember one sentence: add new numbered optional fields, add new methods, reserve retired numbers, and never change, reuse, remove, or retype an existing field number.
---