API evolution: changing interfaces without breaking clients

Code inside one program can change its function signatures freely, because the compiler finds every caller and they all ship together. An API can’t. Its callers are other teams’ services, mobile apps that users won’t update for months, and messages sitting in a queue that were written by last week’s code. Evolving an API is the craft of changing it without breaking any of them.

This article covers the two directions of compatibility and why requests and responses differ, the practical rules for protobuf and JSON, how REST, gRPC and GraphQL each handle change, the expand-and-contract pattern for changes that can’t be made compatibly, and the tooling that catches breakage before it ships.

Backward and forward compatibility

• Backward compatible: new code can read data written by old code. A new server accepts requests from old clients.
• Forward compatible: old code can read data written by new code. An old client handles responses from a new server.

You need both, more often than it seems. During a rolling deploy, old and new versions of a service run side by side, behind the same load balancer, reading the same queues and database. A client can hit a new instance on one request and an old one on the next. A rollback turns forward compatibility into backward compatibility. And anything persisted, such as events in Kafka or rows in a table, will be read by code that hasn’t been written yet.

Requests and responses break differently

The rule that falls out of this is asymmetric, and remembering the direction is most of the job. The server is the reader of requests and the clients are the readers of responses, so:

• In responses, you may add fields; you may not remove fields, make them nullable, or add new values to an enum, because old clients read them.
• In requests, you may add optional fields and accept new enum values; you may not add a required field, or stop accepting something old clients send.

In both, you may never change what an existing field means. Turning amount from dollars into cents, with the same name and type, is the worst breaking change of all, because no tool will flag it.

REST, gRPC and GraphQL

The three common styles make different trade-offs, and they evolve differently:

• REST with JSON, described by an OpenAPI document. Universally supported and easy to debug, but the schema is optional and checks happen at runtime, if at all. Evolution depends on discipline and on readers ignoring fields they don’t know.
• gRPC with protobuf. A typed schema compiled into clients, a compact binary encoding over HTTP/2, streaming, and deadlines propagated natively. Evolution rules are precise and machine-checkable, because fields are identified by number.
• GraphQL. Clients ask for exactly the fields they need, so adding fields is always safe, and the server knows which clients use which fields. Fields are retired with the built-in @deprecated directive and removed once usage drops to zero. Many teams version a GraphQL API not by URL but continuously, one field at a time.

A common combination is gRPC between internal services, where both sides are under your control, and REST or GraphQL at the edge, for browsers, mobile apps and third parties.

Protobuf’s rules

Protocol buffers encode each field as a tag (field number and wire type) followed by its value. Names don’t appear on the wire at all. Readers skip fields with numbers they don’t know, and use defaults for fields that are absent. Almost every rule follows from that:

Proto3 has no required fields, deliberately: a field that is required today can never be removed. Treat every field as possibly absent, and validate presence in application code where it matters.

JSON’s rules

JSON has no schema of its own, so its rules are conventions both sides must follow:

• Be a tolerant reader (Fowler, 2011): ignore unknown fields, don’t fail on extra ones, read only what you need. A client that validates responses against a strict schema with additionalProperties: false breaks on every additive change.
• Treat enums as open. Clients should handle an unknown value, for example a new status: "on_hold", with a sensible default instead of crashing in an exhaustive switch. Servers should still avoid adding values to response enums without warning.
• Never repurpose a field. Add a new one with the new meaning, and deprecate the old.
• Distinguish absent, null and empty consistently, and document which one a field can be. Many client bugs come from a field that was always present suddenly being null.

Expand, migrate, contract

Some changes can’t be made compatibly in one step: renaming a field, changing its type, splitting one into two. The answer is to split the change into steps that each are compatible. The pattern goes by several names: expand and contract, or parallel change (Danilo Sato, 2014).

1. Expand: add the new field alongside the old one, and make writers populate both.
2. Migrate: backfill old data, then move readers to the new field, one consumer at a time.
3. Contract: once nothing reads the old field, stop writing it, and finally remove it.

The same three steps apply to database columns (in detail in the article on schema migrations), API fields, event schemas and whole endpoints. They take longer than a single change, often weeks across several releases. That is the price of never having a moment when something is broken.

Versioning and deprecation

When a breaking change is unavoidable for external clients, it needs a new version. Common schemes, roughly from coarse to fine:

• In the URL (/v2/orders): visible and easy to route, but each major version is effectively a second API to maintain.
• In a header or media type: keeps URLs stable, but is less visible.
• Dated versions, as Stripe does: each account is pinned to the API version it first used, and the server translates responses back through a chain of small, per-change transformations. Breaking changes become cheap to ship, because old clients never see them unless they upgrade.

Retiring a version is a process, not a deploy. Announce it, measure who still calls the old version, contact them, and signal it in responses: the Deprecation header (RFC 9745) says an endpoint is deprecated, and Sunset (RFC 8594) says when it will stop working.

Contract testing

Two kinds of automated check catch breaking changes before they ship:

• Schema diffing compares the new schema against the last released one and fails the build on incompatible changes: buf breaking for protobuf, OpenAPI diff tools, schema registry compatibility modes for Avro and JSON Schema. It is cheap and catches structural breaks.
• Consumer-driven contracts (Pact is the common tool) go further. Each consumer records the requests it makes and the parts of the responses it relies on. The provider’s CI replays every consumer’s contract against the new build, so a provider learns it would break a specific client, and which field, before deploying. Unlike schema diffs, this catches behavioural breaks, for the fields consumers actually use.

Hyrum’s law

Hyrum Wright’s observation from working on Google’s internal libraries: “With a sufficient number of users of an API, it does not matter what you promise in the contract: all observable behaviors of your system will be depended on by somebody.” The order of results in a list nobody said was sorted, the exact text of an error message, the latency of a call, a field that happened to be always present. Changing any of them will break someone.

You can’t avoid that entirely, but you can limit what is observable: document ordering explicitly or randomise it, use error codes instead of messages, and keep internal details out of responses. Everything you expose becomes part of the contract, whether or not you meant it to.

A breaking-change checker

The core of a schema-diff tool, for one field. The same change gets a different verdict in a request and a response, because the reader is on the other side.

// Classify a change to one field of an API as breaking or not. The same change can
// be safe in a response and breaking in a request, because the reader is different.
type Field = { type: string; required: boolean; enumValues?: string[] };
type Where = "request" | "response";

export function isBreaking(where: Where, before: Field | undefined, after: Field | undefined): string | null {
  // Added field.
  if (!before && after) {
    if (where === "request" && after.required) return "new required request field: old clients don't send it";
    return null; // optional request field, or any response field: tolerant readers ignore it
  }
  // Removed field.
  if (before && !after) {
    if (where === "response") return "removed response field: clients may read it";
    return null; // the server just stops reading it; old clients sending it is harmless
  }
  if (!before || !after) return null;
  if (before.type !== after.type) return `type changed ${before.type} → ${after.type}`;
  if (where === "request" && !before.required && after.required) return "request field became required";
  if (where === "response" && before.required && !after.required) return "response field may now be missing";
  // Enums: servers may accept more values; clients may only see the values they know.
  const added = (after.enumValues ?? []).filter((v) => !(before.enumValues ?? []).includes(v));
  const removed = (before.enumValues ?? []).filter((v) => !(after.enumValues ?? []).includes(v));
  if (where === "response" && added.length) return `new enum values ${added.join(", ")} reach old clients`;
  if (where === "request" && removed.length) return `enum values ${removed.join(", ")} no longer accepted`;
  return null;
}

References

1. Kleppmann, M. (2017). Designing Data-Intensive Applications, ch. 4: Encoding and Evolution. O’Reilly. dataintensive.net
2. Protocol Buffers. Language Guide (proto 3): Updating A Message Type. protobuf.dev
3. Google. AIP-180: Backwards compatibility. google.aip.dev
4. Fowler, M. (2011). TolerantReader. martinfowler.com. martinfowler.com/bliki/TolerantReader.html
5. Sato, D. (2014). ParallelChange. martinfowler.com. martinfowler.com/bliki/ParallelChange.html
6. Leach, B. (2017). APIs as infrastructure: future-proofing Stripe with versioning. Stripe blog. stripe.com/blog/api-versioning
7. Wilde, E. (2019). RFC 8594: The Sunset HTTP Header Field. IETF. rfc-editor.org/rfc/rfc8594
8. Dalal, S. & Wilde, E. (2025). RFC 9745: The Deprecation HTTP Response Header Field. IETF. rfc-editor.org/rfc/rfc9745
9. Wright, H. Hyrum’s Law. hyrumslaw.com
10. Pact. Consumer-driven contract testing. docs.pact.io