# AEP-1 AEP Purpose and Guidelines
Service APIs on the Internet continue to proliferate; having a machine-readable
API is an expectation and prerequisite to adoption for many services. By some
estimates, there are now more than 20,000 public REST APIs available.
As this corpus continues to grow, many companies struggle with API Governance:
even as companies grow and disparate teams work to deliver discrete services,
APIs ought to remain simple, intuitive, and consistent.
Therefore, it is increasingly necessary to have a corpus of documentation for
API producers, reviewers, and other interested parties to reference. The AEP
collection provides a way to provide consistent documentation for API design
guidance.
## What is an AEP?
AEP stands for **API Enhancement Proposal**, which is a design document
providing high-level, concise documentation for API development.
Companies that adopt the AEP program use them as a source of truth for
API-related documentation, and the means by which service producers discuss and
come to consensus on API guidance. AEPs are maintained as Markdown files with
metadata in the AEP GitHub repository.
### Technical leadership
See [governance](/governance).
## States
At any given time, AEPs may exist in a variety of states as they work their way
through the process. The following is a summary of each state.
### Reviewing
Initial discussion on most AEPs occurs in the initial pull request to submit
the AEP. Once this PR is merged, the AEP exists in the "Reviewing" state. This
means that the authors and the technical steering committee have reached a
general consensus on the proposal.
At this stage, the committee may request changes or suggest alternatives to the
proposal before moving forward, but there is a general expectation that the
proposal will move forward and it is usually safe to "early adopt" it.
An AEP be in the reviewing state for at least 14 days before being
approved, and the committee send appropriate communication regarding
the pending approval.
### Approved
Once an AEP has been agreed upon, it enters "approved" state and is considered
"best current practice".
AEPs be edited after they are approved, either to correct grammar or
word choices, or to clarify semantic guidance (in response to reader
questions). In rare occasions, new guidance be added.
Clarifications and new guidance be reflected in the changelog.
Correction of typos or minor language alterations be done silently.
### Final
If an AEP has been approved for a significant period and the technical steering
committee is certain that no further guidance will be needed, they move
the AEP in to "final" state.
AEPs in the final state be amended with new guidance. They
be editied to correct spelling, grammar, or clarity provided there are no
semantic changes.
### Replaced
If an AEP has been replaced by another AEP, it enters "replaced" state. The AEP
include a notice explaining the replacement and rationale (the
replacement AEP also clearly explain the rationale).
In general, service producers rely primarily on AEPs in the "approved" state.
Service producers rely on AEPs in the "reviewing" state
### Withdrawn
If an AEP is withdrawn by the author or champion, or is rejected by the
technical steering committee after reaching the "reviewing" state, it enters
"withdrawn" state. Withdrawn AEPs remain accessible, but are removed from the
indexes; they provide documentation and reference to inform future discussions.
## Workflow
The following workflow describes the process for proposing an AEP, and moving
an AEP from proposal to implementation to final acceptance.
### Overview
```mermaid
graph LR
classDef orange fill:#FFA500,stroke:#000,stroke-width:1px;
classDef lightskyblue fill:#87CEFA,stroke:#000,stroke-width:1px;
classDef palegreen fill:#98FB98,stroke:#000,stroke-width:1px;
classDef mistyrose fill:#FFE4E1,stroke:#000,stroke-width:1px;
classDef lightsteelblue fill:#B0C4DE,stroke:#000,stroke-width:1px;
github_pr((GitHub PR)):::orange
reviewing[Reviewing]:::lightskyblue
approved[Approved]:::palegreen
final[Final]:::palegreen
withdrawn[Withdrawn]:::mistyrose
replaced[Replaced]:::lightsteelblue
github_pr --> reviewing
reviewing --> approved
reviewing -.-> withdrawn
approved --> final
approved -.-> replaced
final -.-> replaced
```
### Proposing an AEP
In order to propose an AEP, first open a pull request with a draft AEP; the AEP
should conform to the guidance in [AEP-8](/8). Most AEPs be no more than
two pages if printed out.
If the technical steering committee has suggested an AEP number, use that;
otherwise use 99 (and expect to change it during the course of the review).
In most circumstances, the committee will assign the proposal an AEP number and
begin discussion. Once there is consensus, the committee will merge the PR, and
the AEP will enter the "reviewing" state.
The committee reject an AEP outright if they have an obvious reason to
do so (e.g. the proposal was already discussed and rejected in another AEP or
is fundamentally unsound), in which case the PR is not merged.
### Accepting an AEP
The editors will work together to ensure that qualified proposals do not linger
in review.
To gain final approval, an AEP be approved by, at minimum, two members
of the technical steering committee. Additionally, there be any
committee members requesting significant changes (indicated by the use of the
"changes requested" feature on GitHub).
### Withdrawing or Rejecting an AEP
The author of an AEP may decide, after further consideration, that an AEP
should not advance. If so, the author may withdraw the AEP by updating the PR
adding a notice of withdrawal with an explanation of the rationale.
Additionally, the author may be unable to get consensus among the group and the
technical steering committee may elect to reject the AEP. In this situation,
the committee shall amend the PR adding a notice of rejection with an
explanation of the rationale. In both cases, the committee update the
state accordingly and submit the PR.
### Replacing an AEP
In rare cases, it may be necessary to replace an AEP with another one. This is
not general practice: minor edits to approved AEPs are acceptable, and AEPs
only enter final state when there is high confidence that further edits will
not be necessary.
However, if new guidance fundamentally alters the old guidance in some way,
then the technical steering committee create a new AEP that, once
approved, will replace the old one. The old one then enters "Replaced" state,
and will link to the new, current AEP.
[aep.dev]: https://aep.dev/
---
# AEP-2 AEP Numbering and Organization
This document describes the numbering system and general organization of AEPs.
AEP numbers:
- Are unique. No two AEPs share the same number.
- Indicate the order in which they are authored.
To help with organization and discovery, AEPs also have a category and a slug
to easily discovery and refer to the AEP.
See below for an example of an AEP id, category, and slug, expressed via YAML
front-matter:
```yaml
id: 11
category: 'resources'
slug: 'create'
```
## AEP Numbers
### Core AEPs
To uniquely identify an AEP, each one is assigned a permanent number as it is
merged.
- A merged AEP use the next available number.
- AEPs be numbered between 1 and 9999.
### Organization-specific AEPs
- Organizations extend the AEPs with guidance that is relevant to them.
- Organization-specific AEPs use AEP numbers 10000 or greater.
## AEP Categories
To help with discovery of an AEP, each one is assigned a category: a string
that describes a theme or grouping the AEP belongs in.
- The category of an AEP change over time.
As an example, the guidance around custom methods may grow to multiple AEPs. As
such, AEPS may create a new category "custom-methods" and recategorize relevant
AEPs (mutating custom methods, retrieval custom methods) under it.
Categories are expressed in the YAML front-matter of an AEP:
```yaml
category: 'custom-methods'
```
## AEP Slugs
Each AEP is given a slug: a string by which it can be easily referenced and
accessible via an URL on the aep.dev site.
- The slug of an AEP change over time, sometimes to refer to a new
revision of an existing AEP.
- On the AEP site, the AEP is then available by with a path of the slug:
`/{slug}`.
For example, an [AEP-123](/123) may author guidance around a design pattern for
revisions of resources, and be given the slug "revisions". If a new [AEP-1340](/1340)
that provides a significant change is authored, then it may be use the slug
"revisions", with [AEP-123](/123) given a slug like "revisions-deprecated".
Slugs are expressed in the YAML front-matter of an AEP:
```yaml
slug: 'create'
```
And is available via the path "/create" on the site hosting aep.dev.
## Rationale
### Why AEP Slugs
In the original aip.dev project, numbers were the primary identifier for an
AEP. However, as a number does not hint to its purpose or the guidance it
contains, conversations around AIPs were difficult to hold and often the AIPs
were be referred to by an unofficial colloquial name rather than by its number.
As such, AEPs provide the ability for a semantic slug.
### Why AEP Categories
With aip.dev, numbers were an organizational scheme, with AIPs sorted by number
in navigation. Using numbers resulted in difficulties putting a new AEP
alongside related AEPs if a number was not available.
By adding categories, new AEPs can easily be organized, as well as produce more
granular categories as the need arises.
### Why Organization-specific AEPs
Similar to [google.aip.dev](google.aip.dev), the organizations supporting the
AEP project found the need for providing organization-specific guidance that is
a superset of the open specification. By allocating a specific range,
organizations may safely extend the AEPs without concern of conflicting AEP
numbers.
---
# AEP-3 Glossary
This AEP defines common terminology.
## Guidance
The following terminology be used consistently throughout AEPs.
### API
Application Programming Interface. This can be a local interface (such as an
SDK) or a Network API (defined below).
APIs define one or more operations upon resource types.
### API Backend
A set of servers and related infrastructure that implements the business logic
for an API Service.
### API Client
An API Client is a program or library that perform a specific tasks by calling
an API or generic tools, such as CLIs, that expose the API in a user-accessible
fashion or operate on resource data at rest.
Examples of clients include the following:
- Command line interfaces
- Libraries, such as an SDK for a particular programming language
- Scripts that operates on a JSON representation of a resource after reading it
from an API
- Tools, such as a [Declarative client][]
- Visual UIs, such as a web application
### AEP Edition
A specific edition of the AEP specification. This is a specific collection of
AEP guidance that will not change, annotated with the year (e.g. [aep-2025](/2025)). See
[editions](/editions) for more information.
### API Definition
A well-structured representation of an API.
### API Endpoint
Refers to a network address that an API uses to handle incoming requests. One
API may have multiple endpoints, such as `https://pubsub.example.com` and
`https://content-pubsub.example.com`.
### API Gateway
One or more services that together provide common functionality across API
services, such as load balancing and authentication.
### API Method
An individual operation within an API. It is typically represented in Protocol
Buffers by an `rpc` definition, or in HTTP via a `method` and a `path`.
### API Name
The name by which to refer to an API.
See [API Name](/apis#api-name)
### API Request
A single invocation of an API Method. It is often used as the unit for billing,
logging, monitoring, and rate limiting.
### API Resource
An entity upon which one or more methods can operate.
### API Resource Type
The type of a API resource. It is globally unique within an API.
### API Service
An implementation of an API, exposing API methods on one or more network
addresses.
### Consumer
Either a programmatic client or a user that consumes an API. This term should
be used when a statement refers broadly to both programs and users.
### Collection
A collection represents a discrete set of resources of a specific type. A
resource may have one or more collections, with each collection having its own
parent.
### Declarative Clients
Declarative Clients, also known as Infrastructure as Code (IaC), describes a
category of clients that consumes a markup language or code that represents
resources exposed by an API, and executes the appropriate imperative actions to
drive the resource to that desired state. To determine what changes to make and
if a set of updates was successful a declarative client compares server side
resource attributes with client defined values. The comparison feature ensures
accuracy of a creation or an update but it requires services to treat the
client set fields as read-only and diligently preserve those values.
Examples of complexities that declarative clients abstract away include:
- Determing the appropriate imperative action (create / update / delete) to
achieve desired state.
- Ordering of these imperative actions.
[Terraform][] is an example of such a client.
### Resource ID
The unique identifier for a resource, within its parent collection. For
example, in the following resource path:
```
/publishers/consistent-house/books/pride-and-prejudice
```
There are two resource IDs:
- `consistent-house`, which is the id of the publisher.
- `pride-and-prejudice`, which is the id of the book.
See [paths](/paths#resource-id) for more information.
### Root collection
A [collection](#collection) that does not have any parent collections. For
example, `/users/`.
### Root Resource
A root resource is a resource for which a collection exists that has no
parents. In other words, it is a resource that has a collection that appears at
the root of a resource hierarchy.
A root resource may also appear as a child of another resource: for example, a
music streaming services may have global shared playlists (e.g. "top
trending"), but may also have playlists nested under a user. In this case, a
playlist is still considered a root resource, since the collection with the
glocal shared playlists has no parents.
### Schema
A schema describes the structure of the request or response of an
[API method](#api-method), or a [resource](#api-resource). It refers both to an
OpenAPI schema as well as a protobuf message.
### User
A human being which is using an API directly, such as with cURL. This term is
defined to differentiate usage in the AIPs between a human _user_ and a
programmatic _client_.
### Network API
An API that operates across a network of computers. Network APIs communicate
using network protocols including HTTP, and are frequently produced by
organizations separate from those that consume them.
[declarative clients]: #declarative-clients
[terraform]: https://www.terraform.io/
---
# AEP-4 Resource types
Most APIs expose _resources_ (their primary nouns) which users are able to
create, retrieve, and manipulate. APIs are allowed to name their resource types
as they see fit, and are only required to ensure uniqueness within that API.
This means that it is possible (and often desirable) for different APIs to use
the same type name. For example, a Memcache and Redis API would both want to
use `Instance` as a type name.
When mapping the relationships between APIs and their resources, however, it
becomes important to have a single, globally-unique type name. Additionally,
tools such as Kubernetes or GraphQL interact with APIs from multiple providers.
## Guidance
APIs define a resource type for each resource in the API, according to
the following pattern: `{API Name}/{Type Name}`. Also see
[API Name](/apis#api-name).
### Type name
The type name is an identifier that be unique within the API, as
indicated by it's namespacing within the API name.
The type name:
- Only contain ASCII alphanumeric characters.
- Start with a lowercase letter.
- Be of the singular form of the resource.
- Use kebab case.
### Examples
Examples of API names and resource types include:
- `networking.istio.io/instance`
- `pubsub.example.com/topic`
- `pubsub.example.com/subscription`
- `spanner.example.com/database`
- `spanner.example.com/instance`
- `apis.example.com/user/user-event`
### Annotating resource types
APIs annotate the resource types for each resource in the API
- The `type` field be the resource type `{API Name}/{Type Name}`
- The `singular` field be the kebab-case singular type name.
- The `plural` field be the kebab-case plural of the singular.
The `pattern` field match the `pattern` rule in the following grammar,
expressed as [EBNF][EBNF]:
```
pattern = element, { "/", element };
element = variable | literal;
variable = "{", literal, "}";
```
Where `literal` matches the regex `[a-z][a-z0-9\-_]*[a-z0-9]`.
- Patterns match the possible [paths](/paths) of the resource.
- Pattern variables (the segments within braces) match the singular of
the resource whose id is being matched by that value, suffixed with `_id`.
#### Pattern uniqueness
If multiple patterns are defined within a resource, the patterns defined **must
not** overlap in the set of resource paths that they can match. In other words,
a resource path may match at most one of the patterns.
For example, the following two patterns would not be valid for the same
resource:
- `user/{user}`
- `user/{user_part_1}~{user_part_2}`
## Rationale
### Singular and Plural
Well-defined singular and plurals of a resource enable clients to determine the
proper name to use in code and documentation.
google.aip.dev uses UpperCamelCase for resource types, while aep.dev uses
kebab-case. This is to enforce better consistency in the representation of
various multipart strings, as collection identifiers use kebab case.
[API Group]: https://kubernetes.io/docs/concepts/overview/kubernetes-api/#api-groups
[nested collections]: /122.md#collection-identifiers
[Object]: https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md#types-kinds
[resource]: https://github.com/googleapis/googleapis/blob/master/google/api/resource.proto
[service configuration]: https://github.com/googleapis/googleapis/blob/master/google/api/service.proto
[EBNF]: https://en.wikipedia.org/wiki/Extended_Backus%E2%80%93Naur_form
## Changelog
---
# AEP-5 Designing an API
This AEP serves as a high-level guide to designing an AEP-compliant API. AEPs.
## Process summary
1. Enumerate the use cases you would like your API to satisfy.
1. Identify resources.
1. Identify standard operations.
1. Identify custom operations.
## Enumerate use cases
The first step in designing an API is understanding precisely what operations
you would like your user to be able to perform. Enumerate these operations,
attempting to be as granular as possible.
For example, if the API is for VM management in a public cloud, the operations
may include:
- Create a VM.
- List all VMs owned by a company.
- List all VMs owned by a user.
- Restart a running VM.
Or for a multiplayer online game, operations may include:
- Proposing a trade to another player.
- Accepting a proposed trade.
- Find open trade offers containing an item.
- List items in a player's inventory.
Some best practices:
- Attempt to define granular use cases that can be composed to satisfy more
complex use cases.
- Be comprehensive and consider lower-priority use cases: having more use cases
enumerated often leads to better API design.
## Identify resources
Once your use cases are defined, consider how many of those can be represented
by an [API Resource Type][]: entities that are created, read, updated, and
deleted.
Examples include:
- users
- virtual machines
- load balancers
- services
One of the core concepts described by the AEPs is resource-oriented design:
this design paradigm allows for uniform standard operations that reduce the
cognitive overhead in learning about the operations and schemas exposed by your
API.
Resources can relate to each other. For example:
- A parent-child relationship defining ownership/scope (A user who created a
VM).
- A resource dependency, where one resource depends on another to function (A
VM depending on a disk).
See the following AEPs to learn more about resource-oriented design:
- [resource-oriented design](/121)
- [resource paths](/122)
- [resource types](/4)
## Identify standard operations
Once the resources are defined, identify one or more standard methods for each
of those resources. Standard methods operate on the lifecycle of a resource
lifecycle: namely, they create, read, update, delete, and list resources.
Ideally all five standard methods should be exposed for every resource.
See the following AEPs to learn more about the standard methods:
- [Get](/131)
- [List](/132)
- [Create](/133)
- [Update](/134)
- [Delete](/135)
## Identify custom operations
To accomplish some of the user journeys, resources may need to support
operations other than [Create](/133), [Update](/134), [Delete](/135),
[Get](/131), and [List](/132) them. Some examples include:
- restarting a virtual machine
- triggering a CI build
- wiping a disk
Break down each of the actions into granular operations, then follow the
[custom operations](/136) AEP on how to design them.
[API Resource Type]: /3#api-resource-type
---
# AEP-6 Governance
## Article I: Purpose
1. The API Enhancement Proposals (AEP) project exists to help developers and
organizations build clear, consistent network APIs and clients by providing
an extensible set of design guidelines (AEPs) and an ecosystem of tools to
apply those rules in practice.
2. To support this aim, the project provides and maintains tools that enrich
the experience of developing and using APIs that conform to the project's
design guidelines by generating artifacts such as client libraries and
documentation.
## Article II: Steering Committee
1. The AEP project shall be governed by a steering committee, comprising at
least five representatives from at least five distinct organizations.
2. No more than one third of the committee shall represent or be employed by
the same organization.
1. In the event that ambiguities arise regarding what constitutes "the same
organization," a vote of the committee shall decide. Any members from the
organizations under consideration may deliberate, but shall recuse
themselves from the vote.
3. Membership on the steering committee shall be by invitation of the current
steering committee. The steering committee shall meet at least once annually
to consider the addition, removal, or replacement of members.
4. The members of the steering committee shall be maintained in a
[MAINTAINERS.md](https://github.com/aep-dev/aep.dev/blob/main/MAINTAINERS.md)
file in the
[primary AEP GitHub repository](https://github.com/aep-dev/aep.dev).
5. Steering committee members accept the following responsibilities:
1. Ownership of the charter, vision, and direction of the AEP project.
2. Regular attendance and participation at stated committee meetings.
3. Contribution of design guidance or code as appropriate, and reviewing the
contributions of others.
6. Steering committee members who are unable or unwilling to continue to
perform the responsibilities described above may voluntarily step down from
the committee or may be removed from the steering committee without
prejudice. Former committee members will be designated as "emeritus" in the
maintainers list.
7. The steering committee may appoint additional maintainers, to whom it may
delegate tasks as necessary.
## Article III: Source of Truth
1. The AEP GitHub repositories are the source of truth for everything in the
project, including both code and guidelines.
## Article IV: Meetings & Voting
1. The steering committee shall hold a regular meeting on a cadence of its
choosing, and shall keep minutes of said meetings in a GitHub repository set
aside for that purpose, which shall be accessible by all steering committee
members.
1. Meetings and minutes may be public or private, at the discretion of the
steering committee.
2. The steering committee ordinarily operates by consensus, and recording votes
for each decision is not required (but the decision should be recorded in
the minutes). When possible, decisions and voting shall be done using GitHub
issues for transparency.
3. In the event that a vote is required, an email shall be sent to each
steering committee explaining the issue in question and allowing at least 10
business days for voting. The specifics of how elections are handled are at
the discretion of the steering committee.
## Article V: Amendment
This charter may be amended by a two-thirds supermajority of the steering
committee, after conducting an vote as described in V§3.
## Additional Information
### Maintainers
See
[MAINTAINERS.md](https://github.com/aep-dev/aep.dev/blob/main/MAINTAINERS.md)
for the current committee list (known as maintainers in the document).
---
# AEP-8 AEP Style and Guidance
AEP stands for **API Enhancement Proposal**, which is a design document
providing high-level, concise documentation for API design and development. The
goal is for these documents to serve as the source of truth for API-related
documentation and the way API teams discuss and come to consensus on API
guidance.
AEPs are most useful when they are clear and concise, and cover a single topic
or inquiry well. In the same way that AEPs describe consistent patterns and
style for use in APIs, they also _follow_ consistent patterns and style.
## Guidance
- AEPs cover a single, discrete topic, and provide clear, actionable
guidance.
- AEPs duplicate or contradict guidance in another AEP.
- AEPs also cover what _not_ to do, but cover _only_
anti-patterns.
- If AEP guidance is conditional (e.g. a design pattern such as Jobs), the
guidance clearly explain under what conditions the guidance should
be followed.
Guidance contained within an AEP be beneficial to one or more types of
clients or their authors, including but not limited to:
- Asset inventories which can be used to audit and analyze resources.
- Command line interfaces for exploration and simple automation.
- Custom controllers (e.g. auto-scalers) which poll live state and adjust
resource configuration accordingly.
- [Declarative clients][] for orchestration and automation of multiple
resources.
- Recommendation tools which provide guidance on which APIs are useful for
specific use cases, and how to use them.
- SDKs to interact with an API from a programming language, often used heavily
for data-plane operations.
- Security orchestration, automation, and remediation tools.
- Simple scripts to automate or orchestrate tasks.
- Test frameworks.
- Tools that operate on resource data at rest.
- Visual User Interfaces for visualization and one-off manual actions.
- Users.
Examples of enhancements include:
- Requiring new proto annotations that enable more descriptive interfaces on
clients (e.g. requiring `singular` and `plural` on a `google.api.resource`
annotation).
AEP guidance be a significant detriment to a client's usability or
implementation difficulty, or maintenance difficulty.
Examples of detriments include:
- Introduction of a non-uniform pattern in a standard method such that all
clients must introduce additional code without sufficient benefit (e.g. List
behaves like this _except_ for resources that start with the name Foo).
- Renames of well-established fields for minor enhancements in readability
(e.g. rename `expire_time` to `lapse_time` since `lapse` is a common term in
my service).
While the length of AEPs will necessarily vary based on the complexity of the
question, most AEPs be able to cover their content in roughly two
printed pages.
### File structure
AEPs be written in Markdown, and be named using their
four-digit number (example: `0008.md`). AEPs that serve a specific scope
be in the subdirectory for that scope.
AEPs have appropriate front matter.
```yaml
---
id: 8
state: reviewing
created: 2019-05-28
permalink: /8
redirect_from:
- /08
- /008
- /0008
---
```
Front matter for AEPs include:
- The `aep` key:
- `id`: Required. The ID for the given AEP, as an integer.
- `state`: Required. The current state of the AEP, in all lower-case. The
valid states are listed in [AEP-1][], and common states are `draft`,
`reviewing`, and `approved`.
- `created`: Required. The ISO-8601 date (`yyyy-mm-dd`) when the AEP was
originally drafted, with no quotes.
- `updated`: The ISO-8601 date (`yyyy-mm-dd`) when the AEP was last revised.
- `scope`: The scope for the AEP. This match the directory name for
that scope. Required for AEPs with IDs \>= 1000, prohibited otherwise.
- The `permalink` key (required): This be set to
`/{aep.scope}/{aep.id}`. If there is no scope, use `/{aep.id}` instead.
- The `redirect_from` key: This should include a list of any `/{aep.id}`
permutations that a reader would be likely to enter, including:
- `/{aep.id}` (for AEPs where the permalink includes the scope)
- AEP IDs with zero-padding, for each level of zero-padding up to four digits
(for example: `/08`, `/008`, `/0008`).
### Document structure
AEPs begin with a top-level heading with the AEP's title (`# Title`).
The title be a noun (not an imperative). For example, "Bad API
precedents" not "Avoid breaking API precedent".
AEPs then begin with an introduction (with no additional heading),
followed by a `## Guidance` heading. If necessary, the AEP include any
of the following after the guidance, in the following order:
- "Further reading" is a bulleted list of links to other AEPs that are useful
to fully understand the current AEP.
- "Appendices" covering further explanation in the same AEP. These are
relatively rare but are important in cases where an AEP requires a lot of
justification for the decision. Often this is primarily an explanation of
alternatives considered to help explain the guidance.
- "Changelog" is a bulleted list of changes made to the AEP since the first
writing.
The guidance section include subsections that elaborate further on
details. Subsections will automatically create an entry in the table of
contents, and an anchor for citations.
Below is an example AEP shell that uses each major section:
```md
# AEP title
The introductory text explains the background and reason why the AEP exists. It
lays out the basic question, but does not tell the reader what to do.
## Guidance
The "guidance" section helps the reader know what to do. A common format for
the guidance section is a high-level imperative, followed by an example,
followed by a bulleted list explaining the example.
### Subsection
Individual subsections can be cited individually, and further elaborate
details.
## Rationale
The "rationale" section is optional, and helps the reader understand the
motivation behind specific guidance within the AEP.
Deeper explanations of design justification and tradeoffs be in the
rationale instead of other sections, to ensure the rest of the document acts as
an easily actionable reference.
## History
The "history" section is optional, and documents events and context around a
significant edit to an AEP. For example, explanation of rewrite would be
included in this section
While the changelog is a dotted list of one-line summaries of changes to an
AEP, the history section should elaborate on significant events in a
descriptive format.
The section be used to exhaustively enumerate all changes. This is
what the changelog provides.
## Further reading
A bulleted list of (usually) other AEPs, in the following format:
- [AEP-1](/0001): AEP purpose and guidelines
## Changelog
A bulleted list of changes in reverse chronological order, using the following
format:
- **2020-02-18**: Specified ordering.
- **2019-07-01**: Added a subsection clarifying XYZ.
```
AEPs attempt to follow this overall format if possible, but AEPs
deviate from it if necessary (in particular, if the AEP would be more
difficult to understand, even for a reader already accustomed to reading AEPs
in the usual format).
### Requirement keywords
AEPs use the following requirement level keywords: "MUST", "MUST
NOT", "SHOULD", "SHOULD NOT", and "MAY", which are to be interpreted as
described in [RFC 2119][].
When using these terms in AEPs, they be lower-case and **bold**. These
terms be used in other ways.
If "SHOULD" or "SHOULD NOT" are used, they include valid examples of
where other concerns may override the guidance.
### Code examples
API design examples in AEPs be presented in both [OpenAPI][] and
[protocol buffers][]. Examples cover only enough syntax to explain
the concept. When using RPCs in examples, a `google.api.http` annotation
be included.
[OpenAPI][] examples be authored as
[OAS version 3.1](https://spec.openapis.org/oas/v3.1.1.html).
AEPs should use snake-case naming for parameters and properties (fields), for
consistency across OpenAPI and protocol buffers, but other casing conventions
may be used as long as they are applied uniformly.
### Referencing AEPs
When AEPs reference other AEPs, the prosaic text use the format
`AEP-XXXX` without zero-padding (e.g., `AEP-8`, not `AEP-0008`), and
link to the relevant AEP. AEP links point to a particular section of
the AEP if appropriate.
### Error codes
When referencing a error code, the prose use the format
`{google.api.error_code} / {http_status_code}`. For example, `OK / 200`.
## Rationale
### Designing for a broad set of clients
API guidance, similar to any software, is most beneficial when there is a clear
purpose and target beneficiary.
The beneficiaries of improved API design are users. These users interact with
APIs via a variety of clients, depending on their use case as enumerated above.
API guidance must in turn consider the impact broadly across these clients.
## Changelog
- **2024-03-02**: From from https://google.aip.dev/8
[Declarative clients]: /003.md#declarative-clients
[OpenAPI]: https://www.openapis.org/
[protocol buffers]: https://developers.google.com/protocol-buffers/
[rfc 2119]: https://www.ietf.org/rfc/rfc2119.txt
---
# AEP-100 API Design Review FAQ
---
# AEP-101 OpenAPI
This AEP describes the usage of OpenAPI within the AEPs and AEP-compliant APIs.
## Guidance
### OAS versions
AEP-compliant APIs document their HTTP APIs using
[version 3.1.x](https://spec.openapis.org/oas/v3.1.1.html) of the OpenAPI
Specification (OAS).
Upon release of new minor versions of the OAS 3 specification, AEP-compliant
APIs update their specification to use these new versions within 12
months of the release of that version.
AEP tooling which is officially supported by the project support OAS
3.1.x.
---
# AEP-102 APIs and API terminology
This AEP describes the terms used to describe various parts of an API
specification in depth.
## Guidance
### API Name
An API name is a string that uniquely identify the API. It serves as a
namespace where elements of an API are defined, including:
- [resource types][/resource-types]
- [methods][/standard-methods]
In practice, it is difficult to prevent the re-use of the name across different
APIs. Therefore the API should use something, such as the domain name of the
primary server hosting the API and a possible path prefix, as the name.
If a primary server does not exist (for example, a public cloud where every
domain always includes a regional subdomain), then a similar placeholder that
can still ensure uniqueness can be used. For example APIs served at
`{region}.cloud-storage.public-cloud.com` may use
`cloud-storage.public-cloud.com`.
The API Name:
- only uses valid domain name characters as specified in
[RFC 1035](https://datatracker.ietf.org/doc/html/rfc1035#section-2.3.1), or a
backslash `[.-a-z0-9/]`)
- use all lower case.
- use path prefixes that could also be misconstrued as a
resource collection (e.g. `apis.examples.com/users`)
### API Name Examples
- `networking.istio.io`
- `pubsub.example.com`
- `spanner.example.com/v1`
- `apis.example.com/user` (valid, but discouraged to possible confusion with a
collection name.).
## API Services
An API service is distinct from an API name: A name is a common string that is
used to define an API agnostic of the address by which it is served, while an
API service is a live service which exposes an API. For example, a staging and
production environment (`staging.service.com`), or different regions
(`us-west.service.com`) may have different API services, but will share the
same API.
In many cases, the address of an API service and an API name may be identical
(for example, an api `api.example.com` may expose that same API at
`api.example.com`).
An API name be assumed to be an address of an API. Instead, use a
location where that metadata is stored. For example, OpenAPI provides a
[server object](https://spec.openapis.org/oas/v3.1.1.html#server-object) for
this purpose.
---
# AEP-121 Resource-oriented design
Resource-oriented design is a pattern for specifying [RPC][] APIs, based on the
following high-level design principles:
- The fundamental building blocks of an API are individually-named _resources_
(nouns) and the relationships and hierarchy that exist between them.
- A small number of standard _methods_ (verbs) provide the semantics for most
common operations. However, custom methods are available in situations where
the standard methods do not fit.
Readers might notice similarities between these principles and some principles
of [REST][]; resource-oriented design borrows many principles from REST, while
also defining its own patterns where appropriate.
## Guidance
When designing an API, consider the following:
- The resources (nouns) the API will provide.
- The relationships and hierarchies between those resources.
- The schema of each resource.
- The methods (verbs) each resource provides, relying as much as possible on
the standard verbs.
### Resources
A resource-oriented API be modeled as a resource hierarchy, where each
node is either a simple resource or a collection of resources.
A [_collection_](./0003#collection) contains resources of _the same type_. For
example, a publisher has the collection of books that it publishes. A resource
usually has fields, and resources may have any number of sub-resources (usually
collections).
### Methods
Resource-oriented APIs emphasize resources (data model) over the methods
performed on those resources (functionality). A typical resource-oriented API
exposes a large number of resources with a small number of methods on each
resource. The methods can be either the standard methods ([Get][], [List][],
[Create][], [Update][], [Delete][]), or [custom methods][].
If the request to or the response from a standard method (or a custom method in
the same _service_) **is** the resource or **contains** the resource, the
resource schema for that resource across all methods be the same.
| Standard method | Request | Response |
| --------------- | --------------------- | --------------- |
| Create | Contains the resource | Is the resource |
| Get | None | Is the resource |
| Update | Contains the resource | Is the resource |
| Delete | None | None |
| List | None | Is the resource |
_The table above describes each standard method's relationship to the resource,
where "None" indicates that the resource neither **is** nor **is contained** in
the request or the response_
A resource support at minimum [Get][]: clients must be able to
validate the state of resources after performing a mutation such as [Create][],
[Update][], or [Delete][].
A resource also support [List][], except for [singleton resources][]
where more than one resource is not possible.
APIs prefer standard methods over custom methods; the purpose of
custom methods is to define functionality that does not cleanly map to any of
the standard methods. Custom methods offer the same design freedom as
traditional RPC APIs, which can be used to implement common programming
patterns, such as database transactions, import and export, or data analysis.
### Strong Consistency
A method is strongly consistent if, upon completion of a request of that method
(regardless of whether the request is successful,
[long-running][long-running-requests] or synchronous), all user-settable fields
will return the same value on any subsequent requests until another request
which mutates the resource is completed.
[Output only][output only] fields unrelated to the resource [state][]
also return the same value on each subsequent request until another
request which mutates the resource is completed.
- An [output only][] field that takes so long to reach a steady-state that it
would negatively impact the user experience of the request (e.g. an hour-long
instantiation of a VM cluster).
Examples of strong consistency include:
- Following a successful create that is the latest mutation on a resource, a
get request for a resource return the resource.
- Following a successful update that is the latest mutation on a resource, a
get request for a resource return the final values from the update
request.
- Following a successful delete that is the latest mutation on a resource, a
get request for a resource return `NOT_FOUND`.
Clients of resource-oriented APIs often need to orchestrate multiple operations
in sequence (e.g., create resource A, create resource B which depends on A),
and ensuring that resources immediately reflect steady user state after an
operation is complete ensures clients can rely on method completion as a signal
to begin the next operation.
[Output only][output only] fields ideally would follow the same guidelines, but
as these fields can often represent a resources live state, it's sometimes
necessary for these values to change after a successful mutation operation to
reflect a state change.
### Cyclic References
The relationship between resources, such as parent-child or [resource
references][], be representable via a [directed acyclic graph][].
A cyclic relationship between resources increases the complexity of managing
resources. Consider resources A and B that refer to each other. The process to
create said resources are:
1. create resource A without a reference to B. Retrieve id for resource A.
2. create resource B with a reference to A. Retrieve id for resource B.
3. update resource A with the reference to B.
The delete operation may also become more complex, due to reasoning about which
resource must be dereferenced first for a successful deletion.
This requirement does not apply to relationships that are expressed via [output
only][] fields, as they do not require the user to specify the values and in
turn do not increase resource management complexity.
[create]: /133
[custom methods]: /136
[delete]: /135
[directed acyclic graph]: https://en.wikipedia.org/wiki/Directed_acyclic_graph
[get]: /131
[list]: /132
[long-running-requests]: /151
[output only]: /203#output-only
[rest]: https://en.wikipedia.org/wiki/Representational_state_transfer
[resource references]: /122#fields-representing-another-resource
[rpc]: https://en.wikipedia.org/wiki/Remote_procedure_call
[singleton resources]: /156
[soft delete]: /164
[state]: /216
[stateless protocol]: https://en.wikipedia.org/wiki/Stateless_protocol
[update]: /134
## Changelog
- **2024-01-27**: From from https://google.aip.dev/121
---
# AEP-122 Resource paths
Most APIs expose _resources_ (their primary nouns) which users are able to
create, retrieve, and manipulate. Additionally, resources have _paths_: each
resource has a unique (within the API service) identifying path that users use
to reference that resource, and these paths are what users should _store_ as
the canonical identifier for the resources.
## Guidance
### Resource Path
A resource path refers to the the identifier for a specific resource, unique
within an API.
- A resource path be unique with an API, referring to a single
resource.
See the section on [full resource paths](#full-resource-paths) below for more
information on referring to resources across APIs.)
Resource paths are formatted according to the [URI path schema][], but without
the leading slash:
```
publishers/123/books/les-miserables
users/vhugo1802
```
- Resource path components alternate between collection identifiers
(example: `publishers`, `books`, `users`) and resource IDs (example: `123`,
`les-miserables`, `vhugo1802`), _except_ when [singleton resources][] are
present.
- Resource paths use the `/` character to separate individual segments
of the resource path.
- Each segment of a resource path contain a `/` character.
- Resource paths only use characters available in DNS names, as
defined by [RFC-1123](https://tools.ietf.org/html/rfc1123).
- Additionally, resource IDs use upper-case letters.
- If additional characters are necessary, resource paths use
characters that require URL-escaping, or characters outside of ASCII.
- If Unicode characters can not be avoided, resource paths be stored
in Normalization Form C (see [AEP-210][]).
- Each resource have a `path` field that contains its resource path.
- Resources provide the resource ID, i.e. the last segment of the
path, as a separate field named `id`.
- Resources expose tuples, self-links, or other forms of
resource identification.
- All ID fields be strings.
[uri path schema]: https://datatracker.ietf.org/doc/html/rfc3986#appendix-A
[singleton resources]: https://aep.dev/singletons
### Full resource paths
In most cases, resource paths are used within a single API only. However,
sometimes it is necessary for services to refer to resources in an arbitrary
API. In this situation, the service use the _full resource path_, a
schemeless URI with the owning API's service endpoint, followed by the relative
resource path:
```
//apis.example.com/library/publishers/123/books/les-miserables
//apis.example.com/calendar/users/vhugo1802
```
### Collection identifiers
The collection identifier segments in a resource path be the plural
form of the noun used for the resource. (For example, a collection of
`Publisher` resources is called `publishers` in the resource path.)
- Collection identifiers be concise American English terms.
- Collection identifiers be in `kebab-case`.
- Collection identifiers begin with a lower-cased letter and contain
only lower-case ASCII letters, numbers. and hyphens (`/[a-z][a-z0-9-]*/`).
- Collection identifiers be plural.
- In situations where there is no plural word ("info"), or where the singular
and plural terms are the same ("moose"), the non-pluralized (singular) form
is correct. Collection segments "coin" words by adding "s" in
such cases (e.g. avoid "infos").
#### Nested collections
If a resource path contains multiple levels of a hierarchy, and a parent
collection's path is used as a prefix for the child resource's path, the child
collection's path omit the prefix. For example, given a collection of
`UserEvent` resources that would normally be nested underneath `users`:
```
users/vhugo1802/user-events/birthday-dinner-226/user-event-guests/123
```
An API use the less-redundant form:
```
users/vhugo1802/events/birthday-dinner-226/guests/123
```
In this situation, the _message_ is still called `UserEvent` or
`UserEventGuest`; only the collection name is shortened.
### Resource ID
A resource ID segment identifies the resource within its parent collection. In
the resource path `publishers/123/books/les-miserables`, `123` is the resource
ID for the publisher, and `les-miserables` is the resource ID for the book.
- Resource IDs be either always set by users (required on resource
creation), optionally set by users (optional on resource creation,
server-generated if unset), or never set by users (not accepted at resource
creation). They be immutable once created.
- If resource IDs are user-settable, the API document and/or annotate
the field with the allowed formats. User-settable resource IDs
conform to [RFC-1034][]; which restricts to letters, numbers, and hyphen,
with the first character a letter, the last a letter or a number, and a 63
character maximum.
- Additionally, user-settable resource IDs restrict letters to
lower-case (`^[a-z]([a-z0-9-]{0,61}[a-z0-9])?$`).
- Characters outside of ASCII be permitted; however, if
Unicode characters are necessary, APIs follow guidance in
[AEP-210][].
- User-settable IDs be permitted to be a UUID (or any value
that syntactically appears to be a UUID).
- Field annotations use [protovalidate][] in protobuf and [JSON
Schema keywords][] like [`pattern`][pattern] with OAS/JSON Schema.
- If resource IDs are not user-settable, the API document the basic
format, and any upper boundaries (for example, "at most 63 characters").
- For more information, see the [create][] standard method.
[create]: /133.md#user-specified-ids
[rfc-1034]: https://tools.ietf.org/html/rfc1034
[JSON Schema keywords]: https://swagger.io/docs/specification/data-models/keywords/
### Resource ID aliases
It is sometimes valuable to provide an alias for common lookup patterns for
resource IDs. For example, an API with `users` at the top of its resource
hierarchy may wish to provide `users/me` as a shortcut for retrieving
information for the authenticated user.
APIs provide programmatic aliases for common lookup patterns. However,
all data returned from the API use the canonical resource path.
### Resource URIs
The full resource path is a schemeless URI, but slightly distinct from the full
URIs we use to access a resource. The latter adds two components: the protocol
(HTTPS) and the API version:
```
https://apis.example.com/library/v1/publishers/123/books/les-miserables
https://apis.example.com/calendar/v3/users/vhugo1802
```
The version is not included in the full resource path because the full resource
path is expected to persist from version to version. Even though the API
surface may change between major versions, multiple major versions of the same
API are expected to use the same underlying data.
### Fields representing resource paths
When defining a resource, the first field be the resource path,
which be of type `string` and be called `path` for the
resource path. The message include a `google.api.resource`
annotation declaring the type (see [AEP-4](/4) for more on this).
```proto
// A representation of a book in the library.
message Book {
option (google.api.resource) = {
type: "apis.example.com/library/Book"
pattern: "publishers/{publisher_id}/books/{book_id}"
};
// The resource path of the book.
// Format: publishers/{publisher_id}/books/{book_id}
string path = 1;
// Other fields...
}
```
When defining a method that retrieves or acts on an already-existing resource
(such as `GetBook` or `ArchiveBook`), the first field of the request message
be the resource path, which be of type `string` and
be called `path` for the resource path. The field also be
annotated with `api.api.field_info`, referencing the resource type (AEP-4).
```proto
// Request message for ArchiveBook
message ArchiveBookRequest {
// The book to archive.
// Format: publishers/{publisher_id}/books/{book_id}
string path = 1 [
(aep.api.field_info) = { resource_reference: [ "apis.example.com/library/book" ], field_behavior: [ FIELD_BEHAVIOR_REQUIRED ] }
];
// Other fields...
}
```
### Fields representing a resource's parent
When defining a method that retrieves resources from a collection or adds a new
resource to a collection (such as `ListBooks` or `CreateBook`), the first field
of the request message be of type `string` and be called
`parent` for the resource path of the collection. The `parent` field
also be annotated with `api.api.field_info`, referencing the parent's resource
type (AEP-4).
```proto
// Request message for ListBooks.
message ListBooksRequest {
// The publisher to list books from.
// Format: publishers/{publisher_id}
string parent = 1 [(aep.api.field_info) = { resource_reference: [ "apis.example.com/library/publisher" ], field_behavior: [ FIELD_BEHAVIOR_REQUIRED ] }];
// Other fields (e.g. max_page_size, page_token, filter, etc.)...
}
```
If there is more than one possible parent type, the `parent` field
be annotated with multiple `resource_reference` types using
`aep.api.field_info`:
```proto
// Request message for ListBooks.
message ListBooksRequest {
// The parent to list books from.
// Format:
// - publishers/{publisher_id}
// - authors/{author_id}
string parent = 1 [
(aep.api.field_info) = { resource_reference: [ "apis.example.com/library/book" ], field_behavior: [ FIELD_BEHAVIOR_REQUIRED ] }
];
// Other fields (e.g. max_page_size, page_token, filter, etc.)...
}
```
### Fields representing another resource
When referencing a resource path for a different resource:
- The field be of type `string`.
- The value of the field be one of:
- the full resource path.
- the resource path, only if the referenced resource is also local to the API
of the current resource.
- The field name be equivalent to the corresponding message's name
in snake case.
- Field names include a leading adjective if appropriate (such as
`string dusty_book`).
- Field names use the `_path` suffix unless the field would be
ambiguous without it (e.g., `crypto_key_path`)
- In protobuf, fields representing another resource provide the
`aep.api.field_info` annotation with the resource type being referenced.
```proto
// A representation of a book in a library.
message Book {
option (google.api.resource) = {
type: "apis.example.com/library/Book"
pattern: "publishers/{publisher_id}/books/{book_id}"
};
// Path of the book.
// Format is `publishers/{publisher_id}/books/{book_id}`
string path = 1;
// The shelf where the book currently sits.
// Format is `shelves/{shelf}`.
string shelf = 2 [(aep.api.field_info) = { resource_reference: [ "apis.example.com/library/shelf" ], field_behavior: [ FIELD_BEHAVIOR_REQUIRED ] }];
// Other fields...
}
```
## Further reading
- For resource types, see [AEP-4](/4).
[protovalidate]: https://github.com/bufbuild/protovalidate
[pattern]: https://json-schema.org/understanding-json-schema/reference/string#regexp
---
# AEP-124 Resource association
APIs sometimes have resource relationships that can not be cleanly expressed in
a hierarchical structure. For example, a resource may have a many-to-one
relationship with two other resource types instead of just one. Alternatively,
a resource may have a many-to-many relationship with another resource type.
## Guidance
A resource have at most one canonical parent, and `List` requests
require two distinct "parents".
Also see
[resource paths for guidance on fields representing another resource](/0122#fields-representing-another-resource).
### Multiple many-to-one associations
If a resource has a many-to-one relationship with multiple resource types, it
choose at most one of them to be the parent. The resource be
associated with other resources through other fields on the resource.
When listing resources with multiple associations in this way, the RPC
treat the `string parent` field as required as discussed in [list](/list), and
add additional required arguments. The RPC include a
`string filter` field that allows users to filter by other resource
associations as discussed in [filtering](/filtering).
### Many-to-many associations
Many-to-many associations are less common in APIs than they are in relational
databases, in part because they are more difficult to model and present over
network interfaces.
An API contain many-to-many relationships, and use a
repeated field containing a list of resource paths, following the principles
described for repeated fields in [arrays][/arrays].
If the use of a repeated field is too restrictive, or if more metadata is
required along with the association, an API model a many-to-many
relationship using a sub-resource with two one-to-many associations.
### Embedded resources
Resource references as described use string references rather than embedding
one resource inside another because embedding resources can lead to a number of
issues. Retrieving resources with arbitrarily deep nesting may require multiple
serial internal RPCs, leading to complex service dependencies, latency, and
reliability issues.
However, sometimes dereferencing resource references is useful, or even
necessary.
For example, an API may wish to allow filtering of a resource based on a field
of a resource it references. This can be necessary in order to satisfy a query
expressing something like "list books written by authors who were born before
1950", where the author's birth date is a field on the `Author` resource
referenced by the `Book` resource.
In other cases, it may be much cheaper and/or faster for the server to perform
the dereferencing than for the client to make multiple serial requests.
In these cases, an alternative resource association pattern be used,
where the resource reference field is not a string, but is instead the
referenced resource itself.
The default behavior be to populate only the `path` field of the
referenced resource, making it equivalent to a string-based resource reference.
Additional fields be included based on the request: for example, if the
request includes a [view or read mask](./partial-responses) that specifies that
additional fields should be included, or if the request contains a filter that
specifies fields other than `path` within the referenced resource.
Fields other than `path` in embedded resource references be treated as
[output-only](./field-behavior); mutating API methods allow
creating or mutation of embedded resources. For example, if a `Book` resource
has an embedded resource reference to `Author`, the `UpdateBook` method **must
not** allow updating any fields of the `Author` resource other than
`Author.path` (which updates the reference itself, rather than the referenced
resource).
APIs always fully document their behavior with embedded resources,
even if the only supported behavior is populating only the `path` field. If the
referenced resource itself contains embedded resources, the API
include clear documentation about the depth to which embedded resources may be
dereferenced.
---
# AEP-126 Enumerations
It is common for a field to only accept or provide a discrete and limited set
of values. In these cases, it can be useful to use enumerations (generally
abbreviated "enums") in order to clearly communicate what the set of allowed
values are.
## Guidance
APIs expose enum objects for sets of values that are expected to change
infrequently:
```typescript
// Possible formats in which a book may be published.
enum Format {
// The printed format, in hardback.
Hardback = 'HARDBACK',
// The printed format, in paperback.
Paperback = 'PAPERBACK',
// An electronic book format.
Ebook = 'EBOOK',
// An audio recording.
Audiobook = 'AUDIOBOOK',
}
```
- All enum values use a consistent case format across an
organization. In many cases, this is dictated by the IDL the organization
uses.
- Enums document whether the enum is frozen or they expect to add
values in the future.
### When to use enums
Enums can be more accessible and readable than strings or booleans in many
cases, but they do add overhead when they change. Therefore, enums
receive new values infrequently. While the definition of "infrequently" may
change based on individual use cases, a good rule of thumb is no more than once
a year. For enums that change frequently, the API use a string and
document the format.
### Alternatives
Enums be used when there is a competing, widely-adopted standard
representation (such as with [language codes][bcp-47] or [media types][]).
Instead, that standard representation be used. This is true even if
only a small subset of values are permitted, because using enums in this
situation often leads to frustrating lookup tables when trying to use multiple
APIs together.
For enumerated values where the set of allowed values changes frequently, APIs
use a `string` field instead, and document the allowed
values. String fields with enumerated values use a uniform case
system (`snake_case`, `kebab-case`, etc.) throughout an organization.
Boolean fields be used in situations where it is clear that no further
flexibility will be needed. The default value be `false`.
### Compatibility
Adding values to an enum has the potential to be disruptive to existing
clients. Consider code written against the `Format` enum in an earlier version
where only the first two options were available:
```typescript
switch (book.format) {
case Format.Hardback:
// Do something...
break;
case Format.Paperback:
// Do something...
break;
default:
// When new enum values are introduced, pre-existing client code may
// throw errors or act in unexpected ways.
throw new Error('Unrecognized value.');
}
```
Services add new values to existing enums; however, they add
enums carefully; think about what will happen if a client system does not know
about a new value.
Additionally, in IDLs where enum values are presented in a specific order,
services only add new values to the end. An exception to this rule
is if the enum conforms to an external standard (for example, an enum
representing HTTP status codes would add a new 3xx value alongside the others,
not at the end).
## Interface Definitions
## Further reading
- For states, a special type of enum, see [AEP-216](/216).
[bcp-47]: https://en.wikipedia.org/wiki/IETF_language_tag
[media types]: https://en.wikipedia.org/wiki/Media_type
---
# AEP-127 HTTP and gRPC Transcoding
APIs that follow [resource-oriented design](/121) are defined using
[RPCs][rpc], but the resource-oriented design framework allows them to also be
presented as APIs that largely follow REST/JSON conventions. This is important
in order to help developers use their existing knowledge: over 80% of the
public APIs available follow most REST conventions, and developers are
accustomed to that pattern.
## Guidance
Protobuf APIs provide HTTP definitions for each RPC that they define,
except for bi-directional streaming RPCs, which can not be natively supported
using HTTP/1.1. When providing a bi-directional streaming method, an API
also offer an alternative method that does not rely on
bi-directional streaming.
### HTTP method and path
When using protocol buffers, each RPC define the HTTP method and path
using the [`google.api.http`][google.api.http] annotation:
```proto
rpc CreateBook(CreateBookRequest) returns (Book) {
option (google.api.http) = {
post: "/v1/{parent=publishers/*}/books"
body: "book"
};
}
message CreateBookRequest {
// The publisher who will publish this book.
// When using HTTP/JSON, this field is automatically populated based
// on the URI, because of the `{parent=publishers/*}` syntax.
string parent = 1 [
(aep.api.field_info) = { resource_reference: [ "library.example.com/book" ], field_behavior: [ FIELD_BEHAVIOR_REQUIRED ] }
];
// The book to create.
// When using HTTP/JSON, this field is populated based on the HTTP body,
// because of the `body: "book"` syntax.
Book book = 2 [(google.api.field_behavior) = REQUIRED];
// The user-specified ID for the book.
// When using HTTP/JSON, this field is populated based on a query string
// argument, such as `?book_id=foo`. This is the fallback for fields that
// are not included in either the URI or the body.
// Note that clients use camelCase format to communicate the field names
// to the service.
string book_id = 3;
}
```
- The first key (`post` in this example) corresponds to the HTTP method. RPCs
use `get`, `post`, `patch`, or `delete`.
- RPCs use the prescribed HTTP verb for each standard method, as
discussed in [Get](/get), [List](/list), [Create](/create),
[Update](/update), and [Delete](/delete)
- RPCs use the prescribed HTTP verb for custom methods, as
discussed in [Custom Methods](/custom-methods).
- RPCs use `put` or `custom`.
- The corresponding value represents the URI.
- URIs use the `{foo=bar/*}` syntax to represent a variable that
should be populated in the request proto. When extracting a
[resource name](/resource-paths), the variable include the entire
resource name, not just the ID component.
- URIs use nested fields for their variable names. (Additionally,
[AEP-134](/134) mandates this for `Update` requests.)
- URIs use the `*` character to represent ID components, which
matches all URI-safe characters except for `/`. URIs use `**` as
the final segment of a URI if matching `/` is required.
- The `body` key defines which single top-level field in the request will be
sent as the HTTP body. If the body is `*`, then this indicates that the
request object itself is the HTTP body. The request body is encoded as JSON
as defined by protocol buffers' canonical [JSON encoding][json encoding].
- RPCs define a `body` at all for RPCs that use the `GET` or
`DELETE` HTTP verbs.
- RPCs use the prescribed `body` for Create ([Create](/create)) and
Update ([AEP-134](/update)) requests.
- RPCs use the prescribed `body` for custom methods
([Custom Methods](custom-methods)).
- The `body` contain a nested field (or use the `.` character),
- The `body` be the same as a URI parameter.
- The `body` be a `repeated` field.
- In accordance with the [field name casing guidelines][field case], fields
use the `json_name` annotation to specify the snake_case form
when the default JSON conversion would differ, unless this would break
backwards compatibility.
### Multiple URI bindings
Occasionally, an RPC needs to correspond to more than one URI:
```proto
rpc CreateBook(CreateBookRequest) returns (Book) {
option (google.api.http) = {
post: "/v1/{parent=publishers/*}/books"
body: "book"
additional_bindings: {
post: "/v1/{parent=authors/*}/books"
body: "book"
}
additional_bindings: {
post: "/v1/books"
body: "book"
}
};
}
```
- RPCs define any number of additional bindings. The structure is
identical to the `google.api.http` annotation (in fact, it is a recursive
reference).
- RPCs define an additional binding within an additional binding.
- The `body` clause be identical in the top-level annotation and each
additional binding.
[json encoding]: https://developers.google.com/protocol-buffers/docs/proto3#json
[field case]: /140.md#case
[rpc]: https://en.wikipedia.org/wiki/Remote_procedure_call
[google.api.http]: https://buf.build/googleapis/googleapis/docs/main:google.api#google.api.Http
---
# AEP-130 Methods
An API is composed of one or more methods, which represent a specific operation
that a service can perform on behalf of the consumer.
## Guidance
### Categories of Methods
The following enumerates multiple categories of methods that exist, often
grouped up under some object (e.g. collection or resource) that the method
operates upon.
| Category Name | Related AIPs | [Declarative client][] integration | CLI / UI integration | SDK integration |
| ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------- | ---------------------------------- | -------------------- | --------------- |
| _Standard Methods_ | | | | |
| **Standard collection methods**: operate on a collection of resources (List or Create). | [resources], [list], [create] | automatable | automatable | automatable |
| **Standard resource methods**: fetch or mutate a single resource (Get, Update, Delete). | [resources], [get], [update], [delete] | automatable | automatable | automatable |
| **Batch resource methods**: fetch or mutate multiple resources in a collection by name. | [batch-get], [batch-create], [batch-update], [batch-delete] | may be used to optimize queries | automatable | automatable |
| **Aggregated list methods**: fetch or mutate multiple resources of the same type across multiple collections. | [reading-across-collections] | not useful nor automatable | automatable | automatable |
| _Custom Fetch Methods_ | | | | |
| **Custom collection fetch methods**: fetch information across a collection that cannot be expressed via a standard method. | [custom-methods] | handwritten | automatable | automatable |
| **Custom resource fetch methods**: fetch information for a single resource that cannot be expressed via a standard method. | [custom-methods] | handwritten | automatable | automatable |
| _Custom Mutation Methods_ | | | | |
| **Backing up a resource**: storing a copy of a resource at a particular point in time. | [resource-revisions] | unused or handwritten | automatable | automatable |
| **Restoring a resource**: setting a resource to a version from a particular point in time. | [resource-revisions] | unused or handwritten | automatable | automatable |
| **Renaming a resource**: modify the resource's name or id while preserving configuration and data. | [custom-methods] | unused or handwritten | automatable | automatable |
| **Custom collection mutation methods**: perform an imperative operation referencing a collection that may mutate one or more resources within that collection in fashion that cannot be easily achieved by standard methods (e.g. state transitions). | [custom-methods] | unused or handwritten | automatable | automatable |
| **Custom resource mutation methods**: perform an imperative operation on a resource that may mutate it in a way a standard method cannot (e.g. state transitions). | [custom-methods] | unused or handwritten | automatable | automatable |
| _Misc Custom Methods_ | | | |
| **Stateless Methods**: a method that has no permanent effect on any data within the API (e.g. translating text) | [custom-methods] | unused or handwritten | automatable | automatable |
| _None of the above_ | | | | |
| **Streaming methods**: methods that communicate via client, server, or bi-directional streams. | | handwritten | handwritten | automatable |
### Choosing a method category
While designing a method, API authors should choose from the defined categories
in the following order:
1. Standard methods (on collections and resources)
1. Standard batch or aggregate methods
1. Custom methods (on collections, resources, or stateless)
1. Streaming methods
### OperationIDs and RPC names
The OpenAPI specification includes an
[operationId](https://spec.openapis.org/oas/latest.html#fixed-fields-7) field
to uniquely identify an operation, as well as provide a name for the operation
in tools and libraries.
In a similar fashion, gRPC has
[RPCs definition within a service](https://grpc.io/docs/what-is-grpc/core-concepts/#service-definition).
The operationId and RPCs names follow the following format:
- `{standard-method-name}{resource-singular}` for non-list standard methods.
- `List{resource-plural}` for lists.
- `:{custom-method-name}{resource-singular}` for custom methods (omit the
leading colon for gRPC).
Some examples include:
- `CreateBook`
- `GetBook`
- `ApplyBook`
- `ListBooks`
- `:ArchiveBook`
## Rationale
Resource-oriented standard and custom methods are recommended first, as they
can be expressed in the widest variety of clients (Declarative clients, CLIs,
UIs, and so on), and offer the most uniform experience that allows users to
apply their knowledge of one API to another.
If a standard method is unsuitable, then custom methods (that are mounted to a
resource or collection) offer a lesser, but still valuable level of
consistency, helping the user reason about the scope of the action and the
object whose configuration is read to inform that action. Although mutative
custom methods are not uniform enough to have an automated integration with
exclusively resource-oriented clients such as [Declarative clients][], they are
still a pattern that can be easily recognized by CLIs, UIs, and SDKs.
If one cannot express their APIs in a resource-oriented fashion at all, then
the operation falls in a category where the lack of uniformity makes it
difficult for any client aside from SDKs to model the operation. This category
is preferred last due to the fact that a user cannot rely on their knowledge of
similar APIs, as well as the issue that integration with many clients will
likely have to be hand-written.
[batch-create]: /batch-create
[batch-delete]: /batch-delete
[batch-get]: /batch-get
[batch-update]: /batch-update
[create]: /create
[custom-methods]: /custom-methods
[Declarative client]: /003.md#declarative-clients
[Declarative clients]: /003.md#declarative-clients
[delete]: /delete
[get]: /get
[list]: /list
[update]: /update
[reading-across-collections]: /reading-across-collections
[resource-revisions]: /resource-revisions
[resources]: /resources
## Changelog
- **2024-04-10**: Imported from https://aip.dev/130.
---
# AEP-131 Get
In REST APIs, it is customary to make a `GET` request to a resource's URI (for
example, `/v1/publishers/{publisher_id}/books/{book_id}`) in order to retrieve
that resource.
Resource-oriented design [AEP-121](/121) honors this pattern through the `Get` method.
These RPCs accept the URI representing that resource and return the resource.
## Guidance
- APIs provide a get method for resources. The purpose of the get
method is to return data from a single resource.
- Some resources take longer to be retrieved than is reasonable for a regular
API request. In this situation, the API should use a
[long-running operation](/long-running-operations).
### Operation
### Requests
### Responses
The response usually include the fully-populated resource unless
there is a reason to return a partial response (see [AEP-157](/157)).
### Errors
If the user does not have sufficient permission to know that the resource
exists, the service reply with an HTTP 404 error, regardless of
whether or not the resource exists. Permission be checked prior to
checking if the resource exists.
If the user has sufficient permission to know that the resource exists, but is
unable to access it, the service reply with an HTTP 403 error.
If the user does have proper permission, but the requested resource does not
exist, the service reply with an HTTP 404 error.
## Interface Definitions
---
# AEP-132 List
In REST APIs, it is customary to make a `GET` request to a resource
collection's URI (for example, `/publishers/{publisher_id}/books`) in order to
retrieve a list of the resources within that collection.
Resource-oriented design [AEP-121](/121) honors this pattern through the `List` method.
These RPCs accept a parent collection (if one exists), and return a list of
responses matching that input.
## Guidance
APIs provide a `List` method for resource collections.The purpose of
the `List` method is to return data from a finite collection (generally
singular unless the operation supports [reading across collections][]).
When the `GET` method is used on a URI ending in a resource collection, the
result be a list of resources.
### Operation
### Requests
### Responses
`List` operations return a page of results, with each individual
result being a resource.
- The array of resources be named `results` and contain resources with
no additional wrapping.
- The `string nextPageToken` field be included in the list response
schema. It be set if there are subsequent pages, and be
set if the response represents the final page. For more information, see
[AEP-158](/158).
- The response struct include a `int32 total_size` (or
`int64 total_size`) field with the number of items in the collection.
- The value be an estimate (the field clearly document
this if so).
- If filtering is used, the `total_size` field reflect the size of
the collection _after_ the filter is applied.
- The response message include a field corresponding to the resources
being returned, named for the English plural term for the resource, and
include any other repeated fields.
- Fields providing metadata about the list request (such as
`string next_page_token` or `int32 total_size`) be included on the
response (not as part of the resource itself).
### Errors
If the user does not have sufficient permission to know that the collection
exists, the service reply with an HTTP 404 error, regardless of
whether or not the collection exists. Permission be checked prior to
checking whether the collection exists.
If the user does have proper permission, but the requested collection does not
exist (generally because the parent does not exist), the service reply
with an HTTP 404 error.
### Advanced operations
`List` methods allow an extended set of functionality to allow a user
to specify the resources that are returned in a response.
The following table summarizes the applicable AEPs, ordered by the precedence
of the operation on the results.
| Operation | AEP |
| ------------------------------ | ---------------------------------- |
| filter | [AEP-160](/0160) |
| ordering (`order_by`) | [AEP-132](#ordering) |
| pagination (`next_page_token`) | [AEP-158](/0158) |
| skip | [AEP-158](/0158/#skipping-results) |
For example, if both the `filter` and `skip` fields are specified, then the
filter would be applied first, then the resulting set would be the results that
skip N entries of the filtered result.
### Ordering
`List` methods allow clients to specify sorting order; if they do, the
request message contain a `string order_by` field.
- Values be a comma separated list of fields. For example:
`"foo,bar"`.
- The default sorting order is ascending. To specify descending order for a
field, users append a `-` prefix; for example: `"foo,-bar"`, `"-foo,bar"`.
- Redundant space characters in the syntax are insignificant. `"foo, -bar"`,
`" foo , -bar "`, and `"foo,-bar"` are all equivalent.
- Subfields are specified with a `.` character, such as `foo.bar` or
`address.street`.
### Soft-deleted resources
Some APIs need to "[soft-delete][]" resources, marking them as deleted or
pending deletion (and optionally purging them later).
APIs that do this include deleted resources by default in list
requests. APIs with soft deletion of a resource include a
`bool show_deleted` field in the list request that, if set, will cause
soft-deleted resources to be included.
## Interface Definitions
[reading across collections]: /159
[soft-delete]: /164
[resource type]: /004
---
# AEP-133 Create
In REST APIs, it is customary to make a `POST` request to a collection's URI
(for example, `/v1/publishers/{publisher_id}/books`) in order to create a new
resource within that collection.
Resource-oriented design [AEP-121](/121) honors this pattern through the `Create`
method. These RPCs accept the parent collection if one exists, and the resource
to create (and potentially some other parameters), and return the created
resource.
## Guidance
APIs provide a create method for resources unless it is not valuable
for users to do so. The purpose of the create method is to create a new
resource in an already-existing collection.
### Operation
Create methods are specified using the following pattern:
- The HTTP verb be `POST`.
- Some resources take longer to be created than is reasonable for a regular API
request. In this situation, the API use a
[long-running operation](/long-running-operations).
### Requests
Create methods implement a common request message pattern:
- An `id` field be supported.
- The resource field be included and map to the POST body.
- The request message contain any other required fields and
contain other optional fields except those described in this
or another AEP.
### Responses
### Errors
See [errors][], in particular [when to use PERMISSION_DENIED and NOT_FOUND
errors][permission-denied].
### User-specified IDs
An API allow a user to specify the ID component of a resource: not
doing so introduces a non-idempotent request in the API, as sending the same
payload results in creating a new resource each time.
Exceptional cases should have the following behavior:
- The resource allows identical records without a need to disambiguate between
the two (e.g. rows in a table with no primary key).
- The resource will not be exposed in [Declarative clients][].
An API allow the `id` field to be optional, and give the resource a
system-generated ID if one is not specified.
For example:
```
// Using user-specified IDs.
publishers/lacroix/books/les-miserables
// Using system-generated IDs.
publishers/012345678-abcd-cdef/books/12341234-5678-abcd
```
- The `path` field on the resource be ignored.
- The documentation explain what the acceptable format is, and the
format follow the guidance for resource path formatting in
[AEP-122](/122).
- If a user tries to create a resource with an ID that would result in a
duplicate resource path, the service error with `ALREADY_EXISTS`.
- However, if the user making the call does not have permission to see the
duplicate resource, the service error with `PERMISSION_DENIED`
instead.
## Interface Definitions
## Further reading
- For ensuring idempotency in `Create` methods, see [AEP-155](/155).
- For naming resources involving Unicode, see [AEP-210](/210).
## Rationale
### Requiring user-specified ids
[Declarative clients][] use the resource ID as a way to identify a resource for
applying updates and for conflict resolution. The lack of a user-specified ID
means a client is unable to find the resource unless they store the identifier
locally, and can result in re-creating the resource. This in turn has a
downstream effect on all resources that reference it, forcing them to update to
the ID of the newly-created resource.
Having a user-specified ID also means the client can precalculate the resource
path and use it in references from other resources.
[data plane]: /111.md#data-plane
[errors]: /193
[field_behavior]: /203
[Declarative clients]: /003.md#declarative-clients
[permission-denied]: /193.md#permission-denied
[strong consistency]: /121.md#strong-consistency
---
# AEP-134 Update
In REST APIs, it is customary to make a `PATCH` or `PUT` request to a
resource's URI (for example, `/v1/publishers/{publisher_id}/books/{book_id}`)
in order to update that resource.
Resource-oriented design [AEP-121](/121) honors this pattern through the `Update`
method (which mirrors the REST `PATCH` behavior). These methods accept the URI
representing that resource and return the resource.
Also see the [apply](/apply) method, with guidance on how to implement `PUT`
requests.
## Guidance
APIs provide an update method for resources unless it is not
valuable for users to do so. The purpose of the update method is to make
changes to the resources without causing side effects.
### Operation
- The method support partial resource update, and the HTTP verb
be `PATCH`.
- The operation have [strong consistency][].
- Some resources take longer to be created than is reasonable for a regular API
request. In this situation, the API should use a
[long-running operation](/long-running-operations).
- The response schema be the resource itself.
- The response include the fully-populated resource, and
include any fields that were sent and included in the update mask unless
they are input only (see [AEP-203](/203)).
### Requests
Update methods implement a common request pattern:
- The request contain a field for the resource.
- The name of this field be the singular form of the resource's
name.
- The request contain any required fields other than those
described in this section, and contain other optional fields
except those described in this or another AEP.
### Responses
### Errors
See [errors][], in particular [when to use PERMISSION_DENIED and NOT_FOUND
errors][permission-denied].
In addition, if the user does have proper permission, but the requested
resource does not exist, the service error with `NOT_FOUND` (HTTP 404)
unless `allow_missing` is set to `true`.
### Side effects
In general, update methods are intended to update the data within the resource.
Update methods trigger other side effects. Instead, side effects
be triggered by custom methods.
In particular, this entails that [state fields][] be directly
writable in update methods.
### PATCH and PUT
We standardize on `PATCH` because many organizations update stable APIs in
place with backwards-compatible improvements. It is often necessary to add a
new field to an existing resource, but this becomes a breaking change when
using `PUT`.
To illustrate this, consider a `PUT` request to a `Book` resource:
```
PUT /v1/publishers/123/books/456
{"title": "Mary Poppins", "author": "P.L. Travers"}
```
Next consider that the resource is later augmented with a new field (here we
add `rating`, and use a protobuf example without loss of generality):
```proto
message Book {
string title = 1;
string author = 2;
// Subsequently added to v1 in place...
int32 rating = 3;
}
```
If a rating were set on a book and the existing `PUT` request were executed, it
would wipe out the book's rating. In essence, a `PUT` request unintentionally
would wipe out data because the previous version did not know about it.
### FieldMasks in proto and json merge-patch in HTTP
AEP recommends a specific diverence in behavior between the proto and the HTTP
interfaces. Specifically:
- The inclusion of the `update_mask` in the proto variant, requiring the user
to explicitly specify fields to be updated.
- The usage of [IETF RFC 7396 - Json Merge Patch][IETF RFC 7396] for HTTP APIs.
This divergence in behavior is intentional, and exists for the following
reasons:
1. The update mask is a proto-specific concept, due to the lack of ability
across all proto versions to differentiate if the user has explicitly
populated a field or not. JSON has the ability to express whether a field is
present (by omitting it from the JSON payload). Ultimately, this allows the
field mask + proto pair and json to be translatable.
2. RFC 7396 is a popular and well-understood standard for HTTP. Introducing a
new standard for HTTP would have made the AEP HTTP variant less idiomatic.
3. For HTTP-proto bindings, there is a way to generate the proto field mask
from the fields present in the JSON request. This is what is recommended in
the
[API Design Patterns book, section 8.2 ](https://www.oreilly.com/library/view/api-design-patterns/9781617295850/),
describing the Google AIPs from which [AEP-134](/134) is forked. Implementations of
gateway-grpc proto bindings such as
[gateway-grpc](https://grpc-ecosystem.github.io/grpc-gateway/docs/mapping/patch_feature/)
support this translation.
Therefore, given the ability of these two different patch mechanisms to
interoperate while maintaining idiomatic practices, this divergence was
concluded to be the least worst option.
### Etags and preconditions
See [etags](/etags) for more information about adding headers and metadata such
as `ETag` and `If-Match` for supporting resource freshness validation and other
preconditions.
## Interface Definitions
[create]: /133
[errors]: /193
[permission-denied]: /193.md#permission-denied
[state fields]: /216
[strong consistency]: /121.md#strong-consistency
[required]: /203.md#required
[optional]: /203.md#optional
[IETF RFC 7396]: https://datatracker.ietf.org/doc/html/rfc7396
---
# AEP-135 Delete
In REST APIs, it is customary to make a `DELETE` request to a resource's URI
(for example, `/v1/publishers/{publisher_id}/books/{book_id}`) in order to
delete that resource.
Resource-oriented design (AEP-121) honors this pattern through the `Delete`
method. This method accepts the URI representing that resource and usually
returns an empty response.
## Guidance
APIs generally provide a delete method for resources unless it is
not valuable for users to do so.
The Delete method succeed if and only if a resource was present and
was successfully deleted. If the resource did not exist, the method
send a `404 Not found` (`NOT_FOUND`) error.
The method have [strong consistency][]: the completion of a delete
method mean that the existence of the resource has reached a
steady-state and reading resource state returns a consistent `404 Not found`
(`NOT_FOUND`) response.
### Operation
Delete methods are specified using the following pattern:
### Requests
Delete methods implement a common request pattern:
- The HTTP verb be `DELETE`.
- If a delete request contains a body, the body be ignored, and **must
not** cause an error (this is required by [RFC 9110][])
- The request require any fields in the query string. The request
include optional fields in the query string unless described
in another AEP.
### Responses
### Errors
If the user does not have permission to access the resource, regardless of
whether or not it exists, the service error with `403 Forbidden`
(`PERMISSION_DENIED`). Permission be checked prior to checking if the
resource exists.
If the user does have proper permission, but the requested resource does not
exist, the service error with `404 Not found` (`NOT_FOUND`).
### Soft delete
### Cascading delete
Sometimes, it may be necessary for users to be able to delete a resource as
well as all applicable child resources. However, since deletion is usually
permanent, it is also important that users not do so accidentally, as
reconstructing wiped-out child resources may be quite difficult.
If an API allows deletion of a resource that may have child resources, the API
provide a `bool force` field on the request, which the user sets to
explicitly opt in to a cascading delete.
## Interface Definitions
## Further reading
- For soft delete and undelete, see [AEP-164](/164).
- For bulk deleting large numbers of resources based on a filter, see [AEP-165](/165).
## Changelog
- **2024-02-11**: Imported from https://google.aip.dev/135
[strong consistency]: /121.md#strong-consistency
[etag]: /134.md#etags
[RFC 9110]: https://www.rfc-editor.org/rfc/rfc9110.html#name-delete
---
# AEP-136 Custom methods
Resource oriented design [AEP-121](/121) uses custom methods to provide a means to
express actions that are difficult to model using only the standard methods.
Custom methods are important because they provide a means for an API's
vocabulary to adhere to user intent.
## Guidance
Custom methods only be used for functionality that can not be
expressed via standard methods; prefer standard methods if possible, due to
their consistent semantics.
While custom methods vary widely in how they are designed, many principles
apply consistently:
- The HTTP method for custom methods be selected based on the
semantics described by [RFC 7231 section 4.3] and [RFC 5789] for `PATCH`. In
practice, the vast majority of custom methods use `POST` or `GET`.
- Custom methods that serve as an alternative to get or list methods (such as
`Search`) use `GET`, and require no request body. These methods
be idempotent and have no state changes or side effects (they
should be safe as defined in [RFC 7231 section 4.2.1][]).
- Custom methods use `PATCH` or `DELETE`.
- The HTTP URI use a `:` character followed by the custom verb
(`:archive` in the above example), and the verb in the URI match the
verb in the name of the RPC.
- If word separation is required, `kebab-case` be used.
- The name of the RPC be a verb followed by a noun.
- The name of the RPC contain prepositions ("for", "with",
etc.).
### Resource-based custom methods
Custom methods operate on a resource if the API can be modeled as
such:
### Collection-based custom methods
While most custom methods operate on a single resource, some custom methods
operate on a collection instead:
- If the collection has a parent, the field name in the request message
be `parent`.
- The collection key (`books` in the above example) be literal.
### Stateless methods
Some custom methods are not attached to resources at all. These methods are
generally _stateless_: they accept a request and return a response, and have no
permanent effect on data within the API.
- If the method runs in a particular scope (such as a project, as in the above
example), the field name in the request message be the name of the
scope resource. If word separators are necessary, `snake_case` be
used.
- The URI place both the verb and noun after the `:` separator
(avoid a "faux collection key" in the URI in this case, as there is no
collection). For example, `:translateText` is preferable to `text:translate`.
- Stateless methods use `POST` if they involve billing.
### Usage in declarative clients
APIs employ custom methods for functionality that is intended to
be used in a [declarative client](/3#declarative-clients). Declarative clients
use create,read,update,and delete operations to apply desired state, and
integration of custom methods is manual and results in client-side complexity
around state management to determine when the custom method should be invoked.
AEP's supported declarative clients cannot support custom methods.
[rfc 5789]: https://datatracker.ietf.org/doc/html/rfc5789
[rfc 7231 section 4.2.1]: https://datatracker.ietf.org/doc/html/rfc7231#section-4.2.1
[rfc 7231 section 4.3]: https://datatracker.ietf.org/doc/html/rfc7231#section-4.3
---
# AEP-137 Apply
In REST APIs, it is customary to make a `PUT` request to a resource's URI (for
example, `/v1/publishers/{publisher_id}/books/{book_id}`) in order to create or
replace a resource.
[Resource-oriented design](/resources) honors this pattern through the `Apply`
method. These operations accept the resource and its path, which it uses to
create or replace the resource. The operation returns the final resource.
Also see the [update](/update) method, with guidance on how to implement
`PATCH` requests.
## Guidance
APIs provide an apply method for a resource unless it is not
valuable for users to do so. Clients should also consider using
[update](/update) methods instead to ensure forwards compatible requests (see
[PATCH and PUT](#patch-and-put)).
### Behavior
- If a field in the request is not present, the service not modify
this field.
### Operation
Apply methods are specified using the following pattern:
- The HTTP verb be `PUT`.
- Some resources take longer to be applied than is reasonable for a regular API
request. In this situation, the API use a
[long-running operation](/long-running-operations).
- The operation have
[strong consistency](/resources#strong-consistency).
- The HTTP URI path of the `PUT` method be the resource path.
### Requests
Apply methods implement a common request message pattern:
- The resource be included and map to the HTTP request body.
- The request schema contain any other required fields and
contain other optional fields except those described in this
or another AEP.
### Responses
- The response be the resource itself. There is no separate response
schema.
- The response include the fully-populated resource, and
include any fields that were provided unless they are input only (see
[field behavior](/field-behavior-documentation)).
### Errors
If an resource does not have a field set which is labeled with required (either
the protobuf [field behavior](/0203#required) or present in the JsonSchema
`required` field), the request fail with
``INVALID_ARGUMENT / 400 Bad Request`.
For additional guidance, see [errors](/errors), in particular
[when to use PERMISSION_DENIED and NOT_FOUND errors](/errors#permission-denied).
### PATCH and PUT
`PUT` requests with fields missing in the resource may result in overwriting
values in the resource with existing values. Specifically in protobuf, the
fields that do not have the `optional` annotation will use their default zero
value, overwriting any previous value. This gap does not exist in an
[update](/0134), where the `field_mask` field helps clarify which fields to
overwrite.
As such AEP-compliant clients use the `PATCH` HTTP verb.
See [AEP-134's patch and put](/134#patch-and-put) for more information.
## Interface Definitions
## Further reading
- For ensuring idempotency in `Apply` methods, see [idempotency](/idempotency).
- For naming resources involving Unicode, see [unicode](/unicode).
[strong consistency]: /121.md#strong-consistency
---
# AEP-140 Field names
Naming fields in a way that is intuitive to users can often be one of the most
challenging aspects of designing an API. This is true for many reasons; often a
field name that seems entirely intuitive to the author can baffle a reader.
Additionally, users rarely use only one API; they use many APIs together. As a
result, a single company using the same name to mean different things (or
different names to mean the same thing) can often cause unnecessary confusion,
because users can no longer take what they've already learned from one API and
apply that to another.
In short, APIs are easiest to understand when field names are simple,
intuitive, and consistent with one another.
## Guidance
Field names be in correct American English.
Field names clearly and precisely communicate the concept being
presented and avoid overly general names that are ambiguous. That said, field
names avoid including unnecessary words. In particular, avoid
including adjectives that always apply and add little cognitive value. For
example, a `proxy_settings` field might be as helpful as
`shared_proxy_settings` if there is no unshared variant.
- Each word in a field name begin with a number, because it
creates ambiguity when converting between snake case and camel case.
- Fields contain leading, trailing, or adjacent underscores.
### Case
JSON and protobuf fields use `lower_snake_case` names. These names
be mapped to a language-specific idiomatic naming convention in
generated code.
- Over-the-wire references to fields in other contexts (such as query
parameters, path segments, and embedded CEL expressions) not be
transformed in any way, with the following exception:
- If a field name is also be used as a header name, it be
transformed by substituting hyphens (`-`) for underscores (`_`), emitted in
lowercase, and parsed case-insensitively.
#### Support for lowerCamelCase in clients
[ProtoJSON][proto-json] and [gRPC-Gateway][grpc-gateway] by default both
transform `lower_snake_case` protobuf field names into `lowerCamelCase` JSON
field names by default and an earlier draft of this AEP _required_ that JSON
field names be in `lowerCamelCase` (whether or not they are backed by
equivalent protobufs).
While APIs compliant with this AEP are required to use `lower_snake_case` JSON
field names, because of this history, tools for producing or consuming
AEP-compliant APIs support `lowerCamelCase` JSON fields and first-party
tools (those included in the AEP project).
[proto-json]: https://protobuf.dev/programming-guides/json
[grpc-gateway]: https://grpc-ecosystem.github.io/grpc-gateway/
### Uniformity
APIs endeavor to use the same name for the same concept and
different names for different concepts wherever possible. This includes names
across multiple APIs, in particular if those APIs are likely to be used
together.
### Arrays / Repeated fields
Arrays (in OAS) and repeated fields (in protobuf) use the proper
plural form, such as `books` or `authors`. On the other hand, singular fields
use the singular form such as `book` or `author`.
### Prepositions
Field names include prepositions (such as "with", "for", "at",
"by", etc.). For example:
- `error_reason` (**not** `reason_for_error`)
- `author` (**not** `written_by`)
It is easier for field names to match more often when following this
convention. Additionally, prepositions in field names may also indicate a
design concern, such as an overly-restrictive field or a sub-optimal data type.
This is particularly true regarding "with": a field named `book_with_publisher`
likely indicates that the book resource may be improperly structured and worth
redesigning.
### Adjectives
For uniformity, field names that contain both a noun and an adjective
place the adjective _before_ the noun. For example:
- `collected_items` (**not** `items_collected`)
- `imported_objects` (**not** `objects_imported`)
### Verbs
Field names be named to reflect an intent or action. They **must
not** be verbs.
Rather, because the field defines the _desired value_ for mutations, e.g.
Create and Update, and the _current value_ for reads, e.g. Get and List, the
name be a noun. It defines what is so, not what to do.
- `collected_items` (**not** `collect_items`)
- `disabled` (**not** `disable`)
In contrast, method names, whether standard or custom, change facets of
resources and are named as verbs.
### Booleans
Boolean fields omit the prefix "is". For example:
- `disabled` (**not** `is_disabled`)
- `required` (**not** `is_required`)
### String vs. bytes
### URIs
Field names representing URLs or URIs always use `uri` rather than
`url`. This is because while all URLs are URIs, not all URIs are URLs. Field
names use a prefix in front of `uri` as appropriate.
### Reserved words
Field names avoid using names that are likely to conflict with
keywords in common programming languages, such as `new`, `class`, `function`,
`import`, etc. Reserved keywords can cause hardship for developers using the
API in that language.
### Conflicts
Schemas include a field with the same name as the enclosing
schema (ignoring case transformations). This causes conflicts when generating
code in some languages.
### Display names
Many resources have a human-readable name, often used for display in UI. This
field be called `display_name`, and have a uniqueness
requirement.
If an entity has an official, formal name (such as a company name or the title
of a book), an API use `title` as the field name instead. The `title`
field have a uniqueness requirement.
## Further reading
- For naming resource fields, see [paths](./paths).
- For naming fields representing quantities, see [quantities](./quantities).
- For naming fields representing time, see
[time-and-duration](./time-and-duration).
---
# AEP-141 Quantities
Many services need to represent a discrete quantity of items (number of bytes,
number of miles, number of nodes, etc.).
## Guidance
Quantities with a clear unit of measurement (such as bytes, miles, and so on)
include the unit of measurement as the suffix. When appropriate and
unambiguous, units use generally accepted abbreviations (for
example, `distance_km` rather than `distance_kilometers`).
```typescript
// A representation of a non-stop air route.
interface Route {
// The airport where the route begins.
origin: string;
// The destination airport.
destination: string;
// The distance between the origin and destination airports.
// This value is also used to determine the credited frequent flyer miles.
distance_miles: number;
}
```
If the quantity is a number of items (for example, the number of nodes in a
cluster), then the field use the suffix `_count` (**not** the prefix
`num_`):
```typescript
// A cluster of individual nodes.
interface Cluster {
// The number of nodes in the cluster.
node_count: number;
}
```
### Specialized messages
It is sometimes useful to create a message that represents a particular
quantity. This is particularly valuable in two situations.
- Grouping two or more individual quantities together.
- Representing a common concept where the unit of measurement may itself vary.
Consider the example of money, which could need to do both of these:
```typescript
// A representation of an amount of money, in an arbitrary currency.
interface Money {
// The 3-letter currency code defined in ISO 4217.
currency_code: string;
// The whole units of the amount.
// For example if `currency_code` is "USD", then 1 unit is one US dollar.
units: bigint;
// Number of nano (10^-9) units of the amount.
// The value must be between -999,999,999 and +999,999,999 inclusive.
// If `units` is positive, `nanos` must be positive or zero.
// If `units` is zero, `nanos` can be positive, zero, or negative.
// If `units` is negative, `nanos` must be negative or zero.
// For example $-1.75 is represented as `units`=-1 and `nanos`=-750,000,000.
nanos: number;
}
```
APIs create structs to represent quantities when appropriate. When
using these structs as fields, APIs use the name of the struct as
the suffix for the field name if it makes intuitive sense to do so.
## Changelog
- **2019-09-13**: Added the prohibition on unsigned types.
---
# AEP-142 Time and duration
Many services need to represent the concepts surrounding time. Representing
time can be challenging due to the intricacies of calendars and time zones, as
well as the fact that common exchange formats (such as JSON) lack a native
concept of time.
## Guidance
Fields representing time use a `string` field with values conforming
to [RFC 3339][], such as `2012-04-21T15:00:00Z`.
Date/time values in HTTP headers are an exception to this rule -- these
use the HTTP-date format defined in [RFC 9110, Section 5.6.7][].
Services use IDL-specific timestamps or format indicators where
applicable (such as [`google.protobuf.Timestamp`][timestamp] in protocol
buffers, or `format: date-time` in OpenAPI) provided that the service converts
values to RFC 3339 timestamp strings in JSON.
### Timestamps
Fields that represent an absolute point in time (independent of any time zone
or calendar) use a `string` field with RFC 3339 values, and
send values in UTC, with the `Z` time zone explicitly included.
These fields have names ending in `_time`, such as `create_time` or
`update_time`. For array fields, the names end in `_times` instead.
Many timestamp fields refer to an activity (for example, `create_time` refers
to when the applicable resource was created). For these, the field
be named with the `{root_word}_time` form. For example, if a book is being
published, the field storing the time when this happens would use the root form
of the verb "to publish" ("publish"), resulting in a field called
`publish_time`. Fields be named using the past tense (such as
`published_time`, `created_time` or `last_updated_time`).
When accepting timestamps as input, services canonicalize the
timestamp into UTC, and return values in UTC.
### Durations
Fields that represent a span between two points in time (independent of any
time zone or calendar) use an `int` field if there is a single
canonical unit (for example, `ttl_seconds` for an expiring resource, or
`offset_seconds` for position in a video).
If there is no canonical unit, the service use a `string` field with
[ISO 8601][] duration values, such as `P3Y6M4DT12H30M5S`. The field name
end with `_duration`. Designators for zero values be omitted
in accordance with ISO 8601 (for example: `PT12H` is sufficient to represent
"12 hours"), but at least one designator be present (therefore, a zero
duration is `PT0S` or `P0D`).
Services use IDL-specific durations where applicable (such as
[`google.protobuf.Duration`][duration] in protocol buffers) provided that the
service converts values to ISO 8601 duration strings in JSON.
### Fractional seconds
Services support fractional seconds for both timestamps and durations,
but support precision more granular than the nanosecond.
Services also limit the supported precision, and _truncate_
values received from the user to the supported precision.
### Civil dates and times
Fields that represent a calendar date or wall-clock time use partial
ISO 8601 strings.
Fields representing civil dates have names ending in `_date`, while
fields representing civil times or datetimes have names ending in
`_time`.
### Recurring time
A service that needs to document a recurring event use cronspec if
cronspec is able to support the service's use case.
### Compatibility
Occasionally, APIs are unable to use RFC 3339 strings for legacy or
compatibility reasons. For example, an API may conform to a separate
specification that mandates that timestamps be Unix timestamp integers.
In these situations, fields use other types. If possible, the following
naming conventions apply:
- Unix timestamps use a `unix_time` suffix.
- Multipliers of Unix time (such as milliseconds) be used; if
they are unavoidable, the field name use both `unix_time` and
the unit, such as `unix_time_millis`.
- For other integers, include the meaning (examples: `time`, `duration`,
`delay`, `latency`) **and** the unit of measurement (valid values: `seconds`,
`millis`, `micros`, `nanos`) as a final suffix. For example,
`send_time_millis`.
- For strings, include the meaning (examples: `time`, `duration`, `delay`,
`latency`) but no unit suffix.
In all cases, clearly document the expected format, and the rationale for its
use.
[duration]: https://github.com/protocolbuffers/protobuf/blob/master/src/google/protobuf/duration.proto
[iso 8601]: https://www.iso.org/iso-8601-date-and-time-format.html
[rfc 3339]: https://datatracker.ietf.org/doc/html/rfc3339
[timestamp]: https://github.com/protocolbuffers/protobuf/blob/master/src/google/protobuf/timestamp.proto
[rfc 9110, section 5.6.7]: https://datatracker.ietf.org/doc/html/rfc9110#section-5.6.7
---
# AEP-143 Standardized codes
Many common concepts, such as spoken languages, countries, currency, and so on,
have common codes (usually formalized by the [International Organization for
Standardization][iso]) that are used in data communication and processing.
These codes address the issue that there are often different ways to express
the same concept in written language (for example, "United States" and "USA",
or "Español" and "Spanish").
## Guidance
For concepts where a standardized code exists and is in common use, fields
representing these concepts use the standardized code for both input
and output.
- Fields representing standardized concepts use the appropriate data
type for the standard code (usually `string`).
- Fields representing standardized concepts use enums, even if
they only allow a small subset of possible values. Using enums in this
situation often leads to frustrating lookup tables when using multiple APIs
together.
- Fields representing standardized concepts indicate which standard
they follow, preferably with a link (either to the standard itself, the
Wikipedia description, or something similar).
- The field name end in `_code` or `_type` unless the concept has an
obviously clearer suffix.
- When accepting values provided by users, validation be
case-insensitive unless this would introduce ambiguity (for example, accept
both `en-gb` and `en-GB`). When providing values to users, APIs
use the canonical case (in the example above, `en-GB`).
### Content types
Fields representing a content or media type use [IANA media types][]
and be called `content_type`.
### Countries and regions
Fields representing individual countries or nations use the [Unicode
CLDR region codes][cldr] ([list][]), such as `US` or `CH`, and the field
be called `region_code`.
### Currency
Fields representing currency use [ISO-4217 currency codes][iso-4217],
such as `USD` or `CHF`, and the field be called `currency_code`.
### Language
Fields representing spoken languages use [IETF BCP-47 language
codes][bcp-47] ([list][]), such as `en-US` or `de-CH`, and the field
be called `language_code`.
### Time zones
Fields representing a time zone use the [IANA TZ][] codes, and the
field be called `time_zone`.
Fields also represent a UTC offset rather than a time zone (note that
these are subtly different). In this case, the field use the [ISO-8601
format][] to represent this, and the field be named `utc_offset`.
## Changelog
- **2024-02-09**: Adopted text from https://google.aip.dev/143.
[bcp-47]: https://en.wikipedia.org/wiki/IETF_language_tag
[cldr]: http://cldr.unicode.org/
[iana media types]: https://www.iana.org/assignments/media-types/media-types.xhtml
[iana tz]: http://www.iana.org/time-zones
[iso]: https://www.iso.org/
[iso-4217]: https://en.wikipedia.org/wiki/ISO_4217
[iso-8601 format]: https://en.wikipedia.org/wiki/ISO_8601#Time_offsets_from_UTC
[list]: https://www.iana.org/assignments/language-subtag-registry/language-subtag-registry
[money]: https://github.com/aep-dev/aep/blob/main/schemas/type/money.md
---
# AEP-144 Array fields
Within API, methods that operate on schemas containing arrays require
consideration on the strategies for updating elements within the list.
For larger arrays of resources, consider [pagination](/158).
## Guidance
Resources use array fields where appropriate.
- Array fields use a plural field name.
- If the English singular and plural words are identical ("moose", "info"),
the dictionary word be used rather than attempting to coin a new
plural form.
- Array fields have an enforced upper bound that will not cause a
single resource payload to become too large. A good rule of thumb is 100
elements.
- If an array data can not be bounded (in other words, if there is a chance
that the array will be too large to be reasonably returned in a single
request), the API use a sub-resource instead.
### Scalars and structs
Array fields use a scalar type (such as `string`) if they are
certain that additional data will not be needed in the future, as using a
struct type adds significant cognitive overhead and leads to more complicated
code.
However, if additional data is likely to be needed in the future, array fields
use a struct instead of a scalar proactively, to avoid parallel
array fields.
### Update strategies
A resource use one of two strategies to enable updating a array field:
direct update using the [standard `Update`](/134) method, or custom `Add`
and `Remove` methods.
A standard `Update` method has one key limitation: the user is only able to
update _the entire_ array. This means that the user is required to read the
resource, make modifications to the array field value as needed, and send it
back. This is fine for many situations, particularly when the array field is
expected to have a small size (fewer than 10 or so) and race conditions are not
an issue, or can be guarded against with [ETags](/154).
If atomic modifications are required, and if the array is functionally a set
(meaning that order does not matter, duplicate values are not meaningful, and
non-comparable values such as `null` or `NaN` are not used), the API
define custom methods using the verbs `Add` and `Remove`:
#### Request Structure
---
# AEP-145 Ranges
Services often need to represent ranges of discrete or continuous values. These
have wide differences in meaning, and come in many types: integers, floats, and
timestamps, just to name a few, and the expected meaning of a range can vary in
subtle ways depending on the type of range being discussed.
## Guidance
A resource or message representing a range ordinarily use two
separate fields of the same type, with prefixes `start_` and `end_`:
### Inclusive or exclusive ranges
Fields representing ranges use inclusive start values and exclusive
end values (half-closed intervals) in most situations; in interval notation:
`[start_xxx, end_xxx)`.
Exclusive end values are preferable for the following reasons:
- It conforms to user expectations, particularly for continuous values such as
timestamps, and avoids the need to express imprecise "limit values" (e.g.
`2012-04-20T23:59:59`).
- It is consistent with most common programming languages, including C++, Java,
Python, and Go.
- It is easier to reason about abutting ranges: `[0, x), [x, y), [y, z)`, where
values are chainable from one range to the next.
### Exceptions
In some cases, there is significant colloquial precedent for inclusive start
and end values (closed intervals), to the point that using an exclusive end
value would be confusing even for people accustomed to them.
For example, when discussing dates (not to be confused with timestamps), most
people use inclusive end: a conference with dates "April 21-23" is expected to
run for three days: April 21, April 22, and April 23. This is also true for
days of the week: a business that is open "Monday through Friday" is open, not
closed, on Fridays.
In this situation, the prefixes `first` and `last` be used instead:
Fields representing ranges with significant colloquial precedent for inclusive
start and end values use inclusive end values with `first_` and
`last_` prefixes for those ranges only. The service still use
exclusive end values for other ranges where this does not apply, and
clearly document each range as inclusive or exclusive.
---
# AEP-146 Generic fields
Most fields in any API, whether in a request, a resource, or a custom response,
have a specific type or schema. This schema is part of the contract that
developers write their code against.
However, occasionally it is appropriate to have a generic or polymorphic field
of some kind that can conform to multiple schemata, or even be entirely
free-form.
## Guidance
While generic fields are generally rare, a API introduce generic field
where necessary. There are several approaches to this depending on how generic
the field needs to be; in general, APIs attempt to introduce the
"least generic" approach that is able to satisfy the use case.
For example, an API use a completely generic field (such as
`google.protobuf.Struct` in protobuf APIs) when the value of the field must
correspond to one of a known number of schemas. Instead, the API use
a [`oneof`](./oneof) to represent the known schemas.
### Generic fields in protobuf APIs
#### Oneof
A `oneof` be used to introduce a type union: the user or API is able to
specify one of the fields inside the `oneof`. Additionally, a `oneof`
be used with the same type (usually strings) to represent a semantic difference
between the options.
Because the individual fields in the `oneof` have different keys, a developer
can programmatically determine which (if any) of the fields is populated.
A `oneof` preserves the largest degree of type safety and semantic meaning for
each option, and APIs generally prefer them over other generic or
polymorphic options when feasible. However, the `oneof` construct is ill-suited
when there is a large (or unlimited) number of potential options, or when there
is a large resource structure that would require a long series of "cascading
oneofs".
#### Maps
Maps be used in situations where many values _of the same type_ are
needed, but the keys are unknown or user-determined.
Maps are usually not appropriate for generic fields because the map values all
share a type, but occasionally they are useful. In particular, a map can
sometimes be suited to a situation where many objects of the same type are
needed, with different behavior based on the names of their keys (for example,
using keys as environment names).
#### Struct
The [`google.protobuf.Struct`][struct] object be used to represent
arbitrary nested JSON. Keys can be strings, and values can be floats, strings,
booleans, arrays, or additional nested structs, allowing for an arbitrarily
nested structure that can be represented as JSON (and is automatically
represented as JSON when using REST/JSON).
A `Struct` is most useful when the API does not know the schema in advance, or
when a API needs to store and retrieve arbitrary but structured user data.
Using a `Struct` is convenient for users in this case because they can easily
get JSON objects that can be natively manipulated in their environment of
choice.
If a API needs to reason about the _schema_ of a `Struct`, it use
[JSONSchema][] for this purpose. Because JSONSchema is itself JSON, a valid
JSONSchema document can itself be stored in a `Struct`.
#### Any
The [`google.protobuf.Any`][any] object can be used to send an arbitrary
serialized protocol buffer and a type definition.
However, this introduces complexity, because an `Any` becomes useless for any
task other than blind data propagation if the consumer does not have access to
the proto. Additionally, even if the consumer _does_ have the proto, the
consumer has to ensure the type is registered and then deserialize manually,
which is an often-unfamiliar process.
Because of this, `Any` be used unless other options are
infeasible.
### Generic fields in OAS APIs
[any]: https://github.com/protocolbuffers/protobuf/tree/master/src/google/protobuf/any.proto
[struct]: https://github.com/protocolbuffers/protobuf/tree/master/src/google/protobuf/struct.proto
[JSONSchema]: https://json-schema.org/
---
# AEP-148 Standard fields
Certain concepts are common throughout any corpus of APIs. In these situations,
it is useful to have a standard field name and behavior that is used
consistently to communicate that concept.
## Guidance
Standard fields be used to describe their corresponding concept, and
be used for any other purpose.
### Resource paths and IDs
#### path
Every resource have a `string path` field, used for the
[resource path](/resource-paths), which be the first field in the
resource.
#### parent
The `string parent` field refers to the resource name of the parent of a
collection, and be used in [`List`](/list) and [`Create`](/create)
requests except for top-level resources.
### Names
#### display_name
The `string display_name` field be a mutable, user-settable field
where the user can provide a human-readable name to be used in user interfaces.
Display names have uniqueness requirements, and be
limited to \<= 63 characters.
#### title
The `string title` field be the official name of an entity, such as
a company's name. This is a more formal variant of `string display_name`.
#### given_name
The `string given_name` field refer to a human or animal's given name.
Resources use `first_name` for this concept, because the given
name is not placed first in many cultures.
#### family_name
The `string family_name` field refer to a human or animal's family
name. Resources use `last_name` for this concept, because the
family name is not placed last in many cultures.
### Timestamps
See [common components](/common-components) for more information about
`Timestamp` and `Duration`.
#### create_time
The output only `Timestamp create_time` field represent the timestamp
when the resource was created. This be either the time creation was
initiated or the time it was completed.
#### update_time
The output only `Timestamp update_time` field represent the timestamp
when the resource was most recently updated. Any change to the resource made by
users refresh this value; changes to a resource made internally by the
service refresh this value.
#### delete_time
The output only `Timestamp delete_time` field represent the timestamp
that a resource was soft deleted. This correspond to either the time
when the user requested deletion, or when the service successfully soft deleted
the resource. If a resource is not soft deleted, the `delete_time` field
be empty.
Resources that support [soft delete](/soft-delete) provide this
field.
#### expire_time
The `Timestamp expire_time` field represent the time that a given
resource or resource attribute is no longer useful or valid (e.g. a rotating
security key). It be used for [expiration](/resource-expiration).
Services provide an `expire_time` value that is inexact, but the
resource expire before that time.
#### purge_time
The `Timestamp purge_time` field represent the time when a soft
deleted resource will be purged from the system (see [AEP-164](/164)). It be
used for similar forms of expiration as described in [AEP-214](/214). Resources that
support soft delete include this field.
Services provide a `purge_time` value that is inexact, but the resource
be purged from the system before that time.
### Well known string fields
#### IP address
A field that represents an IP address comply with the following:
- use type `string`
- use the name `ip_address` or end with the suffix `_ip_address` e.g.
`resolved_ip_address`
- specify the IP address version format via one of the supported formats
`IPV4`, `IPV6`, or if it can be either, `IPV4_OR_IPV6` (see
[fields](/fields)).
#### uid
The output only `string uid` field refers to a system-assigned unique
identifier for a resource. When provided, this field be a [UUID4][]
and specify this format via the `UUID4` format extension (see
[fields](/fields)).
## Further reading
- For standardized codes, see [AEP-143](/143).
- For the `etag` field, see [AEP-154](/154).
- For the `request_id` field, see [AEP-155](/155).
- For the `filter` field, see [AEP-160](/160).
- For fields related to resource revisions, see [AEP-162](/162).
- For the `validate_only` field, see [AEP-163](/163).
- For fields related to soft delete and undelete, see [AEP-164](/164).
## Rationale
### Well known string fields
Some fields represent very well defined concepts or artifacts that sometimes
also have strict governance of their semantics. For such fields, presenting an
equally standardized API surface is important. This enables development of
improved API consumer tools and documentation, as well as a more unified user
experience across the platform.
[declarative-friendly resources]: /128.md#resources
[kubernetes limits]:
https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/#syntax-and-character-set
[uuid4]:
https://en.wikipedia.org/wiki/Universally_unique_identifier#Version_4_(random)
---
# AEP-151 Long-running operations
Occasionally, a service may need to expose an operation that takes a
significant amount of time to complete. In these situations, it is often a poor
user experience to simply block while the task runs; rather, it is better to
return some kind of promise to the user, and allow the user to check back in
later.
The long-running request pattern is roughly analogous to a [Future][] in Python
or Java, or a [Node.js Promise][]. Essentially, the user is given a token that
can be used to track progress and retrieve the result.
## Guidance
Operations that might take a significant amount of time to complete:
- return an `Operation` resource that can be used to track the
status of the request and ultimately retrieve the result.
- match other AEP guidance that would apply to the method otherwise
(such as status code)
### Operation representation
The response to a long-running request be an [`Operation`][Operation].
### Querying an operation
The service provide an endpoint to query the status of the operation,
which accept the operation path and include other
parameters:
```http
GET /v1/operations/{operation} HTTP/2
Host: library.example.com
Accept: application/json
```
The endpoint return a `Operation` as described above.
### Standard methods
APIs return an `Operation` from the [`Create`](/133),
[`Update`](/134), or [`Delete`](/135) standard methods if appropriate. In
this case, the `response` field be the standard and expected response
type for that standard method.
When creating or deleting a resource with a long-running request, the resource
be included in [`List`](/132) and [`Get`](/131) calls;
however, the resource indicate that it is not usable, generally with
a [state enum](/216).
### Parallel requests
A resource accept multiple requests that will work on it in parallel,
but is not obligated to do so:
- Resources that accept multiple parallel requests place them in a
queue rather than work on the requests simultaneously.
- A resource that does not permit multiple requests in parallel (denying any
new request until the one that is in progress finishes) return
`ALREADY_EXISTS / 409` if a user attempts a parallel request, and include an
error message explaining the situation.
### Expiration
APIs allow their operation resources to expire after sufficient time
has elapsed after the request completed.
### Errors
Errors that prevent a long-running request from _starting_ return an
[error response](/193), similar to any other method.
Errors that occur over the course of a request be placed in the
metadata message. The errors themselves still be represented with a
canonical error object.
## Interface Definitions
[google.rpc.Status]: https://github.com/googleapis/api-common-protos/blob/main/google/rpc/status.proto
[lro]: https://github.com/aep-dev/aep-components/blob/main/proto/aep-api/aep/api/operation.proto
[node.js promise]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Using_promises
[future]: https://docs.python.org/3/library/concurrent.futures.html#concurrent.futures.Future
[Operation]: https://aep.dev/json-schema/type/operation.json
[aep.api.Operation]: https://buf.build/aep/api/docs/main:aep.api#aep.api.Operation
[JSON Schema Operation]: http://aep.dev/json-schema/type/operation.json
---
# AEP-154 Preconditions
Preconditions refers to the concept of a set of conditions that must be met
before a particular operation (generally update of a resource) is performed.
For example, APIs often need to validate that a client and server agree on the
current state of a resource before taking some kind of action on that resource.
e.g. two processes updating the same resource in parallel could create a race
condition, where the latter process "stomps over" the effort of the former one.
`ETags`, `If-Match`, and `If-None-Match` provide a way to deal with this, by
allowing the server to send a checksum based on the current content of a
resource; when the client sends that checksum back, the server can ensure that
the checksums match before acting on the request.
## Guidance
When adding precondition checking to an API (`ETag`, `If-Match`, and
`If-None-Match` headers), the behavior match
[preconditions](https://www.rfc-editor.org/rfc/rfc9110.html#name-preconditions)
as documented in [RFC 9110][].
If a server receives a conditional header it does not support, the service
return an `INAVLID_ARGUMENT / 400` response. A server
support all preconditions, or none of the them.
### Etags
A resource provide an `ETag` header when retrieving a single resource
when it is important to ensure that the client has an up to date resource
before acting on certain requests:
```
200 OK
Content-type: application/json
ETag: "55cc0347-66fc-46c3-a26f-98a9a7d61d0e"
```
The ETag be provided by the server on output, and values
conform to [RFC 9110][]. Resources support the `If-Match` header (and
support the `If-None-Match` header) if and only if resources provide
the ETag.
ETags be based on an opaque checksum or hash of the resource that
guarantees it will change if the resource changes.
### If-Match / If-None-Match
Services that provide ETags support the `If-Match` and
`If-None-Match` headers.
An example of using `If-Match`:
```http
GET /v1/publishers/{publisher_id}/books/{book_id} HTTP/2
Host: library.googleapis.com
Accept: application/json
If-Match: "55cc0347-66fc-46c3-a26f-98a9a7d61d0e"
```
If the service receives a request to modify a resource that includes an
`If-Match` header, the service validate that the value matches the
current ETag. If the `If-Match` header value does not match the ETag, the
service reply with an HTTP 412 error.
If the user omits the `If-Match` header, the service permit the
request. However, services with [strong consistency](./0121#strong-consistency)
or parallelism requirements require users to send ETags all the time
and reject the request with an HTTP 400 error if it does not contain an ETag.
If any conditional headers are supported for any operation within a service,
the same conditional headers be supported for all mutation methods
(`POST`, `PATCH`, `PUT`, and `DELETE`) of any path that supports them, and
be supported uniformly for all operations across the service.
If any validator or conditional headers are supported for any operations in the
service, the use of unsupported conditional headers result in a
`INVALID_ARGUMENT / 400` error response. (In other words, once a service gives
the client reason to believe it understands conditional headers, it **must
not** ever ignore them.)
### Read requests
If a service receives a `GET` or `HEAD` request with an `If-Match` header, the
service proceed with the request if the ETag matches, or send a
`412 Precondition Failed` error if the ETag does not match.
If a service receives a `GET` or `HEAD` request with an `If-None-Match` header,
the service proceed with the request if the ETag does not match, or
return a `304 Not Modified` response if the ETag does match.
### Strong and weak ETags
ETags can be either "strongly validated" or "weakly validated":
- A strongly validated ETag means that two resources bearing the same ETag are
byte-for-byte identical.
- A weakly validated ETag means that two resources bearing the same ETag are
equivalent, but may differ in ways that the service does not consider to be
important.
Resources use either strong or weak ETags, as it sees fit, but
document the behavior. Additionally, weak ETags have a `W/`
prefix as mandated by
[RFC 9110 Etag Comparison](https://www.rfc-editor.org/rfc/rfc9110.html#name-comparison-2).
```
200 OK
Content-type: application/json
ETag: W/"55cc0347-66fc-46c3-a26f-98a9a7d61d0e"
```
**NOTE**: The strong match be used when comparing
[`If-match`](https://www.rfc-editor.org/rfc/rfc9110.html#name-if-match), while
the weak match be used when evaluating the
[`If-None-Match`](https://www.rfc-editor.org/rfc/rfc9110.html#name-if-none-match)
as stated in [RFC-9110][].
Strong ETags and weak ETags be guaranteed to change if any
properties on the resource change that are directly mutable by the client.
Additionally, strong ETags be guaranteed to change if the resource's
representation changes in a meaningful way (meaning the new representation is
not equivalent to the old one).
## Changelog
- **2025-01-18**: Updated to use RFC 9110 instead of RFC 7232, which is
deprecated.
- **2020-09-02**: Clarified that other errors may take precedence over
`FAILED_PRECONDITION` for ETag mismatches.
- **2020-09-02**: Add guidance for ETags on request messages.
- **2019-09-23**: Changed the title to "resource freshness validation".
[RFC 9110]: https://www.rfc-editor.org/rfc/rfc9110.html#section-8.8.3
---
# AEP-155 Idempotency
It is sometimes useful for an API to have a unique, customer-provided
identifier for particular requests. This can be useful for several purposes,
such as:
- De-duplicating requests from parallel processes
- Ensuring the safety of retries
- Auditing
The purpose of idempotency keys is to provide idempotency guarantees: allowing
the same request to be issued more than once without subsequent calls having
any effect. In the event of a network failure, the client can retry the
request, and the server can detect duplication and ensure that the request is
only processed once.
## Guidance
APIs add a `aep.api.IdempotencyKey idempotency_key` parameter to
request messages (including those of standard methods) in order to uniquely
identify particular requests. API servers execute requests with
the same `idempotency_key` more than once.
```proto
message CreateBookRequest {
// The parent resource where this book will be created.
// Format: publishers/{publisher_id}
string parent = 1 [
(aep.api.field_info) = { resource_reference: [ "library.example.com/book" ], field_behavior: [ FIELD_BEHAVIOR_REQUIRED ] }
];
// The book to create.
Book book = 2 [(google.api.field_behavior) = REQUIRED];
// This request is only idempotent if `idempotency_key` is provided.
//
// This key will be honored for at least one hour after the first time it is
// seen by the server.
//
// The key is restricted to 36 ASCII characters. A random UUID is recommended.
aep.api.IdempotencyKey idempotency_key = 3 [
(aep.api.field_info).minimum_lifetime = { seconds: 3600 }
];
}
```
- [`aep.api.IdempotencyKey`][IdempotencyKey] has a `key` and a `first_sent`
timestamp.
- `key` is simply a unique identifier.
- Providing an idempotency key guarantee idempotency.
- If a duplicate request is detected, the server return one of:
- A response equivalent to the response for the previously successful
request, because the client most likely did not receive the previous
response.
- An error indicating that the `first_sent` field of the idempotency key is
invalid or cannot be honored (expired, in the future, or differs from a
previous `first_sent` value with the same `key`).
- An error, if returning an equivalent response is not possible.
For example, if a resource was created, then deleted, and then a
duplicate request to create the resource is received, the server
return an error if returning the previously created resource is not
possible.
- APIs honor idempotency keys for at least an hour.
- When using protocol buffers, idempotency keys that are UUIDs be
annotated with a minimum lifetime using the extension
[`(aep.api.field_info).minimum_lifetime`][minimum_lifetime].
- The `idempotency_key` field be provided on the request message to
which it applies (and it be a field on resources themselves).
- The `first_sent` field can be used by API servers to determine if a key is
expired. API servers reject requests with expired keys, and
reject requests with keys that are in the future. When feasible,
API servers reject requests that use the same `key` but have a
different `first_sent` timestamp.
- The `key` field be able to be a UUID, and allow UUIDs to
be the only valid format. The format restrictions for idempotency keys
be documented.
- Idempotency keys be optional.
## Further reading
- For which codes to retry, see [AEP-194](https://aep.dev/194).
- For how to retry errors in client libraries, see
[AEP-4221](https://aep.dev/4221).
## Rationale
### Naming the field `idempotency_key`
The original content from which this AEP is derived defines a `request_id`
field; we define `idempotency_key` instead for two reasons:
1. There is an [active Internet-Draft][idempotency-key-draft] to standardize an
HTTP header named `Idempotency-Key`.
1. There may be edge cases in which separately identifying idempotent requests
is useful; `request_id` would be more appropriate for such use cases. For
example, an API producer might be testing the idempotency behavior of the
API server, and might want to issue multiple requests with the same
`idempotency_key` and trace the behavior of each request separately.
### Using UUIDs for request identification
When a value is required to be unique, leaving the format open-ended can lead
to API consumers incorrectly providing a duplicate identifier. As such,
standardizing on a universally unique identifier drastically reduces the chance
for collisions when done correctly.
## Changelog
- **2023-23-20**: Adopt AEP from from Google's AIP with the following changes:
- Rename field from `request_id` to `idempotency_key` (plus some minor
releated rewording).
- Add a common component [`aep.api.IdempotencyKey`][IdempotencyKey] and use
this rather than `string` for the `idempotency_key` field; add related
guidance about `IdempotencyKey.first_seen`.
- Remove guidance about annotating `idempotency_key` with
`(google.api.field_info).format`.
- Add guidance about annotating `idempotency_key` with
[`(aep.api.field_info).minimum_lifetime`].
- Update guidance about responses to be more explicit about success and error
cases, while allowing "equivalent" rather than identical responses for
subsequent requests.
- Temporarily removed the section about stale success responses, pending
further discussion.
- **2023-10-02**: Add UUID format extension guidance.
- **2019-08-01**: Changed the examples from "shelves" to "publishers", to
present a better example of resource ownership.
[idempotency-key-draft]: https://datatracker.ietf.org/doc/draft-ietf-httpapi-idempotency-key-header/
[IdempotencyKey]: https://buf.build/aep/api/file/main:aep/api/idempotency_key.proto#L21
[minimum_lifetime]: https://buf.build/aep/api/file/main:aep/api/field_info.proto#L35
---
# AEP-156 Singleton resources
APIs sometimes need to represent a resource where exactly one instance of the
resource always exists within any given parent. A common use case for this is
for a config object.
## Guidance
An API define _singleton resources_. A singleton resource
always exist by virtue of the existence of its parent, with one and exactly one
per parent.
For example:
- Singleton resources have a user-provided or system-generated ID;
their [resource name](/122) includes the name of their parent followed by
one static-segment.
- Example: `users/1234/config`
- Singleton resources are always singular.
- Example: `users/1234/thing`
- Singleton resources parent other resources.
- Singleton resources define the [`Create`](/133) or
[`Delete`](/135) standard methods. The singleton is implicitly created or
deleted when its parent is created or deleted.
- Singleton resources define the [`Get`](/131) and
[`Update`](/134) methods, and define custom methods as
appropriate.
- However, singleton resources define the [`Update`](/134)
method if all fields on the resource are [output only](/203).
- Singleton resources define the [`List`](/132) method, but
implement it according to [AEP-159](/159). See the example below.
- The trailing segment in the path pattern that typically represents the
collection be the `plural` form of the Singleton resource e.g.
`/v1/{parent=users/*}/configs`.
- If a parent resource ID is provided instead of the hyphen `-` as per
[AEP-159](/159), then the service return a collection of one Singleton
resource corresponding to the specified parent resource.
## Rationale
### Support for Standard List
While Singleton resources are not directly part of a collection themselves,
they can be viewed as part of their parent's collection. The one-to-one
relationship of parent-to-singleton means that for every one parent there is
one singleton instance, naturally enabling some collection-based methods when
combined with the pattern of [Reading Across Collections](/159). The
Singleton can present as a collection to the API consumer as it is indirectly
one based on its parent. Furthermore, presenting the Singleton resource as a
pseudo-collection in such methods enables future expansion to a real
collection, should a Singleton be found lacking.
---
# AEP-157 Partial responses
Sometimes, a resource can be either large or expensive to compute, and the API
needs to give the user control over which fields it sends back.
## Guidance
APIs support partial responses in one of two ways:
### `read_mask` parameter
Field masks can be used for granting the user fine-grained control over what
fields are returned. An API support the mask as a request field
named `read_mask`.
- The `read_mask` parameter be optional:
- An explicit value of `"*"` be supported, and return all
fields.
- If `read_mask` is omitted, it default to `"*"`, unless otherwise
documented.
- An API allow read masks with non-terminal repeated fields (unlike
update masks), but is not obligated to do so.
### View enumeration
Alternatively, an API support partial responses with view enums. View
enums are useful for situations where an API only wants to expose a small
number of permutations to the user:
- The enum be specified as a `view` field on the request message.
- The `UNSPECIFIED` value be valid (not an error), and the API
document what the unspecified value will do.
- For List methods, the effective default value be `BASIC`.
- For Get methods, the effective default value be either `BASIC`
or `FULL`.
- APIs add fields to a given view over time. APIs remove a
field from a given view (this is a breaking change).
---
# AEP-158 Pagination
APIs often need to provide collections of data, most commonly in the [List][]
standard method. However, collections can often be arbitrarily sized, and also
often grow over time, increasing lookup time as well as the size of the
responses being sent over the wire. Therefore, it is important that collections
be paginated.
## Guidance
Methods returning collections of data provide pagination _at the
outset_, as it is a [backwards-incompatible change](#backwards-compatibility)
to add pagination to an existing method.
- Request messages for collections define an integer (`int32` for
protobuf) `max_page_size` field, allowing users to specify the maximum number
of results to return.
- If the user does not specify `max_page_size` (or specifies `0`), the API
chooses an appropriate default, which the API document. The API
return an error.
- If the user specifies `max_page_size` greater than the maximum permitted by
the API, the API coerce down to the maximum permitted page size.
- If the user specifies a negative value for `max_page_size`, the API
send an `INVALID_ARGUMENT` error.
- The API return fewer results than the number requested (including
zero results), even if not at the end of the collection.
- Request schemas for collections define a `string page_token`
field, allowing users to advance to the next page in the collection.
- The `page_token` field be required.
- If the user changes the `max_page_size` in a request for subsequent pages,
the service honor the new page size.
- The user is expected to keep all other arguments to the method the same; if
any arguments are different, the API send an `INVALID_ARGUMENT`
error.
- The response be a streaming response.
- Response messages for collections define a `string next_page_token`
field, providing the user with a page token that may be used to retrieve the
next page.
- The field containing pagination results be a repeated field
containing a list of resources constituting a single page of results.
- If the end of the collection has been reached, the `next_page_token` field
be empty. This is the _only_ way to communicate
"end-of-collection" to users.
- If the end of the collection has not been reached (or if the API can not
determine in time), the API provide a `next_page_token`.
- Response messages for collections provide an integer (`int32` for
protobuf) `total_size` field, providing the user with the total number of
items in the list.
- This total be an estimate. If so, the API explicitly
document that.
### Skipping results
The request definition for a paginated operation define an integer
(`int32` for protobuf ) `skip` field to allow the user to skip results.
The `skip` value refer to the number of individual resources to skip,
not the number of pages.
For example:
- A request with no page token and a `skip` value of `30` returns a single page
of results starting with the 31st result.
- A request with a page token corresponding to the 51st result (because the
first 50 results were returned on the first page) and a `skip` value of `30`
returns a single page of results starting with the 81st result.
If a `skip` value is provided that causes the cursor to move past the end of
the collection of results, the response be `200 OK` with an empty
result set, and not provide a `next_page_token`.
### Page Token Opacity
Page tokens provided by APIs be opaque (but URL-safe) strings, and
be user-parseable. This is because if users are able to
deconstruct these, _they will do so_. This effectively makes the implementation
details of your API's pagination become part of the API surface, and it becomes
impossible to update those details without breaking users.
For page tokens which do not need to be stored in a database, and which do not
contain sensitive data, an API obfuscate the page token by defining an
internal protocol buffer message with any data needed, and send the serialized
proto, base-64 encoded.
Page tokens be limited to providing an indication of where to continue
the pagination process only. They provide any form of
authorization to the underlying resources, and authorization be
performed on the request as with any other regardless of the presence of a page
token.
### Page Token Expiration
Many APIs store page tokens in a database internally. In this situation, APIs
expire page tokens a reasonable time after they have been sent, in
order not to needlessly store large amounts of data that is unlikely to be
used. It is not necessary to document this behavior.
### Backwards compatibility
Adding pagination to an existing method is a backwards-incompatible change.
This may seem strange; adding fields to proto messages is generally backwards
compatible. However, this change is _behaviorally_ incompatible.
Consider a user whose collection has 75 resources, and who has already written
and deployed code. If the API later adds pagination fields, and sets the
default to 50, then that user's code breaks; it was getting all resources, and
now is only getting the first 50 (and does not know to advance pagination).
Even if the API set a higher default limit, such as 100, the user's collection
could grow, and _then_ the code would break.
For this reason, it is important to always add pagination to RPCs returning
collections _up front_; they are consistently important, and they can not be
added later without causing problems for existing users.
[list]: /list
## Changelog
- **2024-04-14**: Imported from https://google.aip.dev/158.
---
# AEP-159 Reading across collections
Sometimes, it is useful for a user to be able to retrieve resources across
multiple collections, or retrieve a single resource without needing to know
what collection it is in.
## Guidance
APIs support reading resources across multiple collections by allowing
users to specify a `-` (the hyphen or dash character) as a wildcard character
in a standard [`List`](./list) method:
```
GET /v1/publishers/-/books?filter=...
```
- The method explicitly document that this behavior is supported.
- The resources provided in the response use the canonical name of the
resource, with the actual parent collection identifiers (instead of `-`).
- Services support reading across collections on `List` requests
regardless of whether the identifiers of the child resources are guaranteed
to be unique. However, services support reading across
collections on `Get` requests if the child resources might have a collision.
- Cross-parent requests support `order_by`. If they do, the
field document that it is best effort. This is because cross-parent
requests introduce ambiguity around ordering, especially if there is
difficulty reaching a parent (see [unreachable resources]).
### Unique resource lookup
Sometimes, a resource within a sub-collection has an identifier that is unique
across parent collections. In this case, it may be useful to allow a
[`Get`](./get) method to retrieve that resource without knowing which parent
collection contains it. In such cases, APIs allow users to specify the
wildcard collection ID `-` (the hyphen or dash character) to represent any
parent collection:
```
GET https://library.exampleapis.com/v1/publishers/-/books/{book_id}
```
- The URI pattern still be specified with a variable and permit the
collection to be specified; a URI pattern hard-code the `-`
character. Having a separate route for the wildcard case can lead to
ambiguous or undefined routing behavior (unless the variable pattern excludes
the string `"-"`)
- The method explicitly document that this behavior is supported.
- The resource name in the response use the canonical name of the
resource, with actual parent collection identifiers (instead of `-`). For
example, the request above returns a resource with a name like
`publishers/123/books/456`, _not_ `publishers/-/books/456`.
- The resource ID be unique within parent collections.
### Reading across collections with different path patterns
Sometimes, a resource may have multiple possible path patterns. This typically
happens when a resource, or one of its ancestors, may have more than one
possible parent resource (including having no parent).
For example, `Book` might have the following path patterns:
- `publishers/{publisher_id}/books/{book_id}` for books with a publisher
- `books/{book_id}` for self-published books
In this case, APIs allow users to read across _all_ collections of
books by using the `--` global wildcard sequence:
`GET /v1/--/books`
Here, `--` is not a wildcard for a resource ID within a specific collection.
Rather, it is a wildcard for the entire path pattern. This allows users to
retrieve a book regardless of its resource ancestry.
This pattern may also be used to search all subcollections of a specific
collection. For example, a `Playlist` resource might have the following path
patterns:
- `games/{game}/users/{user}/playlists/{playlist}` for playlists created by a
user within a game
- `games/{game}/zones/{zone}/playlists/{playlist}` for playlists specific to a
game zone
A client could read across all collections of both zone and user playlists
_within_ game `123` with:
`GET /v1/games/123/--/playlists`
## Further reading
- For partial failures due to unreachable resources, see [unreachable
resources].
[unreachable resources]: /unreachable-resources
---
# AEP-160 Filtering
Often, when listing resources (using a [List](./list) method or something
reasonably similar), it is desirable to filter over the collection and only
return results that the user is interested in.
It is tempting to define a structure to handle the precise filtering needs for
each API. However, filtering requirements evolve frequently, and therefore it
is prudent to use a string field with a well-defined syntax. This allows
updates to be able to be made transparently, without waiting for UI or client
updates.
## Guidance
APIs use [CEL][] to provide filtering to users on `List` methods (or
similar methods to query a collection, such as `Search`). If they choose to do
so, they follow the common specification for filters discussed here.
The syntax is formally defined in the [CEL language definition][].
When employing filtering, a request message have exactly one
filtering field, `string filter`. The value of this field be a Boolean
CEL expression.
For example, given the resources
`[{name: "dog" legs: 4}, {name: "cat" legs: 4}, {name: "fish" legs: 0}]`, the
filter string `legs == 4` would result in only the "dog" and "cat" resources
being returned by the method.
### Functions
APIs use CEL's function call syntax in order to support API-specific
extensions. An API document any specific functions it supports,
including any limitations on their usage (for example, if a given function can
only be used on certain fields and/or with certain API methods).
### Limitations
An API support only a subset of the CEL language, and/or to support
filtering on a subset of the fields of a resource. APIs clearly
document any such limitations.
A service specify further structure or limitations for filter queries,
above what is defined here. For example, a service may support the logical
operators defined by CEL, but only permit a certain number of them (to avoid
"queries of death" or other performance concerns).
Further structure or limitations be clearly documented,
violate requirements set forth in this document, and a non-compliant filter
query error with `INVALID_ARGUMENT`.
[cel]: https://github.com/google/cel-spec
[CEL language definition]: https://github.com/google/cel-spec/blob/master/doc/langdef.md
---
# AEP-161 Field masks
Often, when updating resources (using an update method as defined in AIP-134 or
something reasonably similar), it is desirable to specify exactly which fields
are being updated, so that the service can ignore the rest, even if the user
sends new values.
## Guidance
These masks of field names are called "field masks". Fields representing a
field mask use the `google.protobuf.FieldMask` type. Field masks are
most common on Update requests (AIP-134).
Field masks always be relative to the resource:
```proto
message UpdateBookRequest {
// The book to update.
//
// The book's `name` field is used to identify the book to update.
// Format: publishers/{publisher}/books/{book}
Book book = 1 [(aep.api.field_info) = { field_behavior: [ FIELD_BEHAVIOR_REQUIRED ] }];
// The list of fields to update.
// Fields are specified relative to the book
// (e.g. `title`, `rating`; *not* `book.title` or `book.rating`).
google.protobuf.FieldMask update_mask = 2;
}
```
### Read-write consistency
Read and write behavior for field masks be self-consistent if a mask
is present:
- If a user updates a resource with a given mask, and then reads the same
resource with the same mask, the exact same data be returned.
- Exception: [Output only fields](#output-only-fields).
- Similarly, reading a resource with a given mask and then updating the
resource with the returned data and the same mask be a no-op.
### Specifying specific fields
Field masks permit the specification of specific fields in a defined
struct, using the `.` character for traversal.
Because field masks are always relative to the resource, direct fields on the
resource require no traversal (examples: `title`, `rating`). Traversal is used
when resources contain messages (example: `author.given_name`).
### Map fields
Field masks permit the specification of specific fields in a map, if
and only if the map's keys are either strings or integers, using the `.`
character for traversal.
Field masks support string keys that contain characters that are
problematic for the field mask syntax, using the backtick character.
```proto
message Book {
// The name of the book.
// Format: publishers/{publisher}/books/{book}
string name = 1;
// Reviews for the back cover. The key is the author of the review,
// and the value is the text of the review.
//
// Valid field masks: reviews, reviews.smith, reviews.`John Smith`
map reviews = 2;
}
message UpdateBookRequest {
// The book to update.
//
// The book's `name` field is used to identify the book to update.
// Format: publishers/{publisher}/books/{book}
Book book = 1 [(aep.api.field_info) = { field_behavior: [ FIELD_BEHAVIOR_REQUIRED ] }];
// The list of fields to update.
// Fields are specified relative to the book
// (e.g. `title`, `rating`; *not* `book.title` or `book.rating`).
google.protobuf.FieldMask update_mask = 2;
}
```
### Wildcards
Field masks permit the use of the `*` character on a repeated field or
map to indicate the specification of particular sub-fields in the collection:
```proto
message Book {
// The author or authors of the book.
// Valid field masks: authors, authors.*.given_name, authors.*.family_name
// Invalid field masks: authors.0, authors.0.given_name, authors[0], authors[0].given_name
repeated Author authors = 2;
}
message Author {
// The author's given name.
string given_name = 1;
// The author's family name.
string family_name = 2;
}
```
### Output only fields
If a user includes an output only field in an update mask indirectly (by using
a wildcard or specifying an overall message that includes an output-only
subfield), the service ignore any output only fields provided as
input, even if they are cleared or modified.
If a user directly specifies an output only field in an update mask, the
service ignore the output only fields provided as input, even if they
are cleared or modified, to permit the same field mask to be used for input and
output.
### Invalid field mask entries
When reading data, field masks ignore entries that point to a value
that can not exist (either a field that does not exist, or a map key that the
service considers invalid).
When writing data, field masks return an `INVALID_ARGUMENT` error if
an entry points to a value that can not exist; however, the service
permit deletions.
---
# AEP-162 Resource Revisions
Some APIs need to have resources with a revision history, where users can
reason about the state of the resource over time. There are several reasons for
this:
- Users may want to be able to roll back to a previous revision, or diff
against a previous revision.
- An API may create data which is derived in some way from a resource at a
given point in time. In these cases, it may be desirable to snapshot the
resource for reference later.
## Guidance
APIs expose a revision history for a resource. Examples of when it is
useful include:
- When it is valuable to expose older revisions of a resource via an API. This
can avoid the overhead of the customers having to write their own API to
store and enable retrieval of revisions.
- Other resources depend on or descend from different revisions of a resource.
- There is a need to represent the change of a resource over time.
APIs implementing resources with revisions model resource revisions as
a subresource of the resource.
- The resource revision contain a field with a message type of the
parent resource, with a field name of `resource`.
- The value of `resource` be the original resource at the point in
time the revision was created.
- The resource revision contain a `create_time` field (see
[AIP-142][]).
- The resource revision contain a repeated field `aliases`, which would
contain a list of resource IDs that the revision is also known by (e.g.
`latest`)
### Creating Revisions
Depending on the resource, different APIs may have different strategies for
creating a new revision. Specifically examples of strategies include:
- Creating a revision when there is a change to the resource
- Creating a revision when important system state changes
- Creating a revision when specifically requested
APIs use any of these strategies or any other strategy. APIs
document their revision creation strategy.
Resources that support revisions always have at least one revision.
### Resource names for revisions
When referring to the revisions of a resource, the subcollection name
be `revisions`. Resource revisions have paths with the format
`{resource_path}/revisions/{resource_revision}`. For example:
```
publishers/123/books/les-miserables/revisions/c7cfa2a8
```
The resource be named `{resource_singular}Revision` (for example,
`BookRevision`).
### Server-specified aliases
Services reserve specific IDs to be [aliases][alias] (e.g. `latest`).
These are read-only and managed by the service.
```
GET /v1/publishers/{publisher_id}/books/{book_id}/revisions/{revision_id}
```
If specific IDs are reserved, services document those IDs.
- If a `latest` ID exists, it represent the most recently created
revision. The content of
`publishers/{publisher_id}/books/{book_id}/revisions/latest` and
`publishers/{publisher_id}/books/{book_id}` can differ, as the latest
revision may be different from the current state of the resource.
### User-specified aliases
APIs provide a mechanism for users to assign an [alias][] ID to an
existing revision with a custom method "alias":
- The request message have a `path` field:
- The field be
[annotated as required](/field-behavior-documentation).
- The field identify the [resource type](/resource-types) that it
references.
- The request message have a `alias` field:
- The field be [annotated as required][aip-203].
- If the user calls the method with an existing `alias` with `overwrite` set to
true, the request succeed and the alias will be updated to refer to
the provided revision. If `overwrite` is false, the request must fail with an
error code ALREADY_EXISTS (HTTP 409).
### Rollback
A common use case for a resource with a revision history is the ability to roll
back to a given revision. APIs that support this behavior do so with
a `Rollback` custom method:
- The method use the `POST` HTTP verb.
- The method return a resource revision.
- The request message have a `path` field, referring to the resource
revision whose configuration the resource should be rolled back to.
- The field be [annotated as required][aip-203].
- The field identify the [resource type][aip-123] that it
references.
### Child resources
Resources with a revision history have child resources. If they do,
they be a subset of the descendants of the original resource, and a
given revision's descendants must be a subset of the descendants of the
resource at the time the revision was created.
### Standard methods
Any standard methods implement the corresponding AIPs (AIP-131,
AIP-132, AIP-133, AIP-134, AIP-135), with the following additional behaviors:
- List methods: By default, revisions in the list response be
ordered in reverse chronological order. APIs support the
[`order_by`](./list#ordering) field to override the default behavior.
- If the revision supports aliasing, a delete method with the resource path of
the alias (e.g. `revisions/1.0.2`) remove the alias instead of
deleting the resource.
As revisions are nested under the resource, also see [cascading delete][].
## Rationale
### For the name "revision"
There was significant debate about what to call this pattern, with the
following as proposed options:
- snapshots
- revisions
- versions
Among those, revision was chosen because:
- The term "version" is often used in multiple different contexts (e.g. API
version), and using that noun here may result in confusion during
conversations where it could mean one or the other.
- The term "snapshot" is also used for snapshots of datastores, which may not
follow this pattern.
- The term "revision" does not have many conflicts with terms when describing
an API or resource.
## History
### Switching from a collection extension to a subcollection
In aip.dev prior to 2023-09, revisions were more like extensions of an existing
resource by using `@` symbol. List and delete revisions were custom methods on
the resource collection. A single Get method was used to retrieve either the
resource revision, or the resource.
Its primary advantage was allowing a resource reference to seamlessly refer to
a resource, or its revision.
It also had several disadvantages:
- List revisions is a custom method (`:listRevisions`) on the resource.
- Delete revision is a custom method on the resource.
- Not visible in API discovery documenation.
- Resource IDs cannot use the `@` symbol.
The guidance was modified ultimately to enable revisions to behave like a
resource, which reduces users' cognitive load and allows resource-oriented
clients to easily list, get, create, update, and delete revisions.
## Changelog
- **2024-08-09**: Imported from aip.dev.
[alias]: /122.md#resource-id-aliases
[cascading delete]: /135.md#cascading-delete
[UUID4]:
https://en.wikipedia.org/wiki/Universally_unique_identifier#Version_4_(random)
---
# AEP-164 Soft delete
There are several reasons why a client could desire soft delete and undelete
functionality, but one over-arching reason stands out: recovery from mistakes.
A service that supports undelete makes it possible for users to recover
resources that were deleted by accident.
## Guidance
Services support the ability to "undelete", to allow for situations
where users mistakenly delete resources and need the ability to recover.
If a resource needs to support undelete, the `Delete` method simply
mark the resource as having been deleted, but not completely remove it from the
system. If the method behaves this way, it return `200 OK` with the
updated resource instead of `204 No Content`.
Resources that support soft delete have an `expire_time` field as
described in [AEP-148](/148). Additionally, resources include a `DELETED`
state value if the resource includes a `state` field [AEP-216](/216).
### Undelete
A resource that supports soft delete provide an `Undelete` method:
### Create
Create methods adhere to one of the following:
- If a user attempts a create on a soft-deleted resource, the create
succeed, acting as if the resource did not exist previously.
- The create accept a field (query parameter for OAS),
`overwrite_soft_deleted`. If set to `false`, the request fail if a
soft-deleted resource exists. If set to `true`, the request succeed
if the resource exists acting as if the resource did not exist previously.
### List and Get
Soft-deleted resources be returned in `List` [AEP-132](/132) responses by
default (unless `bool show_deleted` is true).
A `Get` [AEP-131](/131) request for a soft deleted resource return with a
`404 Not Found` response unless `bool show_deleted` is true, in which case
soft-deleted resources return the resource (with the `DELETED` state
value if the resource includes a [`state` field](/states)).
Services that soft delete resources choose a reasonable strategy for
purging those resources, including automatic purging after a reasonable time
(such as 30 days), allowing users to set an expiry time [AEP-214](/214), or retaining
the resources indefinitely. Regardless of what strategy is selected, the
service document when soft deleted resources will be completely
removed.
### Declarative-friendly resources
### Errors
Also see [errors](/errors) for additional guidance.
If the user does have proper permission, but the requested resource does not
exist (either it was never created or already expunged), the service
error with `404 Not Found`.
If the user calling a soft `Delete` has proper permission, but the requested
resource is already deleted, the service succeed if `allow_missing` is
`true`, and error with `404 Not Found` if `allow_missing` is `false`.
If the user calling `Undelete` has proper permission, but the requested
resource is not deleted, the service error with `409 Conflict`.
## Further reading
- For the `Delete` standard method, see [AEP-135](/135).
- For long-running operations, see [AEP-151](/151).
- For resource freshness validation (`etag`), see [AEP-154](/154).
- For change validation (`validate_only`), see [AEP-163](/163).
---
# AEP-180 Protobuf Backwards compatibility
APIs are fundamentally contracts with users, and users often write code against
APIs that is then launched into a production service with the expectation that
it continues to work (unless the API has a [stability level](/181) that
indicates otherwise). Therefore, it is important to understand what constitutes
a backwards compatible change and what constitutes a backwards incompatible
change.
## Guidance
Existing client code be broken by a service updating to a new
minor or patch release. Old clients be able to work against newer
servers (with the same major version number).
There are three distinct types of compatibility to consider:
1. Source compatibility: Code written against a previous version
compile against a newer version, and successfully run with a newer version
of the client library.
2. Wire compatibility: Code written against a previous version be able
to communicate correctly with a newer server. In other words, not only are
inputs and outputs compatible, but the serialization and deserialization
expectations continue to match.
3. Semantic compatibility: Code written against a previous version
continue to receive what most reasonable developers would expect. (This can
be tricky in practice, however, and sometimes determining what users will
expect can involve a judgment call.)
### Adding components
In general, new components (interfaces, methods, messages, fields, enums, or
enum values) be added to existing APIs in the same major version.
However, keep the following guidelines in mind when doing this:
- Code written against the previous surface (and thus is unaware of the new
components) continue to be treated the same way as before.
- New required fields be added to existing request messages or
resources.
- Any field being populated by clients have a default behavior
matching the behavior before the field was introduced.
- Any field previously populated by the server continue to be
populated, even if it introduces redundancy.
- For enum values specifically, be aware that it is possible that user code
does not handle new values gracefully.
- Enum values be freely added to enums which are only used in request
messages.
- Enums that are used in response messages or resources and which are
expected to receive new values document this. Enum values still
be added in this situation; however, appropriate caution
be used.
### Removing or renaming components
Existing components (interfaces, methods, messages, fields, enums, or enum
values) be removed from existing APIs in the same major version.
Removing a component is a backwards incompatible change.
### Moving components between files
Existing components be moved between files.
Moving a component from one proto file to another within the same package is
wire compatible, however, the code generated for languages like C++ or Python
will result in breaking change since `import` and `#include` will no longer
point to the correct code location.
### Moving into oneofs
Existing fields be moved into or out of a oneof. This is a
backwards-incompatible change in the Go protobuf stubs.
### Changing the type of fields
Existing fields and messages have their type changed, even if the
new type is wire-compatible, because type changes alter generated code in a
breaking way.
### Changing resource paths
A resource change its [path](/122).
Unlike most breaking changes, this affects major versions as well: in order for
a client to expect to use v2.0 access a resource that was created in v1.0 or
vice versa, the same resource name be used in both versions.
More subtly, the set of valid resource paths change either, for
the following reasons:
- If resource name formats become more restrictive, a request that would
previously have succeeded will now fail.
- If resource name formats become less restrictive than previously documented,
then code making assumptions based on the previous documentation could break.
Users are very likely to store resource names elsewhere, in ways that may be
sensitive to the set of permitted characters and the length of the name.
Alternatively, users might perform their own resource name validation to
follow the documentation.
- For example, Amazon gave customers [a lot of warning][ec2] and had a
migration period when they started allowing longer EC2 resource IDs.
### Semantic changes
Code will often depend on API behavior and semantics, _even when such behavior
is not explicitly supported or documented_. Therefore, APIs change
visible behavior or semantics in ways that are likely to break reasonable user
code, as such changes will be seen as breaking by those users.
#### Default values must not change
Default values are the values set by servers for resources when they are not
specified by the client. This section only applies to static default values
within fields on resources and does not apply to dynamic defaults such as the
default IP address of a resource.
Changing the default value is considered breaking and be done. The
default behavior for a resource is determined by its default values, and this
change across minor versions.
For example:
```proto
message Book {
// google.api.resource and other annotations and fields
// The genre of the book
// If this is not set when the book is created, the field will be given a value of FICTION.
enum Genre {
UNSPECIFIED = 0;
FICTION = 1;
NONFICTION = 2;
}
}
```
Changing to:
```proto
message Book {
// google.api.resource and other annotations and fields
// The genre of the book
// If this is not set when the book is created, the field will be given a value of NONFICTION.
enum Genre {
UNSPECIFIED = 0;
FICTION = 1;
NONFICTION = 2;
}
}
```
would constitute a breaking change.
#### Serializing defaults
APIs change the way a field with a default value is serialized.
For example if a field does not appear in the response if the value is equal to
the default, the serialization change to include the field with
the default. Clients may depend on the presence or absence of a field in a
resource as semantically meaningful, so a change to how serialization is done
for absent values occur in a minor version.
Consider the following proto, where the default value of `wheels` is `2`:
```proto
// A representation of an automobile
message Automobile {
// google.api.resource and other annotations and fields
// The number of wheels on the automobile.
// The default value is 2, when no value is sent by the client.
int wheels = 2;
}
```
First the proto serializes to JSON when the value of `wheels` is `2` as
follows:
```json
{
"name": "my-car"
}
```
Then, the API service changes the serialization to include `wheel` even if the
value is equal to the default value, `2` as follows:
```json
{
"name": "my-car",
"wheels": 2
}
```
This constitutes a change that is not backwards compatible within a major
version.
## Further reading
- For compatibility around field behavior, see [AEP-203][].
- For compatibility around pagination, see [AEP-158][].
- For compatibility around long-running operations, see [AEP-151][].
- For understanding stability levels and expectations, see [AEP-181][].
[ec2]: https://aws.amazon.com/blogs/aws/theyre-here-longer-ec2-resource-ids-now-available/
---
# AEP-191 File and directory structure
A consistent file and directory structure, while making minimal difference
technically, makes API surface definitions easier for users and reviewers to
read.
## Syntax
APIs defined in protocol buffers use `proto3` syntax.
## Single package
APIs defined in protocol buffers define each individual API in a
single package, which end in a version component. For example:
```proto
syntax = "proto3";
package example.cloud.translation.v3;
```
APIs reside in a directory that matches the protocol buffer `package`
directive. For example, the package above dictates that the directory be
`example/cloud/translation/v3`.
## File names
It is often useful to divide API definitions into multiple files. File names
use `snake_case`.
APIs have an obvious "entry" file, generally named after the API
itself. An API with a small number of discrete services have a separate
entry file per service.
APIs with only one file use a filename corresponding to the name of
the API.
API `service` definitions and associated RPC request and response `message`
definitions be defined in the same file.
Bear in mind that the file names often become module names in client libraries,
and customers use them in `import` or `use` statements. Therefore, choosing a
descriptive and language keyword-free filename does matter. For example, a file
called `import.proto` may be problematic in Python.
## File layout
Individual files place higher level and more important definitions
before lower level and less important definitions.
In a proto file, components be in the following order, and each of
these be separated by a blank line:
- Copyright and license notice (if applicable).
- The proto `syntax` statement.
- The proto `package` statement.
- Any `import` statements, in alphabetical order.
- Any file-level `option` statements.
- Any `service` definitions.
- Methods be grouped by the resource they impact, and standard
methods precede custom methods.
- Resource `message` definitions. A parent resource be defined before
its child resources.
- The RPC request and response `message` definitions, in the same order of the
corresponding methods. Each request message precede its
corresponding response message (if any).
- Any remaining `message` definitions.
- Any top-level `enum` definitions.
---
# AEP-192 Documentation
Documentation is one of the most critical aspects of API design. Users of your
API are unable to dig into the implementation to understand the API better;
often, the API surface definition and its corresponding documentation will be
the only things a user has. Therefore, it is important that documentation be as
clear, complete, and unambiguous as possible.
## Guidance
In APIs defined in protocol buffers, public comments be included over
every component (service, method, message, field, enum, and enum value) using
the protocol buffers comment format. This is important even in cases where the
comment is terse and uninteresting, as numerous tools read these comments and
use them.
Services, in particular, have descriptive comments that explain what
the service is and what users are able to do with it.
### Style
Comments be in grammatically correct American English. However, the
first sentence of each comment omit the subject and be in the
third-person present tense:
```proto
// Creates a book under the given publisher.
rpc CreateBook(CreateBookRequest) returns (Book) {
option (google.api.http) = {
post: "/v1/{parent=publishers/*}/books"
body: "book"
};
}
```
### Descriptions
Descriptions of messages and fields be brief but complete. Sometimes
comments are necessarily perfunctory because there is little to be said;
however, before jumping to that conclusion, consider whether some of the
following questions are relevant:
- What is it?
- How do you use it?
- What does it do if it succeeds? What does it do if it fails?
- Is it idempotent?
- What are the units? (Examples: meters, degrees, pixels)
- What are the side effects?
- What are common errors that may break it?
- What is the expected input format?
- What range of values does it accept? (Examples: `[0.0, 1.0)`, `[1, 10]`)
- Is the range inclusive or exclusive?
- For strings, what is the minimum and maximum length, and what characters
are allowed?
- If a value is above the maximum length, do you truncate or send an error?
- Is it always present? (Example: "Container for voting information. Present
only when voting information is recorded.")
- Does it have a default setting? (Example: "If `page_size` is omitted, the
default is 50.")
### Formatting
Any formatting in comments be in [CommonMark][]. Headings and tables
be used, as these cause problems for several tools, and are
unsuitable for client library reference documentation.
Comments use `code font` for field or method names and for literals
(such as `true`).
Raw HTML be used.
"ASCII art" attempts to present a diagram within the protos be
used. The Markdown within the protos is consumed by a large number of
renderers, and any ASCII art is very unlikely to be well-presented by all of
them. If a diagram is useful in order to understand the API, include a link to
a documentation page containing the diagram as an image.
### Cross-references
Comments "link" to another component (service, method, message, field,
enum, or enum value) by using the fully-qualified name of the element as a
Markdown reference link. For example: `[Book][company.example.v1.Book]`
### External links
Comments link to external pages to provide background information
beyond what is described in the public comments themselves. External links
use absolute (rather than relative) URLs, including the protocol
(usually `https`), and assume the documentation is located on
any particular host. For example:
`[Spanner Documentation](https://cloud.google.com/spanner/docs)`
### Trademarked names
When referring to the proper, trademarked names of companies or products in
comments, acronyms be used, unless the acronym is such dominant
colloquial use that avoiding it would obscure the reference (example: [IBM][]).
Comments spell and capitalize trademarked names consistent with the
trademark owner's current branding.
### Deprecations
To deprecate a component (service, method, message, field, enum, or enum
value), the `deprecated`
[option](https://developers.google.com/protocol-buffers/docs/proto#options)
be set to `true`, and the first line of the respective comment
start with `"Deprecated: "` and provide alternative solutions for
developers. If there is no alternative solution, a deprecation reason
be given.
### Internal comments
Comments be explicitly marked as internal by wrapping internal content
in `(--` and `--)`.
Non-public links, internal implementation notes (such as `TODO` and `FIXME`
directives), and other such material be marked as internal.
[commonmark]: https://commonmark.org/
[ibm]: https://en.wikipedia.org/wiki/IBM
---
# AEP-193 Errors
Effective error communication is an important part of designing simple and
intuitive APIs. Services returning standardized error responses enable API
clients to construct centralized common error handling logic. This common logic
simplifies API client applications and eliminates the need for cumbersome
custom error handling code.
## Guidance
Services clearly distinguish successful responses from error
responses.
The structure of the error response be consistent across all APIs of
the service. You use the error response structure below which comes
from the Problem Details for HTTP APIs [RFC 9457].
### Structure
An error response contain the following fields:
| Name | Type | Required | Description |
| -------- | ------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| type | string | Y | A URI reference [URI] that identifies the problem type. If the type URI is a locator (e.g., those with an "http" or "https" scheme), dereferencing it SHOULD provide human-readable documentation for the problem type. |
| status | integer | | The HTTP status code that best describes the type of problem detected. |
| title | string | | A human-readable description of the problem. Messages are likely to be logged in plain text and include information about a specific occurrence nor any sensitive data like PII. Information about specific occurrences should be part of `metadata`. |
| detail | string | | A human-readable explanation specific to this occurrence of the problem. This contain PII and be logged. |
| instance | string | | A unique identifier for the specific occurrence of the problem. |
An error response may contain additional fields appropriate for a specific
error type.
The error title and detail fields help a reasonably technical user
_understand_ and _resolve_ the issue, and assume that the user
is an expert in your particular API. Additionally, the error title and detail
assume that the user will know anything about its underlying
implementation.
The error detail be brief but actionable. Any extra information
be provided in additional properties. If even more information is
necessary, you provide a link where a reader can get more
information or ask questions to help resolve the issue.
The error detail field
- is a developer-facing, human-readable "debug message".
- both explains the error and offers an actionable resolution to it.
- value may significantly change over time for the same error, and should not
be string-matched by any clients to determine the error.
All human-readable error details returned by the service must use the same
language.
A JSON representation of an error response might look like the following:
```json
{
"type": "RESOURCE_EXHAUSTED",
"status": 429,
"title": "Too Many Requests",
"detail": "The zone 'us-east1-a' does not have enough resources available to fulfill the request. Try a different zone, or try again later.",
"instance": "7934df3e-4b63-429b-b0f5-b8d350ec165e"
}
```
#### Dynamic variables
The best, actionable error detail includes dynamic segments. These variable
parts of the message are specific to a particular request. Consider the
following example:
> The Book, "The Great Gatsby", is unavailable at the Library, "Garfield East".
> It is expected to be available again on 2199-05-13.
The preceding error detail is made actionable by the context, both that
originates from the request, the title of the Book and the name of the Library,
and by the information that is known only by the service, i.e. the expected
return date of the Book.
All dynamic variables found in error detail also be present as
additional properties of the error response
```yaml
book_title: 'The Great Gatsby'
library: 'Garfield East'
expected_return_date: '2199-05-13'
```
Once present for a particular error type, additional properties
continue to be included in the the error response to be backwards compatible,
even if the value for a particular property is empty.
#### Localization
The title and detail fields be presented in the service's native
language. If a localized detail is required, it may be included as an
additional property in the error response, and be named
`localizedDetail`.
### Partial errors
APIs support partial errors. Partial errors add significant
complexity for users, because they usually sidestep the use of error codes, or
move those error codes into the response message, where the user must write
specialized error handling logic to address the problem.
However, occasionally partial errors are necessary, particularly in bulk
operations where it would be hostile to users to fail an entire large request
because of a problem with a single entry.
Methods that require partial errors use [long-running operations][],
and the method put partial failure information in the metadata
message. The errors themselves still be represented as an error object
as described in this AEP.
### Permission Denied
If the user does not have permission to access the resource or parent,
regardless of whether or not it exists, the service error with
`PERMISSION_DENIED` (HTTP 403). Permission be checked prior to
checking if the resource or parent exists.
If the user does have proper permission, but the requested resource or parent
does not exist, the service error with `NOT_FOUND` (HTTP 404).
## Rationale
## Further reading
- [RFC 9457 Problem Details for HTTP APIs][RFC 9457]
## Changelog
- **2024-03-21**: Adopt from https://google.aip.dev/193
[RFC 9457]: https://datatracker.ietf.org/doc/html/rfc9457
[grpc status code documentation]: https://github.com/grpc/grpc/blob/master/doc/statuscodes.md
[Code]: https://github.com/googleapis/googleapis/blob/master/google/rpc/code.proto
[Status]: https://github.com/googleapis/googleapis/blob/master/google/rpc/status.proto
[long-running operations]: /151
[URI]: https://datatracker.ietf.org/doc/html/rfc3986
---
# AEP-200 Precedent
Many times, APIs are written in ways that do not match new guidance that is
added to these standards after those APIs have already been released.
Additionally, sometimes it can make sense to intentionally violate standards
for particular reasons, such as maintaining consistency with established
systems, meeting stringent performance requirements, or other practical
concerns. Finally, as carefully as everyone reviews APIs before they are
released, sometimes mistakes can slip through.
Since it often is not feasible to fix past mistakes or make the standards serve
every use case, APIs may be stuck with these exceptions for quite some time.
Further, since new APIs often base their designs (names, types, structures,
etc) on existing APIs, it is possible that a standards violation in one API
could spill over into other APIs, even if original reason for the exception is
not applicable to the other APIs.
As a result of this problem, it is important to "stop the bleeding" of these
standards exceptions into new APIs, and additionally document the reasons for
each exception so that historical wisdom is not lost.
## Guidance
If an API violates "" or "" AEP guidance for any
reason, there be an internal comment linking to this document using
its descriptive link ([aep.dev/not-precedent]()) to ensure others do not copy
the violations or cite the errors as precedent of a "previously approved API".
The comment should also include an explanation of what violates standards and
why it is necessary. For example:
### Local consistency
If an API violates a standard throughout, it would be jarring and frustrating
to users to break the existing pattern only for the sake of adhering to the
global standard.
For example, if all of an API's resources use `creation_time` (instead of the
standard field `create_time` described in [AEP-142](/142)), a new resource in that API
should continue to follow the local pattern.
However, others who might otherwise copy that API should be made aware that
this is contra-standard and not something to cite as precedent when launching
new APIs.
### Pre-existing functionality
Standards violations are sometimes overlooked before launching, resulting in
APIs that become stable and therefore can not easily be modified. Additionally,
a stable API may pre-date a standards requirement.
In these scenarios, it is difficult to make the API fit the standard. However,
the API should still cite that the functionality is contra-standard so that
other APIs do not copy the mistake and cite the existing API as a reason why
their design should be approved.
### Adherence to external spec
Occasionally, APIs must violate standards because specific requests are
implementations of an external specification (for example, OAuth), and their
specification may be at odds with AEP guidelines. In this case, it is likely to
be appropriate to follow the external specification.
### Adherence to existing systems
Similar to the example of an external specification above, it may be proper for
an API to violate AEP guidelines to fit in with an existing system in some way.
This is a fundamentally similar case where it is wise to meet the customer
where they are. A potential example of this might be integration with or
similarity to a partner API.
### Expediency
Sometimes there are users who need an API surface by a very hard deadline or
money walks away. Since most APIs serve a business purpose, there will be times
when an API could be better but cannot get it that way and into users' hands
before the deadline. In those cases, API review councils grant
exceptions to ship APIs that violate guidelines due to time and business
constraints.
### Technical concerns
Internal systems sometimes have very specific implementation needs (e.g., they
rely on operation transforms that speak UTF-16, not UTF-8) and adhering to AEP
guidelines would require extra work that does not add significant value to API
consumers. Future systems which are likely to expose an API at some point
should bear this in mind to avoid building underlying infrastructure which
makes it difficult to follow AEP guidelines.
---
# AEP-203 Field behavior documentation
When defining fields in protocol buffers, it is customary to explain to users
certain aspects of the field's behavior (such as whether it is required or
optional). Additionally, it can be useful for other tools to understand this
behavior (for example, to optimize client library signatures).
## Guidance
APIs use the `aep.api.field_info` annotation to describe well-understood field
behavior, such as a field being required or immutable.
```proto
// The audio data to be recognized.
RecognitionAudio audio = 2 [(aep.api.field_info) = {
field_behavior: [FIELD_BEHAVIOR_REQUIRED]
}];
```
- APIs apply the `aep.api.field_info` annotation on every field on a
message or sub-message used in a request.
- The annotation include any aep.api.FieldBehavior values that
accurately describe the behavior of the field.
- `FIELD_BEHAVIOR_UNSPECIFIED` be used.
- APIs at minimum use one of `FIELD_BEHAVIOR_REQUIRED`,
`FIELD_BEHAVIOR_OPTIONAL`, or `FIELD_BEHAVIOR_OUTPUT_ONLY`.
### field behavior of nested messages
`aep.api.field_info` annotations on a nested message are independent of the
annotations of the parent.
For example, a nested message can have a field behavior of
`FIELD_BEHAVIOR_REQUIRED` while the parent field can be
`FIELD_BEHAVIOR_OPTIONAL`:
```proto
message Title {
string text = 1 [(aep.api.field_info) = {
field_behavior: [FIELD_BEHAVIOR_REQUIRED]
}];
}
message Slide {
Title title = 1 [(aep.api.field_info) = {
field_behavior: [FIELD_BEHAVIOR_OPTIONAL]
}];
}
```
In the case above, if a `title` is specified, the `text` field is required.
## Vocabulary
### Immutable
The use of `FIELD_BEHAVIOR_IMMUTABLE` indicates that a field on a resource
cannot be changed after it's creation. This can apply to either fields that are
input or outputs, required or optional.
When a service receives an immutable field in an update request (or similar),
even if included in the update mask, the service ignore the field if
the value matches, but error with `INVALID_ARGUMENT` if a change is
requested.
Potential use cases for immutable fields (this is not an exhaustive list) are:
- Attributes of resources that are not modifiable for the lifetime of the
application (e.g. a disk type).
### Input only
The use of `FIELD_BEHAVIOR_INPUT_ONLY` indicates that the field is provided in
requests and that the corresponding field will not be included in output.
Additionally, a field only be described as input only if it is a
field in a resource message or a field of a message included within a resource
message. Notably, fields in request messages (a message which only ever acts as
an argument to an RPC, with a name usually ending in `Request`)
be described as input only because this is already implied.
Potential use cases for input only fields (this is not an exhaustive list) are:
- The `ttl` field as described in [AEP-214](/214).
### Optional
The use of `FIELD_BEHAVIOR_OPTIONAL` indicates that a field is not required.
A field be described as optional if it is a field on a request message
(a message that is an argument to an RPC, usually ending in `Request`), or a
field on a submessage.
### Output only
The use of `FIELD_BEHAVIOR_OUTPUT_ONLY` indicates that the field is provided in
responses, but that including the field in a message in a request does nothing
(the server clear out any value in this field and throw
an error as a result of the presence of a value in this field on input).
Similarly, services ignore the presence of output only fields in
update field masks (see: [AEP-161](/161)).
Additionally, a field only be described as output only if it is a
field in a resource message, or a field of a message farther down the tree.
Notably, fields in response messages (a message which only ever acts as a
return value to an RPC, usually ending in `Response`) be
described as output only because this is already implied.
Output only fields be set to empty values if appropriate to the API.
Potential use cases for output only fields (this is not an exhaustive list)
are:
- Create or update timestamps.
- Derived or structured information based on original user input.
- Properties of a resource assigned by the service which can not be altered.
### Required
The use of `FIELD_BEHAVIOR_REQUIRED` indicates that the field be
present (and set to a non-empty value) on the request.
A field only be described as required if _either_:
- It is a field on a resource that a user provides somewhere as input.
- When [creating](./standard-methods) the resource, a value be
provided for the field on the create request.
- When [updating](./standard-methods) the resource, the user omit the
field provided that the field is also absent from the field mask,
indicating no change to the field (otherwise it be provided).
- It is a field on a request message (a message that is an argument to an RPC,
with a name usually ending in `Request`). In this case, a value be
provided as part of the request, and failure to do so cause an error
(usually `INVALID_ARGUMENT`).
Fields be described as required in order to signify:
- A field which will always be present in a response.
- A field which is conditionally required in some situations.
- A field on any message (including messages that are resources) which is never
used as user input.
### Unordered List
The use of `FIELD_BEHAVIOR_UNORDERED_LIST` on a repeated field of a resource
indicates that the service does not guarantee the order of the items in the
list.
A field be described as an unordered list if the service does not
guarantee that the order of the elements in the list will match the order that
the user sent, including a situation where the service will sort the list on
the user's behalf.
## Backwards compatibility
Adding or changing `aep.api.field_info` values can represent a semantic change
in the API that is perceived as incompatible for existing clients. The
following are examples of backwards incompatible changes with
`aep.api.field_info`:
- Adding `FIELD_BEHAVIOR_REQUIRED` to an existing field previously considered
`FIELD_BEHAVIOR_OPTIONAL` (implicitly or otherwise)
- Adding a new field annotated as `FIELD_BEHAVIOR_REQUIRED` to an existing
request message
- Adding `FIELD_BEHAVIOR_OUTPUT_ONLY` to an existing field previously accepted
as input
- Adding `FIELD_BEHAVIOR_INPUT_ONLY` to an existing field previously emitted as
output
- Adding `FIELD_BEHAVIOR_IMMUTABLE` to an existing field previously considered
mutable
- Removing `FIELD_BEHAVIOR_OUTPUT_ONLY` from an existing field previously
ignored as input
There are some changes that _are_ backwards compatible, which are as follows:
- Adding `FIELD_BEHAVIOR_OPTIONAL` to an existing field
- Changing from `FIELD_BEHAVIOR_REQUIRED` to `FIELD_BEHAVIOR_OPTIONAL` on an
existing field
- Removing `FIELD_BEHAVIOR_REQUIRED` from an existing field
- Removing `FIELD_BEHAVIOR_INPUT_ONLY` from an existing field previously
excluded in responses
- Removing `FIELD_BEHAVIOR_IMMUTABLE` from an existing field previously
considered immutable
## Rationale
### Required set of annotations
A field used in a request message must be either an input or an output.
In the case of an output, the `FIELD_BEHAVIOR_OUTPUT_ONLY` annotation is
sufficient.
In the case of an input, a field is either required or optional, and therefore
should have at least the `FIELD_BEHAVIOR_REQUIRED` or `FIELD_BEHAVIOR_OPTIONAL`
annotation, respectively. Only providing `FIELD_BEHAVIOR_INPUT_ONLY` does not
convey the necessity of the field, so specifying either
`FIELD_BEHAVIOR_REQUIRED` or `FIELD_BEHAVIOR_OPTIONAL` is still necessary.
### Requiring field behavior
By including the field behavior annotation for each field, the overall behavior
that the resource exhibits is more clearly defined. Clearly defined field
behavior improves programmatic clients and user understanding.
Requiring the annotation also forces the API author to explicitly consider the
behavior when initially authoring of the API.
Modifying field behavior after initial authoring can result in
backwards-incompatible changes in clients. For example, making an optional
field required results in backwards-incompatible changes in the method
signature of an RPC or a resource in a Declarative client. See the
[Backwards compatibility](#backwards-compatibility) section for more detailed
compatibility guidance.
[fields representing resource paths]:
./resource-paths#fields-representing-resource-paths
## Changelog
- **2024-04-17**: Added initial guidance.
- **2025-09-06**: Removal of the `IDENTIFIER` field.
- **2025-10-11**: Updated to use the new `aep.api.FieldBehavior` enum values
with `FIELD_BEHAVIOR_` prefix.
---
# AEP-205 Beta-blocking changes
APIs often release an alpha version of their API in order to get early feedback
from customers. This API is provisional and can change many times before the
important feedback is incorporated and the API is made stable for beta.
Since the purpose of alpha is to gather feedback, the API does not need to be
perfect yet, and it's not strictly necessary for API authors to address every
usability concern or address every point in the API standards. Often, API
authors and API reviewers will not agree on the best design, and the best way
to find out is by having users try out the API.
However, once the feedback has been collected and the API is going to be
promoted to beta, usability concerns and style issues need to be addressed. In
order to ensure that these issues are not forgotten, they be
explicitly documented in the API.
## Guidance
If an API has usability concerns or violates API standards, and the present
design should receive additional scrutiny before being carried through to the
beta version, there be an internal comment linking to this document
using its descriptive link ([aep.dev/beta-blocker]()) to ensure that the design
is corrected before the API is released to beta.
The comment also indicate what kind of change should be made for beta.
For example:
If an exception to API standards _does_ need to be carried through to beta and
GA, see [AEP-200](/200).
---
# AEP-210 Unicode
APIs should be consistent on how they explain, limit, and bill for string
values and their encodings. This ranges from little ambiguities (like fields
"limited to 1024 characters") all the way to billing confusion (are names and
values of properties in Datastore billed based on characters or bytes?).
In general, if limits are measured in bytes, we are discriminating against
non-ASCII text since it takes up more space. On the other hand, if limits are
measured in "characters", this is ambiguous about whether those are Unicode
"code points", "code units" for a particular encoding (e.g. UTF-8 or UTF-16),
"graphemes", or "grapheme clusters".
## Unicode primer
Character encoding tends to be an area we often gloss over, so a quick primer:
- Strings are just sequences of bytes that represent text according to some
encoding format.
- When we talk about **characters**, we sometimes mean Unicode **code points**,
which are 21-bit unsigned integers `0` through `0x10FFFF`.
- Other times we might mean **grapheme clusters**, which are _perceived_ as
single characters but may be composed of multiple code points. For example,
`á` can be represented as the single code point `U+00E1` or as a sequence of
`U+0061` followed by `U+0301` (the letter `a`, then a combining acute
accent).
- Protocol buffers uses **UTF-8** ("Unicode Transformation Format") which is a
variable-length encoding scheme that represents each code point as a sequence
of 1 to 4 single-byte **code units**.
## Guidance
### Character definition
In API documentation (e.g., API reference documents, blog posts, marketing
documentation, billing explanations, etc), "character" be defined as a
Unicode code point.
### Length units
All string field length limits defined in the API be measured and
enforced in characters as defined above. This means that there is an underlying
maximum limit of (`4 * characters`) bytes, though this limit will only be hit
when using exclusively characters that consist of 4 UTF-8 code units (32 bits).
If you use a database system (e.g. Spanner) which allows you to define a limit
in characters, it is safe to assume that this byte-defined requirement is
handled by the underlying storage system.
### Billing units
APIs use either code points or bytes (using the UTF-8 encoding) as the
unit for billing or quota measurement (e.g., Cloud Translation chooses to use
characters). If an API does not define this, the assumption is that the unit of
billing is characters (e.g., $0.01 _per character_, not $0.01 _per byte_).
### Unique identifiers
Strings used as unique identifiers limit inputs to ASCII characters,
typically letters, numbers, hyphens, and underscores
(`[a-zA-Z][a-zA-Z0-9_-]*`). This ensures that there are never accidental
collisions due to normalization. If an API decides to allow all valid Unicode
characters in unique identifiers, the API reject any inputs that are
not in Normalization Form C.
Unique identifiers use a maximum length of 64 characters, though
this limit may be expanded as necessary. 64 characters should be sufficient for
most purposes as even UUIDs only require 36 characters.
### Normalization
Values always be normalized into Normalization Form C. Unique
identifiers always be stored in Normalization Form C (see the next
section).
Imagine we're dealing with Spanish input "estaré" (the
accented part will be bolded throughout). This text has 6 grapheme clusters,
and can be represented by two distinct sequences of Unicode code points:
- Using 6 code points: `U+0065` `U+0073` `U+0074` `U+0061` `U+0072`
**`U+00E9`**
- Using 7 code points: `U+0065` `U+0073` `U+0074` `U+0061` `U+0072` **`U+0065`
`U+0301`**
Further, when encoding to UTF-8, these code points have two different
serialized representations:
- Using 7 code-units (7 bytes): `0x65` `0x73` `0x74` `0x61` `0x72` **`0xC3`
`0xA9`**
- Using 8 code-units (8 bytes): `0x65` `0x73` `0x74` `0x61` `0x72` **`0x65`
`0xCC` `0x81`**
To avoid this discrepancy in size (both code units and code points), use
[Normalization Form C][] which provides a canonical representation for strings.
[normalization form c]: https://unicode.org/reports/tr15/
### Uniqueness
For the purposes of unique identification (e.g., `name`, `id`, or `parent`),
the value be normalized into [Normalization Form C][] (which happens
to be the most compact). Otherwise we may have what is essentially "the same
string" used to identify two entirely different resources.
In our example above, there are two ways of representing what is essentially
the same text. This raises the question about whether the two representations
should be treated as equivalent or not. In other words, if someone were to use
both of those byte sequences in a string field that acts as a unique
identifier, would it violate a uniqueness constraint?
The W3C recommends using Normalization Form C for all content moving across the
internet. It is the most compact normalized form on Unicode text, and avoids
most interoperability problems. If we were to treat two Unicode byte sequences
as different when they have the same representation in NFC, we'd be required to
reply to possible "Get" requests with content that is **not** in normalized
form. Since that is definitely unacceptable, we treat the two as
identical by transforming any incoming string data into Normalized Form C or
rejecting identifiers not in the normalized form.
There is some debate about whether we should view strings as sequences of code
points encoded into byte sequences (leading to uniqueness determined based on
the byte-representation of said string) or to interpret strings as a higher
level abstraction having many different possible byte-representations. The
stance taken here is that we already have a field type for handling that:
`bytes`. Fields of type `string` already express an opinion of the validity of
an input (it must be valid UTF-8). As a result, treating two inputs that have
identical normalized forms as different due to their underlying byte
representation seems to go against the original intent of the `string` type.
This distinction typically doesn't matter for strings that are opaque to our
services (e.g., `description` or `display_name`), however when we rely on
strings to uniquely identify resources, we are forced to take a stance.
Put differently, our goal is to allow someone with text in any encoding (ASCII,
UTF-16, UTF-32, etc) to interact with our APIs without a lot of "gotchas".
## References
- [Unicode normalization forms](https://unicode.org/reports/tr15/)
- [Datastore pricing "name and value of each property"](https://cloud.google.com/datastore/pricing)
doesn't clarify this.
- [Natural Language pricing](https://cloud.google.com/natural-language/pricing)
uses charges based on UTF-8 code points rather than code units.
- [Text matching and normalization](https://sites.google.com/a/google.com/intl-eng/apis/matching?pli=1)
---
# AEP-211 Authorization checks
The majority of operations, whether reads or writes, require authorization:
permission to do the thing the user is asking to do. Additionally, it is
important to be careful how much information is provided to _unauthorized_
users, since leaking information can be a security concern.
## Guidance
Services check authorization before validating any request, to ensure
both a secure API surface and a consistent user experience. An operation
require multiple permissions or preconditions in order to grant
authorization.
If a request can not pass the authorization check for any reason, the service
error with `403 Forbidden`, and the corresponding error message
look like: "Permission `{p}` denied on resource `{r}` (or it might not
exist)." This avoids leaking resource existence.
If it is not possible to determine authorization for a resource because the
resource does not exist, the service check authorization to read
children on the parent resource, and return `404 Not Found` if the
authorization check passes.
### Multiple operations
A service could encounter a situation where it has two different operations
with two different permissions, either of which would reveal the existence of a
resource if called, but a user only has permission to call one of them.
In this situation, the service only check for authorization applicable
to the operation being called, rather than check for related authorization that
would provide permission to reveal existence. Such algorithms are complicated
to implement correctly and prone to accidental leaks.
For example, consider a scenario where:
- A resource exists within a given collection that a user is unable to read.
- The user _does_ have the ability to create other resources, and the
collection uses user-specified IDs (meaning that a failure because of a
duplicate ID would reveal existance).
In this situation, the get or create methods still only check _their_
permissions when determining what error to return, and not one another's.
## Rationale
[RFC 7231 §6.5.3][] states that services are permitted to use `404 Not Found`
in lieu of `403 Forbidden` in situations where the service does not want to
divulge existance, whereas this AEP argues for the use of `403 Forbidden`
instead. We take this position for the following reasons:
- The practice of "getting `404 Not Found` until you have enough permission to
get `403 Forbidden`" is counter-intuitive and increases the difficulty of
troubleshooting.
- A service _could_ ameliorate this by sending information about missing
permissions while still using the `404 Not Found` status code, but this
constitutes a mixed message.
- While `403 Forbidden` is essentially always an error requiring manual action,
`404 Not Found` is often a valid response that the application can handle
(e.g. "get or create"); overloading it for permission errors deprives
applications of this benefit.
- RFC 7231 §6.5.4 states that `404 Not Found` results are cacheable, but
permission errors are not generally cacheable. Sending explicit cache
controls on a conditional basis could ameliorate this, but would defeat the
purpose.
- The guidance here is more consistent with most other real-world authorization
systems.
[rfc 7231 §6.5.3]: https://tools.ietf.org/html/rfc7231#section-6.5.3
---
# AEP-213 Common components
In general, APIs should be designed to be self-contained. APIs generally need
to be able to move forward independently of one another, and mutual
dependencies can cause downstream APIs to be forced into taking major version
changes or even lead to dependency conflicts.
However, there are also cases where common structures are valuable, especially
where a concept is well-known and it is sufficiently clear that it will not
change. Having a single representation of these common structures is valuable
because it avoids disrepancies between APIs in things like how dates or
monetary values are represented. It also enables shared libraries for common
operations, such as basic arithmetic on monetary values.
Common components serve this use case.
## Guidance
The public representation of APIs be self-contained (for protocol
buffers, this means that all API protos used by the API originate in the same
proto `package`), except for common components, which be used freely in
any API.
An API define a set of API-specific common components which live
outside of its versioning structure. This prevents independent movement of
particular versions and also causes problems for client libraries in many
languages that compile protobuf messages into classes.
An API define alternative representations of any of the existing
common components described below, even within its versioning structure.
## Existing common components
The common components, which public-facing APIs safely depend on, are
defined canonically in the [AEP common components][] repository. These include
READMEs with canonical definitions for each component, and -- when applicable
-- implementations of these definitions, in both JSON Schema and protobuf
formats. The protobufs are also published to the Buf Schema Sepository at
[`buf.build/aep`][buf], and the JSON schemas are published to the [JSON Schema
Store][] with names beginning with `aep-`, for example `aep-type-money` or
`aep-longrunning-operation`.
While the [AEP common components][] repository is canonical and takes
precedence over this list, some of the common components defined there include
representations of the following concepts:
### API design patterns
- `Operation`: Represents the status of a long-running request (see [AEP-151](/151) for
details).
#### gRPC-specific API design patterns
- [`google.api.*`][api] (but _not_ subpackages of `google.api`): Annotations
useful for gRPC/JSON transcoding, supported by frameworks including .NET 7.
- [`google.rpc.*`][rpc]: A small number of components related to gRPC
request/response status and errors.
### Common types
This section provides examples for the sake of illustration; it may not be
exhaustive. The [AEP common components][] repo is canonical.
#### General common types
- [`Color`][color]: RGB or RGBA colors.
- [`Decimal`][decimal]: Decimal numbers.
- [`Fraction`][fraction]: A numeric fraction.
- [`LatLng`][lat_lng]: Geographic coordinates.
- [`Money`][money]: An amount of money in a given currency.
- [`PhoneNumber`][phone_number]: A phone number in most countries.
- [`PostalAddress`][postal_address]: Postal addresses in most countries.
- [`Quaternion`][quaternion]: A geometric quaternion.
#### Date- and time-related types
- [`Date`][date]: A calendar date, with no time or time zone component.
- [`DateTime`][date_time]: A calendar date and wall-clock time, with optional
time zone or UTC offset information.
- [`DayOfWeek`][day_of_week]: An enumeration representing the day of the week.
- [`Duration`][duration]: A duration with nanosecond-level precision.
- [`Interval`][interval]: An interval between two timestamps.
- [`Month`][month]: An enumeration representing the Gregorian month.
- [`TimeOfDay`][time_of_day]: Wall-clock time, with no date or time zone
component.
- [`Timestamp`][timestamp]: A timestamp with nanosecond-level precision.
##### Protobuf well-known types
The [`google.protobuf`][protobuf] package is shipped with protocol buffers
itself, rather than with API tooling. The Well-Known Types defined in this
package should always be used when appropriate, and the [AEP common
components][] repo does not define any protos for these types, even when it
defines a corresponding JSON Schema. These include:
- [`google.protobuf.Duration`][duration]: Durations, with nanosecond-level
precision. The protobuf runtime provides helper functions to convert to and
from language-native duration objects where applicable (such as Python's
[`timedelta`][timedelta]).
- [`google.protobuf.Timestamp`][timestamp]: Timestamps, with nanosecond-level
precision. The protobuf runtime provides helper functions in most languages
to convert to and from language-native timestamp objects (such as Python's
[`datetime`][datetime]).
`google.protobuf` also provides some useful components that correspond to JSON
primitives (and so have no representation at all in the [AEP common
components][] repo), namely:
- [`google.protobuf.Value`][struct]: An arbitrary JSON value. The protobuf
runtime provides helper functions in most languages to convert `Value`
objects to and from JSON.
- [`google.protobuf.Struct`][struct]: JSON-like structures (a dictionary of
primitives, lists, and other dictionaries). The protobuf runtime provides
helper functions in most languages to convert `Struct` objects to and from
JSON.
`google.protobuf.Struct` and `google.protobuf.Value` are designated common
components by this AEP; proto-based APIs use them when representing
arbitrary JSON-like structures.
## Libraries for common types
For the common components in the `aep.type` namespace, which represent common
data types, the [AEP common components][] repo may contain canonical libraries
in a number of languages. These libraries are designed to be idiomatic in a
given language, and should feel similar to using the language's standard
libraries. They should provide basic functionality like adding two `Money`
values, or determining if one `Date` comes before another.
When a language already has a standard library representation of a common type
(such as Python's `datetime` for the `Timestamp` type), there may instead be a
library for converting the JSON or protobuf representation to the standard
library representation.
If a library you want does not exist, and you want to contribute one, please
[open an issue][] on the [AEP common components][] repository in GitHub.
## Appendix: Adding to common components
Occasionally, it may be useful to add protos to these packages or to add to the
list of common components. In order to do this, [open an issue][] on the [AEP
common components][] repository in GitHub.
However, some general guidelines are worth noting for this:
- Schemas only be granted common component status if it is certain
that they will never change (at all -- even in ways that would normally be
considered backwards compatible). Common components are not versioned, and it
must be the case that API creators and consumers can rely on the component to
be a complete and accurate representation indefinitely.
- Schemas must be applicable to a significant number of APIs for consideration
as common components.
- Even after a common component is added, APIs using local versions
continue to do so until they go to the next major version.
[api]: https://github.com/googleapis/googleapis/tree/master/google/api
[rpc]: https://github.com/googleapis/googleapis/tree/master/google/rpc
[color]: https://github.com/aep-dev/aep/tree/master/schemas/type/color.md
[decimal]: https://github.com/aep-dev/aep/tree/master/schemas/type/decimal.md
[fraction]: https://github.com/aep-dev/aep/tree/master/schemas/type/fraction.md
[lat_lng]: https://github.com/aep-dev/aep/tree/master/schemas/type/lat_lng.md
[money]: https://github.com/aep-dev/aep/tree/master/schemas/type/money.md
[phone_number]: https://github.com/aep-dev/aep/tree/master/schemas/type/phone_number.md
[postal_address]: https://github.com/aep-dev/aep/tree/master/schemas/type/postal_address.md
[quaternion]: https://github.com/aep-dev/aep/tree/master/schemas/type/quaternion.md
[date]: https://github.com/aep-dev/aep/tree/master/schemas/type/date.md
[date_time]: https://github.com/aep-dev/aep/tree/master/schemas/type/date_time.md
[day_of_week]: https://github.com/aep-dev/aep/tree/master/schemas/type/day_of_week.md
[duration]: https://github.com/aep-dev/aep/tree/master/schemas/type/duration.md
[interval]: https://github.com/aep-dev/aep/tree/master/schemas/type/interval.md
[month]: https://github.com/aep-dev/aep/tree/master/schemas/type/month.md
[time_of_day]: https://github.com/aep-dev/aep/tree/master/schemas/type/time_of_day.md
[timestamp]: https://github.com/aep-dev/aep/tree/master/schemas/type/timestamp.md
[datetime]: https://docs.python.org/3/library/datetime.html#datetime.datetime
[duration]: https://github.com/protocolbuffers/protobuf/blob/master/src/google/protobuf/duration.proto
[protobuf]: https://github.com/protocolbuffers/protobuf/tree/main/src/google/protobuf
[struct]: https://github.com/protocolbuffers/protobuf/blob/master/src/google/protobuf/struct.proto
[timedelta]: https://docs.python.org/3/library/datetime.html#datetime.timedelta
[timestamp]: https://github.com/protocolbuffers/protobuf/blob/master/src/google/protobuf/timestamp.proto
[open an issue]: https://github.com/aep-dev/aep/issues
[aep common components]: https://github.com/aep-dev/aep
[json schema store]: https://www.schemastore.org/json/
[buf]: https://buf.build/aep
---
# AEP-214 Resource expiration
Customers often want to provide the time that a given resource or resource
attribute is no longer useful or valid (e.g. a rotating security key).
Currently we recommend that customers do this by specifying an exact
"expiration time" into a `Timestamp expire_time` field; however, this adds
additional strain on the user when they want to specify a relative time offset
until expiration rather than a specific time until expiration.
Furthermore, the world understands the concept of a "time-to-live", often
abbreviated to TTL, but the typical format of this field (an integer, measured
in seconds) results in a sub-par experience when using an auto-generated client
library.
## Guidance
1. APIs wishing to convey an expiration rely on a `Timestamp` field
called `expire_time`.
2. APIs wishing to allow a relative expiration time must define a `oneof`
called `expiration` (or `{something}_expiration`) containing both the
`expire_time` field and a separate `Duration` field called `ttl`, the
latter marked as input only.
3. APIs always return the expiration time in the `expire_time` field
and leave the `ttl` field blank when retrieving the resource.
4. APIs that rely on the specific semantics of a "time to live" (e.g., DNS
which must represent the TTL as an integer) use an `int64 ttl`
field (and provide an [aep.dev/not-precedent](/0200) comment in
this case).
### Example
## Rationale
### Alternatives considered
#### A new standard field called `ttl`
We considered allowing a standard field called `ttl` as an alternative way of
defining the expiration, however doing so would require that API services
continually update the field, like a clock counting down. This could
potentially cause problems with the read-modify-write lifecycle where a
resource is being processed for some time, and effectively has its life
extended as a result of that processing time.
#### Always use `expire_time`
This is the current state of the world with a few exceptions. In this scenario,
we could potentially push the computation of `now + ttl = expire_time` into
client libraries; however, this leads to a somewhat frustrating experience in
the command-line and using REST/JSON. Leaving things as they are is typically
the default, but it seems many customers want the ability to define relative
expiration times as it is quite a bit easier and removes questions of time
zones, stale clocks, and other silly mistakes.
---
# AEP-215 Common component versions
---
# AEP-216 States
Many API resources carry a concept of "state": ordinarily, the resource's place
in its lifecycle. For example, a virtual machine may be being provisioned,
available for use, being spun down, or potentially be in one of several other
situations. A job or query may be preparing to run, be actively running, have
completed, and so on.
## Guidance
Resources needing to communicate their state use an enum, which
be called `State` (or, if more specificity is required, end in the
word `State`). This enum be nested within the message it describes
when only used as a field within that message.
### Enum values
Ideally, APIs use the same terminology throughout when expressing the same
semantic concepts. There are usually many words available to express a given
state, but our customers often use multiple APIs together, and it is easier for
them when our terms are consistent.
At a high level:
- Resources that are available for use are `ACTIVE` (preferred over terms such
as "ready" or "available").
- Resources that have completed a (usually terminal) requested action use past
participles (usually ending in `-ED`), such as `SUCCEEDED` (not
"successful"), `FAILED` (not "failure"), `DELETED`, `SUSPENDED`, and so on.
- Resources that are currently undergoing a state change use present
participles (usually ending in `-ING`), such as `RUNNING`, `CREATING`,
`DELETING`, and so on. In this case, it is expected that the state is
temporary and will resolve to another state on its own, with no further user
action.
### Output only
The field referencing the `State` enum in a resource behave and be
documented as "Output only", in accordance with
[field behavior documentation](./field-behavior-documentation).
APIs allow a `State` enum to be directly updated through an
"update" method (or directly set through the "create" method), and
instead use custom state transition methods.
This is because update methods are generally not expected to have side effects,
and also because updating state directly implies that it is possible to set the
state to any available value, whereas states generally reflect a resource's
progression through a lifecycle.
### State transition methods
State transition methods are a special type of
[custom method](./custom-methods) that are responsible for transitioning a
state field from one enum value to another. As part of the transition, other
fields may also change, e.g. an `update_time` field. In addition to the general
guidance for custom methods on resources, the following guidance applies for
method definitions:
- The name of the method be a verb followed by the singular form of
the resource's message name.
- The HTTP verb be `POST`.
In addition to the general guidance for custom methods on resources, the
following guidance applies for request definitions:
- A resource path field be included. It be called `path`.
- The comment for the field document the resource pattern.
- Other fields be included.
## Additional Guidance
### Value uniqueness
Multiple top-level enums within the same package not share the same
values. This is because the C++ protoc code generator flattens top-level enum
values into a single namespace.
State enums live inside the resource definition.
### Prefixes
Using a `STATE_` prefix on every enum value is unnecessary. State enum values
be prefixed with the enum name, except for the default value
`STATE_UNSPECIFIED`.
### Breaking changes
Even though adding states to an existing states enum _can_ break existing user
code, adding states is not considered a breaking change. Consider a state with
only two values: `ACTIVE` and `DELETED`. A user may add code that checks
`if state == ACTIVE`, and in the else cases simply assumes the resource is
deleted. If the API later adds a new state for another purpose, that code will
break.
API documentation actively encourage users to code against state
enums with the expectation that they may receive new values in the future.
APIs add new states to an existing State enum when appropriate, and
adding a new state is _not_ considered a breaking change.
### When to avoid states
Sometimes, a `State` enum may not be what is best for your API, particularly in
situations where a state has a very small number of potential values, or when
states are not mutually exclusive.
Consider the example of a state with only `ACTIVE` and `DELETED`, as discussed
above. In this situation, the API may be better off exposing a
`google.protobuf.Timestamp delete_time`, and instructing users to rely on
whether it is set to determine deletion.
### Common states
The following is a list of states in common use. APIs consider prior
art when determining state names, and value local consistency above
global consistency in the case of conflicting precedent.
#### Resting states
"Resting states" are lifecycle states that, absent user action, are expected to
remain indefinitely. However, the user can initiate an action to move a
resource in a resting state into certain other states (resting or active).
- `ACCEPTED`
- `ACTIVE`
- `CANCELLED`
- `DELETED`
- `FAILED`
- `SUCCEEDED`
- `SUSPENDED`
- `VERIFIED`
#### Active states
"Active states" are lifecycle states that typically resolve on their own into a
single expected resting state.
- `CREATING` (usually becomes `ACTIVE`)
- `DELETING` (usually becomes `DELETED`)
- `PENDING` (usually becomes `RUNNING`)
- `REPAIRING` (usually becomes `ACTIVE`)
- `RUNNING` (usually becomes `SUCCEEDED`)
- `SUSPENDING` (usually becomes `SUSPENDED`)
## Further reading
- For information on enums generally, see [enumerations](./enumerations).
---
# AEP-217 Unreachable resources
Occasionally, a user may ask for a list of resources, and some set of resources
in the list are temporarily unavailable. For example, a user may ask to list
resources across multiple parent locations, but one of those locations is
temporarily unreachable. In this situation, it is still desirable to provide
the user with all the available resources, while indicating that something is
missing.
## Guidance
If a method to retrieve data is capable of partially failing due to one or more
resources being temporarily unreachable, the response message include
a field to indicate this:
- The field be a repeated string/string array, and be named
`unreachable`.
- The field be set to the paths of the resources which are the cause
of the issue, such as the parent or individual resources that could not be
reached. The objects listed as unreachable be _parents_ (or higher
ancestors) rather than the individual resources being requested. For example,
if a location is unreachable, the location is listed.
- The response provide any other information about the issue,
such as error details or codes. Toe enable users to discover what the
underlying issue is, APIs provide an API method that provides
more specific information.
- The service provide a way for the user to get an error with
additional information, and allow the user to repeat the
original call with more restrictive parameters in order to do so.
- The resource paths provided in this field be heterogeneous. The
field document what potential resources may be provided in this
field, and note that it might expand later.
### Pagination
When paginating over a list, it is likely that the service will not know that
there are unreachable parents or resources initially. Further, parents may
alternate between being available and unavailable in unpredictable ways
throughout the process of listing all the requested resources.
These facts lead to the following guidance:
- The response provide any outstanding unreachable locations or
resources in the `unreachable` field on pages _following_ the final page that
contains a resource.
- The response include both requested data and unreachable
resources on the same page.
- For example, if there are two pages of books and one unavailable
publisher, there should be three pages total: first the two pages of
books, and then a final page with no books and the unavailable publisher.
- If the number of unreachable resources to list is very large, the response
honor the `max_page_size` field in the same way as for
resources. In this case, all pages with requested information
precede all pages with unavailable resources or locations.
- The final page's `unreachable` field _only_ include resources or
parents that were partially provided (or missing completely) across the
entirety of the pagination process.
- For example, if a parent or resource was unreachable at the beginning of
pagination and it became reachable again and the entire set of previously
unreachable data was provided to the user on any page, the `unreachable`
field include the intermittently-unreachable parent or
resource.
- On the other hand, if only _some_ of the resources for a given parent are
provided during such an incident as described above, the parent or
resource be included in the `unreachable` field.
## Further reading
- For listing across collections, see
[reading-across-collections](./reading-across-collections).
---
# AEP-231 Batch methods- Get
Some APIs need to allow users to get a specific set of resources at a
consistent time point (e.g. using a read transaction). A batch get method
provides this functionality.
## Guidance
APIs support batch get to retrieve a consistent set of resources.
- The method's name begin with `BatchGet`. The remainder of the method
name be the plural form of the resource being retrieved.
- The HTTP verb be `GET`.
- The HTTP URI end with `:batchGet`.
- The URI path represent the collection for the resource, matching the
collection used for simple CRUD operations. If the operation spans parents, a
[wilcard](./reading-across-collections) be accepted.
- There be a request body.
- If a GET request contains a body, the body be ignored, and **must
not** cause an error.
### Request
The request for a batch get method be specified with the following
pattern:
- The request include an array field which accepts the resource paths
specifying the resources to retrieve. The field be named `paths`.
- If no resource paths are provided, the API error with
`INVALID_ARGUMENT`.
- The parameter be required.
- The documentation for the parameter include the
[resource type](./paths) that it references.
- The parameter should define the pattern of the resource path values.
- The parameter should define the maximum number of paths allowed.
- Batch Get support pagination because transactionality across
API calls would be extremely difficult to implement or enforce, and the
request defines the exact scope of the response anyway.
- The request contain any other required parameters, and **should
not** contain other optional parameters except those described in this or
another AEP.
### Response
The response for a batch get method be specified with the following
pattern:
- The response schema match the method name, with `Response` suffix.
- The response schema have a single array property where each item is
either a retrieved resource or an error structure describing an error that
occurred attempting to retrieve the resource. The error alternative would
only be present for non-transactional batch gets.
- The order of resources/error objects in the response be the same as
the paths in the request.
- The array of resources be named `results` and contain resources with
no additional wrapping.
- There be a `nextPageToken` field as batch get operations are not
pageable.
### Support for transactional get
The batch get method may support a transactional form of operation where the
get either succeeds for all requested resources or fails. When supported, the
method must define a boolean parameter `transactional` that the user must
specify with the value `true` to request a transactional operation.
## Changelog
- **2024-04-22:** Adopt from Google AIP https://google.aip.dev/231 with the
following changes:
- Dropped the "nested requests" pattern.
- Changed the response schema to an object with `results` array property.
- Made transactional operation optional and controlled by asTransaction
parameter.
[aep-122-paths]: /122.md#fields-representing-resource-paths
[aep-122-parent]: /122.md#fields-representing-a-resources-parent
[get-request]: /131.md#get-request
---
# AEP-233 Batch methods- Create
---
# AEP-234 Batch methods- Update
---
# AEP-235 Batch methods- Delete
---
# AEP-300 AEP Editions
This AEP describes how the AEP specification is versioned.
## Edition-based versioning
AEPs will be published in editions, similar but not identical to specifications
such as those for the C++ or Rust programming languages:
- There are two types of editions: "stable" editions which do not change, and
"preview" editions which represent any edition not yet stablized.
- The naming of a stable edition will be based on the year in which it is
published (e.g. `aep-2025`).
- The specification will not change for a stable edition after it is published.
- Upon promoting a preview edition to a stable edition, the next preview
edition will be created (e.g. `aep-2027-preview`). This preview edition will
accumulate changes until it is promoted.
- AEP editions **MAY** have patch version updates which adhere to semantic
versioning (e.g. `aep-2025.25`).
- The patch version will not make any [breaking changes](#breaking-changes).
For example, fixing typos or clarifying existing guidance.
- The patch version will only contain fixes to typos in, and clarification
on, existing guidance.
- Editions will be published every 2 years.
## Breaking changes
A breaking change is a change to the specification that would result in one of
the following:
- A change which could cause a previously compliant API to become
non-compliant.
- A change which would break client functionality that was leveraging guidance
explicitly stated in the specification.
A new AEP edition have breaking changes.
The AEP specification will generally strive to minimize breaking changes, even
across editions.
## First-party clients and tooling
_NOTE_: This proposal only applies to AEP first-party clients. Third party open
source projects or organizations are governed outside of this project, and may
have their own guarantees.
### Client versioning
- Clients _SHOULD_ adhere to [Semantic Versioning 2.0.0](https://semver.org/)
### Client and edition compatibilty
The newest major version of clients and tools **MUST** be compatible with, at
minimum, the 3 latest stable AEP editions.
The following guidance applies to the most recent major versions of clients:
- Clients may provide different support guarantees for older major versions.
- Each major version will state which AEP editions are supported by those
clients.
- This guidance does not apply to accepting PRs on older major versions: those
may be accepted for any maintained branch of the project, at the discretion
of the client maintainers.
- Clients and tools may support features in preview editions, but support for
preview edition features in all clients and tools is not guaranteed.
## Examples of AEP specification changes
The following are examples of specification changes:
- renaming a field (e.g. `name` to `path`)
- changing the syntax for a filter or query language
- updating versioning guidance
- adding a new standard method
- removing guidance
- updating a design pattern (e.g. singletons or revisions)
- adding a new design pattern
## Rationale
### Editions and breaking changes
After enumerating all the possible types of changes to the specification, it
was determined that, to either some API publisher or client, any change to the
specification would be breaking some use case.
Some cases, such as renaming a standard field, were straightforward. Others,
like adding a new design pattern, could conflict with the design of an existing
API, therefore possibly invalidating the API compatibility with a
specification.
Removing guidance could also result in breaking a client which relied upon that
guidance existing (for example, an API that may treat revisions as a
first-class concept for rollback, aliasing, and rolling forward).
Therefore, rather than attempt to scope the breaking changes, all changes of
any kind are left until the next edition.
### Why an edition-based scheme
The AEPs have two goals that are difficult to achieve in concert:
1. Providing a set of modern best practices for remote APIs.
2. Providing a stable ecosystem of tooling that organizations can adopt for
their use.
This is due to the need to constantly evolve the best practices, which may
contradict older best practices and therefore result in a breaking change.
These breaking changes can be difficult for both services producing these APIs
to adopt, as well as complicate clients with multiple different code paths to
handle these different versions of clients.
An edition-based system will help provide clear expectations around the cadence
in which breaking changes could be introduced, as well as act as an anchor on
which other durations could be based (for example, support for a number of
editions in major versions of clients).
### Why clients are versioned separately
Although clients are expected to support recent AEP editions and could have a
similar versioning scheme, clients may also need to introduce breaking changes
for a variety of reasons unrelated to a new AEP edition, including:
1. An interface change in the client itself.
2. A change to support a new integration or interface (for example, supporting
a new version of the Terraform SDK, or a major version of the MCP server
protocol).
This necessitates the ability to express these changes to consumers. As such,
decoupling the client version from the AEP editions is a critical requirement.