AEP Terraform Provider
A core thesis of the AEP project has been that API design constraints can facilitate the creation (and maintenance!) of API client tools. In many companies, teams are staffed to build out CLI tools, Terraform providers, UIs, LLM agents, and various other tools that act as thin layers on top of a company’s APIs. The AEP project aims to ensure that these tools can be easily generated on top of our API design, ensuring that companies can move faster with less overall effort. We’re excited to launch our first version of a fully auto-generated Terraform provider based off the AEPs.
What is Terraform?
Terraform is a powerful Infrastructure-as-Code (IaC) tool that allows you to define and manage your infrastructure using code. Infrastructure-as-Code, in general, is the practice of managing and provisioning infrastructure through machine-readable definition files rather than through manual processes. This means you can automate the creation, modification, and deletion of resources across various cloud providers and on-premises environments, ensuring consistency, repeatability, and version control.
Cloud companies often build out Terraform providers, which are responsible for establishing differences between the cloud and the user’s intent, determining which API requests are necessary, making those requests, and handling any follow-up actions.
How do the AEPs help with this?
Most companies must staff engineers just to maintain their Terraform provider. However, making AEP-compliant APIs can give you providers for no additional effort beyond writing the API! Let’s take a look at a pseudocode version of the Terraform lifecycle.
userIntention := GetTerraformPlan()currentState := ReadResource(userIntention)
if currentState != null: if user wishes to delete: DeleteResource(currentState) else if currentState != userIntention: UpdateResource(userIntention)else: CreateResource(userIntention)Two different sets of problems emerge and the AEPs are well-suited for solving both of them.
The first set of problems revolves around the grouping and calling of APIs. There’s a variety of tools in the market that will call an API method given a set of parameters and an OpenAPI description (or even just a URL). Terraform doesn’t expose individual API methods, but instead exposes a resource schema. This raises a variety of questions when developing a Terraform provider:
- What are my resources?
- How do I know which API methods to call for a given resource?
- Is there a single API method that can be called for each CRUD operation, or will I need to chain together API calls to achieve my result?
- How do I derive my resource schema from four distinct CRUD APIs? Is it simply the union of all the body parameters from the four CRUD API methods?
AEP APIs are resource-oriented by default, so just following the AEPs can address all of the problems above. The Terraform provider can “know” which API methods should be called and how they relate to each other.
What about API responses?
The second set of problems comes from reading API responses. In the pseudocode above, the Read response needs to be parsed and checked against the Terraform plan. This means that the Terraform provider needs to understand the relationship between the API response (beyond simple error checking) and the user’s intention. More questions emerge:
- Does the response of the Read API match the resource schema? What changes need to be made to match the Read API response to the potential resource schema?
- Are there multiple ways that the server will accept a single value? If so, how can I be sure that two semantically equivalent values do not result in a diff against the Terraform plan?
- If a boolean value does not appear in the JSON response, does that mean it’s
false? (And likewise for numeric values being0, string values being the empty string, and so on.)
Many of these questions come down to the preference of the API developer. In fact, some of them cannot be defined by an API schema! The AEPs encode these differences in our specification, so that tools like our Terraform provider can make assumptions about how to parse the responses of our APIs.
The AEP Terraform Provider
The AEP Terraform provider solves these problems. The provider is a template that can be pointed to any OpenAPI spec that features APIs complying with the AEPs. Since the APIs are resource-oriented and AEP-compliant, the provider is able to create a resource schema at runtime that perfectly matches the APIs. Did your team build a new feature? Just update your OpenAPI spec and the provider will add it without users having to update their providers.
Users are able to send over arbitrary headers so that the provider will work with any authentication solution you may have.
When users run terraform apply, the provider automatically knows how to
create Create, Read, Update, and Delete requests, handle authentication, and
return errors. More crucially, it understands how to handle the responses from
those requests. The provider understands what values to expect from the Get
requests and the proper way to handle difference checking against the user’s
intention.
What comes next?
While the Terraform provider lifecycle appears simple at first glance, the details contain a deceptive amount of detail. This is detail that can only be learned through trial-and-error, which is difficult when APIs are often only designed once.
By building out the Terraform provider, we’re even more convinced than ever that the AEP specification can prove the thesis that client tooling can be more easily built on top of consistent APIs.