Proposal for links in OpenAPI v4

moonwalk-links.md

This file describes a proposal for a new design of "links" support for OpenAPI v4 (Moonwalk)

Problems to be solved

There are three key problems with the OpenAPI v3 support for links that this proposal hopes to address:

No support for linking from a full URL

There are some occasions where a response header or body parameter contains a complete URL, e.g. the "Location" header returned in a 201 response from POST, or the "nextLink" property in the response of a pageable OData list operation. OpenAPI 3 links do not support this scenario.

No support for linking to a (possibly inner) request body field

OpenAPI 3 links cannot specify a field within the request body as the target of a link -- only individual parameters or the entire request body.

No support for backward links

OpenAPI 3 only defines "forward" links -- from source to target. Links are defined in the 'response' object and specify how data in the response maps to parameters in a subsequent request. There is no provision for a parameter to point back to a prior operation as the source of data for the parameter.

While it is possible to specify all links in a "forward" fashion, this is cumbersome since data from a single operation could be the source for many subsequent operations. For example, the "id" returned in a POST response could supply the value for the "id" path parameter of a GET, PATCH, and DELETE for the same resource and for any child resources. If the path parameter could "point back" to the post response, this could be specified just once, in a parameter defined in components.parameters, which would be far more concise.

The Proposal

Overview

The Link Object will be extended to support links from fields that contain URLs and links to specific request body properties.

To support "backward links", a links field will be added to the Operation Object to specify operations that can link to this operation, and two new fields will be added to the Link Object to specify the source operationId or source operationRef.

The Link Object

The Link Object of OpenAPI v3 will be modfied/extended as follows:

• A new url field will be added
• The requestBody field will be redefined to support links to fields within the request body
• New fields sourceId and sourceRef will be added to identify the source operation

The url field

A url field will be added to the Link Object.

Field Name	Type	Description
url	Any | {expression}	A constant or an expression to be evaluated and passed to the linked operation. The value should be a URL for the linked operation. Applications may extract values for path or query parameters present in this URL to use as values for these parameters in the linked operation.

The requestBody field

A requestBody field of the Link Object will be redefined as follows.

Field Name	Type	Description
requestBody	Map[string, Any | {expression}]	A map representing values to pass to an operation, as specified with operationId or identified via operationRef, for use in the request body. The key is the JSON path of a request body property in the linked operation, and the value can be a constant or an expression to be evaluated and passed to the linked operation.

Note that this new structure is a generalization of the requestBody field in OpenAPI v3, since "/" can be used to specify the entire request body of the linked operation.

The sourceId and sourceRef fields

Two new fields, sourceId and sourceRef will be added to the Link Object to identify the operation that is the source of information for a parameter or request body.

Field Name	Type	Description
sourceRef	string	A relative or absolute URI reference to an OAS operation. This field is mutually exclusive of the sourceId field, and MUST point to an Operation Object. Relative operationRef values MAY be used to locate an existing Operation Object in the OpenAPI definition.
sourceId	string	The name of an existing, resolvable OAS operation, as defined with a unique operationId. This field is mutually exclusive of the sourceRef field.

The sourceRef and sourceId fields identify the operation that is the source of the linked data. A Link Object may contain both a sourceRef or sourceId AND an operationRef or operationId, which allows the same Link Object to be referenced from both a Response Object and from Parameter or Request Body Objects.

The Operation Object

A links property will be added to the Operation Object with the following definition.

Field Name	Type	Description
links	Map[string, Link Object | Reference Object]	A map of operations links that can be the source of values for this operation's parameters and/or request body properties. The key of the map is a short name for the link, following the naming constraints of the names for Component Objects.

Compatability

The new links design is not compatible with the OpenAPI v3 links structure but it is expected that tooling could translate an OpenAPI v3 definition into OpenAPI v4 with little (ideally no) loss of information.

Examples

Links from URLs

The Location response header

paths:
  /users:
    post:
      responses:
        '201':
          description: Created
          headers:
            Location:
              description: The URL of the User that was created
              type: string
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/User'
          links:
            location:
              operationId: getUser
              # The URL in the location header contains the value for id path parameter
              url: $response.header.location
  /users/{id}:
    get:
      operationId: getUser
      parameters:
      - name: id
        in: path
        required: true
        schema:
          type: string

A nextLink field

paths:
  /users:
    get:
      operationId: listUsers
      responses:
        '200':
          description: Success
          content:
            application/json:
              schema:
                type: object
                properties:
                  value:
                    type: array
                    items:
                      $ref: '#/components/schemas/User'
                  nextLink:
                    description: A URL to retrieve the next page of users
                    type: string
          links:
            nextLink:
              operationId: listUsers
              # The URL in the location header contains the value for id path parameter
              url: $response.body#/nextLink

Links to response body properties

The follow example is a simplification of the real-world "ConfigurationStores_RegenerateKey" operation of the Azure App Configuration resource-manager service.

paths: 
  /listKeys:
    post:
      description: Lists the access keys for the service.
      operationId: ListKeys
      responses:
        '200':
          description: Success
          content:
            application/json:
              schema:
                description: The result of a request to list API keys.
                type: object
                properties:
                  value:
                    type: array
                    items:
                      description: An API key used for authenticating with the service.
                      type: object
                      properties:
                        id:
                          description: The key ID.
                          type: string
                        name:
                          description: A name for the key describing its usage.
                          type: string
          links:
            keyId:
              operationId: RegenerateKey
              # The id field of the first key can be set as id property of the request body of RegenerateKey
              requestBody:
                '/id': '$.response/body#value[0].id'

  /regenerateKey:
    post:
      description: Regenerates an access key for the service.
      operationId: RegenerateKey
      requestBody:
        content:
          application/json:
            schema:
              description: The parameters used to regenerate an API key.
              type: object
              properties:
                id:
                  description: The id of the key to regenerate.
                  type: string
        description: The parameters for regenerating an access key.
        required: true
      responses:
        '200':
          description: Success
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ApiKey'      

Alternatives Considered

Backlink from a Parameter Object or Request Body Object

Rather than add a links field to the operation, we could add links to individual parameters and the response body, but this seems unnecessary, since the Link Object explicitly names the parameters and response body properties that it can supply values for.

One advantage of allowing a backlink on the parameter object is that it could be specified on a parameter definition in components.parameters and then used by any operation that references this parameter. With links specified on the operation, the link will need to be added on every consuming operation.

A futher consideration is that Parameter Objects may be going away in OpenAPI v4, in favor of parameters defined using JSON schema. This would require extending the OpenAPI dialect of JSON schema to add a links attribute. But we don't want to support links everywhere in JSON schema -- just for parameters (and maybe request bodies), so this might be tricky.

Eliminate the "short name" for links

The current structure for the links field in the OpenAPI v3 Response Object, and proposed here for the Operation Object, is a Map[string, Link Object \| Reference Object], where the keys of this map are the "short name for the link". This short name has no real purpose in the API definition and just adds verbosity. A better structure would be [Link Object \| Reference Object] -- the equivalent of the values of the current map structure.

I didn't make this change to minimize the changes from the current structure, but it is appealing (to me at least) and perhaps we should reconsider.

Not addressed

One case that is not addressed by this proposal is extracting paired items from a list in the response. A common case in Azure is a list operation for resources, where each entry in the list contains 1) an id field, and 2) an etag field. We'd like to be able to define a link that would say "You can call delete for a resource using the id and etag fields from any entry in the response of the list resources operation".

We should try to also address the "for any entry in the list" case, for example with a wildcard array index [*]:

          links:
            keyId:
              operationId: RegenerateKey
              # The id field of the first key can be set as id property of the request body of RegenerateKey
              requestBody:
                '/id': '$.response/body#value[*].id'

Extracting paired items could then be achieved by using both [*].id and [*].etag in a link object.

Typo: Compatability -> Compatibility

Thanks for the proposal, I will have a deeper look later.

This proposal also makes it much easier to document HATEOAS APIs. On OAI/OpenAPI-Specification#577, I said the following:

OAS with JSON Schema could describe:

• The starting path(s) with belonging requests/responses
• It should be possible to define a link truly as a link with:
  • Allowed HTTP methods
  • For each HTTP method the schema of the request/response

The paths that are used in the links should not be defined as paths in the OAS, as that would stimulate wrong behavior: just use these endpoints instead of following the links.

Your proposal here allows describing starting paths (a set of operations). The extended links refer to operations, which specify the allowed HTTP methods and schema of request and response. My bullet points are covered with that.

The issue that is not addressed is in the last sentence of the quote. A simple solution to that would be to annotate operations with some sort of flag indicating these operations should only be invoked through links returned from the server, never without that server guidance. Is that something that you can add to your proposal?

This way, we have a pragmatic way to define a mediatype, while retaining the dynamic hypermedia as the engine of application state.

I've never understood HATEOAS. The notion of "just follow this link" with no definition of its parameters, headers, authorization scheme, response schema, response headers, etc does not make sense to me. I'm probably misrepresenting the concept of HATEOAS there but that's just the way it seems to me.

So you are correct that my proposal does not address excluding the paths used in links from the API definition. I think they should be in the API definition so that all the details of the request are well defined.

The purist approach of HATEOAS is that you figure it all out at runtime. IMHO, that is not realistic.

Your approach allows defining links and the operations they relate to. In a HATEOAS API design, I would use that to define a single public GET operation that returns nothing but a bunch of links. That resource is what the client (e.g. a mobile app) would fetch. The app designers would read what possible links are returned. Then they would read the operations and know the possible subsequent requests. The links would generally be optional and the server will populate those links that are applicable in that actual state of the application.

Such an API design has two major benefits:

• The app does not need to keep track of the state to know whether a certain operation will be possible. If the operation is possible, its link is available, otherwise not. So, the defined operations are documented in the OAS spec, the subset of actually possible operations is indicated through link availability. E.g. a pageable resource will on the first page not have a previous link, but it will have a next link if multiple pages exist. The next page will have a previous link.
• The server can change URLs over time without the client being aware (e.g. add a query parameter and populate it from the server, in the the returned link). As long as the operations are the compatible, the app will continue to work.

So, my request to be able to mark operations as "use only through a link" is to guide app designers.

The notion of "just follow this link" with no definition of its parameters, headers, authorization scheme, response schema, response headers, etc does not make sense to me

For a REST API as described by Roy Fielding, the idea is that you'd return not only the link but also define how it is meant to be used in the same response where you gave out the link.

For instance, when returning text/html you might use a <form> to declare that one should do a POST. The client (browser) knows t the media type (text/html) and infers that it should make a POST with whatever input values might be in the said form.

For JSON, you'd technically start the client without reading the global API spec, and in theory the OpenAPI enabled server would return application/openapi+json or some other well defined media type that returns the part of the spec documenting the operation you are linking to.

In theory this could be used for some clever communication of the next possible state. E.g. regular users get told they can only view a resource via a GET, while admin users get told they can additionally DELETE it or replace it edit it via a PUT. Similar to how you only return such a button on an HTML server.

Full HATEOAS support would imply that the clients should be able to dynamically read OpenAPI spec bit by bit. I agree with Bert-R that, this is highly unrealistic and unexpected by your general OpenAPI client.

As such I'm not sure I see a reason to "hide" it from the original spec, given that we don't have the facility to return the spec at all in the ecosystem. HATEAOS clients (such as browsers) wouldn't read the spec in the first place.

This proposal is still really useful because it allows interoperability. E.g. you can make a common "Location" header returning endpoint which is usable via both a regular OpenAPI Client and also a web browser -- which is really nice.

@vaskozl @Bert-R Thanks for your comments. But sadly I think this proposal is dead. There is no agreement on what the current link object means in OpenAPI and the spec offers no guidance, so there is little appetitie in the TSC for fixing or improving a feature that is broken beyond repair.

It pains me to say this because I believe (as I think you do) that this could be an extremely useful feature. But I have given up trying to get others in TSC to find a path forward with this.