Generating Code from OpenAPI Schemas

Monday, May 4, 2020

The idea of contract-first development with OpenAPI pops up every now and then with the team. The idea is pretty simple on paper: write an OpenAPI YAML/JSON file before you touch a line of code. This file declares the APIs and Model data that will be expected by your APIs. Others can consume this file as guidance on how to interact with your APIs. Sounds good, right?

After the schema is written, code must be manifested that is an implementation of that contract. How you could, potentially, generate that code is today’s topic. We’ll be using the Java/Spring Boot version, since I’ve been using that stack a lot lately for work, but some of these concepts apply to other language generators.

First - the Kitchen Sink

The openapi-generator project is a tool meant to generate code from said schema files. In theory, if you already have an OpenAPI schema or you are totally invested in this contract-first idea, you can let this tool generate API and Model classes for you, to use as a basis for your implementation. However, I ran into a couple gotchas, rough documentation points, and a healthy dose of frustration trying to get code generated in a sane (for me) fashion. At the time of this writing, the openapi-generator’s spring generator creates an entire Spring Boot project: code, build files, extra utilities, and a main class, from the given schema. I already had a project in place and just wanted to generate the API and Model code.

To demonstrate this, say you have the following OpenAPI file. TL;DR: a User model and an HTTP POST API to create new users.

openapi: 3.0.0
info:
  version: 1.0.0
  title: User APIs
  description: User Management APIs
tags:
  - name: user
    description: Operations about user
paths:
  /user:
    post:
      tags:
        - user
      summary: Create user
      description: This can only be done by the logged in user.
      operationId: createUser
      requestBody:
        content:
          application/json:
            schema:
              $ref: "#/components/schemas/User"
        description: Created user object
        required: true
      responses:
        default:
          description: successful operation
components:
  schemas:
    User:
      title: a User
      description: A User in the system
      type: object
      properties:
        id:
          type: integer
          format: int64
        username:
          type: string
        firstName:
          type: string
        lastName:
          type: string
        email:
          type: string
        password:
          type: string

If we run this through the Spring generator (I used the OpenAPI Gradle plugin) with the following configuration in our build.gradle:

plugins {
    id 'org.openapi.generator' version '4.3.0'
    id 'java'
}

openApiGenerate {
    generatorName = "spring"
    inputSpec = "$rootDir/api/user.yml"
    outputDir = "$buildDir/generated"
    apiPackage = "io.medgelabs.api.user"
    invokerPackage = "io.medgelabs.api.user"
    modelPackage = "io.medgelabs.api.user"
    configOptions = [
        dateLibrary: "java8"
    ]
}

We’d get a pom.xml with the relevant Spring & Swagger dependencies, generated Model and API files, a couple utility code files, and a Main class that can be run as a Spring Boot app via gradle bootRun.

The User model the spring generator generates is a standard POJO with Jackson annotations:

@Generated(...)
@ApiModel(description = "A User in the system")
public class User {
  @JsonProperty("id")
  private Long id;

  @JsonProperty("username")
  private String username;

  @JsonProperty("firstName")
  private String firstName;

  @JsonProperty("lastName")
  private String lastName;

  @JsonProperty("email")
  private String email;

  @JsonProperty("password")
  private String password;


  // getters, setters, equals, hashCode, and toString omitted for brevity
}

But the APIs, strangely, are generated as interfaces with default methods:

/**
 * NOTE: This class is auto generated by OpenAPI Generator (https://openapi-generator.tech) (4.3.0).
 * https://openapi-generator.tech Do not edit the class manually.
 */
package io.medgelabs.api.user;

import io.swagger.annotations.*;
import java.util.Optional;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.context.request.NativeWebRequest;

@Api(value = "user", description = "the user API")
public interface UserApi {

  default Optional<NativeWebRequest> getRequest() {
    return Optional.empty();
  }

  /**
   * POST /user : Create user This can only be done by the logged in user.
   *
   * @param user Created user object (required)
   * @return successful operation (status code 200)
   */
  @ApiOperation(
      value = "Create user",
      nickname = "createUser",
      notes = "This can only be done by the logged in user.",
      tags = {
        "user",
      })
  @ApiResponses(value = {@ApiResponse(code = 200, message = "successful operation")})
  @RequestMapping(
      value = "/user",
      consumes = {"application/json"},
      method = RequestMethod.POST)
  default ResponseEntity<Void> createUser(
      @ApiParam(value = "Created user object", required = true) @RequestBody User user) {
    return new ResponseEntity<>(HttpStatus.NOT_IMPLEMENTED);
  }
}

You can, hopefully, see why a lot of people pushed for YAML first. That’s a lot of annotating! And then a Controller is generated which implements the API interface:

package io.medgelabs.api.user;

import java.util.Optional;
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.context.request.NativeWebRequest;

@Controller
@RequestMapping("${openapi.userAPIs.base-path:}")
public class UserApiController implements UserApi {

  private final NativeWebRequest request;

  @org.springframework.beans.factory.annotation.Autowired
  public UserApiController(NativeWebRequest request) {
    this.request = request;
  }

  @Override
  public Optional<NativeWebRequest> getRequest() {
    return Optional.ofNullable(request);
  }
}

I believe the intent, here, is to create a separated layer between all those annotations and your business logic. I’m not much a fan of this but your mileage may vary!

Overall this isn’t a terrible start, and is certainly convenient if you already had a spec you were expected to implement, but we still have the whole kitchen sink where we really just needed a bowl.

Just Generate the APIs / Models

Getting rid of the metaphorical sink in favor of just API and Model code requires a bit of configuration. Note that some of these configuration options are for the spring generator. Other generators may require slightly different configuration.

To get rid of the entire project structure, we need to add a pair of systemProperties keys to our build.gradle:

openApiGenerate {
  ...
  systemProperties =[
    apis: '',
    models: '',
  ]
}

This tweaks the generator to only generate API code and Model code files found in your YAML spec. It gets rid of all the extra project files, supporting code, and Main class.

With the existing configuration the code is being generated in the target/generated/ folder. I’m not a fan of hidden source files, so let’s change the generator to output to our src/ folder directly:

openApiGenerate {
    ...
    outputDir = "$rootDir/"
    apiPackage = "io.medgelabs.api.user"
    modelPackage = "io.medgelabs.api.user"
    ...
}

The code will now be output to the src/main/java/io/medgelabs/api/user package. I prefer domain-centric packaging so I generate APIs and Models in the same package. Note that we don’t specify src/main/java in the outputDir key because the generator does that by default. The apiPackage and modelPackage keys control the subfolders, under src/main/java, that their respective code files will reside.

I, also, don’t personally care for the @Generated annotations that OpenAPI adds to the generated code as they cause a lot of commit noise in Git. We can remove them by adding the following configuration:

openApiGenerate {
    ...
    configOptions =[
        hideGenerationTimestamp: 'true', // remove @Generated annotations
    ]
    ...
}

In total, that gives us the following configuration, including the dependencies needed:

plugins {
    id 'org.openapi.generator' version '4.3.0'
    id 'java'
}

// Dependencies required for OpenAPI-generated classes
dependencies {
    implementation 'io.springfox:springfox-swagger2:2.8.0'
    implementation 'org.openapitools:jackson-databind-nullable:0.1.0'
}

openApiGenerate {
    generatorName = 'spring'
    inputSpec = "$rootDir/apis/user.yml"
    outputDir = "$rootDir/"
    apiPackage = "io.medgelabs.api.user"
    modelPackage = "io.medgelabs.api.user"
    generateAliasAsModel = false
    configOptions =[
        hideGenerationTimestamp: 'true', // remove @Generated annotations
        java8: 'true',
        dateLibrary: 'java8',
    ]
    // Only generates apis and models. Everything else is omitted
    systemProperties =[
        apis: '',
        models: '',
    ]
}

How to Support Multiple Files?

This is all well and good but I imagine most of you will quickly ask “what if I have more than one file?". The documentation on how to do multiple generation steps is fairly quick to find scrolling through the README. However, their example shows how to do different generators for the same schema file. I needed the same configuration for different schema files.

Turns out, after a lot of googling, one possible solution comes from a bit of Groovy code. We have to create a new task of the org.openapitools.generator.gradle.plugin.tasks.GenerateTask class for each schema file we have. We could do this in a hard-coded fashion, but I wanted a bit more automation.

First, we need an aggregate task which will run the GenerateTask step for each of our YAML files:

task genApis {
  finalizedBy 'spotlessApply'
}

The finalizedBy step is not required, but I found it useful to run my code formatter, Spotless, immediately after code generation to format everything properly.

Next, we need a convention for where our schemas live. I chose to put all schema files in a folder called apis/. Doing so allows us to have Gradle iterate through that folder and create a GenerateTask, automatically, for each schema found. The code looks like this for Gradle 6.3:

// Create a API Generation task for each YAML file in the subproject's /apis/ folder
fileTree(dir: "$rootDir/apis/", include: '**/*.yml').each {
    def apiName = it.getName().replace('.yml', '')
    def taskName = 'openApiGenerate' + apiName.capitalize()
    tasks.genApis.dependsOn taskName

    tasks.create(taskName, org.openapitools.generator.gradle.plugin.tasks.GenerateTask.class, {
        generatorName = 'spring'
        inputSpec = "$projectDir/apis/".toString() + "${apiName}.yml"
        outputDir = "$projectDir/" // Generates in the project's src/main/java folder directly
        apiPackage = "io.medgelabs.api.${apiName}"
        modelPackage = "io.medgelabs.api.${apiName}"
        generateAliasAsModel = false
        configOptions =[
            hideGenerationTimestamp: 'true', // remove @Generated annotations
            additionalModelTypeAnnotations: '@lombok.Data',
            java8: 'true',
            dateLibrary: 'java8',
        ]
        // Only generates apis and models. Everything else is omitted
        systemProperties =[
            apis: '',
            models: '',
        ]
    })
}

fileTree() is a super-handy function for traversing directories in a sane way. We can use this to grab all our YAML files from the apis/ directory. We use the name of the file to name the task. I prefer short, single-word names if possible which makes this cleaner. Finally, the configuration from above has been altered slightly to make use of the loop variables to create the appropriate configuration.

Now, if we run gradle genApis, we should see our code within the api package!

Customizing the Generated Code

Just a quick closing remark - you can customize the code this Spring generator creates! It uses mustache template files which you can override.

First, add a templateDir key to your Generate task:

openApiGenerate {
    ...
    templateDir = "$rootDir/generator-templates" // Customizations to spring generator
    ...
}

I placed my customizations in a folder called generator-templates/ in the root of the project.

To see what templates you can use, take a look at the generator’s code repository. For the spring generator that is found here. As a fun example, I decided I wanted Lombok models instead of the typical Java POJOs. So I copied the pojo.mustache file from that repository into my generator-templates/ directory. Then, I altered the content to the following:

/**
 * {{#description}}{{.}}{{/description}}{{^description}}{{classname}}{{/description}}
 */{{#description}}
@lombok.Data
@ApiModel(description = "{{{description}}}"){{/description}}
public class {{classname}} {{#parent}}extends {{{parent}}}{{/parent}}{{/parent}} {{#serializableModel}}implements Serializable{{/serializableModel}} {
{{#serializableModel}}
  private static final long serialVersionUID = 1L;
{{/serializableModel}}

  {{#vars}}
    {{#isEnum}}
    {{^isContainer}}
{{>enumClass}}
    {{/isContainer}}
    {{#isContainer}}
    {{#mostInnerItems}}
{{>enumClass}}
    {{/mostInnerItems}}
    {{/isContainer}}
    {{/isEnum}}
  {{#jackson}}
  @JsonProperty("{{baseName}}")
  {{/jackson}}
  {{#gson}}
  @SerializedName("{{baseName}}")
  {{/gson}}
  {{#isContainer}}
  {{#useBeanValidation}}@Valid{{/useBeanValidation}}
  private {{>nullableDataType}} {{name}} = {{#isNullable}}JsonNullable.undefined(){{/isNullable}}{{^isNullable}}{{#required}}{{{defaultValue}}}{{/required}}{{^required}}null{{/required}}{{/isNullable}};
  {{/isContainer}}
  {{^isContainer}}
  private {{>nullableDataType}} {{name}}{{#isNullable}} = JsonNullable.undefined(){{/isNullable}}{{^isNullable}}{{#defaultValue}} = {{{.}}}{{/defaultValue}}{{/isNullable}};
  {{/isContainer}}

  {{/vars}}
}

There’s a lot going on there, If you compare this version to the version in the repository, you will find the getters, setters, equals, hashCode, and toString fragments removed. This is all handled by the @lombok.Data annotation I added at the top. I also removed a bunch of the XML & Date formatting items since I use JSON and the java.time packages.

That’s All For Now!

When all is said and done, this isn’t so bad of a tool! It just took a lot of googling to get here… You can go much farther with the other mustache templates and change the entire structure of the code generated. How much ROI you get from that journey is up to you.

Hope this helped! Thanks for coming by the Lab!