OpenAPI Terraform Provider that configures itself at runtime with the resources exposed by the service provider (defined in a swagger file)

Terraform Provider OpenAPI Build Status GoDoc GoReportCard CodeCov

This terraform provider aims to minimise as much as possible the efforts needed from service providers to create and maintain custom terraform providers. This provider uses terraform as the engine that will orchestrate and manage the cycle of the resources and depends on a swagger file (hosted on a remote endpoint) to successfully configure itself dynamically at runtime.

Powered by https://www.terraform.io

Following The OpenAPI Specification

What are the main pain points that this terraform provider tries to tackle?

  • As as service provider, you can focus on improving the service itself rather than the tooling around it.
  • Due to the dynamic nature of this terraform provider, the service provider can continue expanding the functionality of the different APIs by introducing new versions, and this terraform provider will be able to discover the new resource versions automatically without the need to add support for those as you would when mantining your own custom Terraform provider.
  • Find consistency across APIs provided by different teams encouraging the adoption of OpenAPI specification for describing, producing, consuming, and visualizing RESTful Web services.

Overview

API terraform provider is a powerful full-fledged terraform provider that is able to configure itself at runtime based on a Swagger specification file containing the definitions of the APIs exposed. The dynamic nature of this provider is what makes it very flexible and convenient for service providers as subsequent upgrades to their APIs will not require new compilations of this provider. The service provider APIs are discovered on the fly and therefore the service providers can focus on their services rather than the tooling around it.

Pre-requirements

  • The service provider hosts APIs compliant with OpenApi and swagger spec file is available via a discovery endpoint.

Requirements

  • Terraform v0.12.0 (to execute the terraform provider plugin)
  • Go 1.12.4 (to build the provider plugin)
    • This project uses go modules for dependency management
  • Docker 17.09.0-ce (to run service provider example)
  • Docker-compose 1.16.1 (to run service provider example)

How to use Terraform Provider OpenAPI

Things to know regarding custom terraform providers

  • Terraform expects third party providers to be manually installed in the '.terraform.d/plugins' sub-path in your user's home directory.
  • Terraform expects terraform provider names to follow a specific naming scheme. The naming scheme for plugins is terraform-<type>-NAME_vX.Y.Z, where type is either provider or provisioner.

More information about how terraform discovers third party terraform providers and naming conventions here.

OpenAPI Terraform provider installation

There are multiple ways how the OpenAPI Terraform provider can be installed. Please refer to the OpenAPI Terraform provider installation document to learn more about it.

OpenAPI Terraform provider in action

After having provisioned your environment with the OpenAPI Terraform provider you can now write Terraform configuration files using resources provided by the OpenAPI service. Refer to Using the OpenAPI Terraform Provider doc for more details.

Terraform provider documentation

You can generate the Terraform documentation automatically given an already Terraform compatible OpenAPI document using the The OpenAPI Terraform Documentation Renderer library. The OpenAPI document is the source of truth for both the OpenAPI Terraform provider as well as the user facing documentation.

References

Additionally, the following documents provide deep insight regarding OpenAPI and Terraform as well as frequently asked questions:

  • How to document contains information about how to define a swagger file following good practises that make it work seamlessly with this terraform provider. Additionally, learn more about what is currently supported.
  • Migrating to Terraform 0.12. This document describes how to update configuration created using Terraform v0.11 to v0.12.
  • FAQ document answers for the most frequently asked questions.

Contributing

Please follow the guidelines from:

References

  • go-swagger: Api terraform provider makes extensive use of this library which offers a very convenient implementation to serialize and deserialize swagger specifications.
  • JsonPath: Json path is used in the plugin external configuration file to define values for provider schema properties that are coming from external files.

Authors

  • Daniel I. Khan Ramiro

See also the list of contributors who participated in this project.

Owner
Daniel I. Khan Ramiro
Passionate software engineer, open source enthusiast, problem solver and always looking forward to learning new things. (twitter @Dani_Khan_)
Daniel I. Khan Ramiro
Comments
  • subdocuments: parent_id is required in all arrays elements

    subdocuments: parent_id is required in all arrays elements

    Describe the bug

    parent_id is required in all arrays elements of the subdocument

    To Reproduce

    Using the following swagger spec

    swagger: '2.0'
    paths:
      "/zone":
        post:
          description: ''
          parameters:
          - in: body
            name: body
            required: true
            schema:
              "$ref": "#/definitions/zone"
          responses:
            '200':
              description: OK
              schema:
                "$ref": "#/definitions/zone"
      "/zone/{zone_id}":
        get:
          description: ''
          parameters:
          - in: path
            name: zone_id
            required: true
            type: string
          responses:
            '200':
              schema:
                "$ref": "#/definitions/zone"
      "/zone/{zone_id}/recordset":
        get:
          responses:
            '200':
              description: OK
              schema:
                items:
                  "$ref": "#/definitions/zone_recordset"
                type: array
        post:
          description: ''
          parameters:
          - in: path
            name: zone2_id
            required: true
            type: string
          - in: body
            name: body
            required: true
            schema:
              "$ref": "#/definitions/zone_recordset"
          responses:
            '200':
              description: OK
              schema:
                "$ref": "#/definitions/zone_recordset"
      "/zone/{zone_id}/recordset/{id}":
        get:
          description: ''
          parameters:
          - in: path
            name: zone_id
            required: true
            type: string
          - in: path
            name: id
            required: true
            type: string
          responses:
            '200':
              schema:
                "$ref": "#/definitions/zone_recordset"
    
    definitions:
      zone:
        type: object
        required:
          - name
        properties:
          id:
            type: string
            readOnly: true
          name:
            type: string
      zone_recordset:
        type: object
        properties:
          id:
            type: string
            readOnly: true
          zone_id:
            type: string
            readOnly: true
          record:
            type: array
            items:
              type: object
              properties:
                content:
                  type: string
    

    Terraform

    resource "demo_zone" "test" {
        name = "test.com"
    }
    
    resource "demo_zone_recordset" "test" {
        zone_id = "${demo_zone.test.id}"
    
        record {
            # parent_id is required in all subdocuments arrays
            # zone_id = "${demo_zone.test.id}"
            content = "test"
        }
    }
    

    Terraform plan

    $ terraform plan
    
    Error Missing required argument
    
      on main.tf line 8, in resource "demo_zone_recordset" "test":
      8:     record {
    
    The argument "zone_id" is required, but no definition was found.
    

    Expected behaviour

    parent_id shouldn't be required.

    Checklist

    Don't forget to go through the checklist to make sure the issue is created properly:

    • [x] I have added a corresponding label (feature request/bug/etc) to the issue (right side menu)
    • [x] I have added this issue to the 'API Terraform Provider' GitHub project (right side menu)
  • "error = 'openapi' not found in provider's services configuration"

    Describe the bug

    Attempting to run plan against sample goa environment.

    A clear and concise description on how does it work now and what's the current behaviour.

    Steps to reproduce the behavior:

    1. make local-env
    2. export PROVIDER_NAME=goa && curl -fsSL https://raw.githubusercontent.com/dikhan/terraform-provider-openapi/master/scripts/install.sh | bash -s -- --provider-name $PROVIDER_NAME
    3. create main.tf using example
    4. terraform init && terraform plan

    I also had to set the url using the terraform-provider-openapi.yaml as setting the OTF_VAR_goa_SWAGGER_URL env variable had no e ffect and only errored saying the OTF_VAR_goa_SWAGGER_URL wasn't set.

    Expected behaviour

    A completed plan without error.

    Additional context

    2018-10-06T20:29:14.448Z [DEBUG] plugin: starting plugin: path=/home/vagrant/.terraform.d/plugins/terraform-provider-goa args=[/home/va
    grant/.terraform.d/plugins/terraform-provider-goa]
    2018-10-06T20:29:14.451Z [DEBUG] plugin: waiting for RPC address: path=/home/vagrant/.terraform.d/plugins/terraform-provider-goa
    2018-10-06T20:29:14.542Z [DEBUG] plugin.terraform-provider-goa: plugin address: timestamp=2018-10-06T20:29:14.541Z network=unix address
    =/tmp/plugin059454931
    2018-10-06T20:29:14.543Z [DEBUG] plugin.terraform-provider-goa: 2018/10/06 20:29:14 [INFO] found open api plugin configuration at /home/vagrant/.terraform.d/plugins/terraform-provider-openapi.yaml
    2018-10-06T20:29:14.544Z [DEBUG] plugin.terraform-provider-goa: 2018/10/06 20:29:14 [ERROR] There was an error initialising the terraform provider: plugin init error: error occurred when getting service configuration from plugin configuration file terraform-provider-ope
    napi.yaml - error = 'openapi' not found in provider's services configuration
    2018/10/06 20:29:14 [ERROR] root: eval: *terraform.EvalInitProvider, err: unexpected EOF
    2018/10/06 20:29:14 [ERROR] root: eval: *terraform.EvalSequence, err: unexpected EOF
    2018/10/06 20:29:14 [TRACE] [walkInput] Exiting eval tree: provider.goa
    2018/10/06 20:29:14 [TRACE] dag/walk: upstream errored, not walking "goa_bottles.my_bottle"
    2018/10/06 20:29:14 [TRACE] dag/walk: upstream errored, not walking "provider.goa (close)"
    2018/10/06 20:29:14 [TRACE] dag/walk: upstream errored, not walking "meta.count-boundary (count boundary fixup)"
    2018/10/06 20:29:14 [TRACE] dag/walk: upstream errored, not walking "root"
    
    2018/10/06 20:29:14 [DEBUG] plugin: waiting for all plugin processes to complete...
    2018-10-06T20:29:14.547Z [WARN ] plugin: error closing client during Kill: err="connection is shut down"
    Error: Error asking for user input: 1 error(s) occurred:
    
    * provider.goa: unexpected EOF
    
    
  • OpenAPI provider not working

    OpenAPI provider not working

    Describe the bug

    When attempting to use the OpenAPI provider on Windows system, terraform does not identify it.

    To Reproduce

    Steps to reproduce the behavior:

    1. On Windows 10, install the the openapi executable in both:
      • %APPDATA%\terraform.d\plugins
      • C:\hashicorp\ (location of terraform)
    2. Configure terraform-provider-openapi.yaml
      • change the services to antserver
    3. Copy the openapi executeable to terraform-provider-antserver.exe
    4. Verify provider works from code directory PS C:\Users.....> terraform-provider-antserver.exe 2020/12/12 06:07:06 Running OpenAPI Terraform Provider v0.31.1-d28a81d857163a2dd72072634f10eb3977bb3584; Released on: 2020-07-20T23:37:07Z 2020/12/12 06:07:06 [INFO] found open api plugin configuration at C:\Users\redacted\AppData\terraform.d\plugins/terraform-provider-openapi.yaml 2020/12/12 06:07:06 [DEBUG] serviceConfig = &{SwaggerURL:https://redacted.antsle.us/swagger.json PluginVersion: InsecureSkipVerify:false SchemaConfigurationV1:[] TelemetryConfig:} 2020/12/12 06:07:06 [INFO] Provider antserver is using the following swagger file: https://redacted.antsle.us/swagger.json 2020/12/12 06:07:06 [DEBUG] service configuration = &{SwaggerURL:https://redacted.antsle.us/swagger.json PluginVersion: InsecureSkipVerify:false SchemaConfigurationV1:[] TelemetryConfig:} 2020/12/12 06:07:06 [DEBUG] validating end point terraform compatibility /api/system/setup/send-esn-token 2020/12/12 06:07:06 [DEBUG] resource path '/api/system/setup/send-esn-token' not terraform compliant: path '/api/system/setup/send-esn-token' is not a resource instance path 2020/12/12 06:07:06 [DEBUG] validating end point terraform compatibility /api/network/vlans/{vlan-name}/antlets 2020/12/12 06:07:06 [DEBUG] resource path '/api/network/vlans/{vlan-name}/antlets' not terraform compliant: path '/api/network/vlans/{vlan-name}/antlets' is not a resource instance path 2020/12/12 06:07:06 [DEBUG] validating end point terraform compatibility /api/ads/config 2020/12/12 06:07:06 [DEBUG] resource path '/api/ads/config' not terraform compliant: path '/api/ads/config' is not a resource instance path 2020/12/12 06:07:06 [DEBUG] validating end point terraform compatibility /api/antlets/{antlet-name}/state 2020/12/12 06:07:06 [DEBUG] resource path '/api/antlets/{antlet-name}/state' not terraform compliant: path '/api/antlets/{antlet-name}/state' is not a resource instance path 2020/12/12 06:07:06 [DEBUG] validating end point terraform compatibility /api/network/vlans/connect 2020/12/12 06:07:06 [DEBUG] resource path '/api/network/vlans/connect' not terraform compliant: path '/api/network/vlans/connect' is not a resource instance path 2020/12/12 06:07:06 [DEBUG] validating end point terraform compatibility /api/backups/delete-local-copy 2020/12/12 06:07:06 [DEBUG] resource path '/api/backups/delete-local-copy' not terraform compliant: path '/api/backups/delete-local-copy' is not a resource instance path 2020/12/12 06:07:06 [DEBUG] validating end point terraform compatibility /api/list-zpools 2020/12/12 06:07:06 [DEBUG] resource path '/api/list-zpools' not terraform compliant: path '/api/list-zpools' is not a resource instance path 2020/12/12 06:07:06 [DEBUG] validating end point terraform compatibility /api/system/upgrade-edgelinux 2020/12/12 06:07:06 [DEBUG] resource path '/api/system/upgrade-edgelinux' not terraform compliant: path '/api/system/upgrade-edgelinux' is not a resource instance path 2020/12/12 06:07:06 [DEBUG] validating end point terraform compatibility /api/templates-statuses 2020/12/12 06:07:06 [DEBUG] resource path '/api/templates-statuses' not terraform compliant: path '/api/templates-statuses' is not a resource instance path 2020/12/12 06:07:06 [DEBUG] validating end point terraform compatibility /api/backups/import-local-backup 2020/12/12 06:07:06 [DEBUG] resource path '/api/backups/import-local-backup' not terraform compliant: path '/api/backups/import-local-backup' is not a resource instance path 2020/12/12 06:07:06 [DEBUG] validating end point terraform compatibility /api/ads/drive/{name} 2020/12/12 06:07:06 [DEBUG] resource path '/api/ads/drive/{name}' not terraform compliant: resource instance path '/api/ads/drive/{name}' missing required GET operation 2020/12/12 06:07:06 [DEBUG] validating end point terraform compatibility /api/antlets/{antlet-name}/vusbs/enable 2020/12/12 06:07:06 [DEBUG] resource path '/api/antlets/{antlet-name}/vusbs/enable' not terraform compliant: path '/api/antlets/{antlet-name}/vusbs/enable' is not a resource instance path 2020/12/12 06:07:06 [DEBUG] validating end point terraform compatibility /api/backups/schedule-backups 2020/12/12 06:07:06 [DEBUG] resource path '/api/backups/schedule-backups' not terraform compliant: path '/api/backups/schedule-backups' is not a resource instance path 2020/12/12 06:07:06 [DEBUG] validating end point terraform compatibility /api/antlets/{antlet-name} 2020/12/12 06:07:06 [DEBUG] resource '/api/antlets/{antlet-name}' root path match: [/api/antlets/{antlet-name} /api/antlets/] 2020/12/12 06:07:06 [DEBUG] found resource root path without trailing '/' - /api/antlets ..........
    5. Initialize the terraform project terraform.exe init

    Initializing the backend...

    Initializing provider plugins...

    • Finding latest version of hashicorp/antserver...

    Error: Failed to query available provider packages

    Could not retrieve the list of available versions for provider hashicorp/antserver: provider registry registry.terraform.io does not have a provider named registry.terraform.io/hashicorp/antserver

    Expected behaviour

    When using terraform init the provider would be picked up and used. I also attempted $env:OTF_VAR_ANTSLE_SWAGGER_URL="https://.antsle.us/swagger.json"

    Checklist (for admin only)

    Don't forget to go through the checklist to make sure the issue is created properly:

    • [ ] I have added a corresponding label (bug) to the issue (right side menu)
    • [ ] I have added this issue to the 'API Terraform Provider' GitHub project (right side menu)
  • Error when attempting to use kubernetes api

    Error when attempting to use kubernetes api

    Please check one of the options below and follow the instructions:

    I'm trying use the provider to make api calls to kubernetes. The swagger is here https://raw.githubusercontent.com/kubernetes/kubernetes/master/api/openapi-spec/swagger.json

    My config files looks like this

    version: '1'
    services:
      openapi:
        swagger-url: https://raw.githubusercontent.com/kubernetes/kubernetes/master/api/openapi-spec/swagger.json
        insecure_skip_verify: true
    

    There are other things wrong with this file but at the moment I'm trying to get pass a swagger parsing problem.

    I'm getting these errors when running tf plan

    2018-08-30T01:44:54.269Z [DEBUG] plugin.terraform-provider-k8s: 2018/08/30 01:44:54 [INFO] Provider openapi is using the following remote swagger URL: https://raw.githubusercontent.com/kubernetes/kubernetes/master/api/openapi-spec/swagger.json
    2018-08-30T01:44:55.780Z [DEBUG] plugin.terraform-provider-k8s: 2018/08/30 01:44:55 [DEBUG] findMatchingResourceRoot result - [/api/v1/nodes/{name}/proxy/{path} /api/v1/nodes/{name}/proxy/]
    2018-08-30T01:44:55.780Z [DEBUG] plugin.terraform-provider-k8s: 2018/08/30 01:44:55 [DEBUG] found resource root path without trailing '/' - /api/v1/nodes/{name}/proxy
    2018-08-30T01:44:55.780Z [DEBUG] plugin.terraform-provider-k8s: 2018/08/30 01:44:55 [DEBUG] findMatchingResourceRoot result - [/api/v1/nodes/{name}/proxy/{path} /api/v1/nodes/{name}/proxy/]
    2018-08-30T01:44:55.780Z [DEBUG] plugin.terraform-provider-k8s: 2018/08/30 01:44:55 [DEBUG] found resource root path without trailing '/' - /api/v1/nodes/{name}/proxy
    2018-08-30T01:44:55.780Z [DEBUG] plugin.terraform-provider-k8s: 2018/08/30 01:44:55 [ERROR] There was an error initialising the terraform provider: plugin terraform-provider-openapi init error: error occurred while creating schema provider. Error=path /api/v1/nodes/{name}/proxy POST operation is missing paremeters
    2018/08/30 01:44:55 [ERROR] root: eval: *terraform.EvalInitProvider, err: unexpected EOF
    2018/08/30 01:44:55 [ERROR] root: eval: *terraform.EvalSequence, err: unexpected EOF
    2018/08/30 01:44:55 [TRACE] [walkInput] Exiting eval tree: provider.k8s
    2018-08-30T01:44:55.783Z [DEBUG] plugin: plugin process exited: path=/usr/local/bin/terraform-provider-k8s
    2018/08/30 01:44:55 [DEBUG] plugin: waiting for all plugin processes to complete...
    2018-08-30T01:44:55.784Z [WARN ] plugin: error closing client during Kill: err="connection is shut down"
    
  • Open api plugin issue

    Open api plugin issue

    Describe the bug

    PS C:\Users...\AppData\Roaming\terraform.d\plugins> .\terraform-provider-openapi.exe 2020/08/10 16:00:02 Running OpenAPI Terraform Provider v0.31.1-d28a81d857163a2dd72072634f10eb3977bb3584; Released on: 2020-07-20T23:37:07Z 2020/08/10 16:00:02 [INFO] open api plugin configuration not present at C:\Users...\AppData\terraform.d\plugins/terraform-provider-openapi.yaml 2020/08/10 16:00:02 [DEBUG] serviceConfig = 2020/08/10 16:00:02 [ERROR] There was an error initialising the terraform provider: plugin init error: swagger url not provided, please export OTF_VAR_<provider_name>_SWAGGER_URL env variable with the URL where 'openapi' service provider is exposing the swagger file OR create a plugin configuration file at ~/.terraform.d/plugins following the Plugin configuration schema specifications

    To Reproduce

    On a Windows 10 machine download and set up a provider.yaml

    Steps to reproduce the behavior:

    1. unzip the terraform-provider-openapi
    2. place in correct folder
    3. in powershell or cmd run .\terraform-provider-openapi.exe
    4. error message follows
  • [FeatureRequest: Issue #329] Adding ARM64 Support

    [FeatureRequest: Issue #329] Adding ARM64 Support

    What problem does this Pull Request solve?

    • Adding ARM64 support for Terraform-provider-openapi. Based on the ticket SLTF-5 and Issue #329 . Please link to the issue number here (issue will be closed when PR is merged): Closes #332

    Type of change

    What type of change does your code introduce to the provider? Please put an x (w/o heading/trailing white spaces) in the boxes that apply:

    • [x ] New feature (change that adds new functionality)
    • [ ] Bug-fix (change that fixes current functionality)
    • [ ] Tech debt (enhances the current functionality)
    • [ ] New release (pumps the version)

    Checklist

    Please put an x (w/o heading/trailing white spaces) in the boxes that apply:

    • [ x] I have read and followed the CONTRIBUTING guidelines
    • [ ] I have added tests that prove my fix is effective or that my feature works
    • [x] I have made sure code compiles correctly and all tests are passing by running make test-all
    • [ ] I have added/updated necessary documentation (if appropriate)
    • [x ] I have added the following info to the title of the PR (pick the appropriate option for the type of change). This is important because the release notes will include this information.
      • [x] Feature Request: PRs related to feature requests should have in the title [FeatureRequest: Issue #X] <PR Title>
      • [ ] Bug Fixes: PRs related to bug fixes should have in the title [BugFix: Issue #X] <PR Title>
      • [ ] Tech Debt: PRs related to technical debt should have in the title [TechDebt: Issue #X] <PR Title>
      • [ ] New Release: PRs related to a new release should have in the title [NewRelease] vX.Y.Z

    Checklist for Admins

    • [x] Label is populated
    • [x] PR is assigned to the corresponding project
    • [x] PR has at least 1 reviewer and 1 assignee
  • Unable to initialize Terraform OpenAPI provider

    Unable to initialize Terraform OpenAPI provider

    Describe the bug

    Followed the steps in the documentation to download the binary plugin and move it to the appropriate directory (%AppData%\Roaming\terraform.d\plugins, for Windows in my case). Created a main.tf file with a single resource definition:

    provider "dtr" { apikey_auth = "<my_api_key>" }

    resource "dtr_repositories" "apache_repo" { namespace = " my_dtr_username / terraformtest" }

    However when I try to perform "terraform init" it fails because it's querying the main hashicorp terraform registry for a provider named "dtr", which obviously fails. I've tried passing the "-plugin-dir" cli parameter and it fails in both cases. I'm sure it's something trivial I'm overlooking but I cannot get past this issue.

    Any advice is appreciated!

  • provider name detection can be wrong when using symlinks

    provider name detection can be wrong when using symlinks

    The use of os.Executable() can be wrong when the plugin is a symlink, ln -sF terraform-provider-openapi terraform-provider-<your_provider_name> may yield a name of openapi which is wrong.

    https://github.com/dikhan/terraform-provider-openapi/blob/03d1765264c7e08bdef63534651473fa5c64def8/openapi/provider.go#L67

  • [FeatureRequest: Issue #311] Support Terraform versions greater than or equal to 1.0.0

    [FeatureRequest: Issue #311] Support Terraform versions greater than or equal to 1.0.0

    What problem does this Pull Request solve?

    Please link to the issue number here (issue will be closed when PR is merged): Closes #311

    As discussed on PR-312 and PR-313, created this PR on behalf of the original author (@bradthebuilder ) so we get the feedback from the pipeline (as it's triggered by me and won't have the issues when running the jobs from forks - more details here) and are able to merge the PR following the branch restrictions.

    Type of change

    What type of change does your code introduce to the provider? Please put an x (w/o heading/trailing white spaces) in the boxes that apply:

    • [x] New feature (change that adds new functionality)
    • [ ] Bug-fix (change that fixes current functionality)
    • [ ] Tech debt (enhances the current functionality)
    • [ ] New release (pumps the version)

    Checklist

    Please put an x (w/o heading/trailing white spaces) in the boxes that apply:

    • [x] I have read and followed the CONTRIBUTING guidelines
    • [ ] I have added tests that prove my fix is effective or that my feature works
    • [x] I have made sure code compiles correctly and all tests are passing by running make test-all
    • [ ] I have added/updated necessary documentation (if appropriate)
    • [x] I have added the following info to the title of the PR (pick the appropriate option for the type of change). This is important because the release notes will include this information.
      • [x] Feature Request: PRs related to feature requests should have in the title [FeatureRequest: Issue #X] <PR Title>
      • [ ] Bug Fixes: PRs related to bug fixes should have in the title [BugFix: Issue #X] <PR Title>
      • [ ] Tech Debt: PRs related to technical debt should have in the title [TechDebt: Issue #X] <PR Title>
      • [ ] New Release: PRs related to a new release should have in the title [NewRelease] vX.Y.Z

    Commentary

    All unit and integration tests pass when using TF versions 1.0.1 and 1.0.7 on my system. I'm also able to successfully run make build. I've tested the built provider against my Antsle and it successfully created Antlets using my TF configs.

    Unit Tests Passing

    unit_tests_passing

    Integration Tests Passing

    integration_tests_passing

    Targeting your master branch per your request.

    There is also a single commit addressing what I thought was a typo in some documentation.

    Checklist for Admins

    • [x] Label is populated
    • [x] PR is assigned to the corresponding project
    • [x] PR has at least 1 reviewer and 1 assignee
  • [FeatureRequest: Issue #305] Support colon characters in field names

    [FeatureRequest: Issue #305] Support colon characters in field names

    What problem does this Pull Request solve?

    Please link to the issue number here (issue will be closed when PR is merged): Closes #305

    Type of change

    What type of change does your code introduce to the provider? Please put an x (w/o heading/trailing white spaces) in the boxes that apply:

    • [x] New feature (change that adds new functionality)
    • [ ] Bug-fix (change that fixes current functionality)
    • [ ] Tech debt (enhances the current functionality)
    • [ ] New release (pumps the version)

    Checklist

    Please put an x (w/o heading/trailing white spaces) in the boxes that apply:

    • [x] I have read and followed the CONTRIBUTING guidelines
    • [x] I have added tests that prove my fix is effective or that my feature works
    • [ ] I have made sure code compiles correctly and all tests are passing by running make test-all
    • [ ] I have added/updated necessary documentation (if appropriate)
    • [x] I have added the following info to the title of the PR (pick the appropriate option for the type of change). This is important because the release notes will include this information.
      • [x] Feature Request: PRs related to feature requests should have in the title [FeatureRequest: Issue #X] <PR Title>
      • [ ] Bug Fixes: PRs related to bug fixes should have in the title [BugFix: Issue #X] <PR Title>
      • [ ] Tech Debt: PRs related to technical debt should have in the title [TechDebt: Issue #X] <PR Title>
      • [ ] New Release: PRs related to a new release should have in the title [NewRelease] vX.Y.Z

    Checklist for Admins

    • [x] Label is populated
    • [x] PR is assigned to the corresponding project
    • [x] PR has at least 1 reviewer and 1 assignee

    Commentary

    All unit tests pass on my system.

    The following integration tests in tests/integration/resource_monitors_test.go failed on my laptop before and after my changes were added:

    • TestAccMonitor_CreateRst1
    • TestAccMonitor_CreateDub1
    • TestAccMonitor_MultiRegion_CreateRst1
    • TestAccMonitor_MultiRegion_CreateDub1
    • TestAccMonitor_MultiRegion_Create_Default_Region

    Integration test error stdout

    I figured it had something to do with my local network's DNS not being configured to resolve the API URLs used in the tests (I guess the docker containers bridged to my laptop), so I figured Travis CI would be the gatekeeper in this discrepancy. The errors were present on the master branch before my additions and are still there afterward, so I'm confident my additions at least didn't make things worse. If this is an actual bug and not an issue with my environment, I can work on a separate bug-hunting branch.

    The provider does compile and I tested it against the docker-compose local-env and my Antsle device. It works on my Antsle device perfectly, and 3/4 resources get created with the swaggercodegen example on terraform apply (terraform validate and terraform plan work fine).

    swaggercodegen_firewall_failure

    I think it's because the resource name has more than one number character in it; I noticed when adding my unit tests that the ConvertToTerraformCompliantName function cannot handle more than one distinct number in a field name. Once again, I can work on this in a separate bug branch if we're in agreement.

  • GetMultiRegionHost error is ignored leading to malformed multiregion resources

    GetMultiRegionHost error is ignored leading to malformed multiregion resources

    Proposed changes

    Please add as many details as possible about the change here. Does this Pull Request resolve any open issue? If so, please make sure to link to that issue:

    Fixes: #137

    Type of change

    What type of change does your code introduce to the provider? Please put an x (w/o heading/trailing white spaces) in the boxes that apply:

    • [x] Bug-fix (change that fixes current functionality)
    • [x] Tech debt
    • [ ] New feature (change that adds new functionality)

    Checklist

    Please put an x (w/o heading/trailing white spaces) in the boxes that apply:

    • [x] I have read and followed the CONTRIBUTING guidelines
    • [x] I have made sure code compiles correctly
    • [x] I have run 'make test' locally from the terraform_provider_api folder and no errors were found
    • [x] I have added tests that prove my fix is effective or that my feature works
    • [x] I have added/updated necessary documentation (if appropriate)
  • [FeatureRequest: Issue #352] Support for write-only properties

    [FeatureRequest: Issue #352] Support for write-only properties

    What problem does this Pull Request solve?

    Closes #352

    Type of change

    What type of change does your code introduce to the provider? Please put an x (w/o heading/trailing white spaces) in the boxes that apply:

    • [x] New feature (change that adds new functionality)
    • [ ] Bug-fix (change that fixes current functionality)
    • [ ] Tech debt (enhances the current functionality)
    • [ ] New release (pumps the version)

    Checklist

    Please put an x (w/o heading/trailing white spaces) in the boxes that apply:

    • [x] I have read and followed the CONTRIBUTING guidelines
    • [x] I have added tests that prove my fix is effective or that my feature works
    • [x] I have made sure code compiles correctly and all tests are passing by running make test-all
    • [x] I have added/updated necessary documentation (if appropriate)
    • [x] I have added the following info to the title of the PR (pick the appropriate option for the type of change). This is important because the release notes will include this information.
      • [x] Feature Request: PRs related to feature requests should have in the title [FeatureRequest: Issue #X] <PR Title>
      • [ ] Bug Fixes: PRs related to bug fixes should have in the title [BugFix: Issue #X] <PR Title>
      • [ ] Tech Debt: PRs related to technical debt should have in the title [TechDebt: Issue #X] <PR Title>
      • [ ] New Release: PRs related to a new release should have in the title [NewRelease] vX.Y.Z

    Checklist for Admins

    • [ ] Label is populated
    • [ ] PR is assigned to the corresponding project
    • [ ] PR has at least 1 reviewer and 1 assignee
  • Support for write-only properties

    Support for write-only properties

    Is your feature request related to a problem?

    The plugin seems to expect that all input properties on a terraform resource should be fetchable by the GET request for the resource, however there might be legitimate scenarios where an "input-only"/"write-only" property is needed. For example - providing some credentials to create a resource that only needs it at creation time (i.e on the POST request) and the service doesn't want to store these credentials to be fetched by the GET request.

    Describe the solution you'd like

    A new extension, example - x-terraform-write-only, which can be applied to input properties.

    Using this extension, changes to the property should be ignored if they are changed to null, since the GET request would not return a value for it or return null.

    When performing the diff on terraform state, the value of such properties could be obtained from the terraform state file rather than the API response.

    Acceptance criteria

    • A new extension is introduced for write-only input properties
    • The new extension will ignore changes to null for the target property
    • The extension is compatible with terraform import

    Describe alternatives you've considered

    A clear and concise description of any alternative solutions or features you've considered.

    Additional context

    I've had a look through the feature set and documentation and I haven't found a way to deal with such properties, so feel free to close this issue if there is a workaround that I've missed.

    Checklist (for admin only)

    Don't forget to go through the checklist to make sure the issue is created properly:

    • [ ] I have added a corresponding label (feature request) to the issue (right side menu)
    • [ ] I have added this issue to the 'API Terraform Provider' GitHub project (right side menu)
  • [BugFix: Issue #346] Fixes for x-terraform-ignore-order

    [BugFix: Issue #346] Fixes for x-terraform-ignore-order

    What problem does this Pull Request solve?

    Closes #346

    Type of change

    What type of change does your code introduce to the provider? Please put an x (w/o heading/trailing white spaces) in the boxes that apply:

    • [ ] New feature (change that adds new functionality)
    • [x] Bug-fix (change that fixes current functionality)
    • [ ] Tech debt (enhances the current functionality)
    • [ ] New release (pumps the version)

    Checklist

    Please put an x (w/o heading/trailing white spaces) in the boxes that apply:

    • [x] I have read and followed the CONTRIBUTING guidelines
    • [x] I have added tests that prove my fix is effective or that my feature works
    • [x] I have made sure code compiles correctly and all tests are passing by running make test-all
    • [x] I have added/updated necessary documentation (if appropriate)
    • [x] I have added the following info to the title of the PR (pick the appropriate option for the type of change). This is important because the release notes will include this information.
      • [ ] Feature Request: PRs related to feature requests should have in the title [FeatureRequest: Issue #X] <PR Title>
      • [x] Bug Fixes: PRs related to bug fixes should have in the title [BugFix: Issue #X] <PR Title>
      • [ ] Tech Debt: PRs related to technical debt should have in the title [TechDebt: Issue #X] <PR Title>
      • [ ] New Release: PRs related to a new release should have in the title [NewRelease] vX.Y.Z

    Checklist for Admins

    • [ ] Label is populated
    • [ ] PR is assigned to the corresponding project
    • [ ] PR has at least 1 reviewer and 1 assignee
  • Issues with x-terraform-ignore-order

    Issues with x-terraform-ignore-order

    Describe the bug

    After using the terraform-provider-openapi plugin fairly extensively, several issues have been identified with the behaviour of the plugin when using with the x-terraform-ignore-order flag -

    1. The plugin throws an NPE when the API response does not contain an optional property.
    2. The plugin does not correctly deal with camelCase vs terraform_compliant_name on property names.
    3. The plugin doesn't correctly handle cases where Terraform sets default values even for optional properties but the API does not return the optional property. (Example - in the local state, arrays become [], maps become {}, strings become "", integers become 0, booleans become false)
    4. The plugin does not correctly deal with integers returned from the API when comparing against the local state, because the API response is mapped to a float64 value but the local state is an integer.

    To Reproduce

    A clear and concise description on how does it work now and what's the current behaviour.

    Steps to reproduce the behavior:

    1. Update TestAccCDN_CreateResourceWithIgnoreListOrderExtension with the following - https://github.com/instaclustr/terraform-provider-openapi/blob/b287aa636768f0b7d2fc4a30b14deb528165225a/tests/e2e/gray_box_cdns_test.go#L429-L663
    2. Run the test

    Expected behaviour

    The test passes, and the x-terraform-ignore-order flag correctly handles all of the aforementioned issues.

    Additional context

    Add any other context about the problem here.

    Checklist (for admin only)

    Don't forget to go through the checklist to make sure the issue is created properly:

    • [ ] I have added a corresponding label (bug) to the issue (right side menu)
    • [ ] I have added this issue to the 'API Terraform Provider' GitHub project (right side menu)
  • Nested optional objects skip parent objects when omitted

    Nested optional objects skip parent objects when omitted

    Describe the bug

    If you have an optional object that is located under a nested object, omitting the optional object through the terraform code omits the parent object on the request body too.

    To Reproduce

    With the following swagger YAML -

    paths:
      /v1/characters:
        post:
          parameters:
            -
              name: body
              schema:
                $ref: '#/definitions/Character'
              in: body
              required: true
          responses:
            '200':
              description: Character created
              schema:
                $ref: '#/definitions/Character'
          summary: Create character
      '/v1/characters/{characterId}':
        get:
          parameters:
            -
              name: characterId
              in: path
              required: true
              type: string
          responses:
            '200':
              description: Character fetched.
              schema:
                $ref: '#/definitions/Character'
          summary: Retrieve character
    
    definitions:
      Position:
        title: Root Type for Position
        description: ''
        type: object
        properties:
          x:
            type: integer
          'y':
            type: integer
      Character:
        title: Root Type for Character
        description: ''
        required:
          - health
        type: object
        properties:
          id:
            type: string
            readOnly: true
          health:
            type: integer
          position:
            $ref: '#/definitions/Position'
          pet:
            $ref: '#/definitions/Pet'
      Pet:
        title: Root Type for Pet
        description: ''
        required:
          - health
        type: object
        properties:
          id:
            type: string
            readOnly: true
          health:
            type: integer
          position:
            $ref: '#/definitions/Position'
    

    the following terraform code works fine -

    resource "myprovider_characters_v1" "mychar" {
      health = 10
      position {
        x = 1
        y = 1
      }
      pet {
        health = 30
        position {
          x = 2
          y = 2
        }
      }
    }
    

    The request body is {"health":10,"pet":{"health":30,"position":{"x":2,"y":2}},"position":{"x":1,"y":1}}

    However the following terraform code runs into a problem -

    resource "myprovider_characters_v1" "mychar" {
      health = 10
      position {
        x = 1
        y = 1
      }
      pet {
        health = 30
      }
    }
    

    The problematic request body - {"health":10,"position":{"x":1,"y":1}} - skips the pet object because the optional position object is omitted from the terraform code.

    Expected behaviour

    Omitting optional nested objects continues to include the parent object in the request body.

    Additional context

    Interestingly, omitting position on the Character level does not cause an issue. This is only a problem for optional objects under nested objects.

    Checklist (for admin only)

    Don't forget to go through the checklist to make sure the issue is created properly:

    • [ ] I have added a corresponding label (bug) to the issue (right side menu)
    • [ ] I have added this issue to the 'API Terraform Provider' GitHub project (right side menu)
  • Support for simple listable data source

    Support for simple listable data source

    Is your feature request related to a problem?

    As a user of this plugin, I would like to define my APIs to return a single object containing a list of results as my data source so that I do not have to perform any hacky workarounds on my API to conform with the plugin's requirements.

    Describe the solution you'd like

    Non-CRUD data sources should be created for any GET endpoint that returns a single object, not just ones that return an array.

    Acceptance criteria

    • Data sources are created from all GET endpoints that return a single object.
    • A meta attribute is introduced to exclude certain GET endpoints from being converted to data sources.

    A data source should be creatable for the following API response for example -

    {
      "id": "some_id",
      "results": [
        {
          "myProperty": "myValue"
        }
      ]
    }
    

    Describe alternatives you've considered

    The alternative has been to return a single item array on my GET list requests.

    Example -

    [
      {
        "id": "some_kind_of_id",
        "results": [
          {
            "myProperty": "myValue"
          }
        ]
      }
    ]
    

    As you can see, the above workaround is a bit hacky. Users using my API directly will be fairly confused why I've got such a strange structure - with an array containing a single object containing an array.

    If I don't wrap my response in such a manner and directly return my list of results I get the following error - Error: your query returned contains more than one result. Please change your search criteria to make it more specific because the plugin only wants one result (I think it is so that it can set the ID of the data source).

    Additional context

    Bonus points if an id/x-terraform-id doesn't have to be defined on the API response. I'm not sure Terraform even meaningfully uses the ID of a data source so you could even just set it to some random GUID.

    Checklist (for admin only)

    Don't forget to go through the checklist to make sure the issue is created properly:

    • [ ] I have added a corresponding label (feature request) to the issue (right side menu)
    • [ ] I have added this issue to the 'API Terraform Provider' GitHub project (right side menu)
Terraform provider to help with various AWS automation tasks (mostly all that stuff we cannot accomplish with the official AWS terraform provider)
Terraform provider to help with various AWS automation tasks (mostly all that stuff we cannot accomplish with the official AWS terraform provider)

terraform-provider-awsutils Terraform provider for performing various tasks that cannot be performed with the official AWS Terraform Provider from Has

Dec 8, 2022
Terraform Provider for Azure (Resource Manager)Terraform Provider for Azure (Resource Manager)
Terraform Provider for Azure (Resource Manager)Terraform Provider for Azure (Resource Manager)

Terraform Provider for Azure (Resource Manager) Version 2.x of the AzureRM Provider requires Terraform 0.12.x and later, but 1.0 is recommended. Terra

Oct 16, 2021
Terraform-provider-mailcow - Terraform provider for Mailcow

Terraform Provider Scaffolding (Terraform Plugin SDK) This template repository i

Dec 31, 2021
Terraform-provider-buddy - Terraform Buddy provider For golang

Terraform Provider for Buddy Documentation Requirements Terraform >= 1.0.11 Go >

Jan 5, 2022
Terraform-provider-vercel - Terraform Vercel Provider With Golang

Vercel Terraform Provider Website: https://www.terraform.io Documentation: https

Dec 14, 2022
Terraform-provider-age - Age Terraform Provider with golang

Age Terraform Provider This provider lets you generate an Age key pair. Using th

Feb 15, 2022
Kusk makes your OpenAPI definition the source of truth for API resources in your cluster
Kusk makes your OpenAPI definition the source of truth for API resources in your cluster

Kusk - use OpenAPI to configure Kubernetes What is Kusk? Developers deploying their REST APIs in Kubernetes shouldn't have to worry about managing res

Dec 16, 2022
Terraform-in-Terraform: Execute Modules directly from the Terraform Registry

Terraform-In-Terraform Provider This provider allows running Terraform in Terraform. This might seem insane but there are some edge cases where it com

Dec 25, 2022
A Terraform provider for managing Pi-hole resources

terraform-provider-pihole Pi-hole is an ad blocking application which acts as a DNS proxy that returns empty responses when DNS requests for known adv

Dec 28, 2022
Terraform utility provider for constructing bash scripts that use data from a Terraform module

Terraform Bash Provider This is a Terraform utility provider which aims to robustly generate Bash scripts which refer to data that originated in Terra

Sep 6, 2022
Quick start repository for creating a Terraform provider using terraform-plugin-framework

Terraform Provider Scaffolding (Terraform Plugin Framework) This template repository is built on the Terraform Plugin Framework. The template reposito

Dec 15, 2022
Terraform Provider Scaffolding (Terraform Plugin SDK)

Terraform Provider Scaffolding (Terraform Plugin SDK) This template repository is built on the Terraform Plugin SDK. The template repository built on

Feb 8, 2022
The k8s-generic-webhook is a library to simplify the implementation of webhooks for arbitrary customer resources (CR) in the operator-sdk or controller-runtime.

k8s-generic-webhook The k8s-generic-webhook is a library to simplify the implementation of webhooks for arbitrary customer resources (CR) in the opera

Nov 24, 2022
Hashicups-tf-provider - HashiCups Terraform Provider Tutorial

Terraform Provider HashiCups Run the following command to build the provider go

Jan 10, 2022
A tool to bring existing Azure resources under Terraform's management

Azure Terrafy A tool to bring your existing Azure resources under the management of Terraform. Install go install github.com/magodo/aztfy@latest Usage

Dec 9, 2021
A Terraform module that creates AWS alerts billing for your resources.

terraform-aws-billing-alarms terraform-aws-billing-alarms for project Replace name project to New Project agr 'terraform-aws-billing-alarms' 'new-pr

Oct 20, 2021
A tool to bring existing Azure resources under Terraform's management

Azure Terrafy A tool to bring your existing Azure resources under the management of Terraform. Goal Azure Terrafy imports the resources inside a resou

Jan 1, 2023
Simple tool to move Azure resources based on Terraform state

aztfmove Simple tool to move Azure resources based on Terraform state Goal It is sometimes inevitable to move Azure resources to a new subscription or

Dec 29, 2022