TerraformEnterprise Adapter

This adapter is used to integrate the Itential Automation Platform (IAP) with the TerraformEnterprise System. The API for TerraformEnterprise is available at HERE. The adapter utilizes the TerraformEnterprise API to provide the integrations that are deemed pertinent to IAP. This ReadMe file is intended to provide information on this adapter.

Note: It is possible that some integrations will be supported through the TerraformEnterprise adapter while other integrations will not.

Itential provides information on all of its product adapters in the Customer Knowledge Base. Information in the Customer Knowledge Base is consistently maintained and goes through documentation reviews. As a result, it should be the first place to go for information.

For custom built adapters, it is a starting point to understand what you have built, provide the information for you to be able to update the adapter, and assist you with deploying the adapter into IAP.

Versioning

Itential Product adapters utilize SemVer for versioning. The current version of the adapter can be found in the package.json file or viewed in the IAP GUI on the System page. For Open Source Adapters, the versions available can be found in the Itential OpenSource Repository.

Release History

Any release prior to 1.0.0 is a pre-release. Initial builds of adapters are generally set up as pre-releases as there is often work that needs to be done to configure the adapter and make sure the authentication process to TerraformEnterprise works appropriately.

Release notes can be viewed in CHANGELOG.md or in the Customer Knowledge Base for Itential adapters.

Getting Started

These instructions will help you get a copy of the project on your local machine for development and testing. Reading this section is also helpful for deployments as it provides you with pertinent information on prerequisites and properties.

Adapter Technical Resources

There is adapter documentation available on the Itential Developer Site HERE. This documentation includes information and examples that are helpful for:

Authentication
Properties
Code Files
Action Files
Schema Files
Mock Data Files
Linting and Testing
Troubleshooting

Others will be added over time. Want to build a new adapter? Use the Adapter Builder HERE

Environment Prerequisites

The following is a list of required packages for an adapter.

Node.js
npm
Git

Adapter Prerequisites

The following list of packages are required for Itential product adapters or custom adapters that have been built utilizing the Itential Adapter Builder.

Additional Prerequisites for Development and Testing

If you are developing and testing a custom adapter, or have testing capabilities on an Itential product adapter, you will need to install these packages as well.

chai
eslint
eslint-config-airbnb-base
eslint-plugin-import
eslint-plugin-json
mocha
nyc
testdouble
winston

Creating a Workspace

The following provides a local copy of the repository along with adapter dependencies.

git clone git@gitlab.com:\@itentialopensource/adapters/adapter-terraform_enterprise
npm install

Adapter Properties and Descriptions

This section defines all the properties that are available for the adapter, including detailed information on what each property is for. If you are not using certain capabilities with this adapter, you do not need to define all of the properties. An example of how the properties for this adapter can be used with tests or IAP are provided in the Installation section.

  {
    "id": "ALL ADAPTER PROPERTIES!!!",
    "properties": {
      "host": "system.access.resolved",
      "port": 443,
      "base_path": "/",
      "version": "v1",
      "cache_location": "local",
      "encode_pathvars": true,
      "save_metric": true,
      "stub": false,
      "protocol": "https",
      "authentication": {
        "auth_method": "basic user_password",
        "username": "username",
        "password": "password",
        "token": "token",
        "invalid_token_error": 401,
        "token_timeout": 0,
        "token_cache": "local",
        "auth_field": "header.headers.X-AUTH-TOKEN",
        "auth_field_format": "{token}"
      },
      "healthcheck": {
        "type": "startup",
        "frequency": 300000
      },
      "request": {
        "number_redirects": 0,
        "number_retries": 3,
        "limit_retry_error": [401],
        "failover_codes": [404, 405],
        "attempt_timeout": 5000,
        "global_request": {
          "payload": {},
          "uriOptions": {},
          "addlHeaders": {},
          "authData": {}
        },
        "healthcheck_on_timeout": false,
        "return_raw": false,
        "archiving": false
      },
      "ssl": {
        "ecdhCurve": "",
        "enabled": false,
        "accept_invalid_cert": false,
        "ca_file": "",
        "key_file": "",
        "cert_file": "",
        "secure_protocol": "",
        "ciphers": ""
      },
      "throttle": {
        "throttle_enabled": false,
        "number_pronghorns": 1,
        "sync_async": "sync",
        "max_in_queue": 1000,
        "concurrent_max": 1,
        "expire_timeout": 0,
        "avg_runtime": 200,
        "priorities": []
      },
      "proxy": {
        "enabled": false,
        "host": "localhost",
        "port": 9999,
        "protocol": "http"
      },
      "mongo": {
        "host": "",
        "port": 0,
        "database": "",
        "username": "",
        "password": "",
        "replSet": "",
        "db_ssl": {
          "enabled": false,
          "accept_invalid_cert": false,
          "ca_file": "",
          "key_file": "",
          "cert_file": ""
        }
      }
    },
    "type": "YOUR ADAPTER CLASS"
  }

Connection Properties

These base properties are used to connect to TerraformEnterprise upon the adapter initially coming up. It is important to set these properties appropriately.

A connectivity check tells IAP the adapter has loaded successfully.

Authentication Properties

The following properties are used to define the authentication process to TerraformEnterprise.

Note: Depending on the method that is used to authenticate with TerraformEnterprise, you may not need to set all of the authentication properties.

Examples of authentication field format

"{token}"
"Token {token}"
"{username}:{password}"
"Basic {b64}{username}:{password}{/b64}"

Healthcheck Properties

The healthcheck properties defines the API that runs the healthcheck to tell the adapter that it can reach TerraformEnterprise. There are currently three types of healthchecks.

• None - Not recommended. Adapter will not run a healthcheck. Consequently, unable to determine before making a request if the adapter can reach TerraformEnterprise.
• Startup - Adapter will check for connectivity when the adapter initially comes up, but it will not check afterwards.
• Intermittent - Adapter will check connectivity to TerraformEnterprise at a frequency defined in the frequency property.

Request Properties

The request section defines properties to help handle requests.

SSL Properties

The SSL section defines the properties utilized for ssl authentication with TerraformEnterprise. SSL can work two different ways: set the accept\_invalid\_certs flag to true (only recommended for lab environments), or provide a ca\_file.

Throttle Properties

The throttle section is used when requests to TerraformEnterprise must be queued (throttled). All of the properties in this section are optional.

Proxy Properties

The proxy section defines the properties to utilize when TerraformEnterprise is behind a proxy server.

Mongo Properties

The mongo section defines the properties used to connect to a Mongo database. Mongo can be used for throttling as well as to persist metric data. If not provided, metrics will be stored in the file system.

Testing an Itential Product Adapter

Mocha is generally used to test all Itential Product Adapters. There are unit tests as well as integration tests performed. Integration tests can generally be run as standalone using mock data and running the adapter in stub mode, or as integrated. When running integrated, every effort is made to prevent environmental failures, however there is still a possibility.

Unit Testing

Unit Testing includes testing basic adapter functionality as well as error conditions that are triggered in the adapter prior to any integration. There are two ways to run unit tests. The prefered method is to use the testRunner script; however, both methods are provided here.

node utils/testRunner --unit
 
npm run test:unit

To add new unit tests, edit the test/unit/adapterTestUnit.js file. The tests that are already in this file should provide guidance for adding additional tests.

Integration Testing - Standalone

Standalone Integration Testing requires mock data to be provided with the entities. If this data is not provided, standalone integration testing will fail. When the adapter is set to run in stub mode (setting the stub property to true), the adapter will run through its code up to the point of making the request. It will then retrieve the mock data and return that as if it had received that data as the response from TerraformEnterprise. It will then translate the data so that the adapter can return the expected response to the rest of the Itential software. Standalone is the default integration test.

Similar to unit testing, there are two ways to run integration tests. Using the testRunner script is better because it prevents you from having to edit the test script; it will also resets information after testing is complete so that credentials are not saved in the file.

node utils/testRunner
  answer no at the first prompt
 
npm run test:integration

To add new integration tests, edit the test/integration/adapterTestIntegration.js file. The tests that are already in this file should provide guidance for adding additional tests.

Integration Testing

Integration Testing requires connectivity to TerraformEnterprise. By using the testRunner script it prevents you from having to edit the integration test. It also resets the integration test after the test is complete so that credentials are not saved in the file.

Note: These tests have been written as a best effort to make them work in most environments. However, the Adapter Builder often does not have the necessary information that is required to set up valid integration tests. For example, the order of the requests can be very important and data is often required for creates and updates. Hence, integration tests may have to be enhanced before they will work (integrate) with TerraformEnterprise. Even after tests have been set up properly, it is possible there are environmental constraints that could result in test failures. Some examples of possible environmental issues are customizations that have been made within TerraformEnterprise which change order dependencies or required data.

node utils/testRunner
answer yes at the first prompt
answer all other questions on connectivity and credentials

Test should also be written to clean up after themselves. However, it is important to understand that in some cases this may not be possible. In addition, whenever exceptions occur, test execution may be stopped, which will prevent cleanup actions from running. It is recommended that tests be utilized in dev and test labs only.

Reminder: Do not check in code with actual credentials to systems.

Installing an Itential Product Adapter

If you have App-Artifact installed in IAP, you can follow the instruction for that application to install the adapter into IAP. If not, follow these instructions.

1. Set up the name space location in your IAP node_modules.

cd /opt/pronghorn/current/node_modules
if the @itentialopensource directory does not exist, create it:
   mkdir @itentialopensource

1. Clone the adapter into your IAP environment.

cd \@itentialopensource
git clone git@gitlab.com:\@itentialopensource/adapters/adapter-terraform_enterprise

1. Install the dependencies for the adapter.

cd adapter-terraform_enterprise
npm install

1. If you are running IAP 2019.1 or older, add the adapter properties for TerraformEnterprise (created from Adapter Builder) to the properties.json file for your Itential build. You will need to change the credentials and possibly the host information below. TerraformEnterprise sample properties. If you are running IAP 2019.2 the adapter properties need to go into the database. You can review IAP documentation for how to do this.

2. Restart IAP

systemctl restart pronghorn

Installing a Custom Adapter

If you built this as a custom adapter through the Adapter Builder, it is recommended you go through setting up a development environment and testing the adapter before installing it. There is often configuration and authentication work that is required before the adapter will work in IAP.

1. Move the adapter into the IAP node_modules directory.

Depending on where your code is located, this process is different.
    Could be a tar, move, untar
    Could be a git clone of a repository
    Could also be a cp -R from a coding directory
Adapter should be placed into: /opt/pronghorn/current/node_modules/\@itentialopensource

1. Follow Steps 3-5 (above) to install an Itential adapter to load your properties, dependencies and restart IAP.

Using this Adapter

The adapter.js file contains the calls the adapter makes available to the rest of the Itential Platform. The API detailed for these calls should be available through JSDOC. The following is a brief summary of the calls.

Generic Adapter Calls

The connect call is run when the Adapter is first loaded by he Itential Platform. It validates the properties have been provided correctly.

connect()

The healthCheck call ensures that the adapter can communicate with TerraformEnterprise. The actual call that is used is defined in the adapter properties.

healthCheck(callback)

The refreshProperties call provides the adapter the ability to accept property changes without having to restart the adapter.

refreshProperties(properties)

The encryptProperty call will take the provided property and technique, and return the property encrypted with the technique. This allows the property to be used in the adapterProps section for the credential password so that the password does not have to be in clear text. The adapter will decrypt the property as needed for communications with TerraformEnterprise.

encryptProperty(property, technique, callback)

The getQueue call will return the requests that are waiting in the queue if throttling is enabled.

getQueue(callback)

The addEntityCache call will take the entities and add the list to the entity cache to expedite performance.

addEntityCache(entityType, entities, key, callback)

The capabilityResults call will take the results from a verifyCompatibility and put them in the format to be passed back to the Itential Platform.

capabilityResults(results, callback)

The hasEntity call verifies the adapter has the specific entity.

hasEntity(entityType, entityId, callback)

The verifyCapability call verifies the adapter can perform the provided action on the specific entity.

verifyCapability(entityType, actionType, entityId, callback)

The updateEntityCache call will update the entity cache.

updateEntityCache()

Specific Adapter Calls

Specific adapter calls are built based on the API of the TerraformEnterprise. The Adapter Builder creates the proper method comments for generating JS-DOC for the adapter. This is the best way to get information on the calls.

Troubleshooting the Adapter

Connectivity Issues

1. Verify the adapter properties are set up correctly.

Go into the Itential Platform GUI and verify/update the properties

1. Verify there is connectivity between the Itential Platform Server and TerraformEnterprise Server.

ping the ip address of TerraformEnterprise server
try telnet to the ip address port of TerraformEnterprise

1. Verify the credentials provided for TerraformEnterprise.

login to TerraformEnterprise using the provided credentials

1. Verify the API of the call utilized for TerraformEnterprise Healthcheck.

Go into the Itential Platform GUI and verify/update the properties

Functional Issues

Adapter logs are located in /var/log/pronghorn. In older releases of the Itential Platform, there is a pronghorn.log file which contains logs for all of the Itential Platform. In newer versions, adapters are logging into their own files.

Contributing to TerraformEnterprise

Please check out the Contributing Guidelines.

License & Maintainers

Maintained By

Itential Product Adapters are maintained by the Itential Adapter Team.
Itential OpenSource Adapters are maintained by the community at large.
Custom Adapters are maintained by other sources.

Product License

Apache 2.0