Introducing a New Terraform Provider: Expanding Our Ecosystem

We are proud to announce the release of the official Terraform Provider for Transcend! With options to organize your privacy data in code, it has never been easier to set up governance of your privacy stack.

Transcend offers a cohesive set of products to manage privacy request automation, data mapping, and consent management. In this blog we will show how the Terraform Provider can help with each product. Let’s get started:

Getting started

Our provider is on the Terraform Registry. Here, you will find complete documentation of our provider and all associated resources and data sources. Once you have an API Key from our Admin Dashboard, you’re ready to get started.

terraform {
  required_providers {
    transcend = {
      source = "transcend-io/transcend"
      version = "X.X.X" # fill in with your desired version
    }
  }
}

provider "transcend" {
  key = "<some_api_key"> # Can also be set via the `TRANSCEND_KEY` environment variable
}

Connecting an integration

Wherever your user data is stored, Transcend can help automate its retrieval, modification, or deletion when you receive a privacy request. This personal data can be stored in a variety of places, such as in your databases, logs, Slack messages, or in a third party tool like Twilio, Salesforce, or Zendesk. In Transcend-land, we call each of these systems an Integration, which can be connected in Terraform.

The most conceptually simple Integration is where we send you a webhook with a user’s identifiers and you respond with labeled data about the user. You can setup a server integration with the following code:

resource "transcend_data_silo" "server" {
  type = "server"
  title = "User Data Webhook"
  url = "https://your.company.domain/user/lookup"
  description = "Fetches user data from our internal API"
  owner_emails = ["david@transcend.io"]

  headers {
    name = "someHeaderSentWithWebhook"
    value = "someSecret"
    is_secret = false
  }
}

Now whenever you receive a privacy request, Transcend would send an HTTP request to the `https://some.api.link/some/path` URL with:

• the custom “someHeaderSentWithWebhook” header set
• a JWT to validate that the request came from Transcend
• information about the user who submitted the request and the type of request submitted

After a `terraform apply`, you will be able to see the configured Integration in the Admin Dashboard. As requests come in, you’ll be able to view details about each request, including an audit trail of actions taken relating to user data in this Integration.

In many cases, connecting an integration with the necessary authentication is all you need to do. As an example, if you are connecting Twilio, a SaaS application that manages text messages and other communications data, our integration catalog will know what data types Twilio has on its users and how to retrieve that data using your credentials.

In the example above with our webhook integration, you can manually specify what data can be returned and why you collect that data. Let’s learn how.

Labeling and classifying data

When a server webhook is set up, it can return JSON data immediately. But what if we want to organize that information into subfields—labeling data categories and our purposes for collecting that data? In Transcend, you can configure this data with Data Points.

In our server integration example, let’s pretend our server endpoint takes in a user ID and returns a user’s email address and location data. We would define a data point named `User` that has properties `Email` and `Location`:

resource "transcend_data_point" "server" {
  data_silo_id = transcend_data_silo.server.id
  name = "User"
  title = "User Data"

  properties {
    name = "Email"
    description = "The email address of a customer"

    categories {
      name = "Email"
      category = "CONTACT"
    }
    purposes {
      name = "Other"
      purpose = "ESSENTIAL"
    }
  }

  properties {
    name = "Location"
    description = "The user's estimated location"

    categories {
      name = "Approximate Geolocation"
      category = "LOCATION"
    }
    purposes {
      name = "Other"
      purpose = "ADDITIONAL_FUNCTIONALITY"
    }
  }
}

This helps in a few ways:

• When the privacy request completes, we can use these labels in the report we generate for your end user. This makes their privacy request experience more organized and easier to understand.
• The webhook can now return labeled data, which makes it easier for you to manage.
• Labeling data point properties with categories and purposes gets you started on different types of privacy reports, such as a Record of Processing Activities.

In fact, if we head over to our ROPA view, we’ll see the beginnings of our records already show the categories and purposes we specified in our data point properties:

By automatically adding each data point’s categories and purposes to the ROPA, your ROPA can start to feel more like an extension of your privacy request and data mapping workflows, rather than some scary spreadsheet you manage separately.

One of our favorite ways we’ve seen this used is by our friends at Clubhouse, who enforce that each field in their database can specify a data category and purpose of processing. They then have a script that exports this data, such that it can be synced into Transcend’s backend, with each database table being a data point and each table column with personal data being a property. This is a fantastic way to ensure that your ROPA and privacy request flows are always up to date with the data you collect.

When should I use this provider?

This provider is a great place to start on any new services you are setting up where you want to encode privacy directly into your stack. As you create a new database, sync its fields. As you add a new service that stores personal data, add a server webhook for it. As you add a new SaaS vendor, connect an integration for them right away. 

Or if your existing services are small enough, you can often retroactively add in using this provider (like Clubhouse did to label and govern their data).

But what about cases where you are trying to retroactively discover where your data is? If you have thousands of databases, it may be infeasible to label each field containing personal data by hand. 

In these cases, you can set up “plugins” inside Transcend that can discover integrations to connect. For example, if you use our Okta data silo plugin, we can suggest any SaaS tools your employees can access. Or if you connect our Snowflake, MongoDB, or Postgres data point plugins, we can scan the databases to find fields containing personal data and suggest classifications for those fields.

To make use of our plugins, you can connect the plugins via Terraform (see an example of connecting an AWS Silo Discovery Plugin here that can discover databases and S3 buckets in all regions of your accounts).

For now, you will need to go into the Admin dashboard to schedule when the plugin will run, but soon our provider will support scheduling as well.

Depending on the scale of how many integrations and data points are discovered, it may then be feasible to use Terraform import statements to manage them in code.

Security model

Privacy requests deal with data across all of your systems, meaning that the final report sent back to a user after an Access request will contain all of the sensitive information your company has access to. At Transcend our highest priority is keeping that data secure and confidential, to the extent that our end-to-end encryption model ensures we cannot see the personal data at all.

This is accomplished by deploying Sombra, our end-to-end encryption gateway, inside your cloud. When we want to make a request to Twilio to find the text messages you’ve sent to a user, we send the request without an API key to your gateway, which then uses your KMS to add in the API key and sends the request to Twilio. 

When you get back the personal data, your KMS encrypts that data before sending it back to Transcend. By using this flow in our integrations, we never have access to your customers’ data in plaintext or in any form we can decrypt.

In the context of this Terraform Provider, no personal data is sent through it to Transcend. When we add support in the near future for specifying API keys, those keys will first be sent to your Sombra instance, encrypted, and then sent to Transcend for storage.

Future work

Right now, many of our supported fields are included in the provider. However, there are still some things that will need to be done in our Admin Dashboard, such as connecting integrations with API Keys or other sensitive credentials. Right now, we support connecting many integration types such as our AWS integrations and our hundreds of Automated Vendor Coordination integrations, but for Oauth and API Key integrations that deal with sensitive data, we are still working to support those in Terraform in a secure way.

There are also some fields that are useful to ROPA generation, like retention periods and setting up plugin runs, that we are continuing to add support for.

About Transcend

Transcend is the company that makes it easy to encode privacy across your entire tech stack. Our mission is to make it simple for companies to give users control of their data.

Automate data subject request workflows with Privacy Requests, ensure nothing is tracked without user consent using Transcend Consent, or discover data silos and auto-generate reports with Data Mapping.