Managing Google Cloud Pub/Sub Log Sources with Terraform (Beta)

Manage Google Cloud Pub/Sub log sources as code in Terraform

Overview

Managing Google Cloud Pub/Sub log sources with Terraform is in open beta starting with Panther version 1.121, and is available to all customers. Please share any bug reports and feature requests with your Panther support team.

You can define your Google Cloud Pub/Sub log source in Terraform using the Panther Terraform provider.

Other methods to create a Pub/Sub log source include using the Panther API directly and manual creation in the Panther Console.

How to define your Panther Pub/Sub log source in Terraform

The following sections outline how to define your Pub/Sub log source in HashiCorp Configuration Language (HCL).

Prerequisite

• Before starting, ensure you have an API URL and token with the Manage Log Sources permission. This is required to complete Step 3.

  • If needed, follow these instructions for creating an API token in the Panther Console.

Step 1: Choose an authentication method

• Select an authentication method for your Pub/Sub source from the options listed on Pub/Sub Source.

The authentication method you select will determine the variables you define in Step 2, below.

Step 2: Define variables

• Define a variables.tf file with the Panther variables shown in the code block below.

variable "panther_api_token" {
  description = "Panther API token"
  type        = string
}

variable "panther_api_url" {
  description = "Panther API URL"
  type        = string
}

variable "integration_label" {
  description = "The name of the integration."
  type        = string
}

variable "credentials_type" {
  description = "Authentication method used."
  type        = string
}

variable "subscription_id" {
  description = "Google Cloud Pub/Sub subscription ID"
  type        = string
}

// Auth variables are specific to credentials_type. See table below
variable "credentials" {
  description = "Service account JSON key or WIF credential configuration file content"
  type        = string
  sensitive   = true
}

variable "project_id" {
  description = "Google Cloud Project ID (required for WIF, optional for service account)"
  type        = string
}

// (Optional) Regional endpoint
variable "regional_endpoint" {
  description = "Regional endpoint for Pub/Sub"
  type        = string
}

// (Optional) Relevant only when log_stream_type = "JsonArray"
variable "json_array_envelope_field" {
  description = "Envelope field for json array stream"
  type        = string
}

// (Optional) Relevant only when log_stream_type = "XML"
variable "xml_root_element" {
  description = "Root element name for XML log streams"
  type        = string
}

Authentication method-specific variables

In your variables.tf file, include the values in the Additional variables column below for the authentication method you chose in Step 1.

Authentication method	credentials_type value	Additional variables
Service account authentication	service_account	credentials (JSON keyfile content)
Workload Identity Federation authentication	wif	credentials (credential configuration file content), project_id

Step 3: Provide values for the defined variables

• Add a *.tfvars file that assigns values to the variables you defined in Step 2. Note that to complete this section, you will need the API URL and token outlined in the Prerequisite section.

  • Your panther_api_url value should be your root API URL. This is either:

    • A GraphQL API URL without the /public/graphql suffix

    • A REST API URL as-is (REST URLs do not have a suffix after the root URL)

panther_api_token         = "XXXXXXXXXX"
panther_api_url           = "https://your-panther-url/v1"
integration_label         = "test-pubsub-integration"
credentials_type          = "service_account" // service_account or wif
subscription_id           = "my-subscription"
credentials               = "{ ... }" // JSON keyfile or credential config content
project_id                = "my-gcp-project" // Required for WIF, optional for service_account
regional_endpoint         = "us-central1-pubsub.googleapis.com" // Optional
json_array_envelope_field = "records"
xml_root_element          = "" // Optional, relevant only when log_stream_type = "XML"

Step 4: Define the Terraform provider

• Add the Panther Terraform provider.

terraform {
  required_providers {
    panther = {
      source = "panther-labs/panther"
      version = "~> 0.2.10"
    }
  }
}

Step 5: Define Panther Pub/Sub log source

The following HCL configuration defines the Pub/Sub log source in Panther.

provider "panther" {
  token = var.panther_api_token
  url   = var.panther_api_url
}

resource "panther_pubsubsource" "demo_pubsub_source" {
  integration_label   = var.integration_label
  log_stream_type     = "JSON" // Options: JSON, JsonArray, Auto, Lines, and XML
  log_types           = ["GCP.AuditLog"]
  credentials_type    = var.credentials_type
  subscription_id     = var.subscription_id
  credentials         = var.credentials
  project_id          = var.project_id
  regional_endpoint   = var.regional_endpoint // Optional
  // (Optional) Configure based on log_stream_type
  log_stream_type_options = {
    json_array_envelope_field = var.json_array_envelope_field // Relevant for "JsonArray"
    xml_root_element          = var.xml_root_element          // Relevant for "XML"
  }
}