# Managing AWS S3 Log Sources with Terraform

## Overview

You can define your Panther S3 log source completely in Terraform—meaning you can create the S3 bucket and associated infrastructure in AWS, and onboard it to your Panther instance. Panther is a [Terraform provider](https://registry.terraform.io/providers/panther-labs/panther/latest).

Other methods to create an S3 log source include [using the Panther API](https://docs.panther.com/api/graphql/log-source#creating-an-s3-log-source) directly and [manual creation in the Panther Console](https://docs.panther.com/data-onboarding/data-transports/aws/s3).

## How to define your Panther S3 log source in Terraform

The following sections outline how to define your S3 log source in HashiCorp Configuration Language (HCL). You will define both AWS and Panther infrastructure in HCL.

{% hint style="warning" %}
The first log source you create in Panther must be done in the Panther Console. If you use Terraform to set up your first Panther log source, you may run into a "pending confirmation" issue detailed in [this Knowledge Base article](https://help.panther.com/articles/2327494518-why-is-my-sns-topic-stuck-in-a-pending-confirmation-state-for-the-sqs-confirmation-for-panther).
{% endhint %}

### Prerequisite

* Before starting, ensure you have an API URL and token with the `Manage Log Sources` permission. This is required to complete [Step 2](#step-2-provide-values-for-the-defined-variables).
  * If needed, follow [these instructions for creating an API token in the Panther Console](https://docs.panther.com/api#how-to-create-a-panther-api-token).

### Step 1: Define variables

* Create a `variables.tf` file with the following AWS and Panther variables:

```hcl
variable "aws_account_id" {
  type        = string
  description = "The AWS account ID where the template is being deployed"
}

variable "panther_aws_account_id" {
  type        = string
  description = "The AWS account ID of your Panther instance"
}

variable "panther_aws_region" {
  type = string
  default = "us-east-1"
  description = "The region where the Panther instance is deployed"
}

variable "panther_aws_partition" {
  type        = string
  default     = "aws"
  description = "AWS partition of the account running the Panther backend e.g aws, aws-cn, or aws-us-gov"
}

variable "s3_bucket_name" {
  type        = string
  description = "The S3 Bucket name to onboard"
}

variable "log_source_name" {
  type = string
  description = "The name of the log source to be created in Panther"
}

variable "panther_api_token" {
  type = string
}

variable "panther_api_url" {
  type = string
}
```

### Step 2: Provide values for the defined variables

* Add a `*.tfvars` file that assigns values to the variables you defined in Step 1. Note that to complete this section, you will need the API URL and token outlined in the [Prerequisite section](#prerequisite).
  * Your `panther_api_url` value should be your root API URL. This is either:
    * A [GraphQL API URL](https://docs.panther.com/api/graphql#step-1-identify-your-panther-graphql-api-url) without the `/public/graphql` suffix
    * A [REST API URL](https://docs.panther.com/api/rest#step-1-identify-your-panther-rest-api-url) as-is (REST URLs do not have a suffix after the root URL)

```hcl
aws_account_id         = "XXXXXXXXXX"
panther_aws_account_id = "XXXXXXXXXX"
panther_aws_region     = "us-east-1"
panther_aws_partition  = "aws"
s3_bucket_name         = "test-bucket"
log_source_name        = "test-integration"
panther_api_token      = "XXXXXXXXXX"
panther_api_url        = "https://your-panther-url/v1"
```

### Step 3: Define Terraform providers

* Include both the the [AWS](https://registry.terraform.io/providers/hashicorp/aws/latest/docs) and [Panther](https://registry.terraform.io/providers/panther-labs/panther/latest) Terraform providers:

```hcl
terraform {
  required_providers {
    panther = {
      source = "panther-labs/panther"
    }
    aws = {
      source  = "hashicorp/aws"
    }
  }
}
```

### Step 4: Define AWS infrastructure

In AWS, you need to create an S3 bucket and an SNS topic. To ingest logs from an S3 bucket, the bucket must write notifications to an SNS topic on object creation. A subscription on this topic is needed to push object information onto Panther's input queue.

<figure><img src="https://4011785613-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LgdiSWdyJcXPahGi9Rs-2910905616%2Fuploads%2Fgit-blob-8445590731154281af27b2407835de8f061c2860%2FTF(1).png?alt=media" alt="An AWS infrastructure diagram is shown. A Log Source Bucket is pointing to a Panther Notifications Topic, which points to a Panther Input Notifications Queue" width="563"><figcaption></figcaption></figure>

#### Define S3 bucket

The following HCL configuration defines the S3 bucket and associated IAM role for accessing its contents. This role requires read permissions on the S3 bucket, as it will be assumed by your Panther instance to read incoming logs.

```hcl
resource "aws_s3_bucket" "log_bucket" {
  bucket = var.s3_bucket_name
}

resource "aws_iam_role" "log_processing_role" {
  name = "PantherLogProcessingRole-${var.s3_bucket_name}"

  # Policy that grants an entity permission to assume the role
  assume_role_policy = jsonencode({
    Version : "2012-10-17",
    Statement : [
      {
        Action : "sts:AssumeRole",
        Effect : "Allow",
        Principal : {
          AWS : "arn:${var.aws_partition}:iam::${var.panther_aws_account_id}:root"
        }
        Condition : {
          Bool : { "aws:SecureTransport" : true }
        }
      }
    ]
  })

  tags = {
    Application = "Panther"
  }
}


# Provides an IAM role inline policy for reading S3 data
resource "aws_iam_role_policy" "read_data_policy" {
  name = "ReadData"
  

  role = aws_iam_role.log_processing_role.id
  policy = jsonencode({
    Version : "2012-10-17",
    Statement : [
      {
        Effect : "Allow",
        Action : [
          "s3:GetBucketLocation",
          "s3:ListBucket",
        ],
        Resource : "arn:${var.aws_partition}:s3:::${aws_s3_bucket.log_bucket.bucket}"
      },
      {
        Effect : "Allow",
        Action : "s3:GetObject",
        Resource : "arn:${var.aws_partition}:s3:::${aws_s3_bucket.log_bucket.bucket}/*"
    }, ]
  })
}


```

#### Define SNS topic

The following HCL configuration creates the SNS topic and related policy for enabling S3 bucket notifications. It also creates a subscription to forward messages to Panther's input data notifications queue.

The same SNS topic can be used for multiple S3 buckets integrations.

```hcl
resource "aws_sns_topic" "panther_notifications_topic" {
  name = "panther-notifications-topic"
}

resource "aws_sns_topic_policy" "default" {
  arn = aws_sns_topic.panther_notifications_topic.arn

  policy = data.aws_iam_policy_document.panther_notifications_topic_policy.json
}

data "aws_iam_policy_document" "panther_notifications_topic_policy" {
  statement {
    sid = "AllowS3EventNotifications"
    actions = [
      "sns:Publish",
    ]
    effect = "Allow"
    principals {
      type = "Service"
      identifiers = ["s3.amazonaws.com"]
    }
    resources = [
      aws_sns_topic.panther_notifications_topic.arn,
    ]
  }
  statement {
    sid = "AllowCloudTrailNotification"
    actions = [
      "sns:Publish",
    ]
    effect = "Allow"
    principals {
      type = "Service"
      identifiers = ["cloudtrail.amazonaws.com"]
    }
    resources = [
      aws_sns_topic.panther_notifications_topic.arn,
    ]
  }
  statement {
    sid = "AllowSubscriptionToPanther"
    actions = [
      "sns:Subscribe",
    ]
    effect = "Allow"
    principals {
      type = "AWS"
      identifiers = ["arn:${var.panther_aws_partition}:iam::${var.panther_aws_account_id}:root"]
    }
    resources = [
      aws_sns_topic.panther_notifications_topic.arn,
    ]
  }
}

resource "aws_s3_bucket_notification" "panther_event_notifications" {
  bucket = aws_s3_bucket.log_bucket.id

  topic {
    topic_arn = aws_sns_topic.panther_notifications_topic.arn
    events        = ["s3:ObjectCreated:*"]
  }
}

resource "aws_sns_topic_subscription" "panther_notifications_subscription" {
  topic_arn = aws_sns_topic.panther_notifications_topic.arn
  protocol  = "sqs"
  endpoint  = "arn:${var.panther_aws_partition}:sqs:${var.panther_aws_region}:${var.panther_aws_account_id}:panther-input-data-notifications-queue"
  raw_message_delivery = false
}

```

### Step 5: Define Panther S3 log source

The following HCL configuration defines the S3 log source in Panther.

{% hint style="info" %}
Note that `panther_managed_bucket_notifications_enabled` is set to `false`. This indicates that all of the infrastructure related to this log source is being managed externally, in this case through Terraform.
{% endhint %}

```hcl
provider "panther" {
  token = var.panther_api_token
  url   = var.panther_api_url
}

resource "panther_s3_source" "demo_source" {
  aws_account_id                          = var.aws_account_id
  name                                    = var.log_source_name
  log_processing_role_arn                 = aws_iam_role.log_processing_role.arn
  log_stream_type                         = "JSON"
  panther_managed_bucket_notifications_enabled = false
  bucket_name                             = aws_s3_bucket.log_bucket.bucket
  prefix_log_types = [{
    excluded_prefixes = []
    log_types         = ["AWS.CloudTrail"]
    prefix            = ""
  }]
}
```