# Configuring Databricks for Panther

## Overview

This page describes how to configure [Databricks](https://www.databricks.com/) for use as your Panther data storage backend. As you complete the steps below, you will collect and store various configuration values, then provide them to Panther.

{% hint style="warning" %}
You should complete the process on this page only after arriving at [Step 9](https://docs.panther.com/system-configuration/panther-deployment-types/set-up#step-9-if-using-databricks-configure-databricks-for-panther) on [set-up](https://docs.panther.com/system-configuration/panther-deployment-types/cloud-connected/set-up "mention").
{% endhint %}

This process will:

* Create a Databricks workspace for Panther (along with associated Databricks infrastructure in AWS)
* Create an IAM role in AWS to allow Databricks to read from the Panther S3 staging bucket.
* Create an external storage credential.
* Create an external storage integration so Databricks can read data from S3 for loading.
* Create service principals—one for loading (read/write) and one for querying (read-only).
* Create secrets with KMS keys in AWS to hold OAuth credentials for the service principals.
* Create a catalog in Databricks for Panther tables, with permissions for the service principals.
* Create load, optimize, query, and scheduled query warehouses.

## How to configure Databricks for Panther

### Prerequisites

* You have a Databricks account.
* You have completed the instructions on [set-up](https://docs.panther.com/system-configuration/panther-deployment-types/cloud-connected/set-up "mention") and can log in to the Panther Console.
* You are logged into the AWS console in the AWS account you'd like to use for Panther compute. This is needed because Databricks will create a workspace on your behalf.

{% hint style="warning" %}
This AWS account should not be the AWS account where Panther is hosted.
{% endhint %}

* You have the Databricks and AWS permissions listed in the following pages:
  * [Create a classic workspace](https://docs.databricks.com/aws/en/admin/workspace/create-workspace#requirements)
  * [Create a storage credential and external location for S3 using Catalog Explorer or SQL](https://docs.databricks.com/aws/en/connect/unity-catalog/cloud-storage/s3/s3-external-location-manual)
  * [Create a SQL warehouse](https://docs.databricks.com/aws/en/compute/sql-warehouse/create#requirements)

### Step 1: Make a copy of the configuration table

Throughout the configuration process, you'll collect values that you'll send to Panther at the end. To organize theses values, make a copy of the table below.

| Parameter                                 | Value |
| ----------------------------------------- | ----- |
| `databricks_load_role_arn`                |       |
| `databricks_load_secret_kms_key_arn`      |       |
| `databricks_query_secret_kms_key_arn`     |       |
| `databricks_load_secret_arn`              |       |
| `databricks_query_secret_arn`             |       |
| `databricks_catalog`                      |       |
| `databricks_load_warehouse_id`            |       |
| `databricks_optimize_warehouse_id`        |       |
| `databricks_query_warehouse_id`           |       |
| `databricks_scheduled_query_warehouse_id` |       |

### Step 2 (Optional): Create a Databricks workspace

It's recommended to create a dedicated Databricks workspace for Panther, for organizational purposes, but not required. If you'd like to use an existing workspace, skip this step.

{% hint style="info" %}
For additional support while creating a workspace, see the Databricks [Create a workspace with automated configuration](https://docs.databricks.com/aws/en/admin/workspace/create-workspace#create-a-workspace-with-automated-configuration) documentation.
{% endhint %}

1. Log in into the Databricks console.
2. In the left-hand navigation menu, click **Workspaces**.
3. Click **Create workspace**.
4. Fill out the **Create Workspace** modal:
   * **Workspace name**: enter a memorable name.
   * **Region**: select the region that matches your AWS deployment of Panther.
   * **Storage and compute**: select **Use your existing cloud account**.\
     ![An arrow is drawn from a "Workspaces" navigation item to a "Create workspace" button. In the foreground, there is a "Create Workspace" modal.](https://4011785613-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LgdiSWdyJcXPahGi9Rs-2910905616%2Fuploads%2FFP3zY1Ol3VYloFfQSP1Y%2FScreenshot%202025-12-11%20at%2012.36.44%E2%80%AFPM.png?alt=media\&token=46e43128-d9bf-4b88-b05c-395223950470)
5. Click **Continue**.
6. Under **Cloud resources**, in the **Cloud credentials** field, select **Add cloud credentials**.\
   ![Under a "Create workspace" header, there are various form fields, like "Workspace name," "Region," and "Cloud credentials."](https://4011785613-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LgdiSWdyJcXPahGi9Rs-2910905616%2Fuploads%2Fy8IfHW0Sds2v751WaKcT%2FScreenshot%202025-12-11%20at%2012.38.58%E2%80%AFPM.png?alt=media\&token=f58fcbed-1ddb-412b-bdf2-5417655347a0)
7. On the **Add cloud credentials** modal, leave **Add automatically** selected, then click **OK**.
   * Both the **Cloud credentials** and **Cloud storage** fields will be provided an **Add automatically** value. Leave these as-is.
8. Click **Log in to AWS and create workspace**.
9. On the **Review AWS Resources** modal, click **Initiate workspace creation**.
10. On the AWS modal, click **Allow access**.
11. Return to your Databricks browser tab, and wait a few minutes for the new workspace to appear in the **Workspaces** list. When it appears, click **Open** to enter the workspace environment.

### Step 3: Enable variant shredding in your workspace

{% hint style="info" %}
For additional support while enabling variant shredding, see the Databricks [Enable shredding](https://docs.databricks.com/aws/en/delta/variant-shredding#enable-shredding) documentation.
{% endhint %}

1. In your Databricks workspace, in the upper-right corner, click your profile icon, then **Previews**.
2. To the right of **Variant Shredding for Optimized Read Performance on Semi-Structured Data**, click the toggle **On**.\
   ![](https://4011785613-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LgdiSWdyJcXPahGi9Rs-2910905616%2Fuploads%2Fgit-blob-6addcda10edcb8e1981426efdb240fdacb23a337%2FScreenshot%202025-10-30%20at%2011.29.48%E2%80%AFAM.png?alt=media)

### Step 4: Create a Panther role for the storage credential

{% hint style="info" %}
For additional support while creating an IAM role, see the Databricks [Step 1: Create an IAM role](https://docs.databricks.com/aws/en/connect/unity-catalog/cloud-storage/s3/s3-external-location-manual#step-1-create-an-iam-role) documentation.
{% endhint %}

1. In the AWS account where you created the Databricks workspace infrastructure, create an IAM role named `panther-databricks-s3-reader-role-<region>`, accepting all defaults.
2. In your Panther Console, retrieve the **Processed Data Bucket** value:
   1. Click the gear icon (Settings) > **General**.
   2. Click **Data Lake**.
   3. Under **Databricks Configuration**, copy the **Processed Data Bucket** value.\
      ![](https://4011785613-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LgdiSWdyJcXPahGi9Rs-2910905616%2Fuploads%2Fgit-blob-5be9237e052ed673e513d5b50f47b9f854ce561a%2FScreenshot%202025-10-30%20at%2011.11.47%E2%80%AFAM.png?alt=media)
3. Update the role's trust relationship:
   1. In the AWS console, in the **Roles** list, click the newly created role to view its details page.
   2. Click **Trust relationships**.
   3. Click **Edit trust policy**.
   4. Replace the JSON in the code editor with the JSON below:<br>

      <div data-gb-custom-block data-tag="hint" data-style="info" class="hint hint-info"><p>The following trust policy sets <code>"sts:ExternalId": "TBD"</code> as a placeholder—you will update this later. You will also later add a self-assumption statement.</p></div>

      ```json
      {
          "Version": "2012-10-17",
          "Statement": [
              {
                  "Effect": "Allow",
                  "Principal": {
                      "AWS": [
                          "arn:aws:iam::414351767826:role/unity-catalog-prod-UCMasterRole-14S5ZJVKOTYTL",
                          "arn:aws:iam::<your account for this role>:role/panther-databricks-s3-reader-role-<region>"
                      ]
                  },
                  "Action": "sts:AssumeRole",
                  "Condition": {
                      "StringEquals": {
                          "sts:ExternalId": "TBD"
                      }
                  }
              }
          ]
      }
      ```
   5. Click **Update policy.**
4. Update the role's permissions:
   1. On the role's details page, click **Permissions**.
   2. Click **Add permissions** > **Create inline policy**.
   3. In the **Policy editor** section, click **JSON**.
   4. Replace the JSON in the code editor with the JSON below:<br>

      <div data-gb-custom-block data-tag="hint" data-style="info" class="hint hint-info"><p>In the policy below, replace <code>&#x3C;Processed Data Bucket from Panther settings></code> with the <strong>Processed Data Bucket</strong> value you retrieved above.</p></div>

      ```json
      { 	
      "Statement": [ 		
        { 			
         "Action": [ 				
           "s3:ListBucket", 				
            "s3:GetBucketLocation" 			
         ], 			
         "Effect": "Allow", 			
         "Resource": "arn:aws:s3:::<Processed Data Bucket from Panther settings>" 		
       }, 		
       { 			
         "Action": "s3:GetObject", 			
          "Effect": "Allow", 			
          "Resource": "arn:aws:s3:::<Processed Data Bucket from Panther settings>/*" 		
       } 	
       ], 	
       "Version": "2012-10-17" 
      }
      ```
   5. Click **Next**.
   6. Under **Policy details**, enter a **Policy name**.
   7. Click **Create policy**.
5. On the role's details page, copy the **ARN**, and add it as the `databricks_load_role_arn` value in your [configuration table](#step-1-make-a-copy-of-the-configuration-table).
   * Leave the browser window with the role details page open, as you will return to it in Step 5.

### Step 5: Create a storage credential

{% hint style="info" %}
For additional support while creating a storage credential, see the Databricks [Step 2: Give Databricks the IAM role details](https://docs.databricks.com/aws/en/connect/unity-catalog/cloud-storage/s3/s3-external-location-manual#step-2-give-databricks-the-iam-role-details) documentation.
{% endhint %}

Create a Databricks [storage credential](https://docs.databricks.com/aws/en/sql/language-manual/sql-ref-storage-credentials#credential) to represent the AWS IAM role you just created:

1. In your Databricks workspace, click **Catalog**, then **External Data**.\
   ![](https://4011785613-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LgdiSWdyJcXPahGi9Rs-2910905616%2Fuploads%2Fgit-blob-7c66d9e3150d25358ef4f8a87e5a9ea46a56c582%2FScreenshot%202025-10-30%20at%2012.31.13%E2%80%AFPM.png?alt=media)
2. Click **Credentials**.
3. Click **Create credential**.
4. Fill in the **Create a new credential** form:
   1. **Credential Type**: select **AWS IAM Role**.
   2. **Credential name**: enter `panther-storage-credential`.
   3. **IAM role (ARN)**: enter the ARN of the IAM role you created above (which is `databricks_load_role_arn` in the [configuration table](#step-1-make-a-copy-of-the-configuration-table)).
5. Click **Create**.
   * On the **Credential created** page, copy the **External ID** value, and store it in a secure location, as you will need it in the next step.

### Step 6: Update the IAM role trust relationship policy

{% hint style="info" %}
For additional support while updating the IAM role, see the Databricks [Step 3: Update the IAM role trust relationship policy](https://docs.databricks.com/aws/en/connect/unity-catalog/cloud-storage/s3/s3-external-location-manual#step-3-update-the-iam-role-trust-relationship-policy) documentation.
{% endhint %}

1. Return to the AWS console, to the details page for the `panther-databricks-s3-reader-role-<region>` IAM role you created above.
2. Click **Trust relationships**.
3. Click **Edit trust policy**.
4. In the `"sts:ExternalId": "TBD"` line, replace `TBD` with the **External ID** value you copied in Databricks above.
5. Click **Update policy**.

### Step 7: Create an external storage location

{% hint style="info" %}
For additional support while updating the IAM role, see the Databricks [Create an external location for an AWS S3 bucket](https://docs.databricks.com/aws/en/connect/unity-catalog/cloud-storage/s3/s3-external-location-manual#-create-an-external-location-for-an-aws-s3-bucket) documentation.
{% endhint %}

1. In your Databricks workspace, click **Catalog**, then **External Data**.\
   ![](https://4011785613-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LgdiSWdyJcXPahGi9Rs-2910905616%2Fuploads%2Fgit-blob-7c66d9e3150d25358ef4f8a87e5a9ea46a56c582%2FScreenshot%202025-10-30%20at%2012.31.13%E2%80%AFPM.png?alt=media)
2. Click **Create external location**.
3. Click **Manual**, then **Next**.
4. Fill in the **Create a new external location manually** form:
   * **External location name**: enter `panther-processed-data`.
   * **Storage type**: select **S3.**
   * **URL**: enter the **Processed Data Bucket** value you retrieved from the Settings page in the Panther Console in Step 3.
   * **Storage credential**: select `panther-storage-credential`.
5. Click **Create**.
6. You will be routed to a page with a **Permission Denied** warning box—click **Force create**.\
   ![](https://4011785613-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LgdiSWdyJcXPahGi9Rs-2910905616%2Fuploads%2Fgit-blob-14e6e1ccf829f5587ed9e573de2965cb2dfee357%2FScreenshot%202025-10-31%20at%2012.45.58%E2%80%AFPM.png?alt=media)

### Step 8: Create a load service principal in Databricks

1. Access your Databricks workspace settings:
   1. In the upper-right corner, click your initial.
   2. Click **Settings**.\
      ![](https://4011785613-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LgdiSWdyJcXPahGi9Rs-2910905616%2Fuploads%2Fgit-blob-d5d21f8e58937167fbd9056eb13de6c408bd6f7b%2Funknown.png?alt=media)
2. In the **Settings** navigation bar, under **Workspace admin**, click **Identity and access**.
3. To the right of **Service principals**, click **Manage**.\
   ![](https://4011785613-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LgdiSWdyJcXPahGi9Rs-2910905616%2Fuploads%2Fgit-blob-df9072e7689801a9d7cc031688b04da9f67c88e9%2FScreenshot%202025-10-30%20at%2012.51.56%E2%80%AFPM.png?alt=media)
4. Click **Add service principal**.
5. In the **Add service principal** modal, click **Add new**.
6. In the **Service principal name** field, enter `panther-load`.
7. Click **Add**.
8. In the table, click **panther-load** to view its details page.
9. Click **Secrets**.
10. Click **Generate secret**.
11. Under **Lifetime (days)**, enter `730` (the maximum).
12. Click **Generate**.
13. Copy the **Secret** and **Client ID** values and store them in a secure location, as you'll need them in a later step (as an alternative to copying these values, you can leave this browser tab open).

### Step 9: Create a load secret KMS key in AWS

1. In your AWS console, ensure you are in the correct region. Navigate to [Key Management Service](https://aws.amazon.com/kms/).
2. In the left-hand navigation menu, click **Customer managed keys**.
3. Click **Create Key**.
4. Under **Key type**, select **Symmetric**. Under **Key usage**, select **Encrypt and decrypt**.
5. Click **Next**
6. Enter an **Alias** value, then click **Next**.
7. Under **Key administrators**, optionally select users and/or roles, then click **Next**.
8. On the **Define key usage permissions&#x20;*****- optiona**l* page, under **Other AWS accounts**, click **Add another AWS account**.
   1. In the field that appears, enter the AWS account ID for the account your Panther deployment is in. You can find this value in the Panther Console, in the [general settings footer](https://docs.panther.com/system-configuration/..#general-settings).
   2. Click **Next**.
9. Switch to a browser tab with the Panther Console open, and retrieve the **Delta Controller Role ARN** and **Delta Admin Role ARN** values:
   1. Click the gear icon (Settings) > **General**.
   2. Click **Data Lake**.
   3. Under **Databricks Configuration**, note the **Delta Controller Role ARN** and **Delta Admin Role ARN** values.\
      ![](https://4011785613-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LgdiSWdyJcXPahGi9Rs-2910905616%2Fuploads%2Fgit-blob-cf523729c1aa1d9e8d34f225f788e4cd55566b1b%2FScreenshot%202025-10-30%20at%2011.11.47%E2%80%AFAM.png?alt=media)
10. In the AWS console, under **Key policy**, click **Edit**, then replace the JSON in the code editor with the JSON below:

{% hint style="info" %}
In the policy below, replace:

* `<Delta Controller Role ARN from Panther settings>` with the **Delta Controller Role ARN** value you retrieved above
* `<Delta Admin Role ARN from Panther settings>` with the **Delta Admin Role ARN** value you retrieved above
* `<AWS Account ID you are working in>` with the Account ID of the account you are working in
  {% endhint %}

```json
{
	"Version": "2012-10-17",
	"Statement": [
		{
			"Sid": "Panther",
			"Effect": "Allow",
			"Principal": {
				"AWS": [
					"<Delta Controller Role ARN from Panther settings>",
					"<Delta Admin Role ARN from Panther settings>"
				]
			},
			"Action": "kms:Decrypt",
			"Resource": "*"
		},
		{
			"Sid": "root",
			"Effect": "Allow",
			"Action": [
				"kms:*"
			],
			"Resource": "*",
			"Principal": {
				"AWS": "arn:aws:iam::<AWS Account ID you are working in>:root"
			}
		}
	]
}
```

11. Click **Next**.
12. On the **Review** page, review the configuration, then click **Finish**.
13. In the Customer managed keys list, click the alias of the key you just created, to view its detail page.
14. Copy the key ARN into the table above for the `databricks_load_secret_kms_key_arn` row.

{% hint style="info" %}
In [Step 12](#step-12-optional-create-a-query-secret-kms-key), you will either reuse this KMS key or create an additional one.
{% endhint %}

### Step 10: Create a load secret in AWS

1. In your AWS console, ensure you are in the correct region. Navigate to [Secrets Manager](https://docs.aws.amazon.com/secretsmanager/latest/userguide/intro.html).

{% hint style="warning" %}
You should not be in the AWS account hosting your Panther infrastructure.
{% endhint %}

2. Click **Store a new secret**.
3. Under **Secret type**, select **Other type of secret**.
4. Under **Key/value pairs**, in the **Key/value** tab, enter the following key value pairs:

   <table data-header-hidden><thead><tr><th width="217.18145751953125">Key</th><th>Value</th></tr></thead><tbody><tr><td>Key</td><td>Value</td></tr><tr><td><code>secret</code></td><td>&#x3C;the <strong>Secret</strong> value you generated in Databricks in <a href="#step-8-create-a-load-service-principal-in-databricks">Step 8</a>></td></tr><tr><td><code>client-id</code></td><td>&#x3C;the <strong>Client ID</strong> value you generated in Databricks in <a href="#step-8-create-a-load-service-principal-in-databricks">Step 8</a>></td></tr><tr><td><code>databricks-host</code></td><td>&#x3C;the URL of your Databricks workspace><br><br>While viewing the workspace you created above in your Databricks console, copy the URL of the page. For example, <code>https://dbc-023ca860-3666.cloud.databricks.com</code></td></tr></tbody></table>
5. Under **Encryption key**, select the `databricks_load_secret_kms_key_arn` KMS key you created in the previous step.
6. Click **Next**.
7. In the **Secret name** field, enter `panther-databricks-admin-access`, then click **Next**.
8. Without making any changes on the **Configure rotation&#x20;*****- optional*** page, click **Next**.
9. Review the secret settings, then click **Store**.
10. In the **Secrets** list, click **panther-databricks-admin-access**, to view its details page.
11. In the **Resource permissions** tile, click **Edit permissions**.
12. Under **Resource permissions**, replace the JSON in the code editor with the JSON below:

{% hint style="info" %}
In the policy below:

* Replace `<Delta Controller Role ARN from Panther settings>` with the **Delta Controller Role ARN** value you retrieved above
* Replace `<Delta Admin Role ARN from Panther settings>` with the **Delta Admin Role ARN** value you retrieved above
* If you will reuse the load secret KMS key you created in [Step 9](#step-9-create-a-load-secret-kms-key-in-aws) in [Step 12](#step-12-optional-create-a-query-secret-kms-key), replace `<Databricks Role ARN from Panther settings>` with the **Databricks Role ARN** value you retrieved above
* If you will not reuse the load secret KMS key you created in [Step 9](#step-9-create-a-load-secret-kms-key-in-aws) (i.e., you will create a new query secret KMS key in [Step 12](#step-12-optional-create-a-query-secret-kms-key)), remove `"<Databricks Role ARN from Panther settings>"` completely
  {% endhint %}

```json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Panther",
      "Effect": "Allow",
      "Principal": {
        "AWS": [
            "<Delta Controller Role ARN from Panther settings>",
            "<Delta Admin Role ARN from Panther settings>",
            "<Databricks Role ARN from Panther settings>"
        ]
      },
      "Action": "secretsmanager:GetSecretValue",
      "Resource": "*"
    }
  ]
}
```

13. Click **Save**.
14. Copy the ARN of the newly created secret and add it as the `databricks_load_secret_arn` value in your [configuration table](#step-1-make-a-copy-of-the-configuration-table).
15. In the Databricks console, return to the **External Data** page (click **Catalog** > **External Data**).
16. Under **External Locations**, click the **panther-processed-data** location you created above.
17. Click **Permissions**.
18. Click **Grant**.
19. Under **Principals**, search for and select **panther-load**.
20. Under **Privileges**, check the boxes for **BROWSE** and **READ FILES**.
21. Click **Confirm**.

### Step 11: Create a query service principal in Databricks

1. Access your Databricks workspace settings:
   1. In the upper-right corner, click your initial.
   2. Click **Settings**.\
      ![](https://4011785613-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LgdiSWdyJcXPahGi9Rs-2910905616%2Fuploads%2Fgit-blob-d5d21f8e58937167fbd9056eb13de6c408bd6f7b%2Funknown.png?alt=media)
2. In the **Settings** navigation bar, under **Workspace admin**, click **Identity and access**.
3. To the right of **Service principals**, click **Manage**.\
   ![](https://4011785613-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LgdiSWdyJcXPahGi9Rs-2910905616%2Fuploads%2Fgit-blob-df9072e7689801a9d7cc031688b04da9f67c88e9%2FScreenshot%202025-10-30%20at%2012.51.56%E2%80%AFPM.png?alt=media)
4. Click **Add service principal**.
5. In the **Add service principal** modal, click **Add new**.
6. In the **Service principal name** field, enter `panther-query`.
7. Click **Add**.
8. In the table, click **panther-query** to view its details page.
9. Click **Secrets**.
10. Click **Generate secret**.
11. Under **Lifetime (days)**, enter `730` (the maximum).
12. Click **Generate**.
13. Copy the **Secret** and **Client ID** values and store them in a secure location, as you'll need them in a later step (as an alternative to copying these values, you can leave this browser tab open).

### Step 12 (Optional): Create a query secret KMS key

In the next step, you'll create an additional secret in AWS. You can either create a new KMS key to associate to this secret, or reuse the KMS key you created in Step 9 (added to your configuration table as `databricks_load_secret_kms_key_arn`).

* If you'd like to reuse the KMS key you created above, copy the value of `databricks_load_secret_kms_key_arn` to `databricks_query_secret_kms_key_arn` in the configuration table above.
* If you'd like to create a new KMS key, repeat [Step 9: Create a load secret KMS key in AWS](#step-9-create-a-load-secret-kms-key-in-aws), then add the ARN for the key as `databricks_query_secret_kms_key_arn` in the configuration table above.

### Step 13: Create a query secret in AWS

1. In your AWS console, ensure you are in the correct region. Navigate to [Secrets Manager](https://docs.aws.amazon.com/secretsmanager/latest/userguide/intro.html).

{% hint style="warning" %}
This should NOT be created in the AWS account hosting your Panther infrastructure.
{% endhint %}

2. Click **Store a new secret**.
3. Under **Secret type**, select **Other type of secret**.
4. Under **Key/value pairs**, in the **Key/value** tab, enter the following key value pairs:

   <table data-header-hidden><thead><tr><th width="217.18145751953125">Key</th><th>Value</th></tr></thead><tbody><tr><td>Key</td><td>Value</td></tr><tr><td><code>secret</code></td><td>&#x3C;the <strong>Secret</strong> value you generated in Databricks in <a href="#step-11-create-a-query-service-principal-in-databricks">Step 11</a>></td></tr><tr><td><code>client-id</code></td><td>&#x3C;the <strong>Client ID</strong> value you generated in Databricks in <a href="#step-11-create-a-query-service-principal-in-databricks">Step 11</a>></td></tr><tr><td><code>databricks-host</code></td><td>&#x3C;the URL of your Databricks workspace><br><br>While viewing your Databricks workspace in the Databricks console, copy the URL of the page. For example, <code>https://dbc-023ca860-3666.cloud.databricks.com</code></td></tr></tbody></table>
5. Under **Encryption key**, select the `databricks_query_secret_kms_key_arn` KMS key you created in the previous step (or the `databricks_load_secret_kms_key_arn` KMS key, if you are reusing that one).
6. Click **Next**.
7. In the **Secret name** field, enter `panther-databricks-query-access`, then click **Next**.
8. Without making any changes on the **Configure rotation&#x20;*****- optional*** page, click **Next**.
9. Review the settings, then click **Store**.
10. In the **Secrets** list, click **panther-databricks-query-access**, to view its details page.
11. In the **Resource permissions** tile, click **Edit permissions**.
12. Under **Resource permissions**, replace the JSON in the code editor with the JSON below:

{% hint style="info" %}
In the policy below, replace:

* `<Databricks Role ARN from Panther settings>` with the **Databricks Role ARN** value you retrieved above
  {% endhint %}

```json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Panther",
      "Effect": "Allow",
      "Principal": {
        "AWS": [
            "<Databricks Role ARN from Panther settings>"
        ]
      },
      "Action": "secretsmanager:GetSecretValue",
      "Resource": "*"
    }
  ]
}
```

13. Click **Save**.
14. Copy the ARN of the newly created secret and add it as the `databricks_query_secret_arn` value in your [configuration table](#step-1-make-a-copy-of-the-configuration-table).

### Step 14: Create an S3 bucket and external location

1. In your AWS console, ensure you are in the correct region. Navigate to [S3](https://docs.aws.amazon.com/AmazonS3/latest/userguide/Welcome.html).

{% hint style="warning" %}
You should not be in the AWS account hosting your Panther infrastructure.
{% endhint %}

2. Click **Create bucket**.
3. Enter a **Bucket name**.
4. Click **Create bucket**.
5. In the Databricks workspace you created above, click **Catalog**, then **External Data**.\
   ![](https://4011785613-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LgdiSWdyJcXPahGi9Rs-2910905616%2Fuploads%2Fgit-blob-7c66d9e3150d25358ef4f8a87e5a9ea46a56c582%2FScreenshot%202025-10-30%20at%2012.31.13%E2%80%AFPM.png?alt=media)
6. Click **Create external location**.
7. Click **AWS Quickstart (Recommended)**, then **Next**.
8. In the **Bucket Name** field, enter the name of the bucket you just created.
9. Under **Personal Access Token**, click **Generate new token**.
   * Copy this value, as you'll need it in the follow steps. Alternatively, you can leave this page open.
10. Click **Launch in Quickstart**.
    * A new browser tab will open in AWS, on a **Quick create stack** screen with the CloudFormation template pre-loaded.
11. In the **Parameters** section, in the **Databricks Personal Access Token** field, enter the **Personal Access Token** you generated above in Databricks.
12. Click **Create stack**.
13. After the stack has completed deploying, return to your Databricks console browser tab. On the **Create external location with Quickstart** screen, click **Ok**.\
    ![](https://4011785613-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LgdiSWdyJcXPahGi9Rs-2910905616%2Fuploads%2Fgit-blob-ea25a95983540cd9f8bdbea308848423d4973758%2FScreenshot%202025-11-13%20at%201.51.00%E2%80%AFPM.png?alt=media)
    * Verify that the **External Locations** list contains the one you just created.

### Step 15: Create a Databricks catalog

1. In your Databricks workspace, click **Catalog**.
2. Click **Add data** > **Create a catalog**.\
   ![](https://4011785613-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LgdiSWdyJcXPahGi9Rs-2910905616%2Fuploads%2Fgit-blob-db7d214539e16b1700c4b47bcfdf437d380f9732%2FScreenshot%202025-11-05%20at%2012.08.38%E2%80%AFPM.png?alt=media)
3. Fill in the **Create a new catalog** form:
   * **Catalog name**: enter a name for your catalog, e.g., `panther`.

     <div data-gb-custom-block data-tag="hint" data-style="info" class="hint hint-info"><p>It's recommended to name the catalog <code>panther</code>, but not required.</p></div>
   * **Type**: select **Standard**.
   * **Select external location**: choose the external location you created in [Step 14](#step-14-create-an-s3-bucket-and-external-location).

     <div data-gb-custom-block data-tag="hint" data-style="warning" class="hint hint-warning"><p>Do not choose <strong>panther-processed-data</strong>.</p></div>

     \
     ![](https://4011785613-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LgdiSWdyJcXPahGi9Rs-2910905616%2Fuploads%2Fgit-blob-860d8df774dc6201495017a87c86574b2126bb46%2FScreenshot%202025-11-14%20at%2010.30.48%E2%80%AFAM.png?alt=media)
4. Click **Create**.
5. On the **Catalog created!** modal, click **View catalog**.
6. Click **Permissions**.
7. Click **Grant**.
8. In the **Grant on panther** modal, fill in the form:
   * **Principals**: type and select `panther-load`.
   * Select the following permissions:
     * **USE CATALOG**
     * **USE SCHEMA**
     * **BROWSE**
     * **SELECT**
     * **MODIFY**
     * **CREATE SCHEMA**
     * **CREATE TABLE**\
       ![](https://4011785613-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LgdiSWdyJcXPahGi9Rs-2910905616%2Fuploads%2Fgit-blob-394374ffa67af1e3d59f9bcc845bbb473a62c4f4%2FScreenshot%202025-11-14%20at%2010.48.29%E2%80%AFAM.png?alt=media)
9. Click **Confirm**.
10. Click **Grant**.
11. In the **Grant on panther** modal, fill in the form:
    * **Principals**: type and select `panther-query`.
    * Select the following permissions:
      * **USE CATALOG**
      * **USE SCHEMA**
      * **BROWSE**
      * **SELECT**\
        ![](https://4011785613-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LgdiSWdyJcXPahGi9Rs-2910905616%2Fuploads%2Fgit-blob-0ce210c66af22724c02b0b7a58bd413213f8295a%2FScreenshot%202025-11-14%20at%2010.54.34%E2%80%AFAM.png?alt=media)
12. Click **Confirm**.
13. Add the catalog name as the `databricks_catalog` value in your [configuration table](#step-1-make-a-copy-of-the-configuration-table).

### Step 16: Create a panther-load SQL warehouse

{% hint style="info" %}
For additional SQL warehouse creation support, see the Databricks [Create a SQL warehouse](https://docs.databricks.com/aws/en/compute/sql-warehouse/create) documentation.
{% endhint %}

1. In your Databricks workspace, click **Compute**.
2. Click **SQL warehouses**.
3. Click **Create SQL warehouse**.
4. Fill out the **New SQL warehouse** form:
   * **Name**: enter `panther-load`.
   * **Cluster size**: select **2X-Small**.
   * **Scaling**: set the **Max** value to `40` (the maximum allowed).
   * **Type**: select **Pro**.

{% hint style="warning" %}
Do not use **Serverless**.
{% endhint %}

5. Click **Create**.
6. In the **Manage permissions** modal, add the **panther-load** user, then select **Can use** permissions.\
   ![](https://4011785613-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LgdiSWdyJcXPahGi9Rs-2910905616%2Fuploads%2Fgit-blob-cf4c503fdbc8ac6265c57d15a44659089505cdb7%2FScreenshot%202025-11-05%20at%201.34.24%E2%80%AFPM.png?alt=media)
7. Click **Add**.
8. Click the **X** in the upper-right corner to close the **Manage permissions** modal.
9. On the **panther-load** warehouse details page, copy the **ID** (next to the name) and add it as the `databricks_load_warehouse_id` value in your [configuration table](#step-1-make-a-copy-of-the-configuration-table).\
   ![](https://4011785613-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LgdiSWdyJcXPahGi9Rs-2910905616%2Fuploads%2Fgit-blob-48716b09f2db92922980be4ce1f71cadc99112d5%2FScreenshot%202025-11-05%20at%201.38.43%E2%80%AFPM.png?alt=media)

### Step 17: Create a panther-optimize SQL warehouse

This warehouse runs nightly table maintenance jobs.

{% hint style="info" %}
For additional SQL warehouse creation support, see the Databricks [Create a SQL warehouse](https://docs.databricks.com/aws/en/compute/sql-warehouse/create) documentation.
{% endhint %}

1. In your Databricks workspace, click **Compute**.
2. Click **SQL warehouses**.
3. Click **Create SQL warehouse**.
4. Fill out the **New SQL warehouse** form:
   * **Name**: enter `panther-optimize`.
   * **Cluster size**: select **2X-Small**.
   * **Scaling**: set the **Max** value to `40` (the maximum allowed).
   * **Type**: select **Serverless**.
5. Click **Create**.
6. In the **Manage permissions** modal, add the **panther-load** user, then select **Can use** permissions.\
   ![](https://4011785613-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LgdiSWdyJcXPahGi9Rs-2910905616%2Fuploads%2Fgit-blob-cf4c503fdbc8ac6265c57d15a44659089505cdb7%2FScreenshot%202025-11-05%20at%201.34.24%E2%80%AFPM.png?alt=media)
7. Click **Add**.
8. Click the **X** in the upper-right corner to close the **Manage permissions** modal.
9. On the **panther-optimize** warehouse details page, copy the **ID** (next to the name) and add it as the `databricks_optimize_warehouse_id` value in your [configuration table](#step-1-make-a-copy-of-the-configuration-table).\
   ![](https://4011785613-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LgdiSWdyJcXPahGi9Rs-2910905616%2Fuploads%2Fgit-blob-c6697074956cf9040ce4a69b363516f8d549e8d9%2FScreenshot%202025-11-05%20at%201.44.47%E2%80%AFPM.png?alt=media)

### Step 18: Create a panther-query SQL warehouse

{% hint style="info" %}
For additional SQL warehouse creation support, see the Databricks [Create a SQL warehouse](https://docs.databricks.com/aws/en/compute/sql-warehouse/create) documentation.
{% endhint %}

1. In your Databricks workspace, click **Compute**.
2. Click **SQL warehouses**.
3. Click **Create SQL warehouse**.
4. Fill out the **New SQL warehouse** form:
   * **Name**: enter `panther-query`.
   * **Cluster size**: select **Medium**.
   * **Scaling**: set the **Max** value to `40` (the maximum allowed).
   * **Type**: select **Serverless** or **Pro**.

{% hint style="info" %}
This SQL warehouse can be **Serverless** or **Pro**, but **Serverless** is recommended. **Pro** warehouses start up slowly.
{% endhint %}

5. Click **Create**.
6. In the **Manage permissions** modal, add the **panther-query** user, then select **Can use** permissions.\
   ![](https://4011785613-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LgdiSWdyJcXPahGi9Rs-2910905616%2Fuploads%2Fgit-blob-f1b9a840a4f1fd234c2a3ad47ee78d9ea2ce093b%2FScreenshot%202025-11-05%20at%201.50.14%E2%80%AFPM.png?alt=media)
7. Click **Add**.
8. Click the **X** in the upper-right corner to close the **Manage permissions** modal.
9. On the **panther-query** warehouse details page, copy the **ID** (next to the name) and add it as the `databricks_query_warehouse_id` value in your [configuration table](#step-1-make-a-copy-of-the-configuration-table).\
   ![](https://4011785613-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LgdiSWdyJcXPahGi9Rs-2910905616%2Fuploads%2Fgit-blob-d3871aca92acc0f84205e15b79b83ff404fa306e%2FScreenshot%202025-11-05%20at%201.51.10%E2%80%AFPM.png?alt=media)

### Step 19: Create a panther-scheduled-query SQL warehouse

{% hint style="info" %}
For additional SQL warehouse creation support, see the Databricks [Create a SQL warehouse](https://docs.databricks.com/aws/en/compute/sql-warehouse/create) documentation.
{% endhint %}

1. In your Databricks workspace, click **Compute**.
2. Click **SQL warehouses**.
3. Click **Create SQL warehouse**.
4. Fill out the **New SQL warehouse** form:
   * **Name**: enter `panther-scheduled-query`.
   * **Cluster size**: select **3X-Large**.
   * **Scaling**: set the **Max** value to `40` (the maximum allowed).
   * **Type**: select **Serverless** or **Pro**.

{% hint style="info" %}
This SQL warehouse can be **Serverless** or **Pro**, but **Serverless** is recommended. **Pro** warehouses start up slowly.
{% endhint %}

5. Click **Create**.
6. In the **Manage permissions** modal, add the **panther-query** user, then select **Can use** permissions.\
   ![](https://4011785613-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LgdiSWdyJcXPahGi9Rs-2910905616%2Fuploads%2Fgit-blob-f1b9a840a4f1fd234c2a3ad47ee78d9ea2ce093b%2FScreenshot%202025-11-05%20at%201.50.14%E2%80%AFPM.png?alt=media)
7. Click **Add**.
8. Click the **X** in the upper-right corner to close the **Manage permissions** modal.
9. On the **panther-scheduled-query** warehouse details page, copy the **ID** (next to the name) and add it as the `databricks_scheduled_query_warehouse_id` value in your [configuration table](#step-1-make-a-copy-of-the-configuration-table).\
   ![](https://4011785613-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LgdiSWdyJcXPahGi9Rs-2910905616%2Fuploads%2Fgit-blob-050d443fa678787a928bd8f03b50c8edf6dfa19f%2FScreenshot%202025-11-05%20at%202.00.40%E2%80%AFPM.png?alt=media)

### Step 20: Send configuration values to Panther

* Now that the [configuration table you created in Step 1](#step-1-make-a-copy-of-the-configuration-table) is completely filled-in, share it with the Panther team.

### Step 21: Return to the post-setup recommendations

* Return to the [Post-setup recommendations](https://docs.panther.com/system-configuration/panther-deployment-types/set-up#post-setup-recommendations) on [set-up](https://docs.panther.com/system-configuration/panther-deployment-types/cloud-connected/set-up "mention").