Sync from S3 Source
This feature is available in version 1.28 and newer.

Set up a Lookup Table

Example scenario: Let's say you want to know what groups and permission levels are associated with the employees at your company. In this scenario, your company has an AWS S3 source with an up-to-date copy of their Active Directory listing that includes groups and permissions information.
To configure the Lookup Table, follow these steps in your Panther Console:
  1. 1.
    From the left sidebar, click Enrichment > Lookup Tables.
  2. 2.
    In the upper right side of the page, click Create New to add a new Lookup Table.
  3. 3.
    Configure the Lookup Table Basic Information:
    • Enter a descriptive Lookup Name.
    • Enter a Description (optional) and a Reference (optional).
      • Description is meant for content about the table, while Reference can be used to hyperlink to an internal resource.
    • Make sure the Enabled? toggle is set to Yes.
      • Note: This is required to import your data later in this process.
  4. 4.
    Click Continue.
  5. 5.
    Configure the Associated Log Types:
    • Select the Log Type from the dropdown.
    • Type in the name of the Selectors, the foreign key fields from the log type you want enriched with your Lookup Table.
    • Click Add Log Type to add another if needed.
      In the example screen shot above, we selected AWS.VPCFlow logs and typed in account to represent keys in the VPC Flow logs.
  6. 6.
    Click Continue.
  7. 7.
    Configure the Table Schema. Note: If you have not already created a new schema, please see our documentation on creating schemas. Once you have created a schema, you will be able to select it from the dropdown on the Table Schema page while configuring a Lookup Table.
    1. 1.
      Select a Schema Name from the dropdown.
    2. 2.
      Select a Primary Key Name from the dropdown. This should be a unique column on the table, such as accountID.
  8. 8.
    Click Continue.
  9. 9.
    On the "Choose Import Method" page, click Set Up next to "Sync Data from an S3 Bucket."
  10. 10.
    Set up your S3 source.
    • Enter the Account ID, the 12-digit AWS Account ID where the S3 bucket is located.
    • Enter the S3 URI, the unique path that identifies the specific S3 bucket.
    • Optionally, enter the KMS Key if your data is encrypted using KMS-SSE.
    • Enter the Update Period, the cadence your S3 source gets updated (defaulted to 1 hour).
  11. 11.
    Click Continue.
  12. 12.
    Set up an IAM Role.
    • Please see the next section, Creating an IAM Role, for instructions on the three options available to do this.
  13. 13.
    Click Finish Setup. A source setup success page will populate.
  14. 14.
    Optionally, next to to Set an alarm in case this lookup table doesn't receive any data?, toggle the setting to YES to enable an alarm.
    • Fill in the Number and Period fields to indicate how often Panther should send you this notification.
    • The alert destinations for this alarm are displayed at the bottom of the page. To configure and customize where your notification is sent, see documentation on Panther Destinations.
Note: Notifications generated for a Lookup Table upload failing are accessible in the System Errors tab within the Alerts & Errors page in the Panther Console.

Creating an IAM Role

There are three options for creating an IAM Role to use with your Panther Lookup Table using an S3 source:
The Set Up an IAM Role screen shows options for how to can set up your IAM role.

Create an IAM role using AWS Console UI

  1. 1.
    On the "Set Up an IAM role" page, during the process of creating a Lookup Table with an S3 source, locate the tile labeled "Using the AWS Console UI". On the right side of the tile, click Select.
  2. 2.
    Click Launch Console UI.
    • You will be redirected to the AWS console in a new browser tab, with the template URL pre-filled.
    • The CloudFormation stack will create an AWS IAM role with the minimum required permissions to read objects from your S3 bucket.
    • Click the "Outputs" tab of the CloudFormation stack in AWS, and note the Role ARN.
  3. 3.
    Navigate back to your Panther account.
  4. 4.
    On the "Use AWS UI to set up your role" page, enter the Role ARN.
  5. 5.
    Click Finish Setup.
Create an IAM role using CloudFormation Template File
  1. 1.
    On the "Set Up an IAM role" page, during the process of creating a Lookup Table with an S3 source, locate the tile labeled "CloudFormation Template File". On the right side of the tile, click Select.
  2. 2.
    Click CloudFormation template, which downloads the template to apply it through your own pipeline.
  3. 3.
    Upload the template file in AWS:
    1. 1.
      Open your AWS console and navigate to the CloudFormation product.
    2. 2.
      Click Create stack.
    3. 3.
      Click Upload a template file and select the CloudFormation template you downloaded.
  4. 4.
    On the "CloudFormation Template" page in Panther, enter the Role ARN.
  5. 5.
    Click Finish Setup.
Create an IAM role manually
  1. 1.
    On the "Set Up an IAM role" page, during the process of creating a Lookup Table with an S3 source, click the link that says I want to set everything up on my own.
  2. 2.
    Create the required IAM role. You may create the required IAM role manually or through your own automation. The role must be named using the format PantherLUTsRole-${Suffix}(e.g., PantherLUTsRole-MyLookupTable).
    • The IAM role policy must include the statements defined below:
      "Version": "2012-10-17",
      "Statement": [
      {
      "Action": "s3:GetBucketLocation",
      "Resource": "arn:aws:s3:::<bucket-name>",
      "Effect": "Allow"
      },
      {
      "Action": "s3:GetObject",
      "Resource": "arn:aws:s3:::<bucket-name>/<input-file-path>",
      "Effect": "Allow"
      }
      ]
      }
    • If your S3 bucket is configured with server-side encryption using AWS KMS, you must include an additional statement granting the Panther API access to the corresponding KMS key. In this case, the policy will look something like this:
      "Version": "2012-10-17",
      "Statement": [
      {
      "Action": "s3:GetBucketLocation",
      "Resource": "arn:aws:s3:::<bucket-name>",
      "Effect": "Allow"
      },
      {
      "Action": "s3:GetObject",
      "Resource": "arn:aws:s3:::<bucket-name>/<input-file-path>",
      "Effect": "Allow"
      },
      {
      "Action": ["kms:Decrypt", "kms:DescribeKey"],
      "Resource": "arn:aws:kms:<region>:<your-accound-id>:key/<kms-key-id>",
      "Effect": "Allow"
      }
      ]
      }
  3. 3.
    On the "Setting up role manually" page in Panther, enter the Role ARN.
    • This can be found in the "Outputs" tab of the CloudFormation stack in your AWS account.
  4. 4.
    Click Finish Setup, and you will be redirected to the Lookup Tables list page with your new Employee Directory table listed.