Managing Enrichment Providers with the Panther Analysis Tool

Configure custom and Panther-managed enrichments using PAT

Overview

Enrichment sources (also called Lookup Tables) allow you to add more context to your incoming logs. You can manage schemas and mappings for your custom enrichments and certain Panther-managed enrichments through the Panther Analysis Tool (PAT).

This guide will walk you through the following:

• Creating and uploading a custom schema for a custom enrichment source using the pantherlog tool.

• Modifying the Selectors and LogTypes in the enrichment source's YAML configuration file.

  • Note that log types and Selectors can be set manually or automatically mapped by indicator fields.

• Uploading the enrichment source's YAML configuration file via PAT.

• Testing the enrichment in the Panther Console.

• If your team uses CLI workflows, it's recommended to use PAT and CI/CD to manage your enrichment, instead of doing so via Detection Packs in the Console.

• If you choose to manage enrichment tables through PAT after enabling them in the Panther Console, you must first disable the Detection Packs in the Panther Console. Simultaneous use of both the Panther Console and PAT to manage enrichment sources is not supported.

This guide applies to custom enrichments and these Panther-managed enrichment sources.

"Bring your own API key" log pullers and Panther log source pullers cannot be enabled in the CLI workflow using PAT.

Custom enrichments vs. Panther-managed enrichments

• Custom enrichments are managed by you. You'll need to create and upload a schema, then upload the enrichment table's YAML configuration file.

• Panther-managed enrichment providers are managed by Panther. Their schemas are Panther-defined, and their YAML configuration files (which you can modify to your needs) can be found in the panther-analysis repo in GitHub.

How to manage custom and Panther-managed enrichments with PAT

Custom enrichments

Prerequisites

• A YAML configuration file. You must create the YAML configuration file yourself.

• A data sample (if you need to create a new schema) or an existing YAML schema created in Panther.

Step 1: Create and upload a schema

Custom enrichments must be associated with a schema you have created and uploaded to Panther. If you have already created a schema in Panther that you want associate to your custom enrichment, you can skip this step.

1. Create the schema using your sample log data.

   • You can use pantherlog to infer a schema from a sample set of data. To generate a schema from a sample JSON log file, use the infer command:

     $ ./pantherlog infer sample_logs.jsonl > schema.yml

   • Remember to review the inferred schema and make any necessary adjustments before uploading it to Panther. For more information about this process, see the pantherlog documentation.

2. Upload the schema.

   • Once you have created the schema, you can upload it to Panther by following the Uploading log schemas with the Panther Analysis Tool instructions.

Step 2: Create the YAML configuration file

• For custom enrichments, you must create the YAML configuration file from scratch. Reference the Lookup Table Specification Reference to see which keys this file must include.

Step 3: Upload the custom enrichment via PAT

Once you have created your custom custom enrichment configuration file, you can upload it to Panther using the PAT upload command:

panther_analysis_tool upload

You will need to provide an API token and host with --api-token and --api-host, respectively, for the upload to occur. Other options include filtering, minimum tests, and more.

Ensure you've uploaded the corresponding schema before uploading the YAML configuration file.

Step 4: Test the custom enrichment

There are several methods to test if your custom enrichment has been set up correctly.

Method 1: Enriching test data in the Panther Console or CLI

Console

In the Panther Console's detection editor, click Enrich Test Data to verify if your custom enrichment is working correctly. This allows you to input test data and see the output of the enrichment process within your unit test.

For Enrich Test Data to work, the unit test must have a p_log_type identifying the correct log type. This serves as the basis for Panther's enrichment logic.

CLI

Method 2: Checking the panther_signals database

You can verify that your changes have taken effect by checking the panther_signals.public.correlation_signals database/table for the p_enrichment field. Ensure that the field includes the custom enrichment details you would expect to see.

Method 3: Using SQL queries

You can also perform a LEFT JOIN between event logs and the enrichment table in SQL. Ensure that the selector is defined in the query. This allows you to verify if the data from your logs is being correctly matched with the data in your custom enrichment.

For example, this query will attempt to match event data to the custom enrichment using a custom selector (which should be the same as the selector you've defined in the YAML configuration file):

SELECT *
FROM panther_logs.public.<log_type> AS e
LEFT JOIN panther_lookups.public.<lookup_table_name> AS lt
ON e.<field_path> = lt.<field_path>
WHERE e.p_occurs_since('1 day')

Panther-managed enrichments