Managing Lookup Tables and Enrichment Providers with the Panther Analysis Tool
Manage Custom Lookup Tables and Enrichment Providers using PAT
Last updated
Was this helpful?
Manage Custom Lookup Tables and Enrichment Providers using PAT
Last updated
Was this helpful?
You can manage and (also known as Panther-managed Lookup Table) schemas and mappings through the .
This guide will walk you through the following:
Creating and uploading a custom schema for a Custom Lookup Table using the .
Modifying the Selectors
and LogTypes
in the Lookup Table/Enrichment Provider YAML configuration file.
Note that log types and Selectors can or .
Uploading the Lookup Table/Enrichment Provider YAML configuration file via PAT.
Testing the enrichment in the Panther Console.
This guide applies to all except for , which cannot be enabled in the CLI workflow using PAT.
In Panther, there is a distinction between Custom Lookup Tables and Enrichment Providers (also known as Panther-managed Lookup Tables):
are user-managed. You'll need to create and upload a schema, then upload the Lookup Table's YAML configuration file.
are Panther-managed Lookup Tables. Their schemas are Panther-defined, and their YAML configuration files (which you can modify to your needs) can be found in the in GitHub.
A YAML configuration file. You must create the YAML configuration file yourself.
A data sample (if you need to create a new schema) or an existing YAML schema created in Panther.
Custom Lookup Tables must be associated with a schema you have created and uploaded to Panther. If you have already created a schema in Panther that you want associate to your Lookup Table, you can skip this step.
Create the schema using your sample log data.
You can use pantherlog
to infer a schema from a sample set of data. To generate a schema from a sample JSON log file, use the infer
command:
Upload the schema.
You will need to provide an API token and host with --api-token
and --api-host
, respectively, for the upload to occur. Other options include filtering, minimum tests, and more.
Ensure you've uploaded the corresponding schema before uploading the YAML configuration file.
There are several methods to test if your Lookup Table has been set up correctly.
In the Panther Console's detection editor, click Enrich Test Data to verify if your Lookup Table is working correctly. This allows you to input test data and see the output of the enrichment process within your unit test.
For Enrich Test Data to work, the unit test must have a p_log_type
identifying the correct log type. This serves as the basis for Panther's enrichment logic.
panther_rule_matches
databaseYou can verify that your changes have taken effect by checking the panther_rule_matches
database for the p_enrichment
field. Ensure that the field includes the Lookup Table details you would expect to see.
You can also perform a LEFT JOIN
between event logs and the lookup table in SQL. Ensure that the selector is defined in the query. This allows you to verify if the data from your logs is being correctly matched with the data in your Lookup Table.
For example, this query will attempt to match event data to the Lookup Table using a custom selector (which should be the same as the selector you've defined in the YAML configuration file):
Remember to review the inferred schema and make any necessary adjustments before uploading it to Panther. For more information about this process, see the .
Once you have created the schema, you can upload it to Panther by following the instructions.
For custom Lookup Tables, you must create the YAML configuration file from scratch. Reference the to see which keys this file must include.
Once you have created your custom Lookup Table configuration file, you can upload it to Panther using the Panther Analysis Tool's :
Verify your Lookup Table is working properly by using PAT's enrich-test-data
command. .
A YAML configuration file. You can use the .
If you are enabling an Enrichment Provider (i.e. a Panther-managed Lookup Table), you can modify the configuration file that to meet your needs.
Note than even if this log type (Cloudflare.Firewall
) and Selector (ClientIP
) weren't set manually in this fashion, Cloudflare.Firewall
and p_any_ip_addresses
would be added as a log type and Selector, respectively, due to the .
Once you have modified your Enrichment Provider configuration file, you can upload it to Panther using the Panther Analysis Tool's :
Verify your Enrichment Provider is working properly by using PAT's enrich-test-data
command. .