Filtering

Filter incoming log data

Overview

Raw event filters allow you to filter your log data ingested into Panther, using regex expressions or substrings patterns. Filtering helps you realize the value of your high-volume logs and use logs that were previously cost-prohibitive when connected with Panther.

Once you have enabled a raw event filter, monitor its performance by viewing filtered event metrics.

Types of raw event filters

There are currently two types of filters:

• Regex filters: Events that match the regex expression will be dropped.

  • Regex filters use Google's RE2 engine.

• Substring filters: Events that include the pattern at least once will be dropped.

Creating a raw event filter

Filters are applied on raw events—not normalized data visible in the data lake, which can differ. Ensure you are constructing filters based on raw data. Basing filters on normalized data could cause false positives and unintentionally dropped data.

To create a raw event filter:

1. In the left-hand navigation bar of your Panther Console, click Configure > Log Sources.

2. Click the name of the log source you'd like to add a filter to.

3. Click the Filters tab.

4. Click New Filter.
5. Provide a value in the Add Filter name or short description field, then click the checkmark to the right of the field to save it.

6. In the Exclusion Condition section, click the + to the right of Exclude if.
7. Click Condition, and select one of the options below. Learn more about the different ways to construct exclusion statements in Types of raw event filters.

   • Matches Regex

   • Contains
8. If you used the Matches Regex condition, enter a regular expression. If you used the Contains condition, enter a string value.

9. In the Quick Test section, enter a raw event to test against the filter you just created.

   • You can click View raw data to see raw events received by the source. To the right of an event, click Test event to populate the Raw Event field in Quick Test with the event.
10. Click Run Test.

    • Notice whether the test event matches the exclusion pattern.

11. Click Save.

Enabling or disabling a raw event filter

To enable or disable a filter, by clicking on the toggle in the righthand side corner.

1. In the left-hand navigation bar of your Panther Console, click Configure > Log Sources.

2. Click the name of the log source you'd like to add a filter to.

3. Click the Filters tab.

4. On the right-hand side of the tile of the filter you would like to enable or disable, click the toggle next to Enabled.

Viewing filtered event metrics

To see your total number of filtered events for the current month and a chart of filtered out events for a configurable timeframe, see the Data Ingestion Dashboard in the Panther Console at Configure > Log Sources. Read more about this feature in Monitoring Log Sources.