Links

Raw Event Filters

Filter out events before they're parsed by a log schema

Overview

Raw event filters let you define conditions under which to filter out ingested events. They act on raw data before it is parsed by a log schema, and can be defined using regex expressions or substrings patterns.
Once you have enabled a raw event filter, monitor its performance by viewing filtered event metrics.

Types of raw event filters

There are currently two types of filters:
  • Regex filters: Events that match the regex expression will be dropped.
  • Substring filters: Events that include the pattern at least once will be dropped.
In a log event filter, an Exclusion Condition is shown. The filter reads, "Exclude if" and a select box is open, showing two options: "Matches Regex" and "Contains"

How to create a raw event filter

Raw event filters are applied on unparsed events—not normalized data, like that which is visible in the data lake. Ensure you are constructing filters based on raw data. Basing a raw data filter on normalized data could cause false positives and unintentionally dropped data.
To create a filter for parsed events instead, see Normalized Event Filters.
To create a raw event filter:
  1. 1.
    In the left-hand navigation bar of your Panther Console, click Configure > Log Sources.
  2. 2.
    Click the name of the log source you'd like to add a filter to.
  3. 3.
    Click the Filters tab.
  4. 4.
    On the right-hand side of the Raw Data Filters tile, click Add Filter.
    The "Filters" tab of an "AY Okta" Log source is shown. There is an arrow drawn to a blue "Add Filter" button.
  5. 5.
    A new filter form will be expanded. Configure the filter:
    1. 1.
      Optionally edit the filter's name by clicking the pencil icon (
      pencil icon
      ) to the right of the placeholder name.
    2. 2.
      In the Exclusion Condition section, click the + to the right of Exclude if.
      An "Exclusion Condition" is shown. There is an "Exclude if" statement, with a plus sign to its right. The plus sign is circled.
    3. 3.
      Click Condition, and select one of the options below. Learn more about the different ways to construct exclusion statements in Types of raw event filters.
      • Matches Regex
      • Contains
        In a log event filter, an Exclusion Condition is shown. The filter reads, "Exclude if" and a select box is open, showing two options: "Matches Regex" and "Contains"
    4. 4.
      If you selected the Matches Regex condition, enter a regular expression. If you selected the Contains condition, enter a string value.
    5. 5.
      In the Quick Test section, enter a raw event to test against the filter you just created.
      • You can click View raw data to see raw events received by the source. To the right of an event, click Test event to populate the Raw Event field in Quick Test with the event.
        A list of raw events is shown. Each row has a "Test event" button on the right-hand side. The "Test event" button in the first row is circled.
    6. 6.
      Click Run Test.
      • Notice whether the test event matches the exclusion pattern.
    7. 7.
      The filter is enabled by default. If you would like to disable it, click the Enabled toggle.
  6. 6.
    In the upper-right corner, click Save.

Enabling or disabling a raw event filter

After an ingestion filter has been created, you can enable or disable it:
  1. 1.
    In the left-hand navigation bar of your Panther Console, click Configure > Log Sources.
  2. 2.
    Click the name of the log source you'd like to enable or disable a filter on.
  3. 3.
    Click the Filters tab.
  4. 4.
    Locate the filter you'd like to enable or disable, and set its toggle to Enabled or Disabled.

Viewing filtered event metrics