# Raw Event Filters

## Overview

Raw event filters let you define conditions under which to filter out ingested events. They act on raw data *before* it is parsed by a log schema, and can be defined using regex expressions or substrings patterns.

Raw event filters can be created as either inclusion or exclusion filters. Learn about the [difference between inclusion and exclusion filters here](https://docs.panther.com/data-onboarding/ingestion-filters/..#inclusion-vs.-exclusion-filters).

Once you have enabled a raw event filter, monitor its performance by [viewing filtered event metrics](#viewing-filtered-event-metrics).

{% hint style="warning" %}
If you create multiple (inclusion or exclusion) raw event filters, there is no guarantee of the order they will run in. Take special note of this when creating multiple inclusion filters. Due to filters running in a random order, events you intend to include could be dropped if another filter that is evaluated first does not include them.
{% endhint %}

### Types of raw event filters

There are currently two types of filters:

* Regex filter&#x73;**:** Events that match the regex expression will be dropped.
  * Regex filters use [Google's RE2 engine](https://github.com/google/re2/wiki/Syntax).
* Substring filter&#x73;**:** Events that include the pattern at least once will be dropped.

<figure><img src="https://4011785613-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LgdiSWdyJcXPahGi9Rs-2910905616%2Fuploads%2Fgit-blob-bca754e651c79bb2cc803f9912e84e3b67ab0ed7%2FScreenshot%202023-07-13%20at%203.27.26%20PM.png?alt=media" alt="In a log event filter, an Exclusion Condition is shown. The filter reads, &#x22;Exclude if&#x22; and a select box is open, showing two options: &#x22;Matches Regex&#x22; and &#x22;Contains&#x22;"><figcaption></figcaption></figure>

## How to create a raw event filter

{% hint style="warning" %}
Raw event filters are applied on unparsed events—*not* normalized events, like those visible in the data lake. Ensure you are constructing filters based on raw data. Basing a raw data filter on normalized data could cause false positives and unintentionally dropped data.

To create a filter for parsed events instead, see [Normalized Event Filters](https://docs.panther.com/data-onboarding/ingestion-filters/normalized-event).
{% endhint %}

To create a raw event filter:

1. In the left-hand navigation bar of your Panther Console, click **Configure** > **Log Sources**.
2. Click the name of the log source you'd like to add a filter to.
3. Click the **Filters** tab.
4. On the right-hand side of the **Raw Event Filters** tile, click **Add Filter**.\
   ![The "Filters" tab of an "AY Okta" Log source is shown. There is an arrow drawn to a blue "Add Filter" button.](https://4011785613-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LgdiSWdyJcXPahGi9Rs-2910905616%2Fuploads%2Fgit-blob-b2c23c73bc34c9428348e777ffffc2c28c7ad667%2FScreenshot%202024-02-12%20at%2010.04.04%20AM.png?alt=media)
5. In the new filter form that is expanded, configure the filter:
   1. (Optional) Click the pencil icon (<img src="https://4011785613-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LgdiSWdyJcXPahGi9Rs-2910905616%2Fuploads%2Fgit-blob-c6bdd7f21ad5a1cd45edc84b22f568495de30bec%2FScreenshot%202024-02-09%20at%2012.04.14%20PM.png?alt=media" alt="pencil icon" data-size="line">) to edit the filter's name.
   2. In the **Condition** dropdown, make a selection:
      * **Exclude if**: Choose this if you'd like to create an [exclusion filter](https://docs.panther.com/data-onboarding/ingestion-filters/..#inclusion-vs.-exclusion-filters).
      * **Include if**: Choose this if you'd like to create an [inclusion filter](https://docs.panther.com/data-onboarding/ingestion-filters/..#inclusion-vs.-exclusion-filters).
   3. Click **+** to add a condition.\
      ![An "Exclusion Condition" is shown. There is an "Exclude if" statement, with a plus sign to its right. The plus sign is circled.](https://4011785613-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LgdiSWdyJcXPahGi9Rs-2910905616%2Fuploads%2Fgit-blob-5b60a962845e170d031163099b25e9b05681f98f%2FScreenshot%202023-07-13%20at%2012.42.52%20PM.png?alt=media)
   4. Click **Condition**, and select one of the options below. Learn more about the different ways to construct conditions in [Types of raw event filters](#types-of-raw-event-filters).
      * **Matches Regex**
      * **Contains**\
        ![In a log event filter, an Exclusion Condition is shown. The filter reads, "Exclude if" and a select box is open, showing two options: "Matches Regex" and "Contains"](https://4011785613-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LgdiSWdyJcXPahGi9Rs-2910905616%2Fuploads%2Fgit-blob-a9fcfc49f1df3bc218fef05d66842d3db6a50023%2FScreenshot%202023-07-13%20at%2012.48.03%20PM.png?alt=media)
   5. If you selected the **Matches Regex** condition, enter a regular expression. If you selected the **Contains** condition, enter a string value.
   6. In the **Quick Test** section, enter a raw event to test against the filter you just created.
      * You can click **View raw data** to see raw events received by the source. To the right of an event, click **Test event** to populate the **Raw Event** field in **Quick Test** with the event.\
        ![A list of raw events is shown. Each row has a "Test event" button on the right-hand side. The "Test event" button in the first row is circled.](https://4011785613-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LgdiSWdyJcXPahGi9Rs-2910905616%2Fuploads%2Fgit-blob-64f9ca204689fc4b840c17600ed63875d886ecec%2FScreenshot%202023-07-13%20at%2012.58.31%20PM.png?alt=media)
   7. Click **Run Test**.
      * Notice whether the test event matches the pattern.
   8. The filter is enabled by default. If you would like to disable it, click the **Enabled** toggle.
6. In the upper-right corner, click **Save**.

## Enabling or disabling a raw event filter

After an ingestion filter has been created, you can enable or disable it:

1. In the left-hand navigation bar of your Panther Console, click **Configure** > **Log Sources**.
2. Click the name of the log source you'd like to enable or disable a filter on.
3. Click the **Filters** tab.
4. Locate the filter you'd like to enable or disable, and set its toggle to **Enabled** or **Disabled**.\
   ![Under a "Dev account" header, an arrow is pointing from a Filters tab to an "Enabled" toggle on a tile called "Event Based Filter cc4e0a."](https://4011785613-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LgdiSWdyJcXPahGi9Rs-2910905616%2Fuploads%2Fgit-blob-fc2cd88025de20d4c17cbc48979984311410c686%2FScreenshot%202024-02-12%20at%204.58.34%20PM.png?alt=media)\\

## Viewing filtered event metrics

* See [Viewing filtered event volume](https://docs.panther.com/monitoring-log-sources#viewing-filtered-event-volume).
