Detections

Use detections to analyze data and trigger alerts on suspicious behavior

Overview

Detections are segments of logic run over real-time or historical log events, to identify suspicious behavior and generate signals, and optionally alerts. Your team can leverage detections-as-code by creating detections in Python, or use Panther's Simple Detections features to collaborate on detection creation and management, no matter your level of technical expertise.

There are four types of detections:

• Rules: Detect suspicious activity in security logs in real-time.

  • In the Console, rules can be created by using the Simple Detection builder, or by writing Python.

  • In the CLI workflow, rules can be written in Simple Detections YAML or Python.

  For more information, see Rules and Scheduled Rules.

• Scheduled Rules: Run against results from scheduled queries on your data lake.

  • In the Console, scheduled rules can be created by writing Python.

  • In the CLI workflow, scheduled rules can be created by writing Python.

  For more information, see Rules and Scheduled Rules.

• Correlation rules: Detect when a group or sequence of events has occurred.

  • In the Console, correlation rules can be created by writing YAML.

    • In the CLI workflow, correlation rules can be created by writing YAML.

  For more information, see Correlation Rules.

• Policies: Scan and evaluate cloud infrastructure configurations to identify misconfigurations.

  • In the Console, policies can be created by writing Python.

  • In the CLI workflow, policies can be created by writing Python.

  For more information, see Policies.

When there's a match on a rule, scheduled rule, or correlation rule, a signal is created—and, if alerting is enabled, a rule match. Policies, on each match, generate a compliance failure.

Rule matches can trigger alerts according to the detection's event threshold and deduplication configurations. Alerts are then routed to destinations based on configurations on the detection and destination. Learn more about the difference between signals, rule matches, and alerts below.

You'll need to decide whether you'd like to manage your detections in the Panther Console or using the CLI workflow. Then you can get started enabling Panther-managed detections, or writing your own.

Getting started with detections

Choose a detection management workflow

You can create and manage Panther detections using one of the following methods:

• Panther Console

  • Construct your own detections in the Console using the Simple Detection builder, or by writing Python.

  • Enable Panther-managed Python detections (individually, or in Detection Packs) to get started quickly, with no additional configuration necessary.

• Panther CLI workflow

  • Write detections locally as either Simple Detections or Python detections.

  • Learn more about managing Panther detections using a Continuous Integration and Continuous Deployment workflow in our CI/CD Guide.

  • CLI workflows include using Panther Analysis Tool (PAT), an open source utility for testing, packaging, and deploying Panther detections from source code.

If you use Simple Detections—the set of features that allows users of varying technical skill to create and edit detections—it is possible to use both the Panther Console and the CLI workflow to collaborate on detections, to an extent. Learn more in Simple Detections.

Migrating to a CI/CD workflow

You may want to get started quickly by enabling Panther-managed Detection Packs in the Panther Console, but then later transition to using the CI/CD workflow. (Using both the Console and CI/CD workflows simultaneously is not supported.) To migrate your workflow to CI/CD, follow the instructions on Migrating to a CI/CD Workflow.

Enable or write detections

The quickest way to start detecting threats with Panther is to turn on the already written Panther-managed detections that come with your Panther instance. These built-in rules and policies are applicable to various log sources, and Panther periodically releases improvements to their core detection logic. Panther-managed rules can be customized using Rule Filters, or you can clone them and edit the detection logic of the cloned version to suit your exact needs.

It's also possible to quickly generate rules in Panther by converting Sigma rules.

If you'd rather create your own detections from scratch, learn how on the following pages:

• Using the Simple Detection Builder

• Writing Simple Detections

• Writing Python Detections

• Correlation Rules

Simple Detections

Simple Detections refers to the set of Panther features that makes it easy for users with all levels of technical skill to create and manage detections in Panther. These features include:

• Simple Detection builder: In the Panther Console, you can use the Simple Detection builder to create and edit rules without writing code.

• Simple Detections written in the CLI workflow: When you write Simple Detections in the CLI workflow (in YAML), then upload them to Panther, they will be represented in the Panther Console in the Simple Detection builder.

Together, these features allow team members with various levels of YAML experience to understand and collaborate on detections.

If your team uses the CLI workflow to manage detection content, the changes made to detections using the Simple Detection builder in the Console will be overwritten on next upload (except for Inline Filters created in the Console, which will be preserved).

If you create or edit detections using the Simple Detection builder in the Console, copy the resulting YAML representation and include it in your local detections files, in order to prevent the changes from being overwritten on next upload.

Working with your data in detections

Schema definitions

Panther’s Schemas provide helpful information on the types of fields contained within your data, which makes it easier to understand how to interact with your data when writing a Detection.

Schema definitions can be found in:

• Panther's documentation

  • See schemas for each integration within the Supported Logs section of the documentation. Find more information about Custom Log schemas in Custom Logs.

• The Panther Console

  • Log in to the Panther Console and navigate to Data > Schemas.

Pulling samples out of Data Explorer

Data Explorer makes it easier to understand and investigate data, location of data, and data types when writing Python code. It contains all the data Panther parses from your log sources and stores the data in tables.

Explore and find log events by searching the relevant table for the log type you are interested in writing a detection for.

You can preview example table data without writing SQL. To generate a sample SQL query for that log source, click the eye icon next to the table type:

When the query has produced results, you will see the example log events in the Results table. You can download these as a CSV file.

To copy the log event to be used in Unit Tests while writing detections, click View JSON.

Signals vs. rule matches vs. alerts

First, let's examine when signals are generated vs. rule matches. Each time there is a match on a rule, scheduled rule, or correlation rule:

• A signal is generated.

  • This is true regardless of any configuration set on the rule, scheduled rule, or correlation rule.

• If the rule, scheduled rule, or correlation rule has alerting enabled (i.e., in the Console, Create Alert is set to ON, or in the CLI workflow CreateAlert is set to true [or not set]), a rule match is generated.

  • If alerting is disabled for the detection, rule matches are not created. See these instructions for how to create a rule with alerting disabled.

Only rule matches (not signals) can then produce alerts. Alerts are generated from rule matches according to the event threshold and deduplication configurations on the rule, scheduled rule, or correlation rule.

Each time there is a match on a policy, a compliance failure is created (and not a signal). Policies cannot be configured to disable alerting.

Detection features

Simple Detections

Panther's Simple Detections functionality includes the Simple Detection builder in the Panther Console, and Simple Detections in the CLI workflow.

Together, these features allow team members with varying levels of technical skill to collaborate on detection management.

Panther-managed Detections

Panther comes with a number of Panther-managed detections, which are detections for which Panther has written the core logic, and periodically updates. Using Panther-managed detections saves you the effort of having to write your own, and provides the ongoing benefit of receiving improvements to core detection logic over time, as Panther releases new versions. For more information, see Using Panther-managed detections.

Detection Packs

Panther packs logically group and update detections via the Panther Console. Detection packs can group any number of Panther features including but not limited to detections, queries, global helpers, data models, or Lookup Tables. packs are defined in this open source repository: panther-labs/panther-analysis. For more information, see Detection Packs.

Inline Filters

Inline Filters are conditional statements that are evaluated before a detection's rule function. You can apply Inline Filters to easily tune detections, including Panther-managed ones. A filter must return true (i.e., match the event) for the rule function, which is written in code, to then be run. Based on the detection's log type, you can select a field to filter on. From there, you will specify the operator and, if applicable, input a value. To learn more, see Modifying Detections with Inline Filters.

Derivation

You can create one or more Derived Detections from a single Base Detection in Panther. Derived Detections inherit the Base Detection's core detection logic, which is immutable, as well as its metadata field values, which can be overwritten for each Derived Detection.

Learn more on Derived Detections.

Testing

Panther's detection testing ensures that detections behave as expected and generate alerts once deployed correctly. Test inputs are utilized to determine whether or not an alert will generate in order to promote reliability as code evolves and protect against regressions. For more information, see Testing.

Data Replay

Data Replay (Beta) allows rules to be tested against historical log data to preview the outcome of a rule before enabling it. Data Replay can simulate what types of alerts you are likely to receive before deploying the detection. For more information, see Data Replay.

Caching

Panther examines events one-by-one and provides a way to cache results across invocations. To accommodate stateful checks, Panther rules can cache values by using built-in helper functions. For more information, see Caching.

Data Models

Panther's data models provide a way to configure a set of unified fields across all log types. Data models allow you to monitor particular fields across many log types at once, avoiding cumbersome and complex individual log monitoring. For more information, see Data Models.

Global Helper Functions

Panther supports the common programming pattern to extract repeated code into helper functions, via the global analysis type. Import global helper functions in your detections by inserting certain commands at the top of your analysis function body then calling the global function as if it were any other Python library. For more information, see Global Helper Functions.

Framework Mapping

Panther supports the ability to map rules, policies, and scheduled rules to compliance frameworks (including MITRE ATT&CK®) to track coverage against that framework. Reports can be mapped to your detection within the Detections > All Detections navigation section of the Panther Console. For more information, see Framework Mapping and MITRE ATT&CK® Matrix.