Correlation Rule Reference

Construct YAML correlation rules in either the Console or the CLI workflow

Overview

Correlation rules are in open beta starting with Panther version 1.108, and are available to all customers. Please share any bug reports and feature requests with your Panther support team.

Correlation rules can be written in YAML locally, in the CLI workflow, or in a code editor in the Panther Console. See instructions for how to create a correlation rule on Correlation Rules.

Correlation rule syntax

Each correlation rule can be composed of the following fields, at the top level:

  • Detection key

    Detection: 
  • Metadata keys

    AnalysisType: correlation_rule
    CreateAlert:
    Enabled: 
    RuleID:
    Reports: 
    Tags: 
    Tests: 
  • Alert keys (static)

    Severity:
    Description:
    DisplayName:
    OutputIds:
    Reference:
    Runbook:
    SummaryAttributes: 

Learn more about each of these keys, including which are required and optional, in the Correlation rule top-level fields reference, below.

Correlation rule writing tips

  • Use anchors and aliases. Anchors, defined with & , let you identify an item in your YAML. That item can later be referenced with an alias, defined with *.

    • Example: &failed_login (anchor) and *failed_login (alias)

Correlation rule top-level fields

The table below contains all available keys for YAML detections. Required fields are in bold.

Field Name

Description

Expected Value

AnalysisType

Indicates whether this analysis is a rule, scheduled_rule, policy, correlation_rule, or global

correlation_rule

Enabled

Whether this correlation_rule is enabled

Boolean

RuleID

The unique identifier of the rule

String

Severity

Which severity an associated alert should have

One of the following strings: Info, Low, Medium, High, or Critical

Correlation rule definition including sequence or group

CreateAlert

Whether the correlation rule should create an alert or not (default: true)

Boolean

Description

A brief description of the rule

String

DisplayName

A user-friendly name to show in the Panther Console and alerts. The RuleID will be displayed if this field is not set.

String

OutputIds

Static destination overrides. These will be used to determine how alerts from this rule are routed, taking priority over default routing based on severity.

List of strings

Reference

The reason this rule exists, often a link to documentation

String

Reports

A mapping of framework or report names to values this rule covers for that framework

Map of strings to list of strings

Runbook

The actions to be carried out if this rule returns an alert, often a link to documentation

String

SummaryAttributes

A list of fields that alerts should summarize.

List of strings

Tags

Tags used to categorize this rule

List of strings

Defines unit tests for this detection.

Detection fields

NameTypeValidationDescription

Object

N/A

The scheduling of a correlation rule

EventEvaluationOrder

String

Accepted values:

  • Chronological (default)

  • ReverseChronological

If Chronological, events are analyzed from oldest to newest. If ReverseChronological, events are analyzed from newest to oldest.

LookbackWindowMinutes

Scalar

15 ≤ x ≤ 21600 (15 days)

Default: 15

Indicates how many minutes in the past a correlation rule should look to evaluate the group or sequence. Learn more about LookbackWindowMinutes on Correlation Rules

List

2 ≤ x ≤ 50

Only one of Group or Sequence is required/allowed

A list of rule references that define the group Learn more about groups on Correlation Rules

MinMatchCount

Scalar

Only can be used if Group is present

2 ≤ x < number of rules in Group

Cannot use when number of rules in Group is 2 or >8.

Cannot use when any rules in Group use Absence: true

MinMatchCount specifies the minimum number of individual rules, scheduled rules, or correlation rules (defined in Group) that must match in order for this correlation rule to match. Learn more about MinMatchCount on Correlation Rules.

List

Only can be used if Group is present

x ≥ 1

Length must equal the length of Group

A mapping of MatchCriteria that defines the event fields the group should match on to pass

List

2 ≤ x ≤ 50

Only one of Group or Sequence is required/allowed

A list of rule references that define the sequence

Learn more about Sequences on Correlation Rules

List

Only can be used if Sequence is present

x ≥ 1

A list of transitions that defines the requirements to transition from one step of the sequence to the next. This can include event values to match on or a time frame.

Schedule fields

Each correlation rule runs on a schedule. The Schedule field defines that interval. Learn more about setting Schedule on Correlation Rules.

NameTypeValidationDescription

CronExpression

String

Only one of CronExpression or RateMinutes is required/allowed

A cron expression to describe how often a correlation rule should run

RateMinutes

Scalar

x ≥ 2 Only one of CronExpression or RateMinutes is required

The rate of minutes to describe how often a correlation rule should run

TimeoutMinutes

Scalar

x ≤ RateMinutes

The time frame in which a correlation rule can run. If a correlation rule takes longer than this period of time to evaluate, it will be cancelled for that time frame and a system health notification will be generated

Group and Sequence fields (rule references)

Within Group and Sequence, include a list of references to the rules included in this correlation rule. Each rule reference can include the following fields:

NameTypeValidationDescription

ID

String

Only required if using Transitions or MatchCriteria

A unique identifier to a rule being referenced in a sequence or group

RuleID

String

The ruleID must exist in Panther beforehand

The id of the rule, scheduled rule, or correlation rule to include in the sequence or group

MinMatchCount

Scalar

x ≥ 1 Default: 1

The minimum number of signals required for this step in the sequence to pass

MaxMatchCount

Scalar

x ≥ 0 x > MinMatchCount (if set)

The maximum number of signals allowed for this step in the sequence to pass

Absence

Boolean

N/A

Whether the absence of a signal needs to be true for the step to pass

MatchCriteria fields

The MatchCriteria key can be used alongside Group for more granular results. It indicates which event field each rule is matched on. There can only be one type of field matched on per correlation rule (e.g., all IP address fields or all email address fields).

The MatchCriteria field contains a key that is a unique label. That key contains a list of GroupID and Match pairings.

See examples of a Group with and without MatchCriteria on Correlation Rules.

NameTypeValidationDescription

GroupID

String

The ID defined in Group. Used to link the rule reference to the field that should be matched on

Match

String

  • For rules associated to multiple log types, scheduled rules, or correlation rules: only p_ fields are allowed

  • For rules associated to only one log type: any field is allowed

  • Validated based on the log schema. Cannot end on a field of type object or JSON. Anything within a JSON object is not validated.

  • Must be the same type as other Match values

The field in the event for the rule referenced that should be matched on

e.g., p_alert_context.username

Transitions fields

The Transitions key, used only alongside Sequence, defines how one step can traverse to another.

Transitions must be in the same order as the rules listed under Sequence.

Learn more about transitions on Correlation Rules.

NameTypeValidationDescription

ID

String

N/A

A unique identifier to a transition in a sequence.

From

String

The ID in the Rule References section of a sequence that indicates the starting step.

To

String

The ID in the Rule References section of a sequence that indicates the ending step.

WithinTimeFrameMinutes

Scalar

1 ≤ x ≤ 1440

The time frame in minutes within which two steps in a sequence (defined by From and To) must occur in order to pass.

List

len(x) = 1

Defines which event fields must match

Match fields

Match is a child field of Transitions, used with Sequence. These fields allow you to define which event fields, for each rule, scheduled rule, or correlation rule, must have matching values.

There can only be one type of field matched on per correlation rule (e.g., all IP address fields or all email address fields).

Example using same match values
Detection:
  - Sequence:
      - ID: Failed Login
        RuleID: Standard.BruteForceByIP
        MinMatchCount: 10
      - ID: Successful Login
        RuleID: Okta.Login.Success
    Transitions:
      - ID: Brute Force Login Success
        From: Failed Login
        To: Successful Login
        WithinTimeFrameMinutes: 10
        Match:
          - On: p_alert_context.ip     // these should be the same
      - ID: Gained Root Access
        From: Successful Login
        To: Root Login
        Match:
          - From: p_alert_context.ip  // these should be the same
            To: p_alert_context.sourceIPAddress
NameKey typeValidationDescription

On

String

  • From and To must be empty

  • For rules associated to multiple log types, scheduled rules, or correlation rules: only p_ fields are allowed

  • For rules associated to only one log type: any field is allowed

  • Validated based on the log schema. Cannot end on a field of type object or JSON. Anything within a JSON object is not validated.

  • Must use KQL syntax

The field in the event referenced by the Transitions From and To that should be matched on. Used when the event fields' names match exactly. E.g., p_udm.x or p_any_usernames

From

String

  • On must be empty

  • For rules associated to multiple log types, scheduled rules, or correlation rules: only p_ fields are allowed

  • For rules associated to only one log type: any field is allowed

  • Validated based on the log schema. Cannot end on a field of type object or JSON. Anything within a JSON object is not validated.

  • Must use KQL syntax

The field in the event referenced by the Transitions From and To that should be matched on.

Used when the event fields' names don't exactly match, but should be grouped together. E.g., p_alert_context.username = p_alert_context.user

To

String

  • On must be empty

  • For rules associated to multiple log types, scheduled rules, or correlation rules: only p_ fields are allowed

  • For rules associated to only one log type: any field is allowed

  • Validated based on the log schema. Cannot end on a field of type object or JSON. Anything within a JSON object is not validated.

  • Must use KQL syntax

The field in the event referenced by the Transitions From and To that should be matched on.

Used when the event fields' names don't exactly match, but should be grouped together. E.g., p_alert_context.username = p_alert_context.user

Tests fields

NameKey typeDescription

Name

String

Descriptive name for the test case. All test cases must have unique names.

ExpectedResult

Boolean

Whether the correlation rule should generate a match (i.e., return True or False) for this test case. For example if the test specifies ExpectedResult: False and the correlation rule does generate a match (i.e., return True), the test will fail because the expectation does not match the actual behavior.

RuleOutputs

Array<RuleOutput>

A list of RuleOutput objects. Each RuleOutput represents mock signal(s) generated from a particular rule referenced in your correlation rule definition, including definitions for which event fields/values matched.

RuleOutput fields

NameKey typeDescription

ID

String

A reference to one of the steps in your correlation rule's Group or Sequence field. If the items in your Group or Sequence have IDs defined, this value must match that ID. If the items in your Group or Sequence do not IDs defined, this value must match the RuleID associated with that Group or Sequence item.

Matches

Object<String, MatchValue>

An object that represents the event fields and values that this step in a Group or Sequence matched on. The key is the field name matched on (subfields can be specified using JSON path notation, e.g.,p_alert_context.username)—and the value is a MatchValue object.

MatchValue fields

A MatchValue object defines an event value that was matched on, and the times the match(es) occurred.

Currently, a correlation rule may only specify one event value to match on throughout, therefore this object may only contain a single key/value pair.

Example:

Matches:
    p_any_emails: # String (field name matched on)
        [email protected]: [0, 1, 2] # MatchValue object (value using relative timestamps)

or

Matches:
    p_any_ip_addresses: # String (field name matched on)
        1.2.3.4: # MatchValue key
            - "2006-01-02T15:04:05Z" # MatchValue value (using absolute timestamps)
KeyKey typeDescription

[Value matched on](String)

Array<String> OR Array<Number>

The key is an arbitrary string that your rule matched on. The value is an array of timestamps where the timestamps can either be relative (to the time the test case is ran) or absolute:

  • Relative timestamps: Represented as an array of numbers where the number is how many minutes the match occurred after the start of the test case scenario.

  • Absolute timestamps: Represented as an array of timestamps in RFC3339 format.

A single test cannot mix relative and absolute timestamps. For example, if a test uses absolute timestamps for one RuleOutput, it must use absolute timestamps for all RuleOutputs.

Last updated