Correlation Rule Reference

Construct YAML correlation rules in either the Console or the CLI workflow

Overview

Correlation rules are in open beta starting with Panther version 1.108, and are available to all customers. Please share any bug reports and feature requests with your Panther support team.

Correlation rules can be written in YAML locally, in the CLI workflow, or in a code editor in the Panther Console. See instructions for how to create a correlation rule on Correlation Rules.

Correlation rule syntax

Each correlation rule can be composed of the following fields, at the top level:

• Detection key

  Detection: 

• Metadata keys

  AnalysisType: correlation_rule
  CreateAlert:
  Enabled: 
  RuleID:
  Reports: 
  Tags: 
  Tests: 

• Alert keys (static)

  Severity:
  Description:
  DisplayName:
  OutputIds:
  Reference:
  Runbook:
  SummaryAttributes: 

Learn more about each of these keys, including which are required and optional, in the Correlation rule top-level fields reference, below.

Correlation rule writing tips

• Use anchors and aliases. Anchors, defined with & , let you identify an item in your YAML. That item can later be referenced with an alias, defined with *.

  • Example: &failed_login (anchor) and *failed_login (alias)

  • Do not use anchors and aliases within Tests. Doing so can cause errors.

Correlation rule top-level fields

The table below contains all available keys for YAML detections. Required fields are in bold.

Field Name	Description	Expected Value
AnalysisType	Indicates whether this analysis is a rule, scheduled_rule, policy, correlation_rule, or global	correlation_rule
Enabled	Whether this correlation_rule is enabled	Boolean
RuleID	The unique identifier of the rule	String
Severity	Which severity an associated alert should have	One of the following strings: Info, Low, Medium, High, or Critical
Detection	Correlation rule definition including sequence or group	See Detection fields
CreateAlert	Whether the correlation rule should create an alert or not (default: true)	Boolean
Description	A brief description of the rule	String
DisplayName	A user-friendly name to show in the Panther Console and alerts. The RuleID will be displayed if this field is not set.	String
OutputIds	Static destination overrides. These will be used to determine how alerts from this rule are routed, taking priority over default routing based on severity.	List of strings
Reference	The reason this rule exists, often a link to documentation	String
Reports	A mapping of framework or report names to values this rule covers for that framework	Map of strings to list of strings
Runbook	The actions to be carried out if this rule returns an alert, often a link to documentation	String
SummaryAttributes	A list of fields that alerts should summarize.	List of strings
Tags	Tags used to categorize this rule	List of strings
Tests	Defines unit tests for this detection.	See Tests fields

Detection fields

Name	Type	Validation	Description
Schedule	Object	N/A	The scheduling of a correlation rule
EventEvaluationOrder	String	Accepted values: Chronological (default) ReverseChronological	If Chronological, events are analyzed from oldest to newest. If ReverseChronological, events are analyzed from newest to oldest.
LookbackWindowMinutes	Scalar	15 ≤ x ≤ 21600 (15 days) Default: 15	Indicates how many minutes in the past a correlation rule should look to evaluate the group or sequence. Learn more about LookbackWindowMinutes on Correlation Rules
Group	List	2 ≤ x ≤ 50 Only one of Group or Sequence is required/allowed	A list of rule references that define the group Learn more about groups on Correlation Rules
MinMatchCount	Scalar	Only can be used if Group is present 2 ≤ x < number of rules in Group Cannot use when number of rules in Group is 2 or >8. Cannot use when any rules in Group use Absence: true	MinMatchCount specifies the minimum number of individual rules, scheduled rules, or correlation rules (defined in Group) that must match in order for this correlation rule to match. Learn more about MinMatchCount on Correlation Rules.
MatchCriteria	List	Only can be used if Group is present x ≥ 1 Length must equal the length of Group	A mapping of MatchCriteria that defines the event fields the group should match on to pass
Sequence	List	2 ≤ x ≤ 50 Only one of Group or Sequence is required/allowed	A list of rule references that define the sequence Learn more about sequences on Correlation Rules
Transitions	List	Only can be used if Sequence is present x ≥ 1	A list of transitions that defines the requirements to transition from one step of the sequence to the next. This can include event values to match on or a time frame.

Schedule fields

Each correlation rule runs on a schedule. The Schedule field defines that interval. Learn more about setting Schedule on Correlation Rules.

Name	Type	Validation	Description
CronExpression	String	Only one of CronExpression or RateMinutes is required/allowed	A cron expression to describe how often a correlation rule should run. Learn more about CronExpression format in How to use the Scheduled Search crontab.
RateMinutes	Scalar	x ≥ 2 Only one of CronExpression or RateMinutes is required	The rate of minutes to describe how often a correlation rule should run
TimeoutMinutes	Scalar	x ≤ RateMinutes	The time frame in which a correlation rule can run. If a correlation rule takes longer than this period of time to evaluate, it will be cancelled for that time frame and a system health notification will be generated

Group and Sequence fields (rule references)

Within Group and Sequence, include a list of references to the rules included in this correlation rule. Each rule reference can include the following fields:

Name	Type	Validation	Description
ID	String	Only required if using Transitions or MatchCriteria	A unique identifier to a rule being referenced in a sequence or group
RuleID	String	The ruleID must exist in Panther beforehand	The id of the rule, scheduled rule, or correlation rule to include in the sequence or group
MinMatchCount	Scalar	x ≥ 1 Default: 1	The minimum number of signals required for this step in the sequence to pass
MaxMatchCount	Scalar	x ≥ 0 x > MinMatchCount (if set)	The maximum number of signals allowed for this step in the sequence to pass
Absence	Boolean	N/A	Whether the absence of a signal needs to be true for the step to pass

MatchCriteria fields

The MatchCriteria key can be used alongside Group for more granular results. It indicates which event field each rule is matched on. There can only be one type of field matched on per correlation rule (e.g., all IP address fields or all email address fields).

The MatchCriteria field contains a key that is a unique label. That key contains a list of GroupID and Match pairings.

See examples of a Group with and without MatchCriteria on Correlation Rules.

Name	Type	Validation	Description
GroupID	String		The ID defined in Group. Used to link the rule reference to the field that should be matched on
Match	String	For rules associated to multiple log types, scheduled rules, or correlation rules: only p_ fields are allowed For rules associated to only one log type: any field is allowed Validated based on the log schema. Cannot end on a field of type object or JSON. Anything within a JSON object is not validated. Must be the same type as other Match values Must use Kusto Query Language (KQL) syntax	The field in the event for the rule referenced that should be matched on e.g., p_alert_context.username

Transitions fields

The Transitions key, used only alongside Sequence, defines how one step can traverse to another.

Transitions must be in the same order as the rules listed under Sequence.

Learn more about transitions on Correlation Rules.

Name	Type	Validation	Description
ID	String	N/A	A unique identifier to a transition in a sequence.
From	String		The ID in the rule references section of a sequence that indicates the starting step.
To	String		The ID in the rule references section of a sequence that indicates the ending step.
WithinTimeFrameMinutes	Scalar	1 ≤ x ≤ 1440 AND x ≤ LookbackWindowMinutes	The time frame in minutes within which two steps in a sequence (defined by From and To) must occur in order to pass.
Match	List	len(x) = 1	Defines which event fields must match

Match fields

Match is a child field of Transitions, used with Sequence. These fields allow you to define which event fields, for each rule, scheduled rule, or correlation rule, must have matching values.

There can only be one type of field matched on per correlation rule (e.g., all IP address fields or all email address fields).

Example using same match values

Detection:
  - Sequence:
      - ID: Failed Login
        RuleID: Standard.BruteForceByIP
        MinMatchCount: 10
      - ID: Successful Login
        RuleID: Okta.Login.Success
    Transitions:
      - ID: Brute Force Login Success
        From: Failed Login
        To: Successful Login
        WithinTimeFrameMinutes: 10
        Match:
          - On: p_alert_context.ip     // these should be the same
      - ID: Gained Root Access
        From: Successful Login
        To: Root Login
        Match:
          - From: p_alert_context.ip  // these should be the same
            To: p_alert_context.sourceIPAddress

Name	Key type	Validation	Description
On	String	From and To must be empty For rules associated to multiple log types, scheduled rules, or correlation rules: only p_ fields are allowed For rules associated to only one log type: any field is allowed Validated based on the log schema. Cannot end on a field of type object or JSON. Anything within a JSON object is not validated. Must use KQL syntax	The field in the event referenced by the Transitions From and To that should be matched on. Used when the event fields' names match exactly. E.g., p_udm.x or p_any_usernames
From	String	On must be empty For rules associated to multiple log types, scheduled rules, or correlation rules: only p_ fields are allowed For rules associated to only one log type: any field is allowed Validated based on the log schema. Cannot end on a field of type object or JSON. Anything within a JSON object is not validated. Must use KQL syntax	The field in the event referenced by the Transitions From and To that should be matched on. Used when the event fields' names don't exactly match, but should be grouped together. E.g., p_alert_context.username = p_alert_context.user
To	String	On must be empty For rules associated to multiple log types, scheduled rules, or correlation rules: only p_ fields are allowed For rules associated to only one log type: any field is allowed Validated based on the log schema. Cannot end on a field of type object or JSON. Anything within a JSON object is not validated. Must use KQL syntax	The field in the event referenced by the Transitions From and To that should be matched on. Used when the event fields' names don't exactly match, but should be grouped together. E.g., p_alert_context.username = p_alert_context.user

Tests fields

Do not use anchors and aliases within Tests. Doing so can cause errors.

Name	Key type	Description
Name	String	Descriptive name for the test case. All test cases must have unique names.
ExpectedResult	Boolean	Whether the correlation rule should generate a match (i.e., return True or False) for this test case. For example if the test specifies ExpectedResult: False and the correlation rule does generate a match (i.e., return True), the test will fail because the expectation does not match the actual behavior.
RuleOutputs	Array<RuleOutput>	A list of RuleOutput objects. Each RuleOutput represents mock signal(s) generated from a particular rule referenced in your correlation rule definition, including definitions for which event fields/values matched.

RuleOutput fields

Name	Key type	Description
ID	String	A reference to one of the steps in your correlation rule's Group or Sequence field. If the items in your Group or Sequence have IDs defined, this value must match that ID. If the items in your Group or Sequence do not IDs defined, this value must match the RuleID associated with that Group or Sequence item.
Matches	Object<String, MatchValue>	An object that represents the event fields and values that this step in a Group or Sequence matched on. The key is the field name matched on (subfields can be specified using JSON path notation, e.g.,p_alert_context.username)—and the value is a MatchValue object.

MatchValue fields

A MatchValue object defines an event value that was matched on, and the times the match(es) occurred.

Currently, a correlation rule may only specify one event value to match on throughout, therefore this object may only contain a single key/value pair.

Example:

Matches:
    p_any_emails: # String (field name matched on)
        [e-mail]: [0, 1, 2] # MatchValue object (value using relative timestamps)

or

Matches:
    p_any_ip_addresses: # String (field name matched on)
        1.2.3.4: # MatchValue key
            - "2006-01-02T15:04:05Z" # MatchValue value (using absolute timestamps)

Key	Key type	Description
[Value matched on](String)	Array<String> OR Array<Number>	The key is an arbitrary string that your rule matched on. The value is an array of timestamps where the timestamps can either be relative (to the time the test case is ran) or absolute: Relative timestamps: Represented as an array of numbers where the number is how many minutes the match occurred after the start of the test case scenario. Absolute timestamps: Represented as an array of timestamps in RFC3339 format. A single test cannot mix relative and absolute timestamps. For example, if a test uses absolute timestamps for one RuleOutput, it must use absolute timestamps for all RuleOutputs.