# Correlation Rule Reference

## Overview

{% hint style="info" %}
Correlation rules are in open beta starting with Panther version 1.108, and are available to all customers. Please share any bug reports and feature requests with your Panther support team.
{% endhint %}

Correlation rules can be written in YAML locally, in the CLI workflow, or in a code editor in the Panther Console. See instructions for how to create a correlation rule on [Correlation Rules](https://docs.panther.com/detections/correlation-rules/..#how-to-create-a-correlation-rule).

## Correlation rule syntax

Each correlation rule can be composed of the following fields, at the top level:

* Detection key

  ```yaml
  Detection: 
  ```
* Metadata keys

  ```yaml
  AnalysisType: correlation_rule
  CreateAlert:
  Enabled: 
  RuleID:
  CreatedBy: 
  Reports: 
  Tags: 
  Tests: 
  ```
* Alert keys (static)

  ```yaml
  Severity:
  Description:
  DisplayName:
  OutputIds:
  Reference:
  Runbook:
  SummaryAttributes: 
  ```

Learn more about each of these keys, including which are required and optional, in the [Correlation rule top-level fields reference, below](#correlation-rule-top-level-fields).

### Correlation rule writing tips

* Use [anchors and aliases](https://medium.com/@kinghuang/docker-compose-anchors-aliases-extensions-a1e4105d70bd). Anchors, defined with `&` , let you identify an item in your YAML. That item can later be referenced with an alias, defined with `*`.
  * Example: `&failed_login` (anchor) and `*failed_login` (alias)
  * Do not use anchors and aliases within `Tests`. Doing so can cause [errors](https://docs.panther.com/detections/correlation-rules/..#correlation-rule-errors).

## Correlation rule top-level fields

The table below contains all available keys for YAML detections. Required fields are in **bold**.

<table data-header-hidden data-full-width="false"><thead><tr><th width="183">Field Name</th><th width="330">Description</th><th width="373.5081967213115">Expected Value</th></tr></thead><tbody><tr><td>Field Name</td><td>Description</td><td>Expected Value</td></tr><tr><td><strong><code>AnalysisType</code></strong></td><td>Indicates whether this analysis is a rule, scheduled_rule, policy, correlation_rule, or global</td><td><code>correlation_rule</code></td></tr><tr><td><strong><code>Enabled</code></strong></td><td>Whether this correlation_rule is enabled</td><td>Boolean</td></tr><tr><td><strong><code>RuleID</code></strong></td><td>The unique identifier of the rule</td><td>String</td></tr><tr><td><strong><code>Severity</code></strong></td><td>Which <a href="../../rules#alert-severity">severity</a> an associated alert should have</td><td>One of the following strings: <code>Info</code>, <code>Low</code>, <code>Medium</code>, <code>High</code>, or <code>Critical</code></td></tr><tr><td><a href="#detection-fields"><strong><code>Detection</code></strong></a></td><td>Correlation rule definition including sequence or group</td><td>See <a href="#detection-fields">Detection fields</a></td></tr><tr><td><code>CreateAlert</code></td><td>Whether the correlation rule should create an alert or not (default: true)</td><td>Boolean</td></tr><tr><td><code>Description</code></td><td>A brief description of the rule</td><td>String</td></tr><tr><td><code>DisplayName</code></td><td>A user-friendly name to show in the Panther Console and alerts. The <code>RuleID</code> will be displayed if this field is not set.</td><td>String</td></tr><tr><td><code>OutputIds</code></td><td>Static destination overrides. These will be used to determine how alerts from this rule are routed, taking priority over default routing based on severity.</td><td>List of strings</td></tr><tr><td><code>Reference</code></td><td>The reason this rule exists, often a link to documentation</td><td>String</td></tr><tr><td><code>Reports</code></td><td>A mapping of framework or report names to values this rule covers for that framework</td><td>Map of strings to list of strings</td></tr><tr><td><code>Runbook</code></td><td>The actions to be carried out if this rule returns an alert. It's recommended to provide a descriptive runbook, as <a href="../../../alerts#panther-ai-alert-triage">Panther AI alert triage</a> will take it into consideration.</td><td>String</td></tr><tr><td><code>SummaryAttributes</code></td><td>A list of fields that alerts should summarize.</td><td>List of strings</td></tr><tr><td><code>Tags</code></td><td>Tags used to categorize this rule</td><td>List of strings</td></tr><tr><td><a href="#tests-fields"><code>Tests</code></a></td><td>Defines unit tests for this detection.</td><td>See <a href="#tests-fields">Tests fields</a></td></tr><tr><td><code>CreatedBy</code></td><td>The author of this detection. Can be set as a Panther user UUID, email address, or an arbitrary text value. Learn more in <a href="../../../panther-developer-workflows/detections-repo/pat/pat-commands#the-createdby-detection-field">The <code>CreatedBy</code> detection field</a>.</td><td>String</td></tr></tbody></table>

## `Detection` fields

<table data-full-width="false"><thead><tr><th width="178">Name</th><th width="73">Type</th><th width="246">Validation</th><th width="262">Description</th></tr></thead><tbody><tr><td><a href="#schedule-fields"><strong><code>Schedule</code></strong></a></td><td>Object</td><td>N/A</td><td>The scheduling of a correlation rule</td></tr><tr><td><code>EventEvaluationOrder</code></td><td>String</td><td><p>Accepted values:</p><ul><li><code>Chronological</code> (default)</li><li><code>ReverseChronological</code></li></ul></td><td><br>If <code>Chronological</code>, events are analyzed from oldest to newest.<br>If <code>ReverseChronological</code>, events are analyzed from newest to oldest.</td></tr><tr><td><code>LookbackWindowMinutes</code></td><td>Scalar</td><td><p><code>15</code> ≤ x ≤ <code>21600</code> (15 days)</p><p>Default: <code>15</code></p></td><td>Indicates how many minutes in the past a correlation rule should look to evaluate the group or sequence. <a href="..#setting-the-schedule-and-lookback-window">Learn more about <code>LookbackWindowMinutes</code> on Correlation Rules</a></td></tr><tr><td><a href="#group-and-sequence-fields-rule-references"><strong><code>Group</code></strong></a></td><td>List</td><td><p><code>2</code> ≤ x ≤ <code>50</code><br></p><p>Only one of <code>Group</code> or <code>Sequence</code> is required/allowed</p></td><td>A list of <a href="#group-and-sequence-fields-rule-references">rule references</a> that define the group<br><br><a href="..#group-correlation-rules">Learn more about groups on Correlation Rules</a></td></tr><tr><td><code>MinMatchCount</code></td><td>Scalar</td><td><p>Only can be used if <code>Group</code> is present</p><p><code>2</code> ≤ x &#x3C; number of rules in <code>Group</code></p><p>Cannot use when number of rules in <code>Group</code> is <code>2</code> or ><code>8</code>.</p><p>Cannot use when any rules in <code>Group</code> use <code>Absence: true</code></p></td><td><code>MinMatchCount</code> specifies the minimum number of individual rules, scheduled rules, or correlation rules (defined in <code>Group</code>) that must match in order for this correlation rule to match. Learn more about <code>MinMatchCount</code> <a href="..#minmatchcount">on Correlation Rules</a>.</td></tr><tr><td><a href="#matchcriteria-fields"><code>MatchCriteria</code></a></td><td>List</td><td><p>Only can be used if <code>Group</code> is present</p><p>x ≥ <code>1</code></p><p><br>Length must equal the length of <code>Group</code></p></td><td>A mapping of MatchCriteria that defines the event fields the group should match on to pass</td></tr><tr><td><a href="#group-and-sequence-fields-rule-references"><strong><code>Sequence</code></strong></a></td><td>List</td><td><p><code>2</code> ≤ x ≤ <code>50</code></p><p>Only one of <code>Group</code> or <code>Sequence</code> is required/allowed</p></td><td><p>A list of <a href="#group-and-sequence-fields-rule-references">rule references</a> that define the sequence<br></p><p><a href="..#sequence-correlation-rules">Learn more about sequences on Correlation Rules</a></p></td></tr><tr><td><a href="#transitions-fields"><code>Transitions</code></a></td><td>List</td><td><p>Only can be used if <code>Sequence</code> is present</p><p>x ≥ <code>1</code></p></td><td>A list of transitions that defines the requirements to transition from one step of the sequence to the next. This can include event values to match on or a time frame.</td></tr></tbody></table>

### `Schedule` fields

Each correlation rule runs on a schedule. The `Schedule` field defines that interval. Learn more about setting `Schedule` [on Correlation Rules](https://docs.panther.com/detections/correlation-rules/..#setting-schedule).

<table data-full-width="false"><thead><tr><th width="191">Name</th><th width="81">Type</th><th width="262">Validation</th><th width="511">Description</th></tr></thead><tbody><tr><td><strong><code>CronExpression</code></strong></td><td>String</td><td>Only one of <code>CronExpression</code> or <code>RateMinutes</code> is required/allowed</td><td>A cron expression to describe how often a correlation rule should run. Learn more about <code>CronExpression</code> format in <a href="../../../search/scheduled-searches#how-to-use-the-scheduled-search-crontab">How to use the Scheduled Search crontab</a>.</td></tr><tr><td><strong><code>RateMinutes</code></strong></td><td>Scalar</td><td>x ≥ 2<br>Only one of <code>CronExpression</code> or <code>RateMinutes</code> is required</td><td>The rate of minutes to describe how often a correlation rule should run</td></tr><tr><td><strong><code>TimeoutMinutes</code></strong></td><td>Scalar</td><td>x ≤ <code>RateMinutes</code></td><td>The time frame in which a correlation rule can run. If a correlation rule takes longer than this period of time to evaluate, it will be cancelled for that time frame and a system health notification will be generated</td></tr></tbody></table>

### `Group` and `Sequence` fields (rule references)

Within `Group` and `Sequence`, include a list of references to the rules included in this correlation rule. Each rule reference can include the following fields:

<table data-full-width="false"><thead><tr><th width="181">Name</th><th width="99">Type</th><th width="199">Validation</th><th width="489">Description</th></tr></thead><tbody><tr><td><strong><code>ID</code></strong></td><td>String</td><td>Only required if using <code>Transitions</code> or <code>MatchCriteria</code></td><td>A unique identifier to a rule being referenced in a sequence or group<br></td></tr><tr><td><strong><code>RuleID</code></strong></td><td>String</td><td>The ruleID must exist in Panther beforehand</td><td>The id of the rule, scheduled rule, or correlation rule to include in the sequence or group</td></tr><tr><td><code>MinMatchCount</code></td><td>Scalar</td><td>x ≥ <code>1</code><br>Default: <code>1</code></td><td>The minimum number of signals required for this step in the sequence to pass</td></tr><tr><td><code>MaxMatchCount</code></td><td>Scalar</td><td>x ≥ 0<br>x > <code>MinMatchCount</code> (if set)</td><td>The maximum number of signals allowed for this step in the sequence to pass</td></tr><tr><td><code>Absence</code></td><td>Boolean</td><td>N/A</td><td>Whether the absence of a signal needs to be true for the step to pass</td></tr></tbody></table>

### `MatchCriteria` fields

The `MatchCriteria` key can be used alongside `Group` for more granular results. It indicates which event field each rule is matched on. There can only be one type of field matched on per correlation rule (e.g., *all* IP address fields or *all* email address fields).

The `MatchCriteria` field contains a key that is a unique label. That key contains a list of `GroupID` and `Match` pairings.

See examples of a `Group` with and without `MatchCriteria` on [Correlation Rules](https://docs.panther.com/detections/correlation-rules/..#matchcriteria).\\

<table data-full-width="false"><thead><tr><th width="140">Name</th><th width="82">Type</th><th width="648">Validation</th><th width="653">Description</th></tr></thead><tbody><tr><td><strong><code>GroupID</code></strong></td><td>String</td><td></td><td>The <code>ID</code> defined in <a href="#group-and-sequence-fields-rule-references"><code>Group</code></a>. Used to link the rule reference to the field that should be matched on</td></tr><tr><td><strong><code>Match</code></strong></td><td>String</td><td><ul><li>For rules associated to multiple log types, scheduled rules, or correlation rules: only <a href="../../search/panther-fields"><code>p_</code> fields</a> are allowed</li><li>For rules associated to only one log type: any field is allowed</li><li>Validated based on the log schema. Cannot end on a field of type <code>object</code> or <code>JSON</code>. Anything within a JSON object is not validated.</li><li>Must be the same type as other <code>Match</code> values</li><li>Must use <a href="https://learn.microsoft.com/en-us/azure/data-explorer/kusto/query/scalar-data-types/dynamic#dynamic-object-accessors">Kusto Query Language (KQL) syntax</a></li></ul></td><td><p>The field in the event for the rule referenced that should be matched on</p><p>e.g., <code>p_alert_context.username</code></p></td></tr></tbody></table>

### `Transitions` fields

The `Transitions` key, used only alongside `Sequence`, defines how one step can traverse to another.

Transitions must be in the same order as the rules listed under `Sequence`.

Learn more about transitions [on Correlation Rules](https://docs.panther.com/detections/correlation-rules/..#transitions).

<table data-full-width="false"><thead><tr><th width="268">Name</th><th width="107">Type</th><th width="110">Validation</th><th width="546">Description</th></tr></thead><tbody><tr><td><code>ID</code></td><td>String</td><td>N/A</td><td>A unique identifier to a transition in a sequence.<br></td></tr><tr><td><strong><code>From</code></strong></td><td>String</td><td></td><td>The <code>ID</code> in the <a href="#group-and-sequence-fields-rule-references">rule references</a> section of a sequence that indicates the starting step.</td></tr><tr><td><strong><code>To</code></strong></td><td>String</td><td></td><td>The <code>ID</code> in the <a href="#group-and-sequence-fields-rule-references">rule references</a> section of a sequence that indicates the ending step.</td></tr><tr><td><code>WithinTimeFrameMinutes</code></td><td>Scalar</td><td><code>1</code> ≤ x ≤ <code>1440</code> AND x ≤ <code>LookbackWindowMinutes</code></td><td>The time frame in minutes within which two steps in a sequence (defined by <code>From</code> and <code>To)</code> must occur in order to pass.</td></tr><tr><td><a href="#match-fields"><code>Match</code></a></td><td>List</td><td>len(x) = <code>1</code></td><td>Defines which event fields must match</td></tr></tbody></table>

#### `Match` fields

`Match` is a child field of `Transitions`, used with `Sequence`. These fields allow you to define which event fields, for each rule, scheduled rule, or correlation rule, must have matching values.

There can only be one type of field matched on per correlation rule (e.g., *all* IP address fields or *all* email address fields).

<details>

<summary>Example using same match values</summary>

```yaml
Detection:
  - Sequence:
      - ID: Failed Login
        RuleID: Standard.BruteForceByIP
        MinMatchCount: 10
      - ID: Successful Login
        RuleID: Okta.Login.Success
    Transitions:
      - ID: Brute Force Login Success
        From: Failed Login
        To: Successful Login
        WithinTimeFrameMinutes: 10
        Match:
          - On: p_alert_context.ip     // these should be the same
      - ID: Gained Root Access
        From: Successful Login
        To: Root Login
        Match:
          - From: p_alert_context.ip  // these should be the same
            To: p_alert_context.sourceIPAddress
```

</details>

<table data-full-width="false"><thead><tr><th width="88.14453125">Name</th><th width="92">Key type</th><th width="571.6771240234375">Description</th><th width="543.953125">Validation</th></tr></thead><tbody><tr><td><strong><code>On</code></strong></td><td>String</td><td><p>The field in the event referenced by the <a href="#transitions-fields"><code>Transitions</code></a> <code>From</code> and <code>To</code> that should be matched on. Used when the event fields' names match exactly. E.g., <code>p_udm.x</code> or <code>p_any_usernames</code></p><p><br>If the value is an array-type field, at least one item must be present in both arrays (assuming <code>Absence: true</code> is not being used for either event).</p></td><td><ul><li><code>From</code> and <code>To</code> must be empty</li><li>For rules associated to multiple log types, scheduled rules, or correlation rules: only <a href="../../search/panther-fields"><code>p_</code> fields</a> are allowed</li><li>For rules associated to only one log type: any field is allowed</li><li>Validated based on the log schema. Cannot end on a field of type <code>object</code> or <code>JSON</code>. Anything within a JSON object is not validated.</li><li>Must use <a href="https://learn.microsoft.com/en-us/azure/data-explorer/kusto/query/scalar-data-types/dynamic#dynamic-object-accessors">KQL syntax</a></li></ul></td></tr><tr><td><strong><code>From</code></strong></td><td>String</td><td><p>The field in the event referenced by the <a href="#transitions-fields"><code>Transitions</code></a> <code>From</code> and <code>To</code> that should be matched on.</p><p>Used when the event fields' names don't exactly match, but should be grouped together. E.g., <code>p_alert_context.username = p_alert_context.user</code></p><p><br>If the value is an array-type field, <code>To</code> can be either (assuming <code>Absence: true</code> is not being used for either event):</p><ul><li>A scalar field, in which case the <code>To</code> value must be in the <code>From</code> array</li><li>Another array, in which case at least one item must be present in both arrays</li></ul></td><td><ul><li><code>On</code> must be empty</li><li>For rules associated to multiple log types, scheduled rules, or correlation rules: only <a href="../../search/panther-fields"><code>p_</code> fields</a> are allowed</li><li>For rules associated to only one log type: any field is allowed</li><li>Validated based on the log schema. Cannot end on a field of type <code>object</code> or <code>JSON</code>. Anything within a JSON object is not validated.</li><li>Must use <a href="https://learn.microsoft.com/en-us/azure/data-explorer/kusto/query/scalar-data-types/dynamic#dynamic-object-accessors">KQL syntax</a></li></ul></td></tr><tr><td><strong><code>To</code></strong></td><td>String</td><td><p>The field in the event referenced by the <a href="#transitions-fields"><code>Transitions</code></a> <code>From</code> and <code>To</code> that should be matched on.</p><p>Used when the event fields' names don't exactly match, but should be grouped together. E.g., <code>p_alert_context.username = p_alert_context.user</code><br><br>If the value is an array-type field, <code>From</code> can be either (assuming <code>Absence: true</code> is not being used for either event):</p><ul><li>A scalar field, in which case the <code>From</code> value must be in the <code>To</code> array</li><li>Another array, in which case at least one item must be present in both arrays</li></ul></td><td><ul><li><code>On</code> must be empty</li><li>For rules associated to multiple log types, scheduled rules, or correlation rules: only <a href="../../search/panther-fields"><code>p_</code> fields</a> are allowed</li><li>For rules associated to only one log type: any field is allowed</li><li>Validated based on the log schema. Cannot end on a field of type <code>object</code> or <code>JSON</code>. Anything within a JSON object is not validated.</li><li>Must use <a href="https://learn.microsoft.com/en-us/azure/data-explorer/kusto/query/scalar-data-types/dynamic#dynamic-object-accessors">KQL syntax</a></li></ul></td></tr></tbody></table>

## `Tests` fields

Do not use [anchors and aliases](#correlation-rule-writing-tips) within `Tests`. Doing so can cause [errors](https://docs.panther.com/detections/correlation-rules/..#correlation-rule-errors).

<table data-full-width="false"><thead><tr><th width="195">Name</th><th width="197">Key type</th><th width="646">Description</th></tr></thead><tbody><tr><td><strong><code>Name</code></strong></td><td>String</td><td>Descriptive name for the test case. All test cases must have unique names.</td></tr><tr><td><strong><code>ExpectedResult</code></strong></td><td>Boolean</td><td>Whether the correlation rule should generate a match (i.e., return <code>True</code> or <code>False</code>) for this test case. For example if the test specifies <code>ExpectedResult: False</code> and the correlation rule <em>does</em> generate a match (i.e., return <code>True</code>), the test will fail because the expectation does not match the actual behavior.</td></tr><tr><td><strong><code>RuleOutputs</code></strong></td><td>Array&#x3C;<a href="#ruleoutput-fields"><code>RuleOutput</code></a>></td><td>A list of <a href="#ruleoutput-fields"><code>RuleOutput</code></a> objects. Each <a href="#ruleoutput-fields"><code>RuleOutput</code></a> represents mock <a href="../signals">signal(s)</a> generated from a particular rule referenced in your correlation rule definition, including definitions for which event fields/values matched.</td></tr></tbody></table>

### `RuleOutput` fields

<table data-full-width="false"><thead><tr><th width="127">Name</th><th width="256">Key type</th><th width="648">Description</th></tr></thead><tbody><tr><td><strong><code>ID</code></strong></td><td>String</td><td>A reference to one of the steps in your correlation rule's <code>Group</code> or <code>Sequence</code> field. If the items in your <a href="#group-and-sequence-fields-rule-references"><code>Group</code></a> or <a href="#group-and-sequence-fields-rule-references"><code>Sequence</code></a> have <code>ID</code>s defined, this value must match that <code>ID</code>. If the items in your <code>Group</code> or <code>Sequence</code> do not <code>ID</code>s defined, this value must match the <code>RuleID</code> associated with that <code>Group</code> or <code>Sequence</code> item.</td></tr><tr><td><strong><code>Matches</code></strong></td><td>Object&#x3C;String, <a href="#matchvalue-fields"><code>MatchValue</code></a>></td><td>An object that represents the event fields and values that this step in a <code>Group</code> or <code>Sequence</code> matched on. The key is the field name matched on (subfields can be specified using JSON path notation, e.g.,<code>p_alert_context.username</code>)—and the value is a <a href="#matchvalue-fields"><code>MatchValue</code></a> object.</td></tr></tbody></table>

### `MatchValue` fields

A `MatchValue` object defines an event value that was matched on, and the times the match(es) occurred.

{% hint style="warning" %}
Currently, a correlation rule may only specify one event value to match on throughout, therefore this object may only contain a single key/value pair.
{% endhint %}

#### Example:

```yaml
Matches:
    p_any_emails: # String (field name matched on)
        ali.ford@example.net: [0, 1, 2] # MatchValue object (value using relative timestamps)
```

or

```yaml
Matches:
    p_any_ip_addresses: # String (field name matched on)
        1.2.3.4: # MatchValue key
            - "2006-01-02T15:04:05Z" # MatchValue value (using absolute timestamps)
```

<table data-full-width="false"><thead><tr><th width="232">Key</th><th width="151">Key type</th><th width="625">Description</th></tr></thead><tbody><tr><td><strong><code>[Value matched on]</code>(String)</strong></td><td>Array&#x3C;String> OR Array&#x3C;Number></td><td><p>The key is an arbitrary string that your rule matched on. The value is an array of timestamps where the timestamps can either be relative (to the time the test case is ran) or absolute:</p><ul><li>Relative timestamps: Represented as an array of numbers where the number is how many minutes the match occurred after the start of the test case scenario.</li><li>Absolute timestamps: Represented as an array of timestamps in RFC3339 format.</li></ul><p>A single test cannot mix relative and absolute timestamps. For example, if a test uses absolute timestamps for one <code>RuleOutput</code>, it must use absolute timestamps for all <code>RuleOutputs</code>.</p></td></tr></tbody></table>