# Correlation Rules (Beta)
## **Overview**
{% hint style="info" %}
Correlation rules are in open beta starting with Panther version 1.108, and are available to all customers. Please share any bug reports and feature requests with your Panther support team.
{% endhint %}
{% hint style="warning" %}
This feature is currently not compatible with a [Databricks backend.](https://docs.panther.com/search/backend/databricks)
{% endhint %}
Using correlation rules in Panther, you can track multiple actions across log types. In a correlation rule, you specify a [group](#group-correlation-rules) or specific [sequence](#sequence-correlation-rules) of [signals](https://docs.panther.com/detections/signals) that must occur in a certain window of time in order to be considered a match—then generating a signal and optionally, an [alert](https://docs.panther.com/alerts).
You can also include the *absence* of a signal in your correlation rule criteria. Because matches on correlation rules are determined by signals, not by rule matches or alerts, it's possible to include rules, scheduled rules, and correlation rules that [have alerting disabled](https://docs.panther.com/signals#how-to-create-a-rule-that-only-produces-signals).
Correlation rules may be particularly useful if you want to generate an alert when, for example:
* A certain Okta user logs in successfully after at least one hundred unsuccessful login attempts, then logs in to AWS as a root user ([see full example below](#brute-force-okta-login-to-aws-root-login))
* Advanced security settings were disabled for a GitHub repository, which then was not archived ([see full example below](#github-repository-security-policy-disabled-without-subsequent-archival))
[Learn how to create custom correlation rules below](#how-to-create-a-correlation-rule), and more about the YAML keys that make up correlation rules on [Correlation Rule Reference](https://docs.panther.com/detections/correlation-rules/correlation-rule-reference). You can also leverage [Panther-managed correlation rules](https://github.com/panther-labs/panther-analysis/tree/main/correlation_rules).
{% hint style="warning" %}
Using correlation rules may increase your Snowflake compute costs. To learn how to make your correlation rules more cost efficient, see the guidelines in [Making correlation rules more efficient](#making-correlation-rules-more-efficient), below.
{% endhint %}
## How correlation rules work
Correlation rules are written in YAML and reference previously created [rules](https://docs.panther.com/detections/rules), [scheduled rules](https://docs.panther.com/detections/rules), and/or correlation rules. Each correlation rule [runs on a schedule](#setting-schedule), and [defines a "lookback window,"](#setting-lookbackwindowminutes) i.e., the amount of time in the past the rule should look to find [signals](https://docs.panther.com/detections/signals) (or absences of signals).
You can apply additional criteria to your correlation rule, such as:
* Requiring a minimum number of signals to be found for a certain rule, or a maximum
* Requiring certain event values to match from one rule to another (e.g., requiring signals for all individual rules to contain the same IP address)
* Requiring subsequent steps in a [sequence](#sequence-correlation-rules) to have occurred within a certain amount of time
### What happens when there is a match on a correlation rule
Matches on correlation rules generate [signals](https://docs.panther.com/detections/signals). When a correlation rule has alerting enabled, rule matches are generated, which can create alerts according to the correlation rule's [deduplication configuration](#deduplication-of-events). [Learn more about the difference between signals, rule matches, and alerts here](https://docs.panther.com/detections/..#signals-vs.-rule-matches-vs.-alerts).
When an alert is generated for a correlation rule, the individual rules, scheduled rules, and correlation rules referenced in the correlation rule may or may not also generate their own alerts. This depends on:
* Whether each individual rule, scheduled rule, and correlation rule has alerting enabled
* The `Threshold` value on individual rules or scheduled rules, and the `MinMatchCount` value on the correlation rule. An edge case in which the correlation rule could generate an alert but not the rules that make it up is if the event threshold for an individual rule (set with the `Threshold` key) is higher than the `MinMatchCount` on the correlation rule, and the number of actual rule matches is somewhere in the middle.
### Group vs. sequence
There are two types of correlation rules: [groups](#group-correlation-rules) and [sequences](#sequence-correlation-rules). Both group and sequence correlation rules define a collection of rules for which signals must be found (or *not* found, by setting `Absence: true`).
Group correlation rules will generate an alert if all rules have produced signals (or absences), no matter the order in which signals were found. Sequence rules, however, define a particular order in which signals (or absences) must be found in order to generate an alert.
### Setting the schedule and lookback window
Each correlation rule defines both of the following:
* A schedule: Defined by the [`Schedule`](https://docs.panther.com/detections/correlation-rule-reference#schedule-fields) field (which uses either `RateMinutes` or `CronExpression`), the schedule indicates how frequently the correlation rule should run.
* A lookback window: Defined by the `LookbackWindowMinutes` field, the lookback window specifies the number of minutes in the past the correlation rule should look to find [signals](https://docs.panther.com/detections/signals) (or absences of signals) for the rules, scheduled rules, or correlation rules included in its group or sequence.
When setting the values of `Schedule` and `LookbackWindowMinutes`, it's recommended to take a few factors into account:
* How important it is for you to be alerted in a timely manner for matches on this correlation rule
* For example, it might suffice to run lower priority correlation rules only every 24 hours; alternatively, you might want to run higher priority correlation rules every 15 minutes.
* How far apart in time the first and last signal the correlation rule is looking for can be to be considered part of the same occurrence that should generate a match
* See [Ensuring signals aren't split between lookback windows](#ensuring-signals-arent-split-between-lookback-windows)
* The maximum latency expected from the data source(s) evaluated by the rules associated to the correlation rule
* See [Accounting for log source latency in `LookbackWindowMinutes`](#accounting-for-log-source-latency-in-lookbackwindowminutes)
`Schedule` and `LookbackWindowMinutes` values can have an impact on Snowflake compute costs. See [Making correlation rules more efficient](#making-correlation-rules-more-efficient) for more information.
#### Ensuring signals aren't split between lookback windows
It's recommended to set your lookback window such that you won't miss occurrences of the thing you're searching for because the required signals were split across the lookback windows of multiple correlation rule runs.
Example of the situation to avoid
To illustrate this situation we're trying to avoid, take this simplified correlation rule configuration:
```yaml
# Bad example; do not replicate
- Group:
- RuleID: First.Rule
- RuleID: Second.Rule
- RuleID: Third.Rule
Schedule:
RateMinutes: 60
LookbackWindowMinutes: 90
```
In this scenario, every hour, the correlation rule looks back over the previous 90 minutes to find signals for the three rules included in the `Group`. Let's say each of these rules matched/generated a signal at the times below:
* Signal for `First.Rule` generated at 12:58 PM
* Signal for `Second.Rule` generated at 1:05 PM
* Signal for `Third.Rule` generated at 1:40 PM
The correlation rule is scheduled to run at 30 minutes past the hour. It runs at the following times:
* 1:30 PM (looking back to 12:00 PM): Sees only the first and second signals (12:58 and 1:05 PM), and therefore does not generate a match.
* 2:30 PM (looking back to 1:00 PM): Sees only the second and third signals (1:05 and 1:40 PM), and again does not generate a match.
It's likely that the author of this correlation rule would have expected that if signals for all three rules were generated within 42 minutes (between 12:58 and 1:40 PM), the correlation rule would have matched. However, because all three of the required signals weren't found within the lookback window of a single run of the correlation rule, a match was not generated.
To ensure the signals required for your correlation rule to match are not split across different runs (i.e., they are included in the same lookback window), first determine how far apart in time the first and last signal can be in order to be considered part of the same occurrence that should be alerted on. For convenience, let's call this value "maximum signal timespan minutes."
(This should not be confused with [`WithinTimeFrameMinutes`](https://docs.panther.com/detections/correlation-rules/correlation-rule-reference#transitions-fields), which is the time frame within which two steps in a sequence must occur in order to pass. "Maximum signal timespan minutes" and `WithinTimeFrameMinutes` can be equal if the correlation rule is a sequence and specifies only two steps.)
It's recommended to configure your `LookbackWindowMinutes` value so that every possible window of length "maximum signal timespan minutes" is covered within at least one run of the correlation rule. It's generally possible to do this with the following formula:
* If `RateMinutes` <= "maximum signal timespan minutes":
* `LookbackWindowMinutes` = 2 x "maximum signal timespan minutes" + log ingest latency
* If `RateMinutes` > "maximum signal timespan minutes":
* `LookbackWindowMinutes` = "maximum signal timespan minutes" + `RateMinutes` + log ingest latency
To understand the inclusion of log ingest latency, see [Accounting for log source latency in `LookbackWindowMinutes`](#accounting-for-log-source-latency-in-lookbackwindowminutes)
#### Accounting for log source latency in `LookbackWindowMinutes`
Log ingest latency is the maximum latency expected from the data sources evaluated by the rules associated to the correlation rule.
Because signals are fetched within the lookback window based on the time their associated event occurred (`p_event_time`), *not* the time the event was ingested into Panther (`p_parse_time`), it's necessary to take ingest latency into account when determining `LookbackWindowMinutes` to ensure you are processing all "new" data since the last time the correlation rule ran.
For example, if you have configured your correlation rule to run every hour (e.g., by setting `RateMinutes` to `60`) and the source of the logs processed by the rules associated to the correlation rule states that it may delay log forwarding to Panther by up to three hours, you might set `LookbackWindowMinutes` to `60 + 3*60`, or `240`. To further illustrate this, consider that because of the three-hour ingestion delay, data received by Panther at `9:01am` can have a `p_event_time` of as early as `6:01am`. If the correlation rule runs every hour, on the hour, at `10:00am`, it would need to look back to at least `6:01am`.
### Deduplication of events
The deduplication period in correlation rules is the value of the [`LookbackWindowMinutes`](https://docs.panther.com/detections/correlation-rule-reference#detection-fields) field. This means overlapping correlation rules within the same `LookbackWindowMinutes` time frame will only contain the unique events that caused that correlation rule to match.
Deduplication set on individual rules and scheduled rules referenced in a correlation rule (with `dedup()`, `DedupPeriodMinutes`, `Threshold`, or set in the Console) is not applicable to the correlation rule.
### Correlation rule errors
While working with correlation rules, you may receive one of the following errors:
* [Simple Detections error code](https://docs.panther.com/detections/rules/writing-simple-detections/simple-detection-error-codes): You have used incorrect syntax to construct a correlation rule
* [Detection error](https://docs.panther.com/alerts#overview): Execution of your correlation rule has failed
* [System error](https://docs.panther.com/system-configuration/notifications/system-errors): Your correlation rule has timed out (likely due to a too-large `LookbackWindowMinutes` value)
## Group correlation rules
A group correlation rule defines a collection of rules for which [signals](https://docs.panther.com/detections/signals) (or a lack of signals) must occur in a given lookback window. The signals can occur in any order.
If you would like the collection of events to occur in a specific order, use a [sequence correlation rule](#sequence-correlation-rules) instead.
### MatchCriteria
In a group correlation rule, the `MatchCriteria` key defines fields, per rule, scheduled rule, and correlation rule, that must have matching values in order for the correlation rule to pass.
For rules associated to multiple log types, scheduled rules, or correlation rules, only [`p_` fields](https://docs.panther.com/search/panther-fields) can be matched on. (For rules associated to only one log type, any field may be matched on.)
If match criteria is not defined, because there is no requirement for certain event field values to match, the correlation rule is less specific.
Learn more about `MatchCriteria` on [Correlation Rule Reference](https://docs.panther.com/detections/correlation-rule-reference#matchcriteria-fields).
### MinMatchCount
In a group correlation rule, `MinMatchCount` is an optional field that specifies the minimum number of individual rules, scheduled rules, or correlation rules (defined in `Group`) that must match in order for this correlation rule to match.
For example, if you list five rules within `Group` and add `MinMatchCount: 2`, the correlation rule will match if any two of the five rules generate a signal.
`MinMatchCount` is also an available field on individual rules defined in `Group`. See both fields in use together in the [Group with MinMatchCount example](#group-with-minmatchcount), below.
### Group examples
The examples below refer to these JSON events:
Example events
These sample events contain a subset of fields, for readability.
```json
[
{
"p_rule_id": "Standard.BruteForceByIP",
"event_type": "failed_login",
"p_event_time": "2023-12-08 10:27:03.496000000",
"p_alert_context": {
"ip": "136.24.229.58",
"geolocation": "San Francisco, California in United States"
}
},
{
"p_rule_id": "Standard.BruteForceByIP",
"event_type": "failed_login",
"p_event_time": "2023-12-08 10:27:03.623000000",
"p_alert_context": {
"ip": "136.24.229.58",
"geolocation": "San Francisco, California in United States"
}
},
{
"p_rule_id": "Standard.BruteForceByIP",
"event_type": "failed_login",
"p_event_time": "2023-12-08 10:27:03.861000000",
"p_alert_context": {
"ip": "136.24.229.58",
"geolocation": "San Francisco, California in United States"
}
},
{
"p_rule_id": "Okta.Login.Success",
"event_type": "successful_login",
"p_event_time": "2023-12-08 10:27:54.317000000",
"p_alert_context": {
"ip": "136.24.229.58",
"geolocation": "San Francisco, California in United States"
}
},
{
"p_rule_id": "AWS.Console.RootLogin",
"eventType": "AwsApiCall",
"p_event_time": "2023-12-08 10:30:13.274000000",
"p_alert_context": {
"sourceIPAddress": "136.24.229.58"
}
}
]
```
{% tabs %}
{% tab title="Group with MatchCriteria" %}
In this example, `MatchCriteria` specifies that the `IP` field in all four rules must contain the same value.
```yaml
Detection:
- Group:
- ID: &failed_login Failed Login
RuleID: Standard.BruteForceByIP
MinMatchCount: 7
- ID: &successful_login Successful Login
RuleID: Okta.Login.Success
- ID: &root_access Root Access
RuleID: AWS.Console.RootLogin
- ID: &missing_crowdstrike Missing Crowdstrike
RuleID: Crowdstrike.Detection.passthrough
Absence: true
MatchCriteria:
ip:
- GroupID: *failed_login
Match: p_alert_context.ip
- GroupID: *successful_login
Match: p_alert_context.ip
- GroupID: *root_access
Match: p_alert_context.sourceIPAddress
- GroupID: *missing_crowdstrike
Match: p_any_ip_addresses
EventEvaluationOrder: Chronological
LookbackWindowMinutes: 60
Schedule:
RateMinutes: 30
TimeoutMinutes: 7
```
{% endtab %}
{% tab title="Group without MatchCriteria" %}
In this example, if signals are found for the first three rules but *not* the fourth one, the correlation rule will pass.
```yaml
Detection:
- Group:
- RuleID: Standard.BruteForceByIP
MinMatchCount: 7
- RuleID: Okta.Login.Success
- RuleID: AWS.Console.RootLogin
- RuleID: Crowdstrike.Detection.passthrough
Absence: true
LookbackWindowMinutes: 60
Schedule:
RateMinutes: 30
TimeoutMinutes: 3
```
{% endtab %}
{% tab title="Group with MinMatchCount" %}
In this example, the `MinMatchCount` value under `Detection` specifies that if signals are found for any two of the three rules defined in `Group`, the correlation rule will match. Note there is also a `MinMatchCount` defined on a rule in `Group`, which is used in combination with the outer-level `MinMatchCount`. See the list below for examples of when this correlation rule would pass.
```yaml
Detection:
- Group:
- RuleID: Standard.BruteForceByIP
MinMatchCount: 7
- RuleID: Okta.Login.Success
- RuleID: AWS.Console.RootLogin
MinMatchCount: 2
LookbackWindowMinutes: 60
Schedule:
RateMinutes: 30
TimeoutMinutes: 3
```
The above correlation rule would pass in any of the following scenarios:
* When there is at least one signal from `Okta.Login.Success` and at least one signal from `AWS.Console.RootLogin`
* When there is at least seven signals from `Standard.BruteForceByIP` and at least one signal from `Okta.Login.Success`
* When there is at least seven signals from `Standard.BruteForceByIP` and at least one signal from `AWS.Console.RootLogin`
* When there are signals from all three rules (with at least seven signals from `Standard.BruteForceByIP`)
{% endtab %}
{% endtabs %}
## Sequence correlation rules
A sequence correlation rule defines a collection of rules for which [signals](https://docs.panther.com/detections/signals) (or a lack of signals) must occur in a specific order within a given lookback window.
The order of the sequence is defined by the order of rules defined within the `Sequence` key.
If you would like the correlation rule to match merely if all rules have signals (or absences), without requiring a specific order, use a [group correlation rule](#group-correlation-rules) instead.
### Transitions
Within a sequence, you can optionally define transitions. Transitions define additional criteria for how one step in a sequence can traverse to the next, including how much time can occur between steps, as well as which event fields must have matching values. Using `Transitions` with `WithinTimeFrameMinutes` and/or `Match` increases the specificity of your correlation rule.
If transitions are defined, there must be one fewer transition than the number of rules included in `Sequence`. Additionally, the items within `Transitions` must be in the same order as the `Sequence` list.
Currently, there can only be one type of field matched on per correlation rule (e.g., *all* IP address fields or *all* email address fields). For rules associated to multiple log types, scheduled rules, or correlation rules, only [`p_` fields](https://docs.panther.com/search/panther-fields) can be matched on. (For rules associated to only one log type, any field may be matched on.)
Learn more about transitions on [Correlation Rule Reference](https://docs.panther.com/detections/correlation-rule-reference#transitions-fields).
### Sequence examples
The examples below refer to these JSON events:
Example events
These sample events contain a subset of fields, for readability.
```json
[
{
"p_rule_id": "Standard.BruteForceByIP",
"event_type": "failed_login",
"p_event_time": "2023-12-08 10:27:03.496000000",
"p_alert_context": {
"ip": "136.24.229.58",
"geolocation": "San Francisco, California in United States"
}
},
{
"p_rule_id": "Standard.BruteForceByIP",
"event_type": "failed_login",
"p_event_time": "2023-12-08 10:27:03.623000000",
"p_alert_context": {
"ip": "136.24.229.58",
"geolocation": "San Francisco, California in United States"
}
},
{
"p_rule_id": "Standard.BruteForceByIP",
"event_type": "failed_login",
"p_event_time": "2023-12-08 10:27:03.861000000",
"p_alert_context": {
"ip": "136.24.229.58",
"geolocation": "San Francisco, California in United States"
}
},
{
"p_rule_id": "Okta.Login.Success",
"event_type": "successful_login",
"p_event_time": "2023-12-08 10:27:54.317000000",
"p_alert_context": {
"ip": "136.24.229.58",
"geolocation": "San Francisco, California in United States"
}
},
{
"p_rule_id": "AWS.Console.RootLogin",
"eventType": "AwsApiCall",
"p_event_time": "2023-12-08 10:30:13.274000000",
"p_alert_context": {
"sourceIPAddress": "136.24.229.58"
}
}
]
```
{% tabs %}
{% tab title="Sequence w/ Transitions and Match " %}
Using `Transitions` with `Match` within `Sequence` is useful if you'd like to define event fields whose values must match.
```yaml
Detection:
- Sequence:
- ID: Failed Login
RuleID: Standard.BruteForceByIP
MinMatchCount: 7
- ID: Successful Login
RuleID: Okta.Login.Success
MinMatchCount: 1
- ID: Root Login
RuleID: AWS.Console.RootLogin
MinMatchCount: 1
- ID: Missing Crowdstrike
RuleID: Crowdstrike.Detection.passthrough
Absence: true
Transitions:
- ID: Brute Force Login Success
From: Failed Login
To: Successful Login
WithinTimeFrameMinutes: 10
Match:
- On: client.ipAddress
- ID: Gained Root Access
From: Successful Login
To: Root Login
Match:
- From: client.ipAddress
To: p_alert_context.sourceIPAddress
- ID: Absence of Crowdstrike
From: Root Login
To: Missing Crowdstrike
Match:
- From: p_alert_context.sourceIPAddress
To: p_any_ip_addresses
LookbackWindowMinutes: 60
EventEvaluationOrder: Chronological
Schedule:
RateMinutes: 30
TimeoutMinutes: 3
```
{% endtab %}
{% tab title="Sequence w/ Transitions w/o Match" %}
Using `Transitions` within `Sequence` without `Match` is useful if you'd just like to specify a time frame within which two steps in a sequence must occur, using `WithinTimeFrameMinutes`.
```yaml
Detection:
- Sequence:
- ID: Failed Login
RuleID: Standard.BruteForceByIP
MinMatchCount: 7
- ID: Successful Login
RuleID: Okta.Login.Success
- ID: Root Login
RuleID: AWS.Console.RootLogin
- ID: Missing Crowdstrike
RuleID: Crowdstrike.Detection.passthrough
Absence: true
Transitions:
- ID: Brute Force Login Success
From: Failed Login
To: Successful Login
WithinTimeFrameMinutes: 10 # Define timeframe
- ID: Gained Root Access
From: Successful Login
To: Root Login
- ID: Absence of Crowdstrike
From: Root Login
To: Missing Crowdstrike
LookbackWindowMinutes: 60
Schedule:
RateMinutes: 30
TimeoutMinutes: 3
```
{% endtab %}
{% tab title="Sequence w/o Transitions" %}
You can omit `Transitions` from within `Sequence` if you don't want to specify event fields to match on or a time frame within which steps must occur.
```yaml
Detection:
- Sequence:
- RuleID: Standard.BruteForceByIP
MinMatchCount: 7
- RuleID: Okta.Login.Success
- RuleID: AWS.Console.RootLogin
- RuleID: Crowdstrike.Detection.passthrough
Absence: true
LookbackWindowMinutes: 60
Schedule:
RateMinutes: 30
TimeoutMinutes: 3
```
{% endtab %}
{% endtabs %}
## Testing correlation rules
You can add unit tests to a correlation rule to evaluate whether, given certain conditions, a match on the correlation rule would be generated (potentially creating an alert—see [What happens when there is a match on a correlation rule](#what-happens-when-there-is-a-match-on-a-correlation-rule)).
{% hint style="info" %}
Correlation rule tests exist to test the correlation logic only. To test the rule logic of the individual rules making up the correlation rule, use the unit tests on the individual rules themselves.
{% endhint %}
Unit tests are defined on a correlation rule within the **Unit Tests** tab (in the Panther Console) or the top-level `Tests` field (in the CLI workflow), and are structured similarly to [unit tests for rules or policies](https://docs.panther.com/detections/testing).
Each unit test on a correlation rule must include a `Name`, `ExpectedResult`, and `RuleOutputs` field. Learn more about the YAML structure for unit tests, including how to construct a `RuleOutputs` value [here, on Correlation Rule Reference](https://docs.panther.com/detections/correlation-rule-reference#tests-fields).
{% hint style="info" %}
After writing tests for correlation rules, you can run them using the [Panther Analysis Tool `test` command](https://docs.panther.com/panther-developer-workflows/detections-repo/pat/pat-commands#test-running-unit-tests). Running `pat test` for correlation rules requires an API token—see [Authenticating with an API token](https://docs.panther.com/panther-developer-workflows/detections-repo/pat/install-configure-and-authenticate-with-pat#authenticating-with-an-api-token) for more information.
{% endhint %}
### Unit test examples
Using the following correlation rule:
```yaml
- Sequence:
- ID: OktaLoginFailure
RuleID: Okta.Login.Failure
MinMatchCount: 10
- ID: OktaLoginSuccess
RuleID: Okta.Login.Success
MinMatchCount: 1
- ID: RootLogin
RuleID: AWS.Console.RootLogin
MinMatchCount: 1
Transitions:
- ID: Okta Brute Force Login
From: OktaLoginFailure
To: OktaLoginSuccess
WithinTimeFrameMinutes: 10
Match:
- On: client.ipAddress
- ID: AWS Root Login
From: OktaLoginSuccess
To: RootLogin
Match:
- From: client.ipAddress
To: srcIpAddress7
Schedule:
RateMinutes: 15
TimeoutMinutes: 2
LookbackWindowMinutes: 60
```
We can write the following tests:
{% hint style="info" %}
If the below tests were included in the rule's YAML file (as is required when managing your detections in the CLI workflow), they would be positioned under a `Tests` key.
{% endhint %}
{% tabs %}
{% tab title="Test w/absolute timestamps" %}
If there are ten login attempts followed by a successful login, then a root login, we expect the correlation rule to return `true`. This test uses absolute timestamps—learn more about how to use absolute timestamps [here, on Correlation Rule Reference](https://docs.panther.com/detections/correlation-rule-reference#matchvalue-fields).
{% code overflow="wrap" %}
```yaml
Name: Successful Login with Timestamps
ExpectedResult: true
RuleOutputs:
- ID: OktaLoginFailure
Matches:
client.ipAddress:
# Since the original rule has a MinMatchCount of 10, we need at least 10 timestamps
123.123.123.123: ["2006-01-02T15:04:05Z", "2006-01-02T15:04:06Z","2006-01-02T15:04:07Z", "2006-01-02T15:04:08Z", "2006-01-02T15:04:09Z", "2006-01-02T15:04:10Z", "2006-01-02T15:04:11Z", "2006-01-02T15:04:12Z", "2006-01-02T15:04:13Z", "2006-01-02T15:04:14Z"]
- ID: OktaLoginSuccess
Matches:
client.ipAddress:
123.123.123.123: ["2006-01-02T15:04:15Z"]
- ID: RootLogin
Matches:
srcIpAddress7:
123.123.123.123: ["2006-01-02T15:04:16Z"]
```
{% endcode %}
{% endtab %}
{% tab title="Test w/relative timestamps" %}
If there are ten login attempts followed by a successful login, then a root login, we expect the correlation rule to return `true`. This test uses relative timestamps—learn more about how to use relative timestamps [here, on Correlation Rule Reference](https://docs.panther.com/detections/correlation-rule-reference#matchvalue-fields).
{% code overflow="wrap" %}
```yaml
Name: Successful Login with Relative Minutes
ExpectedResult: true
RuleOutputs:
- ID: OktaLoginFailure
Matches:
client.ipAddress:
# Since the original rule has a MinMatchCount of 10, we need at least 10 timestamps
123.123.123.123: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
- ID: OktaLoginSuccess
Matches:
client.ipAddress:
123.123.123.123: [11]
- ID: RootLogin
Matches:
srcIpAddress7:
123.123.123.123: [12]
```
{% endcode %}
{% endtab %}
{% tab title="Test w/ExpectedResult: false" %}
This tests expects the rule to evaluate to `false` because there are only *nine* login attempts followed by a successful login, not 10.
```yaml
Name: Nine Login Failures Before Success
ExpectedResult: false
RuleOutputs:
- ID: OktaLoginFailure
Matches:
client.ipAddress:
123.123.123.123: [1, 2, 3, 4, 5, 6, 7, 8, 9]
- ID: OktaLoginSuccess
Matches:
client.ipAddress:
123.123.123.123: [10]
- ID: RootLogin
Matches:
srcIpAddress7:
123.123.123.123: [11]
```
{% endtab %}
{% endtabs %}
## Limitations of correlation rules
* If a sequence uses transitions:
* The number of transitions allowed is one fewer than the number of rules included in the sequence collection.
* The order of items in the `Transitions` list must correspond to the order of rules in the `Sequence` field.
* If you use event field matching:
* The value of the previous `Match.On`/`Match.From` must match the next `Match.On`/`Match.To` value.
* For rules associated to more than one log type, scheduled rule, or correlation rule, values for `Match.On`/`Match.From`/`Match.To` or `MatchCriteria.Match` must be one of the `p_` fields listed on [Standard Fields](https://docs.panther.com/search/panther-fields).
* Only one type of field may be matched on throughout the correlation rule. For example, only IP addresses can be matched on, or only email addresses.
* It is not possible to use event field matching for values that do not match. For example, it is not possible for a Correlation Rule to pass when an IP address from one step in the Correlation Rule does not match the IP address in the next step.
## How to create a correlation rule
You can write correlation rules in the Panther Console or locally. For explanations of the YAML keys used to construct a correlation rule, see [Correlation Rule Reference](https://docs.panther.com/detections/correlation-rules/correlation-rule-reference).
In addition to creating custom correlation rules, you can also leverage [Panther-managed correlation rules](https://github.com/panther-labs/panther-analysis/tree/main/correlation_rules), available in the `correlation_rules` directory of the `panther-analysis` repository.
### Using the flow chart visualizer in the Console
You can use the flow chart visualizer in the Panther Console while working with Correlation Rules. This renders the correlation rules visually while you construct them in YAML, and the UI provides immediate validation feedback on your rules.
### Creating a correlation rule in YAML in the Console
To create a correlation rule in the Panther Console, you can either select detections from the list view to generate YAML, or construct the YAML yourself.
{% tabs %}
{% tab title="Select rules and generate YAML" %}
1. In the left-hand navigation bar of your Panther Console, click **Detections**.
2. In the list of detections, click the checkbox on the left-hand side of each detection you'd like to include in your correlation rule.
* The order in which you click the detections will be the order of the generated [sequence](#sequence-correlation-rules) correlation rule.
3. Click **Correlate**.\
4. On the detection creation page, finish configuring your correlation rule:
* **Name**: Enter a descriptive name for the correlation rule.
* **ID** (optional)**:** Click the pen icon and enter a unique ID for your correlation rule.
* In the upper-right corner, the **Enabled** toggle will be set to `ON` by default. If you'd like to disable the rule, flip the toggle to `OFF`.
* Under the **YAML Editor** tab:
* If desired, make modifications to the generated correlation rule. The rule, by default:
* Is a [`Sequence`](https://docs.panther.com/detections/correlation-rule-reference#detection-fields) correlation rule.
* You can change this to [`Group`](https://docs.panther.com/detections/correlation-rule-reference#detection-fields) by clicking **Organize Rules as Group**.\
\
Learn more in [Switching between sequence and group in the Console](#switching-between-sequence-and-group-in-the-console).
* Sets [`MinMatchCount`](https://docs.panther.com/detections/correlation-rule-reference#group-and-sequence-fields-rule-references) to `1` for all rules and scheduled rules included in the correlation rule.
* Within [`Transitions`](https://docs.panther.com/detections/correlation-rule-reference#transitions-fields), sets [`Match.On`](https://docs.panther.com/detections/correlation-rule-reference#match-fields) for each transition to `p_any_usernames`.
* You can update the `Match.On` values, or instead use [`Match.To`](https://docs.panther.com/detections/correlation-rule-reference#match-fields) and [`Match.From`](https://docs.panther.com/detections/correlation-rule-reference#match-fields).
* Sets [`Schedule.RateMinutes`](https://docs.panther.com/detections/correlation-rule-reference#schedule-fields) to `15` and [`Schedule.TimeoutMinutes`](https://docs.panther.com/detections/correlation-rule-reference#schedule-fields) to `2`.
* Sets [`LookbackWindowMinutes`](https://docs.panther.com/detections/correlation-rule-reference#detection-fields) to `60`.
* More information about correlation detection YAML syntax can be found on [Correlation Rule Reference](https://docs.panther.com/detections/correlation-rules/correlation-rule-reference), including a full list of required and optional fields.
* Under the **Alert Settings** tab:
* Within the **Basics** tab, configure the following fields:
* **Create Alert**: This `ON/OFF` toggle indicates whether an [alert](https://docs.panther.com/alerts) should be created when there are matches, or only a [signal](https://docs.panther.com/detections/signals).
* (Only applicable if **Create Alert** is set to `ON`) **Severity:** Select a [severity level](#alert-severity) for the alerts triggered by this detection.
* (Only applicable if **Create Alert** is set to `ON`) **Destination Overrides:** Optionally choose destinations to receive alerts for this detection, regardless of severity.
* (Only applicable if **Create Alert** is set to `ON`) Within the **Context** sub-tab, optionally provide values for the following fields:
* **Description**: Enter additional context about the rule.
* **Runbook**: Enter the procedures and operations relating to this rule.
* Learn more on [Alert Runbooks](https://docs.panther.com/alerts/alert-runbooks).
* It's recommended to provide a descriptive runbook, as [Panther AI alert triage](https://docs.panther.com/alerts#ai-alert-triage) will take it into consideration.
* **Reference**: Enter an external link to more information relating to this rule.
* **Summary Attributes**: Enter the attributes you want to showcase in the alerts that are triggered by this detection.
* To use a nested field as a summary attribute, use the Snowflake dot notation in the Summary Attribute field to traverse a path in a JSON object:
`:..`
The alert summary will then be generated for the referenced object in the alert. [Learn more about traversing semi-structured data in Snowflake here.](https://docs.snowflake.com/en/user-guide/querying-semistructured.html#label-traversing-semistructured-data)
* For more information on Alert Summaries, see [Assigning and Managing Alerts](https://docs.panther.com/alerts/alert-management).
* **Tags**: Enter custom tags to help you understand the rule at a glance (e.g., `HIPAA`.)
* In the **Framework Mapping** section:
1. Click **Add New** to enter a report.
2. Provide values for the following fields:
* **Report Key**: Enter a key relevant to your report.
* **Report Values**: Enter values for that report.
* Under the **Unit Tests** tab, optionally add tests:
1. Click **+ Add new unit test**.
* Boilerplate code for a test for this correlation rule will be populated.
2. Make any necessary adjustments to the populated text, and fill in the remainder of the test, including the contents of `Matches`.
3. Below the code editor, click **Run Test** to evaluate the test.\
5. In the upper-right corner, click **Deploy**.
{% endtab %}
{% tab title="Write YAML from scratch" %}
1. In the left-hand navigation bar of your Panther Console, click **Build** > **Detections**.
2. Click **Create New**.
3. On the **Correlation Rule** tile, click **Start**.
4. On the detection creation page, fill in the following fields:
* **Name**: Enter a descriptive name for the correlation rule.
* **ID** (optional)**:** Click the pen icon and enter a unique ID for your correlation rule.
* In the upper-right corner, the **Enabled** toggle will be set to `ON` by default. If you'd like to disable the rule, flip the toggle to `OFF`.
* Under the **YAML Editor** tab, define your correlation rule.
* By default, the correlation rule is organized as a [`Sequence`](https://docs.panther.com/detections/correlation-rule-reference#detection-fields). You can change this to [`Group`](https://docs.panther.com/detections/correlation-rule-reference#detection-fields) by clicking **Organize Rules as Group**.\
\
Learn more in [Switching between sequence and group in the Console](#switching-between-sequence-and-group-in-the-console).
* More information about correlation detection YAML syntax can be found on [Correlation Rule Reference](https://docs.panther.com/detections/correlation-rules/correlation-rule-reference), including a full list of required and optional fields.
* Under the **Alert Settings** tab:
* Within the **Basics** tab, configure the following fields:
* **Create Alert**: This `ON/OFF` toggle indicates whether an [alert](https://docs.panther.com/alerts) should be created when there are matches, or only a [signal](https://docs.panther.com/detections/signals).
* (Only applicable if **Create Alert** is set to `ON`) **Severity:** Select a [severity level](#alert-severity) for the alerts triggered by this detection.
* (Only applicable if **Create Alert** is set to `ON`) **Destination Overrides:** Optionally choose destinations to receive alerts for this detection, regardless of severity.
* (Only applicable if **Create Alert** is set to `ON`) Within the **Context** sub-tab, optionally provide values for the following fields:
* **Description**: Enter additional context about the rule.
* **Runbook**: Enter the procedures and operations relating to this rule.
* Learn more on [Alert Runbooks](https://docs.panther.com/alerts/alert-runbooks).
* It's recommended to provide a descriptive runbook, as [Panther AI alert triage](https://docs.panther.com/alerts#ai-alert-triage) will take it into consideration.
* **Reference**: Enter an external link to more information relating to this rule.
* **Summary Attributes**: Enter the attributes you want to showcase in the alerts that are triggered by this detection.
* To use a nested field as a summary attribute, use the Snowflake dot notation in the Summary Attribute field to traverse a path in a JSON object:
`:..`
The alert summary will then be generated for the referenced object in the alert. [Learn more about traversing semi-structured data in Snowflake here.](https://docs.snowflake.com/en/user-guide/querying-semistructured.html#label-traversing-semistructured-data)
* For more information on Alert Summaries, see [Assigning and Managing Alerts](https://docs.panther.com/alerts/alert-management).
* **Tags**: Enter custom tags to help you understand the rule at a glance (e.g., `HIPAA`.)
* In the **Framework Mapping** section:
1. Click **Add New** to enter a report.
2. Provide values for the following fields:
* **Report Key**: Enter a key relevant to your report.
* **Report Values**: Enter values for that report.
* Under the **Unit Tests** tab, optionally add tests:
1. Click **+ Add new unit test**.
* Boilerplate code for a test for this correlation rule will be populated.
2. Make any necessary adjustments to the populated text, and fill in the remainder of the test, including the contents of `Matches`.
3. Below the code editor, click **Run Test** to evaluate the test.\
5. In the upper-right corner, click **Deploy**.
{% endtab %}
{% endtabs %}
#### Switching between sequence and group in the Console
You can switch your correlation rule from a sequence to a group, and vice versa, using the **Organize Rules as Group/Sequence** button at the top right of the **YAML Editor** panel.
If your correlation rule is already a sequence and you click **Organize Rules as Group**:
* The `Sequence` key will be changed to `Group`.
* The `Transitions` section will be removed.
* A `MatchCriteria` section will be added.
* If you have previously edited a `MatchCriteria` section, that version will be added. Otherwise, a default `MatchCriteria` section will be provided.
If your correlation rule is already a group and you click **Organize Rules as Sequence**:
* The `Group` key will be changed to `Sequence`.
* The `MatchCriteria` section will be removed.
* A `Transitions` section will be added.
* If you have previously edited a `Transitions` section, that version will be added. Otherwise, a default `Transitions` section will be provided.
### Creating a correlation rule in YAML in the CLI workflow
Expand instructions
If you're writing Correlations detections locally (instead of in the Panther Console), we recommend managing your local detection files in a version control system like GitHub or GitLab.
**Folder setup**
If you group your correlation rules into folders, each folder name must contain `rules` in order for them to be found during upload (using either PAT or the bulk uploader in the Console).
**File setup**
Each correlation rule consists of:
* A YAML specification file (a file with a `.yml` extension) containing the detection logic, as well as metadata attributes of the detection.
More information about correlation detection YAML syntax can be found on [Correlation Rule Reference](https://docs.panther.com/detections/correlation-rules/correlation-rule-reference), including a full list of required and optional fields.
* Create a YAML file (e.g. `my_new_correlation_rule.yml`) using the template below (including a top-level `Detection` key):
```yaml
AnalysisType: correlation_rule
DisplayName: Example Correlation Rule to Check the Format of the Spec
Enabled: true
RuleID: Correlation.Type.Behavior.MoreContext
Severity: High
Reports:
ReportName (like CIS, MITRE ATT&CK):
- The specific report section relevant to this correlation rule
Tags:
- Tags
- Go
- Here
Description: >
This correlation rule exists to validate the CLI workflows of the Panther CLI
Runbook: >
First, find out who wrote this the spec format, then notify them with feedback.
Reference: https://www.a-clickable-link-to-more-info.com
Detection:
- Sequence:
- ID: Failed Login
RuleID: Okta.Login.Fail
MinMatchCount: 7
- ID: Successful Login
RuleID: Okta.Login.Success
MinMatchCount: 1
LookbackWindowMinutes: 15
Schedule:
RateMinutes: 5
TimeoutMinutes: 3
Tests:
- Name: Rule should return false if no matches
ExpectedResult: false
RuleOutputs:
- ID: Failed Login
Matches: {} # Empty mapping implies no matches
```
After this rule is uploaded to Panther, it will be viewable in the Console.
## Correlation rule full examples
### Discovering exfiltrated GitHub credentials
This `Discovering.Exfiltrated.Credentials` correlation rule checks every 10 minutes to see if there has been a [signal](https://docs.panther.com/detections/signals) for the `AWS.CloudTrail.IaaS` rule (defined in the second tab below) *not* followed by a signal for the `GitHub.CICD` rule (defined in the third tab below) in the last 10 minutes.
{% tabs %}
{% tab title="Discovering Exfiltrated Credentials" %}
```yaml
AnalysisType: correlation_rule
RuleID: 'Discovering.Exfiltrated.Credentials'
DisplayName: 'Discovering.Exfiltrated.Credentials'
Enabled: true
Severity: High
Description: >
There was at least one IaaS activity match not followed
by a CI/CD activity within 10 minutes.
Detection:
- Sequence:
- ID: IaaS Activity
RuleID: AWS.CloudTrail.IaaS
- ID: CICD Activity
RuleID: Github.CICD
Absence: true
Transitions:
- From: IaaS Activity
To: CICD Activity
WithinTimeFrameMinutes: 10
Schedule:
RateMinutes: 10
TimeoutMinutes: 3
Tests:
- Name: IaaS Activity without CICD Activity
ExpectedResult: true
RuleOutputs:
- ID: IaaS Activity
Matches:
username:
my_username: [1]
- Name: IaaS Activity with CICD Activity
ExpectedResult: false
RuleOutputs:
- ID: IaaS Activity
Matches:
username:
my_username: [1]
- ID: CICD Activity
Matches:
username:
my_username: [2]
```
{% endtab %}
{% tab title="AWS.CloudTrail.IaaS" %}
```yaml
AnalysisType: rule
RuleID: 'AWS.CloudTrail.IaaS'
DisplayName: 'AWS CloudTrail IaaS'
Enabled: true
LogTypes:
- AWS.CloudTrail
Severity: Info
CreateAlert: false
Detection:
- KeyPath: userIdentity.arn
Condition: IsIn
Values:
- DeploymentUpdateGitHubRole
- KeyPath: eventName
Condition: IsIn
Values:
- StartSession
- ListResources
- UpdateResource
- DescribeResource
- WriteLog
```
{% endtab %}
{% tab title="GitHub.CICD" %}
```yaml
AnalysisType: rule
RuleID: 'GitHub.CICD'
DisplayName: 'GitHub CICD'
Enabled: true
LogTypes:
- GitHub.Audit
Severity: Info
CreateAlert: false
Detection:
- KeyPath: repository
Condition: Equals
Value: panther-labs/example-repo
- KeyPath: action
Condition: Equals
Value: workflows.created_workflow_run
- KeyPath: name
Condition: Equals
Value: CI
```
{% endtab %}
{% endtabs %}
### Brute force Okta login to AWS root login
This `Brute.Force.Login` correlation rule checks every 30 minutes to see if there has been a [signal](https://docs.panther.com/detections/signals) for the [`Standard.BruteForceByIP`](https://github.com/panther-labs/panther-analysis/blob/main/rules/standard_rules/brute_force_by_ip.py) rule followed by a signal for the `Okta.Login.Success` rule (defined in the second tab below) followed by a signal for the [`AWS.Console.RootLogin`](https://github.com/panther-labs/panther-analysis/blob/main/rules/aws_cloudtrail_rules/aws_console_root_login.py) rule, with additional time frame and event IP value matching requirements.
{% tabs %}
{% tab title="Brute Force Login" %}
```yaml
AnalysisType: correlation_rule
RuleID: 'Brute.Force.Login'
DisplayName: 'Brute.Force.Login'
Enabled: true
Severity: High
Description: >
At least 100 failed logins followed by a successful login
that occurred within 10 minutes of the last failed login
and then a root login from the AWS console.
Detection:
- Sequence:
- ID: Failed Login
RuleID: Standard.BruteForceByIP
MinMatchCount: 100
- ID: Successful Login
RuleID: Okta.Login.Success
MinMatchCount: 1
- ID: Root Login
RuleID: AWS.Console.RootLogin
MinMatchCount: 1
Transitions:
- ID: Brute Force Login Success
From: Failed Login
To: Successful Login
WithinTimeFrameMinutes: 10
Match:
- From: p_alert_context.ip
To: client.ipAddress
- ID: Gained Root Access
From: Successful Login
To: Root Login
Match:
- From: client.ipAddress
To: p_alert_context.sourceIPAddress
LookbackWindowMinutes: 60
EventEvaluationOrder: Chronological
Schedule:
RateMinutes: 30
TimeoutMinutes: 3
Tests:
# See additional tests for this correlation rule above, in "Unit test examples"
- Name: Successful Login with Relative Minutes
ExpectedResult: true
RuleOutputs:
- ID: Failed Login
Matches:
client.ipAddress:
# Since the original rule has a MinMatchCount of 10, we need at least 10 timestamps
123.123.123.123: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
- ID: Successful Login
Matches:
client.ipAddress:
123.123.123.123: [11]
- ID: Root Login
Matches:
srcIpAddress7:
123.123.123.123: [12]
```
{% endtab %}
{% tab title="Okta.Login.Success" %}
```yaml
AnalysisType: rule
RuleID: 'Okta.Login.Success'
DisplayName: 'Okta Login Success'
Enabled: true
LogTypes:
- Okta.SystemLog
Severity: Info
CreateAlert: false
Detection:
- DeepKey:
- outcome
- result
Condition: Equals
Value: SUCCESS
- Key: eventType
Condition: Equals
Value: user.session.start
```
{% endtab %}
{% endtabs %}
### GitHub repository security policy disabled without subsequent archival
This `Github.Repo.Security.Policy.Disabled.Without.Archival` correlation rule checks every 10 minutes to see if there has been a [signal](https://docs.panther.com/detections/signals) for the [`GitHub.Advanced.Security.Change` ](https://github.com/panther-labs/panther-analysis/blob/main/rules/github_rules/github_advanced_security_change.py)rule *not* followed by a signal for the `GitHub.Repo.Archived` rule (defined in the second tab below) where there is also a matching value for the `p_alert_context.repo` event field within the last 10 minutes.
{% tabs %}
{% tab title="Github.Repo.Security.Policy.Disabled.Without.Archival" %}
```yaml
AnalysisType: correlation_rule
RuleID: 'Github.Repo.Security.Policy.Disabled.Without.Archival'
DisplayName: 'Github Repo Security Policy Disabled Without Archival'
Enabled: true
Tags:
- Github
- Repo Archived
Severity: High
Detection:
- Sequence:
- ID: GitHub Advanced Security Change
RuleID: GitHub.Advanced.Security.Change
- ID: Github Repo Archived
RuleID: Github.Repo.Archived
Absence: true
Transitions:
- ID: TR1
From: GitHub Advanced Security Change
To: Github Repo Archived
WithinTimeFrameMinutes: 10
Match:
- On: p_alert_context.repo
Schedule:
RateMinutes: 10
TimeoutMinutes: 3
Tests:
- Name: Github Repo Security Policy Disabled Without Archival
ExpectedResult: true
RuleOutputs:
- ID: GitHub Advanced Security Change
Matches:
p_alert_context.repo:
my_production_repo: [1]
- Name: Github Repo Security Policy Disabled With Archival
ExpectedResult: false
RuleOutputs:
- ID: GitHub Advanced Security Change
Matches:
p_alert_context.repo:
my_production_repo: [1]
- ID: Github Repo Archived
Matches:
p_alert_context.repo:
my_production_repo: [2]
```
{% endtab %}
{% tab title="GitHub.Repo.Archived" %}
```yaml
AnalysisType: rule
RuleID: 'GitHub.Repo.Archived'
DisplayName: 'GitHub Repo Archived'
Enabled: true
LogTypes:
- GitHub.Audit
Severity: Info
CreateAlert: false
AlertContext:
- KeyName: repo
KeyValue:
KeyPath: repo
Detection:
- KeyPath: action
Condition: Equals
Value: repo.archived
```
{% endtab %}
{% endtabs %}
## Making correlation rules more efficient
Correlation rules use complex pattern recognitions, which means they have the potential to be computationally expensive. To reduce Snowflake costs associated with correlation rules, keep the following guidelines in mind.
### **Run the correlation rule as infrequently as possible**
How often a correlation rule runs—determined by its `Schedule` value—can have a large impact on its cost. It's therefore recommended to ensure your correlation rule is not running more often than it needs to (while still meeting your detection needs—see [Setting `Schedule`, above](#setting-schedule), for considerations when configuring this field).
A general guideline when thinking about the relationship between a correlation rule's run frequency and cost is: Each time you double the interval on which the correlation rule is run (e.g. by doubling `RateMinutes`), you halve the cost it generates.
### **Set `LookbackWindowMinutes` as low as possible**
The amount of data a correlation rule runs over is largely defined by its `LookbackWindowMinutes` value, and this data quantity is a leading factor in the correlation rule's processing time and resulting cost. It's recommended to reduce the amount of data your correlation rule processes by setting its `LookbackWindowMinutes` value as low as possible (while still meeting your detection needs—see [Setting `LookbackWindowMinutes`, above](#setting-lookbackwindowminutes), for considerations when configuring this field).
As an example, say you'd like to identify when two rules each generate a signal within 10 minutes of one another (using `WithinTimeFrameMinutes`). While setting `LookbackWindowMinutes` to exactly `10` is not advisable, you can safely use a value of as low as `15`.
### **Choose match fields with the lowest cardinality possible**
Match field cardinality is positively correlated with correlation rule cost. If your correlation rule uses event value matching, when choosing which fields to match on, it's recommended to use fields that have lower cardinality.
Cardinality of a match field can be influenced by a few factors, including:
* The number of possible values the field can have—the more possible values, the higher the cardinality.
* For example, if `field_a` can have one of three possible values (e.g., `"yellow"`, `"red"`, or `"blue"`), but `field_b` will only ever have one of two values (e.g., `"purple"` or `"green"`), `field_b` has lower cardinality than `field_a`.
* The field's data type—typically, fields with a non-scalar data type (i.e., that are an `array` or `object`) have higher cardinality than fields with a scalar data type (i.e., `string`, `boolean`, or `number`).
* For example, if your log schema designates both an `email` and `username` field as a `username` [indicator](https://docs.panther.com/search/panther-fields#indicator-fields), meaning your `p_any_usernames` field will include them both in an `array` (e.g., `p_any_usernames: ["Bob Smith", "bob.smith@example.com"]`), that `p_any_usernames` field will have a higher cardinality than the `email` field, which has a `string` type with a single value.
.png?alt=media)
