YAML Match Expression Reference

Construct match expressions to define logic

Overview

This functionality, as part of the Simple Detections feature set, is in closed beta starting with Panther version 1.81. To request access to the feature or share any bug reports or feature requests, please contact your Panther support team.

A match expression is a conditional logic expression, defined in YAML, that will return true or false. Match expressions are used various places, for example:

In the Detection key of a YAML detection
In the InlineFilters key of a Python or YAML detection
In the Expressions key of a List Comprehension match expression (effectively creating match expressions within a match expression)
In the Conditions key of the DynamicSeverities key of a YAML detection

There are seven types of match expressions, with varying structures. You can group match expressions using combinators to form complex conditions.

You can use enrichment in YAML detections with enrichment match expressions.

How match expressions work

Match expressions work roughly the following way:

A key from an incoming event is identified with a key specifier
A condition is provided, which describes the nature of the check
A value (or list of values) is provided to compare against

(There are exceptions to this pattern, where zero or multiple event keys are identified for comparison, or no value is needed.)

Types of match expressions

There are seven types of match expressions, each with a unique syntax or structure.

The match expression examples in the following sections refer to this JSON event:

Example event:

You can refer to this event in the example match expressions in this section.

{
  "audit_event": "RECLASSIFICATION",
  "status": "success",
  "date": "2006-08-24",
  "actor": "International Astronomical Union",
  "old_classification": "PLANET",
  "new_classification": "DWARF_PLANET",
  "entity": {
    "name": "Pluto",
    "diameter": {
      "distance": 2370,
      "unit": "km"
    },
    "moons": [
      {
        "name": "Charon",
        "year_discovered": 1978,
        "discovered_by": ["James W. Christy"],
        "diameter": {
          "distance": 1212,
          "unit": "km"
        }
      },
      {
        "name": "Nix",
        "year_discovered": 2005,
        "discovered_by": [
          "H. A. Weaver",
          "S. A. Stern",
          "M. J. Mutchler",
          "A. J. Steffl",
          "M. W. Buie",
          "W. J. Merline",
          "J. R. Spencer",
          "E. F. Young",
          "L. A. Young"
        ],
        "diameter": {
          "distance": 49.8,
          "unit": "km"
        }
      },
      {
        "name": "Hydra",
        "year_discovered": 2005,
        "discovered_by": ["A. J. Steffl", "M. J. Mutchler"],
        "diameter": {
          "distance": 51
          "unit": "km"
        }
      },
      {
        "name": "Kerberos",
        "year_discovered": 2011,
        "discovered_by": ["M. R. Showalter", "D. P. Hamilton"]
        "diameter": {
          "distance": 19,
          "unit": "km",
        }
      },
      {
        "name": "Styx",
        "year_discovered": 2012,
        "discovered_by": ["M. R. Showalter", "D. P. Hamilton"]
        "diameter": {
          "distance": 25
          "unit": "km" 
        }
      }
    ],
    "single_year_in_earth_years": 247.94,
    "year_discovered": 1930,
    "discovered_by": "Clyde Tombaugh",
    "orbit_region": "Kuiper belt",
    "atmosphere": {
      "composition": ["nitrogen", "methane", "carbon monoxide"],
      "surface_pressure": {
        "number": 1,
        "unit": "pascals"
      }
    }
  },
  "p_enrichment": {
    "discoverer_info": {
        "discovered_by": {
            "p_match": "Clyde Tombaugh",
	    "year_born": 1906,
	    "year_died": 1997
	}
    }
  } 
}

In each example match expression below, a single key specifier is used to identify the event key(s), however any of the key specifiers (e.g., Key, DeepKey or KeyPath) can be used in most cases.

Key/value match expressions

Key/value match expressions compare the value of a certain event key to a given value.

You can see all allowed values of Condition in the Scalar matching conditions table below, where the "Applicable Match Expressions" column includes "Key/value."

Example:

- Key: audit_event
  Condition: Equals
  Value: reclassification

Key/values match expressions

Key/values match expressions evaluate whether some event value is a member in a given list.

The only permitted Condition values for key/values match expressions are IsIn and IsNotIn, visible in the List matching conditions table below.

Example:

- KeyPath: entity.name
  Condition: IsIn
  Values:
    - Mercury
    - Venus
    - Earth
    - Mars
    - Jupiter
    - Saturn
    - Uranus
    - Neptune

Multi-key match expressions

Multi-key match expressions cannot be viewed and edited in the Simple Detection builder in the Panther Console, nor can they be used in Inline Filters. See a full list of Simple Detection builder limitations here, and Inline Filters limitations here.

Multi-key match expressions allow you to compare the values of two event keys. You can form a multi-key match expression by including a Condition and list of Values that contains two event keys.

See all allowed values of Condition in the Scalar matching conditions table below, where the "Applicable Match Expressions" column includes "Multi-key."

- Condition: IsGreaterThan
  Values: # These values can be any combination of `Key`, 'DeepKey', or 'KeyPath, with up to two clauses.
    - KeyPath: entity.diameter.distance # Pluto is bigger than...
    - KeyPath: entity.moons[0].diameter.distance # Charon

List comprehension match expressions

List comprehension match expressions cannot be viewed and edited in the Simple Detection builder in the Panther Console, nor can they be used in Inline Filters. See a full list of Simple Detection builder limitations here, and Inline Filters limitations here.

A list comprehension match expression takes a key with a list value, and, for each item in the list, evaluates it against a list of match expressions. The list of match expressions is stored under the Expressions key.

For list comprehension match expressions, four values for Condition are valid:

AnyElement
AllElements
OnlyOneElement
NoElement

Learn more about these Condition values in the List comprehension conditions table below.

Example using AnyElement:

- KeyPath: entity.moons
  Condition: AnyElement
  Expressions:
    - Key: name
      Condition: Equals
      Value: "Death Star"
    - Key: year_discovered
      Condition: IsGreaterThan
      Value: 2000

Within the Expressions key, you can add either:

Nested list comprehension match expressions. Each level of nesting can only contain one list comprehension match expression. This structure effectively creates nested for loops. The innermost list comprehension can have as many match expressions as you like, so long as they are not list comprehension match expressions. Example:
```
- KeyPath: entity.moons
  Condition: AnyElement
  Expressions:
    - Key: discovered_by
      Condition: AllElements
      Expressions:
        - Key: $Element$
          Condition: Equals
          Value: "James W. Christy"
```

Any combination of match expressions that are not list comprehension match expressions. Example:

- Key: someKey
  Condition: AnyElement
  Expressions:
    - Key: a
      Condition: Equals
      Value: 123
    - Key: b
      Condition: IsIn
      Values:
      - 345
      - 678

Existence match expressions

An existence match expression determines whether a specific key matches a specified existence condition.

See all allowed values of Condition in the Scalar matching conditions table below, where the "Applicable Match Expressions" column includes "Existence."

Example:

- Key: entity.atmosphere
  Condition: Exists

Absolute match expressions

Absolute match expressions cannot be viewed and edited in the Simple Detection builder in the Panther Console, nor can they be used in Inline Filters. See a full list of Simple Detection builder limitations here, and Inline Filters limitations here.

In certain cases, you may want a match expression to always evaluate to True or False. You can do this by creating a match expression that simply contains a Condition key set to AlwaysTrue or AlwaysFalse.

- Condition: AlwaysTrue

Enrichment match expressions

Enrichment match expressions cannot be viewed and edited in the Simple Detection builder in the Panther Console, nor can they be used in Inline Filters. See a full list of Simple Detection builder limitations here, and Inline Filters limitations here.

Enrichment match expressions are only used to interact with a JSON event's p_enrichment struct, which is present if there is a match on a Lookup Table, Enrichment Provider, or Identify Provider Profile.

To create an enrichment match expression, include an Enrichment key. The contents of Enrichment indicate the path to the event key on which to build the condition. Within Enrichment, include the following fields:

Table: The name of the Lookup Table, Enrichment Provider table, or Identity Provider Profile table.
- These names are user-defined in the case of Lookup Tables, and Panther-defined in the case of Enrichment Providers and Identity Provider Profiles.
Selector: The name of the selector field from the incoming log.
FieldPath: The name of any one of the event fields within the Selector field. This key takes JSON path notation, meaning you can denote nested fields, if needed.

Once you have defined Enrichment and its sub-fields, the rest of the match expression can be modeled after the structure of any of the other match expression types—except for List comprehension match expressions (these expressions are not supported for use with enrichment) and Absolute match expressions (these expressions do not reference any event values).

For example, the enrichment match expression below—which includes Condition and Value keys—resembles a key/value match expression:

- Enrichment:
    Table: discoverer_info
    Selector: discovered_by
    FieldPath: p_match
  Condition: Equals
  Value: "Clyde Tombaugh"

Combinators

The OnlyOne and None combinators cannot be viewed and edited in the Simple Detection builder in the Panther Console, nor can they be used in Inline Filters. See a full list of Simple Detection builder limitations here, and Inline Filters limitations here.

Combinators are YAML keys that describe the relationship between match expressions or groups of match expressions. There are four combinators: All, Any, OnlyOne, and None.

If more than one match expression is provided directly under a key that takes match expressions (e.g., Detection, InlineFilters or Expressions), and no combinator is specified, All is assumed.

When writing YAML detections in the CLI workflow, there is no limit on the number of nested levels you can create with combinators. (In the Simple Detection builder in the Panther Console, there is a three-level limit.) Note that if match expressions are nested more than three levels deep when working locally, the Panther Console will not render them in the Simple Detection builder—they will only be viewable in raw YAML.

Combinator

Boolean operator name

Description

All

Logical AND

All match expressions in the list evaluate to true. The first failing match ends evaluation of the list.

Any

Logical OR

Any single match expression in the list evaluates to true. The first true match ends evaluation of the list.

OnlyOne

Logical XOR

Only one single match expression in the list evaluates to true.

None

Logical NOT AND

No match expression in the list evaluates to true.

Example of one level of nesting using OnlyOne:

# Exactly one of the listed expressions must evaluate 
# to true for the `OnlyOne` block to evaluate to true
- OnlyOne:
  - Key: someKey
    Condition: DoesNotEqual
    Value: 1234
  - Key: someOtherKey
    Condition: Contains
    Value: "invalid login"
  - Key: someOtherOtherKey
    Condition: IsIn
    Values:
      - 314
      - 567

Example of multiple levels of nesting using Any and All:

- Any:
      - All:
          - Key: eventSource
            Condition: StartsWith
            Value: 'dynamodb'
          - Any:
              - Key: eventName
                Condition: StartsWith
                Value: List
              - Key: eventName
                Condition: StartsWith
                Value: Describe
              - Key: eventName
                Condition: StartsWith
                Value: Get
      - All:
          - Key: eventSource
            Condition: StartsWith
            Value: 'ec2'
          - Any:
              - Key: eventName
                Condition: StartsWith
                Value: Describe
              - Key: eventName
                Condition: StartsWith
                Value: Get

Key specifiers

Key specifiers are a group of YAML keys that allow you to indicate a certain key in your log event JSON. There are three key specifiers: Key, DeepKey, and KeyPath.

If the event property provided as the value of the key specifier does not exist, evaluation of the match expression stops and returns false.

`Key`

Key lets you access a top-level event property. Example:

Key: username

`DeepKey`

DeepKey lets you access a nested event property. DeepKey accepts a list. You can also use DeepKey to access elements off an array when you know the absolute index.

Example using DeepKey, which accesses the deep field foo.bar.baz, then indexes on an array index 2:

- DeepKey:
  - foo
  - bar
  - baz
  - 2

`KeyPath`

KeyPath lets you access a nested event property using subfield indexing (.) and array indexing ([x]) (additional JSON path features are not supported). The value of KeyPath could also just be a top-level field.

Example:

KeyPath: foo.bar.baz
# or 
KeyPath: foo

`Condition`

There are various types of Condition values you can choose from, including scalar, list, and list comprehension groupings.

Certain Condition values cannot be viewed and edited in the Simple Detection builder in the Panther Console, nor can they be used in Inline Filters. See a full list of Simple Detection builder limitations here, and Inline Filters limitations here.

Scalar matching conditions

Use scalar matching conditions alongside the Value key (not Values). The match expression's key specifier can reference a list or a single value, but Value must be singular.

Condition

Key’s Value Types

Description

Applicable Match Expressions

Exists

Any

Checks if the key specified exists.

Existence, Enrichment

DoesNotExist

Any

Checks if the key specified does not exist.

Existence, Enrichment

IsNull

Any

Checks if the key is null - e.g. {"key": null}

Existence, Enrichment

IsNotNull

Any

Checks if the key is not null - e.g. {"key": 1234}

Existence, Enrichment

IsNullOrEmpty

Any

Checks if the key is null or has an non-empty value. Empty values include: empty list ([]), empty string "", and the number 0.

Existence, Enrichment

IsNotNullOrEmpty

Any

Checks if the key is not null or has a non-empty value. Empty values include: empty list ([]), empty string "", and the number 0.

Existence, Enrichment

Equals

Any

Checks if a key’s value is equal to a specified value.

Key/value, Multi-key, Enrichment

DoesNotEqual

Any

Checks if a key’s value is not equal to a specified value.

Key/value, Multi-key, Enrichment

IEquals

String

Checks if a key’s string value is equal to a specified string, case insensitive.

Key/value, Multi-key, Enrichment

IDoesNotEqual

String

Checks if a key’s string value is not equal to a specified string, case insensitive.

Key/value, Multi-key, Enrichment

StartsWith

String

Checks if a key’s string value starts with a specified value. If the key is not a string, returns false.

Key/value, Multi-key, Enrichment

IStartsWith

String

Checks if a key’s string value starts with a specified value, case insensitive. If the key is not a string, returns false.

Key/value, Multi-key, Enrichment

DoesNotStartWith

String

Checks if a key’s string value does not start with a specified value. If they key is not a string, returns false.

Key/value, Multi-key, Enrichment

IDoesNotStartWith

String

Checks if a key’s string value does not start with a specified value, case insensitive. If they key is not a string, returns false.

Key/value, Multi-key, Enrichment

EndsWith

String

Checks if a key’s string value ends with a specified value. If the key is not a string, returns false.

Key/value, Multi-key, Enrichment

IEndsWith

String

Checks if a key’s string value ends with a specified value, case insensitive. If the key is not a string, returns false.

Key/value, Multi-key, Enrichment

DoesNotEndWith

String

Checks if a key’s string value does not end with a specified value. If they key is not a string, returns false.

Key/value, Multi-key, Enrichment

IDoesNotEndWith

String

Checks if a key’s string value does not end with a specified value, case insensitive. If they key is not a string, returns false.

Key/value, Multi-key, Enrichment

Contains

String or List

Checks if a key’s string value contains a specified value. If the key is not a string, returns false. OR Checks if an array of string(case sensitive)/int/float/bool contains a value.

Key/value, Multi-key, Enrichment

IContains

String

Checks if a key’s string value contains a specified value, case insensitive. If the key is not a string, returns false.

Key/value, Multi-key, Enrichment

DoesNotContain

String or List

Checks if a key’s string value does not contain a specified value. If they key is not a string, returns false. OR Checks if an array of string(case sensitive)/int/float/bool does not contains a value.

Key/value, Multi-key, Enrichment

IDoesNotContain

String

Checks if a key’s string value does not contain a specified value, case insensitive. If they key is not a string, returns false.

Key/value, Multi-key, Enrichment

IsIPAddress

String or Int

Checks if a key’s string or int value can be resolved into an IP Address. Functionally equivalent to ANY(IsIPv4Address, IsIPv6Address)

Key Existence Match, Enrichment

IsIPv4Address

String or Int

Checks if a key’s string or int value can be resolved into an IPv4 Address.

Key Existence Match, Enrichment

IsIPv6Address

String or Int

Checks if a key’s string or int value can be resolved into an IPv6 Address.

Key Existence Match, Enrichment

IsIPAddressPrivate

String or Int

Checks if a key’s string or int value can be resolved into a private IP Address.

Key Existence Match, Enrichment

IsIPAddressPublic

String or Int

Checks if a key’s string or int value can be resolved into a public IP Address.

Key Existence Match, Enrichment

IsIPAddressInCIDR

String or Int

Checks if a key’s string or int value rendered as an IP address is inside a CIDR range.

Key/value, Multi-key, Enrichment

IsIPAddressNotInCIDR

String or Int

Checks if a key’s string or int value rendered as an IP address is not inside a CIDR range.

Key/value, Multi-key, Enrichment

CIDRContainsIPAddresses

List of IP’s as Strings

Checks if a key’s string value rendered as a CIDR range contains a specified IP address.

Key/value, Multi-key, Enrichment

CIDRDoesNotContainIPAddresses

List of IP’s as Strings

Checks if a key’s value interpreted a list of IP addresses is in specified CIDR range

Key/value, Multi-key, Enrichment

IsGreaterThan

Int or Float

Check if key’s value is greater than specified value.

Key/value, Multi-key, Enrichment

IsGreaterThanOrEqual

Int or Float

Check if key’s value is greater than or equal to specified value.

Key/value, Multi-key, Enrichment

IsLessThan

Int or Float

Check if key’s value is less than specified value.

Key/value, Multi-key, Enrichment

IsLessThanOrEqual

Int or Float

Check if key’s value is less than or equal to specified value.

Key/value, Multi-key, Enrichment

List matching conditions

Use list matching conditions alongside Values (not Value). The match expression's key specifier can reference a single value or a list, but Values must reference a list.

Condition

Key Type

Value Type

Description

Applicable Match Expressions

IsIn

Scalar

List

Evaluate if the value of the specified key is a member of the list.

Key/values, Enrichment

IsNotIn

Scalar

List

Evaluate if the value of the specified key is not a member of the list.

Key/values, Enrichment

List comprehension conditions

List comprehension conditions can only be used within list comprehension match expressions.

Condition

Key Type

Description

Applicable Match Expressions

AnyElement

List

Evaluate if the list of values from the specified key matches any expression in a list of match conditions.

List comprehension

AllElements

List

Evaluate if the list of values from the specified key matches all expressions in a list of match conditions.

List comprehension

OnlyOneElement

List

Evaluate if the list of values from the specified key only has a single element that matches a list of match conditions.

List comprehension

NoElement

List

Evaluate if the list of values from the specified key does not match all expressions in the list of match conditions.

List comprehension

`Value(s)`

Values for the Value and Values keys can have one of four data types: integer, float, boolean, or string. Integers, floats, and booleans should not be enclosed in quotation marks. Strings can be enclosed in quotation marks, but do not have to be.

See examples of how differently formatted Value values will evaluate below:

Detection:
  - Key: someKey
    Condition: Equals
    Value: 2 # this will evaluate to the integer 2
  - Key: someKey
    Condition: Equals
    Value: "2" # this will evaluate to the string "2"
  - Key: someKey
    Condition: Equals
    Value: 2.2 # this will evaluate to the float 2.2
  - Key: someKey
    Condition: Equals
    Value: "2.2" # this will evaluate to the string "2.2"
  - Key: someKey
    Condition: Equals
    Value: true # this will evaluate to the boolean value of true
  - Key: someKey
    Condition: Equals
    Value: "true" # this will evaluate to the string "true"
  - Key: someKey
    Condition: Equals
    Value: some string thing # this will evaluate to the string "some string thing"
  - Key: someKey
    Condition: Equals
    Value: "some string thing" # this will evaluate to the string "some string thing"

PreviousWriting YAML Detections NextSimple Detection Error Codes

Last updated 1 year ago