YAML Match Expression Reference

Construct match expressions to define logic

Overview

This functionality, as part of the Simple Detections feature set, is in closed beta starting with Panther version 1.81. To request access to the feature or share any bug reports or feature requests, please contact your Panther support team.

A match expression is a conditional logic expression, defined in YAML, that will return true or false. Match expressions are used various places, for example:

• In the Detection key of a YAML detection

• In the InlineFilters key of a Python or YAML detection

• In the Expressions key of a List Comprehension match expression (effectively creating match expressions within a match expression)

• In the Conditions key of the DynamicSeverities key of a YAML detection

There are seven types of match expressions, with varying structures. You can group match expressions using combinators to form complex conditions.

You can use enrichment in YAML detections with enrichment match expressions.

How match expressions work

Match expressions work roughly the following way:

• A key from an incoming event is identified with a key specifier

• A condition is provided, which describes the nature of the check

• A value (or list of values) is provided to compare against

(There are exceptions to this pattern, where zero or multiple event keys are identified for comparison, or no value is needed.)

Types of match expressions

There are seven types of match expressions, each with a unique syntax or structure.

The match expression examples in the following sections refer to this JSON event:

Example event:

You can refer to this event in the example match expressions in this section.

{
  "audit_event": "RECLASSIFICATION",
  "status": "success",
  "date": "2006-08-24",
  "actor": "International Astronomical Union",
  "old_classification": "PLANET",
  "new_classification": "DWARF_PLANET",
  "entity": {
    "name": "Pluto",
    "diameter": {
      "distance": 2370,
      "unit": "km"
    },
    "moons": [
      {
        "name": "Charon",
        "year_discovered": 1978,
        "discovered_by": ["James W. Christy"],
        "diameter": {
          "distance": 1212,
          "unit": "km"
        }
      },
      {
        "name": "Nix",
        "year_discovered": 2005,
        "discovered_by": [
          "H. A. Weaver",
          "S. A. Stern",
          "M. J. Mutchler",
          "A. J. Steffl",
          "M. W. Buie",
          "W. J. Merline",
          "J. R. Spencer",
          "E. F. Young",
          "L. A. Young"
        ],
        "diameter": {
          "distance": 49.8,
          "unit": "km"
        }
      },
      {
        "name": "Hydra",
        "year_discovered": 2005,
        "discovered_by": ["A. J. Steffl", "M. J. Mutchler"],
        "diameter": {
          "distance": 51
          "unit": "km"
        }
      },
      {
        "name": "Kerberos",
        "year_discovered": 2011,
        "discovered_by": ["M. R. Showalter", "D. P. Hamilton"]
        "diameter": {
          "distance": 19,
          "unit": "km",
        }
      },
      {
        "name": "Styx",
        "year_discovered": 2012,
        "discovered_by": ["M. R. Showalter", "D. P. Hamilton"]
        "diameter": {
          "distance": 25
          "unit": "km" 
        }
      }
    ],
    "single_year_in_earth_years": 247.94,
    "year_discovered": 1930,
    "discovered_by": "Clyde Tombaugh",
    "orbit_region": "Kuiper belt",
    "atmosphere": {
      "composition": ["nitrogen", "methane", "carbon monoxide"],
      "surface_pressure": {
        "number": 1,
        "unit": "pascals"
      }
    }
  },
  "p_enrichment": {
    "discoverer_info": {
        "discovered_by": {
            "p_match": "Clyde Tombaugh",
	    "year_born": 1906,
	    "year_died": 1997
	}
    }
  } 
}

In each example match expression below, a single key specifier is used to identify the event key(s), however any of the key specifiers (e.g., Key, DeepKey or KeyPath) can be used in most cases.

Key/value match expressions

Key/value match expressions compare the value of a certain event key to a given value.

You can see all allowed values of Condition in the Scalar matching conditions table below, where the "Applicable Match Expressions" column includes "Key/value."

Example:

- Key: audit_event
  Condition: Equals
  Value: reclassification

Key/values match expressions

Key/values match expressions evaluate whether some event value is a member in a given list.

The only permitted Condition values for key/values match expressions are IsIn and IsNotIn, visible in the List matching conditions table below.

Example:

- KeyPath: entity.name
  Condition: IsIn
  Values:
    - Mercury
    - Venus
    - Earth
    - Mars
    - Jupiter
    - Saturn
    - Uranus
    - Neptune

Multi-key match expressions

Multi-key match expressions cannot be viewed and edited in the Simple Detection builder in the Panther Console, nor can they be used in Inline Filters. See a full list of Simple Detection builder limitations here, and Inline Filters limitations here.

Multi-key match expressions allow you to compare the values of two event keys. You can form a multi-key match expression by including a Condition and list of Values that contains two event keys.

See all allowed values of Condition in the Scalar matching conditions table below, where the "Applicable Match Expressions" column includes "Multi-key."

- Condition: IsGreaterThan
  Values: # These values can be any combination of `Key`, 'DeepKey', or 'KeyPath, with up to two clauses.
    - KeyPath: entity.diameter.distance # Pluto is bigger than...
    - KeyPath: entity.moons[0].diameter.distance # Charon

List comprehension match expressions

List comprehension match expressions cannot be viewed and edited in the Simple Detection builder in the Panther Console, nor can they be used in Inline Filters. See a full list of Simple Detection builder limitations here, and Inline Filters limitations here.

A list comprehension match expression takes a key with a list value, and, for each item in the list, evaluates it against a list of match expressions. The list of match expressions is stored under the Expressions key.

For list comprehension match expressions, four values for Condition are valid:

• AnyElement

• AllElements

• OnlyOneElement

• NoElement

Learn more about these Condition values in the List comprehension conditions table below.

Example using AnyElement:

- KeyPath: entity.moons
  Condition: AnyElement
  Expressions:
    - Key: name
      Condition: Equals
      Value: "Death Star"
    - Key: year_discovered
      Condition: IsGreaterThan
      Value: 2000

Within the Expressions key, you can add either:

• Nested list comprehension match expressions. Each level of nesting can only contain one list comprehension match expression. This structure effectively creates nested for loops. The innermost list comprehension can have as many match expressions as you like, so long as they are not list comprehension match expressions. Example:

  - KeyPath: entity.moons
    Condition: AnyElement
    Expressions:
      - Key: discovered_by
        Condition: AllElements
        Expressions:
          - Key: $Element$
            Condition: Equals
            Value: "James W. Christy"

• Any combination of match expressions that are not list comprehension match expressions. Example:

  - Key: someKey
    Condition: AnyElement
    Expressions:
      - Key: a
        Condition: Equals
        Value: 123
      - Key: b
        Condition: IsIn
        Values:
        - 345
        - 678

Existence match expressions

An existence match expression determines whether a specific key matches a specified existence condition.

See all allowed values of Condition in the Scalar matching conditions table below, where the "Applicable Match Expressions" column includes "Existence."

Example:

- Key: entity.atmosphere
  Condition: Exists

Absolute match expressions

Absolute match expressions cannot be viewed and edited in the Simple Detection builder in the Panther Console, nor can they be used in Inline Filters. See a full list of Simple Detection builder limitations here, and Inline Filters limitations here.

In certain cases, you may want a match expression to always evaluate to True or False. You can do this by creating a match expression that simply contains a Condition key set to AlwaysTrue or AlwaysFalse.

- Condition: AlwaysTrue

Enrichment match expressions

Enrichment match expressions cannot be viewed and edited in the Simple Detection builder in the Panther Console, nor can they be used in Inline Filters. See a full list of Simple Detection builder limitations here, and Inline Filters limitations here.

Enrichment match expressions are only used to interact with a JSON event's p_enrichment struct, which is present if there is a match on a Lookup Table, Enrichment Provider, or Identify Provider Profile.

To create an enrichment match expression, include an Enrichment key. The contents of Enrichment indicate the path to the event key on which to build the condition. Within Enrichment, include the following fields:

• Table: The name of the Lookup Table, Enrichment Provider table, or Identity Provider Profile table.

  • These names are user-defined in the case of Lookup Tables, and Panther-defined in the case of Enrichment Providers and Identity Provider Profiles.

• Selector: The name of the selector field from the incoming log.

• FieldPath: The name of any one of the event fields within the Selector field. This key takes JSON path notation, meaning you can denote nested fields, if needed.

Once you have defined Enrichment and its sub-fields, the rest of the match expression can be modeled after the structure of any of the other match expression types—except for List comprehension match expressions (these expressions are not supported for use with enrichment) and Absolute match expressions (these expressions do not reference any event values).

For example, the enrichment match expression below—which includes Condition and Value keys—resembles a key/value match expression:

- Enrichment:
    Table: discoverer_info
    Selector: discovered_by
    FieldPath: p_match
  Condition: Equals
  Value: "Clyde Tombaugh"

Combinators

The OnlyOne and None combinators cannot be viewed and edited in the Simple Detection builder in the Panther Console, nor can they be used in Inline Filters. See a full list of Simple Detection builder limitations here, and Inline Filters limitations here.

Combinators are YAML keys that describe the relationship between match expressions or groups of match expressions. There are four combinators: All, Any, OnlyOne, and None.

If more than one match expression is provided directly under a key that takes match expressions (e.g., Detection, InlineFilters or Expressions), and no combinator is specified, All is assumed.

When writing YAML detections in the CLI workflow, there is no limit on the number of nested levels you can create with combinators. (In the Simple Detection builder in the Panther Console, there is a three-level limit.) Note that if match expressions are nested more than three levels deep when working locally, the Panther Console will not render them in the Simple Detection builder—they will only be viewable in raw YAML.

Combinator	Boolean operator name	Description
All	Logical AND	All match expressions in the list evaluate to true. The first failing match ends evaluation of the list.
Any	Logical OR	Any single match expression in the list evaluates to true. The first true match ends evaluation of the list.
OnlyOne	Logical XOR	Only one single match expression in the list evaluates to true.
None	Logical NOT AND	No match expression in the list evaluates to true.

Example of one level of nesting using OnlyOne:

# Exactly one of the listed expressions must evaluate 
# to true for the `OnlyOne` block to evaluate to true
- OnlyOne:
  - Key: someKey
    Condition: DoesNotEqual
    Value: 1234
  - Key: someOtherKey
    Condition: Contains
    Value: "invalid login"
  - Key: someOtherOtherKey
    Condition: IsIn
    Values:
      - 314
      - 567

Example of multiple levels of nesting using Any and All:

- Any:
      - All:
          - Key: eventSource
            Condition: StartsWith
            Value: 'dynamodb'
          - Any:
              - Key: eventName
                Condition: StartsWith
                Value: List
              - Key: eventName
                Condition: StartsWith
                Value: Describe
              - Key: eventName
                Condition: StartsWith
                Value: Get
      - All:
          - Key: eventSource
            Condition: StartsWith
            Value: 'ec2'
          - Any:
              - Key: eventName
                Condition: StartsWith
                Value: Describe
              - Key: eventName
                Condition: StartsWith
                Value: Get

Key specifiers

Key specifiers are a group of YAML keys that allow you to indicate a certain key in your log event JSON. There are three key specifiers: Key, DeepKey, and KeyPath.

If the event property provided as the value of the key specifier does not exist, evaluation of the match expression stops and returns false.

Key

Key lets you access a top-level event property. Example:

Key: username

DeepKey

DeepKey lets you access a nested event property. DeepKey accepts a list. You can also use DeepKey to access elements off an array when you know the absolute index.

Example using DeepKey, which accesses the deep field foo.bar.baz, then indexes on an array index 2:

- DeepKey:
  - foo
  - bar
  - baz
  - 2

KeyPath

KeyPath lets you access a nested event property using subfield indexing (.) and array indexing ([x]) (additional JSON path features are not supported). The value of KeyPath could also just be a top-level field.

Example:

KeyPath: foo.bar.baz
# or 
KeyPath: foo

Condition

There are various types of Condition values you can choose from, including scalar, list, and list comprehension groupings.

Certain Condition values cannot be viewed and edited in the Simple Detection builder in the Panther Console, nor can they be used in Inline Filters. See a full list of Simple Detection builder limitations here, and Inline Filters limitations here.

Scalar matching conditions

Use scalar matching conditions alongside the Value key (not Values). The match expression's key specifier can reference a list or a single value, but Value must be singular.

Condition	Key’s Value Types	Description	Applicable Match Expressions
Exists	Any	Checks if the key specified exists.	Existence, Enrichment
DoesNotExist	Any	Checks if the key specified does not exist.	Existence, Enrichment
IsNull	Any	Checks if the key is null - e.g. {"key": null}	Existence, Enrichment
IsNotNull	Any	Checks if the key is not null - e.g. {"key": 1234}	Existence, Enrichment
IsNullOrEmpty	Any	Checks if the key is null or has an non-empty value. Empty values include: empty list ([]), empty string "", and the number 0.	Existence, Enrichment
IsNotNullOrEmpty	Any	Checks if the key is not null or has a non-empty value. Empty values include: empty list ([]), empty string "", and the number 0.	Existence, Enrichment
Equals	Any	Checks if a key’s value is equal to a specified value.	Key/value, Multi-key, Enrichment
DoesNotEqual	Any	Checks if a key’s value is not equal to a specified value.	Key/value, Multi-key, Enrichment
IEquals	String	Checks if a key’s string value is equal to a specified string, case insensitive.	Key/value, Multi-key, Enrichment
IDoesNotEqual	String	Checks if a key’s string value is not equal to a specified string, case insensitive.	Key/value, Multi-key, Enrichment
StartsWith	String	Checks if a key’s string value starts with a specified value. If the key is not a string, returns false.	Key/value, Multi-key, Enrichment
IStartsWith	String	Checks if a key’s string value starts with a specified value, case insensitive. If the key is not a string, returns false.	Key/value, Multi-key, Enrichment
DoesNotStartWith	String	Checks if a key’s string value does not start with a specified value. If they key is not a string, returns false.	Key/value, Multi-key, Enrichment
IDoesNotStartWith	String	Checks if a key’s string value does not start with a specified value, case insensitive. If they key is not a string, returns false.	Key/value, Multi-key, Enrichment
EndsWith	String	Checks if a key’s string value ends with a specified value. If the key is not a string, returns false.	Key/value, Multi-key, Enrichment
IEndsWith	String	Checks if a key’s string value ends with a specified value, case insensitive. If the key is not a string, returns false.	Key/value, Multi-key, Enrichment
DoesNotEndWith	String	Checks if a key’s string value does not end with a specified value. If they key is not a string, returns false.	Key/value, Multi-key, Enrichment
IDoesNotEndWith	String	Checks if a key’s string value does not end with a specified value, case insensitive. If they key is not a string, returns false.	Key/value, Multi-key, Enrichment
Contains	String or List	Checks if a key’s string value contains a specified value. If the key is not a string, returns false. OR Checks if an array of string(case sensitive)/int/float/bool contains a value.	Key/value, Multi-key, Enrichment
IContains	String	Checks if a key’s string value contains a specified value, case insensitive. If the key is not a string, returns false.	Key/value, Multi-key, Enrichment
DoesNotContain	String or List	Checks if a key’s string value does not contain a specified value. If they key is not a string, returns false. OR Checks if an array of string(case sensitive)/int/float/bool does not contains a value.	Key/value, Multi-key, Enrichment
IDoesNotContain	String	Checks if a key’s string value does not contain a specified value, case insensitive. If they key is not a string, returns false.	Key/value, Multi-key, Enrichment
IsIPAddress	String or Int	Checks if a key’s string or int value can be resolved into an IP Address. Functionally equivalent to ANY(IsIPv4Address, IsIPv6Address)	Key Existence Match, Enrichment
IsIPv4Address	String or Int	Checks if a key’s string or int value can be resolved into an IPv4 Address.	Key Existence Match, Enrichment
IsIPv6Address	String or Int	Checks if a key’s string or int value can be resolved into an IPv6 Address.	Key Existence Match, Enrichment
IsIPAddressPrivate	String or Int	Checks if a key’s string or int value can be resolved into a private IP Address.	Key Existence Match, Enrichment
IsIPAddressPublic	String or Int	Checks if a key’s string or int value can be resolved into a public IP Address.	Key Existence Match, Enrichment
IsIPAddressInCIDR	String or Int	Checks if a key’s string or int value rendered as an IP address is inside a CIDR range.	Key/value, Multi-key, Enrichment
IsIPAddressNotInCIDR	String or Int	Checks if a key’s string or int value rendered as an IP address is not inside a CIDR range.	Key/value, Multi-key, Enrichment
CIDRContainsIPAddresses	List of IP’s as Strings	Checks if a key’s string value rendered as a CIDR range contains a specified IP address.	Key/value, Multi-key, Enrichment
CIDRDoesNotContainIPAddresses	List of IP’s as Strings	Checks if a key’s value interpreted a list of IP addresses is in specified CIDR range	Key/value, Multi-key, Enrichment
IsGreaterThan	Int or Float	Check if key’s value is greater than specified value.	Key/value, Multi-key, Enrichment
IsGreaterThanOrEqual	Int or Float	Check if key’s value is greater than or equal to specified value.	Key/value, Multi-key, Enrichment
IsLessThan	Int or Float	Check if key’s value is less than specified value.	Key/value, Multi-key, Enrichment
IsLessThanOrEqual	Int or Float	Check if key’s value is less than or equal to specified value.	Key/value, Multi-key, Enrichment

List matching conditions

Use list matching conditions alongside Values (not Value). The match expression's key specifier can reference a single value or a list, but Values must reference a list.

Condition	Key Type	Value Type	Description	Applicable Match Expressions
IsIn	Scalar	List	Evaluate if the value of the specified key is a member of the list.	Key/values, Enrichment
IsNotIn	Scalar	List	Evaluate if the value of the specified key is not a member of the list.	Key/values, Enrichment

List comprehension conditions

List comprehension conditions can only be used within list comprehension match expressions.

Condition	Key Type	Description	Applicable Match Expressions
AnyElement	List	Evaluate if the list of values from the specified key matches any expression in a list of match conditions.	List comprehension
AllElements	List	Evaluate if the list of values from the specified key matches all expressions in a list of match conditions.	List comprehension
OnlyOneElement	List	Evaluate if the list of values from the specified key only has a single element that matches a list of match conditions.	List comprehension
NoElement	List	Evaluate if the list of values from the specified key does not match all expressions in the list of match conditions.	List comprehension

Value(s)

Values for the Value and Values keys can have one of four data types: integer, float, boolean, or string. Integers, floats, and booleans should not be enclosed in quotation marks. Strings can be enclosed in quotation marks, but do not have to be.

See examples of how differently formatted Value values will evaluate below:

Detection:
  - Key: someKey
    Condition: Equals
    Value: 2 # this will evaluate to the integer 2
  - Key: someKey
    Condition: Equals
    Value: "2" # this will evaluate to the string "2"
  - Key: someKey
    Condition: Equals
    Value: 2.2 # this will evaluate to the float 2.2
  - Key: someKey
    Condition: Equals
    Value: "2.2" # this will evaluate to the string "2.2"
  - Key: someKey
    Condition: Equals
    Value: true # this will evaluate to the boolean value of true
  - Key: someKey
    Condition: Equals
    Value: "true" # this will evaluate to the string "true"
  - Key: someKey
    Condition: Equals
    Value: some string thing # this will evaluate to the string "some string thing"
  - Key: someKey
    Condition: Equals
    Value: "some string thing" # this will evaluate to the string "some string thing"