Simple Detection Match Expression Reference

Construct match expressions to define logic

Overview

A match expression is a conditional logic expression, defined in YAML, that will return true or false. Match expressions are used various places, for example:

There are seven types of match expressions, with varying structures. You can group match expressions using combinators to form complex conditions.

You can use enrichment in Simple Detections with enrichment match expressions.

How match expressions work

Match expressions work roughly the following way:

  • A key from an incoming event is identified with a key specifier

  • A condition is provided, which describes the nature of the check

  • A value (or list of values) is provided to compare against

(There are exceptions to this pattern, where zero or multiple event keys are identified for comparison, or no value is needed.)

Types of match expressions

There are seven types of match expressions, each with a unique syntax or structure.

The match expression examples in the following sections refer to this JSON event:

Example event:

You can refer to this event in the example match expressions in this section.

{
  "audit_event": "RECLASSIFICATION",
  "status": "success",
  "date": "2006-08-24",
  "actor": "International Astronomical Union",
  "old_classification": "PLANET",
  "new_classification": "DWARF_PLANET",
  "entity": {
    "name": "Pluto",
    "diameter": {
      "distance": 2370,
      "unit": "km"
    },
    "moons": [
      {
        "name": "Charon",
        "year_discovered": 1978,
        "discovered_by": ["James W. Christy"],
        "diameter": {
          "distance": 1212,
          "unit": "km"
        }
      },
      {
        "name": "Nix",
        "year_discovered": 2005,
        "discovered_by": [
          "H. A. Weaver",
          "S. A. Stern",
          "M. J. Mutchler",
          "A. J. Steffl",
          "M. W. Buie",
          "W. J. Merline",
          "J. R. Spencer",
          "E. F. Young",
          "L. A. Young"
        ],
        "diameter": {
          "distance": 49.8,
          "unit": "km"
        }
      },
      {
        "name": "Hydra",
        "year_discovered": 2005,
        "discovered_by": ["A. J. Steffl", "M. J. Mutchler"],
        "diameter": {
          "distance": 51
          "unit": "km"
        }
      },
      {
        "name": "Kerberos",
        "year_discovered": 2011,
        "discovered_by": ["M. R. Showalter", "D. P. Hamilton"]
        "diameter": {
          "distance": 19,
          "unit": "km",
        }
      },
      {
        "name": "Styx",
        "year_discovered": 2012,
        "discovered_by": ["M. R. Showalter", "D. P. Hamilton"]
        "diameter": {
          "distance": 25
          "unit": "km" 
        }
      }
    ],
    "single_year_in_earth_years": 247.94,
    "year_discovered": 1930,
    "discovered_by": "Clyde Tombaugh",
    "orbit_region": "Kuiper belt",
    "atmosphere": {
      "composition": ["nitrogen", "methane", "carbon monoxide"],
      "surface_pressure": {
        "number": 1,
        "unit": "pascals"
      }
    }
  },
  "p_enrichment": {
    "discoverer_info": {
        "discovered_by": {
            "p_match": "Clyde Tombaugh",
	    "year_born": 1906,
	    "year_died": 1997
	}
    }
  } 
}

In each example match expression below, a single key specifier is used to identify the event key(s), however any of the key specifiers (e.g., Key, DeepKey or KeyPath) can be used in most cases.

Key/value match expressions

Key/value match expressions compare the value of a certain event key to a given value.

You can see all allowed values of Condition in the Scalar matching conditions table below, where the "Applicable Match Expressions" column includes "Key/value."

Example:

- Key: audit_event
  Condition: Equals
  Value: reclassification

Key/values match expressions

Key/values match expressions evaluate whether some event value is a member in a given list.

The only permitted Condition values for key/values match expressions are IsIn and IsNotIn, visible in the List matching conditions table below.

Example:

- KeyPath: entity.name
  Condition: IsIn
  Values:
    - Mercury
    - Venus
    - Earth
    - Mars
    - Jupiter
    - Saturn
    - Uranus
    - Neptune

Multi-key match expressions

Multi-key match expressions cannot be viewed and edited in the Simple Detection builder in the Panther Console, nor can they be used in Inline Filters. See a full list of Simple Detection builder limitations here, and Inline Filters limitations here.

Multi-key match expressions allow you to compare the values of two event keys. You can form a multi-key match expression by including a Condition and list of Values that contains two event keys.

See all allowed values of Condition in the Scalar matching conditions table below, where the "Applicable Match Expressions" column includes "Multi-key."

- Condition: IsGreaterThan
  Values: # These values can be any combination of `Key`, 'DeepKey', or 'KeyPath, with up to two clauses.
    - KeyPath: entity.diameter.distance # Pluto is bigger than...
    - KeyPath: entity.moons[0].diameter.distance # Charon

List comprehension match expressions

List comprehension match expressions cannot be viewed and edited in the Simple Detection builder in the Panther Console, nor can they be used in Inline Filters. See a full list of Simple Detection builder limitations here, and Inline Filters limitations here.

A list comprehension match expression takes a key with a list value, and, for each item in the list, evaluates it against a list of match expressions. The list of match expressions is stored under the Expressions key.

For list comprehension match expressions, four values for Condition are valid:

  • AnyElement

  • AllElements

  • OnlyOneElement

  • NoElement

Learn more about these Condition values in the List comprehension conditions table below.

Example using AnyElement:

- KeyPath: entity.moons
  Condition: AnyElement
  Expressions:
    - Key: name
      Condition: Equals
      Value: "Death Star"
    - Key: year_discovered
      Condition: IsGreaterThan
      Value: 2000

Within the Expressions key, you can add either:

  • Nested list comprehension match expressions. Each level of nesting can only contain one list comprehension match expression. This structure effectively creates nested for loops. The innermost list comprehension can have as many match expressions as you like, so long as they are not list comprehension match expressions. Example:

    - KeyPath: entity.moons
      Condition: AnyElement
      Expressions:
        - Key: discovered_by
          Condition: AllElements
          Expressions:
            - Key: $Element$
              Condition: Equals
              Value: "James W. Christy"
  • Any combination of match expressions that are not list comprehension match expressions. Example:

    - Key: someKey
      Condition: AnyElement
      Expressions:
        - Key: a
          Condition: Equals
          Value: 123
        - Key: b
          Condition: IsIn
          Values:
          - 345
          - 678

Existence match expressions

An existence match expression determines whether a specific key matches a specified existence condition.

See all allowed values of Condition in the Scalar matching conditions table below, where the "Applicable Match Expressions" column includes "Existence."

Example:

- Key: entity.atmosphere
  Condition: Exists

Absolute match expressions

Absolute match expressions cannot be viewed and edited in the Simple Detection builder in the Panther Console, nor can they be used in Inline Filters. See a full list of Simple Detection builder limitations here, and Inline Filters limitations here.

In certain cases, you may want a match expression to always evaluate to True or False. You can do this by creating a match expression that simply contains a Condition key set to AlwaysTrue or AlwaysFalse.

- Condition: AlwaysTrue

Enrichment match expressions

Enrichment match expressions cannot be viewed and edited in the Simple Detection builder in the Panther Console, nor can they be used in Inline Filters. See a full list of Simple Detection builder limitations here, and Inline Filters limitations here.

Enrichment match expressions are only used to interact with a JSON event's p_enrichment struct, which is present if there is a match on a Lookup Table, Enrichment Provider, or Identify Provider Profile.

To create an enrichment match expression, include an Enrichment key. The contents of Enrichment indicate the path to the event key on which to build the condition. Within Enrichment, include the following fields:

  • Table: The name of the Lookup Table, Enrichment Provider table, or Identity Provider Profile table.

    • These names are user-defined in the case of Lookup Tables, and Panther-defined in the case of Enrichment Providers and Identity Provider Profiles.

  • Selector: The name of the selector field from the incoming log.

  • FieldPath: The name of any one of the event fields within the Selector field. This key takes JSON path notation, meaning you can denote nested fields, if needed.

Once you have defined Enrichment and its sub-fields, the rest of the match expression can be modeled after the structure of any of the other match expression types—except for List comprehension match expressions (these expressions are not supported for use with enrichment) and Absolute match expressions (these expressions do not reference any event values).

For example, the enrichment match expression below—which includes Condition and Value keys—resembles a key/value match expression:

- Enrichment:
    Table: discoverer_info
    Selector: discovered_by
    FieldPath: p_match
  Condition: Equals
  Value: "Clyde Tombaugh"

Combinators

The OnlyOne and None combinators cannot be viewed and edited in the Simple Detection builder in the Panther Console, nor can they be used in Inline Filters. See a full list of Simple Detection builder limitations here, and Inline Filters limitations here.

Combinators are YAML keys that describe the relationship between match expressions or groups of match expressions. There are four combinators: All, Any, OnlyOne, and None.

If more than one match expression is provided directly under a key that takes match expressions (e.g., Detection, InlineFilters or Expressions), and no combinator is specified, All is assumed.

When writing Simple Detections in the CLI workflow, there is no limit on the number of nested levels you can create with combinators. (In the Simple Detection builder in the Panther Console, there is a three-level limit.) Note that if match expressions are nested more than three levels deep when working locally, the Panther Console will not render them in the Simple Detection builder—they will only be viewable in raw YAML.

Example of one level of nesting using OnlyOne:

# Exactly one of the listed expressions must evaluate 
# to true for the `OnlyOne` block to evaluate to true
- OnlyOne:
  - Key: someKey
    Condition: DoesNotEqual
    Value: 1234
  - Key: someOtherKey
    Condition: Contains
    Value: "invalid login"
  - Key: someOtherOtherKey
    Condition: IsIn
    Values:
      - 314
      - 567

Example of multiple levels of nesting using Any and All:

- Any:
      - All:
          - Key: eventSource
            Condition: StartsWith
            Value: 'dynamodb'
          - Any:
              - Key: eventName
                Condition: StartsWith
                Value: List
              - Key: eventName
                Condition: StartsWith
                Value: Describe
              - Key: eventName
                Condition: StartsWith
                Value: Get
      - All:
          - Key: eventSource
            Condition: StartsWith
            Value: 'ec2'
          - Any:
              - Key: eventName
                Condition: StartsWith
                Value: Describe
              - Key: eventName
                Condition: StartsWith
                Value: Get

Key specifiers

Key specifiers are a group of YAML keys that allow you to indicate a certain key in your log event JSON. There are three key specifiers: Key, DeepKey, and KeyPath.

If the event property provided as the value of the key specifier does not exist, evaluation of the match expression stops and returns false.

Key

Key lets you access a top-level event property. Example:

Key: username

DeepKey

DeepKey lets you access a nested event property. DeepKey accepts a list. You can also use DeepKey to access elements of an array when you know the absolute index.

Example using DeepKey, which accesses the deep field foo.bar.baz, then indexes on an array index 2:

- DeepKey:
  - foo
  - bar
  - baz
  - 2

KeyPath

KeyPath lets you access:

Example:

KeyPath: foo[*].bar
# or
KeyPath: foo.bar[2]
# or
KeyPath: foo.bar[2].baz
# or
KeyPath: foo.bar.baz
# or 
KeyPath: foo

Condition

There are various types of Condition values you can choose from, including scalar, list, and list comprehension groupings.

Certain Condition values cannot be viewed and edited in the Simple Detection builder in the Panther Console, nor can they be used in Inline Filters. See a full list of Simple Detection builder limitations here, and Inline Filters limitations here.

Scalar matching conditions

Use scalar matching conditions alongside the Value key (not Values). The match expression's key specifier can reference a list or a single value, but Value must be singular.

List matching conditions

Use list matching conditions alongside Values (not Value). The match expression's key specifier can reference a single value or a list, but Values must reference a list.

List comprehension conditions

List comprehension conditions can only be used within list comprehension match expressions.

Value(s)

Values for the Value and Values keys can have one of the following four data types:

  • Integer

  • Float

  • Boolean

  • String

Integers, floats, and booleans should not be enclosed in quotation marks. Strings can be enclosed in quotation marks, but do not have to be.

See examples of how differently formatted Value values will evaluate below:

Detection:
  - Key: someKey
    Condition: Equals
    Value: 2 # this will evaluate to the integer 2
  - Key: someKey
    Condition: Equals
    Value: "2" # this will evaluate to the string "2"
  - Key: someKey
    Condition: Equals
    Value: 2.2 # this will evaluate to the float 2.2
  - Key: someKey
    Condition: Equals
    Value: "2.2" # this will evaluate to the string "2.2"
  - Key: someKey
    Condition: Equals
    Value: true # this will evaluate to the boolean value of true
  - Key: someKey
    Condition: Equals
    Value: "true" # this will evaluate to the string "true"
  - Key: someKey
    Condition: Equals
    Value: some string thing # this will evaluate to the string "some string thing"
  - Key: someKey
    Condition: Equals
    Value: "some string thing" # this will evaluate to the string "some string thing"

Last updated