Writing YAML Detections

Construct YAML detections in the CLI workflow

Overview

YAML detections, as part of the Simple Detections feature set, are in closed beta starting with Panther version 1.81. To request access to the feature or share any bug reports or feature requests, please contact your Panther support team.

In Panther, you can create rules in YAML (in addition to Python).

YAML detections created in the CLI workflow, then uploaded to Panther, will be viewable and editable in the Simple Detection builder in the Console. This may be valuable if members of your team have varying levels of experience with YAML.

If you aren't sure whether to write detections locally in YAML or Python, see the Using Python vs. YAML section.

If your team uses the CLI workflow to manage detection content, the changes made to detections using the Simple Detection builder in the Console will be overwritten on next upload (except for Inline Filters created in the Console, which will be preserved).

If you create or edit detections using the Simple Detection builder in the Console, copy the resulting YAML representation and include it in your local detections files, in order to prevent the changes from being overwritten on next upload.

Limitations of YAML detections

• Scheduled rules and policies cannot be created in YAML.

  • Only rules can be created in YAML.

• Panther-managed YAML detections are not yet available.

  • It is possible, however, to use Panther-managed Python detections alongside your own YAML detections.

• Many helper functions available in Python, including those for specific log sources, are not represented in YAML.

  • Some global helpers have been converted into YAML keys, e.g., deep_get() is DeepKey.

• It is not possible to make external API calls in YAML detections, including to fetch values from your Dynamo KV store to use caching.

How to create a rule in YAML

Creating a rule in YAML in the CLI workflow

If you're writing YAML detections locally (instead of in the Panther Console), we recommend managing your local detection files in a version control system like GitHub or GitLab.

Folder setup

If you group your rules into folders, each folder name must contain rules in order for them to be found during upload (using either PAT or the bulk uploader in the Console).

We recommend grouping rules into folders based on log/resource type, e.g., suricata_rules or aws_s3_policies.

File setup

Each rule and scheduled rule consists of:

• A YAML specification file (a file with a .yml extension) containing the detection logic, as well as metadata attributes of the detection.

  • This file has similar syntax to the Python YAML files, with some additional keys.

YAML rules contain lists of boolean logic called match expressions that detect suspicious behaviors. Returning a value of True indicates suspicious activity, which triggers an alert.

Learn more about YAML detection syntax, and see a complete list of required and optional YAML fields below.

• Create a YAML file (e.g. my_new_rule.yml) using the template below (including a top-level Detection key):

  AnalysisType: rule
  DedupPeriodMinutes: 60 # 1 hour
  DisplayName: Example Rule to Check the Format of the Spec
  Enabled: true
  RuleID: Type.Behavior.MoreContext
  Severity: High
  LogTypes:
    - LogType.GoesHere
  Reports:
    ReportName (like CIS, MITRE ATT&CK):
      - The specific report section relevant to this rule
  Tags:
    - Tags
    - Go
    - Here
  Description: >
    This rule exists to validate the CLI workflows of the Panther CLI
  Runbook: >
    First, find out who wrote this the spec format, then notify them with feedback.
  Reference: https://www.a-clickable-link-to-more-info.com
  Detection:
    - KeyPath: hostName
      Condition: Contains
      Value: prod

After this rule is uploaded to Panther, it will be viewable in the Console in the Simple Detection builder.

YAML detection syntax

Each custom YAML detection can be composed of:

• Detection key

  Detection: 

• Filter key

  InlineFilters: 

• Metadata keys

  AnalysisType: 
  Enabled: 
  RuleID:
  LogTypes: 
  Reports: 
  Tags: 
  Tests: 

• Alert keys (dynamic)

  DynamicSeverities: 
  AlertTitle: 
  AlertContext: 
  GroupBy: 

• Alert keys (static)

  Severity:
  Description:
  DedupPeriodMinutes:
  Threshold: 
  DisplayName:
  OutputIds:
  Reference:
  Runbook:
  SummaryAttributes: 

Learn more about each of these keys, including which are required and optional, in the YAML rule specification reference below.

Detection

Within the Detection key, include one or more match expressions.

InlineFilters

Learn more about using InlineFilters in YAML detections on Modifying Detections with Inline Filters.

Dynamic alert keys in YAML detections

Alert fields are fields in a YAML detection definition that are applicable to the alerts generated by that detection.

Alert fields can be static or dynamic. With static alert fields, you provide a set value in the detection definition, which does not change based on the incoming event. Dynamic alert fields, however, can use information in the event to determine the value.

DynamicSeverities

Use DynamicSeverities to dynamically set the severity of an alert generated by a match on this detection. This field is dynamic because you can use values from the event to determine the severity.

When DynamicSeverities is present, its value overrides the value of the Severity key. Severity is still required, and its value will be the fallback value if there are no matches on any of the match expressions contained within DynamicSeverities.

Within the DynamicSeverities key, include one or more ChangeTo keys, each with a corresponding Conditions key. The value of ChangeTo should be one of the alert severities. The ChangeTo blocks are evaluated in order, from top to bottom, and evaluation stops once a match has been found.

Within Conditions, include one or more match expressions. The Conditions list has the following limitations:

• No list comprehension, multi-key, or absolute match expressions may be used.
• No combinators may be used.

Example:

DynamicSeverities:
  - ChangeTo: CRITICAL
    Conditions:
      - Key: status
        Condition: Equals
        Value: severe
  - ChangeTo: MEDIUM
    Conditions:
      - KeyPath: user.name
        Condition: StartsWith
        Value: admin_
      - DeepKey:
          - user
          - roles
        Condition: Contains
        Value: wheel

AlertTitle

Use AlertTitle to dynamically set the title of an alert generated by a match on this detection. This field is dynamic because you can use values from the event in the title.

The value of AlertTitle should be a string. You can reference event values by using curly braces. Inside the curly braces, use JSON path syntax.

Example:

AlertTitle: "User {actor.username} impersonated {target.username} at {time}"

AlertContext

AlertContext lets you identify event data to pass onto generated alerts, formatted as a dictionary.

Within AlertContext, include one or more KeyName and KeyValue pairs. KeyName takes a string of your choice, which will become the key in the alert context dictionary. Within KeyValue, use a key specifier to indicate an event key—its value will be the value in the alert context dictionary.

Example:

AlertContext:
  - KeyName: ip
    KeyValue:
      Key: sourceIP
  - KeyName: user
    KeyValue:
      DeepKey:
        - user
        - username
  - KeyName: resource
    KeyValue:
      KeyPath: resource.arns

GroupBy

GroupBy sets the deduplication string for your detection. Learn more about deduplication, including the order of precedence for how the deduplication string is set, on Rules and Scheduled Rules.

Within the GroupBy key, include a list of one or more event keys defined with key specifiers.

Example:

GroupBy:
  - Key: sourceIP
  - KeyPath: user.name
  - KeyPath: resource.arn

The values of the keys provided under GroupBy are joined with a colon to form the deduplication string. The outputted deduplication string for the above example would be:

<value of sourceIP>:<value of user.name>:<value of resource.arn>

YAML rule specification reference

The table below contains all available keys for YAML detections. Required fields are in bold.

If you are writing a Python rule, instead see the Python rule specification reference.

Field Name	Description	Expected Value
AnalysisType	Indicates whether this analysis is a rule, scheduled_rule, policy, or global	rule
Enabled	Whether this rule is enabled	Boolean
RuleID	The unique identifier of the rule	String
LogTypes	The list of logs to apply this rule to	List of strings
Severity	Which severity an associated alert should have	One of the following strings: Info, Low, Medium, High, or Critical This field is overwritten by DynamicSeverities, but is required even if DynamicSeverities is defined
Detection	The list of match expressions to apply to the event data	List of match expressions
DynamicSeverities	Alternate severities based on custom sets of conditions	List of dynamic severity configurations, consisting of ChangeTo and Conditions fields. ChangeTo is a Severity value and Conditions is a list of match expressions.
Description	A brief description of the rule	String
GroupBy	Set of event values that will be used to deduplicate alerts by	List of event keys
DedupPeriodMinutes	The time period (in minutes) during which similar events of an alert will be grouped together	15,30,60,180 (3 hours),720 (12 hours), or 1440 (24 hours)
DisplayName	A user-friendly name to show in the Panther Console and alerts. The RuleID will be displayed if this field is not set.	String
OutputIds	Static destination overrides. These will be used to determine how alerts from this rule are routed, taking priority over default routing based on severity.	List of strings
Reference	The reason this rule exists, often a link to documentation	String
Reports	A mapping of framework or report names to values this rule covers for that framework	Map of strings to list of strings
Runbook	The actions to be carried out if this rule returns an alert, often a link to documentation	String
SummaryAttributes	A list of fields that alerts should summarize.	List of strings
Threshold	How many events need to trigger this rule before an alert will be sent.	Integer
Tags	Tags used to categorize this rule	List of strings
Tests	Unit tests for this rule	List of maps
InlineFilters	The list of filters in the form of match expressions to filter in data	List of match expressions (limited to filter-compatible versions)
AlertTitle	An alternate DisplayName that can use event values to create a dynamic title for alerts	String
AlertContext	Event values to add to the Event under custom keys to create a dynamic alert context	List of key name and key value pairs