Writing Detections
Use Panther Detections to analyze data, run queries, and trigger alerts on suspicious behavior

Overview

Panther has three core detection types to trigger alerts on suspicious behavior:
  • Rules
    • Panther's Rules are Python functions for detecting suspicious log activity and generating alerts. Rules specifically apply to security logs. For more information, see Rules.
  • Scheduled Rules
    • Panther's Scheduled Rules are Python functions that run on query results against your data lake. For more information, see Scheduled Rules.
  • Policies
    • Panther's Policies are Python functions that scan and evaluate cloud infrastructure configurations to identify misconfigured infrastructure and subsequently generate alerts. Policies specifically apply to cloud resources. For more information, see Policies.

Getting started with writing detections

Choose a method for writing and managing detections

You can create and manage Panther detections using one of the following methods:
  • Panther Console.
    • Create individual rules via the UI or enable Panther-managed Detection Packs to quickly get started with no additional configuration necessary.
    • Learn more about managing Panther detections using a Continuous Integration and Continuous Deployment workflow in our CI/CD Guide.

Migrating to a CI/CD workflow

You can get started quickly by enabling Panther-managed Detection Packs in the Panther Console, but later on you may want to start using a CI/CD workflow. To migrate your workflow to CI/CD, follow the steps in the guide Managing Detections via CI/CD.
Managing detections via both the Panther Console and a Git-based workflow simultaneously may result in unexpected behavior.

How to write detections

Panther Console
Panther Analysis Tool
  1. 1.
    Log in to the Panther Console.
  2. 2.
    Click Build > Detections.
  3. 3.
    Click Create New.
  4. 4.
    Select your detection type between a Rule, Policy, or Scheduled Rule.
  5. 5.
    Fill out the required Basic Info as follows:
    • Enabled: Toggle the button in the right-hand corner to ON.
    • Name: Enter a descriptive name for your new detection.
    • Severity: Select a detection severity from the drop-down options.
    • The lower right-hand drop-down menu differs depending on the detection type you chose:
      • Rule: Select the applicable Log Types.
      • Policy: Select the applicable Resource Types.
      • Scheduled Rule: Select the Scheduled Query or queries this rule should apply to.
    • Unique ID (optional): Click the pen icon and enter a unique ID for your detection.
  6. 6.
    Click the Functions & Tests tab and write a Python function to define your Detection in the Rule Function text editor.
  7. 7.
    Click {} Create Test to run a test against the Detection you defined in the previous step.
  8. 8.
    Optional: Adjust your Rule Settings or add a report on Report Mapping.
  9. 9.
    Click Save in the upper right corner.
You will land on your detection's information page - click Edit in the upper right-hand corner to manage or update your detection.
You can use PAT to test and upload locally managed Detections and optionally integrate with a CI/CD setup.
The steps below explain how to use PAT to write detections. For detailed instructions, see Panther Developer Workflows.
  1. 1.
    Create an API token.
    • PAT requires an API Token to authenticate against your Panther instance. Follow our documentation for Creating an API Token. You will then pass this API token as an argument to the panther_analysis_tool command for operations such as uploading/deleting detections, custom schemas, saved queries, and more.
  2. 4.
  3. 5.
    For further testing, packaging, and deploying capabilities, install our Python CLI called the Panther Analysis Tool.

Working with your data in Panther

Schema definitions

Panther’s Schemas provide helpful information on the types of fields contained within your data, which makes it easier to understand how to interact with your data when writing a Detection.
Schema definitions can be found in:
  • Panther's documentation
    • See schemas for each integration within the Supported Logs section of the documentation, and find more information about Custom Log schemas in Custom Logs.
  • The Panther Console
    • Log in to the Panther Console and navigate to Data > Schemas.

Pulling samples out of Data Explorer

Data Explorer makes it easier to understand and investigate data, location of data, and data types when writing Python code. It contains all the data Panther parses from your log sources and stores the data in tables.
Explore and find log events by searching the relevant table for the log type you are interested in writing a detection for.
You can preview example table data without writing SQL. To generate a sample SQL query for that log source, click the eye icon next to the table type:
When the query has produced results, you will see the example log events in the Results table. You can download these as a CSV file.
To copy the log event to be used in Unit Tests while writing detections, click View JSON.

Panther Detection examples and best practices

Python best practices

Python Enhancement Proposals publishes resources on how to cleanly and effectively write and style your Python code. For example, you can use autopep8 to automatically ensure that the Detections you write all follow a consistent style.

Understanding the structure of a Detection in Panther

See the Detection Functions section below to view all available functions within a Panther Detection.
The only required function is def rule(event), but other functions make your Alerts more dynamic. See the section Configuring Detection functions dynamically for examples and for more information about the different functions.

Basic Rule template

def rule(event):
if event.get("Something"):
return True
return False
return True triggers an alert, while return False does not trigger an alert.
For more templates, see the panther_analysis repo on Github.

Detection writing best practices

Writing tests for your detections

Before enabling new detections, it is recommended to write tests that define scenarios where alerts should or should not be generated. Best practice dictates at least one positive and one negative to ensure the most reliability.

Casing for event fields

Lookups for event fields are not case sensitive. event.get("Event_Type") or event.get("event_type") will return the same result.

Understanding top level fields and nested fields

Top-level fields represent the parent fields in a nested data structure. For example, a record may contain a field called user under which there are other fields such as ip_address. In this case, user is the top-level field, and ip_address is a nested field underneath it.
Nesting can occur many layers deep, and so it is valuable to understand the schema structure and know how to access a given field for a detection.

Accessing top-level fields safely

Basic Rules match a field’s value in the event, and a best practice to avoid errors is to leverage Python’s built-in get() function.
The example below is a best practice because it leverages a get() function. get() will look for a field, and if the field doesn't exist, it will return None instead of an error, which will result in the detection returning False.
def rule(event):
return event.get('field') == 'value'
In the example below, if the field exists, the value of the field will be returned. Otherwise, False will be returned:
def rule(event):
if event.get('field')
return event.get('field')
return False
Bad practice example The example below is bad practice because the code is explicit about the field name. If the field doesn't exist, Python will throw a KeyError: def rule(event):
return event['field'] == 'value'

Using Global Helper functions

Once many detections are written, a set of patterns and repeated code will begin to emerge. This is a great use case for Global Helper functions, which provide a centralized location for this logic to exist across all detections. For example, see the deep_get() function referenced in the next section.

Accessing nested fields safely

If the field is nested deep within the event, use a Panther-supplied function called deep_get() to safely access the fields value. deep_get() must be imported by the panther_base_helpers library.
deep_get() takes two or more arguments:
  • The event object itself (required)
  • The top-level field name (required)
  • Any nested fields, in order (as many nested fields as needed)
Example:
AWS CloudTrail logs nest the type of user accessing the console underneath userIdentity.
JSON CloudTrail root activity:
{
"eventVersion": "1.05",
"userIdentity": {
"type": "Root",
"principalId": "1111",
"arn": "arn:aws:iam::123456789012:root",
"accountId": "123456789012",
"userName": "root"
},
...
}
Here is how you could check that value safely with deep_get:
from panther_base_helpers import deep_get
def rule(event):
return deep_get(event, "userIdentity", "type") == "Root"

Checking fields for specific values

You may want to know when a specific event has occurred. If it did occur, then the detection should trigger an alert. Since Panther stores everything as normalized JSON, you can check the value of a field against the criteria you specify.
For example, to detect the action of granting Box technical support access to your Box account, the Python below would be used to match events where the event_type equals ACCESS_GRANTED:
def rule(event):
return event.get("event_type") == "ACCESS_GRANTED"
If the field is event_type and the value is equal to ACCESS_GRANTED then the rule function will return true and an Alert will be created.

Checking fields for Integer values

You may need to compare the value of a field against integers. This allows you to use any of Python’s built-in comparisons against your events.
For example, you can create an alert based on HTTP response status codes:
# returns True if 'status_code' equals 404
def rule(event):
if event.get("status_code"):
return event.get("status_code") == 404
else:
return False
# returns True if 'status_code' greater than 400
def rule(event):
if event.get("status_code"):
return event.get("status_code") > 404
else:
return False
Reference:

Using the Universal Data Model

Data Models provide a way to configure a set of unified fields across all log types. By default, Panther comes with built-in Data Models for several log types. Custom Data Models can be added in the Panther Console or via the Panther Analysis Tool.
event.udm() can only be used with log types that have an existing Data Model in your Panther environment.
Example:
import panther_event_type_helpers as event_type
def rule(event):
# filter events on unified data model field ‘event_type’
return event.udm("event_type") == event_type.FAILED_LOGIN
References:

Using multiple conditions

The and keyword is a logical operator and is used to combine conditional statements. It is often required to match multiple fields in an event using the and keyword. When using and, all statements must be true: “string_a” == “this”`` and ``string_b” == “that”
Example:
To track down successful root user access to the AWS console you need to look at several fields:
from panther_base_helpers import deep_get
def rule(event):
return (event.get("eventName") == "ConsoleLogin" and
deep_get(event, "userIdentity", "type") == "Root" and
deep_get(event, "responseElements", "ConsoleLogin") == "Success")
The or keyword is a logical operator and is used to combine conditional statements. When using or, either of the statements may be true: “string_a” == “this”`` or ``string_b” == “that”
Example:
This example detects if the field contains either Port 80 or Port 22:
# returns True if 'port_number' is 80 or 22
def rule(event):
return event.get("port_number") == 80 or event.get("port_number") == 22

Searching values in lists

Comparing and matching events against a list of IP addresses, domains, users etc. is very quick and easy in Python. This is often used in conjunction with choosing not to alert on an event if the field being checked also exists in the list. This helps with reducing false positives for known behavior in your environment.
Example: If you have a list of IP addresses that you would like to add to your allow list, but you want to know if an IP address comes through outside of that list, we recommend using a Python set. Sets are similar to Python lists and tuples, but are more memory efficient.
# Set - Recommended over tuples or lists for performance
ALLOW_IP = {'192.0.0.1', '192.0.0.2', '192.0.0.3'}
def rule(event):
return event.get("ip_address") not in ALLOW_IP
In the example below, we use the Panther helper pattern_match_list:
from panther_base_helpers import pattern_match_list
USER_CREATE_PATTERNS = [
"chage", # user password expiry
"passwd", # change passwords for users
"user*", # create, modify, and delete users
]
def rule(event):
# Filter the events
if event.get("event") != "session.command":
return False
# Check that the program matches our list above
return pattern_match_list(event.get("program", ""), USER_CREATE_PATTERNS)

Matching events with regex

If you want to match against events using regular expressions - to match subdomains, file paths, or a prefix/suffix of a general string - you can use regex. In Python, regex can be used by importing the re library and looking for a matching value.
In the example below, the regex pattern will match Administrator or administrator against the nested value of the privilegeGranted field.
import re
from panther_base_helpers import deep_get
#The regex pattern is stored in a variable
# Note: This is better performance than putting it in the rule function, which is evaluated on each event
ADMIN_PATTERN = re.compile(r"[aA]dministrator")
def rule(event):
# using the deep_get function we can pull out the nested value under the "privilegeGranted" field
value_to_search = deep_get(event, "debugContext", "debugData", "privilegeGranted")
# finally we use the regex object we created earlier to check against our value
# if there is a match, "True" is returned
return (bool(ADMIN_PATTERN.search(value_to_search, default="")))
In the example below, we use the Panther helper pattern_match:
from panther_base_helpers import pattern_match
def rule(event):
return pattern_match(event.get("operation", ""), "REST.*.OBJECT")
References:

Panther Detection functions and features

Detection alerting functions

Panther's detection auxiliary functions are Python functions that control analysis logic, generated alert title, event grouping, routing of alerts, and metadata overrides. Rules are customizable and can import from standard Python libraries or global helpers.
Applicable to both Rules and Policies, each function listed takes a single argument of event (Rules) or resource (Policies). Advanced users may define functions, variables, or classes outside of the functions defined below.
The only required function is def rule(event), but other functions make your Alerts more dynamic.
Detection Alerting Function Name
Description
Return Value
Default Return Value
title
The generated alert title
String
If not defined, the Display Name, RuleID, or PolicyID is used
dedup
The string to group related events with, limited to 1000 characters
String
If not defined, the titlefunction output is used.
alert_context
Additional context to pass to the alert destination(s)
Dict[String: Any]
An empty Dict
severity
The level of urgency of the alert
INFO, LOW, MEDIUM, HIGH, or CRITICAL
The severity as defined in the detection metadata
description
An explanation about why the rule exists
String
The description as defined in the detection metadata
reference
A reference URL to an internal document or online resource about the rule
String
The reference as defined in the detection metadata
runbook
A list of instructions to follow once the alert is generated
String
The runbook as defined in the detection metadata
destinations
The label or ID of the destinations to specifically send alerts to. An empty list will suppress all alerts.
List[Destination Name/ID]
The destinations as defined in the detection metadata

Configuring Detection functions dynamically

title

The title function is optional, but it is recommended to include it to provide additional context. In the example below, the log type, relevant username, and a static string are returned to the destination. The function checks to see if the event is related the AWS.CloudTrail log type and return the AWS Account Name if that is true.
If the dedup function is not present, the title is used to group related events for deduplication purposes.
Example:
def title(event):
# use unified data model field in title
log_type = event.get("p_log_type")
title_str = (
f"{log_type}: User [{event.udm('actor_user')}] has exceeded the failed logins threshold"
)
if log_type == "AWS.CloudTrail":
title_str += f" in [{lookup_aws_account_name(event.get('recipientAccountId'))}]"
return title_str
Reference: Template Rule

dedup

Deduplication is the process of grouping suspicious events together into a single alert to prevent receiving duplicate alerts for the same behavior that may have multiple indicators. Any event that triggers a detection is grouped together with other events that triggered the same detection and subsequent deduplication string within the designated deduplication period. It returns a string used to group related events. It is limited to 1000 characters. If this function is not present, the title string will be used to group events.
Example:
def dedup(event):
user_identity = event.get("userIdentity", {})
if user_identity.get("type") == "AssumedRole":
return helper_strip_role_session_id(user_identity.get("arn", ""))
return user_identity.get("arn")

severity

In some scenarios, you may need to upgrade or downgrade the severity level of an alert. In the example below, a HIGH severity alert is returned if an API token is created - otherwise we create an INFO level alert. The severity levels of an Alert can be mapped to INFO, LOW, MEDIUM, HIGH, or CRITICAL.
Example:
def severity(event):
if event.get('eventType') == 'system.api_token.create':
return "HIGH"
return "INFO"
Reference: Template Rule

destinations

By default, Alerts are sent to specific destinations based on severity level or log type event. Each Detection has the ability to override their default destination and send the Alert to one or more specific destination(s). In some scenarios, a destination override is required, providing more advance criteria based on the logic of the Rule.
Example:
A rule used for multiple log types utilizes the destinations function to reroute the Alert to another destination if the log type is "AWS.CloudTrail". The Alert is suppressed to this destination using return [] if the log type is not CloudTrail.
def destinations(event):
if event.get("p_log_type") == "AWS.CloudTrail":
return ["slack-security-alerts"] ### Name or UUID of destination
# Do not send alert to an external destination
return []
Reference: Template Rule

alert_context

This function allows the detection to pass any event details as additional context, such as usernames, IP addresses, or success/failure, to the Alert destination(s).
Example:
The code below returns all event data in the alert context.
def rule(event):
return (
event.get("actionName") == "UPDATE_SAML_SETTINGS"
and event.get("actionResult") == "SUCCEEDED"
)
def alert_context(event):
return {
"user": event.udm("actor_user"),
"ip": event.udm("source_ip")
}

alert runbook, reference, and description

These functions can provide additional context around why an alert was triggered and how to resolve the related issue. Depending on what conditions are met, a string can be overridden and returned to the specified field in the alert.
The example below dynamically provides a link within the runbook field in an alert.
def runbook(event):
log_type = event.get("p_log_type")
if log_type == "OnePassword.SignInAttempt":
return: f"<https://link/to/resource>"
elif log_type == "Okta.SystemLog":
return: f"<https://link/to/resource/2>"
else:
return: f"<https://default/link>"

Detection features

Detection Packs
Data Models
Global Helper Functions
Caching
Data Replay
Testing
Report Mapping
Alert Summaries
Triaging Alerts

Troubleshooting Detections

Visit the Panther Knowledge Base to view articles about detections that answer frequently asked questions and help you resolve common errors and issues.
Copy link
On this page
Overview
Getting started with writing detections
Choose a method for writing and managing detections
How to write detections
Working with your data in Panther
Panther Detection examples and best practices
Python best practices
Understanding the structure of a Detection in Panther
Detection writing best practices
Panther Detection functions and features
Detection alerting functions
Configuring Detection functions dynamically
Detection features
Troubleshooting Detections