> For the complete documentation index, see [llms.txt](https://docs.panther.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.panther.com/~/changes/Dd8nx2iqd1Pp2OzWJaWk/detections/rules/python.md). # Writing Python Detections Construct Python detections in the Console or CLI workflow ## Overview You can write your own Python detections in the Panther Console or locally, following the [CLI workflow](/~/changes/Dd8nx2iqd1Pp2OzWJaWk/panther-developer-workflows/ci-cd.md). When writing Python detections, try to follow [these best practices](#python-detection-writing-best-practices), and remember that [certain alert fields can be set dynamically](#alert-functions-in-python-detections). You can alternatively use the [no-code detection builder](/~/changes/Dd8nx2iqd1Pp2OzWJaWk/detections/rules/simple-detection-builder.md) in the Console to create rules, or write them locally in [YAML](/~/changes/Dd8nx2iqd1Pp2OzWJaWk/detections/rules/yaml.md). If you aren't sure whether to write detections locally in YAML or Python, see the [Using Python vs. YAML](https://docs.panther.com/~/changes/Dd8nx2iqd1Pp2OzWJaWk/detections/rules/pages/-MXJ6kW1VEatIf8lr3tM#using-python-vs.-yaml) section. {% hint style="info" %} Before you write a new Python detection, see if there's a [Panther-managed detection](/~/changes/Dd8nx2iqd1Pp2OzWJaWk/detections/panther-managed.md) that meets your needs (or *almost* meets your needs—Panther-managed rules can be tuned with [Inline Filters](/~/changes/Dd8nx2iqd1Pp2OzWJaWk/detections/rules/inline-filters.md)). Leveraging a Panther-managed detection not only saves you from the effort of writing one yourself, but also provides the ongoing benefit of continuous updates to core detection logic, as Panther releases new versions. {% endhint %} ## How to create detections in Python ### How to create a rule in Python You can write a Python rule in both the Panther Console and CLI workflow.

Creating a rule in Python in the Console

1. In the left-hand navigation bar of your Panther Console, click **Build** > **Detections**. 2. Click **Create New**. 3. On the **New Detection** page, select **Rule** for the detection type. 4. In the **Basic Info** section, provide values for the following fields: * **Name**: Enter a descriptive name for the rule. * **ID** (optional)**:** Click the pen icon and enter a unique ID for your rule. 5. In the upper-right corner, click **Continue**. 6. On the next page, configure your rule: * In the upper-right corner, the **Enabled** toggle will be set to `ON` by default. If you'd like to disable the rule, flip the toggle to `OFF`. * In the **For the Following Source** section: * **Log Types**: Select the log types this rule should apply to. * In the **Detect** section: * In the **Rule Function** text editor, write a Python `rule` function to define your detection. * For detection templates and examples, see the [panther\_analysis GitHub repository](https://github.com/panther-labs/panther-analysis/tree/master/templates). * In the **Set Alert Fields** section: * **Severity**: Select a [severity level](#alert-severity) for the alerts triggered by this detection. * In the **Optional Fields** section, optionally provide values for the following fields: * **Description**: Enter additional context about the rule. * **Runbook**: Enter the procedures and operations relating to this rule. * To see examples of runbooks for built-in rules, see [Alert Runbooks](/~/changes/Dd8nx2iqd1Pp2OzWJaWk/alerts/alert-runbooks.md). * **Reference**: Enter an external link to more information relating to this rule. * **Destination Overrides:** Choose destinations to receive alerts for this detection, regardless of severity. Note that destinations can also be set dynamically, in the rule function. See [Routing Order Precedence](/~/changes/Dd8nx2iqd1Pp2OzWJaWk/alerts/destinations.md#routing-order-precedence) to learn more about routing precedence. * **Deduplication Period** and **Events Threshold**: Enter the deduplication period and threshold for rule matches. To learn how deduplication works, see [Deduplication](#deduplication). * **Summary Attributes**: Enter the attributes you want to showcase in the alerts that are triggered by this detection. * To use a nested field as a summary attribute, use the Snowflake dot notation in the Summary Attribute field to traverse a path in a JSON object: `:..` The alert summary will then be generated for the referenced object in the alert. [Learn more about traversing semi-structured data in Snowflake here.](https://docs.snowflake.com/en/user-guide/querying-semistructured.html#label-traversing-semistructured-data) * For more information on Alert Summaries, see [Assigning and Managing Alerts](/~/changes/Dd8nx2iqd1Pp2OzWJaWk/alerts/alert-management.md). * **Custom Tags**: Enter custom tags to help you understand the rule at a glance (e.g., `HIPAA`.) * In the **Framework Mapping** section: 1. Click **Add New** to enter a report. 2. Provide values for the following fields: * **Report Key**: Enter a key relevant to your report. * **Report Values**: Enter values for that report. * In the **Test** section: * In the **Unit Test** section, click **Add New** to [create a test](/~/changes/Dd8nx2iqd1Pp2OzWJaWk/detections/testing.md) for the rule you defined in the previous step. 7. In the upper-right corner, click **Save**. After you have created a rule, you can modify it using [Inline Filters](/~/changes/Dd8nx2iqd1Pp2OzWJaWk/detections/rules/inline-filters.md).

Creating a rule in Python in the CLI workflow

If you're writing detections locally (instead of in the Panther Console), we recommend managing your local detection files in a version control system like GitHub or GitLab. We advise that you start your custom detection content by creating either [a public fork](https://docs.panther.com/panther-developer-workflows/ci-cd/detections-repo/public-fork) or a [private cloned repo](https://docs.panther.com/panther-developer-workflows/ci-cd/detections-repo/private-cloned-repo) from Panther's [open-source panther-analysis repository](https://github.com/panther-labs/panther-analysis). #### Folder setup If you group your rules into folders, each folder name must contain `rules` in order for them to be found during upload (using either PAT or the bulk uploader in the Console). We recommend grouping rules into folders based on log/resource type, e.g., `suricata_rules` or `aws_s3_policies`. You can use the [panther-analysis](https://github.com/panther-labs/panther-analysis) repo as a reference. #### File setup Each rule and scheduled rule consists of: * A Python file (a file with a `.py` extension) containing your detection logic. * A YAML specification file (a file with a `.yml` extension) containing metadata attributes of the detection. * By convention, we give this file the same name as the Python file. Rules are Python functions to detect suspicious behaviors. Returning a value of `True` indicates suspicious activity, which triggers an alert. 1. Write your rule and save it (in your folder of choice) as `my_new_rule.py`: ```python def rule(event): return 'prod' in event.get('hostName') ``` 2. Create a metadata file using the template below: ```yaml AnalysisType: rule DedupPeriodMinutes: 60 # 1 hour DisplayName: Example Rule to Check the Format of the Spec Enabled: true Filename: my_new_rule.py RuleID: Type.Behavior.MoreContext Severity: High LogTypes: - LogType.GoesHere Reports: ReportName (like CIS, MITRE ATT&CK): - The specific report section relevant to this rule Tags: - Tags - Go - Here Description: > This rule exists to validate the CLI workflows of the Panther CLI Runbook: > First, find out who wrote this the spec format, then notify them with feedback. Reference: https://www.a-clickable-link-to-more-info.com ``` When this rule is uploaded, each of the fields you would normally populate in the Panther Console will be auto-filled. See [Rule specification reference](#python-rule-specification-reference) for a complete list of required and optional fields.

### How to create a scheduled rule in Python You can write a Python scheduled rule in both the Panther Console and CLI workflow.

Creating a scheduled rule in Python in the Console

1. In the left-hand navigation bar of your Panther Console, click **Build** > **Detections**. 2. Click **Create New**. 3. On the **New Detection** page, select **Scheduled Rule** for the detection type. 4. In the **Basic Info** section, provide values for the following fields: * **Name**: Enter a descriptive name for the scheduled rule. * **ID** (optional)**:** Click the pen icon and enter a unique ID for your scheduled rule. 5. In the upper-right corner, click **Continue**. 6. On the next page, configure your scheduled rule: * In the upper-right corner, the **Enabled** toggle will be set to `ON` by default. If you'd like to disable the scheduled rule, flip the toggle to `OFF`. * In the **For the Following Scheduled Queries** section: * **Scheduled Queries**: Select one or more [Scheduled Searches](/~/changes/Dd8nx2iqd1Pp2OzWJaWk/search/scheduled-searches.md) this scheduled rule should apply to. * In the **Detect** section: * In the **Rule Function** text editor, write a Python `rule` function to define your detection. * If all your filtering logic is already taken care of in the SQL of the associated [scheduled query](/~/changes/Dd8nx2iqd1Pp2OzWJaWk/search/scheduled-searches.md), you can configure the `rule` function to simply return `true` for each row: ```python def rule(event): return True ``` * For detection templates and examples, see the [panther\_analysis GitHub repository](https://github.com/panther-labs/panther-analysis/tree/master/templates) * In the **Set Alert Fields** section: * **Severity**: Select a [severity level](#alert-severity) for the alerts triggered by this detection. * In the **Optional Fields** section, optionally provide values for the following fields: * **Description**: Enter additional context about the rule. * **Runbook**: Enter the procedures and operations relating to this rule. * To see examples of runbooks for built-in rules, see [Alert Runbooks](/~/changes/Dd8nx2iqd1Pp2OzWJaWk/alerts/alert-runbooks.md). * **Reference**: Enter an external link to more information relating to this rule. * **Destination Overrides:** Choose destinations to receive alerts for this detection, regardless of severity. Note that destinations can also be set dynamically, in the rule function. See [Routing Order Precedence](/~/changes/Dd8nx2iqd1Pp2OzWJaWk/alerts/destinations.md#routing-order-precedence) to learn more about routing precedence. * **Deduplication Period** and **Events Threshold**: Enter the deduplication period and threshold for rule matches. To learn how deduplication works, see [Deduplication](#deduplication). * **Summary Attributes**: Enter the attributes you want to showcase in the alerts that are triggered by this detection. * To use a nested field as a summary attribute, use the Snowflake dot notation in the Summary Attribute field to traverse a path in a JSON object: `:..` The alert summary will then be generated for the referenced object in the alert. [Learn more about traversing semi-structured data in Snowflake here.](https://docs.snowflake.com/en/user-guide/querying-semistructured.html#label-traversing-semistructured-data) * For more information on Alert Summaries, see [Assigning and Managing Alerts](/~/changes/Dd8nx2iqd1Pp2OzWJaWk/alerts/alert-management.md). * **Custom Tags**: Enter custom tags to help you understand the rule at a glance (e.g., `HIPAA`.) * In the **Framework Mapping** section: 1. Click **Add New** to enter a report. 2. Provide values for the following fields: * **Report Key**: Enter a key relevant to your report. * **Report Values**: Enter values for that report. * In the **Test** section: * In the **Unit Test** section, click **Add New** to [create a test](/~/changes/Dd8nx2iqd1Pp2OzWJaWk/detections/testing.md) for the rule you defined in the previous step. 7. In the upper-right corner, click **Save**. * Once you've clicked **Save**, the scheduled rule will become active. The SQL returned from the associated [scheduled query](/~/changes/Dd8nx2iqd1Pp2OzWJaWk/search/scheduled-searches.md) (at the interval defined in the query) will be run through the scheduled rule (if, that is, any rows are returned). After you have created a rule, you can modify it using [Inline Filters](/~/changes/Dd8nx2iqd1Pp2OzWJaWk/detections/rules/inline-filters.md).

Creating a scheduled rule in Python in the CLI workflow

If you're writing detections locally (instead of in the Panther Console), we recommend managing your local detection files in a version control system like GitHub or GitLab. We advise that you start your custom detection content by creating either [a public fork](https://docs.panther.com/panther-developer-workflows/ci-cd/detections-repo/public-fork) or a [private cloned repo](https://docs.panther.com/panther-developer-workflows/ci-cd/detections-repo/private-cloned-repo) from Panther's [open-source panther-analysis repository](https://github.com/panther-labs/panther-analysis). #### Folder setup If you group your rules into folders, each folder name must contain the string `rules` in order for them to be found during upload (using either PAT or the bulk uploader in the Console). We recommend grouping rules into folders based on log/resource type, e.g., `suricata_rules` or `aws_s3_policies`. You can use the [panther-analysis](https://github.com/panther-labs/panther-analysis) repo as a reference. #### File setup Each scheduled rule consists of: * A Python file (a file with a `.py` extension) containing your detection logic. * A YAML specification file (a file with a `.yml` extension) containing metadata attributes of the detection. * By convention, we give this file the same name as the Python file. Scheduled rules allow you to analyze the output of a [scheduled search](/~/changes/Dd8nx2iqd1Pp2OzWJaWk/search/scheduled-searches.md) with Python. Returning a value of `True` indicates suspicious activity, which triggers an alert. 1. Write your query and save it as `my_new_scheduled_query.yml`: ```yaml AnalysisType: scheduled_query QueryName: My New Scheduled Query Name Enabled: true Tags: - Optional - Tags Description: > An optional Description Query: 'SELECT * FROM panther_logs.aws_cloudtrail LIMIT 10' SnowflakeQuery: 'SELECT * FROM panther_logs.public.aws_cloudtrail LIMIT 10' AthenaQuery: 'SELECT * FROM panther_logs.aws_cloudtrail LIMIT 10' Schedule: # Note: CronExpression and RateMinutes are mutually exclusive, only # configure one or the other CronExpression: '0 * * * *' RateMinutes: 1 TimeoutMinutes: 1 ``` 2. Write your rule and save it as `my_new_rule.py`: ```python # Note: See an example rule for more options # https://github.com/panther-labs/panther-analysis/blob/master/templates/example_rule.py def rule(_): # Note: You may add additional logic here return True ``` 3. Create a metadata file and save it as `my_new_schedule_rule.yml`:

AnalysisType: scheduled_rule
   Filename: my_new_rule.py 
   RuleID: My.New.Rule
   DisplayName: A More Friendly Name
   Enabled: true
   ScheduledQueries:
     - My New Scheduled Query Name
   Tags:
     - Tag
   Severity: Medium
   Description: >
     An optional Description
   Runbook: >
     An optional Runbook 
   Reference: An optional reference.link 
   Tests:
     -
       Name: Name 
       ExpectedResult: true
       Log:
         {
           "JSON": "string"
         }

When this scheduled rule is uploaded, each of the files will connect a scheduled query with a rule, and fill in the fields you would normally populate in the Panther Console will be auto-filled. See [Rule specification reference below](#python-rule-specification-reference) for a complete list of required and optional fields.

### How to create a policy in Python * To learn how to create a policy, see the [How to write a policy instructions on Policies](/~/changes/Dd8nx2iqd1Pp2OzWJaWk/detections/policies.md#how-to-write-a-policy). ## Python detection syntax A local Python detection is made up of two files: a Python file and a YAML file. When a Python detection is created in the Panther Console, there is only a Python text editor (not a YAML one). The keys listed in the YAML column, below, are set in fields in the user interface.

The Python file can contain: The YAML file can contain:

The Python file can contain:	The YAML file can contain:
Detection logic `def rule(event): # or def policy(resource):` Alert functions (dynamic) `def severity(event): def title(event): def dedup(event): def destinations(event): def runbook(event): def reference(event): def description(event): def alert_context(event):`	Filter key `InlineFilters:` Metadata keys `AnalysisType: # rule, scheduled_rule, or policy Enabled: FileName: RuleID: # or PolicyId: LogTypes: Reports: Tags: Tests: ScheduledQueries: # only applicable to scheduled rules Suppressions: # only applicable to policies` Alert keys (static) `Severity: Description: DedupPeriodMinutes: Threshold: DisplayName: OutputIds: Reference: Runbook: SummaryAttributes:`

Detection logic

def rule(event): # or def policy(resource):

Alert functions (dynamic)

def severity(event):
def title(event):
def dedup(event):
def destinations(event):
def runbook(event):
def reference(event):
def description(event):
def alert_context(event):

Filter key
```
InlineFilters: 
```

Metadata keys

AnalysisType: # rule, scheduled_rule, or policy
Enabled: 
FileName: 
RuleID: # or PolicyId:
LogTypes: 
Reports: 
Tags: 
Tests: 
ScheduledQueries: # only applicable to scheduled rules
Suppressions: # only applicable to policies

Alert keys (static)

Severity:
Description:
DedupPeriodMinutes:
Threshold: 
DisplayName:
OutputIds:
Reference:
Runbook:
SummaryAttributes:

### **Basic Python rule structure** Only a `rule()` function and the YAML keys shown below are required for a Python rule. Additional Python alert functions, however, can make your alerts more dynamic. Additional YAML keys are available, too—see [Python rule specification reference](#python-rule-specification-reference).

rule.py rule.yml

rule.py	rule.yml
`def rule(event): if event.get("Something"): return True return False`	`AnalysisType: rule Enabled: true Filename: rule.py RuleID: my.rule LogTypes: - Some.Schema Severity: INFO`

def rule(event): 
    if event.get("Something"): 
        return True 
    return False

AnalysisType: rule
Enabled: true
Filename: rule.py
RuleID: my.rule
LogTypes: 
    - Some.Schema
Severity: INFO

For more templates, see the [panther-analysis repo on GitHub](https://github.com/panther-labs/panther-analysis/tree/master/templates). ### `InlineFilters` Learn more about using `InlineFilters` in Python rules on [Modifying Detections with Inline Filters](/~/changes/Dd8nx2iqd1Pp2OzWJaWk/detections/rules/inline-filters.md#creating-filters-in-the-developer-workflow). ### Alert functions in Python detections Panther's detection auxiliary functions are Python functions that control analysis logic, generated alert title, event grouping, routing of alerts, and metadata overrides. Rules are customizable and can import from standard Python libraries or [global helpers](/~/changes/Dd8nx2iqd1Pp2OzWJaWk/detections/rules/python/globals.md). Applicable to both rules and policies, each function below takes a single argument of `event` (rules) or `resource` (policies). Advanced users may define functions, variables, or classes outside of the functions defined below. Each of the below alert functions are optional, but can add dynamic context to your alerts.

Detection alert function name	Description	Overrides	Return Value
`severity`	The level of urgency of the alert	In YAML: `Severity` key In Console: Severity field	`INFO, LOW, MEDIUM, HIGH, or CRITICAL`
`title`	The generated alert title	In YAML: `DisplayName` > `RuleID` or `PolicyID` In Console: Name field > ID field	`String`
`dedup`	The string to group related events with, limited to 1000 characters	In Python/YAML: `title()` > `DisplayName` > `RuleID` or `PolicyID` In Console: `title()` > Name field > ID field	`String`
`alert_context`	Additional context to pass to the alert destination(s)	Does not override a YAML nor Console field	`Dict[String: Any]`
`description`	An explanation about why the rule exists	In YAML: `Description` key In Console: Description field	`String`
`reference`	A reference URL to an internal document or online resource about the rule	In YAML: `Reference` key In Console: Reference field	`String`
`runbook`	A list of instructions to follow once the alert is generated	In YAML: `Runbook` key In Console: Runbook field	`String`
`destinations`	The label or ID of the destinations to specifically send alerts to. An empty list will suppress all alerts.	In YAML: `OutputIds` key In Console: Destination Overrides field	`List[Destination Name/ID]`

#### `severity` In some scenarios, you may need to upgrade or downgrade the severity level of an alert. The severity levels of an alert can be mapped to INFO, LOW, MEDIUM, HIGH, CRITICAL, or DEFAULT. Return DEFAULT to fall back to the statically defined rule severity. In all cases, the severity string returned is case insensitive, meaning you can return, for example, `Critical` or `default`, depending on your style preferences. Example where a HIGH severity alert is returned if an API token is created - otherwise we create an INFO level alert: ```python def severity(event): if event.get('eventType') == 'system.api_token.create': return "HIGH" return "INFO" ``` Reference: [Template Rule](https://github.com/panther-labs/panther-analysis/blob/master/templates/example_rule.py#L33) Example using `DEFAULT`: ```python def severity(event): if event.get('eventType') == 'system.api_token.create': return "HIGH" return "DEFAULT" ``` #### `title` The title function is optional, but it is recommended to include it to provide additional context. In the example below, the log type, relevant username, and a static string are returned to the destination. The function checks to see if the event is related the AWS.CloudTrail log type and return the AWS Account Name if that is true. Learn more about how an alert title is set on [Rules and Scheduled Rules](/~/changes/Dd8nx2iqd1Pp2OzWJaWk/detections/rules.md#title-of-associated-alerts). Example: ```python def title(event): # use unified data model field in title log_type = event.get("p_log_type") title_str = ( f"{log_type}: User [{event.udm('actor_user')}] has exceeded the failed logins threshold" ) if log_type == "AWS.CloudTrail": title_str += f" in [{lookup_aws_account_name(event.get('recipientAccountId'))}]" return title_str ``` Reference: [Template Rule](https://github.com/panther-labs/panther-analysis/blob/master/templates/example_rule.py#L15) #### `dedup` Deduplication is the process of grouping related events into a single alert to prevent receiving duplicate alerts. Events triggering the same detection that also share a deduplication string, within the deduplication period, are grouped together in a single alert. The `dedup` function is one way to define a deduplication string. It is limited to 1000 characters. Learn more about deduplication on [Rules and Scheduled Rules](/~/changes/Dd8nx2iqd1Pp2OzWJaWk/detections/rules.md#deduplication). Example: ```python def dedup(event): user_identity = event.get("userIdentity", {}) if user_identity.get("type") == "AssumedRole": return helper_strip_role_session_id(user_identity.get("arn", "")) return user_identity.get("arn") ``` Reference: [AWS S3 Bucket Deleted Rule](https://github.com/panther-labs/panther-analysis/blob/4b6b79846fb4cb1596908fd31ce983c75a39baaa/aws_cloudtrail_rules/aws_s3_bucket_deleted.py) #### **`destinations`** By default, Alerts are sent to specific destinations based on severity level or log type event. Each Detection has the ability to override their default destination and send the Alert to one or more specific destination(s). In some scenarios, a destination override is required, providing more advance criteria based on the logic of the Rule. Example: A rule used for multiple log types utilizes the destinations function to reroute the Alert to another destination if the log type is "AWS.CloudTrail". The Alert is suppressed to this destination using `return ["SKIP"]` if the log type is not CloudTrail. ```python def destinations(event): if event.get("p_log_type") == "AWS.CloudTrail": return ["slack-security-alerts"] ### Name or UUID of destination # Do not send alert to an external destination return ["SKIP"] ``` Reference: [Template Rule](https://github.com/panther-labs/panther-analysis/blob/master/templates/example_rule.py#L59) #### `alert_context` This function allows the detection to pass any event details as additional context, such as usernames, IP addresses, or success/failure, to the Alert destination(s). Example: The code below returns all event data in the alert context. ```python def rule(event): return ( event.get("actionName") == "UPDATE_SAML_SETTINGS" and event.get("actionResult") == "SUCCEEDED" ) def alert_context(event): return { "user": event.udm("actor_user"), "ip": event.udm("source_ip") } ``` #### `runbook`, `reference`, and `description` These functions can provide additional context around why an alert was triggered and how to resolve the related issue. Depending on what conditions are met, a string can be overridden and returned to the specified field in the alert. The example below dynamically provides a link within the `runbook` field in an alert. ```python def runbook(event): log_type = event.get("p_log_type") if log_type == "OnePassword.SignInAttempt": return: f"" elif log_type == "Okta.SystemLog": return: f"" else: return: f"" ``` ## Python best practices Python Enhancement Proposals [publishes resources](https://peps.python.org/pep-0008/) on how to cleanly and effectively write and style your Python code. For example, you can use [autopep8](https://pypi.org/project/autopep8/) to automatically ensure that your written detections all follow a consistent style. ### Available Python libraries The following Python libraries are available to be used in Panther in addition to `boto3`, provided by [AWS Lambda](https://docs.aws.amazon.com/lambda/latest/dg/lambda-runtimes.html):

Package	Version	Description	License
Package	Version	Description	License
`jsonpath-ng`	`1.5.2`	JSONPath Implementation	Apache v2
`policyuniverse`	`1.3.3.20210223`	Parse AWS ARNs and Policies	Apache v2
`requests`	`2.23.0`	Easy HTTP Requests	Apache v2

## Python detection writing best practices ### Writing tests for your detections Before enabling new detections, it is [recommended to write tests ](/~/changes/Dd8nx2iqd1Pp2OzWJaWk/detections/testing.md)that define scenarios where alerts should or should not be generated. Best practice dictates at least one positive and one negative to ensure the most reliability. ### Casing for event fields Lookups for event fields are not case sensitive. `event.get("Event_Type")` or `event.get("event_type")` will return the same result. ### Understanding top level fields and nested fields Top-level fields represent the parent fields in a nested data structure. For example, a record may contain a field called `user` under which there are other fields such as `ip_address`. In this case, `user` is the top-level field, and `ip_address` is a nested field underneath it. Nesting can occur many layers deep, and so it is valuable to understand the schema structure and know how to access a given field for a detection. ### **Accessing top-level fields safely** Basic Rules match a field’s value in the event, and a best practice to avoid errors is to leverage Python’s built-in `get()` function. The example below is a best practice because it leverages a `get()` function. `get()` will look for a field, and if the field doesn't exist, it will return `None` instead of an error, which will result in the detection returning `False`. ```python def rule(event): return event.get('field') == 'value' ``` In the example below, if the field exists, the value of the field will be returned. Otherwise, `False` will be returned: ```python def rule(event): if event.get('field') return event.get('field') return False ``` {% hint style="warning" %} **Bad practice example**\ The rule definition below is bad practice because the code is explicit about the field name. If the field doesn't exist, Python will throw a `KeyError`: ```python def rule(event): return event['field'] == 'value' ``` {% endhint %} Reference: [Safely Accessing Event Fields](https://docs.panther.com/writing-detections/rules#safely-accessing-event-fields) ### Using Global Helper functions Once many detections are written, a set of patterns and repeated code will begin to emerge. This is a great use case for [Global Helper functions](/~/changes/Dd8nx2iqd1Pp2OzWJaWk/detections/rules/python/globals.md), which provide a centralized location for this logic to exist across all detections. For example, see the `deep_get()` function referenced in the next section. ### Accessing nested fields safely If the field is nested deep within the event, use a Panther-supplied function called `deep_get()` to safely access the fields value. `deep_get()` must be imported by the `panther_base_helpers` library. `deep_get()` takes two or more arguments: * The event object itself (required) * The top-level field name (required) * Any nested fields, in order (as many nested fields as needed) Example: AWS CloudTrail logs nest the type of user accessing the console underneath `userIdentity`. JSON CloudTrail root activity: ```json { "eventVersion": "1.05", "userIdentity": { "type": "Root", "principalId": "1111", "arn": "arn:aws:iam::123456789012:root", "accountId": "123456789012", "userName": "root" }, ... } ``` Here is how you could check that value safely with `deep_get`: ```python from panther_base_helpers import deep_get def rule(event): return deep_get(event, "userIdentity", "type") == "Root" ``` Reference: [AWS Console Root Login](https://github.com/panther-labs/panther-analysis/blob/cd220c87982011d4ad156c7daecd2857c358d154/rules/aws_cloudtrail_rules/aws_console_root_login.py) ### **Checking fields for specific values** You may want to know when a specific event has occurred. If it did occur, then the detection should trigger an alert. Since Panther stores everything as normalized JSON, you can check the value of a field against the criteria you specify. For example, to detect the action of granting Box technical support access to your Box account, the Python below would be used to match events where the `event_type` equals `ACCESS_GRANTED`: ```python def rule(event): return event.get("event_type") == "ACCESS_GRANTED" ``` If the field is `event_type` and the value is equal to `ACCESS_GRANTED` then the rule function will return `true` and an Alert will be created. ### Checking fields for Integer values You may need to compare the value of a field against integers. This allows you to use any of Python’s built-in comparisons against your events. For example, you can create an alert based on HTTP response status codes: ```python # returns True if 'status_code' equals 404 def rule(event): if event.get("status_code"): return event.get("status_code") == 404 else: return False # returns True if 'status_code' greater than 400 def rule(event): if event.get("status_code"): return event.get("status_code") > 404 else: return False ``` Reference: * [box\_access\_granted.py](https://github.com/panther-labs/panther-analysis/blob/cd220c87982011d4ad156c7daecd2857c358d154/rules/box_rules/box_access_granted.py) * [Python Operators](https://www.w3schools.com/python/python_operators.asp) ### **Using the Universal Data Model** [Data Models](/~/changes/Dd8nx2iqd1Pp2OzWJaWk/detections/rules/python/data-models.md) provide a way to configure a set of unified fields across all log types. By default, Panther comes with built-in Data Models for several log types. Custom Data Models can be added in the Panther Console or via the [Panther Analysis Tool](https://docs.panther.com/writing-detections/panther-analysis-tool#data-models). `event.udm()` can only be used with log types that have an existing Data Model in your Panther environment. Example: ```python import panther_event_type_helpers as event_type def rule(event): # filter events on unified data model field ‘event_type’ return event.udm("event_type") == event_type.FAILED_LOGIN ``` References: * [Data Models Guide](https://docs.panther.com/writing-detections/data-models) * [Data Models](https://github.com/panther-labs/panther-analysis/tree/master/data_models) * [Brute Force by IP](https://github.com/panther-labs/panther-analysis/blob/cd220c87982011d4ad156c7daecd2857c358d154/indexes/standard.md) ### Using multiple conditions The `and` keyword is a logical operator and is used to combine conditional statements. It is often required to match multiple fields in an event using the `and` keyword. When using `and`, all statements must be true:\ `"string_a" == "this"`**`and`**`"string_b" == "that"` Example: To track down successful root user access to the AWS console you need to look at several fields: ```python from panther_base_helpers import deep_get def rule(event): return (event.get("eventName") == "ConsoleLogin" and deep_get(event, "userIdentity", "type") == "Root" and deep_get(event, "responseElements", "ConsoleLogin") == "Success") ``` The `or` keyword is a logical operator and is used to combine conditional statements. When using `or`, either of the statements may be true:\ "`string_a" == "this"`` `**`or`**` ``"string_b" == "that"` Example: This example detects if the field contains either Port 80 **or** Port 22: ```python # returns True if 'port_number' is 80 or 22 def rule(event): return event.get("port_number") == 80 or event.get("port_number") == 22 ``` ### Searching values in lists Comparing and matching events against a list of IP addresses, domains, users etc. is very quick and easy in Python. This is often used in conjunction with choosing not to alert on an event if the field being checked also exists in the list. This helps with reducing false positives for known behavior in your environment. Example: If you have a list of IP addresses that you would like to add to your allow list, but you want to know if an IP address comes through outside of that list, we recommend using a Python set. Sets are similar to Python lists and tuples, but are more memory efficient. ```python # Set - Recommended over tuples or lists for performance ALLOW_IP = {'192.0.0.1', '192.0.0.2', '192.0.0.3'} def rule(event): return event.get("ip_address") not in ALLOW_IP ``` In the example below, we use the Panther helper `pattern_match_list`: ```python from panther_base_helpers import pattern_match_list USER_CREATE_PATTERNS = [ "chage", # user password expiry "passwd", # change passwords for users "user*", # create, modify, and delete users ] def rule(event): # Filter the events if event.get("event") != "session.command": return False # Check that the program matches our list above return pattern_match_list(event.get("program", ""), USER_CREATE_PATTERNS) ``` Reference: [Teleport Create User Accounts](https://github.com/panther-labs/panther-analysis/blob/cd220c87982011d4ad156c7daecd2857c358d154/rules/gravitational_teleport_rules/teleport_create_user_accounts.py) ### Matching events with regex If you want to match against events using regular expressions - to match subdomains, file paths, or a prefix/suffix of a general string - you can use regex. In Python, regex can be used by importing the `re` library and looking for a matching value. In the example below, the regex pattern will match Administrator or administrator against the nested value of the privilegeGranted field. ```python import re from panther_base_helpers import deep_get #The regex pattern is stored in a variable # Note: This is better performance than putting it in the rule function, which is evaluated on each event ADMIN_PATTERN = re.compile(r"[aA]dministrator") def rule(event): # using the deep_get function we can pull out the nested value under the "privilegeGranted" field value_to_search = deep_get(event, "debugContext", "debugData", "privilegeGranted") # finally we use the regex object we created earlier to check against our value # if there is a match, "True" is returned return (bool(ADMIN_PATTERN.search(value_to_search, default=""))) ``` In the example below, we use the Panther helper `pattern_match`: ```python from panther_base_helpers import pattern_match def rule(event): return pattern_match(event.get("operation", ""), "REST.*.OBJECT") ``` References: * [re.compile](https://docs.python.org/3/library/re.html#functions) * [Pythex: simple RegEx editor and tester](https://pythex.org/) * [AWS S3 Insecure Access](https://github.com/panther-labs/panther-analysis/blob/cd220c87982011d4ad156c7daecd2857c358d154/rules/aws_s3_rules/aws_s3_insecure_access.py) ## Python rule specification reference Required fields are in **bold**.

Field Name	Description	Expected Value
Field Name	Description	Expected Value
`AnalysisType`	Indicates whether this analysis is a rule, scheduled_rule, policy, or global	Rules: `rule` Scheduled Rules: `scheduled_rule`
`Enabled`	Whether this rule is enabled	Boolean
`FileName`	The path (with file extension) to the python rule body	String
`RuleID`	The unique identifier of the rule	String Cannot include `%`
`LogTypes`	The list of logs to apply this rule to	List of strings
`Severity`	What severity this rule is	One of the following strings: `Info`, `Low`, `Medium`, `High`, or `Critical`
`ScheduledQueries` (field only for Scheduled Rules)	The list of Scheduled Query names to apply this rule to	List of strings
`Description`	A brief description of the rule	String
`DedupPeriodMinutes`	The time period (in minutes) during which similar events of an alert will be grouped together	`15`,`30`,`60`,`180` (3 hours),`720` (12 hours), or `1440` (24 hours)
`DisplayName`	A friendly name to show in the UI and alerts. The `RuleID` will be displayed if this field is not set.	String
`OutputIds`	Static destination overrides. These will be used to determine how alerts from this rule are routed, taking priority over default routing based on severity.	List of strings
`Reference`	The reason this rule exists, often a link to documentation	String
`Reports`	A mapping of framework or report names to values this rule covers for that framework	Map of strings to list of strings
`Runbook`	The actions to be carried out if this rule returns an alert, often a link to documentation	String
`SummaryAttributes`	A list of fields that alerts should summarize.	List of strings
`Threshold`	How many events need to trigger this rule before an alert will be sent.	Integer
`Tags`	Tags used to categorize this rule	List of strings
`Tests`	Unit tests for this rule.	List of maps

## Python Policy Specification Reference Required fields are in **bold**. A complete list of policy specification fields:

Field Name	Description	Expected Value
Field Name	Description	Expected Value
`AnalysisType`	Indicates whether this specification is defining a policy or a rule	`policy`
`Enabled`	Whether this policy is enabled	Boolean
`FileName`	The path (with file extension) to the python policy body	String
`PolicyID`	The unique identifier of the policy	String Cannot include `%`
`ResourceTypes`	What resource types this policy will apply to	List of strings
`Severity`	What severity this policy is	One of the following strings: `Info`, `Low`, `Medium`, `High`, or `Critical`
`Description`	A brief description of the policy	String
`DisplayName`	What name to display in the UI and alerts. The `PolicyID` will be displayed if this field is not set.	String
`Reference`	The reason this policy exists, often a link to documentation	String
`Reports`	A mapping of framework or report names to values this policy covers for that framework	Map of strings to list of strings
`Runbook`	The actions to be carried out if this policy fails, often a link to documentation	String
`Suppressions`	Patterns to ignore, e.g., `aws::s3::*`	List of strings
`Tags`	Tags used to categorize this policy	List of strings
`Tests`	Unit tests for this policy.	List of maps

## Troubleshooting Detections Visit the Panther Knowledge Base to [view articles about detections](https://help.panther.com/Detections) that answer frequently asked questions and help you resolve common errors and issues. --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://docs.panther.com/~/changes/Dd8nx2iqd1Pp2OzWJaWk/detections/rules/python.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.