Derived Detections (Beta)

Create one or more Derived Detections from a single Base Detection in Panther

Overview

Detection derivation is in open beta starting with Panther version 1.93, and is available to all customers. Please share any bug reports and feature requests with your Panther support team.

You can create one or more Derived Detections from a single Base Detection in Panther. Derived Detections inherit the Base Detection's core logic, which is immutable, as well as its metadata and alert field values, which can be overwritten.

Detection derivation is available for rules created as Simple Detections or Python Detections.

Use cases for Derived Detections

Derivation can be particularly useful when:

• You maintain multiple copies of the same rule, each with different metadata

• In the CLI workflow, you use and customize Panther-managed rules, and want to avoid having to resolve merge conflicts when Panther releases updates

  • See a full example on Using Derived Detections to Avoid Merge Conflicts

• You'd like the ability to, while responding to an incident, deploy multiple variations of one detection to gather telemetry that can inform your next decision

• One member of your team (e.g., a Head of Threat Research) would like to create a set of Base Detections that others (e.g., SOC Analysts) can modify

Base Detections and Derived Detections

A Base Detection is any custom or Panther-managed rule from which a Derived Detection has been created.

A Derived Detection is created from a Base Detection and:

• Inherits the Base Detection's metadata/alert field values, but can overwrite certain fields

  • See Limitations of detection derivation for a full list of fields that can be overwritten today.

  • An override made in a Derived Detection completely replaces the field's value it inherited from the Base Detection. For example, if both a Base Detection and Derived Detection define an Inline Filter (in the InlineFilters key in the CLI workflow or Filter to only include events field in the Console), only the Derived Detection's Inline Filter will be applied.

  • Overrides can only be made in one direction, i.e., overrides made on a Derived Detection do not affect Base Detection values. For example, if a Base Detection has Enabled: False and its Derived Detection has Enabled: True, only the Derived Detection will be enabled.

• Inherits the core detection logic of the Base Detection

  • The core detection logic is defined in the Base Detection's rule() function (for Python detections) or Detection field (for Simple Detections).

  • A Derived Detection cannot specify its own detection logic—if in the CLI workflow a Derived Detection includes a Detection key, for example, its contents will be ignored.

What happens when a Base Detection is updated

When the core logic of a Base Detection is updated, the change is propagated to all associated Derived Detections.

When the metadata of a Base Detection is updated, if an associated Derived Detection has already overwritten the value(s) of the updated field(s), there is no change. If an associated Derived Detection has not overwritten the value(s) of the updated field(s), the metadata update is propagated to the Derived Detection.

Disable Base Detections to avoid duplicate alerts

In most cases, Base and Derived Detections are run over the same set of incoming logs (although it is possible to use Inline Filters to target different events). In this scenario, because the detections share core logic, if they are both enabled, they will generate duplicate alerts.

To avoid this, disable the Base Detection. When a disabled Base Detection is updated, its changes will still propagate to its Derived Detections as described above.

Automatically disabling Base Detections in the CLI workflow

In the CLI workflow, there are two ways that you can automatically disable Base Detections:

• Option 1 (Recommended): Add the following setting to your .panther_settings.yml file:

  auto_disable_base: true
  # ... other settings ...

• Option 2: Use --auto-disable-base with the Panther Analysis Tool upload command.

  • When following this option, note that --auto-disable-base must be used with every subsequent upload invocation. If it is omitted, Base Detections will be re-enabled.

If one or more of your Base Detections has already been uploaded to your Panther instance in an enabled state and you then use one of the above methods to automatically disable Base Detections, ensure you are not including --filter enabled: true on your PAT upload command. If you do, Base Detections will be disabled before upload (when the enabled: true filter is applied), meaning the newly disabled Base Detections won't be re-uploaded to your Panther instance (leaving them as-is, or enabled).

Limitations of detection derivation

• Derivation is not available for Scheduled Rules or Policies.

• Only one level of derivation is possible, i.e. a Derived Detection cannot be derived from.

• In the Console workflow, tests are inherited when the Derived Detection is created, but not thereafter when the Base Detection's tests are updated.

• If, in a Python Base Detection, the value of a metadata field is set using a Python function, that value will take precedence over an equivalent static override value supplied in a Derived Detection. For example, if a Python severity() function is present in the Base Detection, its value will take precedence over the Derived Detection's override value supplied in the Severity YAML key (in the CLI workflow) or the Severity field (in the Console).

  • See a full list of the Python functions that set metadata values (called "Alert functions"), as well as which fields in YAML/the Console they override, in the Alert functions in Python detections table.

  • It is possible to overwrite values set by certain Python alert functions by using dynamic alert keys in your Derived Detection. For example, you can override the value of your Python Base Detection's severity() function in your Derived Detection by using the DynamicSeverities field.

    • DynamicSeverities overrides severity()

    • AlertTitle overrides title()

    • AlertContext overrides alert_context()

    • GroupBy overrides dedup()

• If you are creating a Derived Detection in the Console workflow and the Base Detection is a Python detection, you cannot set any alert fields dynamically—they may only be set statically.

  • It is possible to dynamically set alert fields in the Console workflow if the Base Detection is a Simple detection.

  • It is possible to dynamically set alert fields in the CLI workflow (using AlertTitle, DynamicSeverities, AlertContext, and GroupBy) regardless of whether the Base Detection is a Python or YAML detection.

• Currently, only the below fields can be overwritten. These are YAML field names, applicable to the CLI workflow—for those with equivalent fields in the Console, those Console fields may also be overwritten.

  • Enabled

  • Severity

  • Description

  • CreateAlert

  • DedupPeriodMinutes

  • InlineFilters

  • DisplayName

  • OnlyUseBaseRiskScore

  • OutputIds

  • Reference

  • Runbook

  • SummaryAttributes

  • Threshold

  • Tags

  • Reports

  • DynamicSeverities

  • AlertTitle

  • AlertContext

  • GroupBy

  • Tests

How to create a Derived Detection

Panther Console

Creating a Derived Detection in the Panther Console

1. In the left-hand navigation bar of the Panther Console, click Detections.

2. Locate the detection you would like to become the Base Detection for a new Derived Detection, and click its name.

3. In the upper-right corner, click ....

4. Click Derive:
5. On the Basic Info page, optionally edit the Name and ID fields for the Derived Detection.

   • Ensure the name is distinguishable from the Base Detection's name.

6. Click Continue.

7. Scroll down to the Filter and Set Alert Fields sections, and set desired overrides to Inline Filters and alert fields:
8. In the upper-right corner, click Deploy.

   • The detection will have a DERIVED label:

CLI workflow

Creating a Derived Detection in the CLI workflow

1. In the directory where you use the Panther Analysis Tool (PAT), create a new YAML file for your Derived Detection.

2. In this YAML file, add the following fields, which are required for Derived Detections:

   • BaseDetection: Provide the rule ID of any rule currently present in your Panther instance or in the local directory you will upload using PAT.

     • The BaseDetection key is the link between this Derived Detection and its Base Detection, and is what indicates that inheritance should be applied.

   • RuleID: Provide a value that is different than the ID supplied earlier for the BaseDetection key.

   • AnalysisType: Indicate whether this is a rule, scheduled_rule, or policy.

     • Currently, only rules are supported.

3. Add the metadata fields you would like to override, and their new values.

   • See the Limitations section for a list of available override fields.

   • It's possible to use the dynamic alert fields outlined here.

4. Upload your detection using the panther_analysis_tool upload command.

   • It's recommended to automatically disable Base Detections.

CLI workflow example

Below is an example of a Python Base Detection (a standard detection in the panther-analysis repository), and below it, its Derived Detection.

# Base Detection
AnalysisType: rule
Description: An Auth0 User enabled MFA Policy for your organization's tenant.
DisplayName: "Auth0 MFA Policy Enabled"
Enabled: True
Filename: auth0_mfa_policy_enabled.py
Runbook: Assess if this was done by the user for a valid business reason and was expected. This alert indicates a setting change that aligns with best security practices, follow-up may be unnecessary.
Severity: Medium
DedupPeriodMinutes: 60
LogTypes:
    - Auth0.Events
RuleID: "Auth0.MFA.Policy.Enabled"
Threshold: 1
Tests:
    - ExpectedResult: True
      Log:
        data:
            client_id: 1HXWWGKk1Zj3JF8GvMrnCSirccDs4qvr
            client_name: ""
            date: "2023-05-16 17:26:16.782000000"
            description: Set the Multi-factor Authentication policies
            details:
                request:
                    auth:
                        credentials:
                            jti: 0107c849078d8d889af711840197ba7c
                            scopes:
                                - create:actions
                                - create:actions_log_sessions
                                # shortened for brevity
                        strategy: jwt
                        user:
                            email: [e-mail]
                            name: User Name
                            user_id: google-oauth2|105261262156475850461
                    body:
                        - all-applications
                    channel: https://manage.auth0.com/
                    ip: 12.12.12.12
                    method: put
                    path: /api/v2/guardian/policies
                    query: {}
                    userAgent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36
                response:
                    body:
                        - all-applications
                    statusCode: 200
            ip: 12.12.12.12
            log_id: "90020230515215719063964000000000000001223372037488829643"
            type: sapi
            user_agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36
            user_id: google-oauth2|105261262156475850461
        log_id: "90020230515215719063964000000000000001223372037488829643"
        p_any_ip_addresses:
            - 12.12.12.12
        p_any_usernames:
            - google-oauth2|105261262156475850461
        p_event_time: "2023-05-16 17:26:16.782"
        p_log_type: Auth0.Events
        p_parse_time: "2023-05-16 17:28:28.572"
        p_row_id: 2660c447622fa4c3dbb08f9918979102
        p_schema_version: 0
        p_source_id: b9031579-b2c5-45c2-b15c-632b995a4e36
        p_source_label: Org Auth0 Tenant Label
      Name: MFA Policy Enabled First

# Derived Detection

# Required fields
AnalysisType: rule
RuleID: "Auth0.MFA.Policy.Enabled.Custom.Severity"
BaseDetection: "Auth0.MFA.Policy.Enabled"

# Fields being overwritten (i.e., overriding the default values inherited from the Base Detection)
DisplayName: "Auth0 MFA Policy Enabled - Severity Critical"
Enabled: True
Severity: Critical
Description: An Auth0 User enabled MFA Policy for your organization's tenant - severity overridden to critical.

How to view all Derived Detections

To view all Derived Detections in your Panther instance:

1. In the left-hand navigation bar of your Panther Console, click Detections.

2. Click Filters icon.
3. In the Detection Types dropdown field, select Derived Rule.

4. Click Apply Filters.