LogoLogo
Knowledge BaseCommunityRelease NotesRequest Demo
  • Overview
  • Quick Start
    • Onboarding Guide
  • Data Sources & Transports
    • Supported Logs
      • 1Password Logs
      • Apache Logs
      • AppOmni Logs
      • Asana Logs
      • Atlassian Logs
      • Auditd Logs
      • Auth0 Logs
      • AWS Logs
        • AWS ALB
        • AWS Aurora
        • AWS CloudFront
        • AWS CloudTrail
        • AWS CloudWatch
        • AWS Config
        • AWS EKS
        • AWS GuardDuty
        • AWS Security Hub
        • Amazon Security Lake
        • AWS S3
        • AWS Transit Gateway
        • AWS VPC
        • AWS WAF
      • Azure Monitor Logs
      • Bitwarden Logs
      • Box Logs
      • Carbon Black Logs
      • Cisco Umbrella Logs
      • Cloudflare Logs
      • CrowdStrike Logs
        • CrowdStrike Falcon Data Replicator
        • CrowdStrike Event Streams
      • Docker Logs
      • Dropbox Logs
      • Duo Security Logs
      • Envoy Logs
      • Fastly Logs
      • Fluentd Logs
      • GCP Logs
      • GitHub Logs
      • GitLab Logs
      • Google Workspace Logs
      • Heroku Logs
      • Jamf Pro Logs
      • Juniper Logs
      • Lacework Logs
        • Lacework Alert Channel Webhook
        • Lacework Export
      • Material Security Logs
      • Microsoft 365 Logs
      • Microsoft Entra ID Audit Logs
      • Microsoft Graph Logs
      • MongoDB Atlas Logs
      • Netskope Logs
      • Nginx Logs
      • Notion Logs
      • Okta Logs
      • OneLogin Logs
      • Orca Security Logs (Beta)
      • Osquery Logs
      • OSSEC Logs
      • Proofpoint Logs
      • Push Security Logs
      • Rapid7 Logs
      • Salesforce Logs
      • SentinelOne Logs
      • Slack Logs
      • Snowflake Audit Logs (Beta)
      • Snyk Logs
      • Sophos Logs
      • Sublime Security Logs
      • Suricata Logs
      • Sysdig Logs
      • Syslog Logs
      • Tailscale Logs
      • Teleport Logs
      • Tenable Vulnerability Management Logs
      • Thinkst Canary Logs
      • Tines Logs
      • Tracebit Logs
      • Windows Event Logs
      • Wiz Logs
      • Zeek Logs
      • Zendesk Logs
      • Zoom Logs
      • Zscaler Logs
        • Zscaler ZIA
        • Zscaler ZPA
    • Custom Logs
      • Log Schema Reference
      • Transformations
      • Script Log Parser (Beta)
      • Fastmatch Log Parser
      • Regex Log Parser
      • CSV Log Parser
    • Data Transports
      • HTTP Source
      • AWS Sources
        • S3 Source
        • CloudWatch Logs Source
        • SQS Source
          • SNS Source
        • EventBridge
      • Google Cloud Sources
        • Cloud Storage (GCS) Source
        • Pub/Sub Source
      • Azure Blob Storage Source
    • Monitoring Log Sources
    • Ingestion Filters
      • Raw Event Filters
      • Normalized Event Filters (Beta)
    • Data Pipeline Tools
      • Chronosphere Onboarding Guide
      • Cribl Onboarding Guide
      • Fluent Bit Onboarding Guide
        • Fluent Bit Configuration Examples
      • Fluentd Onboarding Guide
        • General log forwarding via Fluentd
        • MacOS System Logs to S3 via Fluentd
        • Syslog to S3 via Fluentd
        • Windows Event Logs to S3 via Fluentd (Legacy)
        • GCP Audit to S3 via Fluentd
      • Observo Onboarding Guide
      • Tarsal Onboarding Guide
    • Tech Partner Log Source Integrations
  • Detections
    • Using Panther-managed Detections
      • Detection Packs
    • Rules and Scheduled Rules
      • Writing Python Detections
        • Python Rule Caching
        • Data Models
        • Global Helper Functions
      • Modifying Detections with Inline Filters (Beta)
      • Derived Detections (Beta)
        • Using Derived Detections to Avoid Merge Conflicts
      • Using the Simple Detection Builder
      • Writing Simple Detections
        • Simple Detection Match Expression Reference
        • Simple Detection Error Codes
    • Correlation Rules (Beta)
      • Correlation Rule Reference
    • PyPanther Detections (Beta)
      • Creating PyPanther Detections
      • Registering, Testing, and Uploading PyPanther Detections
      • Managing PyPanther Detections in the Panther Console
      • PyPanther Detections Style Guide
      • pypanther Library Reference
      • Using the pypanther Command Line Tool
    • Signals
    • Policies
    • Testing
      • Data Replay (Beta)
    • Framework Mapping and MITRE ATT&CK® Matrix
  • Cloud Security Scanning
    • Cloud Resource Attributes
      • AWS
        • ACM Certificate
        • CloudFormation Stack
        • CloudWatch Log Group
        • CloudTrail
        • CloudTrail Meta
        • Config Recorder
        • Config Recorder Meta
        • DynamoDB Table
        • EC2 AMI
        • EC2 Instance
        • EC2 Network ACL
        • EC2 SecurityGroup
        • EC2 Volume
        • EC2 VPC
        • ECS Cluster
        • EKS Cluster
        • ELBV2 Application Load Balancer
        • GuardDuty Detector
        • GuardDuty Detector Meta
        • IAM Group
        • IAM Policy
        • IAM Role
        • IAM Root User
        • IAM User
        • KMS Key
        • Lambda Function
        • Password Policy
        • RDS Instance
        • Redshift Cluster
        • Route 53 Domains
        • Route 53 Hosted Zone
        • S3 Bucket
        • WAF Web ACL
  • Alerts & Destinations
    • Alert Destinations
      • Amazon SNS Destination
      • Amazon SQS Destination
      • Asana Destination
      • Blink Ops Destination
      • Custom Webhook Destination
      • Discord Destination
      • GitHub Destination
      • Google Pub/Sub Destination (Beta)
      • Incident.io Destination
      • Jira Cloud Destination
      • Jira Data Center Destination (Beta)
      • Microsoft Teams Destination
      • Mindflow Destination
      • OpsGenie Destination
      • PagerDuty Destination
      • Rapid7 Destination
      • ServiceNow Destination (Custom Webhook)
      • Slack Bot Destination
      • Slack Destination (Webhook)
      • Splunk Destination (Beta)
      • Tines Destination
      • Torq Destination
    • Assigning and Managing Alerts
      • Managing Alerts in Slack
    • Alert Runbooks
      • Panther-managed Policies Runbooks
        • AWS CloudTrail Is Enabled In All Regions
        • AWS CloudTrail Sending To CloudWatch Logs
        • AWS KMS CMK Key Rotation Is Enabled
        • AWS Application Load Balancer Has Web ACL
        • AWS Access Keys Are Used Every 90 Days
        • AWS Access Keys are Rotated Every 90 Days
        • AWS ACM Certificate Is Not Expired
        • AWS Access Keys not Created During Account Creation
        • AWS CloudTrail Has Log Validation Enabled
        • AWS CloudTrail S3 Bucket Has Access Logging Enabled
        • AWS CloudTrail Logs S3 Bucket Not Publicly Accessible
        • AWS Config Is Enabled for Global Resources
        • AWS DynamoDB Table Has Autoscaling Targets Configured
        • AWS DynamoDB Table Has Autoscaling Enabled
        • AWS DynamoDB Table Has Encryption Enabled
        • AWS EC2 AMI Launched on Approved Host
        • AWS EC2 AMI Launched on Approved Instance Type
        • AWS EC2 AMI Launched With Approved Tenancy
        • AWS EC2 Instance Has Detailed Monitoring Enabled
        • AWS EC2 Instance Is EBS Optimized
        • AWS EC2 Instance Running on Approved AMI
        • AWS EC2 Instance Running on Approved Instance Type
        • AWS EC2 Instance Running in Approved VPC
        • AWS EC2 Instance Running On Approved Host
        • AWS EC2 Instance Running With Approved Tenancy
        • AWS EC2 Instance Volumes Are Encrypted
        • AWS EC2 Volume Is Encrypted
        • AWS GuardDuty is Logging to a Master Account
        • AWS GuardDuty Is Enabled
        • AWS IAM Group Has Users
        • AWS IAM Policy Blocklist Is Respected
        • AWS IAM Policy Does Not Grant Full Administrative Privileges
        • AWS IAM Policy Is Not Assigned Directly To User
        • AWS IAM Policy Role Mapping Is Respected
        • AWS IAM User Has MFA Enabled
        • AWS IAM Password Used Every 90 Days
        • AWS Password Policy Enforces Complexity Guidelines
        • AWS Password Policy Enforces Password Age Limit Of 90 Days Or Less
        • AWS Password Policy Prevents Password Reuse
        • AWS RDS Instance Is Not Publicly Accessible
        • AWS RDS Instance Snapshots Are Not Publicly Accessible
        • AWS RDS Instance Has Storage Encrypted
        • AWS RDS Instance Has Backups Enabled
        • AWS RDS Instance Has High Availability Configured
        • AWS Redshift Cluster Allows Version Upgrades
        • AWS Redshift Cluster Has Encryption Enabled
        • AWS Redshift Cluster Has Logging Enabled
        • AWS Redshift Cluster Has Correct Preferred Maintenance Window
        • AWS Redshift Cluster Has Sufficient Snapshot Retention Period
        • AWS Resource Has Minimum Number of Tags
        • AWS Resource Has Required Tags
        • AWS Root Account Has MFA Enabled
        • AWS Root Account Does Not Have Access Keys
        • AWS S3 Bucket Name Has No Periods
        • AWS S3 Bucket Not Publicly Readable
        • AWS S3 Bucket Not Publicly Writeable
        • AWS S3 Bucket Policy Does Not Use Allow With Not Principal
        • AWS S3 Bucket Policy Enforces Secure Access
        • AWS S3 Bucket Policy Restricts Allowed Actions
        • AWS S3 Bucket Policy Restricts Principal
        • AWS S3 Bucket Has Versioning Enabled
        • AWS S3 Bucket Has Encryption Enabled
        • AWS S3 Bucket Lifecycle Configuration Expires Data
        • AWS S3 Bucket Has Logging Enabled
        • AWS S3 Bucket Has MFA Delete Enabled
        • AWS S3 Bucket Has Public Access Block Enabled
        • AWS Security Group Restricts Ingress On Administrative Ports
        • AWS VPC Default Security Group Restricts All Traffic
        • AWS VPC Flow Logging Enabled
        • AWS WAF Has Correct Rule Ordering
        • AWS CloudTrail Logs Encrypted Using KMS CMK
      • Panther-managed Rules Runbooks
        • AWS CloudTrail Modified
        • AWS Config Service Modified
        • AWS Console Login Failed
        • AWS Console Login Without MFA
        • AWS EC2 Gateway Modified
        • AWS EC2 Network ACL Modified
        • AWS EC2 Route Table Modified
        • AWS EC2 SecurityGroup Modified
        • AWS EC2 VPC Modified
        • AWS IAM Policy Modified
        • AWS KMS CMK Loss
        • AWS Root Activity
        • AWS S3 Bucket Policy Modified
        • AWS Unauthorized API Call
    • Tech Partner Alert Destination Integrations
  • Investigations & Search
    • Search
      • Search Filter Operators
    • Data Explorer
      • Data Explorer SQL Search Examples
        • CloudTrail logs queries
        • GitHub Audit logs queries
        • GuardDuty logs queries
        • Nginx and ALB Access logs queries
        • Okta logs queries
        • S3 Access logs queries
        • VPC logs queries
    • Visualization and Dashboards
      • Custom Dashboards (Beta)
      • Panther-Managed Dashboards
    • Standard Fields
    • Saved and Scheduled Searches
      • Templated Searches
        • Behavioral Analytics and Anomaly Detection Template Macros (Beta)
      • Scheduled Search Examples
    • Search History
    • Data Lakes
      • Snowflake
        • Snowflake Configuration for Optimal Search Performance
      • Athena
  • PantherFlow (Beta)
    • PantherFlow Quick Reference
    • PantherFlow Statements
    • PantherFlow Operators
      • Datatable Operator
      • Extend Operator
      • Join Operator
      • Limit Operator
      • Project Operator
      • Range Operator
      • Sort Operator
      • Search Operator
      • Summarize Operator
      • Union Operator
      • Visualize Operator
      • Where Operator
    • PantherFlow Data Types
    • PantherFlow Expressions
    • PantherFlow Functions
      • Aggregation Functions
      • Date/time Functions
      • String Functions
      • Array Functions
      • Math Functions
      • Control Flow Functions
      • Regular Expression Functions
      • Snowflake Functions
      • Data Type Functions
      • Other Functions
    • PantherFlow Example Queries
      • PantherFlow Examples: Threat Hunting Scenarios
      • PantherFlow Examples: SOC Operations
      • PantherFlow Examples: Panther Audit Logs
  • Enrichment
    • Custom Lookup Tables
      • Creating a GreyNoise Lookup Table
      • Lookup Table Examples
        • Using Lookup Tables: 1Password UUIDs
      • Lookup Table Specification Reference
    • Identity Provider Profiles
      • Okta Profiles
      • Google Workspace Profiles
    • Anomali ThreatStream
    • IPinfo
    • Tor Exit Nodes
    • TrailDiscover (Beta)
  • Panther AI (Beta)
    • Managing Panther AI Response History
  • System Configuration
    • Role-Based Access Control
    • Identity & Access Integrations
      • Azure Active Directory SSO
      • Duo SSO
      • G Suite SSO
      • Okta SSO
        • Okta SCIM
      • OneLogin SSO
      • Generic SSO
    • Panther Audit Logs
      • Querying and Writing Detections for Panther Audit Logs
      • Panther Audit Log Actions
    • Notifications and Errors (Beta)
      • System Errors
    • Panther Deployment Types
      • SaaS
      • Cloud Connected
        • Configuring Snowflake for Cloud Connected
        • Configuring AWS for Cloud Connected
        • Pre-Deployment Tools
      • Legacy Configurations
        • Snowflake Connected (Legacy)
        • Customer-configured Snowflake Integration (Legacy)
        • Self-Hosted Deployments (Legacy)
          • Runtime Environment
  • Panther Developer Workflows
    • Panther Developer Workflows Overview
    • Using panther-analysis
      • Public Fork
      • Private Clone
      • Panther Analysis Tool
        • Install, Configure, and Authenticate with the Panther Analysis Tool
        • Panther Analysis Tool Commands
        • Managing Lookup Tables and Enrichment Providers with the Panther Analysis Tool
      • CI/CD for Panther Content
        • Deployment Workflows Using Panther Analysis Tool
          • Managing Panther Content via CircleCI
          • Managing Panther Content via GitHub Actions
        • Migrating to a CI/CD Workflow
    • Panther API
      • REST API (Beta)
        • Alerts
        • Alert Comments
        • API Tokens
        • Data Models
        • Globals
        • Log Sources
        • Queries
        • Roles
        • Rules
        • Scheduled Rules
        • Simple Rules
        • Policies
        • Users
      • GraphQL API
        • Alerts & Errors
        • Cloud Account Management
        • Data Lake Queries
        • Log Source Management
        • Metrics
        • Schemas
        • Token Rotation
        • User & Role Management
      • API Playground
    • Terraform
      • Managing AWS S3 Log Sources with Terraform
      • Managing HTTP Log Sources with Terraform
    • pantherlog Tool
    • Converting Sigma Rules
  • Resources
    • Help
      • Operations
      • Security and Privacy
        • Security Without AWS External ID
      • Glossary
      • Legal
    • Panther System Architecture
Powered by GitBook
On this page
  • Overview
  • How to use behavioral analytics and anomaly detection macros in Panther
  • Going beyond ad-hoc searches
  • Full examples
  • Viewing the macro source code
  • Peer group analysis
  • Behavioral analytics and anomaly detection macro reference
  • statistical_anomaly
  • new_unique_values
  • statistical_anomaly_peer
  • new_unique_values_peer

Was this helpful?

  1. Investigations & Search
  2. Saved and Scheduled Searches
  3. Templated Searches

Behavioral Analytics and Anomaly Detection Template Macros (Beta)

Detect outliers with Panther-managed macros for behavioral analytics and anomaly detection

PreviousTemplated SearchesNextScheduled Search Examples

Last updated 1 month ago

Was this helpful?

Overview

These template macros are in open beta starting with Panther version 1.112, and are available to all customers. Please share any bug reports and feature requests with your Panther support team.

Panther provides for identifying anomalous and new values across your log data. Using the template macros, you can compare recent log events to historical data to identify activity deviating significantly from the established norm. These behavioral analytics and anomaly detection macros may be useful as part of your User and Entity Behavior Analytics (UEBA) strategy.

These macros work by comparing data in a recent time interval to data in a longer lookback window and determining the level of deviation between the two.

The macros Panther provides are:

  • statistical_anomaly: Identifies outlier values for a numerical field

    • Example: Find VPC hosts that have been sending an unusually high volume of traffic over the last hour, as compared to the last two weeks.

    • Learn more in .

  • statistical_anomaly_peer: Identifies outlier values for a numerical field within a

    • Example: Identify attempts by a user to access a resource that is unusual for members of the same team.

    • Learn more in .

  • new_unique_values: Identifies new values for a given entity

    • Example: Find API tokens that have accessed a resource in the last day they have not accessed in the previous 30 days.

    • Learn more in .

  • new_unique_values_peer: Identifies new values for a given entity within a

    • Example: Detect if an EC2 instance has connected to an IP that is atypical for members of its VPC group.

    • Learn more in

How to use behavioral analytics and anomaly detection macros in Panther

  1. In the left-hand navigation bar of your Panther Console, click Investigate > Data Explorer.

  2. At the top of the SQL editor, add a -- pragma: template statement.

    -- pragma: template
    -- pragma: show macro expanded # Optional
  3. Import one of the available macros:

    {% import 'ueba' <statistical_anomaly, new_unique_values, 
    statistical_anomaly_peer, OR new_unique_values_peer> %}
    -- Specify only one macro, not all four
    • SELECT at least:

      • p_event_time

      • An entity column. The entity is often an ID of some kind (such as an email address, user ID, application ID, or hostname), but can be any data type (such as an IP address). The column must be a top-level field. If the field is nested within an object or array, create an alias for the column with the AS keyword.

      • An aggregation column. For statistical_anomaly queries, this column's contents will be aggregated and scanned for unusual values. For new_unique_values queries, this column's contents will be scanned for new values. As with the entity column, this column must be a top-level field.

    with subquery as (
        select 
            user:email as email, -- entity column
            event_type, -- aggregation column
            p_event_time
        from mytable where p_occurs_since(30d)
    ),
  4. Below the subquery, invoke the macro:

    {{ <statistical_anomaly, new_unique_values, statistical_anomaly_peer, OR new_unique_values_peer>
    (<subquery>, '<entity_col>', ...) }}
  5. Click Run Search.

Going beyond ad-hoc searches

Full examples

See full examples invoking all macros below.

Full example using statistical_anomaly

This query compares VPC traffic observed in the past hour to a baseline set by the past 24 hours, and alerts if any addresses have sent an unusual volume of outbound traffic, potentially indicating a data exfiltration action.

-- pragma: template

{% import 'anomalies' statistical_anomaly %}
WITH subquery AS (
  -- Look for outbound requests:
  SELECT
    p_event_time as p_timeline,
    concat(srcAddr,' -> ',dstAddr,':',dstPort) as traffic,
    *
  FROM
    panther_logs.public.aws_vpcflow
  WHERE
    p_occurs_since('7 day')
    AND dstAddr not like '10.%'
    AND dstPort < 1024
    AND flowDirection = 'egress'
    AND pktDstAwsService is null
),
{{statistical_anomaly('subquery', 'traffic', 'bytes', 'sum', '1', 'hour', 3)}}

Full example using new_unique_values

This query analyzes login events in the last hour and returns cases where an email address was used to login from an IP address that hasn't been seen in the previous 30 days.

This query can be useful for identifying suspicious logins.

--pragma: template
--pragma: show macro expanded
{% import 'anomalies' new_unique_values %}

with subquery as (
    select email, src_ip, p_event_time
    from mytable where p_occurs_since(30d)
),
{{ new_unique_values('subquery', 'email', 'src_ip', '1hr') }}

Full example using statistical_anomaly_peer

The example below uses peer grouping to compare the number of GitHub pull requests created by one user to the number of pull requests created by other members of their team.

{% import 'anomalies' statistical_anomaly_peer %}

with logs as (
    select
        p_event_time,
        actor,
        action as action_name
    from panther_logs.public.github_audit
    where p_occurs_since('90d') -- last 90 days of data
    and action_name = 'pull_request.create' -- only pull request creation events
    and DAYNAME(p_event_time) not in ('Sat', 'Sun') -- ignore weekends
), subquery as (
    select
        logs.p_event_time,
        logs.actor as actor,
        logs.event as event_type
        lut.team as team
    from
        logs
        join
        panther_lookups.public.custom_github_teams as lut
        on logs.actor = lut.actor
),
{{ statistical_anomaly_peer('subquery', 'actor', 'team', 'n', 'count', '1', 'day', 0.1) }}

Full example using new_unique_values_peer

The query in this example seeks to discover whether any users have created a GitHub pull request in a repository that has not been used previously by any user on their team.


{% import 'anomalies' new_unique_values_peer %}

with logs as (
    select
        p_event_time,
        actor,
        action as action_name
        repo,
    from panther_logs.public.github_audit
    where p_occurs_since('90d') -- last 90 days of data
    and action_name = 'pull_request.create' -- only pull request creation events
), subquery as (
    select
        logs.p_event_time,
        logs.actor as actor,
        logs.event as event_type
        lut.team as team
    from
        logs
        join
        panther_lookups.public.custom_github_teams as lut
        on logs.actor = lut.actor
),
{{ new_unique_values_peer('subquery', 'actor', 'team', 'repo', '1d') }}

Viewing the macro source code

You can view the behavioral analytics and anomaly detection macro source code in either your Panther Console or in the panther-analysis repository.

To view the macro source code in your Panther Console:

  1. In the left-hand navigation bar of your Panther Console, click Investigate > Saved Searches.

To view the macro source code in panther-analysis:

Peer group analysis

It's often useful to compare an entity's behavior specifically against the behavior of its peers. For example, has an engineer recently signed into an account that other engineers have not?

Panther provides peer versions of statistical_anomaly and new_unique_values to perform such analysis. In the peer versions, baseline statistics are calculated according to the peer group, then entity behavior is compared to the baseline.

These queries function similarly to the non-peer versions, with the addition of an extra parameter, group_field, to define the peer group. This should be a column whose value is used to group entities together. Some common examples of the group_field value are: user role, job department, VPC ID, and account ID.

Behavioral analytics and anomaly detection macro reference

Below, you can find reference information for how to use the macros provided by Panther. Unless otherwise specified, assume all input arguments must be provided in the order shown here.

statistical_anomaly

The statistical_anomaly macro looks over a dataset for unusual data points within a recent period. It takes a CTE as the base data set and compares the baseline activity for an entity over that period to the most recent activity by the same entity, and calculates how unusual this behavior is.

You must provide the base data set, specify which column contains the entity name and which column to use for data comparison. You must also define the size of the recent period in which to look for anomalies.

Input arguments

Each of the following arguments must be provided to the macro, in the order shown below.

Name
Data type
Desccription

subquery

String

Name of the CTE defined previously, which provides data for the macro to analyze.

entity_field

String

Name of the column to use for grouping; usually a name, IP address, or ID.

agg_field

String

Name of the column to search for outliers.

agg_func

String

Which SQL function to use to aggregate the data in agg_field within a time period. Common value are count, sum, and max.

tmag

String

Number of units for the lookback window in which to look for anomalies. i.e.: the 1 in "1 day".

tunit

String

Unit of time for the lookback window in which to look for anomalies. i.e.: the day in "1 day". Must be singular (no "s" at the end).

zscore

Number

Outlier threshold; results will not be returned unless their calculated zscore value is higher than this.

Returns

The table returned after executing the macro will have the following columns:

Name
Data type
Description

N

Number

The value of the data in agg_field, as aggregated by agg_func, for the given entity over the lookback period.

t1

Timestamp

Start of the lookback period.

t2

Timestamp

End of the lookback period.

<entity_field>

Any

Value of the chosen entity_field.

p_zscore

Number

Calculated zscore of the entity's activities over the lookback period. Higher zscore value means more anomalous.

p_mean

Number

Average value of the agg_field column for this entity over the data in subquery, excluding during the lookback period.

p_stddev

Number

Standard deviation of the agg_field column for this entity over the data in subquery, excluding during the lookback period. Larger p_stddev means the entity's activity was less consistent overall.

new_unique_values

The new_unique_values macro scans a data set and returns a set of new values from a chosen column present in a recent lookback period for a given entity.

Input arguments

The following arguments must be provided to the macro, in the order shown below.

Name
Data type
Description

subquery

String

Name of the CTE defined previously, which contains the base data to use for finding anomalies.

entity_field

String

Name of the column to use for grouping; usually a user name, IP address, or ID.

agg_field

String

Name of the column in which to search for new values.

interval

String

Returns

The table returned after executing the macro will have the following columns:

Name
Data type
Description

<entity_field>

Any

Value of the defined entity_field column.

<agg_field>

Any

Any new values discovered in the agg_field column during the lookback period

statistical_anomaly_peer

Input arguments

The following arguments must be provided to the macro, in the order shown below.

Name
Data type
Desccription

subquery

String

Name of the CTE defined previously, which provides data for the macro to analyze.

entity_field

String

Name of the column to use for identifying an entity; usually a name, IP address, or ID.

group_field

String

Name of the column to use to group entities; for example, a role name or an Account ID.

agg_field

String

Name of the column to search for outliers.

agg_func

String

Which SQL function to use to aggregate the data in agg_field within a time period. Common value are count, sum, and max.

tmag

String

Number of units for the lookback window in which to look for anomalies. i.e.: the 1 in "1 day".

tunit

String

Unit of time for the lookback window in which to look for anomalies. i.e.: the day in "1 day". Must be singular (no "s" at the end).

zscore

Number

Outlier threshold; results will not be returned unless their calculated zscore value is higher than this.

Returns

The table returned after executing the macro will have the following columns:

Name
Data type
Description

N

Number

The value of the data in agg_field, as aggregated by agg_func, for the given entity over the lookback period.

t1

Timestamp

Start of the lookback period.

t2

Timestamp

End of the lookback period.

<entity_field>

Any

Value of the chosen entity_field.

<group_field>

Any

Value of the chosen group_field.

p_zscore

Number

Calculated zscore of the entity's activities over the lookback period. Higher zscore value means more anomalous.

p_mean

Number

Average value of the agg_field column for this entity over the data in subquery, excluding during the lookback period.

p_stddev

Number

Standard deviation of the agg_field column for this entity over the data in subquery, excluding during the lookback period. Larger p_stddev means the entity's activity was less consistent overall.

new_unique_values_peer

Input arguments

The following arguments must be provided to the macro, in the order shown below.

Name
Data type
Description

subquery

String

Name of the CTE defined previously, which contains the base data to use for finding anomalies.

entity_field

String

Name of the column to use for identifying an entity; usually a name, IP address, or ID.

group_field

String

Name of the column to use to group entities; for example, a role name or an Account ID.

agg_field

String

Name of the column in which to search for new values.

interval

String

Returns

The table returned after executing the macro will have the following columns:

Name
Data type
Description

<entity_field>

Any

Value of the defined entity_field column.

<group_field>

Any

Value of the defined group_field column.

<agg_field>

Any

Any new values discovered in the agg_field column during the lookback period

Learn how to view the macros' source code .

You can invoke the Panther-managed behavioral analytics and anomaly detection macros in by following the instructions below. This process is similar to the instructions, but is specific to using the behavioral analytics and anomaly detection template macros.

Optionally, you can also add --pragma: show macro expanded. This will expand the macro into its source code, and is useful for troubleshooting issues with your query. See for more information.

Define a subquery using a . The subquery must:

(If you're using statistical_anomaly_peer or new_unique_values_peer) A field.

Using a WHERE clause, define the lookback window (i.e., the longer period of time that denotes the baseline against which the shorter window is compared). The lookback window should end at the current time; for this reason it is recommended to use p_occurs_since(). .

See the full list of input arguments for each macro in , below.

While Panther's behavioral analytics and anomaly detection queries are useful for threat hunting, they're more powerful when used as a monitoring system. The queries can be , set to run on a schedule, and attached to Scheduled Rules. In this way, you can get alerted whenever anomalous activity is observed.

The query uses where filter clauses to only fetch pull request creation events, as well as to exclude events generated during weekends. It also pulls username/development team associations from a .

The query uses a where filter clause to only fetch pull request creation events. The repo column is extracted in order to analyze for new values. The query also pulls username/development team associations from a .

Search for the Saved Search named anomalies, and click its name.

You will be directed to , where you can view the source code:

In your local Panther code repository or in , view the queries/macros/anomalies.yml file.

Size of the period in which to look for new values. Uses the same syntax as .

Use this macro to determine unusual numerical behavior from an entity compared to its . For example, checking an EC2 instance's traffic volume compared to others with the same tag, or access requests to an S3 object compared to other objects in the same bucket.

Use this macro to identify when an entity has done something that hasn't previously been observed by a member of its .

Size of the lookback period in which to look for new values. Uses the same syntax as .

Common Table Expression (CTE)
Learn more about p_occurs_since() here
saved
custom lookup table
custom lookup table
Panther's upstream panther-analysis repository
below
peer group
Behavioral analytics and anomaly detection macro reference
peer group
peer group
Data Explorer
Data Explorer
statistical_anomaly
peer group
statistical_anomaly_peer
new_unique_values
peer group
new_unique_values_peer.
template macros
Calling template macros in other queries
Debugging template macros
p_occurs_since
p_occurs_since