Knowledge BaseCommunityRelease NotesTry Panther
Search…
⌃K
Links

Rules

Panther's Rules are Python functions to detect suspicious activity in logs then generate alerts

Overview

Rules are Python functions for detecting suspicious activity in logs and generating alerts. Rules apply to security logs while Policies apply to cloud resources.
Common examples of rules include analyzing logs for:
  • Authentication from unknown or unexpected locations
  • Sensitive API calls, such as administrative changes to SaaS services
  • Network traffic access to sensitive data sources, such as databases or virtual machines
  • New suspicious entries added into a system's scheduled tasks, like cron
  • Alerts generated from NIDS, HIDS, or other security systems

Types of rules

Panther has two core types of rules:
  • Real-time rules
    • Real-Time rules are the default mechanism of analyzing data sent to Panther. They have the benefit of low-latency detection and alerting.
    • Use cases: High-signal logs that do not require joins with other data.
  • Scheduled rules
    • Scheduled rules work by running a SQL query on a defined interval (or cron) and using the result of that query as input into a Python-based rule.
    • Use cases: Querying windows of time further in the past, running statistical analysis over data, or joining separate data streams, use a scheduled rule.

How rules work

Rules analyze one event at a time and can use thresholds, deduplication, and event grouping to analyze logs within windows of time. They run on a defined set of log types such as Okta, Box, or your own custom data. A rule contains a rule function, title, and dedup function.
By default, each rule has a threshold of 1 and a deduplication period of 1h, meaning all events returning True from a rule would be appended to the alert within the hour after first being generated.
Provided a threshold of 5 and a deduplication period of 15m, an alert would not trigger until 5 or more rule matches have occurred within 15 minutes.

How to write rules

You can write rules in the Panther Console, or you can write rules locally.
Writing Detections locally means creating Python and metadata files that define a Panther Detection on your own machine. After writing Detections locally, you upload the files to your Panther instance (typically via the Panther Analysis Tool) to control your Detection content.
Note: Anything printed to stdout or stderr by your Python code will end up in CloudWatch. For SaaS/CPaaS customers, Panther engineers can see these CloudWatch logs during routine application monitoring.
Panther Console
Write rules locally
  1. 1.
    Log in to the Panther Console.
  2. 2.
    Click Build > Detections.
  3. 3.
    Click Create New.
  4. 4.
    Select your detection type (Rule or Scheduled Rule)
  5. 5.
    Fill out the required Basic Info as follows:
    • Enabled: Toggle the button in the right-hand corner to ON.
    • Name: Enter a descriptive name for your new detection.
    • Severity: Select a detection severity from the drop-down options.
    • In the lower right-hand drop-down menu, choose the applicable log types.
      • If you selected Scheduled Rule: Select the Scheduled Query or queries this rule should apply to.
    • Unique ID (optional): Click the pen icon and enter a unique ID for your detection.
  6. 6.
    Click the Functions & Tests tab and write a Python function to define your Detection in the Rule Function text editor.
  7. 7.
    Click {} Create Test to run a test against the Detection you defined in the previous step.
  8. 8.
    Optional: Adjust your Rule Settings or add a report on Report Mapping.
  9. 9.
    Click Save in the upper right corner.
You will land on your detection's information page - click Edit in the upper right-hand corner to manage or update your detection.

File setup

Each detection consists of:
  • A Python file (a file with a .py extension) containing your detection/audit logic
  • A YAML or JSON specification file (a file with a .yml or .json extension) containing metadata attributes of the detection.
    • By convention, we give this file the same name as the Python file.
We recommend creating folders based on log/resource type to group your detections, such as suricata_rules or aws_s3_policies. You can use the open source Panther Analysis repo as a reference.
We also recommend managing these files in a version control system (VCS). Most commonly we see GitHub or GitLab used, which are managed git providers.
It's best practice to create a fork of Panther's open-source analysis repository, but you can also create your own repo from scratch.

Writing real-time and scheduled rules locally

Rules are Python functions to detect suspicious behaviors. Returning a value of True indicates suspicious activity, which triggers an alert.
  1. 1.
    ​Write your Rule and save it (in your folder of choice) as my_new_rule.py: def rule(event):
    def rule(event):
    return 'prod' in event.get('hostName')
  2. 2.
    Create a metadata file using the template below:
    AnalysisType: rule
    DedupPeriodMinutes: 60 # 1 hour
    DisplayName: Example Rule to Check the Format of the Spec
    Enabled: true
    Filename: my_new_rule.py
    RuleID: Type.Behavior.MoreContext
    Severity: High
    LogTypes:
    - LogType.GoesHere
    Reports:
    ReportName (like CIS, MITRE ATT&CK):
    - The specific report section relevant to this rule
    Tags:
    - Tags
    - Go
    - Here
    Description: >
    This rule exists to validate the CLI workflows of the Panther CLI
    Runbook: >
    First, find out who wrote this the spec format, then notify them with feedback.
    Reference: https://www.a-clickable-link-to-more-info.com
When this rule is uploaded, each of the fields you would normally populate in the Panther Console will be auto-filled. See Rule Specification Reference below for a complete list of required and optional fields.

Rule Errors

A rule error will use the destination overrides configured for the rule. If there are no destination overrides for the rule itself, it will route to any destinations configured for rule errors. In the event of a query timeout, the Python code for Destinations will not run.

Rule examples

See the Panther Analysis Github repository for Panther rule templates.
For in-depth Detection examples, best practices, and information on functions and features, please see Panther's Writing Detections documentation.

Send an alert when an admin panel is accessed on a web server

As an example, let's write a rule to send an alert when an admin panel is accessed on a web server. The following NGINX log below will be used:
{
"httpReferer": "https://domain1.com/?p=1",
"httpUserAgent": "Chrome/80.0.3987.132 Safari/537.36",
"remoteAddr": "180.76.15.143",
"request": "GET /admin-panel/ HTTP/1.1",
"status": 200,
"time": "2019-02-06 00:00:38 +0000 UTC"
}
A basic rule would look like this:
  • A rule function that looks for 200 (OK) web requests to any URL with the admin-panel string.
    • Return type: Boolean.
  • A title to say that admin panel logins have been logged into from a specific IP address.
    • Return type: String.
  • A dedup function to group all events by the same IP address.
    • Return type: String.
def rule(event):
return event.get('status') == 200 and 'admin-panel' in event.get('request')
​
def title(event):
return f"Successful admin panel login detected from {event.get('remoteAddr')}"
​
​
def dedup(event):
return event.get('remoteAddr')
Then, the following would occur:
  1. 1.
    An alert would be generated and sent to the set of associated destinations, which by default are based on the rule severity
  2. 2.
    The alert would say Successful admin panel login detected from 180.76.15.143
  3. 3.
    Similar events with the same dedup string of 180.76.15.143 would be appended to the alert
  4. 4.
    The recipient of the alert could then check Panther to view all alert metadata, a summary of the events, and run SQL over all of the events to perform additional analysis
A unique alert will be generated for each unique deduplication string, which in this case, is the IP of the requestor.

Rule Specification Reference

Required fields are in bold.
​
Field Name
Description
Expected Value
AnalysisType
Indicates whether this analysis is a rule, policy, or global
Rules: rule Scheduled Rules: scheduled_rule
Enabled
Whether this rule is enabled
Boolean
FileName
The path (with file extension) to the python rule body
String
RuleID
The unique identifier of the rule
String
LogTypes
The list of logs to apply this rule to
List of strings
Severity
What severity this rule is
One of the following strings: Info, Low, Medium, High, or Critical
ScheduledQueries (field only for Scheduled Rules)
The list of Scheduled Query names to apply this rule to
List of strings
Description
A brief description of the rule
String
DedupPeriodMinutes
The time period (in minutes) during which similar events of an alert will be grouped together
15,30,60,180 (3 hours),720 (12 hours), or 1440 (24 hours)
DisplayName
A friendly name to show in the UI and alerts. The RuleID will be displayed if this field is not set.
String
OutputIds
Static destination overrides. These will be used to determine how alerts from this rule are routed, taking priority over default routing based on severity.
List of strings
Reference
The reason this rule exists, often a link to documentation
String
Reports
A mapping of framework or report names to values this rule covers for that framework
Map of strings to list of strings
Runbook
The actions to be carried out if this rule returns an alert, often a link to documentation
String
SummaryAttributes
A list of fields that alerts should summarize.
List of strings
Threshold
How many events need to trigger this rule before an alert will be sent.
Integer
Tags
Tags used to categorize this rule
List of strings
Tests
Unit tests for this rule.
List of maps
Previous
Writing Detections
Next
Data Replay (Beta)
Last modified 5d ago