Alerts & Destinations

Panther detections trigger alerts on suspicious behavior

Overview

Alerts are generated when your rules, scheduled rules, or policies detect suspicious behavior. When an alert is triggered, it is routed to the appropriate Alert Destination(s) using the Alert routing scenarios. You can use AI alert triage to help gather information about an alert and decide what to do next.

Panther can generate three types of alerts:

• Alerts: This classification includes rule matches, policy matches, and scheduled rule matches from enabled detections.

• Detection errors: These are generated due to incorrect code or permissions issues. When this occurs, a rule returns an error and the rule does not complete its run successfully. This includes rule errors and scheduled rule errors.

• System errors: Panther's System Errors alert users when a part of the Panther platform is not functioning correctly. This includes log source inactivity, log classification failures, log source permission failures, alert delivery failures, and cloud account scanning failures.

  • To learn more about these errors, see System Errors.

You can also interact with alerts using the Panther REST API and GraphQL API.

Customizing alerts

You can customize the content of the alerts you receive for detection matches by using the alert functions in Python detections or alert keys in YAML detections.

These functions allow you to, for example, include the matching event (using alert_context() or AlertContext) or add event values to the alert title (using title() or AlertTitle).

Limiting alerts

The alert limiter functionality is intended to safeguard from "alert storms" arising from (likely) misconfigured detections.

If a single detection creates 1,000 alerts within one hour, its CreateAlert field (or Create Alert toggle in the Console) is set to False (or OFF in the Console), which stops the detection from creating additional alerts. (The detection will continue to generate signals on matches.) When this happens, you will receive a System Error notification and alert notifying you of the change.

You can set the CreateAlert/Create Alert value back to True/ON when you are ready—perhaps after some detection tuning.

If you would like to set this alert limit lower than 1,000, please reach out to your Panther Support team.

Alerts with multiple events

If a detection has set a deduplication period and deduplication string, all events matching the detection that share the same deduplication string will be appended to the first alert created, for the length of the deduplication period. This can result in an alert with more than one event associated to it.

Any alert information generated dynamically from a detection (by the alert functions in Python detections or dynamic alert keys in Simple Detections) is generated by the output of the first matching event. The output of the alert functions/keys for additional events attached to the alert does not alter the alert.

For example, if a detection uses severity() to dynamically set the alert severity based on an event property and the first matching event dictates a LOW severity, the alert severity will be set to LOW indefinitely—even if an event later associated to the alert, run through the same severity() function, would dictate a HIGH severity.

Working with alerts

Receiving an alert

To receive alerts outside of the Panther Console, set up an alert destination and ensure it's configured to receive alerts based on the Alert routing scenarios.

When deciding how to receive an alert, consider the following options:

• Natively supported destinations: Panther supports a number of alert destinations natively, like Slack, Jira, and Amazon SNS. See the Alert Destinations documentation for a complete list of supported destinations.
• Custom Webhook: If you'd like to route alerts to a destination that is not natively supported but that has an API, you can use the Custom Webhook option to send alert notifications—see the Custom Webhook documentation for configuration instructions.
• Panther API: If you'd like to receive Panther alerts at a destination that is not natively supported and that does not have an API, you can receive alerts by polling the Panther API for alerts on a schedule. See the available API operations for fetching and manipulating alerts on Alerts (REST) and Alerts & Errors (GraphQL).

Investigating an alert

When you receive an alert to your configured destination, it’s time to investigate. Common investigation workflows include:

• Using AI alert triage.
• Using Search to investigate indicators of compromise (IoCs).
• Using Data Explorer to search robustly using SQL.
• Reviewing data from the Panther Console overview dashboard.

See the Investigations & Search documentation for more information on Panther’s data analysis tools.

Triaging and managing alerts

For more information on triaging, assigning, and managing alerts, see the following documentation pages:

• Assigning and Managing Alerts

  • Triage, use alert summaries, assign, un-assign, view alert history, and add comments to alerts.

  • Quickly tune detections directly from alerts.
  • You can also manage alerts in Slack when using the Slack Bot alert destination.
• Alert Runbooks

  • For many Panther-managed detections, find recommended steps to remediate the issue that triggered the alert.

Viewing alerts

Viewing the alerts list in the Panther Console

1. Log in to your Panther Console.

   • The landing page is the overview dashboard, where you can view alert metrics and a list of the alerts assigned to you. Continue on to the next steps to view a list of all alerts.
2. In the left-hand navigation bar, click Alerts.

   • By default, this page lists alerts from most recent to oldest and displays only Open and Triaged alerts.

   • Near the top of this page, you can view the AI-generated alerts list summary.
3. Filter the list as needed to narrow your results.
4. Click the tabs at the top of the list to filter by the alert type: Alerts, Detection Errors, or System Errors.

Viewing alert details in the Panther Console

While viewing the list of alerts as described above, click an alert title to view the alert details page:

The alert details page includes:

• Basic information

  • This includes the detection that triggered the alert, the associated log types, the assignee, the alert status, the MITRE tactic, and the alert runbook.
• A Start Panther AI Triage option

  • See AI alert triage.
• A list of event matches

  • For each event, you can view event time, event source, the associated p_log_type and p_source_label, and IP information.

• External conversations

  • If there is additional context in a Slack Boomerang, Jira ticket, or Asana task where an alert was delivered, you can click the links in this section to view that information.
  • You can also add comments to an alert in this section.
• Alert history

  • This includes a history of all status changes and comments.

  • If the alert failed to deliver to one of the configured alert destinations, you will see an "Alert delivery failed" error above the history.

The Summary tab on the alert details page is described in Assigning and Managing Alerts.

AI alert triage

Panther AI is in open beta starting with Panther version 1.112, and is available to all customers. Please share any bug reports and feature requests with your Panther support team.

Use of Panther AI features is subject to the AI disclaimer found on the Legal page.

Panther AI alert triage performs an investigation and analysis on an alert. It can be helpful to gather more information about an alert and help you decide what to do next.

AI alert triage may provide a summary of the alert, judgement on whether the alert is a false or true positive, recommended follow-up actions, and an indication of its confidence level. The Panther AI analysis of certain alerts may include a diagram visualizing Panther AI's "thought process" and/or the events that lead to the alert. The analysis will include citations to Panther entities (such as alerts, detections, and searches) when appropriate.

During its alert triage investigation and analysis, Panther AI will only perform read operations—unless you direct it to take a write action, such as by clicking Close alert and mark as Resolved.

To use Panther AI triage:

1. From an alert detail page, click Start Panther AI Triage:

   

   • A Panther AI Alert Triage slide-out panel will appear, where Panther AI will output its findings:

     
2. If the alert's status is Open or Triaged, under Next Steps, optionally click Close alert and mark as Resolved or Close alert and mark as Invalid.

   • The status will be updated to Resolved or Invalid.

     • This is recorded in the alert's Activity log.

   • If the Assign alert to me upon closing toggle is set to ON, the alert assignee will be set as you.

     • This is recorded in the alert's Activity log.

   • A comment with a high-level alert summary will be left in the alert's Activity log.

Reference

Alert timestamps

Each generated alert in Panther is enriched with the following timestamps:

p_alert_creation_time	The first time an event matched this rule
p_event_time	The time the event reported itself as happening
p_parse_time	The time the event was processed by Panther
p_alert_update_time	The last time an event matched this rule (in the case of deduplication)