# Alerts & Destinations ## Overview Alerts are generated when your [rules, scheduled rules, or policies](https://docs.panther.com/detections) detect suspicious behavior. When an alert is triggered, it is routed to the appropriate [Alert Destination(s)](https://docs.panther.com/alerts/destinations) using the [Alert routing scenarios](https://docs.panther.com/destinations#alert-routing-scenarios). In the Panther Console, you can view an [AI-generated triage of your alerts](https://docs.panther.com/ai/panther-ai-and-alerts#panther-ai-triage-of-alerts-list) and use [AI alert triage](https://docs.panther.com/ai/panther-ai-and-alerts#panther-ai-alert-triage) to help gather information about an alert and decide what to do next. Panther can generate three types of alerts: * **Alerts**: This classification includes rule matches, policy matches, and scheduled rule matches from enabled [detections](https://docs.panther.com/detections). * **Detection errors:** These are generated due to incorrect code or permissions issues. When this occurs, a rule returns an error and the rule does not complete its run successfully. This includes [rule errors and scheduled rule errors](https://docs.panther.com/detections/rules#rule-errors-and-scheduled-rule-errors). * **System errors:** Panther's System Errors alert users when a part of the Panther platform is not functioning correctly. This includes log source inactivity, log classification failures, log source permission failures, alert delivery failures, and cloud account scanning failures. * To learn more about these errors, see [System Errors](https://docs.panther.com/system-configuration/notifications/system-errors). You can also interact with alerts using the Panther [REST API](https://docs.panther.com/panther-developer-workflows/api/rest/alerts) and [GraphQL API](https://docs.panther.com/panther-developer-workflows/api/graphql/alerts-and-errors). ## Customizing alerts You can customize the content of the alerts you receive for detection matches by using the [alert functions in Python detections](https://docs.panther.com/detections/rules/python#alert-functions-in-python-detections) or [alert keys in YAML detections](https://docs.panther.com/detections/rules/writing-simple-detections#dynamic-alert-keys-in-yaml-detections). These functions allow you to, for example, include the matching event (using [`alert_context()`](https://docs.panther.com/detections/rules/python#alert_context) or [`AlertContext`](https://docs.panther.com/detections/rules/writing-simple-detections#alertcontext)) or add event values to the alert title (using [`title()`](https://docs.panther.com/detections/rules/python#title) or [`AlertTitle`](https://docs.panther.com/detections/rules/writing-simple-detections#alerttitle)). ## Limiting alerts The alert limiter functionality is intended to safeguard from "alert storms" arising from (likely) misconfigured detections. If a single detection creates 1,000 alerts within one hour, its `CreateAlert` field (or **Create Alert** toggle in the Console) is set to `False` (or `OFF` in the Console), which stops the detection from creating additional alerts. (The detection will continue to generate [signals](https://docs.panther.com/detections/signals) on matches.) When this happens, you will receive a [System Error](https://docs.panther.com/system-configuration/notifications/system-errors) notification and alert notifying you of the change. You can set the `CreateAlert`/**Create Alert** value back to `True`/`ON` when you are ready—perhaps after some detection tuning. If you would like to set this alert limit lower than 1,000, please reach out to your Panther Support team. ## Alerts with multiple events If a detection has set a [deduplication period and deduplication string](https://docs.panther.com/detections/rules#deduplication-of-alerts), all events matching the detection that share the same deduplication string will be appended to the first alert created, for the length of the deduplication period. This can result in an alert with more than one event associated to it. Any alert information generated dynamically from a detection (by the [alert functions in Python detections](https://docs.panther.com/detections/rules/python#alert-functions-in-python-detections) or [dynamic alert keys in Simple Detections](https://docs.panther.com/detections/rules/writing-simple-detections#dynamic-alert-keys-in-simple-detections)) is generated by the output of the first matching event. The output of the alert functions/keys for *additional* events attached to the alert does not alter the alert. For example, if a detection uses [`severity()`](https://docs.panther.com/detections/rules/python#severity) to dynamically set the alert severity based on an event property and the first matching event dictates a `LOW` severity, the alert severity will be set to `LOW` indefinitely—even if an event later associated to the alert, run through the same `severity()` function, would dictate a `HIGH` severity. ## Working with alerts ### Receiving an alert To receive alerts outside of the Panther Console, set up an alert destination and ensure it's configured to receive alerts based on the [Alert routing scenarios](https://docs.panther.com/destinations#alert-routing-scenarios). When deciding how to receive an alert, consider the following options: * **Natively supported destinations**: Panther supports a number of alert destinations natively, like Slack, Jira, and Amazon SNS. See the [Alert Destinations documentation](https://docs.panther.com/alerts/destinations) for a complete list of supported destinations. * **Custom Webhook**: If you'd like to route alerts to a destination that is not natively supported but that has an API, you can use the Custom Webhook option to send alert notifications—see the [Custom Webhook documentation](https://docs.panther.com/alerts/destinations/custom_webhook) for configuration instructions. * **Panther API**: If you'd like to receive Panther alerts at a destination that is not natively supported and that does not have an API, you can receive alerts by polling the [Panther API](https://docs.panther.com/panther-developer-workflows/api) for alerts on a schedule. See the available API operations for fetching and manipulating alerts on [Alerts](https://docs.panther.com/panther-developer-workflows/api/rest/alerts) (REST) and [Alerts & Errors](https://docs.panther.com/panther-developer-workflows/api/graphql/alerts-and-errors) (GraphQL). Alerts can also be delivered to a destination after the alert has been created with [manual alert dispatch](#manual-alert-dispatch). ### Investigating an alert When you receive an alert to your configured destination, it’s time to investigate. Common investigation workflows include: * Using [AI alert triage](https://docs.panther.com/ai/panther-ai-and-alerts#panther-ai-alert-triage). * Using [Search](https://docs.panther.com/search/search-tool) to investigate indicators of compromise (IoCs). * Using [Data Explorer](https://docs.panther.com/search/data-explorer) to search robustly using SQL. * Reviewing data from the [Panther Console overview dashboard](https://docs.panther.com/search/visualization-and-dashboards/panther-managed). See the [Investigations & Search documentation](https://docs.panther.com/search) for more information on Panther’s data analysis tools. ### Triaging and managing alerts For more information on triaging, assigning, and managing alerts, see the following documentation pages: * [Assigning and Managing Alerts](https://docs.panther.com/alerts/alert-management) * Triage, use alert summaries, assign, un-assign, view alert history, and add comments to alerts. * [Quickly tune detections](https://docs.panther.com/detections/rules/inline-filters#add-filters-from-an-alert-event) directly from alerts. * You can also [manage alerts in Slack](https://docs.panther.com/alerts/alert-management/slack) when using the [Slack Bot alert destination](https://docs.panther.com/alerts/destinations/slack-bot). * [Alert Runbooks](https://docs.panther.com/alerts/alert-runbooks) * For many Panther-managed detections, find recommended steps to remediate the issue that triggered the alert. ## Viewing alerts ### Viewing the alerts list in the Panther Console 1. Log in to your Panther Console. * The landing page is the [overview dashboard](https://docs.panther.com/search/visualization-and-dashboards/panther-managed), where you can view alert metrics and a list of the alerts assigned to you. Continue on to the next steps to view a list of all alerts. 2. In the left-hand navigation bar, click **Alerts**. * By default, this page lists alerts from most recent to oldest and displays only **Open** and **Triaged** alerts. * Near the top of this page, you can view the [AI-generated alerts list triage](https://docs.panther.com/ai/panther-ai-and-alerts#panther-ai-triage-of-alerts-list). 3. Filter the list as needed to narrow your results.
4. Click the tabs at the top of the list to filter by the alert type: **Alerts**, **Detection Errors**, or **System Errors**.
### Viewing alerts in Search To view alerts in [Search](https://docs.panther.com/search/search-tool): 1. In the [database filter](https://docs.panther.com/search/search-tool#using-database-table-and-date-range-filters), select **Signals**. 2. In the [table filter](https://docs.panther.com/search/search-tool#using-database-table-and-date-range-filters), select **Alerts**. 3. Create additional filter chips as desired. 4. Click **Search**. ### Viewing alert details in the Panther Console While viewing the list of alerts as described above, click an alert title to view the alert details page:
An alert tile on the alerts list page is shown, with a title reading "AWS login detected without MFA for x in x account x". This title is circled. Below and next to the title are fields like Severity, Assignee and Alert Status.
The alert details page includes: * Basic information * This includes the detection that triggered the alert, the associated log types, the assignee, the alert status, alert quality, context tags, the [MITRE tactic](https://docs.panther.com/detections/report-mapping), and the [alert runbook](https://docs.panther.com/alerts/alert-runbooks). * A **Start Panther AI Triage** option * See [AI alert triage](https://docs.panther.com/ai/panther-ai-and-alerts#panther-ai-alert-triage). * A **Dispatch Alert** option * See [manual alert dispatch](#manual-alert-dispatch). * A list of event matches * For each event, you can view event time, event source, the associated `p_log_type` and `p_source_label`, and IP information. * External conversations * If there is additional context in a [Slack Boomerang](https://docs.panther.com/alert-management/slack#send-boomerang), Jira ticket, or Asana task where an alert was delivered, you can click the links in this section to view that information. * You can also [add comments to an alert](https://docs.panther.com/alert-management#adding-comments-to-alerts) in this section. * Alert history * This includes a history of all status changes and comments. * If the alert failed to deliver to one of the configured alert destinations, you will see an "Alert delivery failed" error above the history. * Alerts manually delivered to destinations will appear in this section too. Delivering an alert to the same destination multiple times will update the last delivery time of the entry in the table, but will not appear as a second item in the list.
The Summary tab on the alert details page is described in [Assigning and Managing Alerts](https://docs.panther.com/alerts/alert-management). ## Manual alert dispatch {% hint style="info" icon="circle-exclamation" %} anual alert dispatch isupported for `pypanther` alerts. {% endhint %} On the alert details page, the "Dispatch Alert" button allows you to manually delivery an alert to a configured destination.
Alert details page with "Dispatch Alert" button highlighted
Alert destinations need to be individually updated to allow manual alert routing to them. If you do not have any enabled yet, you will see a popup after clicking **Dispatch Alert** that will take you to the Destinations page. See [Scenario 4: Manual alert dispatch](https://docs.panther.com/destinations#scenario-4-manual-alert-dispatch) for steps to enable alert destinations for manual dispatch.
Dispatch Alert manual setting not enabled popup
After clicking on **Dispatch Alert**, select one or more destinations to deliver the alert to. Click **Send to # Destination(s)** at the bottom of the slide-out to send them. You can see the manual alert delivery history in the alert activity.
## Reference ### Alert timestamps Each generated alert in Panther is enriched with the following timestamps:
p_alert_creation_timeThe first time an event matched this rule
p_event_timeThe time the event reported itself as happening
p_parse_timeThe time the event was processed by Panther
p_alert_update_timeThe last time an event matched this rule (in the case of deduplication)