Knowledge BaseCommunityRelease NotesTraining VideosRequest Demo
Search
⌃K
Links

Data Replay (Beta)

Preview the outcome of a rule against real data before enabling it

Overview

Data Replay is currently in open beta, and is available to all customers. Please share any bug reports and feature requests with your account team.
With Panther's Data Replay, rules can be tested against historical log data to preview the outcome of a rule before enabling it.
When writing or updating a detection, use Data Replay to simulate what type of alerts you are likely to receive before deploying the detection. Alerts generated through a replay will not be delivered to the rule's destinations or displayed on the main Alerts & Errors dashboard.
Data Replay is available in the Panther Console. In the CLI workflow, you can use Panther Analysis Tool's benchmark command to evaluate rule performance.

Limitations

Before starting a Data Replay, note the following limitations:
  • Time range is configurable but must be within the last 15 days
  • Time range must be older than 24 hours
  • Replay must complete in under an hour
  • Access to the DynamoDB cache is blocked to prevent polluting production data
  • Access to Lookup Tables is blocked to prevent modifying production data from a replay
  • Network calls from within a rule are blocked

Starting a replay

A rule's deduplication period and events threshold will be taken into account during Data Replay. If a rule has Rule Filters, they will be applied.
This consolidated user interface for viewing and editing detections is in open beta starting with Panther version 1.74, and is available to all customers. Please share any bug reports and feature requests with your Panther support team.
  1. 1.
    In the left-hand navigation bar of your Panther Console, click Build > Detections.
  2. 2.
    Click on the name of the detection you want to use in a Data Replay.
  3. 3.
    Scroll down to the Data Replay tile, within the Test section.
    The Data Replay section of the Panther console shows a Log Types field, with OneLogin.Events selected, as well as a field that has two tabbed options: Data(GB) and Time. Data(GB) is currently selected, with a value of 20GB. At the bottom is a blue "Create Replay" button.
    ​
  4. 4.
    In the Log Types field, select the log types you wish to replay for the detection.
  5. 5.
    On the right side of the Data Replay section, select Time or Data(GB), depending on how you want to limit the set of data your replay will run over:
    • Time: Choose this option to run a replay on the data collected during a specified period of time.
      • Click the Date Start or Date End fields to open the date modal. You can select a preset day range or pick custom dates in the available range:
      The date modal displays a calendar view, and fields at the bottom for "From Time" and "To Time."
    • Data(GB): Choose this option to run a replay on a data set for a specified size (in gigabytes.)
      • Select one of the options in the Data dropdown.
      • Note that the data set is not guaranteed to be the most recent n number of gigabytes.
  6. 6.
    Click Create Replay.
    • The estimated run time and data size will be displayed.
    • Note: If you have made any changes to the rule, you must save your changes before starting a Replay.
As the Replay processes, the progress will be displayed live, including the estimated volume processed and matched events. You can exit this page at any time without cancelling the Replay.
If you have a high rate of matched events, we recommend stopping the replay to tune your detection.
The Panther Console displays a message that says the Replay is processing.

Viewing replay results

After the Replay is complete, you will see an alert summary on the detection's page that includes total alerts, total rule errors, volume processed, and total matched events. All alerts will be shown as they would appear if the alert had been enabled during this range of time.
On this page, you can filter alerts and view the events associated with each alert.
The Replay Test Summary shows the number of alerts, rule errors, estimated volume processed, and total matched events.
Click on an alert for more information. On the alert's page, click the Details tab to see which destinations the alerts would have been sent to if the rule was live. This page also includes the start time, end time, and duration of the replay.
Note: No alerts are sent to their destinations while replaying data.
The alert's Details page shows more information, including the destination where you would expect the alert to be sent.
Previous
Testing
Next
Framework Mapping and MITRE ATT&CK® Matrix
Last modified 3d ago