Knowledge BaseCommunityRelease NotesTry Panther
Search
⌃K
Links

Saved and Scheduled Queries

Save and optionally schedule queries

Overview

You can avoid repeatedly creating the same queries in Panther's Data Explorer and Query Builder by saving your queries. You can also schedule queries created in Data Explorer, which allows you to then run results against a rule and alert on matches. This workflow includes the following features:
  • ​Create a Saved Query, a preserved SQL expression.
  • ​Create a Scheduled Query, a Saved Query that you can schedule to run on a designated interval.
  • ​Create a Scheduled Rule, a detection that's associated with a Scheduled Query. The data returned each time the query executes is run against the detection, alerting when matches are found.
By default, each Panther account is limited to 10 active Scheduled Queries. This limit is only precautionary, and can be increased via a support request. There is no additional cost from Panther for raising this limit, however you may incur extra charges from the database backend, depending on the volume of data processed.

How to create a Saved Query

You can save a query in Panther's Data Explorer or Query Builder. Queries saved in both tools are considered Saved Queries. Follow these instructions for how to save a query in Data Explorer, and these instructions for how to save a query in Query Builder.

How to create a Scheduled Query

A Scheduled Query is a Saved Query that has been configured to run on a schedule. Using the Panther Console, currently only Saved Queries created in Data Explorer can be scheduled—Saved Queries created in Query Builder cannot be scheduled. You can alternatively create and upload Scheduled Queries using the developer workflow.
Remember that creating a Scheduled Query alone won't run the returned data against detections or send alerts. To do this, also create a Scheduled Rule, and associate it with your Scheduled Query.
​Customer-configured Snowflake accounts: Your company will incur costs on your database backend every time a Scheduled Query runs. Please make sure that your queries can complete inside the specified timeout period. This does not apply to accounts that use Panther-managed Snowflake.
Using Data Explorer
Using developer workflows

How to create a Scheduled Query in Data Explorer

To learn how to schedule your Saved Query created in Data Explorer, follow one of the below sets of instructions:

How to create a Scheduled Query using developer workflows

Writing Scheduled Queries locally means creating metadata files that define SQL queries on your own machine. Upload the files to your Panther instance (typically via the Panther Analysis Tool) to control your Scheduled Query content.
​
We recommend managing your local detection files in a version control system like GitHub or GitLab.
It's best practice to create a fork of Panther's open-source analysis repository, but you can also create your own repo from scratch.

​

File setup

Each saved query consists of:
  • A YAML file (.yml or .json extension) containing metadata attributes of the Scheduled Query.
View an example Scheduled Query YAML file here.
​

Folder setup

If you group your queries into folders, each folder name must contain queries in order for them to be found during upload (using either PAT or the bulk uploader in the Console).
We recommend grouping queries into folders based on log/resource type. You can use the open source Panther Analysis repo as a reference.
​

Write the Scheduled Query

Write your Scheduled Query and save it in your folder as new-scheduled-query.yml:
AnalysisType: scheduled_query
QueryName: ScheduledQuery_Example
Description: Example of a scheduled query for PAT
Enabled: true
Query:
- Your query appears here
Tags:
- Your tags
Schedule:
CronExpression: "0 0 29 2 *"
RateMinutes: 0
TimeoutMinutes: 2
​

Upload the content with PAT

  • Use the PAT upload command: panther_analysis_tool upload --path <path-to-your-query> --api-token <your-api-token> --api-host https://api.<your-panther-instance-name>.runpanther.net/public/graphql
    • Replace the values:
      • <api-token> : The API key you generated.
      • <your-panther-instance-name> : The fairytale name of your instance (e.g. carrot-tuna.runpanther.net).
      • <path-to-your-query> : The path to your Saved Query on your own machine.
​
When your Scheduled Query is uploaded, each of the fields you would normally populate in the Panther Console will be auto-filled. See Scheduled Query Specification Reference for a complete list of required and optional fields.

How to use the Scheduled Query crontab

Panther's Scheduled Query crontab uses the standard crontab notation consisting of five fields: minutes, hours, day of month, month, day of week. Additionally, you will find a query timeout selector (with a maximum value currently set at 10 minutes). The expression will run on UTC.
The interpreter uses a subset of the standard crontab notation:
┌───────── minute (0 - 59)
│ ┌──────── hour (0 - 23)
│ │ ┌────── day of month (1 - 31)
│ │ │ ┌──── month (1 - 12)
│ │ │ │ ┌── day of week (0 - 6 => Sunday - Saturday)
│ │ │ │ │
↓ ↓ ↓ ↓ ↓
* * * * *
If you want to specify day by day, you can separate days with dashes (1-5 is Monday through Friday) or commas, for example 0,1,4 in the Day of Week field will execute the command only on Sundays, Mondays and Thursdays. Currently, we do not support using named days of the week or month names.
Using the crontab allows you to be more specific in your schedule than the Period frequency option:
The Cron expression screen displays options for selecting a time range for the scheduled query to run.

Using Saved and Scheduled Queries

How to delete or download a Saved Query

You can delete Saved Queries individually or in bulk. Note that if a Saved Query is scheduled (i.e., it's a Scheduled Query), it must be unlinked from any Scheduled Rules it's associated to in order to be deleted.
  1. 1.
    Log in to the Panther Console, then navigate to Investigate > Saved Queries.
  2. 2.
    In the list of Saved Queries, find the query or queries you'd like to download or delete. Check the box to the left of the name of each query.
  3. 3.
    At the top of the page, click either Download or Delete.
    The top of the Saved Queries page is shown, with two buttons: "Download" and "Delete"
    ​
    • If you clicked Download, a saved_queries.zip file will be downloaded.
    • If you clicked Delete, an Attention! modal will pop up. Click Confirm.
      A modal titled "Attention!" is shown, with the text, "Are you sure you want to delete these (2) selected Saved Queries" and there are "Cancel" and "Confirm" buttons
      ​

How to deactivate a Scheduled Query

  1. 1.
    Log in to the Panther Console, then navigate to Investigate > Saved Queries.
  2. 2.
    Find the Scheduled Query you'd like to deactivate, and in the upper right corner of its tile, click the three dots icon.
    The image shows a query from the list of queries in the Panther Console. In the right side, there is a red arrow pointing to the 3 dots icon.
    ​
  3. 3.
    In the dropdown menu, click Edit Query Metadata.
  4. 4.
    In the Update Query form, toggle the setting Is it active? to OFF to disable the query.
    The "Update Query" form is displayed. It contains fields for Query Name, Tags, Description, and Default Database. The toggle next to "Is it active?" is set to "Off."
    ​
  5. 5.
    Click Update Query to save your changes.

Update a Saved Query's metadata

To edit a Saved Query's name, tags, description, and default database (and, for Scheduled Queries, whether it's active, and the period or cron expression):
  1. 1.
    Log in to the Panther Console, then navigate to Investigate > Saved Queries.
  2. 2.
    Locate the query you'd like to edit, and click the three dots icon in the upper right corner of its tile.
    The 3 dots icon is expanded to a dropdown menu. The option "Edit query metadata" is highlighted.
    ​
  3. 3.
    In the dropdown menu, click Edit Query Metadata.
  4. 4.
    Make changes in the Update Query form as needed.
    The "Update query" form is displayed. It contains fields for Query Name, Tags, Description, and Default Database. Next to "Is this a scheduled query?" the toggle is set to "On." There is a blue button at the bottom labeled "Update query".
    ​
  5. 5.
    Click Update Query.

Search for Saved Queries

On the Saved Queries page, you can search for queries using:
  • The search bar at the top of the queries list
  • The date range selector in the upper right corner
  • The Filters option in the upper right corner
    • Filter by whether the query is scheduled, whether its active, its type (Native SQL or Query Builder), or by up to 100 tags.
in the Saved Queries list, use the date range or filters in the upper right corner to search for queries. In the image, the date range selector is circled and the Filters button is circled.
Click on the name of the Saved Query to be taken directly to Data Explorer (for Native SQL queries) or Query Builder (for Query Builder queries) with the query populated.

Use LIMITs in Scheduled Queries

In the Panther Data Lake settings page, you can optionally enable a setting that will check if a Scheduled Query has a LIMIT clause specified. Use this option if you're concerned about a Scheduled Query unintentionally returning thousands of results, potentially resulting in alert delays, Denial of Service (DoS) for downstream systems and general cleanup overhead from poorly tuned queries.
  1. 1.
    In the upper right corner of the Panther Console, click the gear icon. In the dropdown menu that appears, click General.
    The gear icon's dropdown menu is expanded, showing options for General, Users, User Roles, API Tokens, and API Playground.
    ​
  2. 2.
    Click the Data Lake tab.
  3. 3.
    Scroll down to the Scheduled Queries header. Below the header, you will see the LIMIT clause toggle setting:
    At the top, a tab labeled Data Lake is selected. Near the bottom of the screen, there is a header called "Scheduled Queries." The option "LIMIT Clause for Scheduled Queries" is set to "Off."
    ​
  4. 4.
    Toggle the LIMIT Clause for Scheduled Queries setting to ON to start enforcing LIMITs in Scheduled Queries.
    The toggle next to "LIMIT Clause for Scheduled Queries" is set to "On."
    ​
When this field is set to ON, any new Scheduled Queries marked as active cannot be saved unless a LIMIT clause is specified in the query definition.
​
The image shows the query creation screen. There is a red banner at the top that says "Unable to create Saved query. This scheduled query does not contain a LIMIT clause in the SQL expression. Update the SQL expression to add a LIMIT clause to save this scheduled query."
​
Existing Scheduled Queries without a LIMIT clause will appear with a warning message in the list of Saved Queries, and edits cannot be saved unless a LIMIT clause is included.
A Scheduled Query without a LIMIT clause shows a warning banner that says "This Scheduled Query does not contain a LIMIT clause in the SQL expression."
The setting only checks for the existence of a LIMIT clause anywhere in the Saved Query. It does not check specifically for outer LIMIT clauses.

Scheduled Query specification reference

Required fields are in bold.
A complete list of scheduled query specification fields:
Scheduled Query Field Name
Description
Expected Value of Scheduled Query Field
AnalysisType
Indicates whether this analysis is a Rule, Policy, Scheduled Query, or global.
scheduled_query
QueryName
A friendly name to show in the UI.
String
Enabled
Whether this rule is enabled.
Boolean
Tags
Tags used to categorize this rule.
List of strings
Description
A brief description of the rule.
String
Query
A query that can run on any backend. If this field is specified, you should not specify a SnowflakeQuery or a AthenaQuery.
String
SnowflakeQuery
A query specifically for a Snowflake backend.
String
AthenaQuery
A query specifically for Athena.
String
Schedule
The schedule that this query should run. Expressed with a CronExpression or in Rate Minutes. TimeoutMinutes is required to release the query if it takes longer than expected. Note that cron and rate minutes are mutually exclusive.
CronExpression: "0 0 29 2 *"
RateMinutes: 0
TimeoutMinutes: 2
Map

​

Previous
Standard Fields
Next
Scheduled Query Examples
Last modified 3d ago