# Panther Analysis Tool Commands ## Overview You can manage your Panther detection content using the Panther Analysis Tool (PAT). PAT lets you [upload](#upload-uploading-packages-to-panther-directly), [test](#test-running-tests-with-pat), and [delete](#delete-deleting-rules-policies-or-saved-queries-with-pat) assets, among other actions. Each of the PAT commands accepts certain [options](#pat-command-options-sub-commands). For example, you can use [--filter](#filtering-pat-commands) with several of the commands to narrow the scope of the action. ## PAT commands See the full list of available PAT commands in the following codeblock. Beneath it, find additional information about several of the commands. PAT commands can be run using `panther_analysis_tool` or `pat`. [Learn more about these aliases here](https://docs.panther.com/panther-developer-workflows/detections-repo/install-configure-and-authenticate-with-pat#pat-cli-aliases). To understand which Panther permissions you need to execute each PAT command, see [Permissions required per command](#permissions-required-per-command). ``` $ pat --help Usage: pat [OPTIONS] COMMAND [ARGS]... Panther Analysis Tool: A command line tool for managing Panther policies and rules. ╭─ Options ───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ --version Show the version and exit │ │ --debug Enable debug mode │ │ --skip-version-check Skip Panther version check │ │ --install-completion Install completion for the current shell. │ │ --show-completion Show completion for the current shell, to copy it or customize the installation. │ │ --help Show this message and exit. │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯ ╭─ Commands ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮ │ release Create release assets for repository containing panther detections. Generates a file called │ │ panther-analysis-all.zip and optionally generates panther-analysis-all.sig │ │ test Validate analysis specifications and run policy and rule tests. │ │ debug Run a single rule test in a debug environment, which allows you to see print statements and use │ │ breakpoints. │ │ publish Publishes a new release, generates the release assets, and uploads them. Generates a file called │ │ panther-analysis-all.zip and optionally generates panther-analysis-all.sig │ │ upload Upload specified policies and rules to a Panther deployment. │ │ delete Delete policies, rules, or saved queries from a Panther deployment. │ │ update-custom-schemas Update or create custom schemas on a Panther deployment. │ │ test-lookup-table Validate a Lookup Table spec file. │ │ validate Validate your bulk uploads against your panther instance. │ │ zip Create an archive of local policies and rules for uploading to Panther. │ │ check-connection Check your Panther API connection │ │ benchmark Performance test one rule against one of its log types. The rule must be the only item in the working │ │ directory or specified by --path, --ignore-files, and --filter. This feature is an extension of Data Replay │ │ and is subject to the same limitations. │ │ enrich-test-data Enrich test data with additional enrichments from the Panther API. │ │ check-packs Ensure that packs don't have missing detections. │ │ init Initialize a new panther project │ │ update Update and merge the latest content from Panther Analysis with your own. Rerun this every time you want to │ │ update your content. Items with the same ID as a Panther Analysis Item and a BaseVersion field will be │ │ merged with the latest Panther Analysis Item. Items that have merge conflicts will be skipped to be │ │ resolved manually with the `merge` command. Use the --write-merge-conflicts flag to write all merge │ │ conflicts to their respective files instead of skipping them. │ │ install Install and enable an analysis item from Panther Analysis into your local repository. │ │ explore Explore the latest Panther Analysis content │ │ merge Merge an analysis item with the latest Panther Analysis content │ │ migrate Migrate all analysis items that are based off Panther Analysis content by adding a BaseVersion field and │ │ getting them up to date. │ │ fmt Format and standardize the code in the current directory to help with making less changes in other │ │ commands. │ ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯ ``` ### `init`: Preparing your Panther directory This command should only be run once, when first setting up your Panther directory. It: * Creates a `.gitignore` file, if needed, and adds to it * Enables `git rerere` * Creates a `.cache` directory, which locally stores content from `panther-analysis` for fast loading * Pulls the latest content from `panther-analysis` ### `migrate`: Transitioning content from a legacy management flow `migrate` tells PAT to start tracking existing analysis items for future updates. Running `pat migrate` looks in your repository, and: * For each analysis item with an ID that matches an ID in `panther-analysis`: * Adds a `BaseVersion` field to the YAML file. * Merges your item with the latest version from `panther-analysis`. * If there are merge conflicts, PAT leaves the item un-merged. To resolve the conflict, run `pat migrate `. * You can optionally use the `--auto-accept` flag to completely avoid merge conflicts.

Please note: pat migrate <ID> is for solving merge conflicts while you are still migrating. Items that ran into merge conflicts during pat migrate have not been migrated yet. This is very similar to pat merge <ID>, which is used to solve merge conflicts after running pat update. The merge command should not be used during the migration phase.

* For each analysis item with an ID that does not match an ID in `panther-analysis`, no change is made to the item, as it is considered custom. The output of this command is stored in `./migration_status.md`. ### `update`: Fetching and merging the latest panther-analysis content Rerun this every time you want to update your content. This command fetches the latest `panther-analysis` content, and merges it into your own. PAT knows that an item needs updating by determining if an analysis item has the same ID as an item in `panther-analysis` and if the `BaseVersion` is less than the latest version of the item in `panther-analysis`. If an item has the same ID as something in `panther-analysis` but has no `BaseVersion` field it will not receive updates. Likewise, if an item does not have a matching ID in `panther-analysis` but has a `BaseVersion` field, it will not receive updates. Items that have merge conflicts will not be updated as a part of `pat update`, and must be resolved manually with the `merge` command. A list of all the items that had merge conflicts will be printed out in the output of the `update` command. {% hint style="info" %} If you have merge conflicts arising from `pat update`, you can resolve them with `pat merge` (though this is not required to test and upload the content that was successfully updated). {% endhint %} Use the `--write-merge-conflicts` flag to write all merge conflicts to their respective files instead of skipping them (This may be useful if you're using AI to solve conflicts). Use `--auto-accept panthers` or `--auto-accept yours` to automatically resolve merge conflicts (note that the item could still have issues or failing tests even after resolving conflicts, each item should be reviewed after resolving). Use the `--preview` flag to do a dry-run of the update. This flag will have the command print the IDs of the items that will be updated and the items that will have merge conflicts without updating any of the files. ### `explore`: Viewing available `panther-analysis` content Run `pat explore` to view the `panther-analysis` content in your `.cache` directory. (This command fetches new p-a content if it is out of date with the latest p-a release.) You can search by content ID, type, or description. You can access the **Analysis Content** tree using `tab`. Press `enter` on any analysis type to filter by it. Press `enter` on any single analysis ID to filter by it.
The search bar can be used to filter the items in the table. You can search for plain text, which will filter items containing the text in their YAML or python files, case sensitive. You can also search using the filters available in the command line. For example, a search of `data "prod env" AnalysisType=rule Severity=Critical,High` would do a case sensitive search for any item with `data` in its YAML or python, `prod env` in its YAML or python, the analysis type of `rule`, and a severity of `Critical` or `High`. You can use single or double quotes to group words together in the search. All quotes must have their matching end quote to apply the search. Double quotes can be escaped with a backslash (`\`). Single quotes cannot be escaped. Pressing `enter` on a specific item will open its source code. For analysis types that have YAML and Python files, you will be shown both, side-by-side.
{% hint style="info" %} When viewing a certain item, you can use `^v` to install and enable it (rather than using the separate `pat install` command). {% endhint %} You can exit the explorer GUI with `ctrl+q` or `ctrl+c`. ### `install`: Copy and enable a `panther-analysis` item Run `pat install` to copy items from `panther-analysis` into your own repository. (To understand what you'd like to install, it's recommended to run `pat explore`.) If you install an analysis item that has a dependency (like a global helper or data model), the dependency will be installed (replicated in your own repository) as well. ### `fmt`: Format and standardize the code Run to format and standardize the code in the current directory. There are some formatting changes that are required as part of using PAT. This command lets you easily make formatting changes separately from all other changes the tool makes. Use this to keep logic changes and formatting changes independent of each other while interacting with PAT. Changes it makes to YAML files are: * Normalizes all boolean casing (e.g. `True` -> `true`) * Standardizes all indentation to 2 spaces * Replaces explicit `null`s with empty values (e.g. `Field: null` -> `Field:` ) * Replaces all inline style fields with block style, and JSON with block style * Removes any newlines within strings without explicit multiline markers (like `|` and `>`) ### `merge`: Merge an analysis item with the latest Panther Analysis version of the item {% hint style="info" %} `pat merge ` is for solving merge conflicts after a `pat update`. If you are still migrating code, use `pat migrate ` to migrate items with merge conflicts. {% endhint %} After running `pat update`, you might run into merge conflicts that prevent the analysis item from being cleanly updated. To merge your item with the latest version from Panther Analysis and solve merge conflicts, run `pat merge `, with `ID` being the ID of the analysis item you want to solve merge conflicts for. YAML merge conflicts will be solved inside a custom YAML merge conflict resolver tool. This will allow you to quickly compare things field by field, choosing your version or Panther's version to resolve the conflict. This tool allows you to avoid merge conflicts that occur from formatting changes, as this tool compares changes field by field instead of line by line. It will only compare top-level fields in your YAML. After any YAML conflicts, the command will let you solve Python merge conflicts. By default, this will happen inside of `vi`. You can specify a different editor to use by providing the `--editor` flag or setting the `EDITOR` environment variable. The editor must be available as a CLI command to be used. Some editors do not install their CLI commands by default, so please be sure to verify you have your preferred editor available via the CLI. After successfully solving the merge conflicts, the command will update the `BaseVersion` field in the YAML for you. If you prefer to solve merge conflicts differently, you can supply the following flags: * `--auto-accept`: Works the same as it does with other commands. It accepts `yours` or `panthers` as values. Will avoid merge conflicts by automatically solving with one side of the merge. * `--write-merge-conflicts`: Writes all the merge conflicts, unresolved, to their files. Doing so will not resolve any conflicts and will not avoid whitespace YAML conflicts. `BaseVersion` will be updated. ### `test`: Running unit tests Use PAT to load the defined specification files and evaluate unit tests locally: ```bash panther_analysis_tool test --path ``` To filter rules or policies based on certain attributes: ```bash panther_analysis_tool test --path --filter RuleID=Category.Behavior.MoreInfo ``` {% hint style="info" %} Running `pat test` for [correlation rules](https://docs.panther.com/detections/correlation-rules) and [Simple Detections](https://docs.panther.com/detections#simple-detections) requires an API token. See [Authenticating with an API token](https://docs.panther.com/panther-developer-workflows/detections-repo/install-configure-and-authenticate-with-pat#authenticating-with-an-api-token) for more information. {% endhint %} ### `debug`: Debug a test using print statements and breakpoints Use `debug` to run a specific unit test in debug mode: ```bash panther_analysis_tool debug My.RuleID "My unit test name" ``` See print statements: ```bash INFO: Testing analysis items in /panther-analysis/rules/debug Debug.RuleThatPrints Hello world! ``` See your errors with traceback info: ```bash INFO: Testing analysis items in /panther-analysis/rules/debug Debug.RuleWithError ERROR: Test exception for debug tracing File "/panther-analysis/rules/debug/rule_with_error.py", line 4, in rule sub_func() File "/panther-analysis/rules/debug/rule_with_error.py", line 7, in sub_func raise ValueError('Test exception for debug tracing') ``` ### `benchmark`: Evaluating rule performance You can use `benchmark` to test the performance of an existing or draft rule against one hour of data, for one log type. It can be particularly useful to iterate on a rule that is timing out. This is a long-running command intended to be used manually, as needed—not in a regular CI/CD workflow. The API token used with this command must be granted the "Read Panther Metrics" (also known as `SummaryRead`) and "Manage Rules" (also known as `RuleModify`) permissions. Because `benchmark` is an extension of [Data Replay](https://docs.panther.com/detections/testing/data-replay), it is subject to the same [limitations](https://docs.panther.com/detections/testing/data-replay#limitations). You must provide a single rule to `benchmark`, either by having just one rule in the working directory (`./` or `--path`), or through the use of `--ignore-files` or `--filter`. If you do not specify a certain hour of data, the system will select the historical hour with the highest volume of data. To specify a specific hour to run against, use `--hour`. (Most common time formats are supported, e.g., `2023-07-31T09:00:00-7:00`—minutes, seconds, etc. will be truncated). For example: ```bash panther_analysis_tool benchmark --hour ``` For a rule with multiple log types, one must be specified using `--log-type`. For example: ```bash panther_analysis_tool benchmark --log-type ``` The output of `benchmark` will be written to both stdout, and to the directory indicated by the `--out` option. ### `enrich-test-data`: Enriching test data with Enrichment content Use `enrich-test-data` to enrich the test content of your Rules and Scheduled Rules with data from connected [Enrichment providers](https://docs.panther.com/enrichment) and custom [Lookup Tables](https://docs.panther.com/enrichment/custom). This allows you to build more sophisticated test cases for detections that rely on enrichment content. `enrich-test-data` is simple to use, but may introduce substantial changes to your analysis YAML files. The command will modify files based on the following criteria: * If the Rule or Scheduled Rule does not have test cases, the YAML file will not be modified. * If the log type does not support enrichment, the YAML file will not be modified. * If the log type supports enrichment and there are test cases: * Test cases represented as inline JSON content will be reformatted into YAML. * The YAML file will be formatted according to common YAML conventions, using two spaces for indentation. Similar to other commands, `enrich-test-data` works from the current directory, recursively. If you run the command at the root directory of your `panther-analysis` copy, it will attempt to enrich all Rules and Scheduled Rules. To enrich content in a single directory, navigate to that directory before running the command. You can run `enrich-test-data` in PAT versions 0.26 and beyond using the following command: ```bash panther_analysis_tool enrich-test-data ``` The output of the command will be written to stdout, including a list of any Rules or Scheduled Rules that were enriched. ### `validate`: Ensuring detection content is ready to be uploaded The `validate` command verifies your detection content is ready to be uploaded to your Panther instance by running the same checks that happen during the upload process. Because some of these checks require configuration information in your Panther instance, `validate` makes an API call. To validate your detections against your Panther instance using PAT: 1. If you have not already, [generate an API token in your Panther Console](https://docs.panther.com/api#how-to-create-a-panther-api-token). 2. Run the following command: {% code fullWidth="false" %} ```bash panther_analysis_tool validate --path --api-token --api-host https://api..runpanther.net/public/graphql ``` {% endcode %} * You may exclude the `--api-token` and `--api-host` options if you are [setting configuration values](https://docs.panther.com/panther-developer-workflows/detections-repo/install-configure-and-authenticate-with-pat#configuring-pat) another way, i.e., by using environment variables or a configuration file. ### `zip`: Creating a package to upload to the Panther Console To create a package for uploading manually to the Panther Console, run the following command: ``` $ panther_analysis_tool zip --path tests/fixtures/valid_policies/ --out tmp [INFO]: Testing analysis packs in tests/fixtures/valid_policies/ AWS.IAM.MFAEnabled [PASS] Root MFA not enabled fails compliance [PASS] User MFA not enabled fails compliance [INFO]: Zipping analysis packs in tests/fixtures/valid_policies/ to tmp [INFO]: /tmp/panther-analysis-2020-03-23T12-48-18.zip ``` #### Uploading content in the Panther Console 1. In the lefthand side of the Panther Console, click **Detections**. 2. Click the **Upload** button in the upper right corner. 3. Drag and drop your .zip file onto the page, or click **Select file**.
### `upload`: Uploading packages to Panther directly {% hint style="info" %} Starting with PAT version 0.22.0, if you have authenticated with an API token and execute the `upload` command, an asynchronous bulk upload will automatically be performed, to prevent timeout issues. If you did not use an API token to authenticate, you can use the `--batch` option. The `--batch` option is only available in versions of PAT after 0.19.0. {% endhint %} The upload command uploads your detection content to your Panther instance. To use `upload`: 1. If you have not already, [generate an API token in your Panther Console](https://docs.panther.com/api#step-2-create-an-api-token). 2. Run `panther_analysis_tool test` to ensure your unit tests are passing. 3. Run the following command:\ `panther_analysis_tool upload --path --api-token --api-host https://api..runpanther.net/public/graphql` * You may exclude the `--api-token` and `--api-host` options if you are [setting configuration values](https://docs.panther.com/panther-developer-workflows/detections-repo/install-configure-and-authenticate-with-pat#configuring-pat) another way, i.e., by using environment variables or a configuration file. When using `upload`, detections and Lookup Tables with existing IDs are overwritten. Locally deleted detections will not automatically be deleted in your Panther instance on upload—they must be removed with the [`delete`](#delete-deleting-rules-policies-or-saved-queries) command (or manually deleted in your Panther Console). When using the CLI workflow, it's recommended to set a detection's `Enabled` property to `false`, instead of deleting. If you update the ID of an entity (i.e., the value of `RuleId` or `PolicyId`) and use `upload`—but do not also use `delete` to manually remove the old entity, both versions will exist in your Panther instance. If you intend to update the ID without creating a duplicate detection, use `delete` with the old ID. #### The `CreatedBy` detection field By default, detections uploaded in the CLI workflow (i.e., with [PAT](#upload-uploading-packages-to-panther-directly) or via the [REST API](https://docs.panther.com/panther-developer-workflows/api/rest)) are shown on the **Detections** list page in the Panther Console as created by Panther:
A table with headers and one row is shown. A header/row value pair (Created By / Panther) is circled.
If you'd like to customize the **Created By** value for a detection uploaded in the CLI workflow, you can add a `CreatedBy` key to your detection's YAML file. The value of `CreatedBy` can be: * A Panther user's email address: in the Console, the Created By field will show the associated user. * A Panther user's unique ID: in the Console, the Created By field will show the associated user. * This value can be looked up using the [list users REST API endpoint](https://docs.panther.com/api/rest/users#get-users). * An arbitrary text value: in the Panther Console, the provided value will be displayed with **(External)** appended to it.
A table with headers and four row values is shown. A "Created By" column with the values in the first three rows is circled.
The `CreatedBy` key can be used in definitions for Python rules, scheduled rules, correlation rules, simple detections, and policies. If you download a detection from the Panther Console: * If it was created in the Panther Console, it will not contain `CreatedBy`. * If it was uploaded in the CLI workflow: * With `CreatedBy`, it will contain `CreatedBy`. * Without `CreatedBy`, it will not contain `CreatedBy`. ### `delete`: Deleting Rules, Policies, or Saved Queries While `panther_analysis_tool upload --path ` will upload everything from ``, it will not delete anything in your Panther instance if you simply remove a local file from ``. Instead, you can use the `panther_analysis_tool delete` command to explicitly delete detections from your Panther instance.\ \ To delete a specific detection, you can run the following command: ```bash panther_analysis_tool delete --analysis-id MyRuleId ``` This will interactively ask you for a confirmation before it deletes the detection. If you would like to delete without confirming, you can use the following command: ```bash panther_analysis_tool delete --analysis-id MyRuleId --no-confirm ``` You can delete up to 1000 detections at once with PAT. ### `update-custom-schemas`: Creating or updating custom schemas Use `update-custom-schemas` to create or update custom schemas. {% hint style="warning" %} After using this command to create a schema, wait at least 15 minutes before using `upload` to upload detections that reference the new schema. {% endhint %} ## Permissions required per command Below is a mapping of permissions required for each command. See [Permission names in the Console and API](https://docs.panther.com/system-configuration/rbac#permission-names-in-the-console-and-api) for more information.
CommandRequired permission(s)
check-connectionGeneralSettingsRead
test (when testing detections that use Inline Filters)Bulk Upload OR Bulk Upload Validate OR View Rules
benchmarkSummaryRead, RuleRead, RuleModify
validateBulkUpload OR BulkUploadValidate
uploadBulkUpload
deletePolicyModify, RuleModify, DataAnalyticsModify
update-custom-schemasLogSourceRead, LogSourceModify
## PAT command options Each PAT command has options (also known as flags), which you can view by running `$ pat -h`.
Example output of pat test -h ``` $ panther_analysis_tool test -h usage: panther_analysis_tool test [-h] [--filter KEY=VALUE [KEY=VALUE ...]] [--minimum-tests MINIMUM_TESTS] [--path PATH] [--ignore-extra-keys IGNORE_EXTRA_KEYS] [--ignore-files IGNORE_FILES [IGNORE_FILES ...]] [--skip-disabled-tests] [--available-destination AVAILABLE_DESTINATION] [--sort-test-results] [--ignore-table-names] optional arguments: -h, --help show this help message and exit --filter KEY=VALUE [KEY=VALUE ...] --minimum-tests MINIMUM_TESTS The minimum number of tests in order for a detection to be considered passing. If a number greater than 1 is specified, at least one True and one False test is required. --path PATH The relative path to Panther policies and rules. --ignore-extra-keys IGNORE_EXTRA_KEYS Meant for advanced users; allows skipping of extra keys from schema validation. --ignore-files IGNORE_FILES [IGNORE_FILES ...] Relative path to files in this project to be ignored by panther-analysis tool, space separated. Example ./foo.yaml ./bar/baz.yaml --skip-disabled-tests --available-destination AVAILABLE_DESTINATION A destination name that may be returned by the destinations function. Repeat the argument to define more than one name. --sort-test-results Sort test results by whether the test passed or failed (passing tests first), then by rule ID --ignore-table-names Allows skipping of table names from schema validation. Useful when querying non-Panther or non-Snowflake tables --valid-table-names VALID_TABLE_NAMES [VALID_TABLE_NAMES ...] Fully qualified table names that should be considered valid during schema validation (in addition to standard Panther/Snowflake tables), space separated. Accepts '*' as wildcard character matching 0 or more characters. Example foo.bar.baz bar.baz.* foo.*bar.baz baz.* *.foo.* ```
If an option takes multiple arguments, you must pass each value individually with the option. For example, to pass two arguments to `--valid-table-names`: * Correct: `--valid-table-names table1 --valid-table-names table2` * Incorrect: `--valid-table-names table1 table2` If a multi-argument option accepts values in the format `KEY=VALUE1,VALUE2` (like [`--filter`](#filter-filtering-pat-commands)), each one must be passed to the flag separately (though the argument's value can still be represented as a comma-separated array). For example: * Correct: `--filter AnalysisType=policy,rule --filter Severity=High` * Incorrect: `--filter AnalysisType=policy,rule Severity=High` ### `--filter`: Filtering PAT commands The `test`, `zip`, `upload`, and `release` commands all support filtering. Filtering works by passing the `--filter` flag with a list of filters specified in the format `KEY=VALUE1,VALUE2`. The keys can be any valid field in a policy or rule. When using a filter, only analysis types that matches each filter specified will be considered. For example, the following command will test only items with the `AnalysisType` of policy AND the `Severity` of High: ``` panther_analysis_tool test --path tests/fixtures/valid_policies --filter AnalysisType=policy --filter Severity=High [INFO]: Testing analysis packs in tests/fixtures/valid_policies AWS.IAM.BetaTest [PASS] Root MFA not enabled fails compliance [PASS] User MFA not enabled fails compliance ``` The following command will test items with the `AnalysisType` policy OR rule, AND the `Severity` High: ``` panther_analysis_tool test --path tests/fixtures/valid_policies --filter AnalysisType=policy,rule --filter Severity=High [INFO]: Testing analysis packs in tests/fixtures/valid_policies AWS.IAM.BetaTest [PASS] Root MFA not enabled fails compliance [PASS] User MFA not enabled fails compliance AWS.CloudTrail.MFAEnabled [PASS] Root MFA not enabled fails compliance [PASS] User MFA not enabled fails compliance ``` When writing policies or rules that refer to the `global` analysis type, be sure to include them in your filter. You can include an empty string as a value in a filter, and it will mean the filter is only applied if the field exists. The following command will return an error, because the policy in question imports a global but the global does not have a severity so it is excluded by the filter: ``` panther_analysis_tool test --path tests/fixtures/valid_policies --filter AnalysisType=policy,global --filter Severity=Critical [INFO]: Testing analysis packs in tests/fixtures/valid_policies AWS.IAM.MFAEnabled [ERROR] Error loading module, skipping Invalid: tests/fixtures/valid_policies/example_policy.yml No module named 'panther' [ERROR]: [('tests/fixtures/valid_policies/example_policy.yml', ModuleNotFoundError("No module named 'panther'"))] ``` For this query to work as expected, we need to allow for the severity field to be absent: ``` panther_analysis_tool test --path tests/fixtures/valid_policies --filter AnalysisType=policy,global --filter Severity=Critical,"" [INFO]: Testing analysis packs in tests/fixtures/valid_policies AWS.IAM.MFAEnabled [PASS] Root MFA not enabled fails compliance [PASS] User MFA not enabled fails compliance ``` Filters work for the `zip`, `upload`, and `release` commands in the same way they work for the `test` command. ### `--minimum-tests`: Requiring a certain number of unit tests You can set a minimum number of unit tests with the `--minimum-tests` flag. Detections that don't have the minimum number of tests will be considered failing. If `--minimum-tests` is set to `2` or greater, it will also enforce that at least one test must return `True` and one must return `False`. In the example below, even though each of the rule tests passed, they're still considered failing because they do not have the correct test coverage: ``` panther_analysis_tool test --path tests/fixtures/valid_policies --minimum-tests 2 % panther_analysis_tool test --path okta_rules --minimum-tests 2 [INFO]: Testing analysis packs in okta_rules Okta.AdminRoleAssigned [PASS] Admin Access Assigned Okta.BruteForceLogins [PASS] Failed login Okta.GeographicallyImprobableAccess [PASS] Non Login [PASS] Failed Login -------------------------- Panther CLI Test Summary Path: okta_rules Passed: 0 Failed: 3 Invalid: 0 -------------------------- Failed Tests Summary Okta.AdminRoleAssigned ['Insufficient test coverage, 2 tests required but only 1 found.', 'Insufficient test coverage: expected at least one passing and one failing test.'] Okta.BruteForceLogins ['Insufficient test coverage, 2 tests required but only 1 found.', 'Insufficient test coverage: expected at least one passing and one failing test.'] Okta.GeographicallyImprobableAccess ['Insufficient test coverage: expected at least one passing and one failing test.'] ``` ## Frequently asked questions
Do I need a versions.yml file in my repository? No, you do not need to maintain or include `versions.yml` in your repository.
After I update a detection from upstream, do I need to do anything with PAT for it to be aware of my changes? No. Your changes are preserved as long as `BaseVersion` is present on the detection. PAT uses `BaseVersion` to track the upstream version your customizations are based on, so you do not need to update it manually. See [Getting updates of Panther-managed detections](https://docs.panther.com/panther-developer-workflows/detections-repo/pat#getting-updates-of-panther-managed-detections) for more details.
What is the current recommended tool for schema uploads and testing? Use `panther_analysis_tool update-custom-schemas` [to create or update custom schemas](https://docs.panther.com/panther-developer-workflows/detections-repo/pat/pat-commands#update-custom-schemas-creating-or-updating-custom-schemas). For local schema testing, use the [pantherlog tool](https://docs.panther.com/panther-developer-workflows/pantherlog).
Do I need an indexes/ folder in my repository? No. You do not need to include or maintain the `indexes/` folder in your repo. It contains human-readable reference documents that list which Panther-managed detections exist for each log type and what they detect. You can browse them anytime in the [public panther-analysis repo](https://github.com/panther-labs/panther-analysis/tree/main/indexes).
Do saved queries overwrite themselves by name when uploaded via PAT? Yes. The query name acts as the unique identifier. When you upload a query with the same name, it will be updated. Use `--query-id` with `pat delete` to manage queries in your repository.