# Global Helper Functions ## Overview A common pattern in programming is to extract repeated code into helper functions—Panther supports this pattern with the `global` analysis type. Panther provides a number of [built-in global helpers](#built-in-globals), and you can also [add your own custom globals](#adding-custom-globals). Learn more about certain [common helpers](#common-helpers) below. {% hint style="warning" %} Global helpers are not best suited to frequent changes. If you *do* need to make frequent changes, consider instead using [Custom Lookup Tables](https://docs.panther.com/enrichment/custom), which support automatic syncing with S3 and don't require code changes within Panther for updates. {% endhint %} ## Panther-managed globals By default, Panther provides the following collections of built-in global helpers: * [`panther_base_helpers`](https://github.com/panther-labs/panther-analysis/blob/main/global_helpers/panther_base_helpers.py): Contains various generic and log source-specific helpers. Learn more about certain functions defined in `panther_base_helpers` below, in [Common helpers](#common-helpers). * [`panther_detection_helpers`](https://docs.panther.com/detections/rules/caching#cache-helper-functions-in-panther_detection_helpers): Provides caching functions. Learn more about the `panther_detection_helpers` package [on Python Rules Caching](https://docs.panther.com/detections/rules/caching#cache-helper-functions-in-panther_detection_helpers). ### Customizing Panther-managed globals Certain globals require configuration input from you, e.g., to provide values for an allowlist. If you need to make more significant changes, such as modifying logic or adding custom methods, it's generally recommended to [create a new global](#adding-custom-globals) for any. This reduces the likelihood of having to deal with complex merge conflicts when updating your detection sources. If you'd still like to modify the global helper, see the Panther Knowledge Base article: [What’s the best way for me to customize Panther-managed global helper functions?](https://help.panther.com/articles/3976369474-managing-panther-global-helpers-overwriting-vs-cloning) ## Using globals ### Viewing globals {% tabs %} {% tab title="Console" %} * To view Panther-provided and custom helper files in the Console, in the left-hand navigation bar of your Panther Console, click **Detections,** then click the **Helpers** tab.
An arrow is drawn from "Detections" to "Helpers," and a page titled "Helpers is shown, with a table with various entries, including "crowdstrike_event_streams_helpers" and "gcp_environment".
{% endtab %} {% tab title="CLI workflow" %} * To view Panther-provided helper files in the CLI workflow, see the [`global_helpers` directory of the `panther-analysis` GitHub repository](https://github.com/panther-labs/panther-analysis/tree/main/global_helpers). {% endtab %} {% endtabs %} ### Adding custom globals New globals can be created from the [Panther Analysis Tool](https://docs.panther.com/panther-developer-workflows/detections-repo/pat) or in your Panther Console. {% hint style="warning" %} It is highly discouraged to make external API requests from within your detections in Panther. In general, detections are processed at a very high scale, and making API requests can overload receiving systems and cause your rules to exceed the [15-second runtime limit](https://docs.panther.com/detections/rules/..#rule-errors-and-scheduled-rule-errors). {% endhint %} {% tabs %} {% tab title="Panther Console" %} To create a new global in the Panther Console: 1. In the left-hand navigation bar of your Panther Console, click **Detections**. 2. Click the **Helpers** tab. 3. In the upper right corner, click **Create New**.\ ![On the Helpers page in the Panther Console, there is a blue "Create New" button in the upper right.](https://4011785613-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LgdiSWdyJcXPahGi9Rs-2910905616%2Fuploads%2Fgit-blob-844434c91c2f59fec117f94d881083449c99145a%2FScreen%20Shot%202022-08-02%20at%2012.05.56%20PM.png?alt=media) 4. Type your Python functions, then click **Create**. This global can now be imported in your rules or policies.
The Helper Settings page shows fields for Helper Name and Description. Under Helper Definition, there is a code block with sample Python code written in it.
{% endtab %} {% tab title="Panther Analysis Tool" %} Global functions allow common logic to be shared across either rules or policies. To declare them as code, add them into the `global_helpers` folder with a similar pattern to rules and policies. {% hint style="info" %} Globals defined outside of the `global_helpers` folder will not be loaded. {% endhint %} 1. Create your Python file (`global_helpers/acmecorp.py`): ```python from fnmatch import fnmatch RESOURCE_PATTERN = 'acme-corp-*-[0-9]' def matches_internal_naming(resource_name): return fnmatch(resource_name, RESOURCE_PATTERN) ``` 2\. Create your specification file: ```yaml AnalysisType: global GlobalID: acmecorp Filename: acmecorp.py Description: A set of helpers internal to acme-corp ``` 3\. Use this helper in a policy (or a rule): ```python import acmecorp def policy(resource): return acmecorp.matches_internal_naming(resource['Name']) ``` {% endtab %} {% endtabs %} ### Importing globals in detections Import global helpers using an `import` statement at the top of your analysis file, then call the helper as if it were any other Python library. For example: ```python import panther_aws_helpers def rule(event): return event['name'] == 'test-bucket' def title(event): # Lookup the account name from an account Id account_name = panther_aws_helpers.lookup_aws_account_name(event['accountId']) return 'Suspicious request made to account ' + account_name ``` ### Removing references to helper functions from detections * If you decide to remove dependencies from your detections, we recommend [staggering the changes](https://help.panther.com/Detections/Detection_Features/Why_do_I_receive_rule_import_errors_after_removing_helper_functions_from_my_Panther_detection%3F). ## Common helpers ### `deep_get()` {% hint style="warning" %} `deep_get()` is also available as a [function on the event object itself](https://docs.panther.com/detections/rules/python/..#deep_get). For convenience, it's recommended to use that event function instead of this global helper function. {% endhint %} Located in [panther\_base\_helpers](https://github.com/panther-labs/panther-analysis/blob/master/global_helpers/panther_base_helpers.py). `deep_get()` can be used to return keys that are nested within Python dictionaries. This function is useful for safely returning nested keys and avoiding an `AttributeError` when a key is not present. If the key you are trying to access is nested inside a list, consider using [`deep_walk()`](#deep_walk) instead. #### Example With the following JSON, the deep\_get function would return the value of result. ```python { "outcome": { "reason": "VERIFICATION_ERROR", "result": "FAILURE" }} ``` ```python deep_get(event, "outcome", "result") == "FAILURE" ``` This can be found in the [Geographically Improbable Okta Login](https://github.com/panther-labs/panther-analysis/blob/cd220c87982011d4ad156c7daecd2857c358d154/rules/okta_rules/okta_geo_improbable_access.py) detection. #### default `deep_get()` takes in an optional `default` parameter. If a key is not present at the expected location or the value at that location is `None`, the default value will be returned. ```python deep_get(event, "outcome", "nonexistent_key", default="Key Not Found") == "Key Not Found" ``` ### `deep_walk()` {% hint style="warning" %} `deep_walk()` is also available as a [function on the event object itself](https://docs.panther.com/detections/rules/python/..#deep_walk). For convenience, it's recommended to use that event function instead of this global helper function. {% endhint %} Located in [panther\_base\_helpers](https://github.com/panther-labs/panther-analysis/blob/3ecca4a198646ab6dcad0320968fe075420572a7/global_helpers/panther_base_helpers.py#L313-L370). `deep_walk()` can be used to return values associated with keys that are deeply nested in Python dictionaries, which may contain any number of dictionaries or lists. This functionality is the key differentiator between `deep_walk()` and [`deep_get()`](#deep_get). As with `deep_get()`, this traversal is safe and will avoid any exceptions or errors. In the event that a key is not present in the structure, the default value is returned. #### Example With the following object, `deep_walk()` would return the value of `very_nested_key`: 
{"key": {"multiple_nested_lists_with_dict": [[[{"very_nested_key": "very_nested_value"}]]]}}
```python deep_walk(event, "key", "multiple_nested_lists_with_dict", "very_nested_key", default="") == "very_nested_value" ``` This can be found in the [GCP Service Account Access Denied](https://github.com/panther-labs/panther-analysis/blob/master/rules/gcp_audit_rules/gcp_service_account_access_denied.py) detection. #### default Like `deep_get()`, `deep_walk()` takes an optional `default` parameter. If a key is not present in the provided event, the key is `None`, or the key is an empty list, the default value is returned instead. Using the above example: ```python deep_walk(event, "key", "multiple_nested_lists_with_dict", "very_nested_nonexistent_key", default="") == "" ``` #### return\_val Unlike `deep_get()`, `deep_walk()` can return three distinct value classifications: * `all` * `first` * `last` **`all`** By default, `deep_walk()` will return `all` values for a given key. This is useful for cases where a key is duplicated in an event; however, if the number of values returned by `all` is one, only that value is returned. For example: ```json {"key": {"inner_key": [{"nested_key": "nested_value"}, {"nested_key": "nested_value2"}]}} ``` ```python deep_walk(event, "key", "inner_key", "nested_key", default="") == ['nested_value', 'nested_value2'] ``` When using `all` and returning multiple values, the elements in the list can be accessed like any other Python list. **`first`** To return only the first found value for a key, specify `return_val="first"`. For example: ```python deep_walk(event, "key", "inner_key", "nested_key", default="", return_val="first") == "nested_value" ``` **`last`** To return only the last found value for a key, specify `return_val="last"`. For example: ```python deep_walk(event, "key", "inner_key", "nested_key", default="", return_val="last") == "nested_value2" ``` ### `is_ip_in_network()` Located in [panther\_base\_helpers](https://github.com/panther-labs/panther-analysis/blob/master/global_helpers/panther_base_helpers.py). `is_ip_in_network()` is a function to check if an IP address is within a list of IP ranges. This function can be used with a list of known internal networks for added context to the detection. Example: ```python SHARED_IP_SPACE = [ "192.168.0.0/16", ] if is_ip_in_network(event.get("ipaddr"), SHARED_IP_SPACE): ... ``` An example can be found in the [OneLogin Active Login Activity](https://github.com/panther-labs/panther-analysis/blob/cd220c87982011d4ad156c7daecd2857c358d154/rules/onelogin_rules/onelogin_active_login_activity.py) detection. ### `pattern_match()` Located in [panther\_base\_helpers](https://github.com/panther-labs/panther-analysis/blob/master/global_helpers/panther_base_helpers.py). Wrapper around [fnmatch](https://docs.python.org/3/library/fnmatch.html) for basic pattern globs. This can be used when simple pattern matching is needed without the requirement of using regex. Example: With the following JSON the pattern\_match() function would return true. ```python { "operation": "REST.PUT.OBJECT" } ``` ```python pattern_match(event.get("operation", ""), "REST.*.OBJECT") ``` An example can be found in the AWS S3 Access Error detection. ### `pattern_match_list()` Located in [panther\_base\_helpers](https://github.com/panther-labs/panther-analysis/blob/master/global_helpers/panther_base_helpers.py). Similar to `pattern_match()`, `pattern_match_list()` can check that a string matches any pattern in a given list. Example: With the following JSON the pattern\_match\_list() function would return true. ```python { "userAgent": "aws-sdk-go/1.29.7 (go1.13.7; darwin; amd64) APN/1.0 HashiCorp/1.0 Terraform/0.12.24 (+https://www.terraform.io)" } ``` ```python ALLOWED_USER_AGENTS = { "* HashiCorp/?.0 Terraform/*", # 'console.ec2.amazonaws.com', # 'cloudformation.amazonaws.com', } pattern_match_list(event.get("userAgent"), ALLOWED_USER_AGENTS) ``` An example can be found in the [AWS EC2 Manual Security Group Change](https://github.com/panther-labs/panther-analysis/blob/cd220c87982011d4ad156c7daecd2857c358d154/rules/aws_cloudtrail_rules/aws_ec2_security_group_modified.py) detection. ### `aws_strip_role_session_id()` Located in [panther\_aws\_helpers](https://github.com/panther-labs/panther-analysis/blob/main/global_helpers/panther_aws_helpers.py). `aws_strip_role_session_id()` strips the session ID our of the arn. Example: With the following value, `aws_strip_role_session_id()` would return `arn:aws:sts::123456789012:assumed-role/demo` ```python { "arn": "arn:aws:sts::123456789012:assumed-role/demo/sessionName" } ``` ```python aws_strip_role_session_id(user_identity.get("arn", "")) ``` An example can be found in the [AWS Unauthorized API Call](https://github.com/panther-labs/panther-analysis/blob/main/rules/aws_cloudtrail_rules/aws_unauthorized_api_call.py) detection. ### `is_base64()` Located in [panther\_base\_helpers](https://github.com/panther-labs/panther-analysis/blob/master/global_helpers/panther_base_helpers.py). `is_base64()` checks if the string is base64 encoded, and if so, returns the decoded string. If not, it returns an empty string. See an example in the [`Crowdstrike.Base64EncodedArgs` detection](https://github.com/panther-labs/panther-analysis/blob/release/rules/crowdstrike_rules/crowdstrike_base64_encoded_args.py). ### `get_string_set()` Located in [panther\_detection\_helpers](https://pypi.org/project/panther-detection-helpers/). Learn more about the `panther_detection_helpers` package [on Python Rules Caching](https://docs.panther.com/detections/rules/caching#cache-helper-functions-in-panther_detection_helpers). `get_string_set` is used to get a value from a Panther-managed cache based on its key. This is useful to retrieve state between detection invocations. [An example used in panther-analysis can be found here](https://github.com/panther-labs/panther-analysis/blob/a0b6039dbe1966d29ddd80937b9a790a59ef9cfe/rules/standard_rules/impossible_travel_login.py#L105-L106). ### `put_string_set()` Located in [panther\_detection\_helpers](https://pypi.org/project/panther-detection-helpers/). Learn more about the `panther_detection_helpers` package [on Python Rules Caching](https://docs.panther.com/detections/rules/caching#cache-helper-functions-in-panther_detection_helpers). `put_string_set` is used to store a value into a Panther-managed cache based on its key. This is useful to store state between detection invocations. [An example used in panther-analysis can be found here](https://github.com/panther-labs/panther-analysis/blob/a0b6039dbe1966d29ddd80937b9a790a59ef9cfe/rules/standard_rules/impossible_travel_login.py#L111-L115).