# PantherFlow Examples: Threat Hunting Scenarios ## Pivoting from an alert to a log search Let's say we've received a [Wiz](https://docs.panther.com/pantherflow/example-queries/broken-reference) alert about an EC2 instance being potentially misconfigured. From the alert, we can pull the associated AWS instance IDs. Then, we can search across all of our AWS logs for activity on those instances. ```kusto let alert_data = panther_signals.public.signal_alerts | where p_event_time > time.ago(7d) | where p_alert_id == '00411934608291e0fccd928590194fd6' | summarize instances = arrays.flatten(agg.make_set(p_any_aws_instance_ids)), mintime = agg.min(p_event_time), maxtime = agg.max(p_event_time); union panther_logs.public.aws* | where p_event_time between time.parse_timestamp(toscalar(alert_data | project mintime)) - 30m .. time.parse_timestamp(toscalar(alert_data | project maxtime)) + 30m | where arrays.overlap(p_any_aws_instance_ids, toscalar(alert_data | project instances)) ``` The statements above leverage: * [`let` statement](https://docs.panther.com/statements#let-statements) functionality * Operators: [`union`](https://docs.panther.com/pantherflow/operators/union), [`project`](https://docs.panther.com/pantherflow/operators/project), [`summarize`](https://docs.panther.com/pantherflow/operators/summarize), [`where`](https://docs.panther.com/pantherflow/operators/where), and [`between`](https://docs.panther.com/expressions#between-comparisons) * Functions: [`arrays.flatten()`](https://docs.panther.com/functions/array#arrays.flatten), [`arrays.overlap()`](https://docs.panther.com/functions/array#arrays.overlap), [`agg.make_set()`](https://docs.panther.com/functions/aggregation#agg.make_set), [`agg.min()`](https://docs.panther.com/functions/aggregation#agg.min), [`agg.max()`](https://docs.panther.com/functions/aggregation#agg.max), [`time.ago()`](https://docs.panther.com/functions/date-time#time.ago), [`time.parse_timestamp()`](https://docs.panther.com/functions/date-time#time.parse_timestamp), and [`toscalar()`](https://docs.panther.com/functions/other#toscalar) ## Pulling IPs from one table to search in another While threat hunting, it's common to need to pull values from one table and pivot to searching those values in another table. The queries below fetch IP addresses from the VPC Flow logs table, then search for them in Okta System logs. ```kusto let IPs = panther_logs.public.aws_vpcflow | where p_event_time > time.ago(5d) | where flowDirection == 'ingress' | summarize IP = agg.make_set( re.substr(srcAddr, '(\\d{1,3})\\.(\\d{1,3})\\.(\\d{1,3})\\.(\\d{1,3})') ); panther_logs.public.okta_systemlog | where p_event_time > time.ago(1d) | where arrays.overlap(toscalar(IPs), p_any_ip_addresses) ``` The statements above leverage: * [`let` statement](https://docs.panther.com/statements#let-statements) functionality * Operators: [`summarize`](https://docs.panther.com/pantherflow/operators/summarize) and [`where`](https://docs.panther.com/pantherflow/operators/where) * Functions: [`arrays.overlap()`](https://docs.panther.com/functions/array#arrays.overlap), [`agg.make_set()`](https://docs.panther.com/functions/aggregation#agg.make_set), [`re.substr()`](https://docs.panther.com/functions/regular-expression#re.substr), [`time.ago()`](https://docs.panther.com/functions/date-time#time.ago), and [`toscalar()`](https://docs.panther.com/functions/other#toscalar) ## CIDR matching with a regular expression This query searches AWS logs for an IP address that matches a regex expression. ```kusto union aws_alb , amazon_eks_audit, aws_cloudtrail, aws_s3serveraccess | where p_event_time > time.ago(7d) | extend ip = coalesce(clientIp, sourceIPs, sourceIPAddress, remoteip) | summarize events=agg.count() by ip, p_log_type | where re.matches(ip, "^34\\.222\\..+\\..+$") | sort events desc ``` The statements above leverage: * Operators: [`union`](https://docs.panther.com/pantherflow/operators/union), [`where`](https://docs.panther.com/pantherflow/operators/where), [`extend`](https://docs.panther.com/pantherflow/operators/extend), [`summarize`](https://docs.panther.com/pantherflow/operators/summarize), and [`sort`](https://docs.panther.com/pantherflow/operators/sort) * Functions: [`coalesce()`](https://docs.panther.com/functions/other#coalesce), [`agg.count()`](https://docs.panther.com/functions/aggregation#agg.count), and [`re.matches()`](https://docs.panther.com/functions/regular-expression#re.matches) Results: | events | ip | p\_log\_type | | ------ | ---------------- | ---------------- | | `5866` | `34.222.253.62` | `AWS.CloudTrail` | | `184` | `34.222.140.16` | `AWS.CloudTrail` | | `176` | `34.222.42.181` | `AWS.CloudTrail` | | `171` | `34.222.87.204` | `AWS.CloudTrail` | | `88` | `34.222.241.235` | `AWS.CloudTrail` | | ... | | | ## Investigating an alert for API key creation In this scenario, we've received an alert for a match on the Panther-managed [AWS User API Key Created](https://github.com/panther-labs/panther-analysis/blob/main/rules/aws_cloudtrail_rules/aws_iam_user_key_created.yml) detection, which runs over CloudTrail data and alerts when an AWS API key is created for an AWS user by another user. {% hint style="info" %} This threat scenario is also [walked through on video, on the Risky Business podcast](https://risky.biz/video/product-demo-the-pantherflow-piped-query-language-for-the-panther-siem/), as well as in the Panther blog: [Investigating Amazon EKS Privilege Escalation with PantherFlow](https://panther.com/blog/investigating-amazon-eks-privilege-escalation-with-pantherflow). {% endhint %}
In the upper-left corner is the Panther logo, and on the left is a navigation bar where Alerts is selected. The right side shows an alert for a detection "AWS User API Key Created"
The event tells us that an actor named `ariel.ropek` has created API keys for a new user called `snidely-whiplash`. Let's investigate to see if this behavior is a false positive or real compromise. 1. First, let's look at `ariel.ropek`'s activity during the hour surrounding the alert: ```kusto panther_logs.public.aws_cloudtrail | where p_event_time between time.parse_timestamp('2024-11-13 19:00') .. time.parse_timestamp('2024-11-13 20:00') | extend p_actor = strings.split(userIdentity.arn, '/')[arrays.len(strings.split(userIdentity.arn, '/'))-1] | where p_actor == 'ariel.ropek' | sort p_event_time desc ``` \ Results:
Under a header reading "4 events" is a table with various columns, like time, database, log type, and p_actor.
Interesting! We see that not only did `ariel.ropek` create a new user named `snidely-whiplash`, but that they attached an `AdministratorAccess` policy to it:
In a slide-out panel, a JSON log is shown. A node with the key "requestParameters" is circled.
2. Let’s add `snidely-whiplash` to our query to view their activity: ```kusto panther_logs.public.aws_cloudtrail | where p_event_time between time.parse_timestamp('2024-11-13 19:00') .. time.parse_timestamp('2024-11-13 20:00') | extend p_actor = strings.split(userIdentity.arn, '/')[arrays.len(strings.split(userIdentity.arn, '/'))-1] | where p_actor in ['ariel.ropek', 'snidely-whiplash'] | sort p_event_time desc ``` \ Results:
Under a "55 events" header is a table with various columns, like time, database, log type, and p_actor.
There are many more results when `snidely-whiplash` is included—we can see that user ran commands in EKS. They created a new role, associated the `AmazonEKSClusterAdminPolicy` to it, then assumed the role under a new session name, `snidely-whiplash-session`.
In a slide-out panel on the right, a JSON log is shown. Two fields are circled: eventName and policyArn.
3. Now that we know the user took action in EKS, we need to use [`union`](https://docs.panther.com/pantherflow/operators/union) to expand our search to include EKS Audit and Authenticator logs.\ \ CloudTrail, EKS Audit, and EKS Authenticator logs each have a different schema; however, we can use [`coalesce()`](https://docs.panther.com/functions/other#coalesce) to create a data model that maps these log sources to common `actor` and `action` fields: ```kusto union panther_logs.public.aws_cloudtrail, panther_logs.public.amazon_eks_audit, panther_logs.public.amazon_eks_authenticator | where p_event_time between time.parse_timestamp('2024-11-13 19:00') .. time.parse_timestamp('2024-11-13 20:00') | extend p_aws_arn = coalesce(userIdentity.arn, user.username, arn) | extend p_actor = strings.split(p_aws_arn, '/')[arrays.len(strings.split(p_aws_arn, '/'))-1] | extend p_action = coalesce(eventName, strings.cat(verb, ' ', objectRef.resource), path) | where p_actor in ['ariel.ropek', 'snidely-whiplash', 'snidely-whiplash-session'] | sort p_event_time desc ``` \ Results:
Under a "77 events" header is a table with various columns, including time, database, log type, and p_actor.
Looking at the `p_action` column, we can see that using `snidely-whiplash-session`, the user created a Kubernetes pod (`create pods`). Clicking into the full event, we see the pod they created is privileged:
On the right-hand side is a slide-out panel showing a JSON log. A "securityContext" node is circled.
\ At this point, we can conclude that this is very likely a malicious actor. In sum, we've discovered: 1. An AWS user (`ariel-ropek`) created a new user (`snidley-whiplash`) with admin privileges. 2. `snidley-whiplash` pivoted into EKS and elevated their privileges there, becoming a cluster admin. 3. They created a privileged pod in EKS. 4. We can view the full attack chain in a chart with [`visualize`](https://docs.panther.com/pantherflow/operators/visualize): ```kusto union panther_logs.public.aws_cloudtrail, panther_logs.public.amazon_eks_audit, panther_logs.public.amazon_eks_authenticator | where p_event_time between time.parse_timestamp('2024-11-13 19:00') .. time.parse_timestamp('2024-11-13 20:00') | extend p_aws_arn = coalesce(userIdentity.arn, user.username, arn) | extend p_actor = strings.split(p_aws_arn, '/')[arrays.len(strings.split(p_aws_arn, '/'))-1] | extend p_action = coalesce(eventName, strings.cat(verb, ' ', objectRef.resource), path) | where p_actor in ['ariel.ropek', 'snidely-whiplash', 'snidely-whiplash-session'] | summarize events = agg.count() by p_actor, p_action | visualize bar xcolumn = p_action | sort p_actor ``` \ Results:
A bar chart titled "p_action vs events" is shown.
In the visualization above, the attack chain can be tracked from right to left, starting with `ariel.ropek`'s actions.