Data Explorer SQL Search Examples

Overview

These examples demonstrate how to use Data Explorer in the Panther Console to investigate and answer questions using the data within your Security Data Lake.

For examples on syntax and correct use of statements for Snowflake environments, please see the Snowflake documentation: SQL Command Reference and Querying semi-structured data.

The examples on this page apply to all log types, while the sub-pages contain examples that apply to specific log types:

• VPC Flow logs

• CloudTrail logs

• GuardDuty logs

• S3 Access logs

• NGINX and ALB Access logs

• GitHub Audit logs

Considerations

All queries should control the result size. This can be done with a LIMIT or GROUP BY clause.

Did this IP address have any activity in my network (and in what logs)?

This is often one of the first questions asked in an investigation. Given there is some known bad indicator such as an IP address, then if there is related activity in your network/systems, a detailed investigation will be needed.

Snowflake's contains method can be a partial match for the data, and Snowflake's array_contains method requires an exact match for the data in question.

In the example below, the Panther field p_any_ip_addresses is used. Panther extracts a number of common indicator fields over all data sources into standard fields (see Panther Fields).

SELECT
 p_log_type, count(1) AS row_count
FROM panther_views.public.all_logs
WHERE p_occurs_between('2021-01-01', '2021-01-02')
     AND array_contains('95.123.145.92'::variant, p_any_ip_addresses)
GROUP BY p_log_type
LIMIT 100

What are the top 10 IPs by row count over all logs?

Ranking activity (top or bottom) is a useful technique to gain visibility into a network. High ranking activity might help locate IP addresses involved in a DDOS attack while low ranking (change ORDER BY to ASC) might highlight sneaky activity.

SELECT
  ip.value as ip,
  count(1) as total_rows
FROM panther_views.public.all_logs, LATERAL FLATTEN(input => p_any_ip_addresses) ip
WHERE p_occurs_between('2021-01-01', '2021-01-02')
GROUP BY ip.value
ORDER BY  total_rows DESC
LIMIT 100

What are the top 10 IPs by log type over all logs?

This is a variant of the above query where we are ranking the IPs by how many data sources they show activity. This shows the degree of "reach" the IP address has over all your systems.

SELECT
  ip,
  count(distinct p_log_type) as datasets
FROM
(
SELECT
  p_log_type,
  ip. value as ip
FROM panther_views.public.all_logs, LATERAL FLATTEN(input => p_any_ip_addresses) ip
WHERE p_occurs_between('2021-01-01', '2021-01-02')
GROUP BY ip.value, p_log_type
)
GROUP BY ip
ORDER BY  datasets DESC
LIMIT 100