Monitoring Log Sources

How to monitor data metrics, log source health, and schemas for individual log sources.

Overview

Once a log source is onboarded in Panther, you can monitor data processing metrics and the log source's health on the log source's operations page. You can also attach new schemas and view raw data associated with the log source.

How to monitor log sources

Finding the Log Source operations page

1. Log in to your Panther Console.

2. On the left sidebar menu, click Configure > Log Sources.

3. Click on any log source in the list to view that log source's operations page.

Viewing the Log Source Overview

The Overview tab actionable log source metrics and health information. The Basic Info section displays frequently used, click-to-copy metadata, as well as the ingestion pipeline status.

Source Statuses

All statuses display the most recent timestamp for when that event occurred.

• Source Created: Log source configuration is complete and Panther should be able to ingest log data from the source.

• Data Received: Log data is available for normalization.

• Data Ingested: Log data has been processed and normalized.

Overview Stats

All data visualizations reflect the time period selected in the date picker.

• Data Ingested: The amount of uncompressed log data that has been successfully processed and normalized for the selected time period.

• % of Total Data Ingested: The amount of uncompressed log data ingested by this log source vs. the amount of all log data ingested for the selected time period.

• Processed Events: The number of successfully processed and normalized events for the selected time period.

• Data Processed by Log Type Chart: The amount of data ingested by the log type (the type of data).

• Events Processed by Log Type Chart: The number of events processed by the log type (the type of data).

  • Note: There is a corresponding table for each log type.

Viewing the Log Source Schemas

At the top of the operations page, click the Schemas tab to see all schemas that are parsing and normalizing the data for this source.

If you are looking at a custom log source that uses a Data Transport, you will be able to add or remove schemas here.

View in Query Builder

On the right side of a schema in the list, click View Data. You will be redirected to the Query Builder with pre-filled selections that you can run to view data associated with the schema for the source.

View log source raw data

This feature is only available to log sources onboarded with the S3 transport method.

The permission View Log Source Raw Data is required. By default, only users with the Admin role have this permission.

When onboarding an S3 log source with or without log types, you get direct access to the log source's raw data that Panther receives.

To access the log source's raw data:

1. Log in to the Panther Console.

2. In the left sidebar menu, click Configure > Log Sources.

3. Click the Schemas tab then the Edit button

In the Schema Configuration, you can view events for a specific time range. Optionally, you can limit results by applying filters for the S3 key prefix and a search string to match each event.

Viewing the Log Source Health

At the top of the operations page, click the Health tab to see all system health alerts related to the health of the log source you are viewing.

This page displays errors related to data classification, log drop-off, S3 Get.Object, and permissions.

You can learn more about these error types in the documentation: System Health Notifications.

Viewing the Log Source Configuration

In the upper right side of the operations page, click Configuration to see the log source's configuration details.

• Click Delete to delete the log source and associated configurations.

• Click Edit to update the existing configuration.