Data Transports

Panther integrates with various common data transport log ingestion sources

Overview

A Data Transport is a type of log source that sends log types that are not natively supported by Panther—i.e., custom log types. Using Data Transports, you can process custom data types through Panther’s log processing pipeline, map existing detections to custom data types, and map data models to custom data types.

In addition to using a Data Transport to onboard your custom logs, you'll need to create a custom schema to normalize and classify the data.

Panther-supported Data Transports

Panther currently supports the following Data Transport methods:

• HTTP (Webhook)

• AWS

  • S3

  • CloudWatch

  • SQS

    • SNS

  • Amazon EventBridge

• Google Cloud

  • Cloud Storage (GCS)

  • Pub/Sub

• Azure Blob Storage

Ingesting compressed data in Panther

The following Data Transport mechanisms support ingesting compressed data (according to one of the below algorithms):

• HTTP

• Azure Blob Storage

• AWS S3

• GCS

Panther will transparently decompress compressed data, meaning no extra headers are required. Panther's decompression works by first determining the compression algorithm each file was compressed with. This is not inferred from the file extension or metadata, but rather the content of the file itself.

The inner data of a compressed payload must match the stream type you've configured for your Data Transport source.

Supported compression algorithms

Panther supports the following compression formats:

• gzip

• zstd

  • Panther only supports zstd data that was compressed without the use of a dictionary.

Supported columnar formats

Parquet file ingestion is in open beta starting with Panther version 1.96, and is available to all customers. Please share any bug reports and feature requests with your Panther support team.

Panther can transparently detect and ingest Parquet files with a maximum size of 100MB (compressed).

The inner data of Parquet files can be compressed with the following algorithms:

• snappy

• gzip

Troubleshooting Data Transports

Visit the Panther Knowledge Base to view articles about data transports that answer frequently asked questions and help you resolve common errors and issues.