# Data Sources & Transports ## Overview Panther offers built-in integrations for common data sources and data mapping for custom log sources. This page describes available [data source options](#data-sources-and-transports), how to [monitor log source ingestion and health](#monitoring-log-sources), how to [request support for a new log source](#request-support-for-a-log-source), and how to [configure an Event Threshold alarm](#configuring-event-threshold-alarms). For information on ingesting Panther Console audit logs, see the [Panther Audit Logs](https://docs.panther.com/data-onboarding/supported-logs/panther-audit-logs) page. #### Video overview {% embed url="" %} ## Data Sources & Transports ### Data Transports You can create an HTTP (webhook) source, or leverage cloud services like S3 buckets, CloudWatch, SQS, SNS, Azure Blob Storage, or Google Cloud Storage (GCS) to push data to Panther. For more information, see [Data Transports](https://docs.panther.com/data-onboarding/data-transports). ### Supported logs Panther supports pulling logs from vendors via direct integrations that query the API and via AWS EventBridge. In addition, Panther supports pushing logs to common Data Transport sources to ingest logs that have supported schemas but not a direct API integration. For a full list of supported vendors, see the [Supported Logs](https://docs.panther.com/data-onboarding/supported-logs) page. #### Cloud accounts In addition to onboarding AWS as a log source to configure Detections and receive alerts, we recommend configuring Cloud Security Scanning for your AWS account. Cloud Security Scanning works by scanning AWS accounts, modeling the Resources within them, and using Policies to detect misconfigurations. For more information, see [Cloud Security Scanning](https://docs.panther.com/cloud-scanning). ### Custom logs Panther allows you to generate a custom schema if you have a log type that is not yet supported. Panther gives you the ability to build custom schemas, which inform Panther how to parse events correctly. For more information, see [Custom Logs.](https://docs.panther.com/data-onboarding/custom-log-types) ### Monitoring log sources When your log source is onboarded in Panther, you can monitor its individual data processing metrics and health within the log source's operations page, attach new schemas, and view raw data associated with the log source. You can also monitor overall log source ingestion metrics on the Log Source Overview page. For more information, see [Monitoring Log Sources](https://docs.panther.com/data-onboarding/monitoring-log-sources). ### Ingestion filtering Ingestion filters let you define conditions under which incoming data should be dropped—i.e., not ingested into Panther. This dropped data will not contribute to your ingestion quota. These filters can be useful, then, to partially ingest high-volume logs that may have previously been cost-prohibitive when connected with Panther. For more information, see [Ingestion Filters](https://docs.panther.com/data-onboarding/ingestion-filters). ## Configuring event threshold alarms On the final step of configuring your log source with Panther, you have the option to create an alarm in case the source does not process any events within a configurable period of time. For example, if you configure the threshold to 15 minutes, then you will receive an alert if no events are processed in 15 minutes. For instructions, see [Configuring log drop-off alarms for log sources](https://docs.panther.com/system-configuration/notifications/system-errors#configuring-log-drop-offs-alarms-for-log-sources). ## Request support for a log source If you do not see the log source you want within the list at **Integrations > Log Sources**, you can request support of a new log source: 1. Log in to your Panther Console. 2. Navigate to **Configure > Log Sources**. 3. Click **Create New.** 4. Scroll to the bottom of the page and click the **Request it here** hyperlink.\ ![](https://4011785613-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LgdiSWdyJcXPahGi9Rs-2910905616%2Fuploads%2Fgit-blob-abb3b93eeff3dc24a4880b3c45b5cc3641f10800%2FScreenshot%202023-02-27%20at%208.49.25%20AM.png?alt=media) 5. Enter the Log Source name you want to request and the use case it will address. 6. Click **Create Request**. ## Deleting a log source If you no longer want to collect logs from a particular log source, you can delete it in the Panther Console or using the Panther API. After you delete a log source, all events that have already been collected by that source will remain accessible in your Data Lake (meaning they can be queried with [Data Explorer](https://docs.panther.com/search/data-explorer) and [Search](https://docs.panther.com/search/search-tool)). To delete a log source: {% tabs %} {% tab title="Panther Console" %} 1. In the left-hand navigation bar of your Panther Console, click **Configure** > **Log Sources**. 2. In the table of log sources, locate the one you would like to delete. On the right side of its row, click the three dots icon. 3. Click **Delete**.\ ![An arrow is drawn from a three dots icon to a "Delete" value in a pop-up menu.](https://4011785613-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-LgdiSWdyJcXPahGi9Rs-2910905616%2Fuploads%2Fgit-blob-5eda1a07b163809d46902f0d529bdd95fec3fdee%2FScreenshot%202024-08-12%20at%204.26.02%20PM.png?alt=media) 4. In the pop-up confirmation modal, click **Yes, Delete**. {% endtab %} {% tab title="Panther API" %} * Use the `DeleteSource` mutation in the [Panther GraphQL API](https://docs.panther.com/panther-developer-workflows/api/graphql/log-source#deleting-a-log-source). {% endtab %} {% endtabs %} ## Data ingestion size limit Panther can ingest events up to 15 MB. If a log event larger than 15 MB is sent to Panther, it will be skipped and not ingested. If it is being ingested from S3, CloudWatch, GCS, or Azure Blob Storage, the entire file will be dropped and a [System Error will be generated](https://docs.panther.com/system-configuration/notifications/system-errors#s3-getobject-error-notifications). ## IP addresses Panther uses to pull data The IP address Panther uses to fetch your data depends on the nature of the log source: | Type of log source | IP address Panther uses to pull data | | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | [Supported Logs](https://docs.panther.com/data-onboarding/supported-logs) source that is an "API puller"—i.e., whose API Panther polls | Your Panther Console [gateway public IP](https://docs.panther.com/system-configuration#general-settings). | | | An IP address within the [AWS IP address space](https://docs.aws.amazon.com/vpc/latest/userguide/aws-ip-ranges.html). | | [Supported Logs](https://docs.panther.com/data-onboarding/supported-logs) source that uses a GCS storage location OR a [Custom Logs](https://docs.panther.com/data-onboarding/custom-log-types) source that uses a [GCS Data Transport](https://docs.panther.com/data-onboarding/data-transports/google) | An IP address within the [AWS IP address space](https://docs.aws.amazon.com/vpc/latest/userguide/aws-ip-ranges.html). (To request that Panther limit the IP to your Panther Console [gateway public IP](https://docs.panther.com/system-configuration#general-settings), contact Support.) | | [Supported Logs](https://docs.panther.com/data-onboarding/supported-logs) source that uses an Azure storage location OR a [Custom Logs](https://docs.panther.com/data-onboarding/custom-log-types) source that uses an [Azure Data Transport](https://docs.panther.com/data-onboarding/data-transports/azure) | An IP address within the [AWS IP address space](https://docs.aws.amazon.com/vpc/latest/userguide/aws-ip-ranges.html). (To request that Panther limit the IP to your Panther Console [gateway public IP](https://docs.panther.com/system-configuration#general-settings), contact Support.) | ## Troubleshooting Data Sources and Transports Visit the Panther Knowledge Base to [view articles about data sources and transports](https://help.panther.com/Data_Sources) that answer frequently asked questions and help you resolve common errors and issues.