Knowledge BaseCommunityRelease NotesTry Panther
Search
⌃K
Links

SQS Source

Onboarding AWS SQS Logs as a Data Transport log source in the Panther Console

Overview

The steps below enable you to set up an Amazon Simple Queue Service (SQS) source and give you permissions to send data to that queue. Panther pulls events from that queue and allows you to write rules and run queries on the processed data.
SQS has a max message size of 256KB. If you expect to send messages bigger than this, consider using an S3 source instead.
Panther, by default, can process up to 1MiB/second across all SQS sources (5MiB/second for us-east-1, us-west-2, eu-west-1 deployments). If you exceed this limit, processing delays might appear. Please contact the Panther support team if you want this limit to be raised.
See the diagram below to understand how data flows from your application(s) into Panther using SQS.
A diagram shows how data flows from a customer application into Panther, using the SQS Data Transport. The flow is as follows: customer AWS application(s) like Lambda, S3, SNS, etc., SQS (which also takes in Allowed AWS ARNs), Panther application, parse & normalize, real-time detections, Long term retention in Snowflake, Alerts generated, and Alert destination

How to onboard SQS logs into Panther

  1. 1.
    Log in to your Panther Console.
  2. 2.
    In the left sidebar menu click Configure > Log Sources.
  3. 3.
    In the upper right corner, click Create New.
  4. 4.
    Click the Custom Log Formats tile.
  5. 5.
    Click AWS SQS Queue.
  6. 6.
    On the "Configure your source" page, fill in the fields as follows:
    • Name: Enter a descriptive name for your source.
    • Log Types: From the drop-down, select all log types that you wish to monitor.
    • Allowed AWS Principal ARNs: List all ARNs of the AWS principals that will be allowed to publish messages to your SQS queue.
    • Allowed Source ARNs: List all ARNs of the AWS resources (SNS topics, S3 buckets, etc.) that can publish messages to your SQS queue.
      • Note: If none of Allowed AWS Principal ARNs and Allowed Source ARNs properties are set, only Principals of the AWS account where Panther is deployed will be able to publish messages to the queue.
        The "Configure your source" screen in the Panther Console is displayed. There are fields for Name, Log Types, Allowed AWS Principal ARNs, and Allowed Source ARNs.
        ​
  7. 7.
    Click Continue Setup.
    • You will see a success screen on the next page:
      A screen in the Panther Console displays the message "Everything looks good!"
      ​
  8. 8.
    To finish the source setup:
    1. 1.
      Optionally configure a log drop-off alarm.
      • Before you finish the setup, we recommend that you create a log drop-off alarm to alert you if data stops flowing from the log source. Be sure to set an appropriate time interval for when you would like Panther to alert you that the log source is not sending data.
        ​
    2. 2.
      Optionally enable a Detection Pack.
    3. 3.
      Click Finish Setup.
To view your newly created source, click View Log Source.
  • Manage your AWS SQS source here for data and events processed, overall health, source schemas, alarm configuration, etc.
This page displays Data Metrics such as data processed, events processed, and data processed by log type. To view this data, click on a log source.

View Collected Logs

After SQS log sources are configured, you can search your data in Data Explorer. For more information and for example queries, please see the Data Explorer documentation.
Previous
CloudWatch Logs Source
Next
SNS Source
Last modified 2mo ago