Pub/Sub Source
Onboarding Google Cloud Pub/Sub as a Data Transport log source in the Panther Console
Last updated
Was this helpful?
Onboarding Google Cloud Pub/Sub as a Data Transport log source in the Panther Console
Last updated
Was this helpful?
Panther supports configuring as a Data Transport to pull log data directly from Pub/Sub topics.
Panther can authenticate against your source using Google Cloud or a .
In the left-hand navigation bar of your Panther Console, click Configure > Log Sources.
In the upper-right corner, click Create New.
Click the Custom Log Formats tile.
On the Google Cloud Pub/Sub tile, click Start.
On the Basic Info page, fill in the fields:
Name: Enter a descriptive name for the log source.
Log Types: Select the log types Panther should use to parse your logs.
Click Setup.
On the Log Format page, select the of the incoming logs:
Auto
Lines
JSON
JSON Array
Click Continue.
On the Configuration page, follow the steps to create the required infrastructure components.
The instructions below are based on using a Terraform template. If you do not want to use Terraform, you can follow our alternative documentation to .
Before creating GCP infrastructure, you'll need to decide:
The creation method: You can use Terraform to create the infrastructure, or create it manually in the GCP console—see the sub-tabs within each top-level tab below.
To create GCP infrastructure using Terraform (authenticating with a service account):
Click Terraform Template to download the Terraform template.
Fill out the fields in the panther.tfvars
file with your configuration.
Set authentication_method
to "service_account"
.
Initialize a working directory containing Terraform configuration files and run terraform init
.
Copy the corresponding Terraform Command provided and run it in your CLI.
Generate a JSON key file by copying the gcloud Command provided, replacing the value for your service account email address, and running it in your CLI.
You can find the service account email in the output of the Terraform Command.
Under Provide pulling configuration & JSON Keyfile, upload your JSON key file.
Click Setup. You will be directed to a success screen:
If you have not done so already, click Attach or Infer Schemas to attach one or more schemas to the source.
The Trigger an alert when no events are processed setting defaults to YES. We recommend leaving this enabled, as you will be alerted if data stops flowing from the log source after a certain period of time. The timeframe is configurable, with a default of 24 hours.
The authentication method: You can use or a —see the top-level tabs below.
If you'd like Panther to authenticate using a , follow the instructions in one of the tabs below.
On the Infrastructure & Credentials page, click the Service Account tab.
You can also find the Terraform template at .
for the data.
to be used with the topic you created.
.
. To create the account using the gcloud
CLI tool, use the following command format:
Note: You can set conditions or IAM policies on permissions for specific resources. This can be done either in the IAM page of the service account (as seen in the example screenshot below) or in the specific resource's page.
for the service account, which will be used in Panther to authenticate to the GCP infrastructure.
If you'd like Panther to authenticate using , follow the instructions in one of the tabs below.
On the Infrastructure & Credentials page, click the Workload Identity Federation tab.
You can also find the Terraform template at .
for the data.
to be used with the topic you created.
.
Configure Workload Identity Federation with AWS by following the documentation.
As you are , take note of the following examples:
Example :
The value of the google.subject
attribute . You may use to transform or combine attributes from the token issued by AWS. The expression suggested in the table above takes this limit into account, and is an attempt at transforming the ARN into a value that uniquely identifies Panther entities. For more information on the AWS attributes, see "Example 2 - Called by user created with AssumeRole" on .
Example :
attribute.account=="<PANTHER_AWS_ACCOUNT_ID>"
When you are , select AWS.
Note: You can set conditions or IAM policies on permissions for specific resources. This can be done either in the IAM section in GCP (as seen in the example screenshot below) or in the specific resource's page.
, which will be used in Panther to authenticate to the GCP infrastructure.
If you are using a to authenticate:
Enter your Pub/Sub Subscription ID, found in the Subscriptions section of your Google Cloud account.
You can optionally enable one or more .
If you are using to authenticate:
On the Infrastructure & Credentials page, if you have not already, click the Workload Identity Federation tab.
Enter your Pub/Sub Subscription ID and Project ID, found in the Subscriptions section of your Google Cloud account.
You can optionally enable one or more .
After your log source is configured, you can search ingested data using or .