Google Cloud Storage (GCS) Source
Pull logs from Google Cloud Storage (GCS) within Google Cloud Platform (GCP) to use as a Data Transport log source in the Panther Console
This feature is available in versions 1.28 and newer. Panther's direct GCS integration is currently in public beta. Please share any bug reports and feature requests with your account team.
After you configure this integration, Panther will pull log data directly from Google Cloud Storage (GCS) buckets. You can then write rules and run queries on the processed data.
Panther requires certain configurations in Google Cloud Platform (GCP) to authenticate and pull logs. A bucket and a subscription for a topic set up with notifications are required. Panther will ingest new files through Pub/Sub notifications.

Set up the GCS source in Panther

  1. 1.
    Log in to your Panther Console.
  2. 2.
    On the left sidebar navigation, click Integrations > Log Sources.
  3. 3.
    Click Add New Source.
  4. 4.
    On the left side, click the Custom Onboarding tab, then click Select next to Google Cloud Storage.
  5. 5.
    Enter a descriptive name for the source and select the log types you will use, then click Continue Setup.
  6. 6.
    On the "Infrastructure & Credentials" page, follow the instructions on screen to create the infrastructure component with a Terraform template.
  7. 7.
    Upload your JSON key file, then enter the GCS bucket name and the Pub/Sub subscription ID.
    • The subscription ID can be found in the Subscriptions section of your Google Cloud account.
  8. 8.
    Click Continue Setup.
    • The message "Everything looks good" will appear at the top of the screen.
  9. 9.
    Click Finish Setup.
Panther will now start processing the new files that arrive to your GCS bucket.

Configuring the Integration in Google Cloud Platform (GCP)

If you choose to create the infrastructure components manually rather than using a Terraform template during the GCP setup in the Panther Console, follow the instructions below.
  1. 1.
    Log in to your Google Cloud console.
  2. 2.
    Determine which bucket Panther will pull logs from.
  3. 3.
    Create a topic for the notifications.
    • You can create a topic using the gcloud CLI tool with the following command format: gcloud pubsub topics create $TOPIC_ID
  4. 4.
    Configure the bucket to send notifications for new files to the topic you created.
    • You can create a notification using the gcloud CLI tool with the following command format: gsutil notification create -t $TOPIC_NAME -e OBJECT_FINALIZE -f json gs://$BUCKET_NAME
    • Note: Panther only requires the OBJECT_FINALIZE type.
  5. 5.
    Create a subscription to be used with the topic you created. Note that this subscription should not be used by any service other than Panther.
    • You can create a subscription using the gcloud CLI tool with the following command format: gcloud pubsub subscriptions create $SUBSCRIPTION_ID --topic $TOPIC_ID --topic-project $PROJECT_ID
  6. 6.
    Create a new Google Cloud service account and take note of the account email address. Panther will use this identity to be able to access the infrastructure created for this integration.
    • The following permissions are required for the project where the Pub/Sub subscription and topic lives:
      Permissions required
      Role
      Scope
      storage.objects.get
      storage.objects.list
      storage/viewer
      bucket-name
      pubsub.subscriptions.consume
      pubsub/subscriber
      subscription-name
      pubsub.subscriptions.get
      pubsub/viewer
      subscription-name
      monitoring.timeSeries.list
      monitoring/viewer
      project
    • Note: You can set conditions or IAM policies on permissions for specific resources. This can be done either in the IAM page of the service account (as seen in the example screenshot) or in the specific resource's page:
  7. 7.
    Generate a JSON key file for the service account, which will be used in Panther to authenticate to the GCP infrastructure.
    • You can create a JSON key file using the gcloud CLI tool with the following command format: gcloud iam service-accounts keys create $KEYFILE_PATH --iam-account=$SERVICE_ACCOUNT_EMAIL

Viewing Collected Logs

After log sources are configured, you can search your data in Data Explorer. For more information and for example queries, please see the Data Explorer documentation.