To connect GCP logs with Panther, it's recommended to use the Pub/Sub Data Transport source with a log sink, as it results in the lowest latency—roughly five minutes.
Set a default Data Access audit logging configuration for your Google Cloud services:
In your GCP console, navigate to the IAM & Admin service. In the navigation bar, click Audit Logs.
Click Set Default Configuration.
In the Log Types tab, check the boxes for the following types: Admin Read, Data Read, and Data Write.
Click Save.
These instructions for setting a default Data Access audit log configuration for your Google Cloud services are also found in the GCP documentation: Set the default configuration.
Step 1: Create a Google Cloud source in Panther
In the left-hand navigation bar of your Panther Console, click Configure > Log Sources.
Click Create New.
Search for "GCP" then click the Google Cloud tile.
In the slide-out panel, in the Transport Mechanism dropdown in the upper right corner, select Google Cloud Pub/Sub.
It is possible to use any of the Data Transport options, but is recommended to use Pub/Sub in conjunction with a log sink, which you will configure in the next step.
Follow the Panther documentation for configuring your selected Data Transport.
Access Transparency logs record activities taken by Google personnel when accessing customer content. These logs provide visibility into Google Cloud support operations and help meet compliance requirements by recording things like justification for access and what actions were performed.
Google Cloud DNS query logs contain detailed information about DNS queries that your Cloud DNS zones receive. These logs help you monitor DNS activity and troubleshoot DNS-related issues by capturing things like query names, source and destination IP addresses, and protocol information.
Google Cloud VPC Firewall Rules Logging allows you to audit, verify, and analyze the effects of your firewall rules. These logs capture information about network connections that match firewall rules.
External HTTP(S) Load Balancing distributes HTTP and HTTPS traffic to backends hosted on a variety of Google Cloud platforms (such as Compute Engine, Google Kubernetes Engine (GKE), Cloud Storage, and so on), as well as external backends connected over the internet or via hybrid connectivity. HTTP(S) load balancing logs provide information for monitoring and debugging web traffic.
schema: GCP.AccessTransparency
description: |
Access Transparency logs record activities taken by Google personnel when accessing customer content.
These logs provide visibility into Google Cloud support operations and help meet compliance requirements.
referenceURL: https://cloud.google.com/assured-workloads/access-transparency/docs/reading-logs
fields:
- name: logName
required: true
description: The resource name of the log to which this log entry belongs.
type: string
validate:
allowContains: ['cloudaudit.googleapis.com%2Faccess_transparency']
- name: severity
description: The severity of the log entry. The default value is LogSeverity.DEFAULT.
type: string
- name: insertId
description: A unique identifier for the log entry.
type: string
- name: resource
description: The monitored resource that produced this log entry.
type: object
fields:
- name: type
required: true
description: Type of resource that produced this log entry
type: string
- name: labels
description: Labels describing the resource
type: json
- name: timestamp
description: The time the event described by the log entry occurred.
type: timestamp
timeFormats:
- rfc3339
isEventTime: true
- name: receiveTimestamp
description: The time the log entry was received by Logging.
type: timestamp
timeFormats:
- rfc3339
isEventTime: true
- name: labels
description: A set of user-defined (key, value) data that provides additional information about the log entry.
type: json
- name: operation
description: Information about an operation associated with the log entry, if applicable.
type: object
fields:
- name: id
description: Log entries with the same identifier are assumed to be part of the same operation.
type: string
- name: producer
description: An arbitrary producer identifier. The combination of id and producer must be globally unique.
type: string
- name: first
description: This is the first entry in an operation
type: boolean
- name: last
description: This is the last entry in an operation
type: boolean
- name: trace
description: Resource name of the trace associated with the log entry, if any. The trace field provides the link between logs and traces.
type: string
- name: spanId
description: The span ID within the trace associated with the log entry.
type: string
- name: traceSampled
description: The sampling decision of the trace associated with the log entry.
type: boolean
- name: sourceLocation
description: Source code location information associated with the log entry, if any.
type: object
fields:
- name: file
description: Source file name. Depending on the runtime environment, this might be a simple name or a fully-qualified name.
type: string
- name: line
description: Line within the source file. 1-based; 0 indicates no line number available.
type: bigint
- name: function
description: Human-readable name of the function or method being invoked, with optional context such as the class or package name. The format can vary by language
type: string
- name: jsonPayload
required: true
description: The Access Transparency log payload
type: object
fields:
- name: at_sign_type
required: true
description: The type of payload, should be type.googleapis.com/google.cloud.audit.TransparencyLog
rename:
from: '@type'
type: string
- name: location
description: Geographic data about the accessor
type: object
fields:
- name: principalOfficeCountry
description: ISO 3166-1 alpha-2 country code for the permanent desk location of the Google personnel
type: string
- name: principalEmployingEntity
description: The employing entity of the Google personnel (e.g., Google LLC)
type: string
- name: principalPhysicalLocationCountry
description: ISO 3166-1 alpha-2 country code for the physical location from which the access originated
type: string
- name: principalJobTitle
description: Job classification of the Google personnel that accessed the resource (e.g., Engineering, Support)
type: string
- name: product
description: List of GCP services that were accessed
type: array
element:
type: string
- name: reason
description: Justification details for the access
type: array
element:
type: object
fields:
- name: type
description: The reason classification (e.g., CUSTOMER_INITIATED_SUPPORT, GOOGLE_INITIATED_SERVICE, GOOGLE_INITIATED_REVIEW, THIRD_PARTY_DATA_REQUEST, GOOGLE_RESPONSE_TO_PRODUCTION_ALERT)
type: string
- name: detail
description: Specific justification text (e.g., case number, ticket reference)
type: string
- name: permissionDetails
description: IAM permission information about the access
type: array
element:
type: object
fields:
- name: permissionType
description: The maximum IAM permission category (admin_read, admin_write, data_read, data_write)
type: string
- name: logAccessed
description: Boolean indicating if the access was restricted to log data only
type: boolean
- name: eventId
description: Unique event identifier for this access justification
type: string
- name: accesses
description: Specific actions performed by Google personnel
type: array
element:
type: object
fields:
- name: methodName
description: The action type (standard API method, custom method, or GoogleInternal method)
type: string
- name: resourceName
description: Full resource identifier that was accessed (e.g., //googleapis.com/storage/buckets/BUCKET_NAME/objects/OBJECT_NAME)
type: string
- name: accessApprovals
description: List of Access Approval request resource names associated with this access
type: array
element:
type: string
schema: GCP.AuditLog
description: |
Cloud Audit Logs maintains audit logs for each Google Cloud project, folder, and organization: Admin Activity, Data Access, System Event, and Policy Denied.
Google Cloud services write audit log entries to these logs to help you answer the questions of "who did what, where, and when?" within your Google Cloud resources.
referenceURL: https://cloud.google.com/logging/docs/audit
fields:
- name: logName
required: true
description: The resource name of the log to which this log entry belongs.
type: string
- name: severity
description: The severity of the log entry. The default value is LogSeverity.DEFAULT.
type: string
- name: insertId
description: A unique identifier for the log entry.
type: string
- name: resource
description: The monitored resource that produced this log entry.
type: object
fields:
- name: type
required: true
description: Type of resource that produced this log entry
type: string
- name: labels
description: Labels describing the resource
type: json
- name: timestamp
description: The time the event described by the log entry occurred.
type: timestamp
timeFormats:
- rfc3339
isEventTime: true
- name: receiveTimestamp
required: true
description: The time the log entry was received by Logging.
type: timestamp
timeFormats:
- rfc3339
isEventTime: true
- name: labels
description: A set of user-defined (key, value) data that provides additional information about the log entry.
type: json
- name: operation
description: Information about an operation associated with the log entry, if applicable.
type: object
fields:
- name: id
description: Log entries with the same identifier are assumed to be part of the same operation.
type: string
- name: producer
description: An arbitrary producer identifier. The combination of id and producer must be globally unique.
type: string
- name: first
description: This is the first entry in an operation
type: boolean
- name: last
description: This is the last entry in an operation
type: boolean
- name: trace
description: Resource name of the trace associated with the log entry, if any. The trace field provides the link between logs and traces.
type: string
- name: httpRequest
description: Information about the HTTP request associated with this log entry, if applicable.
type: object
fields:
- name: requestMethod
description: The request HTTP method.
type: string
- name: requestURL
description: The scheme (http, https), the host name, the path and the query portion of the URL that was requested.
type: string
indicators:
- url
- name: requestSize
description: The size of the HTTP request message in bytes, including the request headers and the request body.
type: bigint
- name: status
description: The response HTTP status code
type: smallint
- name: responseSize
description: The size of the HTTP response message sent back to the client, in bytes, including the response headers and the response body.
type: bigint
- name: userAgent
description: The user agent sent by the client.
type: string
- name: remoteIP
description: The IP address (IPv4 or IPv6) of the client that issued the HTTP request.
type: string
indicators:
- ip
- name: serverIP
description: The IP address (IPv4 or IPv6) of the origin server that the request was sent to.
type: string
indicators:
- ip
- name: referer
description: The referer URL of the request
type: string
indicators:
- url
- name: latency
description: The request processing latency in seconds on the server, from the time the request was received until the response was sent.
type: string
- name: cacheLookup
description: Whether or not a cache lookup was attempted.
type: boolean
- name: cacheHit
description: Whether or not an entity was served from cache (with or without validation).
type: boolean
- name: cacheValidatedWithOriginServer
description: Whether or not an entity was served from cache (with or without validation).
type: boolean
- name: cacheFillBytes
description: Whether or not an entity was served from cache (with or without validation).
type: bigint
- name: protocol
description: Protocol used for the request.
type: string
- name: spanId
description: The span ID within the trace associated with the log entry.
type: string
- name: traceSampled
description: The sampling decision of the trace associated with the log entry.
type: boolean
- name: sourceLocation
description: Source code location information associated with the log entry, if any.
type: object
fields:
- name: file
description: Source file name. Depending on the runtime environment, this might be a simple name or a fully-qualified name.
type: string
- name: line
description: Line within the source file. 1-based; 0 indicates no line number available.
type: bigint
- name: function
description: Human-readable name of the function or method being invoked, with optional context such as the class or package name. The format can vary by language
type: string
- name: protoPayload
required: true
description: The AuditLog payload
type: object
fields:
- name: '@type'
required: true
description: The type of payload
type: string
- name: serviceName
description: The name of the API service performing the operation
type: string
- name: methodName
description: The name of the service method or operation. For API calls, this should be the name of the API method.
type: string
- name: resourceName
description: The resource or collection that is the target of the operation. The name is a scheme-less URI, not including the API service name.
type: string
- name: numResponseItems
description: The number of items returned from a List or Query API method, if applicable.
type: bigint
- name: status
description: The status of the overall operation.
type: object
fields:
- name: code
description: The status code, which should be an enum value of google.rpc.Code.
type: int
- name: message
description: A developer-facing error message, which should be in English.
type: string
- name: details
description: A list of messages that carry the error details. There is a common set of message types for APIs to use.
type: json
- name: authenticationInfo
description: Authentication information.
type: object
fields:
- name: principalSubject
description: String representation of identity of requesting party. Populated for both first and third party identities.
type: string
- name: serviceAccountKeyName
description: The name of the service account key used to create or exchange credentials for authenticating the service account making the request. This is a scheme-less URI full resource name.
type: string
indicators:
- domain
- name: principalEmail
description: The email address of the authenticated user making the request.
type: string
indicators:
- email
- name: authoritySelector
description: The authority selector specified by the requestor, if any. It is not guaranteed that the principal was allowed to use this authority.
type: string
- name: thirdPartyPrincipal
description: The third party identification (if any) of the authenticated user making the request. When the JSON object represented here has a proto equivalent, the proto name will be indicated in the @type property.
type: json
- name: serviceAccountDelegationInfo
description: Identity delegation history of an authenticated service account that makes the request. It contains information on the real authorities that try to access GCP resources by delegating on a service account. When multiple authorities present, they are guaranteed to be sorted based on the original ordering of the identity delegation events.
type: array
element:
type: object
fields:
- name: firstPartyPrincipal
description: First party (Google) identity as the real authority.
type: object
fields:
- name: principalEmail
description: The email address of a Google account.
type: string
indicators:
- email
- name: serviceMetadata
description: Metadata about the service that uses the service account.
type: json
- name: thirdPartyPrincipal
description: Third party identity as the real authority.
type: object
fields:
- name: thirdPartyClaims
description: Metadata about third party identity.
type: json
- name: principalSubject
description: String representation of identity of requesting party.
type: string
- name: authorizationInfo
description: Authorization information. If there are multiple resources or permissions involved, then there is one AuthorizationInfo element for each {resource, permission} tuple.
type: array
element:
type: object
fields:
- name: resource
description: The resource being accessed, as a REST-style string.
type: string
- name: permission
description: The required IAM permission
type: string
- name: granted
description: Whether or not authorization for resource and permission was granted.
type: boolean
- name: resourceAttributes
description: Resource attributes used in IAM condition evaluation. This field contains resource attributes like resource type and resource name. To get the whole view of the attributes used in IAM condition evaluation, the user must also look into AuditLog.request_metadata.request_attributes.
type: object
fields:
- name: service
description: The name of the service that this resource belongs to, such as pubsub.googleapis.com. The service may be different from the DNS hostname that actually serves the request.
type: string
- name: name
description: The stable identifier (name) of a resource on the service.
type: string
- name: type
description: The type of the resource. The syntax is platform-specific because different platforms define their resources differently.
type: string
- name: labels
description: The labels or tags on the resource, such as AWS resource tags and Kubernetes resource labels.
type: string
- name: uid
description: The unique identifier of the resource. UID is unique in the time and space for this resource within the scope of the service. It is typically generated by the server on successful creation of a resource and must not be changed. UID is used to uniquely identify resources with resource name reuses. This should be a UUID4.
type: string
- name: requestMetadata
description: Metadata about the request
type: object
fields:
- name: callerIP
description: The IP address of the caller.
type: string
indicators:
- ip
- name: callerSuppliedUserAgent
description: The user agent of the caller. This information is not authenticated and should be treated accordingly.
type: string
- name: callerNetwork
description: The network of the caller. Set only if the network host project is part of the same GCP organization (or project) as the accessed resource.
type: string
- name: requestAttributes
description: Request attributes used in IAM condition evaluation. This field contains request attributes like request time and access levels associated with the request.
type: json
- name: destinationAttributes
description: The destination of a network activity, such as accepting a TCP connection.
type: json
- name: request
description: The operation request. This may not include all request parameters, such as those that are too large, privacy-sensitive, or duplicated elsewhere in the log record. When the JSON object represented here has a proto equivalent, the proto name will be indicated in the @type property.
type: json
- name: response
description: The operation response. This may not include all response parameters, such as those that are too large, privacy-sensitive, or duplicated elsewhere in the log record. When the JSON object represented here has a proto equivalent, the proto name will be indicated in the @type property.
type: json
- name: metadata
description: Other service-specific data about the request, response, and other information associated with the current audited event.
type: json
- name: serviceData
description: Other service-specific data about the request, response, and other activities.
type: json
schema: GCP.DNS
description: Google Cloud DNS query logs contain detailed information about DNS queries that your Cloud DNS zones receive.
referenceURL: https://cloud.google.com/dns/docs/monitoring
fields:
- name: alias_query_response_code
type: string
- name: egressError
type: string
- name: healthyIps
type: string
- name: unHealthyIps
type: string
- name: insertId
required: true
type: string
- name: jsonPayload
required: true
type: object
fields:
- name: authAnswer
type: boolean
- name: dns64Translated
type: boolean
description: Indicates whether the response was translated from an IPv4 address to an IPv6 address using DNS64
- name: serverLatency
type: float
- name: queryName
type: string
indicators:
- domain
- name: vmProjectId
type: string
- name: vmZoneName
type: string
- name: vmInstanceName
type: string
- name: vmInstanceId
type: float
description: Numeric VM instance ID. May lose precision for very large values due to float representation. Use vmInstanceIdString for exact value.
- name: vmInstanceIdString
type: string
description: String representation of the VM instance ID with full precision
- name: responseCode
type: string
- name: destinationIP
type: string
indicators:
- ip
- name: protocol
type: string
- name: structuredRdata
type: array
element:
type: object
fields:
- name: class
type: string
- name: ttl
type: string
- name: domainName
type: string
indicators:
- domain
- name: rvalue
type: string
- name: type
type: string
- name: queryType
type: string
- name: sourceIP
type: string
indicators:
- ip
- name: sourceNetwork
type: string
- name: egressIP
type: string
indicators:
- ip
- name: rdata
type: json
description: DNS answer data in various formats. Can be an object with DNS record fields, an empty string for NXDOMAIN responses, or omitted entirely.
- name: resource
required: true
type: object
fields:
- name: type
required: true
type: string
- name: labels
type: object
fields:
- name: target_type
type: string
- name: location
type: string
- name: source_type
type: string
- name: project_id
type: string
- name: target_name
type: string
- name: timestamp
type: timestamp
timeFormats:
- rfc3339
isEventTime: true
- name: severity
required: true
type: string
- name: logName
required: true
type: string
validate:
allowContains: ['dns.googleapis.com%2Fdns_queries']
- name: receiveTimestamp
required: true
type: timestamp
timeFormats:
- rfc3339
