To connect GCP logs with Panther, it's recommended to use the Pub/Sub Data Transport source with a log sink, as it results in the lowest latency—roughly five minutes.
Set a default Data Access audit logging configuration for your Google Cloud services:
In your GCP console, navigate to the IAM & Admin service. In the navigation bar, click Audit Logs.
Click Set Default Configuration.
In the Log Types tab, check the boxes for the following types: Admin Read, Data Read, and Data Write.
Click Save.
These instructions for setting a default Data Access audit log configuration for your Google Cloud services are also found in the GCP documentation: Set the default configuration.
Step 1: Create a Google Cloud source in Panther
In the left-hand navigation bar of your Panther Console, click Configure > Log Sources.
Click Create New.
Search for "GCP" then click the Google Cloud tile.
In the slide-out panel, in the Transport Mechanism dropdown in the upper right corner, select Google Cloud Pub/Sub.
It is possible to use any of the Data Transport options, but is recommended to use Pub/Sub in conjunction with a log sink, which you will configure in the next step.
Follow the Panther documentation for configuring your selected Data Transport.
schema:GCP.AuditLogdescription:| Cloud Audit Logs maintains audit logs for each Google Cloud project, folder, and organization: Admin Activity, Data Access, System Event, and Policy Denied.
Google Cloud services write audit log entries to these logs to help you answer the questions of "who did what, where, and when?" within your Google Cloud resources.
referenceURL:https://cloud.google.com/logging/docs/auditfields: - name:logNamerequired:truedescription:The resource name of the log to which this log entry belongs.type:string - name:severitydescription:The severity of the log entry. The default value is LogSeverity.DEFAULT.type:string - name:insertIddescription:A unique identifier for the log entry.type:string - name:resourcedescription:The monitored resource that produced this log entry.type:objectfields: - name:typerequired:truedescription:Type of resource that produced this log entrytype:string - name:labelsdescription:Labels describing the resourcetype:json - name:timestampdescription:The time the event described by the log entry occurred.type:timestamptimeFormats: - rfc3339isEventTime:true - name:receiveTimestamprequired:truedescription:The time the log entry was received by Logging.type:timestamptimeFormats: - rfc3339isEventTime:true - name:labelsdescription:A set of user-defined (key, value) data that provides additional information about the log entry.type:json - name:operationdescription:Information about an operation associated with the log entry, if applicable.type:objectfields: - name:iddescription:Log entries with the same identifier are assumed to be part of the same operation.type:string - name:producerdescription:An arbitrary producer identifier. The combination of id and producer must be globally unique.type:string - name:firstdescription:This is the first entry in an operationtype:boolean - name:lastdescription:This is the last entry in an operationtype:boolean - name:trace description: Resource name of the trace associated with the log entry, if any. The trace field provides the link between logs and traces.
type:string - name:httpRequestdescription:Information about the HTTP request associated with this log entry, if applicable.type:objectfields: - name:requestMethoddescription:The request HTTP method.type:string - name:requestURL description: The scheme (http, https), the host name, the path and the query portion of the URL that was requested.
type:stringindicators: - url - name:requestSize description: The size of the HTTP request message in bytes, including the request headers and the request body.
type:bigint - name:statusdescription:The response HTTP status codetype:smallint - name:responseSize description: The size of the HTTP response message sent back to the client, in bytes, including the response headers and the response body.
type:bigint - name:userAgentdescription:The user agent sent by the client.type:string - name:remoteIPdescription:The IP address (IPv4 or IPv6) of the client that issued the HTTP request.type:stringindicators: - ip - name:serverIPdescription:The IP address (IPv4 or IPv6) of the origin server that the request was sent to.type:stringindicators: - ip - name:refererdescription:The referer URL of the requesttype:stringindicators: - url - name:latency description: The request processing latency in seconds on the server, from the time the request was received until the response was sent.
type:string - name:cacheLookupdescription:Whether or not a cache lookup was attempted.type:boolean - name:cacheHitdescription:Whether or not an entity was served from cache (with or without validation).type:boolean - name:cacheValidatedWithOriginServerdescription:Whether or not an entity was served from cache (with or without validation).type:boolean - name:cacheFillBytesdescription:Whether or not an entity was served from cache (with or without validation).type:bigint - name:protocoldescription:Protocol used for the request.type:string - name:spanIddescription:The span ID within the trace associated with the log entry.type:string - name:traceSampleddescription:The sampling decision of the trace associated with the log entry.type:boolean - name:sourceLocationdescription:Source code location information associated with the log entry, if any.type:objectfields: - name:file description: Source file name. Depending on the runtime environment, this might be a simple name or a fully-qualified name.
type:string - name:linedescription:Line within the source file. 1-based; 0 indicates no line number available.type:bigint - name:function description: Human-readable name of the function or method being invoked, with optional context such as the class or package name. The format can vary by language
type:string - name:protoPayloadrequired:truedescription:The AuditLog payloadtype:objectfields: - name:'@type'required:truedescription:The type of payloadtype:string - name:serviceNamedescription:The name of the API service performing the operationtype:string - name:methodName description: The name of the service method or operation. For API calls, this should be the name of the API method.
type:string - name:resourceName description: The resource or collection that is the target of the operation. The name is a scheme-less URI, not including the API service name.
type:string - name:numResponseItemsdescription:The number of items returned from a List or Query API method, if applicable.type:bigint - name:statusdescription:The status of the overall operation.type:objectfields: - name:codedescription:The status code, which should be an enum value of google.rpc.Code.type:int - name:messagedescription:A developer-facing error message, which should be in English.type:string - name:details description: A list of messages that carry the error details. There is a common set of message types for APIs to use.
type:json - name:authenticationInfodescription:Authentication information.type:objectfields: - name:principalSubject description: String representation of identity of requesting party. Populated for both first and third party identities.
type:string - name:serviceAccountKeyName description: The name of the service account key used to create or exchange credentials for authenticating the service account making the request. This is a scheme-less URI full resource name.
type:stringindicators: - domain - name:principalEmaildescription:The email address of the authenticated user making the request.type:stringindicators: - email - name:authoritySelector description: The authority selector specified by the requestor, if any. It is not guaranteed that the principal was allowed to use this authority.
type:string - name:thirdPartyPrincipal description: The third party identification (if any) of the authenticated user making the request. When the JSON object represented here has a proto equivalent, the proto name will be indicated in the @type property.
type:json - name:serviceAccountDelegationInfo description: Identity delegation history of an authenticated service account that makes the request. It contains information on the real authorities that try to access GCP resources by delegating on a service account. When multiple authorities present, they are guaranteed to be sorted based on the original ordering of the identity delegation events.
type:arrayelement:type:objectfields: - name:firstPartyPrincipaldescription:First party (Google) identity as the real authority.type:objectfields: - name:principalEmaildescription:The email address of a Google account.type:stringindicators: - email - name:serviceMetadatadescription:Metadata about the service that uses the service account.type:json - name:thirdPartyPrincipaldescription:Third party identity as the real authority.type:objectfields: - name:thirdPartyClaimsdescription:Metadata about third party identity.type:json - name:principalSubjectdescription:String representation of identity of requesting party.type:string - name:authorizationInfo description: Authorization information. If there are multiple resources or permissions involved, then there is one AuthorizationInfo element for each {resource, permission} tuple.
type:arrayelement:type:objectfields: - name:resourcedescription:The resource being accessed, as a REST-style string.type:string - name:permissiondescription:The required IAM permissiontype:string - name:granteddescription:Whether or not authorization for resource and permission was granted.type:boolean - name:resourceAttributes description: Resource attributes used in IAM condition evaluation. This field contains resource attributes like resource type and resource name. To get the whole view of the attributes used in IAM condition evaluation, the user must also look into AuditLog.request_metadata.request_attributes.
type:objectfields: - name:service description: The name of the service that this resource belongs to, such as pubsub.googleapis.com. The service may be different from the DNS hostname that actually serves the request.
type:string - name:namedescription:The stable identifier (name) of a resource on the service.type:string - name:type description: The type of the resource. The syntax is platform-specific because different platforms define their resources differently.
type:string - name:labels description: The labels or tags on the resource, such as AWS resource tags and Kubernetes resource labels.
type:string - name:uid description: The unique identifier of the resource. UID is unique in the time and space for this resource within the scope of the service. It is typically generated by the server on successful creation of a resource and must not be changed. UID is used to uniquely identify resources with resource name reuses. This should be a UUID4.
type:string - name:requestMetadatadescription:Metadata about the requesttype:objectfields: - name:callerIPdescription:The IP address of the caller.type:stringindicators: - ip - name:callerSuppliedUserAgent description: The user agent of the caller. This information is not authenticated and should be treated accordingly.
type:string - name:callerNetwork description: The network of the caller. Set only if the network host project is part of the same GCP organization (or project) as the accessed resource.
type:string - name:requestAttributes description: Request attributes used in IAM condition evaluation. This field contains request attributes like request time and access levels associated with the request.
type:json - name:destinationAttributesdescription:The destination of a network activity, such as accepting a TCP connection.type:json - name:request description: The operation request. This may not include all request parameters, such as those that are too large, privacy-sensitive, or duplicated elsewhere in the log record. When the JSON object represented here has a proto equivalent, the proto name will be indicated in the @type property.
type:json - name:response description: The operation response. This may not include all response parameters, such as those that are too large, privacy-sensitive, or duplicated elsewhere in the log record. When the JSON object represented here has a proto equivalent, the proto name will be indicated in the @type property.
type:json - name:metadata description: Other service-specific data about the request, response, and other information associated with the current audited event.
type:json - name:serviceDatadescription:Other service-specific data about the request, response, and other activities.type:json
GCP.HTTPLoadBalancer
External HTTP(S) Load Balancing distributes HTTP and HTTPS traffic to backends hosted on a variety of Google Cloud platforms (such as Compute Engine, Google Kubernetes Engine (GKE), Cloud Storage, and so on), as well as external backends connected over the internet or via hybrid connectivity. HTTP(S) load balancing logs provide information for monitoring and debugging web traffic.
