GitLab Logs
Connecting GitLab logs to your Panther Console
Panther supports onboarding GitLab logs using Data Transport mechanisms. Audit logs can be ingested with the HTTP Source, while other GitLab logs can be ingested with Amazon Web Services (AWS) S3 and SQS.
This process outlines how to onboard GitLab Audit logs. To onboard other types of GitLab logs, such as API, Exceptions, Integrations, Git, and Production logs, follow the separate How to onboard non-Audit GitLab logs to Panther process below.
- 1.In the left-hand navigation bar of your Panther Console, click Configure > Log Sources.
- 2.Click Create New.
- 3.Search for “GitLab,” then click its tile.
- In the slide-out panel, the Transport Mechanism dropdown in the upper right corner will be pre-populated with the HTTP option.
- 4.Click Start Setup.
- 5.
- You will be required to use shared secret authentication. This is the only method of authentication GitLab supports.
- The Header Name associated with your Secret Key Value will be locked with a value of
x-panther-gitlab. - Do not proceed to the next step until the creation of your HTTP endpoint has completed.
- 1.In your GitLab console, navigate to your Group.
- 2.Click Security and Compliance > Audit Events.
- 3.In the tab at the top of the page, click Streams, then Add Streaming Destination.
- 4.Enter the information from the HTTP source you created in Step 1:
- The URL provided by Panther.
- The header name
x-panther-gitlaband the secret you configured in Panther.
- 5.Click Add to finish creating the Streaming Destination.
The process below outlines how to onboard non-Audit GitLab logs to Panther, such as API, Exceptions, Git, Integrations, and Production logs. If you'd like to onboard Audit logs, follow the separate How to onboard GitLab Audit Streaming logs to Panther process above.
To connect these logs into Panther:
- 1.Log in to the Panther Console.
- 2.In the left sidebar, click Configure > Log Sources.
- 3.Click Create New.
- 4.Search for the log type you want to onboard, then click its tile.
- 5.Select the data transport method you wish to use for this integration, then follow Panther's instructions for configuring the method:
- 6.Configure GitLab to push logs to the Data Transport source.
- See GitLab's documentation for instructions on pushing logs to your selected Data Transport source.
Panther uses the latest version of GitLab API logs. Some fields differ from the official documentation.
schema: GitLab.API
description: |-
GitLab log for API requests received from GitLab.
NOTE: We are using the latest version of GitLab API logs. Some fields differ from the official documentation
referenceURL: https://docs.gitlab.com/ee/administration/logs.html#api_jsonlog
fields:
- name: time
required: true
description: The request timestamp
type: timestamp
timeFormats:
- rfc3339
isEventTime: true
- name: severity
required: true
description: The log level
type: string
- name: duration_s
required: true
description: The time spent serving the request (in seconds)
type: float
- name: db_duration_s
description: The time spent querying the database (in seconds)
type: float
- name: view_duration_s
description: The time spent rendering the view for the Rails controller (in seconds)
type: float
- name: status
required: true
description: The HTTP response status code
type: smallint
- name: method
required: true
description: The HTTP method of the request
type: string
- name: path
required: true
description: The URL path for the request
type: string
- name: params
description: The URL query parameters
type: array
element:
type: object
fields:
- name: key
required: true
description: Query parameter name
type: string
- name: value
description: Query parameter value
type: json
- name: host
required: true
description: Hostname serving the request
type: string
indicators:
- hostname
- name: ua
description: User-Agent HTTP header
type: string
- name: route
required: true
description: Rails route for the API endpoint
type: string
- name: remote_ip
description: The remote IP address of the HTTP request
type: string
indicators:
- ip
- name: user_id
description: The user id of the request
type: bigint
- name: username
description: The username of the request
type: string
indicators:
- username
- name: gitaly_calls
description: Total number of calls made to Gitaly
type: bigint
- name: gitaly_duration_s
description: Total time taken by Gitaly calls
type: float
- name: redis_calls
description: Total number of calls made to Redis
type: bigint
- name: redis_duration_s
description: Total time to retrieve data from Redis
type: float
- name: correlation_id
description: Request unique id across logs
type: string
indicators:
- trace_id
- name: queue_duration_s
description: Total time that the request was queued inside GitLab Workhorse
type: float
- name: meta.user
description: User that invoked the request
type: string
indicators:
- username
- name: meta.project
description: Project associated with the request
type: string
- name: meta.root_namespace
description: Root namespace
type: string
- name: meta.caller_id
description: Caller ID
type: string
Multi-use schema for GitLab audit events, from both self-hosted audit log files and GitLab's audit event streaming feature.
For more information, see GitLab's documentation on Audit JSON Logs and GitLab's documentation on audit event streaming.
schema: GitLab.Audit
description: 'Multi-use schema for GitLab audit events both from self-hosted audit log files, as well as GitLab''s audit event streaming feature: https://docs.gitlab.com/ee/administration/audit_event_streaming.html'
referenceURL: https://docs.gitlab.com/ee/administration/logs.html#audit_jsonlog
fields:
- name: severity
description: The log level. Present only in audit log files.
type: string
- name: time
description: The event timestamp. Present only in audit log files.
type: timestamp
timeFormats:
- rfc3339
isEventTime: true
- name: author_id
required: true
description: User id that made the change
type: bigint
- name: entity_id
required: true
description: Id of the entity that was modified
type: bigint
- name: entity_type
required: true
description: Type of the modified entity
type: string
- name: change
description: Type of change to the settings. Present only in audit log files.
type: string
- name: from
description: Old setting value. Present only in audit log files.
type: string
- name: to
description: New setting value. Present only in audit log files.
type: string
- name: author_name
required: true
description: Name of the user that made the change
type: string
- name: target_id
required: true
description: Target id of the modified setting
type: bigint
- name: target_type
required: true
description: Target type of the modified setting
type: string
- name: target_details
required: true
description: Details of the target of the modified setting
type: string
- name: created_at
description: Timestamp when event was triggered. Present only in audit event streaming
type: timestamp
timeFormats:
- rfc3339
isEventTime: true
- name: details
description: JSON object containing additional metadata. Present only in audit event streaming
type: json
- name: entity_path
description: Full path of the entity affected by the auditable event. Present only in audit event streaming
type: string
- name: event_type
description: String representation of the type of audit event. Present only in audit event streaming
type: string
- name: id
description: Unique identifier for the audit event. Present only in audit event streaming
type: bigint
- name: ip_address
description: IP address of the host used to trigger the event. Present only in audit event streaming
type: string
indicators:
- ip
GitLab log file containing changes to group or project settings
schema: GitLab.Exceptions
description: GitLab log file containing changes to group or project settings
referenceURL: https://docs.gitlab.com/ee/administration/logs.html#exceptions_jsonlog
fields:
- name: severity
required: true
description: The log level
type: string
- name: time
required: true
description: The event timestamp
type: timestamp
timeFormats:
- rfc3339
isEventTime: true
- name: correlation_id
description: Request unique id across logs
type: string
indicators:
- trace_id
- name: extra.server
description: Information about the server on which the exception occurred
type: object
fields:
- name: os
description: Server OS info
type: object
fields:
- name: name
description: OS name
type: string
- name: version
description: OS version
type: string
- name: build
description: OS build
type: string
- name: runtime
description: Runtime executing gitlab code
type: object
fields:
- name: name
description: Runtime name
type: string
- name: version
description: Runtime version
type: string
- name: extra.project_id
description: Project id where the exception occurred
type: bigint
- name: extra.relation_key
description: Relation on which the exception occurred
type: string
- name: extra.relation_index
description: Relation index on which the exception occurred
type: bigint
- name: exception.class
required: true
description: Class name of the exception that occurred
type: string
- name: exception.message
required: true
description: Message of the exception that occurred
type: string
- name: exception.backtrace
description: Stack trace of the exception that occurred
type: array
element:
type: string
GitLab log file containing all failed requests from GitLab to Git repositories.
schema: GitLab.Git
description: GitLab log file containing all failed requests from GitLab to Git repositories.
referenceURL: https://docs.gitlab.com/ee/administration/logs.html#git_jsonlog
fields:
- name: severity
required: true
description: The log level
type: string
- name: time
required: true
description: The event timestamp
type: timestamp
timeFormats:
- rfc3339
isEventTime: true
- name: correlation_id
description: Unique id across logs
type: string
indicators:
- trace_id
- name: message
required: true
description: The error message from git
type: string
GitLab log with information about integrations activities such as Jira, Asana, and Irker services.
schema: GitLab.Integrations
description: GitLab log with information about integrations activities such as Jira, Asana, and Irker services.
referenceURL: https://docs.gitlab.com/ee/administration/logs.html#integrations_jsonlog
fields:
- name: severity
required: true
description: The log level
type: string
- name: time
required: true
description: The event timestamp
type: timestamp
timeFormats:
- rfc3339
isEventTime: true
- name: service_class
required: true
description: The class name of the integrated service
type: string
- name: project_id
required: true
description: The project id the integration was running on
type: bigint
- name: project_path
required: true
description: The project path the integration was running on
type: string
- name: message
required: true
description: The log message from the service
type: string
- name: client_url
required: true
description: The client url of the service
type: string
indicators:
- url
- name: error
description: The error name if an error has occurred
type: string
GitLab log for Production controller requests received from GitLab
schema: GitLab.Production
description: GitLab log for Production controller requests received from GitLab
referenceURL: https://docs.gitlab.com/ee/administration/logs.html#production_jsonlog
fields:
- name: method
required: true
description: The HTTP method of the request
type: string
- name: path
required: true
description: The URL path for the request
type: string
- name: format
description: The response output format
type: string
- name: controller
description: The Production controller class name
type: string
- name: action
description: The Production controller action
type: string
- name: status
required: true
description: The HTTP response status code
type: bigint
- name: time
required: true
description: The request timestamp
type: timestamp
timeFormats:
- rfc3339
isEventTime: true
- name: params
description: The URL query parameters
type: array
element:
type: object
fields:
- name: key
required: true
description: Query parameter name
type: string
- name: value
description: Query parameter value
type: json
- name: remote_ip
description: The remote IP address of the HTTP request
type: string
indicators:
- ip
- name: user_id
description: The user id of the request
type: bigint
- name: username
description: The username of the request
type: string
indicators:
- username
- name: ua
description: The User-Agent of the requester
type: string
- name: queue_duration_s
description: Total time that the request was queued inside GitLab Workhorse
type: float
- name: gitaly_calls
description: Total number of calls made to Gitaly
type: bigint
- name: gitaly_duration_s
description: Total time taken by Gitaly calls
type: float
- name: redis_calls
description: Total number of calls made to Redis
type: bigint
- name: redis_duration_s
description: Total time to retrieve data from Redis
type: float
- name: redis_read_bytes
description: Total bytes read from Redis
type: bigint
- name: redis_write_bytes
description: Total bytes written to Redis
type: bigint
- name: correlation_id
description: Request unique id across logs
type: string
indicators:
- trace_id
- name: cpu_s
description: Total time spent on CPU
type: float
- name: db_duration_s
description: Total time to retrieve data from PostgreSQL
type: float
- name: view_duration_s
description: Total time taken inside the Rails views
type: float
- name: duration_s
required: true
description: Total time taken to retrieve the request
type: float
- name: meta.caller_id
description: Caller ID
type: string
- name: location
description: (Applies only to redirects) The redirect URL
type: string
- name: exception.class
description: Class name of the exception that occurred
type: string
- name: exception.message
description: Message of the exception that occurred
type: string
- name: exception.backtrace
description: Stack trace of the exception that occurred
type: array
element:
type: string
- name: etag_route
description: Route name etag (on redirects)
type: string
Last modified 4h ago