Docker Logs (Beta)

Stream Docker event logs directly to Panther over HTTPS

Overview

Docker log ingestion is in open beta starting with Panther version 1.77, and is available to all customers. Please share any bug reports and feature requests with your Panther support team.

Panther supports ingesting Docker event logs by streaming them to an HTTP Source after they are forwarded with Fluent Bit.

How to onboard Docker Events to Panther

Step 1: Create a new Docker Events log source in Panther

1. In the left-side navigation bar of your Panther Console, click Configure > Log Sources.

2. Click Create New.

3. Search for "Docker Events," then click its tile.

   • In the slide-out panel, the Transport Mechanism dropdown in the upper-right corner will be pre-populated with the HTTP option.

4. Click Start Setup.

5. Follow Panther's instructions for configuring an HTTP Source.

   • When setting the Stream Type for the source, we recommend choosing JSON, as it corresponds to Format json_lines in the Fluent Bit configuration in the next step.

   • When setting the Auth method for the source, we recommend using Shared Secret.

After creating the HTTP source, the Panther Console will display your HTTP Source URL. Store this value in a secure location, as you will need it in the next steps.

Step 2: Configure Fluent Bit

1. Follow the Getting Started with Fluent Bit instructions to install Fluent Bit as a service.

2. Create a Fluent Bit configuration file.

   • See sample configuration files for Linux and macOS below under Example configuration files.

3. Start Fluent Bit, passing the path to your new configuration file.

   • You must include the path to a parser.conf file that contains the docker parser. This is included by fluent-bit by default. On Linux, it can be found at /etc/fluent-bit/parsers.conf.

     • Example: fluent-bit -c fluentbit.conf -R /etc/fluent-bit/parsers.conf

Example configuration files

Linux

Configure the following in your Fluent Bit configuration file:

• [INPUT] variables:

  • Name: Set this to docker_events.

  • Parser: Set this to docker.

• [OUTPUT] variables:

  • Host: Enter your Panther URL.

    • Example: logs.instance-name.runpanther.net

  • URI: Enter the end of the HTTP Source ingest URL (generated in Step 1 of this process), starting with /http/.

    • Example: /http/cb015ee4-543c-4489-9f4b-testaa16d7a

  • Header: Enter the header name you created and the secret you generated while configuring your HTTP source in the Panther Console in Step 1.

[SERVICE]
  flush      1

[INPUT]
  name       docker_events
  parser     docker

[OUTPUT]
  name       http
  match      *
  host       logs.instance.runpanther.net
  port       443
  URI        /http/70c55034-13f1-4e08-a018-test5f2bb0a8
  Header     x-panther-secret {YOUR_SECRET_HERE}
  Format     json_lines
  TLS        on
  TLS.Verify on

macOS

Configure the following in your Fluent Bit configuration file:

• [INPUT] variables:

  • Name: Set this to docker_events.

    • On macOS, docker_events is not supported by fluent-bit. Instead you can use the Exec input plugin. See the sample configuration for macOS below.

  • Parser: Set this to docker.

• [OUTPUT] variables:

  • Host: Enter your Panther URL.

    • Example: logs.instance-name.runpanther.net

  • URI: Enter the end of the HTTP Source URL (generated in Step 1 of this process), starting with /http/.

    • Example: /http/cb015ee4-543c-4489-9f4b-testaa16d7a

  • Header: Enter the header name you created and the secret you generated while configuring your HTTP source in the Panther Console in Step 1.

• Start Fluent Bit, passing the path to your new configuration file.

  • Note: you must include the path to a parser.conf file that contains the docker parser. This is included by fluent-bit by default. On Linux, it can be found at /etc/fluent-bit/parsers.conf.

  • Example: fluent-bit -c fluentbit.conf -R /etc/fluent-bit/parsers.conf

[SERVICE]
  flush      1

[INPUT]
  name         exec
  parser       docker
  Command      docker events --since 10s --until 0s --format '{{json .}}'
  Interval_Sec 10

[OUTPUT]
  name       http
  match      *
  host       logs.instance.runpanther.net
  port       443
  URI        /http/70c55034-13f1-4e08-a018-2c005f2bb0a8
  Header     x-panther-secret {YOUR_SECRET_HERE}
  Format     json_lines
  TLS        on
  TLS.Verify on

Supported log types

Required fields in the schema are listed as "required: true"

Docker.Events

The following defines the Docker events log schema:

schema: Docker.Events
description: Docker events to audit system-level management operations against containers, images, networks, and volumes
referenceURL: https://docs.docker.com/engine/reference/commandline/events/
fields:
  - name: status
    description: String representing the status of this event's operation
    type: string
  - name: id
    description: Event id
    type: string
  - name: from
    description: Context of where the event originated
    type: string
  - name: Type
    description: Type of object being operated on. See reference for list of object types
    type: string
  - name: Action
    description: The action performed. Different Types have different possible actions. See reference for list of event types
    type: string
  - name: Actor
    description: The actor performing the event. Note in the context of Docker, this is not necessarily an end user, but can be the container itself
    type: object
    fields:
      - name: ID
        description: The actor ID
        type: string
        indicators:
          - actor_id
      - name: Attributes
        description: Event specific details
        type: json
  - name: scope
    description: Different event types have different scopes. Local scoped events are only seen on the node they take place on, and swarm scoped events are seen on all managers.
    type: string
  - name: time
    required: true
    description: Time the event occurred in unix seconds
    type: timestamp
    timeFormats:
      - unix
    isEventTime: true
  - name: timeNano
    description: Event time but in nanoseconds
    type: string