Cribl Onboarding Guide

How to redact sensitive log data using AWS S3, Cribl, and Panther

Overview

The process outlined in this guide outlines how to redact sensitive log data using Cribl Cloud or Cribl Self-Hosted, working specifically within Amazon Web Services (AWS) S3 buckets and Panther. Broadly we will:

  • Authenticate an S3 Bucket in Cribl, with examples of the manual method and Event Breakers.

  • Configure an AWS S3 source and destination using a JSON data format.

  • Provide examples of JSON events, using pre and post-Cribl masks.

How to redact sensitive fields with Cribl

Step 1: Configuring the AWS S3 Source in Cribl

  1. Click Authentication on the left. Click to choose one of the following authentication methods:

    • Manual - Requires Access and Secret Keys (You will need to generate the Access and Secret keys in AWS)

    • Secret - Secret key pair (You will need to generate this key pair in AWS)

  2. Click Event Breakers on the left. Choose the Cribl event breaker.

    • Note: This step allows Cribl to parse the incoming JSON data.

Step 2: Configuring the AWS S3 Destination in Cribl

  1. In the General Settings page, enter the following:

    • S3 Bucket name. In AWS, these S3 buckets can be the same. Objects within the bucket can be differentiated by the prefix filter if needed.

    • Bucket region

    • Prefix for the outputted JSON files

  2. Click Authentication on the left. Click to choose one of the following authentication methods:

    • Manual - Requires Access and Secret Keys (You will need to generate the Access and Secret keys in AWS)

    • Secret - Secret key pair (You will need to generate this key pair in AWS)

Step 3: Configuring the Pipeline in Cribl

    • You will be redirected to the Connection Configuration dashboard.

  1. Paste in the block of JSON below. Note that the block of JSON below has two filters:

    • Mask: Replaces everything in the name field with REDACTED

    • Eval: Removes the fields _raw cribl_breaker crible_pipe _time

    {
      "id": "redaction",
      "conf": {
        "asyncFuncTimeout": 1000,
        "functions": [
          {
            "filter": "true",
            "conf": {
              "rules": [
                {
                  "matchRegex": "/(.*)/i",
                  "replaceExpr": "`REDACTED`"
                }
              ],
              "fields": [
                "name"
              ],
              "depth": 5,
              "flags": []
            },
            "id": "mask",
            "description": "Masking Filter",
            "final": false
          },
          {
            "filter": "true",
            "conf": {
              "remove": [
                "_raw",
                "cribl_breaker",
                "cribl_pipe",
                "_time"
              ]
            },
            "id": "eval",
            "final": true
          }
        ],
        "description": "redaction-pipeline",
        "groups": {}
      }
    }
  2. Save the JSON.

Once the JSON has been saved, the UI will reflect your Masking Rules similar to the screenshot below:

The completed Data Route will look similar to the screen shot below:

JSON Example Pre and Post Cribl

The following example shows a JSON event, using the mask filter mentioned in the previous steps of this guide. After using the mask filter, the name field Bella is replaced with REDACTED.

Pre-Cribl mask:

{ ... "name": "Bella", ...} 

Post-Cribl mask:

{ ... "name":"REDACTED", ...}

We strongly advise that you verify the mask filter is working as expected after configuration. Once the mask filter is confirmed to be working correctly, ingest your masked logs into Panther using the S3 Data Transport.

Last updated