Cloudflare Logs

Connecting Cloudfare logs to your Panther Console

Overview

Panther supports ingesting Cloudflare logs via Cloudflare's Logpush service, which streams logs directly to an HTTP Source, or to Amazon Web Services (AWS) S3.

Note that Cloudflare's Logpush is available to Cloudflare Enterprise customers only. While some Cloudflare log types on this page (e.g., Audit logs) may be pulled without Logpush, Panther's supported schemas rely on the data structure when delivered by Logpush.

How to onboard Cloudflare logs to Panther

You can ingest Cloudflare logs into Panther by streaming them to either an HTTP source or a S3 source.

Step 1: Create an HTTP Source in Panther

In the left-hand navigation bar of your Panther Console, click Configure > Log Sources.
Click Create New.
Search for "Cloudflare," then click its tile.
- In the slide-out panel, the Transport Mechanism dropdown in the upper right corner will be pre-populated with the HTTP option.
Click Start Setup.
Follow Panther's instructions for configuring an HTTP Source.
- You will be required to use shared secret authentication. This is the only method of authentication Cloudflare supports.
- The Header Name associated with your Secret Key Value will be locked with a value of x-panther-cloudflare.

Step 2: Configure a Logpush job in Cloudflare

Locate your Cloudflare account ID. by navigating to your Cloudflare dashboard and copying the ID from the URL.
Create a Cloudflare API token by following Cloudflare's Create an API token documentation.
- Ensure the token has the All accounts - Logs: Edit permission.
- Save the API token for the following step.

Create a Logpush job in Cloudflare by invoking the API, as is shown in the curl example below.

curl -X POST "https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/logpush/jobs" \
    -H "Authorization: Bearer {YOUR_API_TOKEN}" \
    -H "Content-Type:application/json" \
    -d '{
     "enabled": true,
     "name": "my_cloudflare_audit_logs",
     "dataset": "audit_logs",
     "destination_conf": "{LOG_SOURCE_URL}?header_x-panther-cloudflare={SHARED_SECRET}"
     }'

See additional information on setting up a Logpush job on Cloudflare's Enable HTTP destination documentation.

Navigate back to your Cloudflare dashboard to finish the Logpush job configuration.
1. In the left-hand navigation bar, under Analytics & Logs, click Logs.
2. In the Logpush job table, find the row for the Logpush job you created in the previous step, and click Edit.
3. Select the fields you would like Cloudflare to include in the audit log events sent to your HTTP Source. (By default, Cloudflare only includes a subset of all available fields.)
4. Click Save changes.

Prerequisites

Create a new S3 bucket in your AWS account.
- We recommend creating a new S3 bucket specifically for Cloudflare logs. You can use the default settings.
- Please note the region that you are creating it in, as you will need to provide it to Cloudflare.

Step 1: Set up the S3 bucket source in Panther

Follow Panther’s documentation for configuring AWS S3 as a Data Transport.

Step 2: Configure Logpush to stream logs to S3

When choosing the dataset type for your Logpush job, note that Cloudflare has options for Account-scoped data and Zone-scoped data. Audit logs are Account-scoped, whereas Firewall, HttpRequest, and Spectrum are Zone-scoped. The screen shots in this guide demonstrate how to configure Audit logs.

Navigate to your Cloudflare dashboard, then go to Analytics > Logs.
Click Add Logpush job next to the dataset type you want to set up.
Click Select on the dataset you want to choose.
Scroll down and click Next.
Select the fields that you want to include. We recommend that you click the checkbox to select "All fields."
- ID, ResourceType, and When are included by default and are required by Panther. Do not unselect these fields.
Click Next.
Under "Select a destination," locate the Amazon S3 tile and click Select.
Scroll down and click Next.
Configure the destination fields:
- Bucket path: Enter the name of the S3 bucket you created earlier in this documentation.
- Daily subfolders: Choose Yes if you want to organize logs into daily subfolders.
- Bucket region: Select the region that you created your S3 bucket in.
  - To verify the region, navigate to your S3 bucket's Properties.
- Encryption constraint in bucket policy: Set to "Yes" if you enabled SSE encryption on your S3 bucket.
- In the Grant Cloudflare access to upload files to your bucket section, copy the IAM policy that Cloudflare is providing.
  - You will need this in the next steps to give Cloudflare access to put objects in your bucket.
In a separate browser tab, navigate to your AWS account and go to the S3 bucket settings.
At the top of the page, click the Permissions tab.
Scroll down to the Bucket policy section. On the right side of the tile, click Edit. In the text editor, paste the bucket policy you copied from the earlier steps.
Click Save.
Navigate back to the Cloudflare dashboard, and click Validate Access.
- From here, Cloudflare will write an object to your S3 bucket which will contain an ownership challenge that you must provide back to Cloudflare in the next steps.
Navigate back to your S3 bucket in AWS.
Find and download the ownership challenge file, then open it.
Copy the contents of the ownership challenge file. You will need this in the next steps.
Navigate back to your Cloudflare dashboard. In the Ownership token field, paste in the contents of the ownership challenge file.
Click Save.

Additionally, see Cloudflare's documentation for instructions on pushing logs to Amazon S3.

Panther-built Detections

See Panther's built in rules for Cloudflare in panther-analysis in Github.

Supported log types

Required fields in all tables are in bold.

Cloudflare.Audit

When selecting event fields on the Cloudflare UI, make sure you include the When, ID, and ResourceType fields, as they are required by Panther.

# Code generated by Panther; DO NOT EDIT. (@generated)
schema: Cloudflare.Audit
parser:
  native:
    name: Cloudflare.Audit
description: Audit logs summarize the history of changes made within your Cloudflare account. Audit logs include account level actions like login and logout, as well as zone configuration changes.
referenceURL: https://developers.cloudflare.com/logs/reference/log-fields/account/audit_logs
fields:
  - name: ActionResult
    description: Whether the action was successful
    type: boolean
  - name: ActionType
    description: Type of action taken
    type: string
  - name: ActorEmail
    description: Email of the actor
    type: string
    indicators:
      - email
  - name: ActorID
    description: Unique identifier of the actor in Cloudflare's system
    type: string
    indicators:
      - username
  - name: ActorIP
    description: Physical network address of the actor
    type: string
    indicators:
      - ip
  - name: ActorType
    description: Type of user that started the audit trail
    type: string
  - name: ID
    required: true
    description: Unique identifier of an audit log
    type: string
  - name: Interface
    description: Entry point or interface of the audit log
    type: string
  - name: Metadata
    description: Additional audit log-specific information. Metadata is organized in key:value pairs. Key and Value formats can vary by ResourceType.
    type: json
  - name: NewValue
    description: Contains the new value for the audited item
    type: json
  - name: OldValue
    description: Contains the old value for the audited item
    type: json
  - name: OwnerID
    description: The identifier of the user that was acting or was acted on behalf of. If a user did the action themselves, this value will be the same as the ActorID.
    type: string
    indicators:
      - username
  - name: ResourceID
    description: Unique identifier of the resource within Cloudflares system
    type: string
  - name: ResourceType
    required: true
    description: The type of resource that was changed
    type: string
  - name: When
    required: true
    description: When the change happened
    type: timestamp
    timeFormats:
      - cloudflare
    isEventTime: true

Cloudflare.Firewall

When selecting event fields on the Cloudflare UI, make sure you include the "Datetime" field, as it is required by Panther.

Reference: Cloudfare Documentation on Log Field Firewalls.

Column

Type

Description

Action

string

The code of the first-class action the Cloudflare Firewall took on this request

ClientASN

bigint

The ASN number of the visitor

ClientASNDescription

string

The ASN of the visitor as string

ClientCountry

string

Country from which request originated

ClientIP

string

The visitor's IP address (IPv4 or IPv6)

ClientIPClass

string

ClientRefererHost

string

The referer host

ClientRefererPath

string

The referer path requested by visitor

ClientRefererQuery

string

The referer query-string was requested by the visitor

ClientRefererScheme

string

The referer url scheme requested by the visitor

ClientRequestHost

string

The HTTP hostname requested by the visitor

ClientRequestMethod

string

The HTTP method used by the visitor

ClientRequestPath

string

The path requested by visitor

ClientRequestProtocol

string

The version of HTTP protocol requested by the visitor

ClientRequestQuery

string

The query-string was requested by the visitor

ClientRequestScheme

string

The url scheme requested by the visitor

ClientRequestUserAgent

string

Visitor's user-agent string

Description

string

Rule description for this event

Datetime

timestamp

The date and time the event occurred at the edge

EdgeColoCode

string

The airport code of the Cloudflare datacenter that served this request

EdgeResponseStatus

smallint

HTTP response status code returned to browser

Kind

string

The kind of event, currently only possible values are: firewall

MatchIndex

bigint

Rules match index in the chain

Metadata

{ string:string }

Additional product-specific information. Metadata is organized in key:value pairs. Key and Value formats can vary by Cloudflare security product and can change over time

OriginResponseStatus

smallint

HTTP origin response status code returned to browser

OriginatorRayID

string

The RayID of the request that issued the challenge/jschallenge

RayID

string

The RayID of the request

Ref

string

User-defined rule reference for this event

RuleID

string

The Cloudflare security product-specific RuleID triggered by this request

Source

string

The Cloudflare security product triggered by this request

p_event_time

timestamp

Panther added standardized event time (UTC)

p_parse_time

timestamp

Panther added standardized log parse time (UTC)

p_log_type

string

Panther added field with type of log

p_row_id

string

Panther added field with unique id (within table)

p_source_id

string

Panther added field with the source id

p_source_label

string

Panther added field with the source label

p_any_ip_addresses

[string]

Panther added field with collection of ip addresses associated with the row

p_any_domain_names

[string]

Panther added field with collection of domain names associated with the row

p_any_trace_ids

[string]

Panther added field with collection of context trace identifiers

Cloudflare.HttpRequest

When selecting event fields on the Cloudflare UI, make sure you include the "EdgeStartTimestamp" field, as it is required by Panther.

Reference: Cloudfare Documentation on Log Field Requests.

Column

Type

Description

BotDetectionIDs

[bigint]

List of IDs that correlate to the Bot Management Heuristic detections made on a request. Available in Logpush v2 only.

BotScore

bigint

Cloudflare Bot Score (available for Bot Management customers; please contact your account team to enable)

BotScoreSrc

string

Underlying detection engine or source on where a Bot Score is calculated. Possible values are Not Computed | Heuristics | Machine Learning | Behavioral Analysis | Verified Bot

BotTags

[string]

Type of bot traffic (if available). Refer to Bot Tags for the list of potential values. Available in Logpush v2 only.

CacheCacheStatus

string

CacheReserveUsed

boolean

Cache Reserve was used to serve this request. Available in Logpush v2 only.

CacheResponseBytes

bigint

Number of bytes returned by the cache

CacheResponseStatus

smallint

HTTP status code returned by the cache to the edge; all requests (including non-cacheable ones) go through the cache; also see CacheStatus field

CacheTieredFill

boolean

Tiered Cache was used to serve this request

ClientASN

bigint

Client AS number

ClientCountry

string

Country of the client IP address

ClientDeviceType

string

Client device type

ClientIP

string

IP address of the client

ClientIPClass

string

ClientMTLSAuthCertFingerprint

string

The SHA256 fingerprint of the certificate presented by the client during mTLS authentication. Only populated on the first request on an mTLS connection. Available in Logpush v2 only.

ClientMTLSAuthStatus

string

ClientRegionCode

string

The ISO-3166-2 region code of the client IP address.

ClientRequestBytes

bigint

Number of bytes in the client request

ClientRequestHost

string

Host requested by the client

ClientRequestMethod

string

HTTP method of client request

ClientRequestPath

string

URI path requested by the client

ClientRequestProtocol

string

HTTP protocol of client request

ClientRequestReferer

string

HTTP request referrer

ClientRequestScheme

string

The URL scheme requested by the visitor. Available in Logpush v2 only.

ClientRequestSource

string

Identifies requests as coming from an external source or another service within Cloudflare. Refer to ClientRequestSource field for the list of potential values. Available in Logpush v2 only.

ClientRequestURI

string

URI requested by the client

ClientRequestUserAgent

string

User agent reported by the client

ClientSSLCipher

string

Client SSL cipher

ClientSSLProtocol

string

Client SSL (TLS) protocol

ClientSrcPort

int

Client source port

ClientTCPRTTMs

bigint

The smoothed average of TCP round-trip time (SRTT). For the initial request on a connection, this is measured only during connection setup. For a subsequent request on the same connection, it is measured over the entire connection lifetime up until the time that request is received. Available in Logpush v2 only.

ClientXRequestedWith

string

X-Requested-With HTTP header

ContentScanObjResults

[string]

List of content scan results.

ContentScanObjTypes

[string]

List of content types.

json

String key-value pairs for Cookies.

EdgeCFConnectingO2O

boolean

True if the request looped through multiple zones on the Cloudflare edge. This is considered an orange to orange (o2o) request. Available in Logpush v2 only.

EdgeColoCode

string

IATA airport code of data center that received the request

EdgeColoID

bigint

Cloudflare edge colo id

EdgeEndTimestamp

timestamp

Timestamp at which the edge finished sending response to the client

EdgePathingOp

string

Indicates what type of response was issued for this request (unknown = no specific action)

EdgePathingSrc

string

Details how the request was classified based on security checks (unknown = no specific classification)

EdgePathingStatus

string

Indicates what data was used to determine the handling of this request (unknown = no data)

EdgeRateLimitAction

string

The action taken by the blocking rule; empty if no action taken

EdgeRateLimitID

string

The internal rule ID of the rate-limiting rule that triggered a block (ban) or simulate action. 0 if no action taken

EdgeRequestHost

string

Host header on the request from the edge to the origin

EdgeResponseBodyBytes

bigint

Size of the HTTP response body returned to clients. Available in Logpush v2 only.

EdgeResponseBytes

bigint

Number of bytes returned by the edge to the client

EdgeResponseCompressionRatio

float

Edge response compression ratio

EdgeResponseContentType

string

Edge response Content-Type header value

EdgeResponseStatus

smallint

HTTP status code returned by Cloudflare to the client

EdgeServerIP

string

IP of the edge server making a request to the origin

EdgeStartTimestamp

timestamp

Timestamp at which the edge received request from the client

EdgeTimeToFirstByteMs

bigint

Total view of Time To First Byte as measured at Cloudflare’s edge. Starts after a TCP connection is established and ends when Cloudflare begins returning the first byte of a response to eyeballs. Includes TLS handshake time (for new connections) and origin response time. Available in Logpush v2 only.

FirewallMatchesActions

[string]

FirewallMatchesRuleIDs

[string]

Array of RuleIDs of the firewall product that has matched the request. The firewall product associated with the RuleID can be found in FirewallMatchesSources. The length of the array is the same as FirewallMatchesActions and FirewallMatchesSources.

FirewallMatchesSources

[string]

JA3Hash

string

The MD5 hash of the JA3 fingerprint used to profile SSL/TLS clients. Available in Logpush v2 only.

OriginDNSResponseTimeMs

bigint

Time taken to receive a DNS response for an origin name. Usually takes a few milliseconds, but may be longer if a CNAME record is used. Available in Logpush v2 only.

OriginIP

string

IP of the origin server

OriginRequestHeaderSendDurationMs

bigint

Time taken to send request headers to origin after establishing a connection. Note that this value is usually 0. Available in Logpush v2 only.

OriginResponseBytes

bigint

Number of bytes returned by the origin server

OriginResponseDurationMs

bigint

Upstream response time, measured from the first datacenter that receives a request. Includes time taken by Argo Smart Routing and Tiered Cache, plus time to connect and receive a response from origin servers. This field replaces OriginResponseTime. Available in Logpush v2 only.

OriginResponseHeaderReceiveDurationMs

bigint

Time taken for origin to return response headers after Cloudflare finishes sending request headers. Available in Logpush v2 only.

OriginResponseHTTPExpires

timestamp

Value of the origin 'expires' header in RFC1123 format

OriginResponseHTTPLastModified

timestamp

Value of the origin 'last-modified' header in RFC1123 format

OriginResponseStatus

smallint

Status returned by the origin server

OriginResponseTime

bigint

Number of nanoseconds it took the origin to return the response to edge

OriginSSLProtocol

string

SSL (TLS) protocol used to connect to the origin

OriginTCPHandshakeDurationMs

bigint

Time taken to complete TCP handshake with origin. This will be 0 if an origin connection is reused. Available in Logpush v2 only.

OriginTLSHandshakeDurationMs

bigint

Time taken to complete TLS handshake with origin. This will be 0 if an origin connection is reused. Available in Logpush v2 only.

ParentRayID

string

Ray ID of the parent request if this request was made using a Worker script

RayID

string

ID of the request

RequestHeaders

json

String key-value pairs for RequestHeaders

ResponseHeaders

json

String key-value pairs for ResponseHeaders

SecurityAction

string

Rule action of the security rule that triggered a terminating action, if any

SecurityActions

[string]

Array of actions that Cloudflare security products performed on this request.

SecurityLevel

string

The security level configured at the time of this request. This is used to determine the sensitivity of the IP Reputation system

SecurityRuleDescription

string

Rule description of the security rule that triggered a terminating action, if any

SecurityRuleID

string

Rule ID of the security rule that triggered a terminating action, if any

SecurityRuleIDs

[string]

Array of security rule IDs that matched the request

SmartRouteColoID

bigint

The Cloudflare datacenter used to connect to the origin server if Argo Smart Routing is used. Available in Logpush v2 only.

SecuritySources

[string]

Array of Cloudflare security products that matched the request.

UpperTierColoID

bigint

The “upper tier” datacenter that was checked for a cached copy if Tiered Cache is used. Available in Logpush v2 only.

WAFAction

string

Action taken by the WAF, if triggered

WAFAttackScore

bigint

Overall request score generated by the WAF detection module.

WAFFlags

string

Additional configuration flags: simulate (0x1) | null

WAFMatchedVar

string

The full name of the most-recently matched variable

WAFProfile

string

low | med | high

WAFRCEAttackScore

bigint

WAF score for an RCE attack.

WAFRuleID

string

ID of the applied WAF rule

WAFRuleMessage

string

Rule message associated with the triggered rule

WAFSQLiAttackScore

bigint

WAF score for an SQLi attack.

WAFXSSAttackScore

bigint

WAF score for an XSS attack.

WorkerCPUTime

bigint

Amount of time in microseconds spent executing a worker, if any

WorkerStatus

string

Status returned from worker daemon

WorkerSubrequest

boolean

Whether or not this request was a worker subrequest

WorkerSubrequestCount

bigint

Number of subrequests issued by a worker when handling this request

WorkerWallTimeUs

bigint

Real-time in microseconds elapsed between start and end of worker invocation.

ZoneID

bigint

Internal zone ID

ZoneName

string

The human-readable name of the zone (e.g. ‘cloudflare.com’). Available in Logpush v2 only.

p_event_time

timestamp

Panther added standardized event time (UTC)

p_parse_time

timestamp

Panther added standardized log parse time (UTC)

p_log_type

string

Panther added field with type of log

p_row_id

string

Panther added field with unique id (within table)

p_source_id

string

Panther added field with the source id

p_source_label

string

Panther added field with the source label

p_any_ip_addresses

[string]

Panther added field with collection of ip addresses associated with the row

p_any_domain_names

[string]

Panther added field with collection of domain names associated with the row

p_any_trace_ids

[string]

Panther added field with collection of context trace identifiers

Cloudflare.Spectrum

When selecting event fields on the Cloudflare UI, make sure you include the "Timestamp" field, as it is required by Panther.

Reference: Cloudfare Documentation on Log Field Spectrum Events.

Column

Type

Description

Application

string

The unique public ID of the application on which the event occurred

ClientASN

bigint

Client AS number

ClientBytes

bigint

The number of bytes read from the client by the Spectrum service

ClientCountry

string

Country of the client IP address

ClientIP

string

IP address of the client

ClientMatchedIpFirewall

string

ClientPort

int

Client port

ClientProto

string

Transport protocol used by client; tcp | udp | unix

ClientTcpRtt

bigint

The TCP round-trip time in nanoseconds between the client and Spectrum

ClientTlsCipher

string

The cipher negotiated between the client and Spectrum

ClientTlsClientHelloServerName

string

The server name in the Client Hello message from client to Spectrum

ClientTlsProtocol

string

ClientTlsStatus

string

ColoCode

string

IATA airport code of data center that received the request

ConnectTimestamp

timestamp

Timestamp at which both legs of the connection (client/edge, edge/origin or nexthop) were established

DisconnectTimestamp

timestamp

Timestamp at which the connection was closed

Event

string

IpFirewall

boolean

Whether IP Firewall was enabled at time of connection

OriginBytes

bigint

The number of bytes read from the origin by Spectrum

OriginIP

string

Origin IP address

OriginPort

int

Origin port

OriginProto

string

Transport protocol used by origin; tcp | udp | unix

OriginTcpRtt

bigint

The TCP round-trip time in nanoseconds between Spectrum and the origin

OriginTlsCipher

string

The cipher negotiated between Spectrum and the origin

OriginTlsFingerprint

string

SHA256 hash of origin certificate

OriginTlsMode

string

If and how the upstream connection is encrypted; unknown | off | flexible | full | strict

OriginTlsProtocol

string

OriginTlsStatus

string

ProxyProtocol

string

Which form of proxy protocol is applied to the given connection; off | v1 | v2 | simple

Status

bigint

A code indicating reason for connection closure

Timestamp

timestamp

Timestamp at which the event took place

p_event_time

timestamp

Panther added standardized event time (UTC)

p_parse_time

timestamp

Panther added standardized log parse time (UTC)

p_log_type

string

Panther added field with type of log

p_row_id

string

Panther added field with unique id (within table)

p_source_id

string

Panther added field with the source id

p_source_label

string

Panther added field with the source label

p_any_ip_addresses

[string]

Panther added field with collection of ip addresses associated with the row

PreviousCisco Umbrella Logs NextCrowdStrike Logs

Last updated 1 year ago

Was this helpful?