Links

Cloudflare Logs

Connecting Cloudfare logs to your Panther Console

Overview

Panther supports ingesting Cloudfare logs via Cloudflare's Logpush service which streams logs directly to Amazon Web Services (AWS) S3.

How to onboard Cloudflare logs to Panther

Prerequisites

  • Create a new S3 bucket in your AWS account.
    • We recommend creating a new S3 bucket specifically for Cloudflare logs. You can use the default settings.
    • Please note the region that you are creating it in, as you will need to provide it to Cloudflare.

Step 1: Set up the Data Transport in the Panther Console

  • Please follow Panther’s documentation for configuring AWS S3 as a Data Transport.

Step 2: Configure Logpush to stream logs to S3

When choosing the dataset type for your Logpush job, note that Cloudflare has options for Account-scoped data and Zone-scoped data. Audit logs are Account-scoped, whereas Firewall, HttpRequest, and Spectrum are Zone-scoped. The screen shots in this guide demonstrate how to configure Audit logs.
  1. 1.
    Navigate to your Cloudflare dashboard, then go to Analytics > Logs.
  2. 2.
    Click Add Logpush job next to the dataset type you want to set up.
    In Cloudflare under a "Logs" header, there is an option for account-scoped data.
  3. 3.
    Click Select on the dataset you want to choose.
    In Cloudflare under "Select data set," the "Audit logs" option is highlighted.
  4. 4.
    Scroll down and click Next.
  5. 5.
    Select the fields that you want to include. We recommend that you click the checkbox to select "All fields."
    • ID, ResourceType, and When are included by default and are required by Panther. Do not unselect these fields.
      In the "Create Jobpush log" screen in AWS, the boxes are checked next to "All fields" and "General."
  6. 6.
    Click Next.
  7. 7.
    Under "Select a destination," locate the Amazon S3 tile and click Select.
    The AWS S3 tile is selected.
  8. 8.
    Scroll down and click Next.
  9. 9.
    Configure the destination fields:
    • Bucket path: Enter the name of the S3 bucket you created earlier in this documentation.
    • Daily subfolders: Choose Yes if you want to organize logs into daily subfolders.
    • Bucket region: Select the region that you created your S3 bucket in.
      • To verify the region, navigate to your S3 bucket's Properties.
    • Encryption constraint in bucket policy: Set to "Yes" if you enabled SSE encryption on your S3 bucket.
    • In the Grant Cloudflare access to upload files to your bucket section, copy the IAM policy that Cloudflare is providing.
      • You will need this in the next steps to give Cloudflare access to put objects in your bucket.
        Under "Create Logpush job," there are fields to configure the destination.
  10. 10.
    In a separate browser tab, navigate to your AWS account and go to the S3 bucket settings.
  11. 11.
    At the top of the page, click the Permissions tab.
  12. 12.
    Scroll down to the Bucket policy section. On the right side of the tile, click Edit. In the text editor, paste the bucket policy you copied from the earlier steps.
    The "Permission" tab is highlighted. At the bottom, there is a section labeled "Bucket Policy."
  13. 13.
    Click Save.
  14. 14.
    Navigate back to the Cloudflare dashboard, and click Validate Access.
    • From here, Cloudflare will write an object to your S3 bucket which will contain an ownership challenge that you must provide back to Cloudflare in the next steps.
      The page is labeled "Create Logpush job." There is a "Prove ownership" header and a text box below labeled "Ownership token."
  15. 15.
    Navigate back to your S3 bucket in AWS.
  16. 16.
    Find and download the ownership challenge file, then open it.
  17. 17.
    Copy the contents of the ownership challenge file. You will need this in the next steps.
  18. 18.
    Navigate back to your Cloudflare dashboard. In the Ownership token field, paste in the contents of the ownership challenge file.
  19. 19.
    Click Save.

Finish the setup in Panther

Once the Logpush job is configured, go back to Panther.
  1. 1.
    In the Panther Console, navigate to Configure > Log Sources.
  2. 2.
    Click Create New.
  3. 3.
    Search for Cloudflare and start the S3 onboarding process to allow Panther to read from the S3 bucket you just created.
In the Panther Console, the log source page is open and "cloudflare" was typed into the search bar.
Additionally, see Cloudflare's documentation for instructions on pushing logs to Amazon S3.

Panther-built Detections

Supported log types

Required fields in all tables are in bold.

Cloudflare.Audit

When selecting event fields on the Cloudflare UI, make sure you include the When, ID, and ResourceType fields, as they are required by Panther.
# Code generated by Panther; DO NOT EDIT. (@generated)
schema: Cloudflare.Audit
parser:
native:
name: Cloudflare.Audit
description: Audit logs summarize the history of changes made within your Cloudflare account. Audit logs include account level actions like login and logout, as well as zone configuration changes.
referenceURL: https://developers.cloudflare.com/logs/reference/log-fields/account/audit_logs
fields:
- name: ActionResult
description: Whether the action was successful
type: boolean
- name: ActionType
description: Type of action taken
type: string
- name: ActorEmail
description: Email of the actor
type: string
indicators:
- email
- name: ActorID
description: Unique identifier of the actor in Cloudflare's system
type: string
indicators:
- username
- name: ActorIP
description: Physical network address of the actor
type: string
indicators:
- ip
- name: ActorType
description: Type of user that started the audit trail
type: string
- name: ID
required: true
description: Unique identifier of an audit log
type: string
- name: Interface
description: Entry point or interface of the audit log
type: string
- name: Metadata
description: Additional audit log-specific information. Metadata is organized in key:value pairs. Key and Value formats can vary by ResourceType.
type: json
- name: NewValue
description: Contains the new value for the audited item
type: json
- name: OldValue
description: Contains the old value for the audited item
type: json
- name: OwnerID
description: The identifier of the user that was acting or was acted on behalf of. If a user did the action themselves, this value will be the same as the ActorID.
type: string
indicators:
- username
- name: ResourceID
description: Unique identifier of the resource within Cloudflares system
type: string
- name: ResourceType
required: true
description: The type of resource that was changed
type: string
- name: When
required: true
description: When the change happened
type: timestamp
timeFormats:
- cloudflare
isEventTime: true

Cloudflare.Firewall

When selecting event fields on the Cloudflare UI, make sure you include the "Datetime" field, as it is required by Panther.
Column
Type
Description
Action
string
The code of the first-class action the Cloudflare Firewall took on this request
ClientASN
bigint
The ASN number of the visitor
ClientASNDescription
string
The ASN of the visitor as string
ClientCountry
string
Country from which request originated
ClientIP
string
The visitor's IP address (IPv4 or IPv6)
ClientIPClass
string
The classification of the visitor's IP address, possible values are: unknown | clean | badHost | searchEngine | whitelist | greylist | monitoringService |securityScanner | noRecord | scan | backupService | mobilePlatform | tor
ClientRefererHost
string
The referer host
ClientRefererPath
string
The referer path requested by visitor
ClientRefererQuery
string
The referer query-string was requested by the visitor
ClientRefererScheme
string
The referer url scheme requested by the visitor
ClientRequestHost
string
The HTTP hostname requested by the visitor
ClientRequestMethod
string
The HTTP method used by the visitor
ClientRequestPath
string
The path requested by visitor
ClientRequestProtocol
string
The version of HTTP protocol requested by the visitor
ClientRequestQuery
string
The query-string was requested by the visitor
ClientRequestScheme
string
The url scheme requested by the visitor
ClientRequestUserAgent
string
Visitor's user-agent string
Datetime
timestamp
The date and time the event occurred at the edge
EdgeColoCode
string
The airport code of the Cloudflare datacenter that served this request
EdgeResponseStatus
smallint
HTTP response status code returned to browser
Kind
string
The kind of event, currently only possible values are: firewall
MatchIndex
bigint
Rules match index in the chain
Metadata
{ string:string }
Additional product-specific information. Metadata is organized in key:value pairs. Key and Value formats can vary by Cloudflare security product and can change over time
OriginResponseStatus
smallint
HTTP origin response status code returned to browser
OriginatorRayID
string
The RayID of the request that issued the challenge/jschallenge
RayID
string
The RayID of the request
RuleID
string
The Cloudflare security product-specific RuleID triggered by this request
Source
string
The Cloudflare security product triggered by this request
p_event_time
timestamp
Panther added standardized event time (UTC)
p_parse_time
timestamp
Panther added standardized log parse time (UTC)
p_log_type
string
Panther added field with type of log
p_row_id
string
Panther added field with unique id (within table)
p_source_id
string
Panther added field with the source id
p_source_label
string
Panther added field with the source label
p_any_ip_addresses
[string]
Panther added field with collection of ip addresses associated with the row
p_any_domain_names
[string]
Panther added field with collection of domain names associated with the row
p_any_trace_ids
[string]
Panther added field with collection of context trace identifiers

Cloudflare.HttpRequest

When selecting event fields on the Cloudflare UI, make sure you include the "EdgeStartTimestamp" field, as it is required by Panther.
Column
Type
Description
BotScore
bigint
Cloudflare Bot Score (available for Bot Management customers; please contact your account team to enable)
BotScoreSrc
string
Underlying detection engine or source on where a Bot Score is calculated. Possible values are Not Computed | Heuristics | Machine Learning | Behavioral Analysis | Verified Bot
BotTags
[string]
Type of bot traffic (if available). Refer to Bot Tags for the list of potential values. Available in Logpush v2 only.
CacheCacheStatus
string
unknown | miss | expired | updating | stale | hit | ignored | bypass | revalidated
CacheResponseBytes
bigint
Number of bytes returned by the cache
CacheResponseStatus
smallint
HTTP status code returned by the cache to the edge; all requests (including non-cacheable ones) go through the cache; also see CacheStatus field
CacheTieredFill
boolean
Tiered Cache was used to serve this request
ClientASN
bigint
Client AS number
ClientCountry
string
Country of the client IP address
ClientDeviceType
string
Client device type
ClientIP
string
IP address of the client
ClientIPClass
string
unknown | clean | badHost | searchEngine | whitelist | greylist | monitoringService | securityScanner | noRecord | scan |backupService | mobilePlatform | tor
ClientMTLSAuthCertFingerprint
string
The SHA256 fingerprint of the certificate presented by the client during mTLS authentication. Only populated on the first request on an mTLS connection. Available in Logpush v2 only.
ClientMTLSAuthStatus
string
The status of mTLS authentication. Only populated on the first request on an mTLS connection. Available in Logpush v2 only. Possible values are unknown | ok | absent | untrusted | notyetvalid | expired
ClientRequestBytes
bigint
Number of bytes in the client request
ClientRequestHost
string
Host requested by the client
ClientRequestMethod
string
HTTP method of client request
ClientRequestPath
string
URI path requested by the client
ClientRequestProtocol
string
HTTP protocol of client request
ClientRequestReferer
string
HTTP request referrer
ClientRequestScheme
string
The URL scheme requested by the visitor. Available in Logpush v2 only.
ClientRequestSource
string
Identifies requests as coming from an external source or another service within Cloudflare. Refer to ClientRequestSource field for the list of potential values. Available in Logpush v2 only.
ClientRequestURI
string
URI requested by the client
ClientRequestUserAgent
string
User agent reported by the client
ClientSSLCipher
string
Client SSL cipher
ClientSSLProtocol
string
Client SSL (TLS) protocol
ClientSrcPort
int
Client source port
ClientTCPRTTMs
bigint
The smoothed average of TCP round-trip time (SRTT). For the initial request on a connection, this is measured only during connection setup. For a subsequent request on the same connection, it is measured over the entire connection lifetime up until the time that request is received. Available in Logpush v2 only.
ClientXRequestedWith
string
X-Requested-With HTTP header
EdgeCFConnectingO2O
boolean
True if the request looped through multiple zones on the Cloudflare edge. This is considered an orange to orange (o2o) request. Available in Logpush v2 only.
EdgeColoCode
string
IATA airport code of data center that received the request
EdgeColoID
bigint
Cloudflare edge colo id
EdgeEndTimestamp
timestamp
Timestamp at which the edge finished sending response to the client
EdgePathingOp
string
Indicates what type of response was issued for this request (unknown = no specific action)
EdgePathingSrc
string
Details how the request was classified based on security checks (unknown = no specific classification)
EdgePathingStatus
string
Indicates what data was used to determine the handling of this request (unknown = no data)
EdgeRateLimitAction
string
The action taken by the blocking rule; empty if no action taken
EdgeRateLimitID
string
The internal rule ID of the rate-limiting rule that triggered a block (ban) or simulate action. 0 if no action taken
EdgeRequestHost
string
Host header on the request from the edge to the origin
EdgeResponseBodyBytes
bigint
Size of the HTTP response body returned to clients. Available in Logpush v2 only.
EdgeResponseBytes
bigint
Number of bytes returned by the edge to the client
EdgeResponseCompressionRatio
float
Edge response compression ratio
EdgeResponseContentType
string
Edge response Content-Type header value
EdgeResponseStatus
smallint
HTTP status code returned by Cloudflare to the client
EdgeServerIP
string
IP of the edge server making a request to the origin
EdgeStartTimestamp
timestamp
Timestamp at which the edge received request from the client
EdgeTimeToFirstByteMs
bigint
Total view of Time To First Byte as measured at Cloudflare’s edge. Starts after a TCP connection is established and ends when Cloudflare begins returning the first byte of a response to eyeballs. Includes TLS handshake time (for new connections) and origin response time. Available in Logpush v2 only.
FirewallMatchesActions
[string]
Array of actions the Cloudflare firewall products performed on this request. The individual firewall products associated with this action be found in FirewallMatchesSources and their respective RuleIds can be found in FirewallMatchesRuleIDs. The length of the array is the same as FirewallMatchesRuleIDs and FirewallMatchesSources. Possible actions are allow | log | simulate | drop | challenge | jschallenge | connectionClose | challengeSolved | challengeFailed | challengeBypassed | jschallengeSolved | jschallengeFailed | jschallengeBypassed | bypass
FirewallMatchesRuleIDs
[string]
Array of RuleIDs of the firewall product that has matched the request. The firewall product associated with the RuleID can be found in FirewallMatchesSources. The length of the array is the same as FirewallMatchesActions and FirewallMatchesSources.
FirewallMatchesSources
[string]
The firewall products that matched the request. The same product can appear multiple times, which indicates different rules or actions that were activated. The RuleIDs can be found in FirewallMatchesRuleIDs, the actions can be found in FirewallMatchesActions. The length of the array is the same as FirewallMatchesRuleIDs and FirewallMatchesActions. Possible sources are asn | country | ip | ipRange | securityLevel | zoneLockdown | waf | firewallRules | uaBlock | rateLimit |bic | hot | l7ddos | sanitycheck | protect
JA3Hash
string
The MD5 hash of the JA3 fingerprint used to profile SSL/TLS clients. Available in Logpush v2 only.
OriginDNSResponseTimeMs
bigint
Time taken to receive a DNS response for an origin name. Usually takes a few milliseconds, but may be longer if a CNAME record is used. Available in Logpush v2 only.
OriginIP
string
IP of the origin server
OriginRequestHeaderSendDurationMs
bigint
Time taken to send request headers to origin after establishing a connection. Note that this value is usually 0. Available in Logpush v2 only.
OriginResponseBytes
bigint
Number of bytes returned by the origin server
OriginResponseDurationMs
bigint
Upstream response time, measured from the first datacenter that receives a request. Includes time taken by Argo Smart Routing and Tiered Cache, plus time to connect and receive a response from origin servers. This field replaces OriginResponseTime. Available in Logpush v2 only.
OriginResponseHTTPExpires
timestamp
Value of the origin 'expires' header in RFC1123 format
OriginResponseHTTPLastModified
timestamp
Value of the origin 'last-modified' header in RFC1123 format
OriginResponseStatus
smallint
Status returned by the origin server
OriginResponseTime
bigint
Number of nanoseconds it took the origin to return the response to edge
OriginSSLProtocol
string
SSL (TLS) protocol used to connect to the origin
ParentRayID
string
Ray ID of the parent request if this request was made using a Worker script
RayID
string
ID of the request
RequestHeaders
json
String key-value pairs for RequestHeaders
ResponseHeaders
json
String key-value pairs for ResponseHeaders
SecurityLevel
string
The security level configured at the time of this request. This is used to determine the sensitivity of the IP Reputation system
SmartRouteColoID
bigint
The Cloudflare datacenter used to connect to the origin server if Argo Smart Routing is used. Available in Logpush v2 only.
UpperTierColoID
bigint
The “upper tier” datacenter that was checked for a cached copy if Tiered Cache is used. Available in Logpush v2 only.
WAFAction
string
Action taken by the WAF, if triggered
WAFFlags
string
Additional configuration flags: simulate (0x1) | null
WAFMatchedVar
string
The full name of the most-recently matched variable
WAFProfile
string
low | med | high
WAFRuleID
string
ID of the applied WAF rule
WAFRuleMessage
string
Rule message associated with the triggered rule
WorkerCPUTime
bigint
Amount of time in microseconds spent executing a worker, if any
WorkerStatus
string
Status returned from worker daemon
WorkerSubrequest
boolean
Whether or not this request was a worker subrequest
WorkerSubrequestCount
bigint
Number of subrequests issued by a worker when handling this request
ZoneID
bigint
Internal zone ID
ZoneName
string
The human-readable name of the zone (e.g. ‘cloudflare.com’). Available in Logpush v2 only.
p_event_time
timestamp
Panther added standardized event time (UTC)
p_parse_time
timestamp
Panther added standardized log parse time (UTC)
p_log_type
string
Panther added field with type of log
p_row_id
string
Panther added field with unique id (within table)
p_source_id
string
Panther added field with the source id
p_source_label
string
Panther added field with the source label
p_any_ip_addresses
[string]
Panther added field with collection of ip addresses associated with the row
p_any_domain_names
[string]
Panther added field with collection of domain names associated with the row
p_any_trace_ids
[string]
Panther added field with collection of context trace identifiers

Cloudflare.Spectrum

When selecting event fields on the Cloudflare UI, make sure you include the "Timestamp" field, as it is required by Panther.
Column
Type
Description
Application
string
The unique public ID of the application on which the event occurred
ClientASN
bigint
Client AS number
ClientBytes
bigint
The number of bytes read from the client by the Spectrum service
ClientCountry
string
Country of the client IP address
ClientIP
string
IP address of the client
ClientMatchedIpFirewall
string
Whether the connection matched any IP Firewall rules; UNKNOWN | ALLOW | BLOCK_ERROR | BLOCK_IP | BLOCK_COUNTRY | BLOCK_ASN | WHITELIST_IP |WHITELIST_COUNTRY | WHITELIST_ASN
ClientPort
int
Client port
ClientProto
string
Transport protocol used by client; tcp | udp | unix
ClientTcpRtt
bigint
The TCP round-trip time in nanoseconds between the client and Spectrum
ClientTlsCipher
string
The cipher negotiated between the client and Spectrum
ClientTlsClientHelloServerName
string
The server name in the Client Hello message from client to Spectrum
ClientTlsProtocol
string
The TLS version negotiated between the client and Spectrum; unknown | none | SSLv3 | TLSv1 | TLSv1.1 | TLSv1.2 | TLSv1.3
ClientTlsStatus
string
Indicates state of TLS session from the client to Spectrum; UNKNOWN | OK | INTERNAL_ERROR | INVALID_CONFIG | INVALID_SNI | HANDSHAKE_FAILED | KEYLESS_RPC
ColoCode
string
IATA airport code of data center that received the request
ConnectTimestamp
timestamp
Timestamp at which both legs of the connection (client/edge, edge/origin or nexthop) were established
DisconnectTimestamp
timestamp
Timestamp at which the connection was closed
Event
string
connect | disconnect | clientFiltered | tlsError | resolveOrigin | originError
IpFirewall
boolean
Whether IP Firewall was enabled at time of connection
OriginBytes
bigint
The number of bytes read from the origin by Spectrum
OriginIP
string
Origin IP address
OriginPort
int
Origin port
OriginProto
string
Transport protocol used by origin; tcp | udp | unix
OriginTcpRtt
bigint
The TCP round-trip time in nanoseconds between Spectrum and the origin
OriginTlsCipher
string
The cipher negotiated between Spectrum and the origin
OriginTlsFingerprint
string
SHA256 hash of origin certificate
OriginTlsMode
string
If and how the upstream connection is encrypted; unknown | off | flexible | full | strict
OriginTlsProtocol
string
The TLS version negotiated between Spectrum and the origin; unknown | none | SSLv3 | TLSv1 | TLSv1.1 | TLSv1.2 | TLSv1.3
OriginTlsStatus
string
The state of the TLS session from Spectrum to the origin; UNKNOWN | OK | INTERNAL_ERROR | INVALID_CONFIG | INVALID_SNI | HANDSHAKE_FAILED | KEYLESS_RPC
ProxyProtocol
string
Which form of proxy protocol is applied to the given connection; off | v1 | v2 | simple
Status
bigint
A code indicating reason for connection closure
Timestamp
timestamp
Timestamp at which the event took place
p_event_time
timestamp
Panther added standardized event time (UTC)
p_parse_time
timestamp
Panther added standardized log parse time (UTC)
p_log_type
string
Panther added field with type of log
p_row_id
string
Panther added field with unique id (within table)
p_source_id
string
Panther added field with the source id
p_source_label
string
Panther added field with the source label
p_any_ip_addresses
[string]
Panther added field with collection of ip addresses associated with the row