Cloudflare Logs

Connecting Cloudfare logs to your Panther Console

Overview

Panther supports ingesting Cloudflare logs via Cloudflare's Logpush service, which streams logs directly to an HTTP Source, or to Amazon Web Services (AWS) S3.

Note that Cloudflare's Logpush is available to Cloudflare Enterprise customers only. While some Cloudflare log types on this page (e.g., Audit logs) may be pulled without Logpush, Panther's supported schemas rely on the data structure when delivered by Logpush.

How to onboard Cloudflare logs to Panther

You can ingest Cloudflare logs into Panther by streaming them to either an HTTP source or a S3 source.

HTTP Source

Step 1: Create an HTTP Source in Panther

1. In the left-hand navigation bar of your Panther Console, click Configure > Log Sources.

2. Click Create New.

3. Search for "Cloudflare," then click its tile.

   • In the slide-out panel, the Transport Mechanism dropdown in the upper right corner will be pre-populated with the HTTP option.

4. Click Start Setup.

5. Follow Panther's instructions for configuring an HTTP Source.

   • You will be required to use shared secret authentication. This is the only method of authentication Cloudflare supports.

   • The Header Name associated with your Secret Key Value will be locked with a value of x-panther-cloudflare.

Step 2: Configure a Logpush job in Cloudflare

1. Locate your Cloudflare account ID. by navigating to your Cloudflare dashboard and copying the ID from the URL.

2. Create a Cloudflare API token by following Cloudflare's Create an API token documentation.

   • Ensure the token has the All accounts - Logs: Edit permission.

   • Save the API token for the following step.

3. Create a Logpush job in Cloudflare by invoking the API, as is shown in the curl example below.

   curl -X POST "https://api.cloudflare.com/client/v4/accounts/{ACCOUNT_ID}/logpush/jobs" \
       -H "Authorization: Bearer {YOUR_API_TOKEN}" \
       -H "Content-Type:application/json" \
       -d '{
        "enabled": true,
        "name": "my_cloudflare_audit_logs",
        "dataset": "audit_logs",
        "destination_conf": "{LOG_SOURCE_URL}?header_x-panther-cloudflare={SHARED_SECRET}"
        }'

   • See additional information on setting up a Logpush job on Cloudflare's Enable HTTP destination documentation.

4. Navigate back to your Cloudflare dashboard to finish the Logpush job configuration.

   1. In the left-hand navigation bar, under Analytics & Logs, click Logs.

   2. In the Logpush job table, find the row for the Logpush job you created in the previous step, and click Edit.

   3. Select the fields you would like Cloudflare to include in the audit log events sent to your HTTP Source. (By default, Cloudflare only includes a subset of all available fields.)

   4. Click Save changes.

S3 Source

Prerequisites

• Create a new S3 bucket in your AWS account.

  • We recommend creating a new S3 bucket specifically for Cloudflare logs. You can use the default settings.

  • Please note the region that you are creating it in, as you will need to provide it to Cloudflare.

Step 1: Set up the S3 bucket source in Panther

• Follow Panther’s documentation for configuring AWS S3 as a Data Transport.

Step 2: Configure Logpush to stream logs to S3

When choosing the dataset type for your Logpush job, note that Cloudflare has options for Account-scoped data and Zone-scoped data. Audit logs are Account-scoped, whereas Firewall, HttpRequest, and Spectrum are Zone-scoped. The screen shots in this guide demonstrate how to configure Audit logs.

1. Navigate to your Cloudflare dashboard, then go to Analytics > Logs.

2. Click Add Logpush job next to the dataset type you want to set up.

   
3. Click Select on the dataset you want to choose.

   
4. Scroll down and click Next.

5. Select the fields that you want to include. We recommend that you click the checkbox to select "All fields."

   • ID, ResourceType, and When are included by default and are required by Panther. Do not unselect these fields.

     
6. Click Next.

7. Under "Select a destination," locate the Amazon S3 tile and click Select.
8. Scroll down and click Next.

9. Configure the destination fields:

   • Bucket path: Enter the name of the S3 bucket you created earlier in this documentation.

   • Daily subfolders: Choose Yes if you want to organize logs into daily subfolders.

   • Bucket region: Select the region that you created your S3 bucket in.

     • To verify the region, navigate to your S3 bucket's Properties.

   • Encryption constraint in bucket policy: Set to "Yes" if you enabled SSE encryption on your S3 bucket.

   • In the Grant Cloudflare access to upload files to your bucket section, copy the IAM policy that Cloudflare is providing.

     • You will need this in the next steps to give Cloudflare access to put objects in your bucket.
10. In a separate browser tab, navigate to your AWS account and go to the S3 bucket settings.

11. At the top of the page, click the Permissions tab.

12. Scroll down to the Bucket policy section. On the right side of the tile, click Edit. In the text editor, paste the bucket policy you copied from the earlier steps.

    
13. Click Save.

14. Navigate back to the Cloudflare dashboard, and click Validate Access.

    • From here, Cloudflare will write an object to your S3 bucket which will contain an ownership challenge that you must provide back to Cloudflare in the next steps.

      
15. Navigate back to your S3 bucket in AWS.

16. Find and download the ownership challenge file, then open it.

17. Copy the contents of the ownership challenge file. You will need this in the next steps.

18. Navigate back to your Cloudflare dashboard. In the Ownership token field, paste in the contents of the ownership challenge file.

19. Click Save.

Additionally, see Cloudflare's documentation for instructions on pushing logs to Amazon S3.

Panther-built Detections

See Panther's built in rules for Cloudflare in panther-analysis in Github.

Supported log types

Required fields in all tables are in bold.

Cloudflare.Audit

When selecting event fields on the Cloudflare UI, make sure you include the When, ID, and ResourceType fields, as they are required by Panther.

# Code generated by Panther; DO NOT EDIT. (@generated)
schema: Cloudflare.Audit
parser:
  native:
    name: Cloudflare.Audit
description: Audit logs summarize the history of changes made within your Cloudflare account. Audit logs include account level actions like login and logout, as well as zone configuration changes.
referenceURL: https://developers.cloudflare.com/logs/reference/log-fields/account/audit_logs
fields:
  - name: ActionResult
    description: Whether the action was successful
    type: boolean
  - name: ActionType
    description: Type of action taken
    type: string
  - name: ActorEmail
    description: Email of the actor
    type: string
    indicators:
      - email
  - name: ActorID
    description: Unique identifier of the actor in Cloudflare's system
    type: string
    indicators:
      - username
  - name: ActorIP
    description: Physical network address of the actor
    type: string
    indicators:
      - ip
  - name: ActorType
    description: Type of user that started the audit trail
    type: string
  - name: ID
    required: true
    description: Unique identifier of an audit log
    type: string
  - name: Interface
    description: Entry point or interface of the audit log
    type: string
  - name: Metadata
    description: Additional audit log-specific information. Metadata is organized in key:value pairs. Key and Value formats can vary by ResourceType.
    type: json
  - name: NewValue
    description: Contains the new value for the audited item
    type: json
  - name: OldValue
    description: Contains the old value for the audited item
    type: json
  - name: OwnerID
    description: The identifier of the user that was acting or was acted on behalf of. If a user did the action themselves, this value will be the same as the ActorID.
    type: string
    indicators:
      - username
  - name: ResourceID
    description: Unique identifier of the resource within Cloudflares system
    type: string
  - name: ResourceType
    required: true
    description: The type of resource that was changed
    type: string
  - name: When
    required: true
    description: When the change happened
    type: timestamp
    timeFormats:
      - cloudflare
    isEventTime: true

Cloudflare.Firewall

When selecting event fields on the Cloudflare UI, make sure you include the "Datetime" field, as it is required by Panther.

Reference: Cloudfare Documentation on Log Field Firewalls.

Column	Type	Description
Action	string	The code of the first-class action the Cloudflare Firewall took on this request
ClientASN	bigint	The ASN number of the visitor
ClientASNDescription	string	The ASN of the visitor as string
ClientCountry	string	Country from which request originated
ClientIP	string	The visitor's IP address (IPv4 or IPv6)
ClientIPClass	string	The classification of the visitor's IP address, possible values are: unknown | clean | badHost | searchEngine | whitelist | greylist | monitoringService |securityScanner | noRecord | scan | backupService | mobilePlatform | tor
ClientRefererHost	string	The referer host
ClientRefererPath	string	The referer path requested by visitor
ClientRefererQuery	string	The referer query-string was requested by the visitor
ClientRefererScheme	string	The referer url scheme requested by the visitor
ClientRequestHost	string	The HTTP hostname requested by the visitor
ClientRequestMethod	string	The HTTP method used by the visitor
ClientRequestPath	string	The path requested by visitor
ClientRequestProtocol	string	The version of HTTP protocol requested by the visitor
ClientRequestQuery	string	The query-string was requested by the visitor
ClientRequestScheme	string	The url scheme requested by the visitor
ClientRequestUserAgent	string	Visitor's user-agent string
Description	string	Rule description for this event
Datetime	timestamp	The date and time the event occurred at the edge
EdgeColoCode	string	The airport code of the Cloudflare datacenter that served this request
EdgeResponseStatus	smallint	HTTP response status code returned to browser
Kind	string	The kind of event, currently only possible values are: firewall
MatchIndex	bigint	Rules match index in the chain
Metadata	{ string:string }	Additional product-specific information. Metadata is organized in key:value pairs. Key and Value formats can vary by Cloudflare security product and can change over time
OriginResponseStatus	smallint	HTTP origin response status code returned to browser
OriginatorRayID	string	The RayID of the request that issued the challenge/jschallenge
RayID	string	The RayID of the request
Ref	string	User-defined rule reference for this event
RuleID	string	The Cloudflare security product-specific RuleID triggered by this request
Source	string	The Cloudflare security product triggered by this request
p_event_time	timestamp	Panther added standardized event time (UTC)
p_parse_time	timestamp	Panther added standardized log parse time (UTC)
p_log_type	string	Panther added field with type of log
p_row_id	string	Panther added field with unique id (within table)
p_source_id	string	Panther added field with the source id
p_source_label	string	Panther added field with the source label
p_any_ip_addresses	[string]	Panther added field with collection of ip addresses associated with the row
p_any_domain_names	[string]	Panther added field with collection of domain names associated with the row
p_any_trace_ids	[string]	Panther added field with collection of context trace identifiers

Cloudflare.HttpRequest

When selecting event fields on the Cloudflare UI, make sure you include the "EdgeStartTimestamp" field, as it is required by Panther.

Reference: Cloudfare Documentation on Log Field Requests.

Column	Type	Description
BotDetectionIDs	[bigint]	List of IDs that correlate to the Bot Management Heuristic detections made on a request. Available in Logpush v2 only.
BotScore	bigint	Cloudflare Bot Score (available for Bot Management customers; please contact your account team to enable)
BotScoreSrc	string	Underlying detection engine or source on where a Bot Score is calculated. Possible values are Not Computed | Heuristics | Machine Learning | Behavioral Analysis | Verified Bot
BotTags	[string]	Type of bot traffic (if available). Refer to Bot Tags for the list of potential values. Available in Logpush v2 only.
CacheCacheStatus	string	unknown | miss | expired | updating | stale | hit | ignored | bypass | revalidated
CacheReserveUsed	boolean	Cache Reserve was used to serve this request. Available in Logpush v2 only.
CacheResponseBytes	bigint	Number of bytes returned by the cache
CacheResponseStatus	smallint	HTTP status code returned by the cache to the edge; all requests (including non-cacheable ones) go through the cache; also see CacheStatus field
CacheTieredFill	boolean	Tiered Cache was used to serve this request
ClientASN	bigint	Client AS number
ClientCountry	string	Country of the client IP address
ClientDeviceType	string	Client device type
ClientIP	string	IP address of the client
ClientIPClass	string	unknown | clean | badHost | searchEngine | whitelist | greylist | monitoringService | securityScanner | noRecord | scan |backupService | mobilePlatform | tor
ClientMTLSAuthCertFingerprint	string	The SHA256 fingerprint of the certificate presented by the client during mTLS authentication. Only populated on the first request on an mTLS connection. Available in Logpush v2 only.
ClientMTLSAuthStatus	string	The status of mTLS authentication. Only populated on the first request on an mTLS connection. Available in Logpush v2 only. Possible values are unknown | ok | absent | untrusted | notyetvalid | expired
ClientRegionCode	string	The ISO-3166-2 region code of the client IP address.
ClientRequestBytes	bigint	Number of bytes in the client request
ClientRequestHost	string	Host requested by the client
ClientRequestMethod	string	HTTP method of client request
ClientRequestPath	string	URI path requested by the client
ClientRequestProtocol	string	HTTP protocol of client request
ClientRequestReferer	string	HTTP request referrer
ClientRequestScheme	string	The URL scheme requested by the visitor. Available in Logpush v2 only.
ClientRequestSource	string	Identifies requests as coming from an external source or another service within Cloudflare. Refer to ClientRequestSource field for the list of potential values. Available in Logpush v2 only.
ClientRequestURI	string	URI requested by the client
ClientRequestUserAgent	string	User agent reported by the client
ClientSSLCipher	string	Client SSL cipher
ClientSSLProtocol	string	Client SSL (TLS) protocol
ClientSrcPort	int	Client source port
ClientTCPRTTMs	bigint	The smoothed average of TCP round-trip time (SRTT). For the initial request on a connection, this is measured only during connection setup. For a subsequent request on the same connection, it is measured over the entire connection lifetime up until the time that request is received. Available in Logpush v2 only.
ClientXRequestedWith	string	X-Requested-With HTTP header
ContentScanObjResults	[string]	List of content scan results.
ContentScanObjTypes	[string]	List of content types.
Cookies	json	String key-value pairs for Cookies.
EdgeCFConnectingO2O	boolean	True if the request looped through multiple zones on the Cloudflare edge. This is considered an orange to orange (o2o) request. Available in Logpush v2 only.
EdgeColoCode	string	IATA airport code of data center that received the request
EdgeColoID	bigint	Cloudflare edge colo id
EdgeEndTimestamp	timestamp	Timestamp at which the edge finished sending response to the client
EdgePathingOp	string	Indicates what type of response was issued for this request (unknown = no specific action)
EdgePathingSrc	string	Details how the request was classified based on security checks (unknown = no specific classification)
EdgePathingStatus	string	Indicates what data was used to determine the handling of this request (unknown = no data)
EdgeRateLimitAction	string	The action taken by the blocking rule; empty if no action taken
EdgeRateLimitID	string	The internal rule ID of the rate-limiting rule that triggered a block (ban) or simulate action. 0 if no action taken
EdgeRequestHost	string	Host header on the request from the edge to the origin
EdgeResponseBodyBytes	bigint	Size of the HTTP response body returned to clients. Available in Logpush v2 only.
EdgeResponseBytes	bigint	Number of bytes returned by the edge to the client
EdgeResponseCompressionRatio	float	Edge response compression ratio
EdgeResponseContentType	string	Edge response Content-Type header value
EdgeResponseStatus	smallint	HTTP status code returned by Cloudflare to the client
EdgeServerIP	string	IP of the edge server making a request to the origin
EdgeStartTimestamp	timestamp	Timestamp at which the edge received request from the client
EdgeTimeToFirstByteMs	bigint	Total view of Time To First Byte as measured at Cloudflare’s edge. Starts after a TCP connection is established and ends when Cloudflare begins returning the first byte of a response to eyeballs. Includes TLS handshake time (for new connections) and origin response time. Available in Logpush v2 only.
FirewallMatchesActions	[string]	Array of actions the Cloudflare firewall products performed on this request. The individual firewall products associated with this action be found in FirewallMatchesSources and their respective RuleIds can be found in FirewallMatchesRuleIDs. The length of the array is the same as FirewallMatchesRuleIDs and FirewallMatchesSources. Possible actions are allow | log | simulate | drop | challenge | jschallenge | connectionClose | challengeSolved | challengeFailed | challengeBypassed | jschallengeSolved | jschallengeFailed | jschallengeBypassed | bypass
FirewallMatchesRuleIDs	[string]	Array of RuleIDs of the firewall product that has matched the request. The firewall product associated with the RuleID can be found in FirewallMatchesSources. The length of the array is the same as FirewallMatchesActions and FirewallMatchesSources.
FirewallMatchesSources	[string]	The firewall products that matched the request. The same product can appear multiple times, which indicates different rules or actions that were activated. The RuleIDs can be found in FirewallMatchesRuleIDs, the actions can be found in FirewallMatchesActions. The length of the array is the same as FirewallMatchesRuleIDs and FirewallMatchesActions. Possible sources are asn | country | ip | ipRange | securityLevel | zoneLockdown | waf | firewallRules | uaBlock | rateLimit |bic | hot | l7ddos | sanitycheck | protect
JA3Hash	string	The MD5 hash of the JA3 fingerprint used to profile SSL/TLS clients. Available in Logpush v2 only.
OriginDNSResponseTimeMs	bigint	Time taken to receive a DNS response for an origin name. Usually takes a few milliseconds, but may be longer if a CNAME record is used. Available in Logpush v2 only.
OriginIP	string	IP of the origin server
OriginRequestHeaderSendDurationMs	bigint	Time taken to send request headers to origin after establishing a connection. Note that this value is usually 0. Available in Logpush v2 only.
OriginResponseBytes	bigint	Number of bytes returned by the origin server
OriginResponseDurationMs	bigint	Upstream response time, measured from the first datacenter that receives a request. Includes time taken by Argo Smart Routing and Tiered Cache, plus time to connect and receive a response from origin servers. This field replaces OriginResponseTime. Available in Logpush v2 only.
OriginResponseHeaderReceiveDurationMs	bigint	Time taken for origin to return response headers after Cloudflare finishes sending request headers. Available in Logpush v2 only.
OriginResponseHTTPExpires	timestamp	Value of the origin 'expires' header in RFC1123 format
OriginResponseHTTPLastModified	timestamp	Value of the origin 'last-modified' header in RFC1123 format
OriginResponseStatus	smallint	Status returned by the origin server
OriginResponseTime	bigint	Number of nanoseconds it took the origin to return the response to edge
OriginSSLProtocol	string	SSL (TLS) protocol used to connect to the origin
OriginTCPHandshakeDurationMs	bigint	Time taken to complete TCP handshake with origin. This will be 0 if an origin connection is reused. Available in Logpush v2 only.
OriginTLSHandshakeDurationMs	bigint	Time taken to complete TLS handshake with origin. This will be 0 if an origin connection is reused. Available in Logpush v2 only.
ParentRayID	string	Ray ID of the parent request if this request was made using a Worker script
RayID	string	ID of the request
RequestHeaders	json	String key-value pairs for RequestHeaders
ResponseHeaders	json	String key-value pairs for ResponseHeaders
SecurityAction	string	Rule action of the security rule that triggered a terminating action, if any
SecurityActions	[string]	Array of actions that Cloudflare security products performed on this request.
SecurityLevel	string	The security level configured at the time of this request. This is used to determine the sensitivity of the IP Reputation system
SecurityRuleDescription	string	Rule description of the security rule that triggered a terminating action, if any
SecurityRuleID	string	Rule ID of the security rule that triggered a terminating action, if any
SecurityRuleIDs	[string]	Array of security rule IDs that matched the request
SmartRouteColoID	bigint	The Cloudflare datacenter used to connect to the origin server if Argo Smart Routing is used. Available in Logpush v2 only.
SecuritySources	[string]	Array of Cloudflare security products that matched the request.
UpperTierColoID	bigint	The “upper tier” datacenter that was checked for a cached copy if Tiered Cache is used. Available in Logpush v2 only.
WAFAction	string	Action taken by the WAF, if triggered
WAFAttackScore	bigint	Overall request score generated by the WAF detection module.
WAFFlags	string	Additional configuration flags: simulate (0x1) | null
WAFMatchedVar	string	The full name of the most-recently matched variable
WAFProfile	string	low | med | high
WAFRCEAttackScore	bigint	WAF score for an RCE attack.
WAFRuleID	string	ID of the applied WAF rule
WAFRuleMessage	string	Rule message associated with the triggered rule
WAFSQLiAttackScore	bigint	WAF score for an SQLi attack.
WAFXSSAttackScore	bigint	WAF score for an XSS attack.
WorkerCPUTime	bigint	Amount of time in microseconds spent executing a worker, if any
WorkerStatus	string	Status returned from worker daemon
WorkerSubrequest	boolean	Whether or not this request was a worker subrequest
WorkerSubrequestCount	bigint	Number of subrequests issued by a worker when handling this request
WorkerWallTimeUs	bigint	Real-time in microseconds elapsed between start and end of worker invocation.
ZoneID	bigint	Internal zone ID
ZoneName	string	The human-readable name of the zone (e.g. ‘cloudflare.com’). Available in Logpush v2 only.
p_event_time	timestamp	Panther added standardized event time (UTC)
p_parse_time	timestamp	Panther added standardized log parse time (UTC)
p_log_type	string	Panther added field with type of log
p_row_id	string	Panther added field with unique id (within table)
p_source_id	string	Panther added field with the source id
p_source_label	string	Panther added field with the source label
p_any_ip_addresses	[string]	Panther added field with collection of ip addresses associated with the row
p_any_domain_names	[string]	Panther added field with collection of domain names associated with the row
p_any_trace_ids	[string]	Panther added field with collection of context trace identifiers

Cloudflare.Spectrum

When selecting event fields on the Cloudflare UI, make sure you include the "Timestamp" field, as it is required by Panther.

Reference: Cloudfare Documentation on Log Field Spectrum Events.

Column	Type	Description
Application	string	The unique public ID of the application on which the event occurred
ClientASN	bigint	Client AS number
ClientBytes	bigint	The number of bytes read from the client by the Spectrum service
ClientCountry	string	Country of the client IP address
ClientIP	string	IP address of the client
ClientMatchedIpFirewall	string	Whether the connection matched any IP Firewall rules; UNKNOWN | ALLOW | BLOCK_ERROR | BLOCK_IP | BLOCK_COUNTRY | BLOCK_ASN | WHITELIST_IP |WHITELIST_COUNTRY | WHITELIST_ASN
ClientPort	int	Client port
ClientProto	string	Transport protocol used by client; tcp | udp | unix
ClientTcpRtt	bigint	The TCP round-trip time in nanoseconds between the client and Spectrum
ClientTlsCipher	string	The cipher negotiated between the client and Spectrum
ClientTlsClientHelloServerName	string	The server name in the Client Hello message from client to Spectrum
ClientTlsProtocol	string	The TLS version negotiated between the client and Spectrum; unknown | none | SSLv3 | TLSv1 | TLSv1.1 | TLSv1.2 | TLSv1.3
ClientTlsStatus	string	Indicates state of TLS session from the client to Spectrum; UNKNOWN | OK | INTERNAL_ERROR | INVALID_CONFIG | INVALID_SNI | HANDSHAKE_FAILED | KEYLESS_RPC
ColoCode	string	IATA airport code of data center that received the request
ConnectTimestamp	timestamp	Timestamp at which both legs of the connection (client/edge, edge/origin or nexthop) were established
DisconnectTimestamp	timestamp	Timestamp at which the connection was closed
Event	string	connect | disconnect | clientFiltered | tlsError | resolveOrigin | originError
IpFirewall	boolean	Whether IP Firewall was enabled at time of connection
OriginBytes	bigint	The number of bytes read from the origin by Spectrum
OriginIP	string	Origin IP address
OriginPort	int	Origin port
OriginProto	string	Transport protocol used by origin; tcp | udp | unix
OriginTcpRtt	bigint	The TCP round-trip time in nanoseconds between Spectrum and the origin
OriginTlsCipher	string	The cipher negotiated between Spectrum and the origin
OriginTlsFingerprint	string	SHA256 hash of origin certificate
OriginTlsMode	string	If and how the upstream connection is encrypted; unknown | off | flexible | full | strict
OriginTlsProtocol	string	The TLS version negotiated between Spectrum and the origin; unknown | none | SSLv3 | TLSv1 | TLSv1.1 | TLSv1.2 | TLSv1.3
OriginTlsStatus	string	The state of the TLS session from Spectrum to the origin; UNKNOWN | OK | INTERNAL_ERROR | INVALID_CONFIG | INVALID_SNI | HANDSHAKE_FAILED | KEYLESS_RPC
ProxyProtocol	string	Which form of proxy protocol is applied to the given connection; off | v1 | v2 | simple
Status	bigint	A code indicating reason for connection closure
Timestamp	timestamp	Timestamp at which the event took place
p_event_time	timestamp	Panther added standardized event time (UTC)
p_parse_time	timestamp	Panther added standardized log parse time (UTC)
p_log_type	string	Panther added field with type of log
p_row_id	string	Panther added field with unique id (within table)
p_source_id	string	Panther added field with the source id
p_source_label	string	Panther added field with the source label
p_any_ip_addresses	[string]	Panther added field with collection of ip addresses associated with the row