HTTP Source

Onboarding an HTTP (webhook) source for log ingestion in Panther

Overview

Using Panther's HTTP log ingestion capability, you (or a third party) can send log events directly to Panther with webhooks. When you create an HTTP log source, Panther provides a URL endpoint to which you can make POST requests containing data for ingestion.

For log sources without a direct API puller, using an HTTP source means you can ingest data into Panther without setting up additional infrastructure (as is required with other Data Transports, such as S3 or Google Cloud Storage).

HTTP sources in Panther are actually HTTPS sources, meaning the endpoints provided by Panther begin with https://. These sources are interchangeably referred to as "HTTP" or "HTTPS," but are always secured with encryption (TLS).

Payload requirements

When sending data to Panther through an HTTP endpoint, certain specifications apply.

Maximum size

The maximum supported payload size is 1 MB. This limit includes headers.

You can compress your payloads before sending them to maximize throughput.

Compression

Payloads can optionally be sent compressed. Learn more about compression specifications in Ingesting compressed data in Panther.

Stream types

Panther automatically infers the stream type of data ingested through HTTP sources. The following stream types are supported:

• JSON

• Lines

• JSON Array

See a description and example of each stream type here.

Authentication

To secure your HTTPS endpoint, ensuring it only processes requests from authenticated sources, Panther offers various modes of authentication.

Shared secret

With shared secret authentication (also known as token authentication), you configure a header name and shared secret value during source setup. The entity making the POST request to Panther must include a header with that name and secret value. Panther will only accept payloads with headers and secrets that match your configuration.

This method is referred to as shared secret because the secret is shared between Panther and the entity making the POST request, e.g., a third-party SaaS tool emitting log events.

HMAC

With HMAC authentication, you configure a header name and secret key value during source setup. The entity making the POST request to Panther must hash the payload of each request, use the secret key value to generate a signature, then include that signature in a header. Panther will then retrieve the signature from the header and perform the same signature calculation using the secret key. Only requests with matching signatures will be accepted.

Bearer

With bearer authentication, you define a bearer token value during source setup. The entity making the POST request to Panther must include a header of the format Authorization: Bearer <token_value_here>. Panther will only accept request with matching bearer tokens.

Basic

With basic authentication, you define a username and password during source setup. The entity making the POST request to Panther must include a header of the format Authorization: Basic aGVsbG8=. The string after the Basic keyword follows RFC 7617, meaning the sender must encode the value of username:password using base64. Panther only accepts requests with a matching username and password.

None

While highly discouraged, it is possible to configure an endpoint without authentication. This means Panther will not perform any authentication checks on incoming requests, effectively making your URL open to all POST requests.

It is strongly recommended to set up authentication on your endpoint. None is offered for situations in which you do not have control over the headers provided by the sender.

How to set up an HTTP log source in Panther

To set up an HTTP log source in Panther, follow the steps below. You can also view the data ingestion video overview for a quick walkthrough of HTTP source setup.

Creating the HTTP log source in Panther

1. In the left-hand navigation bar of your Panther Console, click Configure > Log Sources.

2. In the upper-right corner, click Create New.

3. Click the Custom Log Formats tile.

4. In the HTTP tile on the slide-out panel, click Start.
5. On the Configure page, in the Basic Information section, provide values for the following fields:

   • Source Name: Enter a descriptive name for the source.

   • Schemas - Optional: Select one or more log types this source will ingest.

     • This field is optional; you can attach schemas to this source after it has been created. Data sent to the HTTP source before a schema is attached can be used to infer a schema.
6. In the Select Authentication Type section, select an Auth method based on your chosen mode of authentication.
7. Follow the instructions below for the Auth method you chose:

   • Shared Secret: Enter a Header Name, and generate a Shared Secret Value.

     • Securely copy your Shared Secret Value, and store it in a safe location. This is the only time you will be shown the Shared Secret Value.

   • HMAC: Enter a Header Name, generate a Secret Key Value, and choose a Hashing Algorithm (either SHA 256 or SHA 1).

     • Securely copy your Secret Key Value, and store it in a safe location. This is the only time you will be shown the Secret Key Value.

   • Bearer: Enter a Bearer Token.

     • In the value you enter, do not include "Bearer."

     • Be sure to securely store your Bearer Token value. It will not be visible in the Panther Console.

   • Basic: Enter Username and Password values.

     • Be sure to securely store your Password value. It will not be visible in the Panther Console.

   • None: No additional configuration required.

8. Click Setup. You will be directed to a success screen:

   

   • Take note of your HTTP Source URL. Webhooks should make POST requests to this endpoint.

   • You can optionally enable one or more Detection Packs.
   • If you have not done so already, click Attach or Infer Schemas to attach one or more schemas to the source. Alternatively, you can infer a schema after data has been received.
   • The Trigger an alert when no events are processed setting defaults to YES. We recommend leaving this enabled, as you will be alerted if data stops flowing from the log source after a certain period of time. The timeframe is configurable, with a default of 24 hours.

   

   • Creation of the endpoint can take up to five minutes, and is usually completed within three.

     • To monitor the creation process, click View Log Source. You will be directed to the source's details page, where you can view Source Status:

       

Sending data to your HTTP log source

After creating your HTTP log source, you can use it to send data to Panther by making POST requests to the source URL.

Ensure your request passes the authentication method you configured for the source.

Example using curl

curl --data '{ "hello": "world" }' \
    -H "x-custom-secret: mysecret" \
    https://{YOUR_URL}

Error handling

If your HTTP endpoint returns a 4xx or 5xx status code, Panther does not then automatically retry the request. It's recommended for the sender to retry the request, as there may have been a transient issue with server availability. If the errors persist, please contact your Panther Support team.

Viewing ingested logs

After your log source is configured, you can search ingested data using Search or Data Explorer.