This guide is aimed to help you quickly set up the necessary AWS resources that can be used to onboard data from various utilities and sources like Fluentd, Syslog, Windows Events, GCP Audit logs, and more.
Fluentd supports Firehose and S3 destination plugins. We have provided sample CloudFormation templates below that can be customized to fit your environment.
Once the template has been deployed and the resources have been created, return to the log source guide to continue configuring the log source.
Firehose to S3 Template (Recommended)
The Fluentd Firehose plugin is generally more performant than the Fluentd S3 plugin
This template creates a Kinesis Firehose resource, an S3 bucket, configures permissions to write to the Firehose stream, the Firehose stream to send its logs to S3, and permissions for Firehose to write to the S3 bucket.
After deploying the template, save the outputs for use in the Fluentd configurations.
The outputs of this template are:
InstanceProfileName - The profile that can be used to assume the role with correct permissions
S3Bucket - The S3bucket that firehose will send events to
FirehoseSendDataRoleArn - Arn of the role to write to Firehose