Fastmatch Log Parser
Overview
The fastmatch
parser uses simple string patterns that specify the position of fields within a log line. As the name suggests, it is very fast and should be considered the preferred method to parse text logs. It can handle most structured text log cases where the order of fields is known. In fact, it is so fast you can specify multiple patterns that will be tested in order, so you can 'solve' cases where there are a few variations in the structure of the log line.
Example using fastmatch
We will be using the following example log line that is using Apache Common Log format:
Here is how we would define a log schema for this log type using fastmatch
:
In the Panther Console, we would follow the How to create a custom schema manually instructions, selecting the FastMatch parser.
In the Fields & Indicators section (below the Parser section shown in the screenshot above), we would define the fields:
Understanding fastmatch patterns
The patterns use %{field_name}
placeholders to set where in the log line a field is expected. For example, to match this text:
We can use this pattern (surrounded by single quotes for clarity):
If you are defining a schema in the Panther Console, you will input your patterns into the Match Patterns field:
Delimiters
The text between two consecutive fields defines the 'delimiter' between them. Delimiters cannot be empty.
In the pattern in the example above, we cannot omit the "@"
between service
and ip
.
The field preceding a delimiter cannot contain the delimiter text. In the example above:
timestamp
cannot contain space" "
service
cannot contain"@"
ip
cannot contain"] ["
log_level
cannot contain"] "
Anonymous fields
Field placeholders without names (%{}
) are ignored.
Tail capture
If the last field in a pattern does not have any delimiter text after it, it will capture everything until the end of the text. In the example above message
will capture "Something when wrong"
Handling quotes
In some cases fields can be quoted within the text:
To properly unescape such fields just surround the field placeholder with quotes:
This works for both single and double quotes.
Last updated