fluentd is an amazing piece of software but can sometimes give one a hard time. Case in point, how can one add a field to an output only if a certain string exists in another record.
Turns out with a little regex, it’s decently simple.
The below code will add a field called “_newfield” with value “OURSTRING” when the “log” record contains the exact same string at the beginning of the line. The regex can be further adjusted as needed.
<filter **>
@type parser
key_name log
reserve_data true
emit_invalid_record_to_error false
<parse>
@type regexp
expression /^(?<_newfield>^OURSTRING)/
</parse>
</filter>
“emit_invalid_record_to_error false” will avoid sending error logs if the regex does not match while “reserve_data true” will preserve the initial message and just add a field to it before passing it out further.