Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion lib/fluent/plugin/parser_syslog.rb
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ class SyslogParser < Parser
RFC3164_CAPTURES = RFC3164_WITHOUT_TIME_AND_PRI_REGEXP.names.freeze
RFC3164_PRI_REGEXP = /\A<(?<pri>[0-9]{1,3})>/

RFC5424_WITHOUT_TIME_AND_PRI_REGEXP = /(?<host>[!-~]{1,255}) (?<ident>[!-~]{1,48}) (?<pid>[!-~]{1,128}) (?<msgid>[!-~]{1,32}) (?<extradata>(?:\-|(?:\[.*?(?<!\\)\])))(?: (?<message>.+))?\z/m
RFC5424_WITHOUT_TIME_AND_PRI_REGEXP = /\G(?<host>[!-~]{1,255}) (?<ident>[!-~]{1,48}) (?<pid>[!-~]{1,128}) (?<msgid>[!-~]{1,32}) (?<extradata>(?:\-|(?:\[.*?(?<!\\)\])))(?: (?<message>.+))?\z/m

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MUST: \G also rejects messages that have more than one space or tab between TIMESTAMP and HOSTNAME. Master parses them, because the unanchored match skipped the extra whitespace. Two examples, as Ruby strings:

"<14>1 2026-04-25T16:43:29Z  host app 123 ID - msg"
"<14>1 2026-04-25T16:43:29Z \thost app 123 ID - msg"

With message_format rfc5424 (or auto), with_priority true and the default regexp engine, master parses both into "host" => "host" and "message" => "msg". With this change, the parser yields nil, nil for both. With the default emit_unmatched_lines false, in_syslog then logs failed to parse message and drops the record.

Allowing only spaces and tabs at the match position keeps these messages. Both inputs from #5630 are still rejected, and so is a header after a line break:

Suggested change
RFC5424_WITHOUT_TIME_AND_PRI_REGEXP = /\G(?<host>[!-~]{1,255}) (?<ident>[!-~]{1,48}) (?<pid>[!-~]{1,128}) (?<msgid>[!-~]{1,32}) (?<extradata>(?:\-|(?:\[.*?(?<!\\)\])))(?: (?<message>.+))?\z/m
RFC5424_WITHOUT_TIME_AND_PRI_REGEXP = /\G[ \t]*(?<host>[!-~]{1,255}) (?<ident>[!-~]{1,48}) (?<pid>[!-~]{1,128}) (?<msgid>[!-~]{1,32}) (?<extradata>(?:\-|(?:\[.*?(?<!\\)\])))(?: (?<message>.+))?\z/m

Could you also add a test that these two messages are still parsed?

RFC5424_CAPTURES = RFC5424_WITHOUT_TIME_AND_PRI_REGEXP.names.freeze
RFC5424_PRI_REGEXP = /\A<(?<pri>\d{1,3})>\d\d{0,2}\s/

Expand Down
14 changes: 14 additions & 0 deletions test/plugin/test_parser_syslog.rb
Original file line number Diff line number Diff line change
Expand Up @@ -530,6 +530,20 @@ def test_parse_does_not_accept_space_after_rfc5424_priority(data)
assert_input_rejected(text, expectation)
end

data(
'non-ASCII host' => '<14>1 2026-04-25T16:43:29Z hôst app 123 ID - msg',
'header on a later line' => "<14>1 2026-04-25T16:43:29Z first line\nhost app 123 ID - second line",
)
def test_regexp_engine_does_not_match_rfc5424_header_later_in_the_message(text)
@parser.configure(
'parser_engine' => 'regexp',
'message_format' => 'rfc5424',
'with_priority' => true,
)

assert_input_rejected(text, :rejects)
end

def test_parse_rfc5424_with_space_containing_time_format
@parser.configure(
'parser_engine' => 'string',
Expand Down