Repository navigation
parser_syslog: anchor the RFC 5424 header regexp at the match position - #5641
rafayqayyum wants to merge 1 commit into
Conversation
Signed-off-by: Rafay Qayyum <rafayqayyum786@gmail.com>
0105482 to
acc2d6d
Compare
Watson1978
left a comment
There was a problem hiding this comment.
Thanks for the fix. Both inputs from #5630 are now rejected, and the new tests fail on master and pass with this change.
I found one kind of input that master parses but this change rejects. Please see the inline comment.
| RFC3164_PRI_REGEXP = /\A<(?<pri>[0-9]{1,3})>/ | ||
|
|
||
| RFC5424_WITHOUT_TIME_AND_PRI_REGEXP = /(?<host>[!-~]{1,255}) (?<ident>[!-~]{1,48}) (?<pid>[!-~]{1,128}) (?<msgid>[!-~]{1,32}) (?<extradata>(?:\-|(?:\[.*?(?<!\\)\])))(?: (?<message>.+))?\z/m | ||
| RFC5424_WITHOUT_TIME_AND_PRI_REGEXP = /\G(?<host>[!-~]{1,255}) (?<ident>[!-~]{1,48}) (?<pid>[!-~]{1,128}) (?<msgid>[!-~]{1,32}) (?<extradata>(?:\-|(?:\[.*?(?<!\\)\])))(?: (?<message>.+))?\z/m |
There was a problem hiding this comment.
MUST: \G also rejects messages that have more than one space or tab between TIMESTAMP and HOSTNAME. Master parses them, because the unanchored match skipped the extra whitespace. Two examples, as Ruby strings:
"<14>1 2026-04-25T16:43:29Z host app 123 ID - msg"
"<14>1 2026-04-25T16:43:29Z \thost app 123 ID - msg"With message_format rfc5424 (or auto), with_priority true and the default regexp engine, master parses both into "host" => "host" and "message" => "msg". With this change, the parser yields nil, nil for both. With the default emit_unmatched_lines false, in_syslog then logs failed to parse message and drops the record.
Allowing only spaces and tabs at the match position keeps these messages. Both inputs from #5630 are still rejected, and so is a header after a line break:
| RFC5424_WITHOUT_TIME_AND_PRI_REGEXP = /\G(?<host>[!-~]{1,255}) (?<ident>[!-~]{1,48}) (?<pid>[!-~]{1,128}) (?<msgid>[!-~]{1,32}) (?<extradata>(?:\-|(?:\[.*?(?<!\\)\])))(?: (?<message>.+))?\z/m | |
| RFC5424_WITHOUT_TIME_AND_PRI_REGEXP = /\G[ \t]*(?<host>[!-~]{1,255}) (?<ident>[!-~]{1,48}) (?<pid>[!-~]{1,128}) (?<msgid>[!-~]{1,32}) (?<extradata>(?:\-|(?:\[.*?(?<!\\)\])))(?: (?<message>.+))?\z/m |
Could you also add a test that these two messages are still parsed?
Which issue(s) this PR fixes:
Fixes #5630
What this PR does / why we need it:
With
message_format rfc5424andparser_engine regexp, the header fields after the timestamp are matched withRFC5424_WITHOUT_TIME_AND_PRI_REGEXP.match(text, idx).Regexp#matchwith a position doesn't anchor the match there, so when the text atidxdoesn't match, the regexp matches later in the message:<14>1 2026-04-25T16:43:29Z hôst app 123 ID - msggave"host" => "st", andfirst line\nhost app 123 ID - second linedroppedfirst lineand parsed the second line.The regexp now starts with
\G, so the fields must start atidx; otherwise the message is reported as not matched, as the issue suggests. The constant is only matched with a position (and exposed throughpatterns), so nothing else changes. The new tests use the issue's two inputs.Docs Changes:
None.
Release Note:
parser_syslog: the regexp engine no longer matches the RFC 5424 header later in the message.