Skip to content

parser_syslog: the regexp engine for RFC 5424 can match the header later in the message and corrupt the record #5630

Open

Description

@Watson1978

Describe the bug

With message_format rfc5424 and the default parser_engine regexp, the parser matches the header fields after the timestamp with re.match(text, idx). Regexp#match with a position does not anchor the match at that position. When the text at idx does not match, the regexp can match later in the text. The record then gets values from the wrong place, and the text before the match is dropped. No warning is logged.

https://github.com/fluent/fluentd/blob/e7d3c3e4/lib/fluent/plugin/parser_syslog.rb#L269

To Reproduce

Run the script below with bundle exec ruby -Ilib repro.rb in this repository.

Results on master (e7d3c3e):

  • <14>1 2026-04-25T16:43:29Z hôst app 123 ID - msg: the regexp engine returns "host" => "st". The string engine returns "host" => "hôst".
  • <14>1 2026-04-25T16:43:29Z first line\nhost app 123 ID - second line: the regexp engine returns "host" => "host" and "message" => "second line". first line is dropped.
require 'fluent/test'
require 'fluent/test/driver/parser'
require 'fluent/plugin/parser_syslog'

Fluent::Test.setup

%w[regexp string].each do |engine|
  d = Fluent::Test::Driver::Parser.new(Fluent::Plugin::SyslogParser).configure(%[
    message_format rfc5424
    with_priority true
    parser_engine #{engine}
  ])
  [
    '<14>1 2026-04-25T16:43:29Z hôst app 123 ID - msg',
    "<14>1 2026-04-25T16:43:29Z first line\nhost app 123 ID - second line",
  ].each do |text|
    d.instance.parse(text) { |_time, record| puts "#{engine}: #{record.inspect}" }
  end
end

Expected behavior

The regexp engine does not skip text. If the header fields do not match at that position, the message is reported as not matched. Anchoring the match at idx (for example with \G) would do this.

Your Environment

- Fluentd version: master (e7d3c3e4). v1.20.1 and v1.19.4 have the same code (not run).
- Package version: -
- Operating system: Linux
- Kernel version: -

Your Configuration

<parse>
  @type syslog
  message_format rfc5424
  with_priority true
</parse>

Your Error Log

No error or warning is logged.

Additional context

parse_rfc3164_regex also uses the same parse_plain. I did not check whether RFC 3164 messages are affected.

Activity

  1. Watson1978 commented on Nov 11, 2026

    @Watson1978
    ContributorAuthor

    I also reproduced this on v1.19.4 with the same script. The results are the same as on master. The current re.match(text, idx) form has been there since v1.10.0 (09f5d11).

  2. rafayqayyum commented on Nov 15, 2026

    @rafayqayyum

    Opened #5641 for this, anchoring the regexp with \G as you suggested; both inputs from the issue are now reported as not matched.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions