Skip to content

parser_syslog: message_format auto parses RFC 3164 messages with an RFC 3339 timestamp as RFC 5424 #5631

Open

Description

@Watson1978

Describe the bug

With message_format auto, the parser decides the format with REGEXP_DETECT_RFC5424:

REGEXP_DETECT_RFC5424 = /\A\<[0-9]{1,3}\>[1-9]\d{0,2}/

https://github.com/fluent/fluentd/blob/e7d3c3e4/lib/fluent/plugin/parser_syslog.rb#L41

It only checks that a digit follows PRI. It does not check the space after VERSION. So an RFC 3164 message whose timestamp starts with the year is detected as RFC 5424. rsyslog's built-in RSYSLOG_ForwardFormat template sends such messages (<%PRI%>%TIMESTAMP:::date-rfc3339% %HOSTNAME% ...). The RFC 5424 parsing then fails, and the parser returns no record.

message_format rfc3164 with time_format parses the same message.

To Reproduce

Run the script below with bundle exec ruby -Ilib repro.rb in this repository.

Results on master (e7d3c3e) for <13>2026-04-25T16:43:29+00:00 web01 nginx: hello:

auto: nil nil
rfc3164: 2026-04-26 01:43:29.000000000 +0900 {"pri" => 13, "host" => "web01", "ident" => "nginx", "message" => "hello"}
require 'fluent/test'
require 'fluent/test/driver/parser'
require 'fluent/plugin/parser_syslog'

Fluent::Test.setup

text = '<13>2026-04-25T16:43:29+00:00 web01 nginx: hello'
%w[auto rfc3164].each do |format|
  d = Fluent::Test::Driver::Parser.new(Fluent::Plugin::SyslogParser).configure(%[
    message_format #{format}
    with_priority true
    time_format %Y-%m-%dT%H:%M:%S%z
  ])
  d.instance.parse(text) { |time, record| puts "#{format}: #{time.inspect} #{record.inspect}" }
end

Expected behavior

message_format auto detects this message as RFC 3164 and parses it, as message_format rfc3164 does. For example, the detection could require a space after VERSION.

Your Environment

- Fluentd version: master (e7d3c3e4). v1.20.1 and v1.19.4 have the same code (not run).
- Package version: -
- Operating system: Linux
- Kernel version: -

Your Configuration

<parse>
  @type syslog
  message_format auto
  with_priority true
  time_format %Y-%m-%dT%H:%M:%S%z
</parse>

Your Error Log

No error is logged by the parser. It returns no record. (I called the parser directly. in_syslog logs "failed to parse message" for such a result.)

Additional context

The RSYSLOG_ForwardFormat definition is from https://docs.rsyslog.com/doc/reference/templates/templates-reserved-names.html

Activity

  1. Watson1978 commented on Nov 11, 2026

    @Watson1978
    ContributorAuthor

    I also reproduced this on v1.19.4 with the same script. The results are the same as on master. REGEXP_DETECT_RFC5424 has had this form since v0.14.14 (b0e9b73).

  2. rafayqayyum commented on Nov 15, 2026

    @rafayqayyum

    Opened #5642 for this: the RFC 5424 detection now requires the space after VERSION, so message_format auto parses your example as RFC 3164.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions