Skip to content

in_tcp: a parser error drops the other lines in the same read and closes the connection #5629

Open

Description

@Watson1978

Describe the bug

in_tcp does not rescue errors from @parser.parse(msg) for each line. When the parser raises, the error goes up to the server helper. The helper logs unexpected error on reading data and closes the connection.

  • With batch emit (the default), the lines already parsed in the same read are not emitted. The lines after the bad line are lost too.
  • With single emit (<extract> with tag_key), the lines before the bad line are emitted. The lines after it are lost.

A parser can raise on a normal input. For example, the json parser with time_format raises ParserError when the time field does not match the format. So one bad line from a client can drop its valid lines.

https://github.com/fluent/fluentd/blob/e7d3c3e4/lib/fluent/plugin/in_tcp.rb#L144
https://github.com/fluent/fluentd/blob/e7d3c3e4/lib/fluent/plugin/in_tcp.rb#L195

To Reproduce

Run Fluentd with the configuration below. Then send these three lines in one write:

{"k":"1"}
{"time":"bad","k":"2"}
{"k":"3"}
ruby -rsocket -e 'TCPSocket.open("127.0.0.1", 5170) { |s| s.write(%Q({"k":"1"}\n{"time":"bad","k":"2"}\n{"k":"3"}\n)); sleep 1 }'

Results on master (e7d3c3e):

  • Batch emit: no event is emitted. {"k":"1"} and {"k":"3"} are lost. The server closes the connection.
  • Single emit (add <extract> with tag_key tag): {"k":"1"} is emitted. {"k":"3"} is lost. The server closes the connection.

Expected behavior

Only the bad line is dropped, with a warning. This is how in_tcp handles a line that does not match the pattern (pattern not matched). The other lines are emitted, and the connection stays open.

Your Environment

- Fluentd version: master (e7d3c3e4). v1.20.1 and v1.19.4 have the same code (not run).
- Package version: -
- Operating system: Linux
- Kernel version: -

Your Configuration

<source>
  @type tcp
  port 5170
  bind 127.0.0.1
  tag test
  <parse>
    @type json
    time_key time
    time_format %Y-%m-%dT%H:%M:%S
  </parse>
</source>
<match test>
  @type stdout
</match>

Your Error Log

[error]: unexpected error on reading data host="127.0.0.1" port=39444 error_class=Fluent::Plugin::Parser::ParserError error="invalid time format: value = bad, error_class = ArgumentError, error = string doesn't match"
  [error]: lib/fluent/plugin/parser.rb:195:in 'Fluent::Plugin::Parser#parse_time'
  [error]: lib/fluent/plugin/parser_json.rb:70:in 'Fluent::Plugin::JSONParser#parse'
  [error]: lib/fluent/plugin/in_tcp.rb:195:in 'block in Fluent::Plugin::TcpInput#start'
  [error]: lib/fluent/plugin_helper/server.rb:661:in 'on_read_with_connection'

No test: {...} line is printed with batch emit.

Additional context

A possible fix is to rescue the error for each line. Then log a warning in the same way as warn_pattern_not_matched (truncated as in #5592), and go on to the next line.

Activity

  1. rafayqayyum commented on Nov 15, 2026

    @rafayqayyum

    Opened #5640 for this: in_tcp now rescues the parser error per line, logs a warning and keeps the connection open, for both batch and single emit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions