Repository navigation
in_tcp: drop only the line that the parser fails on - #5644
Closed
Shriraj1901 wants to merge 1 commit into
Closed
Shriraj1901 wants to merge 1 commit into
Shriraj1901 wants to merge 1 commit into
Conversation
When @parser.parse raised an error for a line (for example ParserError from the json parser when the time field does not match time_format), the error was not rescued in in_tcp. It went up to the server helper, which logged "unexpected error on reading data" and closed the connection. With batch emit the lines that were already parsed in the same read were not emitted, and with single emit the lines after the bad line were lost. One bad line from a client dropped its valid lines. Rescue ParserError per line, log a warning and go on with the next line, as is done for a line that doesn't match the pattern. Fixes fluent#5629 Signed-off-by: shreeraj <shreerajjadhav19@gmail.com>
Contributor
|
Thank you for working on this. #5640 was opened a few hours earlier for the same issue, with a similar change and the same test inputs, so I'm closing this one as a duplicate. Sorry about the overlap. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When @parser.parse raised an error for a line (for example ParserError from the json parser when the time field does not match time_format), the error was not rescued in in_tcp. It went up to the server helper, which logged "unexpected error on reading data" and closed the connection.
With batch emit the lines that were already parsed in the same read were not emitted, and with single emit the lines after the bad line were lost. One bad line from a client dropped its valid lines.
Rescue ParserError per line, log a warning and go on with the next line, as is done for a line that doesn't match the pattern.
Fixes #5629
Which issue(s) this PR fixes:
Fixes #
What this PR does / why we need it:
Docs Changes:
Release Note: