Skip to content

DatasetReader: a row's parse error gives a position within the row, not the stream as protowire-go does #161

Closed

Description

@bfrancojr

When DatasetReader::Next refuses a row, the error's line and column count from the start of that row. protowire-go counts from the start of the stream. So the same input gives a different position in each port, and a C++ caller can't use it to find the bad row in a long stream.

Measured

Input @dataset x.Row ( raw )\n( b" + 256 × A + " )\n, read with max_bytes_literal_length = 191:

Port Next error
protowire-go 903b4a7, UnmarshalOptions{MaxBytesLiteralLength: 191}.NewDatasetReader(...).Next() 2:3: bytes literal decodes to more than MaxBytesLiteralLength=191 bytes
protowire-cpp f320a20 (#162's head), DatasetReader::Create(&in, opts)->Next(&row) 1:3: bytes literal decodes to more than MaxBytesLiteralLength=191 bytes

The message text matches; only the line differs. Errors from the header (Create) already match Go, positions included (1:21:, 1:25: in #162's tests).

Cause (read from the code)

DatasetReader::Next hands ParseDatasetRow the row's own bytes (row_bytes, cut from pending_). ParseDatasetRow builds a fresh Parser whose lexer starts at line 1, column 1. Nothing adds the row's offset in the stream. Go's reader keeps tr.base (a Position) and the lexer's base offset, and reports positions against the whole stream.

Not checked: the column on a row's first line when the row doesn't start at column 1, rows after the first, and positions from Scan's bind errors.

Done when

  • Next's parse errors carry protowire-go's line and column for the same input, for the first row, a later row, and a row that doesn't start at column 1.
  • A test pins each of those three cases against Go's values.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions