From 03283fe3fbb2c9e045e70fff51cd207e0d82963b Mon Sep 17 00:00:00 2001 From: "Benedicto Franco Jr." Date: Sat, 7 Nov 2026 03:46:19 -0800 Subject: [PATCH] release: cut v2.1.0 Bumps the CMake project version to 2.1.0 and cuts the 2.1.0 changelog section from Unreleased: the DatasetReader streaming work (#160, #161, #164, #166, #168, #169, #171) and keeping a comment that no entry follows (#176). Minor, because it adds the public Lexer(input, start) constructor (#161) and four public AST fields (#176). --- CHANGELOG.md | 11 ++++++++++- CMakeLists.txt | 2 +- 2 files changed, 11 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 4f2aafd..2b12425 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -11,6 +11,15 @@ number says nothing about any other port. See the Releases section of ## [Unreleased] +## [2.1.0] — 2026-11-07 + +A minor release. `pxf::DatasetReader` now streams a `@dataset` table as protowire-go's reader does. A row cut short, junk between rows and a broken header are refused with the error `Parse` gives, at the same position, instead of ending the table early. Positions count from the start of the stream. The caller's limits and options apply to every row, and a row on a stream that stays open is returned as soon as it is complete. A comment that no entry follows is now kept, at the end of a document, before a block's `}` and between dataset rows. Minor rather than a patch, because it adds a public constructor and four public AST fields. + +### Added + +- **`pxf::Lexer(std::string_view input, Position start)`** (#161): a lexer over a piece of a larger document, whose positions and offsets count from `start`. `DatasetReader` uses it so a row's positions are the stream's. +- **`trailing_comments` on `Document`, `Block` and `BlockVal`, and `leading_comments` on `DatasetRow`** (#176), as protowire-go has them. The Fixed entry for #176 says what they hold. + ### Fixed - **`DatasetReader` reads a row under the caller's `max_bytes_literal_length`** (#160), as protowire-go does. A row's cells were parsed under the default `kMaxBytesLiteralLength` whatever the options said. A caller who lowered the limit still got a row whose `b"…"` cell was past it, and a caller who raised it had that cell refused. `0` still means the default. Only the nesting depth used to reach the row parser. The header (a `@header` block and the `@dataset` line) was parsed under the defaults for both limits. It is now parsed under the caller's `max_nesting_depth` and `max_bytes_literal_length`, so `Create` refuses such a header as protowire-go's `NewDatasetReader` does, with the same text and position. @@ -23,7 +32,7 @@ number says nothing about any other port. See the Releases section of - **`DatasetReader` reads a long token in time linear in its length** (#168), as protowire-go does. A string, bytes literal, comment or number spread over many reads was lexed again from its start on every 4 KiB read: an unclosed 1 MiB string cost 136 MB of lexing, and 2 MiB cost 540 MB. The scan now lexes a token still open again only once the bytes read since could end it, which is Go's `mayEndToken` rule, and the header scan resumes where it stopped too. The same 1 MiB string now costs 1.05 MB. - **`DatasetReader` returns a complete row from a stream that stays open** (#171), as protowire-go does. On a pipe, a socket or a file still being appended to, `Next` could wait for 4 KiB more input, or for the stream to end, before returning a row that had already arrived. The read took a full 4 KiB (`std::istream::read`), and the scan held a token until four more bytes followed it. `Next` now reads what the stream has, blocking only when nothing is buffered. It takes a token as complete as soon as the bytes the lexer looks at past it have arrived. Input that ends reads as before. - **A row's validator error and schema violation carry no row position** (#166), as protowire-go gives them. `DatasetReader::Scan` and `BindRow` put the row's position in front of every error, so a validator's refusal read `2:1: check: id: denied (...)` and a schema violation `2:1: PXF schema violations: ...`. Only a decode error is moved to the row now, and a hook's error from `on_secret_field` is one, so it keeps the row's position and its column. `Scan` also checks the column paths before the schema, as Go does, so a row type that breaks both gets the column's error. -- **`DatasetReader::Scan` honours every option but `max_message_size`, and protowire-go is changing to match** (#166). Go's `Scan` ignored `skip_validate`, `skip_post_decode`, `discard_unknown`, `on_secret_field` and `validator`, while cpp's honoured them. The owner chose to change Go (trendvidia/protowire-go#280) and keep cpp's behaviour. Each option now has a test that binds the same row through `Scan` and `BindRow`, with the texts Go gives, and `dataset_reader.h` lists the options `Scan` uses. +- **`DatasetReader::Scan` honours every option but `max_message_size`, and protowire-go now matches** (#166; trendvidia/protowire-go#281). Go's `Scan` ignored `skip_validate`, `skip_post_decode`, `discard_unknown`, `on_secret_field` and `validator`, while cpp's honoured them. Go was changed to match (trendvidia/protowire-go#280), and cpp's behaviour stays. Each option now has a test that binds the same row through `Scan` and `BindRow`, with the texts Go gives, and `dataset_reader.h` lists the options `Scan` uses. - **A comment that no entry follows is kept** (#176), as protowire-go keeps it (protowire-go#279, PR #283). The parser attached a comment only to the entry after it. So a comment at the end of a document was dropped, and one before a block's closing `}` was dropped or carried out of the block onto the next entry. Comments between dataset rows went to the next directive, or were dropped at the end. `Document`, `Block` and `BlockVal` now have `trailing_comments`, and `DatasetRow` has `leading_comments`, which `Parse` and `DatasetReader` both set. `FormatDocument` writes each in place, `CanonicalizeKeyed` keeps them when it rewrites a keyed block, and `Rewriter` copies them. A comment inside a row now comes before the row. Comments inside a list still move after it (protowire-go#282). Streaming a 300,000-row table is about 4% slower, and 9% slower when each row carries a comment, since those comments are now kept. ## [2.0.0] — 2026-11-05 diff --git a/CMakeLists.txt b/CMakeLists.txt index 0bad1d6..facca68 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -1,5 +1,5 @@ cmake_minimum_required(VERSION 3.20) -project(protowire VERSION 2.0.0 LANGUAGES CXX) +project(protowire VERSION 2.1.0 LANGUAGES CXX) set(CMAKE_CXX_STANDARD 20) set(CMAKE_CXX_STANDARD_REQUIRED ON)