queryverse · davidanthoff · Sep 1, 2026 · Sep 1, 2026 · Sep 1, 2026 · Sep 1, 2026
diff --git a/NEWS.md b/NEWS.md
@@ -1,3 +1,46 @@
+# ReadStat.jl v2.0.0 Release Notes
+
+Breaking release. Highlights:
+
+* The readers return a `ReadStatTable`: concretely typed `DataValueVector`
+  columns by name or index, plus complete file-level (`filemetadata`) and
+  per-variable (`varmetadata`) metadata — creation/modified time, format
+  version, compression, endianness, encoding, notes, frequency weight,
+  variable labels, display formats, measures, alignments, and all value
+  labels. `ReadStatDataFrame` remains as a deprecated alias with `getproperty`
+  shims for the 1.x fields.
+* Reader keyword arguments: `usecols` (column projection inside the C
+  parse), `row_limit`/`row_offset`, `ntasks` (parallel chunked parsing into
+  shared buffers), `file_encoding`/`handler_encoding`, `user_missing`,
+  `convert_datetime`, `apply_value_labels`, `catalog`, and `progress`
+  (cancellable). New entry points: `readstat` (extension dispatch),
+  `read_meta` (metadata only), `read_sas7bcat` (SAS value-label catalogs),
+  and `read_txt` (fixed-width text via SAS/SPSS/Stata schema files). Every
+  reader also accepts an `IO`.
+* Columns with date/time display formats decode to `Date`/`DateTime`/`HMS`
+  by default (Stata, SAS, and SPSS format tables and epochs); `HMS` carries
+  times of day and durations beyond 24 hours.
+* Value labels: `valuelabels`, and the lazy `LabeledArray`/`LabeledValue`
+  view (via `labeled` or `apply_value_labels=true`) that displays labels but
+  computes on the raw codes. Tagged missing values keep their tags
+  (`missingtags`), including labeled tags; SPSS user-defined missing values
+  can be kept as data (`user_missing=:keep`) and their rules are always in
+  `varmetadata(tbl, col).missing_ranges`.
+* `ReadStatSource`: a lazy handle with `schema`/`colnames`/`coltypes`/
+  `nrows`/`supports`, pushdown-capable `read(src; usecols, rows)`, and
+  single-pass streaming `chunks`.
+* Write support for all formats: `write_dta`, `write_sav`, `write_por`,
+  `write_sas7bdat`, `write_xport`, and `write_sas7bcat`, from columns or a
+  `ReadStatTable`, covering value labels, tagged and user-defined missing
+  values, notes, compression (`.zsav`), and date/time re-encoding; plus the
+  public low-level `ReadStat.Writer` (one-to-one with the C writer API,
+  including Stata strL) and the complete raw C bindings in `ReadStat.CAPI`.
+* Fixes: the `readstat_value_t` ABI on Windows is now verified behaviorally
+  against the C library; cells the parser never delivers are NA over defined
+  storage instead of undefined memory; parsers are freed even when a handler
+  throws.
+* Requires Julia 1.10.
+
 # ReadStat.jl v1.1.0 Release Notes
 * Add support for SAS XPORT
 

diff --git a/Project.toml b/Project.toml
@@ -1,6 +1,6 @@
 name = "ReadStat"
 uuid = "d71aba96-b539-5138-91ee-935c3ee1374c"
-version = "1.2.1-DEV"
+version = "2.0.0-DEV"
 
 [deps]
 Dates = "ade2ca70-3891-5945-98fb-dc099432e06a"

diff --git a/README.md b/README.md
@@ -6,31 +6,97 @@
 
 ## Overview
 
-ReadStat.jl: Read files from Stata, SPSS, and SAS
---
+ReadStat.jl reads and writes the data file formats of Stata (`.dta`), SPSS
+(`.sav`, `.zsav`, `.por`), and SAS (`.sas7bdat`, `.xpt`, `.sas7bcat` value
+catalogs), plus fixed-width text files described by schema files, using the
+[ReadStat](https://github.com/WizardMac/ReadStat) C library. Missing data is
+represented with [DataValues.jl](https://github.com/queryverse/DataValues.jl).
 
-The ReadStat.jl Julia package uses the [ReadStat](https://github.com/WizardMac/ReadStat) C library to parse binary and transport files from Stata, SPSS and SAS. All functions return a `ReadStatDataFrame` whose fields hold the various informations contained in the passed file (column names, column data, labels, formats...).
+For integration with packages like
+[DataFrames.jl](https://github.com/JuliaData/DataFrames.jl) you should use
+the [StatFiles.jl](https://github.com/queryverse/StatFiles.jl) package.
 
-For integration with packages like [DataFrames.jl](https://github.com/JuliaData/DataFrames.jl) you should use the [StatFiles.jl](https://github.com/queryverse/StatFiles.jl) package.
-
-## Usage:
+## Reading
 
 ```julia
 using ReadStat
 
-read_dta("/path/to/something.dta")
+tbl = read_dta("data.dta")        # also read_sav, read_por, read_sas7bdat, read_xport
+tbl = readstat("data.sav")        # infer the format from the extension
+tbl = read_dta(io)                # any IO holding the file works too
+
+tbl[:price]                       # columns by name or index (DataValueVectors)
+names(tbl), size(tbl)
+filemetadata(tbl)                 # file label, timestamps, notes, encoding, ...
+varmetadata(tbl, :price)          # variable label, display format, type, ...
+```
+
+Reads can push work into the C parser:
+
+```julia
+read_dta("data.dta"; usecols = [:price, :mpg])   # column projection
+read_dta("data.dta"; row_offset = 1000, row_limit = 500)
+read_dta("data.dta"; ntasks = 8)                 # parallel chunked parsing
+read_meta("data.dta")                            # metadata only, zero rows
+```
+
+Columns with date/time display formats decode to `Date`, `DateTime`, or
+`HMS` (a time-of-day/duration type with unbounded hours) automatically;
+pass `convert_datetime=false` for the raw numbers.
 
-read_por("/path/to/something.por")
+Value labels are always parsed (`valuelabels(tbl, :rep77)`), and the
+`labeled` view or `apply_value_labels=true` wraps labeled columns in a
+`LabeledArray` whose elements display as their labels but compute as their
+raw codes. SAS value labels live in catalog files:
+`read_sas7bdat("f.sas7bdat"; catalog="formats.sas7bcat")`.
 
-read_sav("/path/to/something.sav")
+Tagged missing values (Stata/SAS `.a`-`.z`) read as NA with their tags
+available via `missingtags(tbl, col)`; SPSS user-defined missing values
+collapse to NA by default or stay data with `user_missing=:keep`.
 
-read_sas7bdat("/path/to/something.sas7bdat")
+## Lazy sources and streaming
 
-read_xport("path/to/something.xpt")
+`ReadStatSource` is a cheap handle for consumers that plan before they read:
+
+```julia
+src = ReadStatSource("big.dta")
+schema(src); colnames(src); coltypes(src); nrows(src)
+read(src; usecols = [:price], rows = 1_000:2_000)
+for chunk in chunks(src; chunksize = 100_000)    # single-pass streaming
+    # chunk is a complete ReadStatTable
+end
 ```
 
+## Writing
+
+```julia
+using DataValues
+
+write_dta("out.dta",
+    Any[DataValueArray([1, 2, 3]), DataValueArray(["a", "b", "c"])],
+    [:id, :tag];
+    labels = ["identifier", "a string"], file_label = "my data")
+
+write_sav("out.sav", tbl)          # write a ReadStatTable, metadata included
+```
+
+`write_dta`, `write_sav`, `write_por`, `write_sas7bdat`, `write_xport`, and
+`write_sas7bcat` cover value labels, tagged and user-defined missing values,
+notes, compression (including `.zsav`), and date/time re-encoding. The
+unexported low-level `ReadStat.Writer` follows the C writer API one-to-one
+(including Stata strL columns), and `ReadStat.CAPI` exposes the raw C
+bindings for anything else.
+
+## Migrating from 1.x
+
+The readers now return a `ReadStatTable`. The old `ReadStatDataFrame` name
+and its field access (`df.data`, `df.headers`, ...) keep working with
+deprecation warnings; switch to indexing and the metadata accessors. Date
+and time columns are now decoded by default (`convert_datetime=false`
+restores raw numbers), and cells the parser never delivers are NA instead
+of undefined memory.
+
 ## Installation
-To install the package, run the following:
 
 ```julia
 Pkg.add("ReadStat")

diff --git a/docs/Project.toml b/docs/Project.toml
@@ -1,5 +1,6 @@
 [deps]
 Documenter = "e30172f5-a6a5-5a46-863b-614d45cd2de4"
+ReadStat = "d71aba96-b539-5138-91ee-935c3ee1374c"
 
 [compat]
 Documenter = "1"
diff --git a/docs/src/index.md b/docs/src/index.md
@@ -1 +1,44 @@
 # Introduction
+
+ReadStat.jl reads and writes the data file formats of Stata (`.dta`), SPSS
+(`.sav`, `.zsav`, `.por`), and SAS (`.sas7bdat`, `.xpt`, and `.sas7bcat`
+value-label catalogs), plus fixed-width text files described by schema
+files, using the [ReadStat](https://github.com/WizardMac/ReadStat) C
+library. Missing data is represented with
+[DataValues.jl](https://github.com/queryverse/DataValues.jl).
+
+## Reading
+
+```julia
+using ReadStat
+
+tbl = read_dta("data.dta")     # read_sav, read_por, read_sas7bdat, read_xport
+tbl = readstat("data.sav")     # dispatch on the file extension
+```
+
+The result is a [`ReadStatTable`](@ref): columns by name (`tbl[:price]`) or
+index, [`filemetadata`](@ref) and [`varmetadata`](@ref) for everything the
+file records, [`valuelabels`](@ref) and [`labeled`](@ref) for value labels,
+and [`missingtags`](@ref) for tagged missing values. All readers accept the
+same keyword arguments for column projection, row selection, parallel
+parsing, encodings, missing-value semantics, and date/time conversion — see
+[`read_dta`](@ref).
+
+For metadata without data, use [`read_meta`](@ref). For planning and
+streaming — schema first, then a projected read or a single-pass chunked
+scan — use [`ReadStatSource`](@ref) with `read` and [`chunks`](@ref).
+
+## Writing
+
+[`write_dta`](@ref), `write_sav`, `write_por`, `write_sas7bdat`,
+`write_xport`, and [`write_sas7bcat`](@ref) write columns or a whole
+`ReadStatTable`, including value labels, missing-value declarations, notes,
+compression, and date/time re-encoding. The public but unexported
+[`ReadStat.Writer`](@ref) mirrors the C writer API one-to-one, and
+`ReadStat.CAPI` holds the raw C bindings.
+
+# API Reference
+
+```@autodocs
+Modules = [ReadStat]
+```
diff --git a/src/C_interface.jl b/src/C_interface.jl