From 0f34c36c9751c5570271f615852ecca168004baa Mon Sep 17 00:00:00 2001 From: Bart Date: Sun, 9 Aug 2026 00:32:13 +0200 Subject: [PATCH 1/6] Read per-node aero tables without a quadratic vcat, and optionally as Arrow `read_node_table` built its value matrix with `reduce(vcat, generator)`. Base's fast `reduce(::typeof(vcat), ::AbstractVector{<:AbstractVecOrMat})` does not match a generator, so it fell back to `foldl` and copied the whole matrix once per row. Parsing straight into a preallocated matrix is 21x faster on a 16 MB surface table (2.49 s to 0.12 s) and 15x over a full dataset: the 90 SK100 pressure tables, 1.33 GB, go from 176.6 s to 11.7 s. No API change, so every existing dataset gets it. `table_format` on `write_section_aero`, threaded through `generate_airfoils`, `obj_to_yaml` and `surfplan_to_aero_yaml`, writes those tables as Arrow instead: another 39x (0.003 s per table) and 2.6x smaller on disk. It defaults to `:csv`, and `read_section_aero` picks the format from the file suffix, so a geometry YAML can reference either and existing data keeps working untouched. Co-Authored-By: Claude Opus 5 --- CHANGELOG.md | 13 ++++++++ Project.toml | 2 ++ src/VortexStepMethod.jl | 1 + src/airfoil_aero/AirfoilAero.jl | 1 + src/airfoil_aero/geometry_gen.jl | 16 +++++---- src/airfoil_aero/section_aero_gen.jl | 44 +++++++++++++++++-------- src/obj_adapter/obj_to_yaml.jl | 8 +++-- src/section_aero.jl | 36 +++++++++++++++----- src/surfplan_adapter/SurfplanAdapter.jl | 7 ++-- test/airfoil_aero/test_airfoil_aero.jl | 11 ++++++- 10 files changed, 106 insertions(+), 33 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 50f48fdf..eb7d3d4d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,18 @@ # Changelog +## Unreleased + +### Added +- `table_format` keyword on `write_section_aero`, `generate_airfoils`, `obj_to_yaml` + and `surfplan_to_aero_yaml`: `:csv` (default, readable) or `:arrow` (binary, ~40× + faster to load and 2.6× smaller). `read_section_aero` detects the format from the + file suffix, so a geometry YAML can reference either. + +### Changed +- `read_node_table` parses into a preallocated matrix instead of `reduce(vcat, …)` + over a generator, which was quadratic in the row count: ~21× faster on a 16 MB + surface table (2.49 s → 0.12 s), benefiting every existing dataset. + ## VortexStepMethod v4.0.0 2026-08-03 ### Added diff --git a/Project.toml b/Project.toml index 9d880e9f..c8665e6f 100644 --- a/Project.toml +++ b/Project.toml @@ -7,6 +7,7 @@ version = "4.0.0" projects = ["examples", "docs", "test"] [deps] +Arrow = "69666777-d1a9-59fb-9406-91d4454c9d45" Colors = "5ae59095-9a9b-59fe-a467-6f913c188581" DefaultApplication = "3f0dd361-4fe0-5fc6-8523-80b14ec94d85" DelimitedFiles = "8bb1440f-4735-579b-a4ab-409b98df4dab" @@ -42,6 +43,7 @@ MakieControlPlots = "6d616b69-6563-4f6e-8472-6f6c706c6f74" VortexStepMethodMakieExt = ["MakieControlPlots"] [compat] +Arrow = "2.8" Colors = "0.13" DefaultApplication = "1" DelimitedFiles = "1" diff --git a/src/VortexStepMethod.jl b/src/VortexStepMethod.jl index c8b8de59..518d8303 100644 --- a/src/VortexStepMethod.jl +++ b/src/VortexStepMethod.jl @@ -1,5 +1,6 @@ module VortexStepMethod +using Arrow using LinearAlgebra using StaticArrays using Logging diff --git a/src/airfoil_aero/AirfoilAero.jl b/src/airfoil_aero/AirfoilAero.jl index 3a750048..3c162956 100644 --- a/src/airfoil_aero/AirfoilAero.jl +++ b/src/airfoil_aero/AirfoilAero.jl @@ -1,5 +1,6 @@ module AirfoilAero +using Arrow using LinearAlgebra using Parameters using Statistics diff --git a/src/airfoil_aero/geometry_gen.jl b/src/airfoil_aero/geometry_gen.jl index 08e3dae1..81b5bade 100644 --- a/src/airfoil_aero/geometry_gen.jl +++ b/src/airfoil_aero/geometry_gen.jl @@ -1,8 +1,8 @@ """ generate_airfoils(airfoils, output_dir; Re, alpha_range=-180:1:180, delta_range=nothing, aero_solver=NeuralFoilSolver(), - reuse_valid_airfoils=true, crease_frac=0.75, verbose=true) - -> (airfoil_rows, ok) + reuse_valid_airfoils=true, crease_frac=0.75, verbose=true, + table_format=:csv) -> (airfoil_rows, ok) Run the 2D solver over a set of already-shrink-wrapped airfoils and write the per section files each geometry route references. Shared by the `.obj` and Surfplan @@ -15,9 +15,10 @@ the wrapped airfoil the solver analyses; `x_raw`/`y_raw` the raw points it enclo Writes into `output_dir` (indexed by `id`), one directory per file kind: `airfoils/{id}.dat` (wrapped shape), `airfoils/{id}_{delta_suffix(δ)}.dat` (per -deflection), `airfoils/{id}_raw.dat` (raw points); `pressure/{id}_cp.csv` / `_cf.csv` -(per-node surface pressure and skin friction); and `polars/{id}.csv` (`POLAR_VECTORS`, -or a `POLAR_MATRICES` grid when `delta_range` is set). `aero_solver` selects the backend +deflection), `airfoils/{id}_raw.dat` (raw points); `pressure/{id}_cp.{table_format}` / +`_cf.{table_format}` (per-node surface pressure and skin friction, `:csv` or the far +faster-loading `:arrow`); and `polars/{id}.csv` (`POLAR_VECTORS`, or a `POLAR_MATRICES` +grid when `delta_range` is set). `aero_solver` selects the backend ([`NeuralFoilSolver`](@ref) default, [`XFoilSolver`](@ref) opt-in). With `reuse_valid_airfoils=true` an airfoil the solver cannot converge is skipped @@ -26,7 +27,8 @@ With `reuse_valid_airfoils=true` an airfoil the solver cannot converge is skippe function generate_airfoils(airfoils, output_dir::String; Re::Real, alpha_range=-180:1:180, delta_range=nothing, aero_solver::AbstractAirfoilSolver=NeuralFoilSolver(), - reuse_valid_airfoils::Bool=true, crease_frac=0.75, verbose::Bool=true) + reuse_valid_airfoils::Bool=true, crease_frac=0.75, verbose::Bool=true, + table_format::Symbol=:csv) mkpath(joinpath(output_dir, "airfoils")) mkpath(joinpath(output_dir, "polars")) mkpath(joinpath(output_dir, "pressure")) @@ -54,7 +56,7 @@ function generate_airfoils(airfoils, output_dir::String; coeff(s -> s.cl), coeff(s -> s.cd), coeff(s -> s.cm)) end paths = write_section_aero(joinpath(output_dir, "airfoils", "$j"), aero; - table_prefix=joinpath(output_dir, "pressure", "$j")) + table_prefix=joinpath(output_dir, "pressure", "$j"), table_format) dat_rel, cp_rel, cf_rel = (relpath(p, output_dir) for p in paths) write_dat(joinpath(output_dir, raw_rel), "section_$(j)_raw", af.x_raw, af.y_raw) diff --git a/src/airfoil_aero/section_aero_gen.jl b/src/airfoil_aero/section_aero_gen.jl index 2bb41947..fa609275 100644 --- a/src/airfoil_aero/section_aero_gen.jl +++ b/src/airfoil_aero/section_aero_gen.jl @@ -94,11 +94,22 @@ end """ write_node_table(path, aero, values) -> path -Write a per-node aero table (`Cp` or `cf`, shaped `n_node × n_alpha × n_delta`) as a -human-readable CSV: header `alpha, delta, n0, n1, …` (node columns in the contour node -order), one row per `(alpha, delta)` with angles in degrees. +Write a per-node aero table (`Cp` or `cf`, shaped `n_node × n_alpha × n_delta`), one row +per `(alpha, delta)` with angles in degrees. The file suffix picks the format: `.arrow` +(columns `alpha`, `delta` and a per-row list column `values`, an order of magnitude +faster to load), anything else a human-readable CSV with header `alpha, delta, n0, n1, …` +(node columns in the contour node order). [`read_node_table`](@ref) reads both. """ function write_node_table(path::AbstractString, aero::SectionAero, values) + if endswith(String(path), ".arrow") + grid = [(jd, ia) for jd in eachindex(aero.delta_range) + for ia in eachindex(aero.alpha_range)] + Arrow.write(String(path), + (alpha=[rad2deg(aero.alpha_range[ia]) for (_, ia) in grid], + delta=[rad2deg(aero.delta_range[jd]) for (jd, _) in grid], + values=[values[:, ia, jd] for (jd, ia) in grid])) + return path + end open(String(path), "w") do io println(io, "alpha,delta," * join(("n$(k - 1)" for k in 1:size(values, 1)), ",")) for jd in eachindex(aero.delta_range), ia in eachindex(aero.alpha_range) @@ -111,19 +122,24 @@ function write_node_table(path::AbstractString, aero::SectionAero, values) end """ - write_section_aero(dat_prefix, aero::SectionAero; table_prefix=dat_prefix) - -> (dat, cp_csv, cf_csv) + write_section_aero(dat_prefix, aero::SectionAero; table_prefix=dat_prefix, + table_format=:csv) -> (dat, cp_table, cf_table) Write a [`SectionAero`](@ref) as human-readable files. The airfoil contours share `dat_prefix`: `{dat_prefix}.dat` (contour at `delta=0`) plus `{dat_prefix}_{delta_suffix(δ)}.dat` per non-zero deflection. The per-node `Cp`/`cf` -tables share `table_prefix` (defaults to `dat_prefix`): `{table_prefix}_cp.csv` and -`{table_prefix}_cf.csv`. Pass a separate `table_prefix` to keep the surface tables out -of the airfoil-shape directory. `read_section_aero` reads them back. The single writer -for surface aero (submodule side); loading lives in the main package. +tables share `table_prefix` (defaults to `dat_prefix`): `{table_prefix}_cp.{ext}` and +`{table_prefix}_cf.{ext}`, where `ext` is `table_format`, either `:csv` (default, +readable) or `:arrow` (binary, an order of magnitude faster to load). Pass a separate +`table_prefix` to keep the surface tables out of the airfoil-shape directory. +`read_section_aero` reads either format back, detecting it from the suffix. The single +writer for surface aero (submodule side); loading lives in the main package. """ function write_section_aero(dat_prefix::AbstractString, aero::SectionAero; - table_prefix::AbstractString=dat_prefix) + table_prefix::AbstractString=dat_prefix, + table_format::Symbol=:csv) + table_format in (:csv, :arrow) || + throw(ArgumentError("table_format must be :csv or :arrow, got :$table_format")) mkpath(dirname(dat_prefix)) mkpath(dirname(table_prefix)) for (jd, d) in enumerate(aero.delta_range) @@ -136,7 +152,9 @@ function write_section_aero(dat_prefix::AbstractString, aero::SectionAero; jz = findfirst(iszero, aero.delta_range) jdat = jz !== nothing && finite(jz) ? jz : findfirst(finite, eachindex(aero.delta_range)) write_dat("$dat_prefix.dat", "section", aero.x[:, jdat], aero.y[:, jdat]) - write_node_table("$(table_prefix)_cp.csv", aero, aero.cp) - write_node_table("$(table_prefix)_cf.csv", aero, aero.cf) - return "$dat_prefix.dat", "$(table_prefix)_cp.csv", "$(table_prefix)_cf.csv" + cp_path = "$(table_prefix)_cp.$table_format" + cf_path = "$(table_prefix)_cf.$table_format" + write_node_table(cp_path, aero, aero.cp) + write_node_table(cf_path, aero, aero.cf) + return "$dat_prefix.dat", cp_path, cf_path end diff --git a/src/obj_adapter/obj_to_yaml.jl b/src/obj_adapter/obj_to_yaml.jl index 08d79eb7..cf49f244 100644 --- a/src/obj_adapter/obj_to_yaml.jl +++ b/src/obj_adapter/obj_to_yaml.jl @@ -81,6 +81,9 @@ share a neighbour's id): `{tag}` is the deflection in degrees with `m` for minus and `p` for the decimal point (e.g. `_dm3.dat`, `_d2p5.dat`) - `polars/{j}.csv` — NeuralFoil polar (alpha, Cd, Cs, Cl, Cm) +- `pressure/{j}_cp.{table_format}`, `_cf.{table_format}` — per-node surface pressure and + skin friction; `table_format` is `:csv` (default, readable) or `:arrow` (binary, an + order of magnitude faster to load) - `geometry.yaml` — `wing_sections` + `wing_airfoils` referencing the above # Returns @@ -94,7 +97,7 @@ function obj_to_yaml(obj_path::String, output_dir::String; reuse_valid_airfoils::Bool=true, max_thickness_ratio::Real=2.0, spanwise_direction=[0.0, 1.0, 0.0], rotation=I, wingtip_distance=0.05, crease_frac=0.75, force::Bool=false, - verbose::Bool=true) + verbose::Bool=true, table_format::Symbol=:csv) (!endswith(obj_path, ".obj")) && (obj_path *= ".obj") isfile(obj_path) || error("OBJ file not found: $obj_path") !isapprox(spanwise_direction, [0.0, 1.0, 0.0]) && @@ -132,7 +135,8 @@ function obj_to_yaml(obj_path::String, output_dir::String; airfoils = [(; id = j, x_fit = stations[j].x_fit, y_fit = stations[j].y_fit, x_raw = stations[j].xa, y_raw = stations[j].ya) for j in unique(ids)] airfoil_rows, ok = generate_airfoils(airfoils, output_dir; Re, alpha_range, - delta_range, aero_solver, reuse_valid_airfoils, crease_frac, verbose) + delta_range, aero_solver, reuse_valid_airfoils, crease_frac, verbose, + table_format) isempty(ok) && error("No section produced a valid polar in $obj_path") # Each section uses its nearest airfoil that actually produced a polar — covering diff --git a/src/section_aero.jl b/src/section_aero.jl index ba4755b1..1f7863c6 100644 --- a/src/section_aero.jl +++ b/src/section_aero.jl @@ -110,15 +110,34 @@ end """ read_node_table(path) -> (alpha, delta, values) -Read a per-node aero CSV (header `alpha, delta, n0, n1, …`; angles in degrees) into -radian `alpha`/`delta` vectors (one entry per row) and a `nrow × n_node` value matrix. +Read a per-node aero table into radian `alpha`/`delta` vectors (one entry per row) and a +`nrow × n_node` value matrix. The file suffix picks the format: `.arrow` (columns +`alpha`, `delta` in degrees and a per-row list column `values`), anything else CSV +(header `alpha, delta, n0, n1, …`, angles in degrees). """ function read_node_table(path::AbstractString) + if endswith(String(path), ".arrow") + table = Arrow.Table(String(path)) + values = Matrix{Float64}(undef, length(table.alpha), length(first(table.values))) + for k in axes(values, 1) + @inbounds values[k, :] .= table.values[k] + end + return deg2rad.(table.alpha), deg2rad.(table.delta), values + end lines = [l for l in readlines(String(path)) if !isempty(strip(l))] - rows = [parse.(Float64, split(l, ',')) for l in lines[2:end]] - alpha = deg2rad.([r[1] for r in rows]) - delta = deg2rad.([r[2] for r in rows]) - values = reduce(vcat, (permutedims(r[3:end]) for r in rows)) + n_row = length(lines) - 1 + n_col = count(==(','), lines[1]) + 1 + alpha = Vector{Float64}(undef, n_row) + delta = Vector{Float64}(undef, n_row) + values = Matrix{Float64}(undef, n_row, n_col - 2) + for k in 1:n_row + fields = split(lines[k + 1], ',') + alpha[k] = deg2rad(parse(Float64, fields[1])) + delta[k] = deg2rad(parse(Float64, fields[2])) + @inbounds for j in 3:n_col + values[k, j - 2] = parse(Float64, fields[j]) + end + end return alpha, delta, values end @@ -127,8 +146,9 @@ end Assemble a [`SectionAero`](@ref) from the human-readable files: the airfoil contour (`dat_file`, plus `{stem}_{delta_suffix(δ)}.dat` per non-zero deflection) and the per-node `Cp` -and `cf` tables (`alpha, delta, n0…` in the `.dat` node order). Returns `nothing` if any -file is missing. This is the single loader for both provided and generated aero. +and `cf` tables in the `.dat` node order, CSV or Arrow as their suffix says (see +[`read_node_table`](@ref)). Returns `nothing` if any file is missing. This is the single +loader for both provided and generated aero. """ function read_section_aero(dat_file::AbstractString, cp_file::AbstractString, cf_file::AbstractString) diff --git a/src/surfplan_adapter/SurfplanAdapter.jl b/src/surfplan_adapter/SurfplanAdapter.jl index ce2720d3..6424a512 100644 --- a/src/surfplan_adapter/SurfplanAdapter.jl +++ b/src/surfplan_adapter/SurfplanAdapter.jl @@ -23,6 +23,8 @@ placement (`wing_sections`) and each airfoil's `.dat` path (`wing_airfoils` default, [`XFoilSolver`](@ref) opt-in). Sections that share an airfoil generate its tables once. `Re` defaults to the export's `wing_airfoils.reynolds`; `alpha_range` defaults to the full `-180:1:180` sweep rather than the export's narrow polar range. +`table_format` writes the per-node surface tables as `:csv` (default, readable) or +`:arrow` (binary, an order of magnitude faster to load). The `.txt` → adapter-YAML step (the upstream Python `SurfplanAdapter`) is the documented prerequisite. Load the result with `Wing(geometry_yaml_path)`. @@ -34,7 +36,8 @@ function surfplan_to_aero_yaml(adapter_dir::AbstractString, output_dir::Abstract aero_solver::AbstractAirfoilSolver=NeuralFoilSolver(), wrap_method::ShrinkWrap=ShrinkWrap(), alpha_range=-180:1:180, delta_range=nothing, Re=nothing, - crease_frac=0.75, force::Bool=false, verbose::Bool=true) + crease_frac=0.75, force::Bool=false, verbose::Bool=true, + table_format::Symbol=:csv) yaml_path = joinpath(output_dir, "geometry.yaml") if !force && isfile(yaml_path) verbose && @info "Reusing existing $yaml_path (pass force=true to regenerate)" @@ -68,7 +71,7 @@ function surfplan_to_aero_yaml(adapter_dir::AbstractString, output_dir::Abstract end airfoil_rows, ok = generate_airfoils(airfoils, output_dir; Re, alpha_range, - delta_range, aero_solver, crease_frac, verbose) + delta_range, aero_solver, crease_frac, verbose, table_format) isempty(ok) && error("No airfoil produced a valid polar from $adapter_dir") remap(id) = id in ok ? id : ok[argmin(abs.(ok .- id))] diff --git a/test/airfoil_aero/test_airfoil_aero.jl b/test/airfoil_aero/test_airfoil_aero.jl index 5b43eba1..28a7dac1 100644 --- a/test/airfoil_aero/test_airfoil_aero.jl +++ b/test/airfoil_aero/test_airfoil_aero.jl @@ -116,7 +116,7 @@ end @test size(cm) == (length(alpha_range), length(delta_range)) end -@testset "SectionAero CSV round-trip and interpolation" begin +@testset "SectionAero table round-trip and interpolation" begin alpha_range = deg2rad.([-5.0, 0.0, 5.0, 10.0]) delta_range = deg2rad.([-3.0, 0.0, 3.0]) xc = [1.0, 0.5, 0.0, 0.5, 1.0] @@ -142,6 +142,15 @@ end _, _, cpa, _ = section_surface(aero, alpha_range[3], delta_range[2]) @test cpa ≈ cp[:, 3, 2] + + dat_a, cp_arrow, cf_arrow = write_section_aero(prefix, aero; table_format=:arrow) + @test endswith(cp_arrow, ".arrow") && endswith(cf_arrow, ".arrow") + arrow = read_section_aero(dat_a, cp_arrow, cf_arrow) + @test arrow.alpha_range == back.alpha_range + @test arrow.delta_range == back.delta_range + @test isequal(arrow.cp, back.cp) + @test isequal(arrow.cf, back.cf) + @test_throws ArgumentError write_section_aero(prefix, aero; table_format=:parquet) end @testset "generate_section_aero builds a surface table" begin From 437a4428a0b9eac99ebe4c429ff597073e87be7a Mon Sep 17 00:00:00 2001 From: Bart Date: Sun, 9 Aug 2026 10:44:07 +0200 Subject: [PATCH 2/6] Convert an existing node-table dataset between CSV and Arrow write_node_rows writes a per-node table from the flat (alpha, delta, values) form read_node_table returns, picking the format from the destination suffix, and convert_node_table pairs the two. write_node_table now delegates to it rather than carrying its own copy of the format branch. obj_to_yaml migrates a cached dataset when table_format differs from what the directory holds, so switching an existing 1.4 GB pressure dir from :csv to :arrow costs one rewrite instead of re-running the NeuralFoil polars that produced it. The source tables are left in place. Co-Authored-By: Claude Opus 5 --- CHANGELOG.md | 4 +++ docs/src/private_functions.md | 3 ++ src/airfoil_aero/AirfoilAero.jl | 4 +-- src/airfoil_aero/section_aero_gen.jl | 25 +++++--------- src/obj_adapter/obj_to_yaml.jl | 50 ++++++++++++++++++++++++++++ src/section_aero.jl | 36 ++++++++++++++++++++ 6 files changed, 103 insertions(+), 19 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index eb7d3d4d..f2737cb6 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,10 @@ and `surfplan_to_aero_yaml`: `:csv` (default, readable) or `:arrow` (binary, ~40× faster to load and 2.6× smaller). `read_section_aero` detects the format from the file suffix, so a geometry YAML can reference either. +- `convert_node_table` and `write_node_rows` rewrite a per-node table in the format + the destination suffix names. `obj_to_yaml` migrates an existing dataset with + them when `table_format` differs from what the directory holds, so a dataset + changes format without re-running the airfoil solver that produced it. ### Changed - `read_node_table` parses into a preallocated matrix instead of `reduce(vcat, …)` diff --git a/docs/src/private_functions.md b/docs/src/private_functions.md index bec6cb73..e1786c54 100644 --- a/docs/src/private_functions.md +++ b/docs/src/private_functions.md @@ -78,6 +78,8 @@ load_matrix_polar_data read_aero_matrix read_dat read_node_table +write_node_rows +convert_node_table delta_suffix interpolate_matrix_nans! remove_vector_nans @@ -146,6 +148,7 @@ write_polar_csv write_polar_matrix_csv write_aero_matrix write_node_table +migrate_node_tables flat_plate_cf neuralfoil_contour_solution fill_node_nans! diff --git a/src/airfoil_aero/AirfoilAero.jl b/src/airfoil_aero/AirfoilAero.jl index 3c162956..531a5ad2 100644 --- a/src/airfoil_aero/AirfoilAero.jl +++ b/src/airfoil_aero/AirfoilAero.jl @@ -1,6 +1,5 @@ module AirfoilAero -using Arrow using LinearAlgebra using Parameters using Statistics @@ -8,7 +7,8 @@ using Interpolations using NPZ using Xfoil using Printf: @sprintf -using ..VortexStepMethod: SectionAero, interpolate_matrix_nans!, delta_suffix +using ..VortexStepMethod: SectionAero, interpolate_matrix_nans!, delta_suffix, + write_node_rows include("kulfan.jl") include("shrink_wrap.jl") diff --git a/src/airfoil_aero/section_aero_gen.jl b/src/airfoil_aero/section_aero_gen.jl index fa609275..d4a236a7 100644 --- a/src/airfoil_aero/section_aero_gen.jl +++ b/src/airfoil_aero/section_aero_gen.jl @@ -101,24 +101,15 @@ faster to load), anything else a human-readable CSV with header `alpha, delta, n (node columns in the contour node order). [`read_node_table`](@ref) reads both. """ function write_node_table(path::AbstractString, aero::SectionAero, values) - if endswith(String(path), ".arrow") - grid = [(jd, ia) for jd in eachindex(aero.delta_range) - for ia in eachindex(aero.alpha_range)] - Arrow.write(String(path), - (alpha=[rad2deg(aero.alpha_range[ia]) for (_, ia) in grid], - delta=[rad2deg(aero.delta_range[jd]) for (jd, _) in grid], - values=[values[:, ia, jd] for (jd, ia) in grid])) - return path + grid = [(jd, ia) for jd in eachindex(aero.delta_range) + for ia in eachindex(aero.alpha_range)] + rows = Matrix{Float64}(undef, length(grid), size(values, 1)) + for (k, (jd, ia)) in enumerate(grid) + rows[k, :] .= @view values[:, ia, jd] end - open(String(path), "w") do io - println(io, "alpha,delta," * join(("n$(k - 1)" for k in 1:size(values, 1)), ",")) - for jd in eachindex(aero.delta_range), ia in eachindex(aero.alpha_range) - row = [rad2deg(aero.alpha_range[ia]), rad2deg(aero.delta_range[jd])] - append!(row, values[:, ia, jd]) - println(io, join(row, ",")) - end - end - return path + return write_node_rows(path, + [aero.alpha_range[ia] for (_, ia) in grid], + [aero.delta_range[jd] for (jd, _) in grid], rows) end """ diff --git a/src/obj_adapter/obj_to_yaml.jl b/src/obj_adapter/obj_to_yaml.jl index cf49f244..78425319 100644 --- a/src/obj_adapter/obj_to_yaml.jl +++ b/src/obj_adapter/obj_to_yaml.jl @@ -89,6 +89,55 @@ share a neighbour's id): # Returns - Path to the written `geometry.yaml`. """ +function obj_to_yaml end + +""" + migrate_node_tables(yaml_path, output_dir, table_format; verbose=true) + +Rewrite a generated dataset's per-node `Cp`/`cf` tables in `table_format` and point +`geometry.yaml` at them, when they are not in that format already. This is what lets +an existing directory change format without re-running the airfoil solver that +produced it — the polars are the slow part and they are untouched. The source tables +are left in place. +""" +function migrate_node_tables(yaml_path::String, output_dir::String, + table_format::Symbol; verbose::Bool=true) + table_format in (:csv, :arrow) || + throw(ArgumentError("table_format must be :csv or :arrow, got :$table_format")) + data = YAML.load_file(yaml_path) + airfoils = get(data, "wing_airfoils", nothing) + airfoils === nothing && return yaml_path + headers = airfoils["headers"] + info_col = findfirst(==("info_dict"), headers) + info_col === nothing && return yaml_path + converted = 0 + for row in airfoils["data"] + info = row[info_col] + info isa AbstractDict || continue + for key in ("cp_file", "cf_file") + haskey(info, key) || continue + relative = String(info[key]) + endswith(relative, ".$table_format") && continue + fresh = string(splitext(relative)[1], ".", table_format) + source = joinpath(output_dir, relative) + isfile(source) || error("Table $source referenced by $yaml_path is missing") + isfile(joinpath(output_dir, fresh)) || + VortexStepMethod.convert_node_table(source, + joinpath(output_dir, fresh)) + info[key] = fresh + converted += 1 + end + end + converted == 0 && return yaml_path + id_col = findfirst(==("airfoil_id"), headers) + type_col = findfirst(==("type"), headers) + write_geometry_yaml(yaml_path, data["wing_sections"]["data"], + [Any[row[id_col], row[type_col], row[info_col]] + for row in airfoils["data"]]) + verbose && @info "Converted $converted node tables to :$table_format" yaml_path + return yaml_path +end + function obj_to_yaml(obj_path::String, output_dir::String; n_sections::Int, Re::Real, alpha_range=-180:1:180, delta_range=nothing, @@ -106,6 +155,7 @@ function obj_to_yaml(obj_path::String, output_dir::String; yaml_path = joinpath(output_dir, "geometry.yaml") if !force && isfile(yaml_path) verbose && @info "Reusing existing geometry (force=true to regenerate)" yaml_path + migrate_node_tables(yaml_path, output_dir, table_format; verbose) return yaml_path end diff --git a/src/section_aero.jl b/src/section_aero.jl index 1f7863c6..d821bff8 100644 --- a/src/section_aero.jl +++ b/src/section_aero.jl @@ -141,6 +141,42 @@ function read_node_table(path::AbstractString) return alpha, delta, values end +""" + write_node_rows(path, alpha, delta, values) -> path + +Write a per-node aero table from radian `alpha`/`delta` vectors (one entry per row) +and a `nrow × n_node` value matrix. The file suffix picks the format, matching +[`read_node_table`](@ref): `.arrow` for the binary form, anything else CSV. Angles +are written in degrees either way. +""" +function write_node_rows(path::AbstractString, alpha, delta, values) + if endswith(String(path), ".arrow") + Arrow.write(String(path), + (alpha=rad2deg.(alpha), delta=rad2deg.(delta), + values=[values[k, :] for k in axes(values, 1)])) + return path + end + open(String(path), "w") do io + println(io, "alpha,delta," * join(("n$(j - 1)" for j in axes(values, 2)), ",")) + for k in axes(values, 1) + row = [rad2deg(alpha[k]), rad2deg(delta[k])] + append!(row, @view values[k, :]) + println(io, join(row, ",")) + end + end + return path +end + +""" + convert_node_table(src, dst) -> dst + +Rewrite a per-node aero table in the format `dst`'s suffix names. Use to move an +existing dataset between CSV and Arrow without re-running the (slow) airfoil solver +that produced it. +""" +convert_node_table(src::AbstractString, dst::AbstractString) = + write_node_rows(dst, read_node_table(src)...) + """ read_section_aero(dat_file, cp_file, cf_file) -> Union{Nothing, SectionAero} From 093e206f9a8d85242561f513b7e60c37ca92bee7 Mon Sep 17 00:00:00 2001 From: Bart Date: Wed, 12 Aug 2026 22:36:03 +0200 Subject: [PATCH 3/6] Auto threads --- bin/run_julia | 1 + 1 file changed, 1 insertion(+) diff --git a/bin/run_julia b/bin/run_julia index 3c14cce7..fa14f603 100755 --- a/bin/run_julia +++ b/bin/run_julia @@ -5,6 +5,7 @@ if [[ $(basename $(pwd)) == "bin" ]]; then fi export JULIA_PKG_SERVER_REGISTRY_PREFERENCE=eager +export JULIA_NUM_THREADS="${JULIA_NUM_THREADS:-auto}" # If any local project preferences file disables CondaPkg, forward that as an # env var so that Julia reads it at runtime even when activating a subproject From 6725c9ee0c492fef69be64581a9e48b2864d7650 Mon Sep 17 00:00:00 2001 From: Bart Date: Wed, 12 Aug 2026 23:53:21 +0200 Subject: [PATCH 4/6] Document migrate_node_tables in its own module and cover it The docstring lives in ObjAdapter, so listing it under the AirfoilAero @docs block left the binding undefined and the docstring uncounted, failing the docs build. Test the format migration end to end, which was the bulk of the uncovered patch. Co-Authored-By: Claude Opus 5 (1M context) --- docs/src/private_functions.md | 2 +- test/obj_adapter/test_obj_adapter.jl | 29 ++++++++++++++++++++++++++++ 2 files changed, 30 insertions(+), 1 deletion(-) diff --git a/docs/src/private_functions.md b/docs/src/private_functions.md index e1786c54..282533cc 100644 --- a/docs/src/private_functions.md +++ b/docs/src/private_functions.md @@ -148,7 +148,6 @@ write_polar_csv write_polar_matrix_csv write_aero_matrix write_node_table -migrate_node_tables flat_plate_cf neuralfoil_contour_solution fill_node_nans! @@ -178,6 +177,7 @@ airfoils_from_yaml write_geometry_yaml resolve_aero_geometry plot_airfoil_fit +migrate_node_tables ``` ## Makie plotting internals diff --git a/test/obj_adapter/test_obj_adapter.jl b/test/obj_adapter/test_obj_adapter.jl index e51368eb..322113ed 100644 --- a/test/obj_adapter/test_obj_adapter.jl +++ b/test/obj_adapter/test_obj_adapter.jl @@ -61,6 +61,35 @@ obj_path = normpath(joinpath(@__DIR__, "..", "..", @test_throws ErrorException obj_to_yaml("missing.obj", outdir; n_sections=3, Re=5e5) end + @testset "obj_to_yaml migrates existing node tables to another format" begin + outdir = mktempdir() + csv_yaml = obj_to_yaml(obj_path, outdir; n_sections=3, Re=5e5, + alpha_range=-4:2:4, aero_solver=NeuralFoilSolver(model_size="medium"), + verbose=false) + csv_info = Dict(YAML.load_file(csv_yaml)["wing_airfoils"]["data"][1][3]) + + arrow_yaml = obj_to_yaml(obj_path, outdir; n_sections=3, Re=5e5, + verbose=false, table_format=:arrow) + info = Dict(YAML.load_file(arrow_yaml)["wing_airfoils"]["data"][1][3]) + @test endswith(info["cp_file"], ".arrow") + @test endswith(info["cf_file"], ".arrow") + @test isfile(joinpath(outdir, info["cp_file"])) + @test isfile(joinpath(outdir, info["cf_file"])) + @test isfile(joinpath(outdir, csv_info["cp_file"])) + @test Wing(arrow_yaml; n_panels=4) isa Wing + + # already in that format: nothing to convert, YAML untouched + again = obj_to_yaml(obj_path, outdir; n_sections=3, Re=5e5, + verbose=false, table_format=:arrow) + @test Dict(YAML.load_file(again)["wing_airfoils"]["data"][1][3]) == info + + @test_throws ArgumentError ObjAdapter.migrate_node_tables(arrow_yaml, outdir, + :parquet) + rm(joinpath(outdir, info["cp_file"])) + @test_throws ErrorException ObjAdapter.migrate_node_tables(arrow_yaml, outdir, + :csv) + end + @testset "center_to_com! rejects non-triangular faces" begin verts = [[0.0, 0.0, 0.0], [1.0, 0.0, 0.0], [0.0, 1.0, 0.0], [1.0, 1.0, 0.0]] @test_throws ArgumentError center_to_com!(verts, [[1, 2, 3, 4]]; prn=false) From 049d8535a48dec21a8f8773aa0f75b73357aace1 Mon Sep 17 00:00:00 2001 From: Bart Date: Thu, 13 Aug 2026 09:16:16 +0200 Subject: [PATCH 5/6] Make every docstring cross-reference resolve Documenter resolves a bare @ref in the module the docstring is rendered under, so refs pointing into another submodule silently degraded to plain text. Give those an explicit qualified target, document the three names that had no docstring to link to (solve_base!, PLATE_FACES, and POLY via its enum), and drop warnonly=[:cross_references] so a broken ref fails the build instead of passing quietly. Co-Authored-By: Claude Opus 5 (1M context) --- docs/make.jl | 1 - docs/src/functions.md | 1 + docs/src/private_functions.md | 1 + ext/VortexStepMethodMakieExt.jl | 31 +++++++++++++++++-------- src/VortexStepMethod.jl | 2 +- src/airfoil_aero/section_aero_gen.jl | 3 ++- src/section_aero.jl | 3 ++- src/solver.jl | 11 +++++++++ src/surfplan_adapter/SurfplanAdapter.jl | 6 +++-- 9 files changed, 43 insertions(+), 16 deletions(-) diff --git a/docs/make.jl b/docs/make.jl index b25fedbb..2688d0c7 100644 --- a/docs/make.jl +++ b/docs/make.jl @@ -14,7 +14,6 @@ makedocs(; VortexStepMethod.ObjAdapter], authors="Uwe Fechner , Bart van de Lint and contributors", sitename="VortexStepMethod.jl", - warnonly=[:cross_references], format = Documenter.HTML(prettyurls = haskey(ENV, "CI")), pages=[ "Home" => "index.md", diff --git a/docs/src/functions.md b/docs/src/functions.md index 72de8257..ef3c6160 100644 --- a/docs/src/functions.md +++ b/docs/src/functions.md @@ -84,6 +84,7 @@ CurrentModule = VortexStepMethod set_va! solve solve! +solve_base! reinit!(body_aero::BodyAerodynamics{P, W, T}) where {P, W, T} linearize calculate_results diff --git a/docs/src/private_functions.md b/docs/src/private_functions.md index 282533cc..94c0f3e1 100644 --- a/docs/src/private_functions.md +++ b/docs/src/private_functions.md @@ -195,6 +195,7 @@ airfoil_skin_geometry panel_normal plate_hinge_local panel_plate_geometry +PLATE_FACES Makie.plot!(ax, panel::VortexStepMethod.Panel) Makie.plot!(ax, body::VortexStepMethod.BodyAerodynamics) Makie.plot!(body::VortexStepMethod.BodyAerodynamics) diff --git a/ext/VortexStepMethodMakieExt.jl b/ext/VortexStepMethodMakieExt.jl index aa1fcfe1..dc6686f5 100644 --- a/ext/VortexStepMethodMakieExt.jl +++ b/ext/VortexStepMethodMakieExt.jl @@ -13,6 +13,12 @@ const PANEL_MESH_OBSERVABLES = Ref{Union{Nothing,Dict}}(nothing) # Global storage for airfoil-skin observables, keyed by body objectid. const AIRFOIL_SKIN_OBSERVABLES = Ref{Union{Nothing,Dict}}(nothing) +""" + PLATE_FACES + +The 4 triangles that mesh the 6 [`panel_plate_geometry`](@ref) vertices of a panel's +flat-plate skin, two per side of the hinge. +""" const PLATE_FACES = [Makie.GLTriangleFace(1, 2, 5), Makie.GLTriangleFace(1, 5, 6), Makie.GLTriangleFace(2, 3, 4), Makie.GLTriangleFace(2, 4, 5)] const PLATE_BORDER_IDX = [1, 2, 3, 4, 5, 6, 1] @@ -1510,8 +1516,8 @@ end Shrink-wrap a section's sliced contour with `wrap_method` and map it back into 3D through the section's local airfoil frame, for overlaying on the 3D slice diagnostic. A nonzero `delta` (degrees) deflects the trailing edge and re-wraps -([`deform_section`](@ref)), showing the geometry the solvers consume. -Returns `nothing` for a degenerate slice. +([`deform_section`](@ref VortexStepMethod.AirfoilAero.deform_section)), showing the +geometry the solvers consume. Returns `nothing` for a degenerate slice. """ function fitted_airfoil_3d(s, wrap_method; delta=0.0, crease_frac=0.75) frame = ObjAdapter.airfoil_frame(s.LE_point, s.TE_point, s.span_dir) @@ -1551,10 +1557,13 @@ end """ generated_slices(out_dir, delta, fit_pts) -> (slices, le, te) -Read the stations of a generated [`obj_to_yaml`](@ref) output directory and their -written `.dat` airfoils — raw slice, wrap, and the `delta`-degree deformed wrap when -it was generated — assembled for [`plot_slices_3d`](@ref). Nothing is re-sliced or -re-wrapped; only the Kulfan fits of the stored coordinates are recomputed (via +Read the stations of a generated +[`obj_to_yaml`](@ref VortexStepMethod.ObjAdapter.obj_to_yaml) output directory and +their written `.dat` airfoils — raw slice, wrap, and the `delta`-degree deformed wrap +when it was generated — assembled for +[`plot_slices_3d`](@ref VortexStepMethod.ObjAdapter.plot_slices_3d). Nothing is +re-sliced or re-wrapped; only the Kulfan fits of the stored coordinates are +recomputed (via `fit_pts`), exactly as the polar pipeline fits them. """ function generated_slices(out_dir, delta, fit_pts) @@ -1605,14 +1614,16 @@ end 3D slice diagnostic with a hover 2D airfoil panel: raw slice points (green), the shrink-wrapped airfoil (crimson) and its Kulfan fit (black dashed), and, for nonzero `delta` (degrees), the deflected re-wrapped airfoil (purple — the -[`deform_section`](@ref) geometry the solvers consume) and its Kulfan fit (orange -dashed). **Hovering a slice** updates the 2D panel. +[`deform_section`](@ref VortexStepMethod.AirfoilAero.deform_section) geometry the +solvers consume) and its Kulfan fit (orange dashed). **Hovering a slice** updates the +2D panel. `path` selects the source: - a mesh `.obj` file: live preview — slices and wraps here with `wrap_method` (`n_slices`, `n_bins`, `wingtip_distance`, `crease_frac`). -- a generated [`obj_to_yaml`](@ref) output directory: audit mode — stations and - airfoils are read from `geometry.yaml` and the written `.dat` files, so the plot +- a generated [`obj_to_yaml`](@ref VortexStepMethod.ObjAdapter.obj_to_yaml) output + directory: audit mode — stations and airfoils are read from `geometry.yaml` and the + written `.dat` files, so the plot shows exactly what the polar pipeline analysed. `delta` must then match a generated deflection value; pass `obj_path` to also draw the mesh (with the same `rotation` used at generation). diff --git a/src/VortexStepMethod.jl b/src/VortexStepMethod.jl index 518d8303..f0dc72bd 100644 --- a/src/VortexStepMethod.jl +++ b/src/VortexStepMethod.jl @@ -262,7 +262,7 @@ end """ LEI_AIRFOIL_BREUKELS -Deprecated alias of [`POLY`](@ref). The Breukels `(tube_diameter, camber)` → coeff +Deprecated alias of `POLY` (see [`AeroModel`](@ref)). The Breukels `(tube_diameter, camber)` → coeff derivation now lives in `AirfoilAero.lei_poly_coeffs`; sections carry the resulting `(cl_coeffs, cd_coeffs, cm_coeffs)`. """ diff --git a/src/airfoil_aero/section_aero_gen.jl b/src/airfoil_aero/section_aero_gen.jl index d4a236a7..d8dfb161 100644 --- a/src/airfoil_aero/section_aero_gen.jl +++ b/src/airfoil_aero/section_aero_gen.jl @@ -98,7 +98,8 @@ Write a per-node aero table (`Cp` or `cf`, shaped `n_node × n_alpha × n_delta` per `(alpha, delta)` with angles in degrees. The file suffix picks the format: `.arrow` (columns `alpha`, `delta` and a per-row list column `values`, an order of magnitude faster to load), anything else a human-readable CSV with header `alpha, delta, n0, n1, …` -(node columns in the contour node order). [`read_node_table`](@ref) reads both. +(node columns in the contour node order). +[`read_node_table`](@ref VortexStepMethod.read_node_table) reads both. """ function write_node_table(path::AbstractString, aero::SectionAero, values) grid = [(jd, ia) for jd in eachindex(aero.delta_range) diff --git a/src/section_aero.jl b/src/section_aero.jl index d821bff8..9e4b3909 100644 --- a/src/section_aero.jl +++ b/src/section_aero.jl @@ -79,7 +79,8 @@ end Filename tag for a non-zero trailing-edge deflection `delta` [rad], e.g. `d5`, `dm3` (−3°), `d2p5` (2.5°). Uses millidegree precision (negatives as `m`, the decimal point as `p`) so sub-degree deflections get distinct `.dat` files instead of colliding. -Shared by [`write_section_aero`](@ref) and [`read_section_aero`](@ref). +Shared by [`write_section_aero`](@ref VortexStepMethod.AirfoilAero.write_section_aero) +and [`read_section_aero`](@ref). """ function delta_suffix(delta) deg = round(rad2deg(delta); digits=3) diff --git a/src/solver.jl b/src/solver.jl index b3e36d03..9d5ff060 100644 --- a/src/solver.jl +++ b/src/solver.jl @@ -616,6 +616,17 @@ end end end +""" + solve_base!(solver::Solver, body_aero::BodyAerodynamics, gamma_distribution=nothing; + log=false) + +Converge the circulation distribution and leave it in `solver.lr.gamma_new`, without +turning it into forces. Fills the solver's panel arrays, builds the AIC matrices, +starts from `gamma_distribution` (or an elliptical/zero guess when it is `nothing` or +`solver.use_gamma_prev` is false) and iterates; a `LOOP` solver that fails to converge +retries once with half the relaxation factor. The circulation half of [`solve!`](@ref), +paired with [`calc_forces!`](@ref). Returns `nothing`. +""" function solve_base!(solver::Solver{P, U, T}, body_aero::BodyAerodynamics, gamma_distribution=nothing; log=false) where {P, U, T} diff --git a/src/surfplan_adapter/SurfplanAdapter.jl b/src/surfplan_adapter/SurfplanAdapter.jl index 6424a512..e1cbe40c 100644 --- a/src/surfplan_adapter/SurfplanAdapter.jl +++ b/src/surfplan_adapter/SurfplanAdapter.jl @@ -13,14 +13,16 @@ using ..AirfoilAero: shrink_wrap, ShrinkWrap, read_dat_coordinates, Generate a pressure-ready `geometry.yaml` (per-node surface `cp`/`cf` tables plus polars) from a SurfplanAdapter aero export, so the wing can be flown with the `AeroPressure` continuous coupling instead of only integrated polars. The Surfplan -counterpart of [`obj_to_yaml`](@ref): where `.obj` slices a mesh, this reads the +counterpart of [`obj_to_yaml`](@ref VortexStepMethod.ObjAdapter.obj_to_yaml): where +`.obj` slices a mesh, this reads the already-clean per-rib airfoil `.dat` profiles the Python adapter exported. Reads `adapter_dir/aero_geometry.yaml` for each section's leading/trailing-edge placement (`wing_sections`) and each airfoil's `.dat` path (`wing_airfoils` `info_dict.dat_file_path`), shrink-wraps every unique profile, and runs the shared [`generate_airfoils`](@ref) core with `aero_solver` ([`NeuralFoilSolver`](@ref) by -default, [`XFoilSolver`](@ref) opt-in). Sections that share an airfoil generate its +default, [`XFoilSolver`](@ref VortexStepMethod.AirfoilAero.XFoilSolver) opt-in). +Sections that share an airfoil generate its tables once. `Re` defaults to the export's `wing_airfoils.reynolds`; `alpha_range` defaults to the full `-180:1:180` sweep rather than the export's narrow polar range. `table_format` writes the per-node surface tables as `:csv` (default, readable) or From b846387250dde2de29ae0d740dee80c92bac6acf Mon Sep 17 00:00:00 2001 From: Bart Date: Thu, 13 Aug 2026 09:47:52 +0200 Subject: [PATCH 6/6] Read Arrow tables from bytes so Windows can rewrite them MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Arrow.Table(path) memory-maps the file, and Windows refuses to unlink or overwrite a mapped file for the rest of the session — so regenerating a dataset, or deleting one, failed after anything had loaded it. Costs 5.6 ms on a 15 MiB table, against the 2.5 s the CSV path took. Co-Authored-By: Claude Opus 5 (1M context) --- src/section_aero.jl | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/src/section_aero.jl b/src/section_aero.jl index 9e4b3909..3e219101 100644 --- a/src/section_aero.jl +++ b/src/section_aero.jl @@ -118,7 +118,8 @@ Read a per-node aero table into radian `alpha`/`delta` vectors (one entry per ro """ function read_node_table(path::AbstractString) if endswith(String(path), ".arrow") - table = Arrow.Table(String(path)) + # bytes, not the mmap Arrow.Table(path) takes: Windows locks a mapped file + table = Arrow.Table(read(String(path))) values = Matrix{Float64}(undef, length(table.alpha), length(first(table.values))) for k in axes(values, 1) @inbounds values[k, :] .= table.values[k]