Conversation
…ransform dispatch Full-codebase audit. Three correctness bugs, three performance defects, and a public API that controlled nothing. Correctness: * analysis_turbo/synthesis_turbo iterated a bare 0:mmax instead of striding by cfg.mres, so for an mres>1 config they populated (and consumed) coefficient columns the transform has no storage for. They disagreed with analysis / synthesis by 0.70 at lmax=6/mres=2 while documenting "same output". Both now stride via cached_m_order, and a @threads guard stops them nesting a static loop when called from an outer threaded region. * The SH_Yrotate, shtns_rotation_apply_cplx and shtns_rotation_apply_real rrules (and the Zygote SH_Yrotate adjoint) captured the primal INPUT and read it lazily in the pullback. All of them support an in-place primal (Rlm === Qlm), which overwrites exactly those coefficients, so the angle gradient came back wrong: in-place dalpha was -1.04 against a finite- difference 9.86. They now snapshot before the primal, matching the fix already applied to SH_Zrotate. * test_mres_diagnostics built Sclean/Tclean with similar() rather than zeros(). Run alone the pages are freshly mmap'd and zero; run late in the suite the allocator recycles them and the test fails. The failure is deceptive: the arrays compare == exactly and norm(a-b) is 0.0, but isapprox is false because norm() of a matrix carrying denormal garbage returns NaN. Performance (lmax=64, nlat=66, single thread, tables enabled): * SHTPlan never consulted cfg.NP_tables -- analysis!/synthesis! hardwired the on-the-fly kernel -- and the vector pair walked the m/theta loop twice, once per component. All four now delegate to the shared orchestrators, which also restores the function barrier that _internal_coefficients' small Union was defeating. analysis! 0.925 -> 0.133 ms, synthesis! 0.916 -> 0.123, analysis_sphtor! 4.91 -> 0.488, synthesis_sphtor! 2.06 -> 0.357. Output is bit-identical to the cfg form on the complex path. * Plm_norm_and_dPdtheta_row! and Plm_norm_dPdtheta_over_sinth_row! ran the serial recurrence twice per row, once into P and again into Pbuf, whose first lmax+1 entries are the same row. 2.5 -> 1.5 us at lmax=200. Dead code: * _get_or_plan had no call sites, so the parallel extension's FFT plan cache was inert -- yet set_fft_plan_cache!/enable_/disable_/fft_plan_cache_enabled were exported, asserted in two testsets, and documented as a tuning knob. Removed the dead machinery and repointed the public API at the cache in fftutils.jl that every transform actually uses, serial and distributed alike. Added a size cap and renamed the environment variable to SHTNSKIT_FFT_PLAN_CACHE, keeping the old name as an alias. Rotation sign conventions: * SH_Zrotate used exp(+i m alpha) while the Wigner engine behind shtns_rotation_apply_real -- and therefore SH_Yrotate -- used exp(-i m alpha), so the two were different rotations of the same field. SH_Xrotate90 had the twin defect, using ZYZ(pi/2, pi/2, -pi/2), which is Rx(-pi/2). Both now use the active convention, verified against a real phi shift on the FFT grid, the ZYZ engine, and the distributed twins. Documented in CHANGELOG.md with verified porting recipes. Also: renamed the `im` parameter to `mval` in the seven remaining mode-limited entry points, since `im` shadows the imaginary unit and `1.0im` is literal juxtaposition; gave shtns_rotation_apply_real an mres-aware error; documented that the :regular and :regular_poles quadratures are only algebraically accurate, with measured round-trip errors. Pre-existing, not caused by this change: 13 @allocated budget tests failed whenever nthreads > 1, verified by stashing and re-running. The cause is @threads task-spawn overhead, a size-independent constant. Budgets now carry a per-thread slack, so the tight check remains the single-threaded run. Serial suite 67789 passing, 0 failing, at 1 and at 4 threads (was 67693/3). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Dbj4TtuW11gke9JwSNs7Nz
…D rules Acts on a second review pass. Each finding was reproduced with a runnable script before being fixed and re-verified after; the ones I could not verify are listed at the end rather than changed blind. Rotations: * shtns_rotation_set_angle_axis used the beta~0 formula at BOTH degenerate poles. At beta~pi the observable Euler combination is alpha-gamma read off the negated first column, so a 180 degree turn about x came back as ZYZ(0,pi,0) -- exactly Ry(pi). Relative error against the true Rx(pi) was 1.33 and against Ry(pi) it was 0. Only exact half-turns were affected. * A Wigner-d rotation through a general beta couples every |m'| <= l, so on an mmax < lmax layout the |m'| > mmax components had nowhere to go and were silently dropped: 14.8% of the field energy at lmax=8/mmax=5, 24.0% at mmax=3. Order-mixing rotations now raise, mirroring the existing mres>1 restriction. The two non-mixing angles stay legal at any mmax -- beta=0 is diagonal and beta=pi is anti-diagonal -- so pure Z-rotations are unaffected. Scaling: * synthesis scales its Fourier bins by phi_inv_scale(cfg) and the inverse FFT divides by nlon, a net factor of 1 under :dft but 1/2pi under :quad. All eight direct evaluators (synthesis_point, the two axisym pairs, and the five in local.jl) applied no factor, so under :quad each disagreed with the grid it claims to sample by exactly 2pi. Default :dft behaviour is unchanged. Configuration: * Assigning cfg.lmax left size(cfg.Nlm) at the old dimensions while the transforms index it as (lmax+1, mmax+1) under @inbounds -- an out-of-bounds read of a live array. lmax/mmax/mres now rebuild the derived spectral layout and drop the stale Legendre tables; nlat/nlon/grid_type/nlm/li/mi/nspat raise, since the quadrature cannot be regenerated in place without knowing the grid type. * The exported SHTConfig(; ...) constructor validated nothing, so a hand-built config could violate nlon >= 2*mmax+1 and then silently synthesize an all-zero field for any mode it could not resolve. It now enforces what the create_* helpers always have. * analysis_sphtor_ml and synthesis_sphtor_ml returned +/-Inf for an out-of-range order where their scalar twins raised; energy_scalar and energy_vector indexed under @inbounds with no size check. Both now validate. Automatic differentiation: * The QST family had no rrule at all, so differentiating a QST pipeline fell through to source tracing and crashed inside FFTW. synthesis_qst and analysis_qst now compose the existing scalar and sphtor adjoints, checked against finite differences to eight digits. * rrules declaring fewer keyword arguments than their primal were skipped entirely the moment a caller passed one, even at its default. analysis accepted none; synthesis and synthesis_sphtor omitted use_rfft. Distributed: * DistributedSpectralPlan2D attached a GC finalizer that called close, which frees the plan's sub-communicators through MPI_Comm_free -- a collective. Garbage collection is rank-local and nondeterministic, so ranks could enter it at divergent points. Cleanup is explicit only now, and close documents the contract. Also: im_from_lm was bounded by lmax rather than mmax, so on an mmax < lmax layout a past-the-end index resolved to an order the config does not store; it takes an optional mmax keyword now. The two vorticity inverse-problem gradients now stride by mres like every other diagnostic (latent only, since analysis pre-zeros its output). Not changed, deliberately: * analysis does not honour phi_scale, so under :quad analysis(synthesis(alm)) is alm/2pi. Making the pair mutually inverse means renormalising the whole analysis family and its adjoints, which is a design decision with a real blast radius rather than a local fix. * _dist_analysis_2d_aligned omits the _keep_one_phi_partner! dedup its three siblings perform, and the 2D synthesis reduction carries an unchecked assumption about matching theta ranges. Both are MPI paths with no MPI in this environment; a wrong guess silently doubles or zeroes results, so they are reported rather than patched. * set_batch_size!/howmany and the twelve-name padding API are exported but unused by the transforms. Removing exported API is a breaking change that needs a maintainer decision. Serial suite 67888 passing, 0 failing, at 1 and at 4 threads (was 67789). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Dbj4TtuW11gke9JwSNs7Nz
…adding API Closes the three items left open by the previous pass. MPI became testable this round (MPICH via MPI.jl), so the distributed findings are settled with evidence rather than reported. analysis now inverts synthesis under every phi_scale: synthesis scaled its Fourier bins by phi_inv_scale(cfg) while analysis always applied a fixed cphi, so the two halves of the pair disagreed about the convention -- under :quad, analysis(synthesis(alm)) came back as alm/2pi exactly. Analysis and every sibling (sphtor, batch, packed-complex, axisym, mode-limited, the distributed and transpose paths, the GPU kernels, and both analysis adjoints) now carry cphi/sigma with sigma = phi_inv_scale/nlon. Default :dft behaviour is bit-identical, since sigma is 1 there. The vorticity inverse-problem gradients used analysis as a stand-in for the synthesis adjoint with a compensating factor; that factor picks up sigma a second time now, which the existing :quad finite-difference tests confirm. An unset phi_scale no longer means :quad for non-Gauss grids. The :auto fallback keyed on grid_type, so a regular grid built through the exported keyword constructor (which defaulted to :auto) disagreed by 2pi with the same grid from create_regular_config. Both constructors emit :dft, so unset now resolves to :dft. 2D distributed alignment: dist_synthesis_distributed_2d_optimized and dist_analysis_distributed_2d(assume_aligned=true) reduce within l_comm rather than the full communicator, which is valid only when the spatial theta split lines up with the spectral l split. Measured on 4 ranks with a 2x2 process grid, the synthesis was off by max|err| = 84.4 against a field of O(10) and the analysis by a relative 1.15 -- silently, with no warning. Both now validate collectively and raise; where the precondition holds they are unchanged and agree with the safe paths to 1.4e-14. New test/parallel/test_mpi_2d_alignment.jl pins both directions and runs in CI on 4 ranks. The related claim that _dist_analysis_2d_aligned is missing the _keep_one_phi_partner! dedup its siblings perform was investigated and is NOT a defect: the siblings reduce over the full communicator, where phi-partners genuinely duplicate a theta slab, while this path reduces within l_comm, where the (now enforced) alignment precondition already guarantees distinct slabs. Verified correct to 2.5e-16 on every decomposition where it is reachable. Padding API: allocate_padded_spatial returns nlat_padded >= nlat rows while every transform requires exactly nlat, so nothing could consume the buffer it produced. New exported spatial_view(cfg, A) bridges the two for both the 2D and batched shapes. It is not a workaround that discards the padding: the SubArray keeps the padded column stride (26 vs 18 at lmax=16), which is the cache-conflict avoidance the padding exists for, and results are bit-identical. set_batch_size!/howmany/spec_dist are genuinely advisory -- the batch entry points take the field count from size(fields, 3). Left in place for SHTns interoperability, but the docstring no longer claims the transforms consult them, which it did. Also corrected docs/phi_scaling.md and docs/src/performance_tips.md, which both described behaviour the code never had: they claimed regular grids default to :quad and showed a create_regular_config(...; phi_scale=:quad) call, but those constructors take no such keyword (the call raises) and both set :dft. Serial suite 67907 passing, 0 failing, at 1 and at 4 threads (was 67888). All 14 MPI test files pass on 4 ranks. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Dbj4TtuW11gke9JwSNs7Nz
Member
Author
|
Superseded by #53, which ports this work onto current This branch is 108 commits behind
Everything else either landed in #53 or was already fixed on Note on this branch's red CIThe Closing as superseded. Not merging, per the title. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Warning
DO NOT MERGE. This branch's merge-base with
mainis 2026-08-11 andmainhas 108 commits since. Merging would be
294 files changed, +12,811 / −46,193— it would delete roughly 46k lines of upstream work. This PR exists as a
reviewable record of an audit; the fixes need re-applying on top of current
mainbefore anything lands.Review commit
23f3f4cconly. The three commits beneath it are pre-existingbranch history.
What this is
A full read of
src/andext/(~21k lines). Every finding below was reproducedwith a runnable script before being fixed, and re-verified after.
Serial suite: 67693 passing / 3 failing → 67789 / 0, at both 1 and 4 threads.
Wall clock 2m18s → 50s.
Correctness
analysis_turbo/synthesis_turboignoredcfg.mres—ext/SHTnsKitLoopVecExt.jlBoth looped a bare
0:mmaxwhile the core usescached_m_order(cfg), so on anmres > 1config they populated — and consumed — coefficient columns the transformhas no storage for. At
lmax=6, mres=2,max|analysis − analysis_turbo|was 0.70;the docstrings claim "same API and output as
SHTnsKit.analysis". Now 6.2e-17.Also added a
@threadsguard so a turbo call from inside an outer threaded regionno longer nests a
:staticloop.Rotation AD angle gradients read the primal input lazily — AdvancedAD + Zygote
SH_Yrotate,shtns_rotation_apply_cplxandshtns_rotation_apply_realcapturedQlm/Zlmand read it inside the pullback. All three support an in-place primal(
Rlm === Qlm), which overwrites exactly those coefficients:SH_Yrotatedαapply_cplxdβapply_realdβFixed by snapshotting before the primal — the same fix already applied to
SH_Zrotateon this branch, which its siblings never received.test/serial/test_mres_diagnostics.jlwas flaky — builtSclean/Tcleanwithsimilar(). Alone it passes (fresh pages are zero); late in the suite the allocatorrecycles them and it fails. Deceptive failure: the arrays compare
==exactly andnorm(a-b) == 0.0, butisapproxisfalsebecausenorm()of a matrix carryingdenormals returns
NaN.Performance (lmax=64, nlat=66, 1 thread, tables enabled)
SHTPlannever consultedcfg.NP_tables, and its vector pair walked the m/θ looptwice, once per component. All four planned transforms now delegate to the shared
orchestrators — which also restores the function barrier that
_internal_coefficients'small
Unionwas defeating.cfgformanalysis!synthesis!analysis_sphtor!synthesis_sphtor!Output is bit-identical to the
cfgform on the complex path (1e-14 on rfft), acrossall 8 combinations of tables × rfft ×
robert_form.Plm_norm_and_dPdtheta_row!/Plm_norm_dPdtheta_over_sinth_row!ran the serialrecurrence twice per row — once into
P, again intoPbuf, whose firstlmax+1entries are the same row (verified bit-identical). 2.5 → 1.5 µs at lmax=200.
Dead code with a live public face
_get_or_planhad zero call sites, so the parallel extension's FFT plan cache wasinert — yet
set_fft_plan_cache!/enable_/disable_/fft_plan_cache_enabledare exported, asserted in two testsets, and documented as a tuning knob
(
SHTNSKIT_CACHE_PENCILFFTS). Removed the dead machinery (307 lines) and repointedthe public API at the cache in
fftutils.jlthat every transform actually uses,serial and distributed alike. Added a size cap; env var is now
SHTNSKIT_FFT_PLAN_CACHE, old name kept as an alias.Rotation sign conventions
SH_Zrotateusedexp(+imα)while the Wigner engine behindshtns_rotation_apply_real— and thereforeSH_Yrotate— usedexp(-imα), so thetwo were different rotations of the same field.
SH_Xrotate90had the twindefect:
ZYZ(π/2, π/2, -π/2), which isRx(-π/2), the inverse of its name.Both now use the active convention, verified against three independent references:
cis(-mα)cis(+mα)ZYZ(α,0,0)/ZYZ(0,0,α)dist_SH_ZrotatetwinsDocumented in
CHANGELOG.mdwith porting recipes, each verified rather than reasoned:negating the angle reproduces the old
SH_Zrotateexactly (0.0); three forward turns(2.3e-15) or
ZXZ(0,-π/2,0)(4.8e-16) reproduce the oldSH_Xrotate90.Note
mainis at v2.0.2 and still shipscis(+mα), so in any port this becomes abreaking change against 2.0.2, not the unreleased-v2.0.0 note written here. The
CHANGELOG entry needs rewriting accordingly.
Also
imparameter tomvalin the seven remaining*_mlentry points —imshadows the imaginary unit and1.0imis literal juxtaposition, the exact trapthat once broke
synthesis_sphtor_ml's S/T coupling.shtns_rotation_apply_realnow explainsmres > 1instead of a bare size mismatch.:regular/:regular_polesquadrature is only algebraicallyaccurate (0.11 relative round-trip error at
nlat = lmax+2, still 0.005 at2.5(lmax+1), vs 1e-15 for:gauss/:driscoll_healy) —create_config's defaultnlat = lmax+2sits at the worst end.@allocatedbudget tests failed atnthreads > 1, verified by stashing and re-running on the original code. Cause is@threadstask-spawn overhead, a size-independent constant. Budgets now carry aper-thread slack, so the tight check remains the single-threaded run.
Still applicable to
mainEvery defect above except the
test_mres_diagnosticsone (that file doesn't exist onmain) is still present onorigin/maintoday — turbo still loops0:mmaxat 5sites, the rotation rrules still take no snapshot,
src/plan.jlstill has zeroNP_tablesreferences, andSH_Zrotatestill disagrees with the Wigner engine.Second pass — findings from a parallel review agent
Fourteen further findings were raised by a separate review of this diff. I
verified each one myself before acting; the outcomes:
Fixed (all reproduced first):
shtns_rotation_set_angle_axishalf-turnsZYZ(0,π,0)= exactlyRy(π)(error vs trueRx(π): 1.33; vsRy(π): 0.0)mmax < lmax|m'| > mmaxcomponents a Wigner-d rotation generates — 14.8 % of field energy lost atlmax=8/mmax=5, 24.0 % atmmax=3. Now raises;β ≡ 0andβ ≡ πstay legal since neither mixes ordersphi_scale:quadeach disagreed with the grid it samples by exactly 2π.:dftunchangedcfg.lmax = 10leftsize(Nlm) == (7,7)@inbounds.lmax/mmax/mresnow rebuild the layout; the rest raiseSHTConfig(; …)validated nothingnlon < 2*mmax+1built fine, then silently synthesized an all-zero fieldanalysis_sphtor_mlreturned±Infenergy_scalar/energy_vector@inboundswith no size check — silently wrong number for a mis-sized spectrumrrulerrules narrower than their primalMPI_Comm_freeim_from_lmbounded bylmaxnotmmaxmresDeliberately not changed, and why:
analysisdoes not honourphi_scale, so under:quadanalysis(synthesis(alm)) == alm/2πexactly. Confirmed. Making the pairmutually inverse means renormalising the whole
analysisfamily and itsadjoints — a design decision with real blast radius, not a local fix.
_dist_analysis_2d_alignedomits the_keep_one_phi_partner!dedup itsthree siblings perform, and the 2D synthesis reduction carries a self-documented
unchecked assumption about matching θ ranges. Both are MPI paths and there is no
MPI in this environment; a wrong guess silently doubles or zeroes results.
set_batch_size!/howmanyand the twelve-name padding API are exported butunused by the transforms. Removing exported API is breaking and needs a
maintainer decision.
One finding the reviewing agent raised and then retracted itself: a NaN it saw
came from
similar()in its own probe script — the same defect fixed intest_mres_diagnostics.jlhere.Suite after this pass: 67888 passing / 0 failing at 1 and at 4 threads.
Third pass — the three items previously left open
MPI became testable this round (MPICH through MPI.jl), so the distributed items
are settled with measurements rather than reported.
analysisnow honoursphi_scale.synthesisscaled its bins byphi_inv_scale(cfg)whileanalysisapplied a fixedcphi, so under:quadaround trip returned
alm/2πexactly. Analysis and every sibling — sphtor, batch,packed-complex, axisym, mode-limited, the distributed/transpose paths, the GPU
kernels, and both analysis adjoints — now carry
cphi/σ,σ = phi_inv_scale/nlon.Round trip is now exactly 1.0 in both modes;
:dftoutput is bit-identicalsince
σ = 1there. The vorticity gradients' compensation picks upσa secondtime as a result, which your existing
:quadfinite-difference tests confirm.Also: an unset
phi_scaleno longer means:quadfor non-Gauss grids. The:autofallback keyed ongrid_type, so a regular grid built through theexported keyword constructor (defaulting to
:auto) disagreed by 2π with thesame grid from
create_regular_config.2D distributed alignment — one real defect, one false positive.
Measured on 4 ranks, 2×2 process grid, where the alignment check reports
false:dist_synthesis_distributed_2d_optimizedmax|err| = 84.4(field of O(10))dist_analysis_distributed_2d(assume_aligned=true)Both now validate collectively. New
test/parallel/test_mpi_2d_alignment.jlpinsboth directions and runs in CI on 4 ranks.
The companion claim — that
_dist_analysis_2d_alignedis missing the_keep_one_phi_partner!dedup its three siblings perform — is not a defect.The siblings reduce over the full communicator, where φ-partners genuinely
duplicate a θ slab; this path reduces within
l_comm, where the alignmentprecondition already guarantees distinct slabs. Verified correct to 2.5e-16 on
every decomposition where it is reachable.
The padding API had no usable path into a transform.
allocate_padded_spatialreturnsnlat_padded ≥ nlatrows; every transformdemands exactly
nlat. New exportedspatial_view(cfg, A)bridges the two (2D andbatched). It does not discard the padding — the
SubArraykeeps the padded columnstride (26 vs 18 at
lmax=16), which is the cache-conflict avoidance the paddingexists for — and results are bit-identical.
set_batch_size!/howmany/spec_distturned out to be genuinely advisory: thebatch entry points take the count from
size(fields, 3). Left in place for SHTnsinteroperability, with a docstring that no longer claims the transforms consult
them.
Two documentation files described behaviour the code never had: both claimed
regular grids default to
:quad, and one showedcreate_regular_config(...; phi_scale=:quad)— a call that raises, since thoseconstructors take no such keyword. Corrected.
Suite after this pass: 67907 passing / 0 failing at 1 and 4 threads, and
14/14 MPI test files passing on 4 ranks.
Not covered
The GPU extension is untestable here (no CUDA) — its changes were read, not run.
MPI is exercised: all 14 files under
test/parallel/pass on 4 ranks, includingthe new 2D alignment contracts.
🤖 Generated with Claude Code
https://claude.ai/code/session_01Dbj4TtuW11gke9JwSNs7Nz