PSF-leakage fit fails on the full v2 area: three causes, and is β(T_PSF) physical? #452

Claude (Opus) on behalf of Cail.

Where this fits. This issue is the diagnosis. It found three problems with the PSF-leakage fit on the full UNIONS v2 area and leaves one open science question, at the end. The fixes live elsewhere:

• One catalogue-masking path for PSF stars and galaxies (manual-mask bit 8 via sky maps) #457 (merged) removes contaminated PSF stars.
• PSF leakage: joint errors-in-variables ρ/τ fit by default #454 replaces the fit's estimator and adds the β(T_PSF) model as an option.
• ρ statistics leave PSF/star fields uncentred while τ centres galaxies, biasing α shear_psf_leakage#58 is a problem in the ρ/τ statistics themselves: α's template didn't match how leakage enters the galaxies. A known-answer test on mocks found it while PSF leakage: joint errors-in-variables ρ/τ fit by default #454 was being validated. Centre selected galaxies and measure galaxy-position alpha templates shear_psf_leakage#59 fixes it, and PSF leakage: joint errors-in-variables ρ/τ fit by default #454 adopts the fix.

Why we looked

cosmo_val fits the ρ/τ leakage model, with α, β and η on τ0/τ2/τ5 over 1–250′. From the fit it builds ξ_sys, the PSF contribution to ξ±. That only makes sense if the model fits.

• On the v1.4 footprint (~10k tiles, Papers I–V) the fit gave χ² 74/57.
• On the full-area v2 catalogue (ShapePipe v2 on DR6, 20,515 tiles) it gives 171/57.
• α and η also came out negative, which is unphysical.

So the v2 ξ_sys can't be trusted until we understand why. We found three causes.

Three causes

1. Contaminated PSF stars (fixed by One catalogue-masking path for PSF stars and galaxies (manual-mask bit 8 via sky maps) #457). One jackknife cell, at M33, carries more than 99% of the first-bin variance of ρ1, ρ3 and ρ4: its star clusters and HII regions enter the PSF star sample. Under the fixed-ρ fit, deleting that cell moves (α, β, η) from (−0.001, 0.81, −1.04) to (+0.027, 1.25, +3.22), so the negative α and η come mostly from this. A star-only scan adds M101, M13 and M92. One catalogue-masking path for PSF stars and galaxies (manual-mask bit 8 via sky maps) #457 vetoes PSF stars that carry ShapePipe's manual-mask bit (bit 8), which covers these objects (PSF star selection: apply the manual mask; M33 shows outlier rejection fails where contaminants dominate shapepipe#993).
2. The estimator (fixed by PSF leakage: joint errors-in-variables ρ/τ fit by default #454). lsq treats the measured ρ as exact and ignores the ρ–τ cross-covariance. At small θ, the noise in ρ, propagated through the model, is 100–500× the τ variance. A joint errors-in-variables (EIV) fit handles this: it fits the true ρ alongside (α, β, η) under one jackknife covariance of ρ and τ together. Dropping the cross term from that fit fakes a pass, 69.4/57 instead of 78.0/57.
3. β grows with PSF size (the open question below). Let β = β₀ + β₁u, with u = T_PSF/⟨T_PSF⟩ − 1. The fit gives β₁ ≈ 1.9 ± 0.3, with Δχ² 8–22 for one parameter. The slope is stable across 200, 400 and 800 patches, across size bounds 3 and 9, and with θ ≤ 60′ cuts. A step model gives β = 0.04 ± 0.27 in the best-seeing quartile and 2.38 ± 0.22 elsewhere. Splitting the sample into the north and south galactic caps, whose seeing differs, improves the fit far more than splitting old from new area: χ²/dof 2.16 → 1.44, against 2.18 → 2.13.

Result. The configuration is size bound 9 (gal_rel_size_max 9), the four-object star mask, joint EIV and β(T_PSF). It gives 64.6/56 (PTE 0.20) at 400 patches and 70.7/56 (PTE 0.089) at 200. Constant β gives 78.0/57 (PTE 0.034).

Caveat: the jackknife covariance doesn't converge. The PSF-error field is coherent beyond 5°.

• Merging cells: grouping jackknife cells into contiguous groups keeps raising the ρ variance. First-bin ρ1 rises 2.6×, 3.8× and 6.4× at 200, 100 and 50 groups. Random groups of the same sizes stay at 1.0×.
• Where it sits: the influence concentrates on the NGC east edge (RA 255–285°, Dec 30–60°). That is the smk_early campaign, observed earliest.
• Consequence: absolute PTEs depend on the patch count, at about the 2× level. The coefficients and Δχ² are stable. Settling the fit quality needs simulations with campaign-scale PSF-error modes.

Open question: is β(T_PSF) physical?

Should β(T_PSF) be the fiducial ξ_sys model?

• The prediction: leakage that scales as β ∝ T_PSF/T_gal predicts β₁ ≈ β₀.
• The fit: in the slope fit (bound 9, masked, joint EIV, 400 patches), β₀ = 1.37 ± 0.10 and β₁ = 1.98 ± 0.37. So β₁ − β₀ = 0.61 ± 0.33, with their correlation of 0.47 included: consistent with proportional at 1.8σ, leaning steeper.
• Constant β, for comparison: 1.28 ± 0.10 on the same selection, and about 1.64 on the full-area clean sample.
• Subsets: the step model's β ≈ 0 in the best-seeing quartile is steeper than proportional. Fitting the best-seeing subset directly gives β = 1.37 ± 0.12, between the two.

Proposal: make β(T_PSF) the fiducial, with α and η constant, and treat the difference from constant β as a systematic. That adds u·δe templates to ξ_sys. #454 measures their auto- and cross-correlations with e_PSF, δe and the size term, behind an option that is off by default.

Figures

M33 drives the ρ3/ρ4 swings; masking it alone makes them smooth:

τ residuals on bound 9 at 400 patches. The β slope (orange) removes the coherent −2 to −4.6σ τ2 miss from 8′ to 100′. An α slope (green) is not supported:

Leave-one-cell-out influence after masking, concentrated on the NGC east edge (smk_early):

The full report (the χ² ladder and the alternatives ruled out) is available from Cail. Only ρ, τ and α/β/η were used; no cosmology was unblinded.