Skip to content

v2 selection: upper size bound 9, FLAGS cut ignores bit 16 - #448

Merged
cailmdaley merged 2 commits into
developfrom
feat/v2-size-bound-9
Oct 31, 2026
Merged

cailmdaley merged 2 commits into
developfrom
feat/v2-size-bound-9

Conversation

@cailmdaley

@cailmdaley cailmdaley commented Oct 30, 2026 •

Copy link
Copy Markdown
Collaborator

ShapePipe v2's upper size cut, T_gal/T_PSF < 3, drops the large galaxies v1 kept. In moment-ratio units it equals a radius-ratio cut of ~1.7. Those large galaxies carry high response at similar PSF coupling, so dropping them raises v2's leakage coefficient β and costs number density. This PR widens the bound to 9 and relaxes the SExtractor FLAGS cut to ignore bit 16 (incomplete aperture data). Bit 16 flags large galaxies preferentially, and with the bound at 9, selecting on it biases m in image sims.

Relation to v1. v1.4's ngmix T columns are Gaussian half-light radii in arcsec: the old ngmix fork labels its radius parameter T. v1's PSF FWHM, 2·T_PSF, has a median of 0.705″, against 0.700″ from HSM on the stars. v2's T is the second moment 2σ². So v1's gal_rel_size_max: 3 (commented "hlr / hlr_psf", which is correct for v1) was a radius ratio, and bound 9 in moment ratio is v1's own window (3² = 9). FLAGS is SExtractor's extraction flag in both versions, and bit 16 means the object's aperture data are incomplete or corrupted. No v1 selection ignored it: v1.X.6 (Paper II's v1.4.6.3) required FLAGS == 0, v1.X.10 <= 2 and v1.X.11 <= 3, and each of those drops bit 16. So ignoring bit 16 is new here. It is needed because bit 16 is set preferentially on the large galaxies the wider bound admits.

Selection (config/calibration/mask_v2.0.yaml):

  • gal_rel_size_max: 3 → 9
  • FLAGS: FLAGS <= 2 → (FLAGS & ~16) <= 2, i.e. keep FLAGS ∈ {0, 1, 2, 16, 17, 18}

Evidence (10k-tile snapshot, #425 calibration; image sims on the four #425 grids):

bound 3, FLAGS ≤ 2 bound 9, (FLAGS & ~16) ≤ 2
n_eff [arcmin⁻²] 5.53 6.95 (+26%)
response R 0.712 0.787
joint β 1.28 1.12
image-sim Δm vs production [10⁻³] — (+1.2 ± 2.5, +1.3 ± 2.4)
shear-weighted ⟨z⟩ shift — −0.0148 ± 0.0003

The FLAGS predicate is load-bearing. Bound 9 with plain FLAGS <= 2 fails the image-sim gate (Δm1 = −4.8 × 10⁻³). Dropping FLAGS entirely passes on the grids, but it would admit 5.3M real FLAGS==3 blends that the sims don't represent.

m vs size bound

Image-sim m against the upper bound for the three FLAGS policies (DES weights on top, unit weights below; shared-tile jackknife). Only m1 under plain FLAGS ≤ 2 moves.

mean z vs size bound

Shear-weighted ⟨z⟩ at bounds 3 and 9 for two photometry choices. The added large galaxies sit lower, at ⟨z⟩ ≈ 0.56.

Before v2 cosmology with this cut:

  • v2 needs its own n(z). Keeping the bound-3 n(z) biases S8 by about +2.5%. A separate effort is building the first multi-bin v2 n(z).
  • The DES weight table must be refit on the new selection. If a fixed des_weight_product is configured (Consistent shear response, weights and additive bias (DES case d) #425), it stays the bound-3 table until regenerated.
  • Open caveat: the m gate holds only for the sims' own population. Real data put more weight in large added galaxies than the sims do (54% of the added weight at RMS radius 5–7.5 px, against 42%). To see what that means for the data, we reweighted the sims to the data's joint size-ratio/SNR distribution and the radius-bin mass. Each galaxy keeps its weight across the metacal shear branches, the correct convention for a change of population. Under that reweighting, widening the bound shifts m by Δm(9−3) = (+5.3 ± 3.1, +5.9 ± 2.7) × 10⁻³, outside the 3 × 10⁻³ budget, and bound-3 production's own m moves by about +4 × 10⁻³. These are conditional estimates: the reweighting uses noisy observed sizes as population labels, and the data's joint radius–size–SNR distribution isn't available. A bound-9 m calibration therefore needs sims that match the data's size distribution. A tighter bound doesn't help: bound 5 shows no established m gain and costs 7% of n_eff.

Grammar: a dat cut can now carry ignore_bits: <int>, which clears those bits before comparing. It lives in masks.apply_condition, and every reader of dat cuts goes through it: Mask, scripts/masking.py, and the dashboard selection histogram, whose private copy of the comparison logic is gone. Mask.get_sign had ≤ and ≥ swapped in its LaTeX labels; that's fixed too.

— Claude on behalf of Cail

🤖 Generated with Claude Code

https://claude.ai/code/session_01JK7o9F5Ad2qR25uH6YFZZg

@cailmdaley
cailmdaley merged commit 2471843 into develop Oct 31, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant