Skip to content

One catalogue-masking path for PSF stars and galaxies (manual-mask bit 8 via sky maps) - #457

Merged
cailmdaley merged 1 commit into
developfrom
feat/psf-star-manual-mask
Nov 1, 2026
Merged

cailmdaley merged 1 commit into
developfrom
feat/psf-star-manual-mask

Conversation

@cailmdaley

@cailmdaley cailmdaley commented Oct 30, 2026 •

Copy link
Copy Markdown
Collaborator

Claude (Opus) on behalf of Cail.

PSF stars and galaxies are now masked by one code path. A mask config is evaluated by one function, masks.catalogue_selection. The star selection behind ρ/τ and the galaxy masking both go through it. A cut whose column a catalogue lacks is read from a sky map the config names. So one predicate, MASK_8_Manual equal False, masks galaxies from their column and PSF stars from the DR6 bit-8 map. There is no ρ/τ-specific veto.

Why stars need it: in run 1, M33's HII regions and clusters enter PSF star selection and dominate the first-bin ρ statistics. This drives the α/η sign flip in #452. ShapePipe's manual mask (bit 8) removes 1,024 of M33's 1,025 oversize detections within 1° and costs 0.5% of PSF stars survey-wide (CosmoStat/shapepipe#993). Galaxies already carry the bit as a column. The v2 star catalogue has no MASK_* columns, so stars take the bit from the map.

Radial profiles around M33, M101, M13, M92, Draco and Ursa Minor: oversize fraction, normal-star ellipticity and size residuals (shapepipe#993)

What changes

  • maps: in a mask config maps a column name to a boolean healsparse map, True = masked. get_masks_from_config reads a column from the catalogue when it's there and from the map otherwise (MapColumns). mask_v2.0.yaml maps MASK_8_Manual to the DR6 bit-8 map. Galaxy catalogues have the column, so their selection is unchanged.
  • catalogue_selection(config, cat, cuts=None) is the AND of the config's cuts. scripts/masking.py:apply_masks and the PSF star selection both call it.
  • A cat_config psf block names its mask config:
    psf:
      mask_config: ../config/calibration/mask_v2.0.yaml  # relative to the cat_config file, or absolute
      mask_cuts: [MASK_8_Manual]                       # stars take only the manual-mask cut
    Without mask_config, star selection is unchanged, so v1 entries are unaffected.
  • Maps are fetched from CANFAR, not read from an ACL-locked copy. A map named by an arc:/vos: URL is fetched once with vcp into ~/.cache/sp_validation/masks/<remote path> ($SP_VALIDATION_MASK_CACHE overrides) and read from there. The canonical bit-8 map is arc:home/mhudson/masks/mask_ugriz_nside131072_n8.hsp (55 MB, 28 Sep 2026). vcp needs a CADC proxy certificate (~/.ssl/cadcproxy.pem); the image ships vcp. A map path can also be a plain local file.

Checks

  • On the real map, through catalogue_selection: every point within 0.2° of M33 is masked, against 0.5% in a 1.0–1.5° control ring. The 28 Sep map and the 6 Aug copy in old_masks/ differ by 429 pixels (≈10⁻⁴ deg²).
  • Tests: a star inside the map is dropped and its neighbours kept; a galaxy column wins over the map; mask_v2.0.yaml's selection equals evaluating each cut directly; a remote map is fetched once into the cache.
  • Fast container suite: 648 pass. The 6 failures are the known candide ones (config paths, the HDF5 star loader, the cached-plot DAG), and they fail identically without this change. Workflow DAG tests: 15 pass.
  • A galaxy catalogue that lacked MASK_8_Manual would now take the bit from the map instead of raising. v2 galaxy catalogues carry the column, and v1 configs name no maps.

Refs #452

🤖 Generated with Claude Code

https://claude.ai/code/session_01JK7o9GrF3X5AAqQo1X5UmX

A mask config's optional `maps` names a sky map per mask column (path or
arc:/vos: URL, fetched once with vcp into ~/.cache/sp_validation/masks).
get_masks_from_config reads a cut's column from the catalogue, or from
the map at the objects' positions when the catalogue lacks it.
catalogue_selection, the AND of those cuts, is what scripts/masking.py
and the rho/tau star selection both evaluate.

A cat_config psf block names its mask config (mask_config, relative to
the catalogue config) and the cuts stars take (mask_cuts). ShapePipe's
star catalogue has no MASK_* columns, so with mask_v2.0.yaml's
MASK_8_Manual -> DR6 bit-8 map, stars inside the manual mask (M33's
HII regions and clusters, shapepipe#993) are dropped under the same cut
galaxies take from their column.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JK7o9GrF3X5AAqQo1X5UmX
@cailmdaley
cailmdaley force-pushed the feat/psf-star-manual-mask branch from e91f741 to eb1b709 Compare October 31, 2026 00:32
@cailmdaley cailmdaley changed the title ρ/τ: veto PSF stars by a configurable mask (manual-mask bit 8) One catalogue-masking path for PSF stars and galaxies (manual-mask bit 8 via sky maps) Oct 31, 2026
@cailmdaley
cailmdaley merged commit c362237 into develop Nov 1, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant