Skip to content

compiler: Block strided subsampling and align SubGrids to global indices - #3106

Open
mloubout wants to merge 3 commits into
mainfrom
subsampling-blocking-mpi
Open

mloubout wants to merge 3 commits into
mainfrom
subsampling-blocking-mpi

Conversation

@mloubout

@mloubout mloubout commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

Fix blocking with strided subdim.

Fix MPI subdim start point

Fix Dimensions used as expressions under MPI, e.g. Eq(u, x + y).

Fixes #2214

@mloubout mloubout added compiler API api (symbolics, types, ...) labels Nov 6, 2026
@codecov

codecov Bot commented Nov 6, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 67.73163% with 101 lines in your changes missing coverage. Please review.
✅ Project coverage is 84.42%. Comparing base (22830c8) to head (fb0ea5d).

Files with missing lines Patch % Lines
tests/test_subsampling.py 60.75% 73 Missing ⚠️
tests/test_mpi.py 37.14% 22 Missing ⚠️
devito/types/dimension.py 88.57% 4 Missing ⚠️
devito/types/grid.py 90.47% 1 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3106      +/-   ##
==========================================
- Coverage   84.52%   84.42%   -0.11%     
==========================================
  Files         260      260              
  Lines       58599    58892     +293     
  Branches     4978     4985       +7     
==========================================
+ Hits        49532    49720     +188     
- Misses       8240     8342     +102     
- Partials      827      830       +3     
Flag Coverage Δ
pytest-gpu-aomp-amdgpuX 68.43% <69.23%> (-0.02%) ⬇️
pytest-gpu-gcc- 79.11% <67.73%> (-0.08%) ⬇️
pytest-gpu-icx- 79.05% <67.73%> (-0.08%) ⬇️
pytest-gpu-nvc-nvidiaX 69.27% <69.23%> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Blocking replaced the iteration Dimensions, which degraded a GuardFactor to a
plain CondEq and left blocked Dimensions guarded. Run `subsampling` before
`_specialize_clusters` instead, so that tiles span the subsampled Dimensions.

Under MPI a SubGrid applied its factor to rank-local indices, so a rank not
starting on a subsampled point read the wrong parent points. Give
ConditionalDimension an `offset`, the intercept of its map onto the parent,
through a SubsamplingOffset symbol localized by MPI like a Thickness.

Decompose a SubGrid through SubGridDistributor, so that a rank owns exactly the
points lying on its own parent points, and derive the subsampled bounds from
the parent's rank-local ones, which stops strided loops overrunning a rank.
@mloubout
mloubout force-pushed the subsampling-blocking-mpi branch from 43c9548 to 7144885 Compare November 6, 2026 23:03
@review-notebook-app

Copy link
Copy Markdown

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

A bare Dimension in an expression compiled to the rank-local loop index, so
Eq(u, x + y) restarted from zero on each MPI rank. Offset it by the new
Grid.origin_isymbols, passed like origin_symbols. Dimensions indexing a
Function stay rank-local, and the code generated without MPI is unchanged.

Fixes #2214
@mloubout
mloubout force-pushed the subsampling-blocking-mpi branch from 4722b62 to 9604a82 Compare November 7, 2026 02:24

@FabioLuporini FabioLuporini left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

With subdomain=, a Function on a SubGrid kept iterating its whole subsampled
Dimension around the SubDomain loop, so every point accumulated over the
entire SubDomain. Subsample the SubDomain's Dimension instead, bound it by the
parent's actual extent, thickness included, and localize the thicknesses that
only bound such a Dimension.

Also keep time guarded in the subsampling pass: it now runs before cluster
specialization, where buffering and other time-based lowering rely upon the
guard.
@mloubout
mloubout force-pushed the subsampling-blocking-mpi branch from a2f26d6 to fb0ea5d Compare November 8, 2026 03:58

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

API api (symbolics, types, ...) compiler

Projects

None yet

Development

Successfully merging this pull request may close these issues.

MPI breaks in presence of SpaceDimensions used as free_symbols in an Eq

2 participants