Skip to content

Add optional force_nofill to skip needsfill - #2027

Open
sdbachman wants to merge 2 commits into
NCAR:mainfrom
sdbachman:needsfill_skip
Open

Add optional force_nofill to skip needsfill#2027
sdbachman wants to merge 2 commits into
NCAR:mainfrom
sdbachman:needsfill_skip

Conversation

@sdbachman

Copy link
Copy Markdown

Plain-language summary

Some applications may need to read/write only over a portion of the model grid, but PIO currently won't allow that because of the "needsfill" attribute. This PR creates a way to (optionally) bypass the path where needsfill is set so that PIO will allow reads/writes to only certain parts of the grid.

Summary

Add an opt-in way for applications to skip PIO’s decomposition coverage / fill-value path (needsfill) when initializing a decomposition.
New optional flag: force_nofill. When set, determine_fill() forces needsfill = false and returns immediately, so the subset rearranger does not build the holegrid / fill machinery for that decomposition.
Existing PIOc_InitDecomp / PIOc_InitDecomp_bc / Fortran PIO_initdecomp call sites are unchanged: omitting the flag preserves current behavior.

Problem

Some applications (e.g. coastal ocean models with land masking) define decompositions whose compute map does not cover every point of the global array. In that case, determine_fill() sets iodesc->needsfill = true, and the subset rearranger allocates and populates holegrid / fill structures.
With ROMS + MASKING on Derecho (parallel I/O via start/count PIO_initdecomp → PIOc_InitDecomp_bc, which always uses PIO_REARR_SUBSET), that path led to heap corruption / glibc malloc aborts during the first initdecomp, before any history I/O completed. Forcing needsfill = false for those decompositions avoided the crash and allowed the run to proceed.
Note: changing the rearranger at PIO_init (e.g. box vs subset) does not affect the block-cyclic (_bc) path, which hardcodes subset rearranging. A per-decomposition opt-out of needsfill is the targeted fix for this class of use case.

Changes

C API (pio.h, pioc.c)
• Add bool force_nofill to io_desc_t.
◦ Add:
◦ PIOc_InitDecomp_flags(..., int force_nofill)
◦ PIOc_InitDecomp_bc_flags(..., int force_nofill)
◦ Keep existing entry points as thin wrappers that pass force_nofill = 0:
◦ PIOc_InitDecomp(...) → PIOc_InitDecomp_flags(..., 0)
◦ PIOc_InitDecomp_bc(...) → PIOc_InitDecomp_bc_flags(..., 0)
• Broadcast force_nofill with the other initdecomp arguments on async IO systems.

Core logic (pio_rearrange.c)

In determine_fill():
if (iodesc->force_nofill) {
iodesc->needsfill = false;
return PIO_NOERR;
}
Fortran API (piolib_mod.F90)
◦ Add optional force_nofill to:
◦ PIO_initdecomp_bc (start/count path)
◦ PIO_initdecomp_dof_i4 / _i8 / internal
• If absent or .false., pass 0 to C (stock behavior).
• If .true., pass 1 and skip the needsfill path.

Example:

call PIO_initdecomp(iosystem, PIO_double, dims, start, count, iodesc, &
force_nofill=.true.)

Compatibility

•	Default behavior unchanged for all existing callers.
•	New C APIs are additive; old symbols remain.
•	Fortran remains source-compatible via an optional argument.

When to use this

Use force_nofill=.true. / force_nofill != 0 when:
• The application guarantees that unwritten / non-owned points do not need PIO fill values (e.g. masked land points that the model never writes, or the file already has acceptable fill / the app does not rely on PIO hole filling), and
• The automatic needsfill / holegrid path is incorrect or harmful for that decomposition.
Do not use it if the application depends on PIO writing fill values for uncovered grid points.

Files touched

•	src/clib/pio.h
•	src/clib/pioc.c
•	src/clib/pio_rearrange.c
•	src/flib/piolib_mod.F90

Incomplete compute maps (e.g. masked land points) set needsfill and
can corrupt heap in the subset rearranger. Allow apps to opt out via
PIOc_InitDecomp*_flags and Fortran PIO_initdecomp(force_nofill=...),
preserving default behavior when the flag is omitted.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant