Skip to content
62 changes: 43 additions & 19 deletions docs/developer/subsystems/checkpoint-output-and-reload-methods.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,17 +11,34 @@ but new code should use `write_timestep(..., petsc_reload=True)`.

## Standard API

`write_timestep()` always writes the mesh file and one HDF5 file per mesh
variable. Mesh-variable files always contain raw coordinate/value datasets under
`/fields`, which are the source data used by `MeshVariable.read_timestep()` for
coordinate/KDTree remapping.
`write_timestep()` writes the mesh file and one HDF5 file per mesh variable.
With XDMF enabled, mesh-variable files contain dimensional
coordinates and values under `/fields`. These are the authoritative arrays for
analysis and the source data used by `MeshVariable.read_timestep()` for
coordinate remapping.

The two optional payloads are selected with explicit flags:

| Flag | Output payload | Reader/use case |
| --- | --- | --- |
| `create_xdmf=True` | `/vertex_fields` or `/cell_fields` compatibility datasets plus a companion `.xdmf` file | ParaView/XDMF visualisation |
| `petsc_reload=True` | PETSc DMPlex section/vector metadata under `/topologies/uw_mesh/dms/...` | `MeshVariable.read_checkpoint()` PETSc-native reload |
| `create_xdmf=True` | Dimensional `/fields`, compact high-order reductions when needed, and a companion `.xdmf` file | Analysis and ParaView/XDMF visualisation |
| `petsc_reload=True` | Native PETSc DMPlex section/global-vector data under `/restart/petsc` | `MeshVariable.read_checkpoint()` exact reload |

The XDMF storage choice follows the finite-element layout:

| Field layout | XDMF representation |
| --- | --- |
| Continuous P1 | Direct `/fields` node values |
| Continuous P2 triangles/tetrahedra | Direct `/fields` values with `Triangle_6`/`Tetrahedron_10` connectivity |
| DG0 | Direct `/fields` cell values |
| DG1 triangles/tetrahedra | Native `/fields` for reload plus exact disconnected-corner `/visualization` data |
| Continuous P3+ or unsupported P2 layout | Compact P1 dataset under `/visualization` |
| DG2+ or unsupported DG1 | Compact DG0 dataset under `/visualization` |

DG1 retains native interpolation coordinates and values under `/fields`, so
`read_timestep()` can reload the solver field. XDMF reads an additional exact
basis conversion at disconnected element corners from `/visualization` because
the native interior DG1 points do not describe the full element geometry.

### Visualisation And Remap

Expand All @@ -48,11 +65,13 @@ output.mesh.Pressure.00000.h5
output.mesh.00000.xdmf
```

The field files contain coordinate/value datasets such as `/fields/<name>` and
`/fields/coordinates`, plus vertex-field datasets for visualisation. Reloading
uses coordinate-based remapping. In practice this means the target variable is
filled by comparing target coordinates to source coordinates, using a KDTree or
similar nearest-neighbour/remap process.
The field files contain `/fields/<name>` and `/fields/coordinates`. P1, P2
triangles/tetrahedra, and DG0 are visualized directly from those datasets. DG1
uses its additional exact disconnected-corner representation. Continuous P3+
fields use one compact P1 visualization reduction, and DG2+ fields use DG0.
Reloading with `read_timestep()` compares target coordinates to the dimensional
source coordinates and converts the saved values back to the active model's
nondimensional solver frame.

### Unified Visualisation And PETSc Reload

Expand All @@ -75,9 +94,9 @@ velocity.read_checkpoint(
)
```

With both `create_xdmf=True` and `petsc_reload=True`, the same variable file can
be used by `read_timestep()` for coordinate/KDTree remapping and by
`read_checkpoint()` for exact PETSc-native reload.
With both flags enabled, the same variable file contains dimensional `/fields`
for analysis, XDMF, and coordinate remapping plus native `/restart/petsc` data
for exact PETSc reload.

### PETSc Reload Without XDMF

Expand All @@ -94,8 +113,8 @@ mesh.write_timestep(
)
```

This still writes raw `/fields` datasets, but it does not write
`/vertex_fields`, `/cell_fields`, or a companion `.xdmf` file.
This writes the native checkpoint directly and does not create a companion
`.xdmf` file or duplicate `/fields` coordinate/value arrays.

Typical PETSc-reload-only files still use the timestep naming convention:

Expand All @@ -105,12 +124,17 @@ restart.mesh.Velocity.00000.h5
restart.mesh.Pressure.00000.h5
```

The variable files contain raw `/fields` datasets and PETSc reload metadata
under `/topologies/uw_mesh/dms/<variable>/`.
Each variable file contains only PETSc reload metadata and one native global
vector under `/restart/petsc/topologies/uw_mesh/dms/<variable>/`.

`read_checkpoint()` also recognizes the former `/uw_checkpoint` group so
existing restart files remain readable.

### Advantages

- Produces XDMF/HDF5 files suitable for visualisation workflows.
- Produces dimensional HDF5 fields suitable for analysis and visualisation.
- Avoids duplicate P1 field arrays when the finite-element layout can be
represented directly.
- Can remap data onto a different mesh or a different node layout.
- Useful for postprocessing where exact finite-element section identity is not
required.
Expand Down
47 changes: 40 additions & 7 deletions docs/developer/subsystems/checkpointing-system.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,25 +18,54 @@ including registered meshes, variables, swarms, and Python-side state bearers.

`Mesh.write_timestep()` is the standard mesh and mesh-variable output method.
It writes one mesh HDF5 file and one HDF5 file per requested mesh variable.
Each variable file always contains `/fields` coordinate/value datasets used by
With `create_xdmf=True`, each variable file contains dimensional
`/fields/coordinates` and `/fields/<name>` datasets. These arrays are the
authoritative analysis output and are also used by
`MeshVariable.read_timestep()`.

Optional payloads are controlled by explicit flags:

| Flag | Payload | Reader / use |
| --- | --- | --- |
| `create_xdmf=True` | XDMF-compatible visualisation datasets and a companion `.xdmf` file | ParaView and other XDMF tools |
| `petsc_reload=True` | PETSc DMPlex section/vector metadata | `MeshVariable.read_checkpoint()` |
| `petsc_reload=True` | PETSc DMPlex section/global-vector metadata | `MeshVariable.read_checkpoint()` |

When nondimensional scaling is active, `/fields` is converted during the write
to the mesh and variable units declared in the model. HDF5 attributes and XDMF
`Information` elements record those units. Analysis scripts can therefore read
physical values directly, without maintaining their own conversion table.

Set `petsc_reload=True` only when an exact restart is needed. It adds the native
nondimensional PETSc payload under `/restart/petsc`; the visualization and
analysis arrays remain dimensional.

### Visualisation and Coordinate Remap

XDMF reads P1 and DG0 values directly from `/fields`. Continuous P2 fields on
triangles and tetrahedra use XDMF `Triangle_6` and `Tetrahedron_10`
connectivity, including their edge nodes, so no P1 projection is stored. DG1
keeps native interpolation coordinates and values under `/fields`, allowing
`read_timestep()` to recover the solver field. For XDMF, the same element
polynomial is evaluated at disconnected cell corners under `/visualization`,
preserving jumps without averaging traces across shared edges or faces.

XDMF cannot represent every UW3 finite-element layout directly. Continuous P3+
fields and unsupported P2 layouts receive one compact P1 dataset under
`/visualization`. DG2+ fields and unsupported DG1 layouts receive one compact
DG0 dataset. Their exact dimensional values remain under `/fields`. Integration
point fields are not supported by this writer.

DG1 therefore has two representations because their coordinate sets serve
different purposes: native `/fields` for analysis and coordinate reload, and an
exact disconnected-corner `/visualization` representation for XDMF. Add
`/restart/petsc` when PETSc-native restart is also required.

```python
mesh.write_timestep(
"output",
index=100,
outputPath="output",
meshVars=[velocity, pressure, temperature],
time=100.0,
create_xdmf=True,
)
```
Expand Down Expand Up @@ -96,10 +125,14 @@ output/restart.mesh.velocity.00100.h5
output/restart.mesh.pressure.00100.h5
```

The variable files contain `/fields` datasets and PETSc reload metadata under
`/topologies/uw_mesh/dms/<variable>/`. `read_checkpoint()` uses PETSc DMPlex
topology, section, vector, and `PetscSF` metadata. It does not use KDTree
remapping.
The variable files contain PETSc reload metadata and one native global vector
under `/restart/petsc/topologies/uw_mesh/dms/<variable>/`. `read_checkpoint()`
uses PETSc DMPlex topology, section, vector, and `PetscSF` metadata. It does not
use dimensional `/fields` values or KDTree remapping. Restart-only output does
not write `/fields`, so the native values are stored only once.

The reader also accepts the former `/uw_checkpoint` group for compatibility
with existing restart files.

### Unified Visualisation and PETSc Reload

Expand Down
Loading
Loading