Skip to content

Write compact dimensional XDMF fields with optional native checkpoints - #756

Open
gthyagi wants to merge 9 commits into
underworldcode:developmentfrom
gthyagi:feature/dimensional-xdmf-output
Open

gthyagi wants to merge 9 commits into
underworldcode:developmentfrom
gthyagi:feature/dimensional-xdmf-output

Conversation

@gthyagi

@gthyagi gthyagi commented Sep 18, 2026

Copy link
Copy Markdown
Contributor

Problem

Unit-aware output currently duplicates mesh-variable values across native /fields and ParaView compatibility groups. Higher-order continuous fields are always reduced to P1, including P2 simplex fields that XDMF and ParaView can represent directly. Benchmark scripts then need their own field-to-unit mappings or HDF5 postprocessing, which can produce inconsistent dimensional values.

Resulting behavior

Mesh.write_timestep(create_xdmf=True) now treats dimensional /fields as the authoritative analysis and coordinate-reload data:

  • Continuous P1 and DG0 values are referenced directly from /fields.
  • Continuous P2 triangle and tetrahedron fields use XDMF Triangle_6 and Tetrahedron_10 connectivity, with UW3 nodes mapped to VTK corner/edge order. No P1 value copy is written.
  • Simplex DG1 fields store one exact dimensional element-corner value array under /fields. All DG1 variables share a single disconnected geometry under /viz/dg1 in the mesh file and appear as attributes on one XDMF DG1 grid. No per-variable visualization geometry or second value array is written.
  • MeshVariable.read_timestep() recognizes the compact DG1 representation, matches complete source and destination cells by geometry, reconstructs each local affine polynomial, and evaluates it at the native DG1 nodes. This preserves separate traces at coincident vertices and works across MPI repartitioning.
  • Continuous P3+ fields and unsupported P2 layouts receive one compact P1 visualization reduction under /visualization.
  • DG2+ fields and unsupported DG1 layouts receive one compact DG0 visualization reduction under /visualization.
  • Tensor values retain UW3's compact component count rather than adding a nine-component copy.
  • The old /vertex_fields, /cell_fields, and /dg1 groups are no longer generated for mesh variables.

When nondimensional scaling is active, /fields/coordinates, /fields/<name>, and shared visualization geometry are written in the mesh and variable units declared by the model. MeshVariable.read_timestep() recognizes this metadata and converts the data back to the active nondimensional solver frame. P2 cell connectivity only indexes the native coordinate/value arrays, so VTK node ordering does not affect KDTree reload.

petsc_reload=True writes one native nondimensional PETSc global vector plus DMPlex section metadata under /restart/petsc. same_layout=True reads the nested vector through a standalone PETSc vector, while the default reader uses globalVectorLoad with PETSc's migration SF for a reconstructed mesh or MPI layout. The reader also accepts the former /uw_checkpoint group so existing restart files remain readable.

With create_xdmf=False, petsc_reload=True, each variable file contains only /restart/petsc; /fields is not written first. With both flags enabled, the file contains dimensional /fields for analysis/remap and /restart/petsc for exact restart.

The mesh file keeps PETSc's native /geometry for restart and, when dimensional scaling is active, a physical /viz/geometry array for XDMF. XDMF 3 function-based geometry scaling was tested but is not accepted by ParaView 6, so this small coordinate-only copy remains. Simplex DG1 adds one shared disconnected /viz/dg1 geometry because XDMF needs independent vertices to display element-local jumps correctly.

Surface signed and unsigned distance fields inherit the mesh coordinate units, so geometric diagnostics are exported in physical length units automatically.

Storage summary

Field layout Reload/analysis data XDMF data
Continuous P1 Native dimensional /fields Direct /fields nodes
Continuous P2 triangle/tetrahedron Native dimensional /fields Direct /fields with Triangle_6/Tetrahedron_10
DG0 Native dimensional /fields Direct /fields cells
DG1 triangle/tetrahedron Exact dimensional corner values in /fields Same values plus one shared /viz/dg1 geometry
Continuous P3+ or unsupported P2 Native dimensional /fields P1 /visualization reduction
DG2+ or unsupported DG1 Native dimensional /fields DG0 /visualization reduction

For restart-only output, the variable file contains one native global vector under /restart/petsc and no /fields or /visualization group.

Dependency

This PR is stacked on #736 and retains its element-local DG1 foundation. The compact layout replaces #736's separate /dg1 group with dimensional /fields, shared mesh geometry, and optional /restart/petsc data.

Validation

  • Current XDMF regression suite: 14 serial tests passed.
  • Focused DG1 and physical-unit suite: 5 serial tests passed.
  • Two-rank DG1 suite: 3 tests passed per rank, including 2D and 3D cases.
  • Cross-decomposition DG1 timestep reload passed from 1 to 2 ranks and from 2 to 1 rank with maximum error 1.7763568394002505e-15.
  • ParaView 6.0.0 loaded the compact output as a base grid plus one shared DG1 grid carrying the DG1 field attributes. It also loaded P2 triangles as vtkQuadraticTriangle (cell type 22) and P2 tetrahedra as vtkQuadraticTetra (cell type 24).
  • Python compilation and git diff --check pass.

Underworld development team with AI support from OpenAI Codex.

Write independent physical vertices and cell-local linear field traces for DG1 triangles and tetrahedra. Use native cell maps and owned-cell PETSc output to preserve discontinuities without coordinate matching or inter-element averaging.

Expose continuous and DG1 fields as separate grids in one XDMF file, support scalar/vector/tensor layouts, and retain native checkpoint data unchanged. Reject unsupported DG visualization explicitly while preserving native-only output.

Add serial and MPI regression coverage for affine fields, jumps, orientation, ownership, tensor packing, and checkpoint reload. Document supported layouts and ParaView usage. Validated 24 focused serial tests, four DG tests on eight ranks, and actual ParaView 2D/3D reads.
Keep mesh geometry and field checkpoint datasets in the nondimensional solver frame, and create dimensional visualization copies using declared mesh and variable units. Record those units in HDF5 and XDMF metadata.

Use nondimensional coordinates for DG1 interpolation before converting its disconnected visualization grid. This removes the need for benchmark-side coordinate wrappers, field-unit mappings, and HDF5 postprocessing.

Cover continuous P2, DG0, and DG1 output in serial and MPI tests.

Underworld development team with AI support from Claude Code
@gthyagi
gthyagi requested a review from lmoresi as a code owner September 18, 2026 05:14
Declare signed and unsigned Surface distance variables with the mesh coordinate unit so physical XDMF export scales geometric distance fields without benchmark-side mappings.

Extend the physical-output regression to cover Surface.abs_distance while preserving its nondimensional checkpoint values.

Underworld development team with AI support from Claude Code
Make dimensional /fields datasets authoritative for analysis and direct XDMF rendering. Export continuous P2 triangles with Triangle_6 connectivity and DG1 simplices with exact disconnected-corner basis conversion, while reducing only unsupported continuous P3+ and DG2+ layouts to compact P1 and DG0 visualization arrays.\n\nStore optional native PETSc restart data under /uw_checkpoint, reuse its existing variable vector for same-layout reload, and retain migration-based reload for reconstructed meshes. Teach read_timestep to convert dimensional fields back to the active model frame and reject DG1 corner data that cannot be inverted by nearest-neighbour remapping.\n\nReplace compatibility-group tests with serial and MPI coverage for direct layouts, high-order reductions through P4/DG4, dimensional units, ParaView-facing topology, and checkpoint round trips. Update checkpoint documentation for the new storage contract.
@gthyagi gthyagi changed the title Write dimensional XDMF output while preserving native checkpoints Write compact dimensional XDMF fields with optional native checkpoints Sep 18, 2026
Keep dimensional DG1 interpolation coordinates and values under /fields so read_timestep can recover solver data exactly. Store the disconnected-corner basis conversion separately under /visualization for XDMF, where it preserves element geometry and discontinuous traces.\n\nRemove the DG1 read_timestep rejection and add serial and two-rank MPI coverage for coordinate reload, PETSc checkpoint reload, dimensional conversion, and ParaView rendering. Update the checkpoint documentation to explain why DG1 requires separate reload and visualization representations.
Write continuous P2 tetrahedra as VTK-ordered Tetrahedron_10 grids so ParaView can render the native quadratic field without a P1 reduction. Keep field coordinates and values in their native order so coordinate-based reload remains independent of visualization connectivity.

For create_xdmf=False with petsc_reload=True, write only the DMPlex checkpoint payload instead of first duplicating values under /fields. Store one owned global vector and reload it directly for the same layout or through PETSc's migration SF for reconstructed layouts and different MPI decompositions.

Add serial and MPI coverage for quadratic tetrahedra, checkpoint-only file structure, same-layout reload, and migrated reload.
Write PETSc-native DMPlex restart data under /restart/petsc so the HDF5 hierarchy states both the payload purpose and storage backend.

Keep read compatibility with grouped and direct-vector files that use the former /uw_checkpoint path, and cover the compatibility path in serial and MPI tests.
Inline the base-grid topology and geometry HDF5 DataItems in generated XDMF instead of relying on XML Reference nodes that ParaView 6's XDMF3 reader rejects. Keep the shared mesh DataItems for compatibility while making both the XDMF3 and legacy readers load exact P2 and DG1 output. Add a regression assertion that generated compact XDMF contains no XML-reference DataItems.
Store simplex DG1 corner geometry and connectivity once in the mesh HDF5 file, while retaining only each variable's exact dimensional corner values under /fields. Emit one XDMF DG1 grid with all compatible variables to remove per-variable visualization geometry duplication.\n\nTeach read_timestep to identify the compact corner-nodal representation and reconstruct native DG1 values cell by cell. Match complete cell geometry rather than coincident vertices so discontinuous traces remain distinct and reload remains exact across MPI repartitioning.\n\nUpdate DG1 and physical-unit tests to cover shared geometry, compact field files, the absence of optional restart data, XDMF references, and exact remapping.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant