Render fieldcompare diff visualizations by PranjalManhgaye · Pull Request #883 · precice/tutorials

PranjalManhgaye · 2026-07-17T20:12:32Z

Summary

Render archived fieldcompare diff VTK files as PNG images when result comparison fails.
Generate one image per numeric point field using distinct sphere glyphs and a generic automatically selected view.
Store images under diff-results/visualizations/ in the system-test artifacts.
Keep visualization best-effort so rendering errors do not hide the original fieldcompare failure.

Fixes #441.

Test plan

Generated PNGs from synthetic scalar and vector VTK fields.
Verified the archive-to-visualization failure path locally.
Rendered five real CI diff VTK files, producing 13 PNGs.
Checked zero and non-zero differences visually.
Ran pre-commit checks successfully.

MakisH

Thank you for the PR! Some first comments, I will continue once I can run it.

Run diff PNG rendering through a dedicated Docker stage with libosmesa6 so local and CI follow the same headless path, and include generated PNGs in the _logs artifact with a job-summary link.

MakisH · 2026-08-05T07:10:35Z

+        max_abs_value = float(np.max(np.abs(values)))
+        color_limit = max_abs_value if max_abs_value > 0 else 1.0
+        plotter.add_mesh(
+            glyphs,
+            scalars=scalar_name,
+            cmap="coolwarm",
+            clim=(-color_limit, color_limit),
+            scalar_bar_args={"title": field_name},
+        )


In this run, fieldcompare complained about the force in the last time window:

field-compare-1 | Comparing '/runs/perpendicular-flap_fluid-openfoam-solid-calculix_2026-08-05-084014/precice-exports/Solid-Mesh-Solid.dt500.vtu' field-compare-1 | and '/runs/perpendicular-flap_fluid-openfoam-solid-calculix_2026-08-05-084014/reference-results-unpacked/fluid-openfoam_solid-calculix/Solid-Mesh-Solid.dt500.vtu' field-compare-1 | Reading '/runs/perpendicular-flap_fluid-openfoam-solid-calculix_2026-08-05-084014/precice-exports/Solid-Mesh-Solid.dt500.vtu' field-compare-1 | Reading '/runs/perpendicular-flap_fluid-openfoam-solid-calculix_2026-08-05-084014/reference-results-unpacked/fluid-openfoam_solid-calculix/Solid-Mesh-Solid.dt500.vtu' field-compare-1 | -- Comparing the field 'Force': FAILED field-compare-1 | -- Report: Deviation above tolerance detected -> [ 8.05858e-03 -5.47671e-05 0.00000e+00] vs. [ 8.05464e-03 -5.46938e-05 0.00000e+00] ([ 0.05 -0.13 inf] %) field-compare-1 | -- Predicate: DefaultEquality (abs_tol: 0., rel_tol: 3.e-07) field-compare-1 | Wrote diff into '/runs/perpendicular-flap_fluid-openfoam-solid-calculix_2026-08-05-084014/precice-exports/diff_Solid-Mesh-Solid.dt500.vtu.vtu' field-compare-1 | File comparison FAILED with 2 PASSED / 1 FAILED / 0 SKIPPED

but looking at the image, this difference is not clearly visible:

The [-1.0,1.0] range is a bit arbitrary. Why is that being picked? I would generally rely on the color scale and range to understand how large a regression is.

Also, if the difference is visualized (and not the absolute value of it), why have the range symmetric? There must be a minimum value and a maximum value.

Notes from our discussion:

We should show the range of the diff files

Even if the values tend to zero, if that doesn't cause any issues, let's have the real values: seeing a range -1.23e-33 to +4.56e-33 is also clear, and it is how ParaView does it

We noticed that the visualizations for the Solid participant are missing from the picture.

I have updated the color scale to use the real min/max from the diff field (clim=(vmin, vmax)), including near-zero ranges, and show that range in the overlay.
On the Solid side: I re-checked locally with real CI diffs and synthetic Solid/Force cases, Solid PNGs are produced. Some look almost uniform when the diff is ~0 (e.g. range 0 to 0); that is a real zero-difference field.

Still, fieldcompare reports:

field-compare-1 | [ FAILED ] runs/perpendicular-flap_fluid-openfoam-solid-calculix_2026-08-13-152820/precice-exports/Solid-Mesh-Fluid.dt500.vtu: ('Force')

but the archive does not include the respective files, so I cannot yet check if this was fixed.

MakisH · 2026-08-13T14:16:51Z

+        except Exception as error:
+            errors.append(f"Could not visualize {diff_file}: {error}")


In https://github.com/precice/tutorials/actions/runs/31705035580, I still don't see the images for the Solid-Mesh-Solid files. Since there are no logs, I cannot see if these files triggered this exception.

We need a log file, probably called system-tests-compare-diff.log, next to system-tests-compare.log.

MakisH · 2026-08-13T14:20:27Z

+        max_abs_value = float(np.max(np.abs(values)))
+        color_limit = max_abs_value if max_abs_value > 0 else 1.0
+        plotter.add_mesh(
+            glyphs,
+            scalars=scalar_name,
+            cmap="coolwarm",
+            clim=(-color_limit, color_limit),
+            scalar_bar_args={"title": field_name},
+        )


Still, fieldcompare reports:

field-compare-1 | [ FAILED ] runs/perpendicular-flap_fluid-openfoam-solid-calculix_2026-08-13-152820/precice-exports/Solid-Mesh-Fluid.dt500.vtu: ('Force')

but the archive does not include the respective files, so I cannot yet check if this was fixed.

MakisH · 2026-08-13T14:21:36Z

+To regenerate the PNG visualizations locally from an archived `diff-results/` folder (for example after downloading a CI artifact):
+
+```bash
+python3 visualize_fieldcompare_diffs.py /path/to/diff-results


I am running locally:

python3 visualize_fieldcompare_diffs.py /home/gc/Downloads/system_tests_run_31705035580_1_full/perpendicular-flap_fluid-openfoam-solid-calculix_2026-08-13-152820/diff-results

but this seems to be stuck without any output for a while. Some output in the script would be good. I assume something is wrong; it cannot be that slow.

Actually, it is very slow. It took at least 30min on my system. I guess the GHA runners are faster.

There, the job timed out, I didn't notice before:

WARNING: Could not render fieldcompare diff visualizations for Perpendicular flap (fluid-openfoam, solid-calculix): Command '['docker', 'compose', '--file', 'docker-compose.diff_visualizer.yaml', 'up', '--exit-code-from', 'diff-visualizer', '--abort-on-container-exit']' timed out after 300 seconds CRITICAL: Fieldcompare returned non zero exit code, therefore Perpendicular flap (fluid-openfoam, solid-calculix) failed

Notice also that only 0m 49s is reported as compare time.

So, overall, the logging could be improved.

MakisH · 2026-08-13T14:46:33Z

+        and path.suffix.lower() in SUPPORTED_SUFFIXES
+        and "diff" in path.name.lower()
+    )
+    for diff_file in diff_files:


This loop could be parallelized using the number of available threads.

Render fieldcompare diff visualizations

b1bea82

PranjalManhgaye requested a review from MakisH July 17, 2026 20:14

MakisH added the systemtests label Jul 18, 2026

MakisH reviewed Jul 18, 2026

View reviewed changes

Comment thread changelog-entries/441.md Outdated

Comment thread tools/tests/requirements.txt

Comment thread tools/tests/visualize_fieldcompare_diffs.py

Render fieldcompare diff visualizations via Docker OSMesa

88d6653

Run diff PNG rendering through a dedicated Docker stage with libosmesa6 so local and CI follow the same headless path, and include generated PNGs in the _logs artifact with a job-summary link.

PranjalManhgaye requested a review from MakisH July 28, 2026 08:52

MakisH reviewed Aug 4, 2026

View reviewed changes

Comment thread changelog-entries/441.md Outdated

Comment thread tools/tests/README.md Outdated

Comment thread tools/tests/README.md Outdated

Comment thread tools/tests/visualize_fieldcompare_diffs.py Outdated

Comment thread tools/tests/dockerfiles/ubuntu_2404/Dockerfile

Address review: simplify docs and inline small helpers

e4ae847

PranjalManhgaye requested a review from MakisH August 4, 2026 17:57

MakisH reviewed Aug 5, 2026

View reviewed changes

MakisH added this to GSoC 2026: System tests improvements Aug 7, 2026

github-project-automation Bot moved this to Planned next in GSoC 2026: System tests improvements Aug 7, 2026

MakisH moved this from Planned next to Needs review in GSoC 2026: System tests improvements Aug 7, 2026

Address review: real color scale, _diffs artifact, local regen docs

4efd8dd

PranjalManhgaye requested a review from MakisH August 12, 2026 16:50

Merge branch 'develop' into issue-441-visualize-diffs

551e35f

MakisH reviewed Aug 13, 2026

View reviewed changes

MakisH self-assigned this Aug 13, 2026

		except Exception as error:
		errors.append(f"Could not visualize {diff_file}: {error}")

Uh oh!

Uh oh!

Conversation

PranjalManhgaye commented Jul 17, 2026

Summary

Test plan

Uh oh!

MakisH left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

MakisH Aug 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

MakisH Aug 13, 2026 •

edited

Loading