calopy2java

English | 中文

A Java 17 port of the smoothing filters of Calopy, an indirect calorimetry analysis framework. The Python implementation is the reference, and JUnit tests compare the Java output with it point by point.

The port covers the 9 methods on Calopy's smoothing page (calopy/src/calopy/maths/filter). Data loading, condition grouping, statistics, RMR, energy balance and the UI are not ported.

---

Project Highlights

• 8 smoothing algorithms plus a pass-through filter, all matching the Python reference on the same 5,256-point input. The rolling-window filters and the splines are bit-identical; the remaining algorithms agree to about 1e-13.
• FITPACK port: the Dierckx FITPACK routines behind scipy.interpolate.UnivariateSpline (fpcurf, fpknot, fpdisc, fpbspl, fpback, fprati, splev) are ported line by line. Knot placement, the smoothing-parameter iteration and scipy's nest resizing all follow the original.
• pygam GAM port: reproduces the pygam.GAM() defaults (20 cubic B-splines, second-difference penalty with λ = 0.6, intercept).
• pandas / scipy edge behaviour matched: even windows, missing values, the Savitzky–Golay interp edges, and the Gaussian standard deviation being truncated to an integer.
• JUnit 5 tests read calopy/python_*.csv directly and assert on them, using relative paths.

---

Filters

Filter	Python counterpart	Notes
RollingWindowMeanFilter	series.rolling(w, center=True, min_periods=1).mean()	Even windows supported; missing values skipped
RollingWindowTriangularFilter	win_type="triang"	Weights from scipy.signal.windows.triang
RollingWindowGaussianFilter	win_type="gaussian", std=int(deviation)	Standard deviation truncated to an integer, as in Python
SavgolFilter	scipy.signal.savgol_filter(y, w, order)	Default mode='interp': polynomial fit at both ends; even windows supported
SingleComponentCosinorFilter	CosinorPy.cosinor1.fit_cosinor	Joint least squares y ~ 1 + cos + sin, curve rebuilt from mesor, amplitude and acrophase
GeneralizedAdditiveFilter	pygam.GAM()	P-spline: 20 cubic B-splines + second-difference penalty
UnivariateSplineFilter	UnivariateSpline(x, y, s)	FITPACK port (com.calopy.maths.spline)
UnivariateSplineAutofitFilter	UnivarateSplineAutofitFilter	Grid search over np.arange(0.02, 6, 0.1), picks s by penalized sum of squares
DoNothingOnSeriesFilter	DoNothingOnSeriesFilter	Pass-through

Missing values are null (pandas NaN) and are handled the same way as in Python:

• Rolling windows skip them, and output a missing value when a window has no valid value.
• Savitzky–Golay spreads them to every window that contains them.
• Cosinor leaves them out of the fit but still produces a value at every position.
• GAM fits on the remaining values and keeps the missing positions missing.
• Splines: scipy returns all-NaN when the input has a NaN, so the Java version returns an all-missing result too. It never treats a missing value as 0.

---

Cross-Validation Against the Python Reference

Input is example_csv.csv, column VO2(3) (5,256 points), with the same parameters the scripts in calopy/*.py used to produce the reference output. The table shows the mean and maximum absolute difference between the Java and Python output.

Filter	MAE before fix	MAE after fix	Max diff after fix
Rolling mean (w=5)	0	0	0
Rolling triangular (w=5)	0	0	0
Rolling Gaussian (w=5, std=1)	0	0	0
Savitzky–Golay (w=9, order=3)	0.00071	9.9e-14	2.3e-13
Cosinor (period 144)	0.98969	3.7e-14	1.7e-13
GAM	2.51335	8.9e-14	3.0e-13
Spline (fixed, s=10)	0.03524	0	0
Spline (autofit)	0.02692	0	0

calopy/python_extended_result.csv (generated by calopy/generate_extended_reference.py) adds cases the original comparison did not cover. All of them pass in the tests with a tolerance of 1e-9:

• The RER column: values around 0.9, where the spline at s=1 actually smooths.
• Even windows: Calopy's UI allows windows down to 2.
• Larger s on VO2: 100 and 500,000.
• Input with missing values.

History of the spline port

1. The first version was a knot-based approximation, with an MAE of 9.82 against Python (output kept in java_spline_result_old.csv).

2. The second version switched to Whittaker–Eilers discrete smoothing and brought the MAE on VO2(3) at s=10 down to 0.027.

   At that setting, however, scipy's spline barely smooths: the Python output differs from the raw data by only MAE 0.019, so returning the raw data unchanged was closer to Python than the second version was. The number therefore did not show agreement. On the RER column at s=1, where the spline really smooths, the second version differed from Python by 43% of the smoothing amount.

3. The current version is a line-by-line FITPACK port and is bit-identical to scipy on every test case.

Reproducing these numbers

import csv

def compare(py_file, py_col, java_file, java_col):
    p = [float(r[py_col]) for r in csv.DictReader(open(py_file))]
    j = [float(r[java_col]) for r in csv.DictReader(open(java_file))]
    diffs = [abs(a - b) for a, b in zip(p, j)]
    print(f"MAE={sum(diffs) / len(diffs):.3g}  max={max(diffs):.3g}")

compare("calopy/python_spline_result.csv", "Spline_Auto_Python",
        "java_spline_result.csv", "Spline_Auto_Java")

---

Why Smoothing Error Matters Downstream

Indirect calorimetry produces continuous VO₂ and VCO₂ time series. The quantities actually used are mostly derived from them: RER is a ratio (VCO₂/VO₂), and energy expenditure is a linear combination of the two. Both pass smoothing error on rather than averaging it out.

• The error is systematic. Neighbouring samples are biased in the same direction, so the error does not cancel when integrated over a time window or aggregated across subjects.
• Ratios amplify it where the denominator is small. The same absolute error in VO₂ gives a relative error in RER proportional to 1/VO₂. Low-VO₂ (resting) intervals are hit hardest, and those are often the intervals of interest.

This is why the port is checked against the reference implementation's output rather than against the raw input. Being close to the raw data says nothing about a smoother's correctness: a filter that does nothing has an MAE of 0 against the raw data.

---

Repository Structure

calopy2java/
├── calopy/                           # Reference Python/Shiny app (MIT, upstream: computational-discovery-research/calopy)
│   ├── python_*_result.csv           # Python reference output
│   ├── python_extended_result.csv    # Python reference output for the extended cases
│   └── *.py                          # Scripts that generate the reference output
├── calojava/                         # Java 17 / Maven port
│   ├── src/main/java/com/calopy/maths/filter   # Filters
│   ├── src/main/java/com/calopy/maths/spline   # FITPACK port
│   ├── src/main/java/com/calopy/maths/gam      # pygam GAM port
│   └── src/test/java                            # JUnit tests
├── example_csv.csv                   # Shared input: 5,256-point physiological time series
├── java_*_result.csv                 # Java output, compared column-wise against the Python reference
└── java_*_result_old.csv             # Output of an earlier version, kept for the before/after comparison

---

How to Run

Python (reference implementation):

pip install -r ./calopy/src/requirements.txt
cd ./calopy/src
shiny run --reload --port 8180 --launch-browser ./app.py

Regenerate the reference output (can be run from any directory):

python calopy/generate_extended_reference.py

Java (calojava):

cd calojava
mvn test            # 17 tests, about 15 seconds
mvn test -Pslow     # also runs the spline autofit comparison on VO2, about 2 minutes

Regenerate the java_*_result.csv files in the repository root:

cd calojava
mvn test-compile dependency:build-classpath -Dmdep.outputFile=cp.txt
java -cp "target/classes:target/test-classes:$(cat cp.txt)" com.calopy.maths.filter.ExportJavaResults

---

Author

Lukas Alexander GitHub: @chillboy67

Reference: Loipfinger S, et al. Nature Metabolism, 2025. DOI: 10.1038/s42255-025-01316-8 Original Calopy: https://calopy.app · MIT License