Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
4 changes: 4 additions & 0 deletions .github/workflows/docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,10 @@ jobs:
path: |
benchmarks/results/f2py.json
benchmarks/results/prik.json
benchmarks/results/f2py-prik-first.json
benchmarks/results/f2py-f2py-first.json
benchmarks/results/prik-prik-first.json
benchmarks/results/prik-f2py-first.json
benchmarks/results/f2py-build.json
benchmarks/results/prik-build.json
docs/user/performance.md
Expand Down
24 changes: 21 additions & 3 deletions .github/workflows/merge-validation.yml
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ jobs:
- name: Ruff format
run: python -m ruff format --check .
- name: Wrapper-plan generator contracts
run: python tools/check_wrapper_codegen_complexity.py
run: python tools/check_codegen_complexity.py
- name: Bandit security scan
run: python -m bandit -c pyproject.toml -r prik --severity-level medium --confidence-level medium
- name: Vulture dead-code scan
Expand Down Expand Up @@ -413,7 +413,7 @@ jobs:
python tools/run_fortran_toolchain_lane.py \
--compiler gfortran \
--junit-dir "$RUNNER_TEMP"
- name: Run full suite without BLAS or LAPACK
- name: Run full suite without real-library examples
env:
PYTHONPATH: .
HYPOTHESIS_PROFILE: ci
Expand All @@ -437,7 +437,7 @@ jobs:
done

native-libraries:
name: BLAS + LAPACK · Ubuntu 24.04 · Python 3.12
name: BLAS + LAPACK + FFTPACK + MINPACK · Ubuntu 24.04 · Python 3.12
needs: [unit-tests, unit-tests-macos]
if: >-
${{ !contains(github.event.pull_request.labels.*.name, 'ignore-real-library-wrappers') }}
Expand Down Expand Up @@ -520,6 +520,20 @@ jobs:
run: |
source examples/lapack/build_all.sh
python -m pytest -q examples/lapack/tests examples/lapack/ci/full_surface.py
- name: Run FFTPACK 31-procedure full-surface audit
env:
PYTHONPATH: .
HYPOTHESIS_PROFILE: ci
run: |
source examples/fftpack/build_all.sh
python -m pytest -q examples/fftpack/tests
- name: Run MINPACK 22-procedure and parameter-array full-surface audit
env:
PYTHONPATH: .
HYPOTHESIS_PROFILE: ci
run: |
source examples/minpack/build_all.sh
python -m pytest -q examples/minpack/tests

documentation-benchmark:
name: Documentation performance benchmark · Ubuntu 24.04 ARM64 · Python 3.12
Expand Down Expand Up @@ -598,6 +612,10 @@ jobs:
path: |
benchmarks/results/f2py.json
benchmarks/results/prik.json
benchmarks/results/f2py-prik-first.json
benchmarks/results/f2py-f2py-first.json
benchmarks/results/prik-prik-first.json
benchmarks/results/prik-f2py-first.json
benchmarks/results/f2py-build.json
benchmarks/results/prik-build.json
docs/user/performance.md
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: Native Libraries
name: Real Libraries

on:
push:
Expand All @@ -12,7 +12,7 @@ env:

jobs:
real-library-wrappers:
name: BLAS + LAPACK · Ubuntu 24.04 · Python 3.12
name: BLAS + LAPACK + FFTPACK + MINPACK · Ubuntu 24.04 · Python 3.12
if: >-
${{
github.event_name != 'pull_request' ||
Expand Down Expand Up @@ -97,3 +97,17 @@ jobs:
run: |
source examples/lapack/build_all.sh
python -m pytest -q examples/lapack/tests examples/lapack/ci/full_surface.py
- name: Run FFTPACK 31-procedure full-surface audit
env:
PYTHONPATH: .
HYPOTHESIS_PROFILE: ci
run: |
source examples/fftpack/build_all.sh
python -m pytest -q examples/fftpack/tests
- name: Run MINPACK 22-procedure and parameter-array full-surface audit
env:
PYTHONPATH: .
HYPOTHESIS_PROFILE: ci
run: |
source examples/minpack/build_all.sh
python -m pytest -q examples/minpack/tests
2 changes: 1 addition & 1 deletion .github/workflows/static-analysis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ jobs:
- name: Ruff format
run: python -m ruff format --check .
- name: Wrapper-plan generator contracts
run: python tools/check_wrapper_codegen_complexity.py
run: python tools/check_codegen_complexity.py
- name: Bandit security scan
run: python -m bandit -c pyproject.toml -r prik --severity-level medium --confidence-level medium
- name: Vulture dead-code scan
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -135,7 +135,7 @@ jobs:
python tools/run_fortran_toolchain_lane.py \
--compiler gfortran \
--junit-dir "$RUNNER_TEMP"
- name: Run full suite without BLAS or LAPACK
- name: Run full suite without real-library examples
env:
PYTHONPATH: .
HYPOTHESIS_PROFILE: ci
Expand Down
8 changes: 4 additions & 4 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ Do not spend context window or analysis on those files unless explicitly request
When asked to change or move an API, import path, command, feature, or behavior, do not add or keep compatibility layers, aliases, shims, fallback paths, or legacy entrypoints unless explicitly requested. A requested change means the old behavior should be removed.
When updating tests, remove obsolete tests that only assert removed/old implementation behavior does not exist. Do not preserve rejection or absence checks for API/features that were intentionally removed unless explicitly requested.

Before wrapper planning begins in `prik/wrapper_codegen/planner.py`, the
Before wrapper planning begins in `prik/codegen/planner.py`, the
post-IR policy stage must have completed every semantic decision needed by
wrapper generation, including object kind, ownership, transfer, destruction,
mutability/writeback, nullability, output projection, release responsibility,
Expand Down Expand Up @@ -52,8 +52,8 @@ the stage breakdown when they help explain the implementation.

Changes limited to wrapper planning, direct bridge/binding lowering, or native
compilation should use the focused owners under
`tests/fortran/infrastructure/wrapper_codegen/`, feature-local
`tests/fortran/*/wrapper_codegen/` directories, and
`tests/fortran/infrastructure/codegen/`, feature-local
`tests/fortran/*/codegen/` directories, and
`tests/fortran/building_shared_library/compiling/` as applicable. Include the
relevant end-to-end feature tests whenever a generated or compiled mechanism
changes; run a broader suite when behavior spans multiple stages.
Expand All @@ -72,7 +72,7 @@ pull-request verification:
- `python3 -m ruff check .`
- `python3 -m ruff format --check .`
- `python3 tools/check_static_analysis_versions.py`
- `python3 tools/check_wrapper_codegen_complexity.py`
- `python3 tools/check_codegen_complexity.py`
- `python3 -m bandit -c pyproject.toml -r prik --severity-level medium --confidence-level medium`
- `python3 -m vulture`
- `python3 tools/check_radon_policy.py --base-ref auto`
Expand Down
23 changes: 15 additions & 8 deletions benchmarks/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ concurrency. The workloads are:
`examples/blas/native/`, with all 155 routines
required in each generated extension.

One untimed warm-up precedes six measured clean builds of each workload by
One untimed warm-up precedes four measured clean builds of each workload by
default. Tool order alternates between rounds. Set
`PRIK_BUILD_BENCHMARK_RUNS` or `PRIK_BUILD_BENCHMARK_WARMUPS` to change those
counts for local investigation.
Expand All @@ -31,9 +31,16 @@ Every workload is measured with two compiler profiles:

Runtime-call measurements continue to use only the optimized profile.

`PRIK_BENCHMARK_FIRST=prik` is the default measurement order. Set it to
`f2py` to reverse the order. The publication workflow alternates this setting
between runs so one tool is not systematically measured first.
`PRIK_BENCHMARK_FIRST=prik` selects the first clean-build tool. Set it to
`f2py` to reverse the first round; later build rounds alternate automatically.

For each runtime group, measurements use an A/B/B/A sequence in the same job:
PRIK then f2py, immediately followed by f2py then PRIK. Each order receives half
of the reduced worker budget, and the final per-tool suites merge both passes
before comparison. The four order-specific JSON files remain in `results/` so
the balance is auditable. This keeps the same 13 workloads while targeting
about 20 minutes on the pinned CI benchmark runner instead of doubling the
previous 30-minute job.

Run the complete correctness check and rigorous benchmark with:

Expand All @@ -46,10 +53,10 @@ and applies each profile consistently to the native Fortran source, generated
Fortran wrapper, and generated C binding. Runtime extensions use the optimized
profile. Runtime cases use latency, medium, and bulk sampling budgets: the
short and noisier cases use more pyperf worker processes and values, while
expensive matrix cases use fewer. Contiguous case groups preserve the table's
call, vector, matrix-sum, and matrix-update order as their results are appended to the same per-tool
JSON suite, so comparison and publication commands remain unchanged. The
script retains f2py's generated sources under `build/f2py` for local
expensive matrix cases use fewer. Each case group runs adjacent PRIK/f2py
measurements in both orders. The order passes are merged into the same final
per-tool JSON suites, so comparison and publication commands remain unchanged.
The script retains f2py's generated sources under `build/f2py` for local
inspection.

To compare existing results without rebuilding:
Expand Down
2 changes: 1 addition & 1 deletion benchmarks/build_time.py
Original file line number Diff line number Diff line change
Expand Up @@ -331,7 +331,7 @@ def run_build_benchmarks(

def parse_args(argv: list[str]) -> argparse.Namespace:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--runs", type=int, default=6)
parser.add_argument("--runs", type=int, default=4)
parser.add_argument("--warmups", type=int, default=1)
parser.add_argument("--first", choices=TOOLS, default="prik")
parser.add_argument("--compiler", default="gfortran")
Expand Down
55 changes: 39 additions & 16 deletions benchmarks/run.sh
Original file line number Diff line number Diff line change
Expand Up @@ -51,11 +51,10 @@ echo "========================================"
echo " Benchmarking clean end-to-end builds"
echo "========================================"
python3 build_time.py \
--runs "${PRIK_BUILD_BENCHMARK_RUNS:-6}" \
--runs "${PRIK_BUILD_BENCHMARK_RUNS:-4}" \
--warmups "${PRIK_BUILD_BENCHMARK_WARMUPS:-1}" \
--first "$benchmark_first"

echo "Benchmark order: ${binding_tools[*]}"
runtime_groups=(
calls
vector-latency
Expand All @@ -65,23 +64,47 @@ runtime_groups=(
matrix-update-latency
matrix-update-bulk
)
for binding_tool in "${binding_tools[@]}"; do
result_args=(-o "results/$binding_tool.json")
for runtime_group in "${runtime_groups[@]}"; do
BINDING_TOOL="$binding_tool" \
PRIK_RUNTIME_BENCHMARK_GROUP="$runtime_group" \
OMP_NUM_THREADS=1 \
OPENBLAS_NUM_THREADS=1 \
MKL_NUM_THREADS=1 \
python3 runtime.py \
--rigorous \
--affinity=0 \
--inherit-environ=BINDING_TOOL,PRIK_RUNTIME_BENCHMARK_GROUP,PRIK_BENCHMARK_CPU_MODEL,OMP_NUM_THREADS,OPENBLAS_NUM_THREADS,MKL_NUM_THREADS \
"${result_args[@]}"
result_args=(--append "results/$binding_tool.json")
runtime_passes=(prik-first f2py-first)
for runtime_group in "${runtime_groups[@]}"; do
for runtime_pass in "${runtime_passes[@]}"; do
case "$runtime_pass" in
prik-first)
binding_tools=(prik f2py)
;;
f2py-first)
binding_tools=(f2py prik)
;;
esac
echo "Runtime benchmark order ($runtime_group, $runtime_pass): ${binding_tools[*]}"
for binding_tool in "${binding_tools[@]}"; do
result_file="results/$binding_tool-$runtime_pass.json"
if [[ -f "$result_file" ]]; then
result_args=(--append "$result_file")
else
result_args=(-o "$result_file")
fi
BINDING_TOOL="$binding_tool" \
PRIK_RUNTIME_BENCHMARK_GROUP="$runtime_group" \
PRIK_RUNTIME_ORDER_PASS="$runtime_pass" \
OMP_NUM_THREADS=1 \
OPENBLAS_NUM_THREADS=1 \
MKL_NUM_THREADS=1 \
python3 runtime.py \
--rigorous \
--affinity=0 \
--inherit-environ=BINDING_TOOL,PRIK_RUNTIME_BENCHMARK_GROUP,PRIK_RUNTIME_ORDER_PASS,PRIK_BENCHMARK_CPU_MODEL,OMP_NUM_THREADS,OPENBLAS_NUM_THREADS,MKL_NUM_THREADS \
"${result_args[@]}"
done
done
done

for binding_tool in prik f2py; do
python3 -m pyperf convert \
"results/$binding_tool-prik-first.json" \
--add "results/$binding_tool-f2py-first.json" \
--output "results/$binding_tool.json"
done

python3 -m pyperf compare_to \
results/f2py.json \
results/prik.json \
Expand Down
21 changes: 13 additions & 8 deletions benchmarks/runtime.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,20 +22,23 @@ def get_function(api: Any, name: str) -> Callable[..., Any]:

tool = os.environ.get("BINDING_TOOL")
benchmark_group = os.environ.get("PRIK_RUNTIME_BENCHMARK_GROUP", "all")
order_pass = os.environ.get("PRIK_RUNTIME_ORDER_PASS")
group_settings = {
"all": (20, 3),
"calls": (64, 4),
"vector-latency": (64, 4),
"vector-bulk": (16, 3),
"matrix-sum-latency": (64, 4),
"matrix-sum-bulk": (4, 3),
"matrix-update-latency": (64, 4),
"matrix-update-bulk": (32, 3),
"all": (8, 3),
"calls": (16, 4),
"vector-latency": (16, 4),
"vector-bulk": (4, 3),
"matrix-sum-latency": (16, 4),
"matrix-sum-bulk": (2, 3),
"matrix-update-latency": (16, 4),
"matrix-update-bulk": (8, 3),
}

if benchmark_group not in group_settings:
choices = ", ".join(group_settings)
raise RuntimeError(f"Unknown PRIK_RUNTIME_BENCHMARK_GROUP {benchmark_group!r}; choose one of: {choices}.")
if order_pass not in {"prik-first", "f2py-first"}:
raise RuntimeError("Set PRIK_RUNTIME_ORDER_PASS to 'prik-first' or 'f2py-first'.")

if tool == "prik":
extension = importlib.import_module("bench_prik")
Expand All @@ -59,6 +62,8 @@ def get_function(api: Any, name: str) -> Callable[..., Any]:
"python_version": sys.version,
"numpy_version": np.__version__,
"platform_details": platform.platform(),
"runtime_order_pass": order_pass,
"runtime_order_protocol": "balanced_ab_ba",
}
if cpu_model := os.environ.get("PRIK_BENCHMARK_CPU_MODEL"):
metadata["cpu_model_name"] = cpu_model
Expand Down
25 changes: 25 additions & 0 deletions conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,35 @@
from pathlib import Path
import sys

import pytest

ROOT = Path(__file__).resolve().parent
if str(ROOT) not in sys.path:
sys.path.insert(0, str(ROOT))


def pytest_addoption(parser: pytest.Parser) -> None:
"""Register repository-wide options before pytest parses its command line.

The Fortran test hooks consume these options from ``tests/fortran``. They
must be registered here because nested conftests are discovered too late
when pytest collects the complete repository suite.
"""
group = parser.getgroup("prik Fortran")
group.addoption(
"--prik-fortran-compiler",
action="store",
default=os.environ.get("PRIK_TEST_FORTRAN_COMPILER", "gfortran"),
metavar="EXECUTABLE",
help="Fortran compiler executable used by compiled Fortran tests.",
)
group.addoption(
"--require-toolchain-smoke",
action="store_true",
help="Require a nonempty, skip-free selection containing only toolchain smoke nodes.",
)


try:
from hypothesis import HealthCheck, settings
except ImportError: # pragma: no cover - base test installs can omit QA extras.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,13 @@
# Compiler-Backed Preprocessing Notes
---
title: Compiler Preprocessing Reference
audience: developers, contributors
prerequisites: repository structure, native project compiler flags
related: source-map.md, c-parser-reference.md, fortran-parser-reference.md, ../user/examples/recipes/compiler-preprocessing.md
status: maintained
publication: draft
---

# Compiler-Backed Preprocessing Reference

`prik/pipeline/preprocessing.py` owns compiler-backed preprocessing for the wrapper
pipeline. The parsers consume one expanded source stream; they do not evaluate
Expand Down Expand Up @@ -40,8 +49,7 @@ Built-in adapters cover GCC-compatible C/Clang (`-E -x c`), GNU Fortran
compiler families. A custom template must write expanded source to stdout:

```bash
python -m prik parse include/api.h --language c \
--preprocess compiler \
python -m prik parse include/api.h --language c --preprocess compiler \
--preprocessor-adapter command-template \
--preprocess-template 'vendor-cc --preprocess {include_dirs} {defines} {source}'
```
Expand Down
Loading
Loading