Skip to content

Fix stencils over-transfer of neighbor data - #737

Open
jpsamaroo wants to merge 7 commits into
masterfrom
jps/mpi-bad-scale-stencil
Open

Fix stencils over-transfer of neighbor data#737
jpsamaroo wants to merge 7 commits into
masterfrom
jps/mpi-bad-scale-stencil

Conversation

@jpsamaroo

@jpsamaroo jpsamaroo commented Aug 13, 2026

Copy link
Copy Markdown
Member

This is a regression introduced by prior stencil optimizations, which improved multi-threaded execution but significantly regressed multi-process and multi-rank.

Written by Claude Opus

@github-actions

github-actions Bot commented Aug 13, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Dagger benchmarks: dirty vs master

Multi-threaded benchmarks (4 threads)

Dagger benchmarks: dirty vs master

Median time

master dirty master / dirty
array/dagger/N=1024 (block 128)/add (X + X) 24.9 ± 2.2 ms 22.8 ± 0.87 ms 1.09 ± 0.11
array/dagger/N=1024 (block 128)/alloc (rand) 15.6 ± 0.56 ms 16 ± 0.51 ms 0.975 ± 0.047
array/dagger/N=1024 (block 128)/broadcast (X .+ 1) 14 ± 1.1 ms 14 ± 0.47 ms 1 ± 0.083
array/dagger/N=1024 (block 128)/map (sin.(X)) 12.8 ± 1.7 ms 13.3 ± 0.79 ms 0.963 ± 0.14
array/dagger/N=1024 (block 128)/norm 15 ± 0.2 ms 15.2 ± 0.26 ms 0.987 ± 0.021
array/dagger/N=1024 (block 128)/reduce (sum) 31.1 ± 1.3 ms 30.8 ± 0.68 ms 1.01 ± 0.049
array/dagger/N=1024 (block 128)/transpose (permutedims) 16 ± 0.45 ms 16.5 ± 1.3 ms 0.968 ± 0.079
array/dagger/N=1024 (block 512)/add (X + X) 6.04 ± 3 ms 3.87 ± 1 ms 1.56 ± 0.88
array/dagger/N=1024 (block 512)/alloc (rand) 2.61 ± 0.88 ms 2.07 ± 0.71 ms 1.26 ± 0.6
array/dagger/N=1024 (block 512)/broadcast (X .+ 1) 2.68 ± 0.13 ms 2.65 ± 0.1 ms 1.01 ± 0.063
array/dagger/N=1024 (block 512)/map (sin.(X)) 5.46 ± 2.2 ms 6.72 ± 1.2 ms 0.812 ± 0.36
array/dagger/N=1024 (block 512)/norm 1.93 ± 0.51 ms 1.87 ± 0.13 ms 1.03 ± 0.28
array/dagger/N=1024 (block 512)/reduce (sum) 2.4 ± 0.54 ms 2.79 ± 1.1 ms 0.858 ± 0.38
array/dagger/N=1024 (block 512)/transpose (permutedims) 4.91 ± 1.8 ms 5.4 ± 1.1 ms 0.91 ± 0.39
array/dagger/N=256 (block 128)/add (X + X) 3.84 ± 2.5 ms 2.2 ± 1 ms 1.75 ± 1.4
array/dagger/N=256 (block 128)/alloc (rand) 1.91 ± 0.68 ms 1.58 ± 0.079 ms 1.21 ± 0.43
array/dagger/N=256 (block 128)/broadcast (X .+ 1) 1.7 ± 0.59 ms 1.32 ± 0.062 ms 1.29 ± 0.45
array/dagger/N=256 (block 128)/map (sin.(X)) 1.47 ± 0.43 ms 1.33 ± 0.29 ms 1.1 ± 0.41
array/dagger/N=256 (block 128)/norm 1.58 ± 2.8 ms 1.53 ± 1.8 ms 1.03 ± 2.2
array/dagger/N=256 (block 128)/reduce (sum) 2.97 ± 0.13 ms 3.71 ± 0.35 ms 0.801 ± 0.083
array/dagger/N=256 (block 128)/transpose (permutedims) 1.63 ± 2.2 ms 2.15 ± 0.24 ms 0.76 ± 1
array/dagger/N=256 (block 256)/add (X + X) 0.971 ± 0.099 ms 1.29 ± 0.097 ms 0.753 ± 0.095
array/dagger/N=256 (block 256)/alloc (rand) 0.923 ± 0.044 ms 1.27 ± 0.47 ms 0.729 ± 0.27
array/dagger/N=256 (block 256)/broadcast (X .+ 1) 0.661 ± 0.23 ms 0.483 ± 0.052 ms 1.37 ± 0.49
array/dagger/N=256 (block 256)/map (sin.(X)) 1.12 ± 0.082 ms 1.09 ± 0.13 ms 1.03 ± 0.14
array/dagger/N=256 (block 256)/norm 0.512 ± 0.049 ms 0.598 ± 0.088 ms 0.856 ± 0.15
array/dagger/N=256 (block 256)/reduce (sum) 0.689 ± 0.036 ms 0.897 ± 0.77 ms 0.767 ± 0.66
array/dagger/N=256 (block 256)/transpose (permutedims) 0.808 ± 0.14 ms 0.86 ± 0.034 ms 0.94 ± 0.17
linalg/dagger/N=1024 (block 128)/cholesky 0.0662 ± 0.0099 s 0.0779 ± 0.0074 s 0.85 ± 0.15
linalg/dagger/N=1024 (block 128)/lu 0.161 ± 0.013 s 0.156 ± 0.0098 s 1.03 ± 0.11
linalg/dagger/N=1024 (block 128)/matmul (A*A) 0.2 ± 0.016 s 0.172 ± 0.044 s 1.16 ± 0.32
linalg/dagger/N=1024 (block 128)/matvec (A*x) 19.3 ± 3.6 ms 21 ± 3.8 ms 0.918 ± 0.24
linalg/dagger/N=1024 (block 128)/qr 0.162 ± 0.0055 s 0.196 ± 0.025 s 0.823 ± 0.11
linalg/dagger/N=1024 (block 128)/solve (A\b via lu) 0.213 ± 0.023 s 0.205 ± 0.019 s 1.04 ± 0.15
linalg/dagger/N=1024 (block 128)/svd 30.1 s 29 s 1.04
linalg/dagger/N=1024 (block 128)/syrk (A'*A) 0.108 ± 0.0064 s 0.129 ± 0.025 s 0.837 ± 0.17
linalg/dagger/N=1024 (block 512)/cholesky 23.1 ± 5 ms 24.7 ± 4.7 ms 0.934 ± 0.27
linalg/dagger/N=1024 (block 512)/lu 0.0495 ± 0.0067 s 0.0428 ± 0.0021 s 1.16 ± 0.17
linalg/dagger/N=1024 (block 512)/matmul (A*A) 0.0506 ± 0.011 s 0.0587 ± 0.019 s 0.861 ± 0.34
linalg/dagger/N=1024 (block 512)/matvec (A*x) 3.16 ± 1.7 ms 2.24 ± 1 ms 1.41 ± 0.98
linalg/dagger/N=1024 (block 512)/qr 0.121 ± 0.0065 s 0.12 ± 0.0031 s 1.01 ± 0.061
linalg/dagger/N=1024 (block 512)/solve (A\b via lu) 0.0664 ± 0.0081 s 0.0572 ± 0.0062 s 1.16 ± 0.19
linalg/dagger/N=1024 (block 512)/svd 0.0355 h 0.0315 h 1.13
linalg/dagger/N=1024 (block 512)/syrk (A'*A) 0.0409 ± 0.0054 s 0.0404 ± 0.003 s 1.01 ± 0.15
linalg/dagger/N=256 (block 128)/cholesky 7.19 ± 3 ms 8.24 ± 6 ms 0.871 ± 0.73
linalg/dagger/N=256 (block 128)/lu 14.3 ± 6.2 ms 13.5 ± 2.9 ms 1.06 ± 0.51
linalg/dagger/N=256 (block 128)/matmul (A*A) 5.81 ± 4.5 ms 8.25 ± 2.7 ms 0.705 ± 0.59
linalg/dagger/N=256 (block 128)/matvec (A*x) 3 ± 0.8 ms 4.29 ± 1.2 ms 0.699 ± 0.28
linalg/dagger/N=256 (block 128)/qr 11.3 ± 4.8 ms 13.6 ± 4.6 ms 0.831 ± 0.45
linalg/dagger/N=256 (block 128)/solve (A\b via lu) 28.7 ± 12 ms 22.4 ± 7.2 ms 1.28 ± 0.7
linalg/dagger/N=256 (block 128)/svd 0.664 ± 0.023 s 0.551 ± 0.076 s 1.21 ± 0.17
linalg/dagger/N=256 (block 128)/syrk (A'*A) 8.95 ± 3.5 ms 8.15 ± 1 ms 1.1 ± 0.45
linalg/dagger/N=256 (block 256)/cholesky 7.81 ± 3.8 ms 5.49 ± 0.24 ms 1.42 ± 0.7
linalg/dagger/N=256 (block 256)/lu 6.74 ± 1.2 ms 4.72 ± 1.9 ms 1.43 ± 0.64
linalg/dagger/N=256 (block 256)/matmul (A*A) 2.59 ± 0.4 ms 2.85 ± 0.68 ms 0.909 ± 0.26
linalg/dagger/N=256 (block 256)/matvec (A*x) 1.12 ± 0.07 ms 1.53 ± 0.77 ms 0.729 ± 0.37
linalg/dagger/N=256 (block 256)/qr 7.35 ± 4.2 ms 6.94 ± 1.8 ms 1.06 ± 0.66
linalg/dagger/N=256 (block 256)/solve (A\b via lu) 14.1 ± 2.9 ms 10.2 ± 4.1 ms 1.38 ± 0.63
linalg/dagger/N=256 (block 256)/svd 0.554 ± 0.0093 s 0.508 ± 0.015 s 1.09 ± 0.037
linalg/dagger/N=256 (block 256)/syrk (A'*A) 4.02 ± 0.41 ms 4.17 ± 0.18 ms 0.964 ± 0.11
sparse/dagger/N=1024 (block 64)/cg solve (laplacian) 0.922 ± 0.028 s 0.937 ± 0.033 s 0.984 ± 0.046
sparse/dagger/N=1024 (block 64)/spgemm (S*S) 0.532 ± 0.0041 s 0.549 ± 0.0073 s 0.969 ± 0.015
sparse/dagger/N=1024 (block 64)/spmv (S*x) 0.0594 ± 0.0036 s 0.0511 ± 0.0027 s 1.16 ± 0.093
sparse/dagger/N=256 (block 16)/cg solve (laplacian) 0.875 ± 0.0037 s 0.967 ± 0.022 s 0.905 ± 0.021
sparse/dagger/N=256 (block 16)/spgemm (S*S) 0.526 ± 0.01 s 0.545 ± 0.0062 s 0.965 ± 0.022
sparse/dagger/N=256 (block 16)/spmv (S*x) 0.0601 ± 0.0078 s 0.0618 ± 0.0082 s 0.973 ± 0.18
stencil/dagger/N=1024 (block 128)/alloc (neighbors Wrap) 19.1 ± 0.67 ms 19.3 ± 0.75 ms 0.985 ± 0.052
stencil/dagger/N=1024 (block 128)/assign (const) 7.98 ± 0.98 ms 8.9 ± 1.1 ms 0.897 ± 0.15
stencil/dagger/N=1024 (block 128)/multi-expr 16.2 ± 0.31 ms 18.1 ± 0.97 ms 0.895 ± 0.051
stencil/dagger/N=1024 (block 128)/neighbors (Clamp) 14.3 ± 0.38 ms 13.5 ± 1.7 ms 1.05 ± 0.13
stencil/dagger/N=1024 (block 128)/neighbors (Pad) 14 ± 1.7 ms 15.6 ± 2.8 ms 0.896 ± 0.2
stencil/dagger/N=1024 (block 128)/neighbors (Reflect) 14.2 ± 1.1 ms 15.1 ± 1.9 ms 0.942 ± 0.14
stencil/dagger/N=1024 (block 128)/neighbors (Wrap) 14.8 ± 0.63 ms 15.3 ± 0.78 ms 0.973 ± 0.064
stencil/dagger/N=1024 (block 128)/update (+) 9.19 ± 0.96 ms 9.72 ± 0.52 ms 0.946 ± 0.11
stencil/dagger/N=1024 (block 512)/alloc (neighbors Wrap) 7.51 ± 1.8 ms 7.25 ± 0.28 ms 1.04 ± 0.25
stencil/dagger/N=1024 (block 512)/assign (const) 1.65 ± 0.58 ms 1.38 ± 0.8 ms 1.19 ± 0.81
stencil/dagger/N=1024 (block 512)/multi-expr 3.79 ± 1.8 ms 2.92 ± 0.71 ms 1.3 ± 0.69
stencil/dagger/N=1024 (block 512)/neighbors (Clamp) 6.49 ± 0.52 ms 5.71 ± 0.13 ms 1.14 ± 0.095
stencil/dagger/N=1024 (block 512)/neighbors (Pad) 7.74 ± 1.9 ms 7.93 ± 2.6 ms 0.976 ± 0.4
stencil/dagger/N=1024 (block 512)/neighbors (Reflect) 6.27 ± 1 ms 6.56 ± 0.31 ms 0.955 ± 0.17
stencil/dagger/N=1024 (block 512)/neighbors (Wrap) 6.46 ± 0.45 ms 6.98 ± 1.4 ms 0.925 ± 0.2
stencil/dagger/N=1024 (block 512)/update (+) 1.81 ± 2.6 ms 2.02 ± 0.64 ms 0.898 ± 1.3
stencil/dagger/N=256 (block 128)/alloc (neighbors Wrap) 2.91 ± 1.1 ms 2.63 ± 1.3 ms 1.1 ± 0.69
stencil/dagger/N=256 (block 128)/assign (const) 1.47 ± 0.44 ms 1.34 ± 1.1 ms 1.1 ± 0.95
stencil/dagger/N=256 (block 128)/multi-expr 4.82 ± 1.6 ms 3.48 ± 1.4 ms 1.38 ± 0.71
stencil/dagger/N=256 (block 128)/neighbors (Clamp) 2.09 ± 0.53 ms 2.32 ± 1 ms 0.898 ± 0.46
stencil/dagger/N=256 (block 128)/neighbors (Pad) 2.83 ± 1.6 ms 1.82 ± 0.41 ms 1.56 ± 0.92
stencil/dagger/N=256 (block 128)/neighbors (Reflect) 1.93 ± 1.4 ms 2.06 ± 2.6 ms 0.94 ± 1.4
stencil/dagger/N=256 (block 128)/neighbors (Wrap) 2.01 ± 1.5 ms 2.1 ± 2.7 ms 0.956 ± 1.4
stencil/dagger/N=256 (block 128)/update (+) 1.18 ± 0.13 ms 1.93 ± 0.96 ms 0.614 ± 0.31
stencil/dagger/N=256 (block 256)/alloc (neighbors Wrap) 1.9 ± 0.22 ms 2.32 ± 0.29 ms 0.818 ± 0.14
stencil/dagger/N=256 (block 256)/assign (const) 0.534 ± 0.072 ms 0.559 ± 0.13 ms 0.956 ± 0.26
stencil/dagger/N=256 (block 256)/multi-expr 1.87 ± 1.5 ms 1.59 ± 0.6 ms 1.18 ± 1.1
stencil/dagger/N=256 (block 256)/neighbors (Clamp) 1.6 ± 0.26 ms 2.24 ± 2 ms 0.716 ± 0.66
stencil/dagger/N=256 (block 256)/neighbors (Pad) 1.78 ± 0.34 ms 1.76 ± 0.062 ms 1.01 ± 0.2
stencil/dagger/N=256 (block 256)/neighbors (Reflect) 1.91 ± 0.19 ms 1.69 ± 0.19 ms 1.13 ± 0.17
stencil/dagger/N=256 (block 256)/neighbors (Wrap) 1.73 ± 0.37 ms 1.58 ± 1.6 ms 1.09 ± 1.1
stencil/dagger/N=256 (block 256)/update (+) 0.748 ± 0.91 ms 0.956 ± 0.33 ms 0.782 ± 0.99
time_to_load 1.13 ± 0.031 s 1.18 ± 0.014 s 0.957 ± 0.029

Allocations / memory

master dirty master / dirty
array/dagger/N=1024 (block 128)/add (X + X) 0.0788 M allocs: 11 MB 0.0787 M allocs: 11 MB 0.999
array/dagger/N=1024 (block 128)/alloc (rand) 0.0428 M allocs: 9.43 MB 0.0431 M allocs: 9.44 MB 0.999
array/dagger/N=1024 (block 128)/broadcast (X .+ 1) 0.0382 M allocs: 9.31 MB 0.0381 M allocs: 9.3 MB 1
array/dagger/N=1024 (block 128)/map (sin.(X)) 0.0328 M allocs: 9.1 MB 0.033 M allocs: 9.11 MB 0.999
array/dagger/N=1024 (block 128)/norm 0.0425 M allocs: 1.41 MB 0.0425 M allocs: 1.41 MB 0.999
array/dagger/N=1024 (block 128)/reduce (sum) 0.0854 M allocs: 2.91 MB 0.0867 M allocs: 2.95 MB 0.984
array/dagger/N=1024 (block 128)/transpose (permutedims) 0.0447 M allocs: 9.73 MB 0.0446 M allocs: 9.72 MB 1
array/dagger/N=1024 (block 512)/add (X + X) 5.59 k allocs: 8.22 MB 5.55 k allocs: 8.22 MB 1
array/dagger/N=1024 (block 512)/alloc (rand) 2.73 k allocs: 8.09 MB 2.76 k allocs: 8.09 MB 1
array/dagger/N=1024 (block 512)/broadcast (X .+ 1) 2.4 k allocs: 8.08 MB 2.41 k allocs: 8.08 MB 1
array/dagger/N=1024 (block 512)/map (sin.(X)) 2.09 k allocs: 8.07 MB 2.1 k allocs: 8.07 MB 1
array/dagger/N=1024 (block 512)/norm 2.7 k allocs: 0.0898 MB 2.71 k allocs: 0.0901 MB 0.997
array/dagger/N=1024 (block 512)/reduce (sum) 3.65 k allocs: 0.125 MB 3.62 k allocs: 0.124 MB 1.01
array/dagger/N=1024 (block 512)/transpose (permutedims) 3.31 k allocs: 8.14 MB 3.31 k allocs: 8.14 MB 1
array/dagger/N=256 (block 128)/add (X + X) 5.59 k allocs: 0.718 MB 5.59 k allocs: 0.718 MB 0.999
array/dagger/N=256 (block 128)/alloc (rand) 2.77 k allocs: 0.593 MB 2.76 k allocs: 0.593 MB 1
array/dagger/N=256 (block 128)/broadcast (X .+ 1) 2.43 k allocs: 0.583 MB 2.43 k allocs: 0.583 MB 1
array/dagger/N=256 (block 128)/map (sin.(X)) 2.1 k allocs: 0.571 MB 2.12 k allocs: 0.572 MB 0.999
array/dagger/N=256 (block 128)/norm 2.69 k allocs: 0.09 MB 2.68 k allocs: 0.0896 MB 1
array/dagger/N=256 (block 128)/reduce (sum) 4.1 k allocs: 0.141 MB 4.15 k allocs: 0.142 MB 0.988
array/dagger/N=256 (block 128)/transpose (permutedims) 3.33 k allocs: 0.636 MB 3.37 k allocs: 0.637 MB 0.998
array/dagger/N=256 (block 256)/add (X + X) 1.73 k allocs: 0.575 MB 1.73 k allocs: 0.575 MB 1
array/dagger/N=256 (block 256)/alloc (rand) 0.754 k allocs: 0.526 MB 0.753 k allocs: 0.526 MB 1
array/dagger/N=256 (block 256)/broadcast (X .+ 1) 0.64 k allocs: 0.522 MB 0.651 k allocs: 0.522 MB 0.999
array/dagger/N=256 (block 256)/map (sin.(X)) 0.574 k allocs: 0.52 MB 0.574 k allocs: 0.52 MB 1
array/dagger/N=256 (block 256)/norm 0.71 k allocs: 24.5 kB 0.704 k allocs: 24.2 kB 1.01
array/dagger/N=256 (block 256)/reduce (sum) 0.816 k allocs: 30.5 kB 0.816 k allocs: 30.5 kB 1
array/dagger/N=256 (block 256)/transpose (permutedims) 1.11 k allocs: 0.551 MB 1.11 k allocs: 0.551 MB 0.999
linalg/dagger/N=1024 (block 128)/cholesky 0.122 M allocs: 15.8 MB 0.123 M allocs: 15.9 MB 0.997
linalg/dagger/N=1024 (block 128)/lu 0.332 M allocs: 24.2 MB 0.329 M allocs: 24.3 MB 0.997
linalg/dagger/N=1024 (block 128)/matmul (A*A) 0.236 M allocs: 15.8 MB 0.237 M allocs: 15.8 MB 0.999
linalg/dagger/N=1024 (block 128)/matvec (A*x) 0.0704 M allocs: 2.61 MB 0.0687 M allocs: 2.55 MB 1.02
linalg/dagger/N=1024 (block 128)/qr 0.238 M allocs: 24.9 MB 0.254 M allocs: 25.5 MB 0.978
linalg/dagger/N=1024 (block 128)/solve (A\b via lu) 0.445 M allocs: 28.5 MB 0.437 M allocs: 28.3 MB 1.01
linalg/dagger/N=1024 (block 128)/svd 3.79 M allocs: 2.32 GB 3.19 M allocs: 2.29 GB 1.01
linalg/dagger/N=1024 (block 128)/syrk (A'*A) 0.173 M allocs: 17 MB 0.173 M allocs: 17.1 MB 1
linalg/dagger/N=1024 (block 512)/cholesky 7.82 k allocs: 10.3 MB 7.86 k allocs: 10.3 MB 1
linalg/dagger/N=1024 (block 512)/lu 16 k allocs: 14.7 MB 15.6 k allocs: 14.6 MB 1
linalg/dagger/N=1024 (block 512)/matmul (A*A) 7.66 k allocs: 8.28 MB 7.67 k allocs: 8.28 MB 1
linalg/dagger/N=1024 (block 512)/matvec (A*x) 6.38 k allocs: 0.253 MB 6.38 k allocs: 0.255 MB 0.995
linalg/dagger/N=1024 (block 512)/qr 11.5 k allocs: 9.59 MB 11.8 k allocs: 9.6 MB 0.999
linalg/dagger/N=1024 (block 512)/solve (A\b via lu) 30 k allocs: 15.2 MB 30.3 k allocs: 15.2 MB 0.999
linalg/dagger/N=1024 (block 512)/svd 0.0493 M allocs: 0.197 GB 0.0502 M allocs: 0.197 GB 1
linalg/dagger/N=1024 (block 512)/syrk (A'*A) 9.15 k allocs: 20.4 MB 9.15 k allocs: 20.4 MB 1
linalg/dagger/N=256 (block 128)/cholesky 7.89 k allocs: 0.938 MB 7.91 k allocs: 0.939 MB 0.999
linalg/dagger/N=256 (block 128)/lu 15.7 k allocs: 1.5 MB 15.6 k allocs: 1.49 MB 1
linalg/dagger/N=256 (block 128)/matmul (A*A) 7.83 k allocs: 0.79 MB 7.78 k allocs: 0.787 MB 1
linalg/dagger/N=256 (block 128)/matvec (A*x) 6.41 k allocs: 0.249 MB 6.36 k allocs: 0.248 MB 1
linalg/dagger/N=256 (block 128)/qr 11.6 k allocs: 1.24 MB 11.6 k allocs: 1.24 MB 0.998
linalg/dagger/N=256 (block 128)/solve (A\b via lu) 30.6 k allocs: 2.11 MB 30.2 k allocs: 2.08 MB 1.01
linalg/dagger/N=256 (block 128)/svd 0.0482 M allocs: 14.6 MB 0.0502 M allocs: 14.7 MB 0.995
linalg/dagger/N=256 (block 128)/syrk (A'*A) 9.18 k allocs: 1.61 MB 9.22 k allocs: 1.61 MB 1
linalg/dagger/N=256 (block 256)/cholesky 3.05 k allocs: 0.634 MB 3.05 k allocs: 0.634 MB 1
linalg/dagger/N=256 (block 256)/lu 5.41 k allocs: 1.24 MB 5.44 k allocs: 1.24 MB 0.999
linalg/dagger/N=256 (block 256)/matmul (A*A) 1.94 k allocs: 0.582 MB 1.95 k allocs: 0.582 MB 0.999
linalg/dagger/N=256 (block 256)/matvec (A*x) 2.45 k allocs: 0.104 MB 2.37 k allocs: 0.0998 MB 1.05
linalg/dagger/N=256 (block 256)/qr 3.46 k allocs: 0.78 MB 3.48 k allocs: 0.781 MB 0.999
linalg/dagger/N=256 (block 256)/solve (A\b via lu) 11.9 k allocs: 1.53 MB 12.1 k allocs: 1.54 MB 0.995
linalg/dagger/N=256 (block 256)/svd 15.4 k allocs: 6.71 MB 15.5 k allocs: 6.72 MB 0.999
linalg/dagger/N=256 (block 256)/syrk (A'*A) 3.07 k allocs: 2.14 MB 3.08 k allocs: 2.14 MB 1
sparse/dagger/N=1024 (block 64)/cg solve (laplacian) 3.09 M allocs: 0.114 GB 2.99 M allocs: 0.111 GB 1.03
sparse/dagger/N=1024 (block 64)/spgemm (S*S) 3.78 M allocs: 0.171 GB 3.88 M allocs: 0.175 GB 0.977
sparse/dagger/N=1024 (block 64)/spmv (S*x) 0.213 M allocs: 7.99 MB 0.184 M allocs: 7.01 MB 1.14
sparse/dagger/N=256 (block 16)/cg solve (laplacian) 3.08 M allocs: 0.113 GB 3.02 M allocs: 0.112 GB 1.02
sparse/dagger/N=256 (block 16)/spgemm (S*S) 3.77 M allocs: 0.134 GB 4.01 M allocs: 0.142 GB 0.943
sparse/dagger/N=256 (block 16)/spmv (S*x) 0.197 M allocs: 7.43 MB 0.189 M allocs: 7.15 MB 1.04
stencil/dagger/N=1024 (block 128)/alloc (neighbors Wrap) 0.0652 M allocs: 10.7 MB 0.0674 M allocs: 10.8 MB 0.991
stencil/dagger/N=1024 (block 128)/assign (const) 25.4 k allocs: 1.04 MB 25.5 k allocs: 1.05 MB 0.992
stencil/dagger/N=1024 (block 128)/multi-expr 0.0587 M allocs: 2.48 MB 0.0585 M allocs: 2.6 MB 0.954
stencil/dagger/N=1024 (block 128)/neighbors (Clamp) 0.0487 M allocs: 2.39 MB 0.0509 M allocs: 2.63 MB 0.907
stencil/dagger/N=1024 (block 128)/neighbors (Pad) 0.0482 M allocs: 2.5 MB 0.05 M allocs: 2.53 MB 0.988
stencil/dagger/N=1024 (block 128)/neighbors (Reflect) 0.0488 M allocs: 2.43 MB 0.0501 M allocs: 2.47 MB 0.985
stencil/dagger/N=1024 (block 128)/neighbors (Wrap) 0.0473 M allocs: 2.13 MB 0.0493 M allocs: 2.34 MB 0.907
stencil/dagger/N=1024 (block 128)/update (+) 0.0327 M allocs: 1.56 MB 0.0332 M allocs: 1.56 MB 0.999
stencil/dagger/N=1024 (block 512)/alloc (neighbors Wrap) 4.82 k allocs: 8.21 MB 4.92 k allocs: 8.21 MB 1
stencil/dagger/N=1024 (block 512)/assign (const) 2.18 k allocs: 0.0952 MB 2.14 k allocs: 0.0936 MB 1.02
stencil/dagger/N=1024 (block 512)/multi-expr 4.77 k allocs: 0.211 MB 4.81 k allocs: 0.211 MB 0.996
stencil/dagger/N=1024 (block 512)/neighbors (Clamp) 3.85 k allocs: 0.233 MB 3.93 k allocs: 0.235 MB 0.993
stencil/dagger/N=1024 (block 512)/neighbors (Pad) 3.69 k allocs: 0.228 MB 3.79 k allocs: 0.23 MB 0.992
stencil/dagger/N=1024 (block 512)/neighbors (Reflect) 3.83 k allocs: 0.265 MB 3.91 k allocs: 0.265 MB 1
stencil/dagger/N=1024 (block 512)/neighbors (Wrap) 3.63 k allocs: 0.167 MB 3.79 k allocs: 0.173 MB 0.965
stencil/dagger/N=1024 (block 512)/update (+) 2.62 k allocs: 0.116 MB 2.65 k allocs: 0.117 MB 0.992
stencil/dagger/N=256 (block 128)/alloc (neighbors Wrap) 4.82 k allocs: 0.705 MB 4.99 k allocs: 0.712 MB 0.99
stencil/dagger/N=256 (block 128)/assign (const) 2.17 k allocs: 0.0953 MB 2.15 k allocs: 0.0944 MB 1.01
stencil/dagger/N=256 (block 128)/multi-expr 4.84 k allocs: 0.213 MB 4.86 k allocs: 0.214 MB 0.997
stencil/dagger/N=256 (block 128)/neighbors (Clamp) 3.81 k allocs: 0.186 MB 3.95 k allocs: 0.19 MB 0.98
stencil/dagger/N=256 (block 128)/neighbors (Pad) 3.73 k allocs: 0.184 MB 3.76 k allocs: 0.184 MB 0.999
stencil/dagger/N=256 (block 128)/neighbors (Reflect) 3.79 k allocs: 0.193 MB 3.91 k allocs: 0.198 MB 0.978
stencil/dagger/N=256 (block 128)/neighbors (Wrap) 3.69 k allocs: 0.293 MB 3.77 k allocs: 0.174 MB 1.69
stencil/dagger/N=256 (block 128)/update (+) 2.65 k allocs: 0.117 MB 2.64 k allocs: 0.117 MB 1.01
stencil/dagger/N=256 (block 256)/alloc (neighbors Wrap) 1.55 k allocs: 0.575 MB 1.6 k allocs: 0.576 MB 0.997
stencil/dagger/N=256 (block 256)/assign (const) 0.811 k allocs: 0.041 MB 0.82 k allocs: 0.0413 MB 0.992
stencil/dagger/N=256 (block 256)/multi-expr 1.73 k allocs: 0.0873 MB 1.75 k allocs: 0.088 MB 0.993
stencil/dagger/N=256 (block 256)/neighbors (Clamp) 1.26 k allocs: 0.0718 MB 1.28 k allocs: 0.0719 MB 0.998
stencil/dagger/N=256 (block 256)/neighbors (Pad) 1.21 k allocs: 0.0703 MB 1.24 k allocs: 0.0708 MB 0.993
stencil/dagger/N=256 (block 256)/neighbors (Reflect) 1.27 k allocs: 0.0796 MB 1.29 k allocs: 0.08 MB 0.994
stencil/dagger/N=256 (block 256)/neighbors (Wrap) 1.2 k allocs: 0.0625 MB 1.23 k allocs: 0.0643 MB 0.973
stencil/dagger/N=256 (block 256)/update (+) 0.922 k allocs: 0.0463 MB 0.922 k allocs: 0.0463 MB 1
time_to_load 0.147 k allocs: 10.8 kB 0.147 k allocs: 10.8 kB 1

Plots

⚠️ Regressions (time > 25.0% and outside the reported ±spread; allocs/memory > 25.0%)

  • array/dagger/N=256 (block 256)/add (X + X) (time): +32.8%

Improvements

  • stencil/dagger/N=256 (block 128)/neighbors (Wrap) (memory): -40.9%
Within noise (18 metric(s) past threshold but inside the ±spread; not counted)
  • stencil/dagger/N=256 (block 128)/update (+) (time): 63.0%
  • linalg/dagger/N=256 (block 128)/matvec (A*x) (time): 43.1%
  • linalg/dagger/N=256 (block 128)/matmul (A*A) (time): 41.8%
  • stencil/dagger/N=256 (block 256)/neighbors (Clamp) (time): 39.6%
  • array/dagger/N=256 (block 256)/alloc (rand) (time): 37.2%
  • linalg/dagger/N=256 (block 256)/matvec (A*x) (time): 37.2%
  • array/dagger/N=256 (block 128)/transpose (permutedims) (time): 31.6%
  • array/dagger/N=256 (block 256)/reduce (sum) (time): 30.3%
  • stencil/dagger/N=256 (block 256)/update (+) (time): 27.9%
  • array/dagger/N=256 (block 256)/broadcast (X .+ 1) (time): -27.0%
  • linalg/dagger/N=256 (block 256)/solve (A\b via lu) (time): -27.6%
  • stencil/dagger/N=256 (block 128)/multi-expr (time): -27.7%
  • linalg/dagger/N=1024 (block 512)/matvec (A*x) (time): -29.2%
  • linalg/dagger/N=256 (block 256)/cholesky (time): -29.8%
  • linalg/dagger/N=256 (block 256)/lu (time): -30.1%
  • stencil/dagger/N=256 (block 128)/neighbors (Pad) (time): -35.7%
  • array/dagger/N=1024 (block 512)/add (X + X) (time): -35.8%
  • array/dagger/N=256 (block 128)/add (X + X) (time): -42.7%
Distributed benchmarks (4 processes)

Dagger benchmarks: dirty vs master

Median time

master dirty master / dirty
array/dagger/N=1024 (block 128)/add (X + X) 0.165 ± 0.033 s 0.141 ± 0.022 s 1.17 ± 0.3
array/dagger/N=1024 (block 128)/alloc (rand) 23.3 ± 0.8 ms 29.7 ± 0.27 ms 0.783 ± 0.028
array/dagger/N=1024 (block 128)/broadcast (X .+ 1) 0.0399 ± 0.00097 s 0.0407 ± 0.0026 s 0.982 ± 0.067
array/dagger/N=1024 (block 128)/map (sin.(X)) 0.0438 ± 0.0008 s 0.0421 ± 0.0018 s 1.04 ± 0.048
array/dagger/N=1024 (block 128)/norm 29 ± 2.4 ms 0.0326 ± 0.0026 s 0.888 ± 0.1
array/dagger/N=1024 (block 128)/reduce (sum) 0.0541 ± 0.0018 s 0.0543 ± 0.0011 s 0.995 ± 0.039
array/dagger/N=1024 (block 128)/transpose (permutedims) 0.0365 ± 0.0055 s 0.0481 ± 0.0022 s 0.758 ± 0.12
array/dagger/N=1024 (block 512)/add (X + X) 14.3 ± 2.6 ms 14.7 ± 0.69 ms 0.969 ± 0.19
array/dagger/N=1024 (block 512)/alloc (rand) 2.37 ± 0.036 ms 3.35 ± 0.16 ms 0.708 ± 0.035
array/dagger/N=1024 (block 512)/broadcast (X .+ 1) 4.3 ± 0.48 ms 4.44 ± 0.19 ms 0.968 ± 0.12
array/dagger/N=1024 (block 512)/map (sin.(X)) 7.38 ± 1.3 ms 8.5 ± 1.5 ms 0.868 ± 0.22
array/dagger/N=1024 (block 512)/norm 1.15 ± 0.2 ms 2.74 ± 0.3 ms 0.421 ± 0.088
array/dagger/N=1024 (block 512)/reduce (sum) 1.76 ± 1.1 ms 0.0335 ± 0.034 s 0.0526 ± 0.062
array/dagger/N=1024 (block 512)/transpose (permutedims) 8.36 ± 0.032 ms 8.17 ± 52 ms 1.02 ± 6.5
array/dagger/N=256 (block 128)/add (X + X) 1.6 ± 0.12 ms 1.56 ± 0.14 ms 1.03 ± 0.12
array/dagger/N=256 (block 128)/alloc (rand) 1.41 ± 0.033 ms 1.45 ± 0.057 ms 0.972 ± 0.045
array/dagger/N=256 (block 128)/broadcast (X .+ 1) 2.56 ± 0.46 ms 2.64 ± 0.52 ms 0.973 ± 0.26
array/dagger/N=256 (block 128)/map (sin.(X)) 2.08 ± 0.42 ms 1.9 ± 0.34 ms 1.1 ± 0.3
array/dagger/N=256 (block 128)/norm 0.864 ± 0.67 ms 0.863 ± 0.7 ms 1 ± 1.1
array/dagger/N=256 (block 128)/reduce (sum) 5.31 ± 35 ms 14.9 ± 34 ms 0.355 ± 2.5
array/dagger/N=256 (block 128)/transpose (permutedims) 1.12 ± 0.037 ms 1.11 ± 0.023 ms 1.01 ± 0.039
array/dagger/N=256 (block 256)/add (X + X) 0.781 ± 0.018 ms 0.821 ± 0.033 ms 0.951 ± 0.044
array/dagger/N=256 (block 256)/alloc (rand) 0.815 ± 0.018 ms 0.822 ± 0.029 ms 0.992 ± 0.042
array/dagger/N=256 (block 256)/broadcast (X .+ 1) 0.423 ± 0.11 ms 0.407 ± 0.11 ms 1.04 ± 0.39
array/dagger/N=256 (block 256)/map (sin.(X)) 0.905 ± 0.0058 ms 0.911 ± 0.0017 ms 0.993 ± 0.0067
array/dagger/N=256 (block 256)/norm 0.365 ± 0.037 ms 0.401 ± 0.032 ms 0.909 ± 0.12
array/dagger/N=256 (block 256)/reduce (sum) 0.517 ± 0.045 ms 0.56 ± 0.012 ms 0.924 ± 0.082
array/dagger/N=256 (block 256)/transpose (permutedims) 0.714 ± 0.022 ms 0.739 ± 0.018 ms 0.966 ± 0.038
linalg/dagger/N=1024 (block 128)/cholesky 0.487 ± 0.016 s 0.478 ± 0.0019 s 1.02 ± 0.034
linalg/dagger/N=1024 (block 128)/lu 0.963 ± 0.74 s 0.515 ± 0.29 s 1.87 ± 1.8
linalg/dagger/N=1024 (block 128)/matmul (A*A) 0.554 ± 0.29 s 0.744 ± 0.017 s 0.744 ± 0.39
linalg/dagger/N=1024 (block 128)/matvec (A*x) 0.112 ± 0.0048 s 0.113 ± 0.016 s 0.994 ± 0.14
linalg/dagger/N=1024 (block 128)/qr 24.8 s 0.288 ± 0.1 s 86.1
linalg/dagger/N=1024 (block 128)/solve (A\b via lu) 0.441 ± 0.17 s 0.365 ± 0.075 s 1.21 ± 0.53
linalg/dagger/N=1024 (block 128)/svd 0.0168 h 59.8 s 1.01
linalg/dagger/N=1024 (block 128)/syrk (A'*A) 0.397 ± 0.21 s 0.344 ± 0.11 s 1.15 ± 0.71
linalg/dagger/N=1024 (block 512)/cholesky 0.0815 ± 0.17 s 0.0695 ± 0.025 s 1.17 ± 2.5
linalg/dagger/N=1024 (block 512)/lu 0.0374 ± 0.002 s 0.0392 ± 0.00064 s 0.952 ± 0.054
linalg/dagger/N=1024 (block 512)/matmul (A*A) 0.0448 ± 0.0042 s 0.0382 ± 0.0021 s 1.17 ± 0.13
linalg/dagger/N=1024 (block 512)/matvec (A*x) 8.27 ± 0.027 ms 7.93 ± 0.2 ms 1.04 ± 0.026
linalg/dagger/N=1024 (block 512)/qr 0.1 ± 0.002 s 0.105 ± 0.0026 s 0.957 ± 0.03
linalg/dagger/N=1024 (block 512)/solve (A\b via lu) 0.0529 ± 0.0042 s 0.0531 ± 0.0044 s 0.996 ± 0.11
linalg/dagger/N=1024 (block 512)/svd 0.0355 h 0.0362 h 0.982
linalg/dagger/N=1024 (block 512)/syrk (A'*A) 0.032 ± 0.00024 s 0.0385 ± 0.0034 s 0.833 ± 0.075
linalg/dagger/N=256 (block 128)/cholesky 28.7 ± 2.2e+02 ms 25.5 ± 2.4e+02 ms 1.12 ± 14
linalg/dagger/N=256 (block 128)/lu 0.582 ± 0.29 s 13.6 ± 3e+02 ms 42.8 ± 9.4e+02
linalg/dagger/N=256 (block 128)/matmul (A*A) 5.91 ± 0.36 ms 5.71 ± 0.071 ms 1.04 ± 0.065
linalg/dagger/N=256 (block 128)/matvec (A*x) 1.89 ± 1.6 ms 1.87 ± 0.06 ms 1.01 ± 0.84
linalg/dagger/N=256 (block 128)/qr 7.66 ± 0.2 ms 7.5 ± 0.054 ms 1.02 ± 0.028
linalg/dagger/N=256 (block 128)/solve (A\b via lu) 0.511 ± 0.26 s 0.445 ± 0.21 s 1.15 ± 0.78
linalg/dagger/N=256 (block 128)/svd 0.794 ± 0.03 s 0.683 ± 0.0011 s 1.16 ± 0.044
linalg/dagger/N=256 (block 128)/syrk (A'*A) 5.55 ± 0.22 ms 4.98 ± 0.71 ms 1.12 ± 0.17
linalg/dagger/N=256 (block 256)/cholesky 2.08 ± 0.069 ms 2.01 ± 0.032 ms 1.03 ± 0.038
linalg/dagger/N=256 (block 256)/lu 3.09 ± 0.086 ms 2.99 ± 0.1 ms 1.03 ± 0.045
linalg/dagger/N=256 (block 256)/matmul (A*A) 1.79 ± 0.15 ms 1.71 ± 0.038 ms 1.04 ± 0.089
linalg/dagger/N=256 (block 256)/matvec (A*x) 0.989 ± 0.021 ms 0.958 ± 0.0066 ms 1.03 ± 0.023
linalg/dagger/N=256 (block 256)/qr 3.91 ± 0.16 ms 3.94 ± 0.24 ms 0.993 ± 0.074
linalg/dagger/N=256 (block 256)/solve (A\b via lu) 5.04 ± 0.14 ms 4.8 ± 0.12 ms 1.05 ± 0.039
linalg/dagger/N=256 (block 256)/svd 0.619 ± 0.0011 s 0.533 ± 0.0059 s 1.16 ± 0.013
linalg/dagger/N=256 (block 256)/syrk (A'*A) 2.56 ± 0.055 ms 2.55 ± 0.084 ms 1 ± 0.04
sparse/dagger/N=1024 (block 64)/cg solve (laplacian) 10.7 s 12.8 s 0.835
sparse/dagger/N=1024 (block 64)/spgemm (S*S) 6.82 s 6.36 s 1.07
sparse/dagger/N=1024 (block 64)/spmv (S*x) 0.547 ± 0.001 s 0.507 ± 0.018 s 1.08 ± 0.038
sparse/dagger/N=256 (block 16)/cg solve (laplacian) 12.8 s 10.8 s 1.18
sparse/dagger/N=256 (block 16)/spgemm (S*S) 6.62 s 6.22 s 1.06
sparse/dagger/N=256 (block 16)/spmv (S*x) 0.511 ± 0.0027 s 0.526 ± 0.036 s 0.971 ± 0.067
stencil/dagger/N=1024 (block 128)/assign (const) 29.1 ± 4.4 ms 0.033 ± 0.0015 s 0.881 ± 0.14
stencil/dagger/N=1024 (block 128)/multi-expr 0.207 ± 0.031 s 0.191 ± 0.016 s 1.08 ± 0.19
stencil/dagger/N=1024 (block 128)/update (+) 0.157 ± 0.011 s 0.166 ± 0.0091 s 0.948 ± 0.084
stencil/dagger/N=1024 (block 512)/assign (const) 2.85 ± 1.8 ms 1.7 ± 0.3 ms 1.68 ± 1.1
stencil/dagger/N=1024 (block 512)/multi-expr 13.9 ± 43 ms 14.4 ± 1.9 ms 0.968 ± 3
stencil/dagger/N=1024 (block 512)/update (+) 16.9 ± 5.3 ms 13.9 ± 2.1 ms 1.21 ± 0.42
stencil/dagger/N=256 (block 128)/assign (const) 0.835 ± 0.025 ms 0.931 ± 0.032 ms 0.896 ± 0.041
stencil/dagger/N=256 (block 128)/multi-expr 1.93 ± 0.11 ms 4.11 ± 0.16 ms 0.469 ± 0.033
stencil/dagger/N=256 (block 128)/neighbors (Clamp) 2.36 ± 0.28 ms
stencil/dagger/N=256 (block 128)/neighbors (Pad) 1.86 ± 0.0089 ms
stencil/dagger/N=256 (block 128)/neighbors (Reflect) 1.95 ± 0.11 ms 1.94 ± 0.04 ms 1.01 ± 0.062
stencil/dagger/N=256 (block 128)/update (+) 1.95 ± 0.22 ms 2.23 ± 0.24 ms 0.873 ± 0.14
stencil/dagger/N=256 (block 256)/assign (const) 0.535 ± 0.037 ms 0.594 ± 0.012 ms 0.901 ± 0.065
stencil/dagger/N=256 (block 256)/multi-expr 1.37 ± 0.062 ms 1.49 ± 0.17 ms 0.923 ± 0.11
stencil/dagger/N=256 (block 256)/neighbors (Clamp) 1.58 ± 0.11 ms
stencil/dagger/N=256 (block 256)/neighbors (Pad) 1.59 ± 0.059 ms
stencil/dagger/N=256 (block 256)/neighbors (Reflect) 1.67 ± 0.064 ms 1.53 ± 0.15 ms 1.09 ± 0.12
stencil/dagger/N=256 (block 256)/update (+) 1.03 ± 0.027 ms 0.769 ± 0.068 ms 1.34 ± 0.12
time_to_load 1.15 ± 0.02 s 1.15 ± 0.012 s 1 ± 0.02

Allocations / memory

master dirty master / dirty
array/dagger/N=1024 (block 128)/add (X + X) 0.262 M allocs: 25.1 MB 0.247 M allocs: 21 MB 1.19
array/dagger/N=1024 (block 128)/alloc (rand) 0.0532 M allocs: 6.9 MB 0.0627 M allocs: 5.34 MB 1.29
array/dagger/N=1024 (block 128)/broadcast (X .+ 1) 0.102 M allocs: 7.25 MB 0.0916 M allocs: 7.32 MB 0.99
array/dagger/N=1024 (block 128)/map (sin.(X)) 0.1 M allocs: 7.14 MB 0.0822 M allocs: 7.14 MB 1
array/dagger/N=1024 (block 128)/norm 0.0634 M allocs: 2.53 MB 0.0688 M allocs: 2.78 MB 0.908
array/dagger/N=1024 (block 128)/reduce (sum) 0.102 M allocs: 4.08 MB 0.107 M allocs: 4.4 MB 0.929
array/dagger/N=1024 (block 128)/transpose (permutedims) 0.0743 M allocs: 7.24 MB 0.0889 M allocs: 8.13 MB 0.89
array/dagger/N=1024 (block 512)/add (X + X) 12.1 k allocs: 20.6 MB 14.9 k allocs: 14.6 MB 1.41
array/dagger/N=1024 (block 512)/alloc (rand) 2.83 k allocs: 8.1 MB 3.88 k allocs: 4.15 MB 1.95
array/dagger/N=1024 (block 512)/broadcast (X .+ 1) 4.54 k allocs: 2.23 MB 4.18 k allocs: 2.25 MB 0.987
array/dagger/N=1024 (block 512)/map (sin.(X)) 4.71 k allocs: 2.21 MB 5.29 k allocs: 2.24 MB 0.988
array/dagger/N=1024 (block 512)/norm 2.7 k allocs: 0.0886 MB 3.59 k allocs: 0.135 MB 0.655
array/dagger/N=1024 (block 512)/reduce (sum) 3.63 k allocs: 0.122 MB 7 k allocs: 0.292 MB 0.418
array/dagger/N=1024 (block 512)/transpose (permutedims) 3.53 k allocs: 8.15 MB 6.52 k allocs: 2.35 MB 3.47
array/dagger/N=256 (block 128)/add (X + X) 6.16 k allocs: 0.74 MB 6.16 k allocs: 0.741 MB 1
array/dagger/N=256 (block 128)/alloc (rand) 2.83 k allocs: 0.595 MB 2.83 k allocs: 0.595 MB 1
array/dagger/N=256 (block 128)/broadcast (X .+ 1) 2.89 k allocs: 0.341 MB 2.98 k allocs: 0.359 MB 0.95
array/dagger/N=256 (block 128)/map (sin.(X)) 2.49 k allocs: 0.553 MB 2.44 k allocs: 0.56 MB 0.987
array/dagger/N=256 (block 128)/norm 2.7 k allocs: 0.0886 MB 2.7 k allocs: 0.0886 MB 1
array/dagger/N=256 (block 128)/reduce (sum) 5.21 k allocs: 0.197 MB 5.15 k allocs: 0.195 MB 1.01
array/dagger/N=256 (block 128)/transpose (permutedims) 3.52 k allocs: 0.646 MB 3.54 k allocs: 0.646 MB 0.999
array/dagger/N=256 (block 256)/add (X + X) 2.03 k allocs: 0.59 MB 2.04 k allocs: 0.591 MB 0.999
array/dagger/N=256 (block 256)/alloc (rand) 0.853 k allocs: 0.53 MB 0.853 k allocs: 0.53 MB 1
array/dagger/N=256 (block 256)/broadcast (X .+ 1) 0.648 k allocs: 0.522 MB 0.648 k allocs: 0.522 MB 1
array/dagger/N=256 (block 256)/map (sin.(X)) 0.571 k allocs: 0.519 MB 0.571 k allocs: 0.519 MB 1
array/dagger/N=256 (block 256)/norm 0.712 k allocs: 24.2 kB 0.712 k allocs: 24.2 kB 1
array/dagger/N=256 (block 256)/reduce (sum) 0.814 k allocs: 30.2 kB 0.814 k allocs: 30.2 kB 1
array/dagger/N=256 (block 256)/transpose (permutedims) 1.3 k allocs: 0.563 MB 1.31 k allocs: 0.563 MB 0.999
linalg/dagger/N=1024 (block 128)/cholesky 0.683 M allocs: 0.0453 GB 0.705 M allocs: 0.0414 GB 1.09
linalg/dagger/N=1024 (block 128)/lu 0.578 M allocs: 0.0404 GB 0.616 M allocs: 0.0385 GB 1.05
linalg/dagger/N=1024 (block 128)/matmul (A*A) 0.628 M allocs: 0.0351 GB 1.24 M allocs: 0.0743 GB 0.473
linalg/dagger/N=1024 (block 128)/matvec (A*x) 0.202 M allocs: 14.9 MB 0.179 M allocs: 9.82 MB 1.52
linalg/dagger/N=1024 (block 128)/qr 0.807 M allocs: 0.0642 GB 0.426 M allocs: 0.0347 GB 1.85
linalg/dagger/N=1024 (block 128)/solve (A\b via lu) 0.749 M allocs: 0.0462 GB 0.795 M allocs: 0.0463 GB 0.997
linalg/dagger/N=1024 (block 128)/svd 17.3 M allocs: 2.58 GB 19.5 M allocs: 2.65 GB 0.977
linalg/dagger/N=1024 (block 128)/syrk (A'*A) 0.442 M allocs: 0.0319 GB 0.499 M allocs: 31.6 MB 1.03
linalg/dagger/N=1024 (block 512)/cholesky 28.1 k allocs: 13.2 MB 9.98 k allocs: 9.56 MB 1.38
linalg/dagger/N=1024 (block 512)/lu 19.1 k allocs: 14.8 MB 19.4 k allocs: 14.8 MB 0.999
linalg/dagger/N=1024 (block 512)/matmul (A*A) 14.2 k allocs: 12.5 MB 14.5 k allocs: 10.5 MB 1.19
linalg/dagger/N=1024 (block 512)/matvec (A*x) 14.4 k allocs: 0.604 MB 14.4 k allocs: 0.595 MB 1.02
linalg/dagger/N=1024 (block 512)/qr 14 k allocs: 9.7 MB 13.3 k allocs: 9.66 MB 1
linalg/dagger/N=1024 (block 512)/solve (A\b via lu) 0.0428 M allocs: 15.8 MB 0.0435 M allocs: 15.8 MB 0.999
linalg/dagger/N=1024 (block 512)/svd 0.171 M allocs: 0.32 GB 0.187 M allocs: 0.311 GB 1.03
linalg/dagger/N=1024 (block 512)/syrk (A'*A) 11.8 k allocs: 20.5 MB 12 k allocs: 20.5 MB 1
linalg/dagger/N=256 (block 128)/cholesky 9.88 k allocs: 1.01 MB 10.5 k allocs: 1.04 MB 0.974
linalg/dagger/N=256 (block 128)/lu 26.5 k allocs: 1.96 MB 27 k allocs: 1.98 MB 0.993
linalg/dagger/N=256 (block 128)/matmul (A*A) 11.3 k allocs: 0.917 MB 11.6 k allocs: 0.925 MB 0.992
linalg/dagger/N=256 (block 128)/matvec (A*x) 7.06 k allocs: 0.275 MB 7.08 k allocs: 0.276 MB 0.998
linalg/dagger/N=256 (block 128)/qr 13.9 k allocs: 1.33 MB 13.9 k allocs: 1.33 MB 0.998
linalg/dagger/N=256 (block 128)/solve (A\b via lu) 0.0732 M allocs: 4.04 MB 0.0749 M allocs: 4.07 MB 0.991
linalg/dagger/N=256 (block 128)/svd 0.166 M allocs: 27.3 MB 0.172 M allocs: 27 MB 1.01
linalg/dagger/N=256 (block 128)/syrk (A'*A) 11.8 k allocs: 1.71 MB 12 k allocs: 1.71 MB 0.996
linalg/dagger/N=256 (block 256)/cholesky 3.52 k allocs: 0.661 MB 3.54 k allocs: 0.662 MB 0.999
linalg/dagger/N=256 (block 256)/lu 6.4 k allocs: 1.29 MB 6.43 k allocs: 1.29 MB 0.999
linalg/dagger/N=256 (block 256)/matmul (A*A) 2.25 k allocs: 0.598 MB 2.26 k allocs: 0.598 MB 0.999
linalg/dagger/N=256 (block 256)/matvec (A*x) 2.8 k allocs: 0.122 MB 2.81 k allocs: 0.123 MB 0.997
linalg/dagger/N=256 (block 256)/qr 4.01 k allocs: 0.81 MB 4.03 k allocs: 0.811 MB 0.999
linalg/dagger/N=256 (block 256)/solve (A\b via lu) 14.1 k allocs: 1.65 MB 14.2 k allocs: 1.65 MB 0.999
linalg/dagger/N=256 (block 256)/svd 17.7 k allocs: 6.84 MB 17.7 k allocs: 6.84 MB 1
linalg/dagger/N=256 (block 256)/syrk (A'*A) 3.57 k allocs: 2.16 MB 3.59 k allocs: 2.16 MB 1
sparse/dagger/N=1024 (block 64)/cg solve (laplacian) 17.3 M allocs: 0.759 GB 21.4 M allocs: 0.926 GB 0.819
sparse/dagger/N=1024 (block 64)/spgemm (S*S) 12.5 M allocs: 0.55 GB 11.8 M allocs: 0.524 GB 1.05
sparse/dagger/N=1024 (block 64)/spmv (S*x) 0.922 M allocs: 0.0381 GB 0.975 M allocs: 0.0391 GB 0.974
sparse/dagger/N=256 (block 16)/cg solve (laplacian) 20.2 M allocs: 0.877 GB 17.8 M allocs: 0.769 GB 1.14
sparse/dagger/N=256 (block 16)/spgemm (S*S) 11.9 M allocs: 0.506 GB 11.3 M allocs: 0.484 GB 1.05
sparse/dagger/N=256 (block 16)/spmv (S*x) 0.898 M allocs: 0.0366 GB 0.974 M allocs: 0.0385 GB 0.952
stencil/dagger/N=1024 (block 128)/assign (const) 0.0539 M allocs: 2.51 MB 0.0602 M allocs: 2.81 MB 0.893
stencil/dagger/N=1024 (block 128)/multi-expr 0.28 M allocs: 12.9 MB 0.322 M allocs: 14.7 MB 0.876
stencil/dagger/N=1024 (block 128)/update (+) 0.216 M allocs: 9.99 MB 0.241 M allocs: 11 MB 0.904
stencil/dagger/N=1024 (block 512)/assign (const) 2.46 k allocs: 0.109 MB 2.47 k allocs: 0.109 MB 0.997
stencil/dagger/N=1024 (block 512)/multi-expr 16.4 k allocs: 0.739 MB 17.2 k allocs: 0.764 MB 0.967
stencil/dagger/N=1024 (block 512)/update (+) 8.67 k allocs: 0.393 MB 13.1 k allocs: 0.595 MB 0.66
stencil/dagger/N=256 (block 128)/assign (const) 2.4 k allocs: 0.108 MB 2.43 k allocs: 0.109 MB 0.983
stencil/dagger/N=256 (block 128)/multi-expr 6.43 k allocs: 0.281 MB 7.98 k allocs: 0.34 MB 0.827
stencil/dagger/N=256 (block 128)/neighbors (Clamp) 5.34 k allocs: 0.251 MB
stencil/dagger/N=256 (block 128)/neighbors (Pad) 4.97 k allocs: 0.236 MB
stencil/dagger/N=256 (block 128)/neighbors (Reflect) 5.08 k allocs: 0.248 MB 5.19 k allocs: 0.25 MB 0.991
stencil/dagger/N=256 (block 128)/update (+) 3.62 k allocs: 0.162 MB 4.57 k allocs: 0.198 MB 0.82
stencil/dagger/N=256 (block 256)/assign (const) 1.07 k allocs: 0.056 MB 1.08 k allocs: 0.0558 MB 1
stencil/dagger/N=256 (block 256)/multi-expr 2.58 k allocs: 0.629 MB 2.93 k allocs: 0.642 MB 0.98
stencil/dagger/N=256 (block 256)/neighbors (Clamp) 1.82 k allocs: 0.598 MB
stencil/dagger/N=256 (block 256)/neighbors (Pad) 1.77 k allocs: 0.596 MB
stencil/dagger/N=256 (block 256)/neighbors (Reflect) 1.89 k allocs: 0.609 MB 1.85 k allocs: 0.607 MB 1
stencil/dagger/N=256 (block 256)/update (+) 1.42 k allocs: 0.57 MB 1.57 k allocs: 0.575 MB 0.991
time_to_load 0.147 k allocs: 10.8 kB 0.147 k allocs: 10.8 kB 1

⚠️ Regressions (time > 35.0% and outside the reported ±spread; allocs/memory > 25.0%)

  • array/dagger/N=1024 (block 512)/reduce (sum) (memory): +139.2%
  • array/dagger/N=1024 (block 512)/norm (time): +137.4%
  • stencil/dagger/N=256 (block 128)/multi-expr (time): +113.1%
  • linalg/dagger/N=1024 (block 128)/matmul (A*A) (memory): +111.5%
  • linalg/dagger/N=1024 (block 128)/matmul (A*A) (allocs): +97.2%
  • array/dagger/N=1024 (block 512)/reduce (sum) (allocs): +92.5%
  • array/dagger/N=1024 (block 512)/transpose (permutedims) (allocs): +84.8%
  • array/dagger/N=1024 (block 512)/norm (memory): +52.7%
  • stencil/dagger/N=1024 (block 512)/update (+) (memory): +51.6%
  • stencil/dagger/N=1024 (block 512)/update (+) (allocs): +50.6%
  • array/dagger/N=1024 (block 512)/alloc (rand) (time): +41.2%
  • array/dagger/N=1024 (block 512)/alloc (rand) (allocs): +36.9%
  • array/dagger/N=1024 (block 512)/norm (allocs): +33.0%
  • stencil/dagger/N=256 (block 128)/update (+) (allocs): +26.3%

Improvements

  • linalg/dagger/N=1024 (block 128)/qr (time): -98.8%
  • array/dagger/N=1024 (block 512)/transpose (permutedims) (memory): -71.2%
  • linalg/dagger/N=1024 (block 512)/cholesky (allocs): -64.4%
  • array/dagger/N=1024 (block 512)/alloc (rand) (memory): -48.7%
  • linalg/dagger/N=1024 (block 128)/qr (allocs): -47.2%
  • linalg/dagger/N=1024 (block 128)/qr (memory): -46.0%
  • linalg/dagger/N=1024 (block 128)/matvec (A*x) (memory): -34.2%
  • array/dagger/N=1024 (block 512)/add (X + X) (memory): -28.9%
  • linalg/dagger/N=1024 (block 512)/cholesky (memory): -27.6%
Within noise (5 metric(s) past threshold but inside the ±spread; not counted)
  • array/dagger/N=1024 (block 512)/reduce (sum) (time): 1801.3%
  • array/dagger/N=256 (block 128)/reduce (sum) (time): 181.5%
  • stencil/dagger/N=1024 (block 512)/assign (const) (time): -40.4%
  • linalg/dagger/N=1024 (block 128)/lu (time): -46.6%
  • linalg/dagger/N=256 (block 128)/lu (time): -97.7%
MPI benchmarks (4 ranks)

Dagger benchmarks: dirty vs master

Median time

master dirty master / dirty
array/dagger/N=1024 (block 128)/add (X + X) 0.184 ± 0.0009 s 0.137 ± 0.0014 s 1.35 ± 0.015
array/dagger/N=1024 (block 128)/alloc (rand) 0.0396 ± 0.0051 s 31.3 ± 0.22 ms 1.27 ± 0.16
array/dagger/N=1024 (block 128)/broadcast (X .+ 1) 15.6 ± 0.063 ms 16.8 ± 0.45 ms 0.929 ± 0.025
array/dagger/N=1024 (block 128)/map (sin.(X)) 17 ± 0.099 ms 18.3 ± 0.042 ms 0.928 ± 0.0058
array/dagger/N=1024 (block 128)/norm 17.9 ± 0.8 ms 18.8 ± 0.12 ms 0.951 ± 0.043
array/dagger/N=1024 (block 128)/reduce (sum) 30.8 ± 1 ms 0.0407 ± 0.0019 s 0.757 ± 0.044
array/dagger/N=1024 (block 128)/transpose (permutedims) 0.0926 ± 5.1e-05 s 0.0697 ± 0.00034 s 1.33 ± 0.0065
array/dagger/N=1024 (block 512)/add (X + X) 18.1 ± 0.042 ms 14.8 ± 0.091 ms 1.23 ± 0.0081
array/dagger/N=1024 (block 512)/alloc (rand) 0.0348 ± 0.00066 s 8.75 ± 4.9 ms 3.97 ± 2.2
array/dagger/N=1024 (block 512)/broadcast (X .+ 1) 2.77 ± 0.039 ms 2.11 ± 0.35 ms 1.31 ± 0.22
array/dagger/N=1024 (block 512)/map (sin.(X)) 4.64 ± 0.4 ms 4.72 ± 0.41 ms 0.983 ± 0.12
array/dagger/N=1024 (block 512)/norm 5.02 ± 0.00078 ms 5.17 ± 0.045 ms 0.97 ± 0.0084
array/dagger/N=1024 (block 512)/reduce (sum) 2.59 ± 0.0015 ms 2.81 ± 0.059 ms 0.923 ± 0.019
array/dagger/N=1024 (block 512)/transpose (permutedims) 14.8 ± 0.019 ms 12.8 ± 0.051 ms 1.16 ± 0.0048
array/dagger/N=256 (block 128)/add (X + X) 13.1 ± 0.19 ms 9.8 ± 0.016 ms 1.34 ± 0.02
array/dagger/N=256 (block 128)/alloc (rand) 22.1 ± 9.9 ms 7.01 ± 3.3 ms 3.15 ± 2.1
array/dagger/N=256 (block 128)/broadcast (X .+ 1) 1.23 ± 0.025 ms 1.31 ± 0.0066 ms 0.94 ± 0.02
array/dagger/N=256 (block 128)/map (sin.(X)) 1.37 ± 0.011 ms 1.45 ± 0.018 ms 0.946 ± 0.014
array/dagger/N=256 (block 128)/norm 1.61 ± 0.031 ms 1.67 ± 0.034 ms 0.963 ± 0.027
array/dagger/N=256 (block 128)/reduce (sum) 2.49 ± 0.04 ms 2.61 ± 0.022 ms 0.954 ± 0.017
array/dagger/N=256 (block 128)/transpose (permutedims) 6.8 ± 0.098 ms 5.33 ± 0.00057 ms 1.28 ± 0.018
array/dagger/N=256 (block 256)/add (X + X) 4.28 ± 0.1 ms 3.75 ± 0.018 ms 1.14 ± 0.027
array/dagger/N=256 (block 256)/alloc (rand) 30.3 ± 18 ms 1.56 ± 0.017 ms 19.4 ± 12
array/dagger/N=256 (block 256)/broadcast (X .+ 1) 0.626 ± 0.039 ms 0.574 ± 0.079 ms 1.09 ± 0.16
array/dagger/N=256 (block 256)/map (sin.(X)) 1.23 ± 0.0049 ms 1.22 ± 0.034 ms 1.01 ± 0.029
array/dagger/N=256 (block 256)/norm 1.16 ± 0.037 ms 1.13 ± 0.06 ms 1.03 ± 0.064
array/dagger/N=256 (block 256)/reduce (sum) 0.899 ± 0.037 ms 0.785 ± 0.031 ms 1.14 ± 0.065
array/dagger/N=256 (block 256)/transpose (permutedims) 3.21 ± 0.78 ms 2.06 ± 0.12 ms 1.56 ± 0.39
linalg/dagger/N=1024 (block 128)/cholesky 12.9 s 3.55 s 3.63
linalg/dagger/N=1024 (block 128)/lu 38.5 s 25 s 1.54
linalg/dagger/N=1024 (block 128)/matmul (A*A) 54.5 s 5.59 s 9.76
linalg/dagger/N=1024 (block 128)/matvec (A*x) 0.197 ± 0.0014 s 0.15 ± 0.00041 s 1.31 ± 0.0099
linalg/dagger/N=1024 (block 128)/qr 22.4 s 3.3 s 6.78
linalg/dagger/N=1024 (block 128)/solve (A\b via lu) 39.3 s 24 s 1.64
linalg/dagger/N=1024 (block 128)/syrk (A'*A) 30.9 s 3.25 s 9.51
linalg/dagger/N=1024 (block 512)/cholesky 0.445 ± 0.022 s 0.129 ± 0.029 s 3.45 ± 0.8
linalg/dagger/N=1024 (block 512)/lu 8.94 s 7.98 s 1.12
linalg/dagger/N=1024 (block 512)/matmul (A*A) 0.852 ± 0.012 s 0.218 ± 0.014 s 3.92 ± 0.25
linalg/dagger/N=1024 (block 512)/matvec (A*x) 21.9 ± 0.15 ms 17 ± 0.065 ms 1.29 ± 0.01
linalg/dagger/N=1024 (block 512)/qr 0.626 ± 0.017 s 0.46 ± 0.15 s 1.36 ± 0.45
linalg/dagger/N=1024 (block 512)/solve (A\b via lu) 8.63 s 7.91 s 1.09
linalg/dagger/N=1024 (block 512)/syrk (A'*A) 0.71 ± 0.0047 s 0.181 ± 0.016 s 3.92 ± 0.36
linalg/dagger/N=256 (block 128)/cholesky 0.412 ± 0.016 s 0.104 ± 0.015 s 3.94 ± 0.57
linalg/dagger/N=256 (block 128)/lu 1.79 ± 0.0096 s 1.46 ± 0.033 s 1.23 ± 0.029
linalg/dagger/N=256 (block 128)/matmul (A*A) 0.812 ± 0.00088 s 0.178 ± 0.011 s 4.57 ± 0.28
linalg/dagger/N=256 (block 128)/matvec (A*x) 19.6 ± 0.46 ms 15.3 ± 0.0084 ms 1.28 ± 0.03
linalg/dagger/N=256 (block 128)/qr 0.638 ± 0.049 s 0.203 ± 0.0018 s 3.15 ± 0.24
linalg/dagger/N=256 (block 128)/solve (A\b via lu) 1.83 ± 0.00039 s 1.44 ± 0.0035 s 1.27 ± 0.0031
linalg/dagger/N=256 (block 128)/syrk (A'*A) 0.663 ± 0.0038 s 0.155 ± 0.0015 s 4.28 ± 0.047
linalg/dagger/N=256 (block 256)/cholesky 0.0562 ± 0.049 s 0.133 ± 0.072 s 0.421 ± 0.43
linalg/dagger/N=256 (block 256)/lu 0.668 ± 0.00014 s 0.517 ± 0.00052 s 1.29 ± 0.0013
linalg/dagger/N=256 (block 256)/matmul (A*A) 8.99 ± 3.8 ms 8.79 ± 4.1 ms 1.02 ± 0.65
linalg/dagger/N=256 (block 256)/matvec (A*x) 5.05 ± 0.17 ms 4.16 ± 0.027 ms 1.22 ± 0.041
linalg/dagger/N=256 (block 256)/qr 10.4 ± 0.92 ms 9.94 ± 0.98 ms 1.04 ± 0.14
linalg/dagger/N=256 (block 256)/solve (A\b via lu) 0.679 ± 0.00019 s 0.527 ± 0.00049 s 1.29 ± 0.0013
linalg/dagger/N=256 (block 256)/syrk (A'*A) 0.111 ± 0.00035 s 9.58 ± 0.78 ms 11.6 ± 0.94
sparse/dagger/N=1024 (block 64)/cg solve (laplacian) 13.8 s 23.9 s 0.579
sparse/dagger/N=1024 (block 64)/spgemm (S*S) 6.74 s 6.23 s 1.08
sparse/dagger/N=1024 (block 64)/spmv (S*x) 0.681 ± 0.00085 s 0.524 ± 0.0061 s 1.3 ± 0.015
sparse/dagger/N=256 (block 16)/cg solve (laplacian) 13 s 20.6 s 0.63
sparse/dagger/N=256 (block 16)/spgemm (S*S) 6.74 s 5.82 s 1.16
sparse/dagger/N=256 (block 16)/spmv (S*x) 0.678 ± 0.00067 s 0.522 ± 0.001 s 1.3 ± 0.0028
stencil/dagger/N=1024 (block 128)/alloc (neighbors Wrap) 0.67 ± 0.00048 s 3.21 s 0.209
stencil/dagger/N=1024 (block 128)/assign (const) 0.082 ± 0.00013 s 0.0606 ± 0.0005 s 1.35 ± 0.011
stencil/dagger/N=1024 (block 128)/multi-expr 0.22 ± 0.0004 s 0.164 ± 0.0012 s 1.34 ± 0.01
stencil/dagger/N=1024 (block 128)/neighbors (Clamp) 0.572 ± 0.00048 s 0.946 ± 0.092 s 0.604 ± 0.059
stencil/dagger/N=1024 (block 128)/neighbors (Pad) 0.573 ± 8.8e-05 s 0.794 ± 0.00037 s 0.722 ± 0.00035
stencil/dagger/N=1024 (block 128)/neighbors (Reflect) 0.571 ± 0.00042 s 0.83 ± 0.0038 s 0.687 ± 0.0032
stencil/dagger/N=1024 (block 128)/neighbors (Wrap) 0.639 ± 5.9e-06 s 0.964 ± 0.014 s 0.663 ± 0.0095
stencil/dagger/N=1024 (block 128)/update (+) 0.138 ± 4.8e-05 s 0.104 ± 0.0013 s 1.33 ± 0.016
stencil/dagger/N=1024 (block 512)/alloc (neighbors Wrap) 0.066 ± 0.00014 s 0.0861 ± 0.002 s 0.766 ± 0.018
stencil/dagger/N=1024 (block 512)/assign (const) 6.41 ± 0.052 ms 5.05 ± 0.006 ms 1.27 ± 0.01
stencil/dagger/N=1024 (block 512)/multi-expr 16.2 ± 0.075 ms 12.3 ± 0.0044 ms 1.32 ± 0.0061
stencil/dagger/N=1024 (block 512)/neighbors (Clamp) 0.0634 ± 0.00013 s 0.0606 ± 0.0002 s 1.05 ± 0.0041
stencil/dagger/N=1024 (block 512)/neighbors (Pad) 0.0635 ± 8.6e-05 s 0.046 ± 0.00022 s 1.38 ± 0.0067
stencil/dagger/N=1024 (block 512)/neighbors (Reflect) 0.0637 ± 4e-05 s 0.0596 ± 0.00016 s 1.07 ± 0.0029
stencil/dagger/N=1024 (block 512)/neighbors (Wrap) 0.0646 ± 0.00047 s 0.0777 ± 9e-05 s 0.831 ± 0.0061
stencil/dagger/N=1024 (block 512)/update (+) 10.8 ± 0.2 ms 8.07 ± 0.059 ms 1.34 ± 0.026
stencil/dagger/N=256 (block 128)/alloc (neighbors Wrap) 0.0378 ± 7.4e-05 s 0.0692 ± 0.00045 s 0.546 ± 0.0037
stencil/dagger/N=256 (block 128)/assign (const) 6.22 ± 0.051 ms 4.65 ± 0.039 ms 1.34 ± 0.016
stencil/dagger/N=256 (block 128)/multi-expr 15.2 ± 0.094 ms 11.4 ± 0.077 ms 1.33 ± 0.012
stencil/dagger/N=256 (block 128)/neighbors (Clamp) 0.0346 ± 0.00022 s 0.0464 ± 4.7e-05 s 0.747 ± 0.0048
stencil/dagger/N=256 (block 128)/neighbors (Pad) 0.0344 ± 3.1e-05 s 0.0321 ± 0.00019 s 1.07 ± 0.0065
stencil/dagger/N=256 (block 128)/neighbors (Reflect) 0.0346 ± 9.3e-05 s 0.0473 ± 0.00078 s 0.732 ± 0.012
stencil/dagger/N=256 (block 128)/neighbors (Wrap) 0.0346 ± 3e-05 s 0.067 ± 0.00043 s 0.516 ± 0.0033
stencil/dagger/N=256 (block 128)/update (+) 9.44 ± 0.074 ms 6.98 ± 0.0083 ms 1.35 ± 0.011
stencil/dagger/N=256 (block 256)/alloc (neighbors Wrap) 5.96 ± 0.015 ms 5.26 ± 0.15 ms 1.13 ± 0.033
stencil/dagger/N=256 (block 256)/assign (const) 2.13 ± 0.033 ms 1.98 ± 0.11 ms 1.07 ± 0.064
stencil/dagger/N=256 (block 256)/multi-expr 4.72 ± 0.19 ms 3.77 ± 0.19 ms 1.25 ± 0.08
stencil/dagger/N=256 (block 256)/neighbors (Clamp) 4.59 ± 0.059 ms 3.86 ± 0.017 ms 1.19 ± 0.016
stencil/dagger/N=256 (block 256)/neighbors (Pad) 5.35 ± 0.6 ms 3.85 ± 0.0017 ms 1.39 ± 0.15
stencil/dagger/N=256 (block 256)/neighbors (Reflect) 4.88 ± 0.076 ms 3.92 ± 0.056 ms 1.25 ± 0.026
stencil/dagger/N=256 (block 256)/neighbors (Wrap) 4.83 ± 0.0091 ms 3.84 ± 0.0027 ms 1.26 ± 0.0025
stencil/dagger/N=256 (block 256)/update (+) 2.95 ± 0.0083 ms 2.74 ± 0.31 ms 1.08 ± 0.12
time_to_load 0.955 ± 0.0031 s 0.969 ± 0.0063 s 0.985 ± 0.0072

Allocations / memory

master dirty master / dirty
array/dagger/N=1024 (block 128)/add (X + X) 0.699 M allocs: 0.0363 GB 0.7 M allocs: 0.0363 GB 1
array/dagger/N=1024 (block 128)/alloc (rand) 0.117 M allocs: 7.36 MB 0.117 M allocs: 7.37 MB 0.999
array/dagger/N=1024 (block 128)/broadcast (X .+ 1) 0.0608 M allocs: 4.29 MB 0.0652 M allocs: 4.35 MB 0.987
array/dagger/N=1024 (block 128)/map (sin.(X)) 0.0549 M allocs: 4.07 MB 0.0592 M allocs: 4.12 MB 0.988
array/dagger/N=1024 (block 128)/norm 0.0738 M allocs: 2.84 MB 0.0768 M allocs: 2.89 MB 0.982
array/dagger/N=1024 (block 128)/reduce (sum) 0.113 M allocs: 4.28 MB 0.118 M allocs: 4.3 MB 0.995
array/dagger/N=1024 (block 128)/transpose (permutedims) 0.331 M allocs: 18.6 MB 0.331 M allocs: 18.6 MB 1
array/dagger/N=1024 (block 512)/add (X + X) 0.0481 M allocs: 4.41 MB 0.0481 M allocs: 4.42 MB 1
array/dagger/N=1024 (block 512)/alloc (rand) 9.36 k allocs: 2.44 MB 9.38 k allocs: 2.44 MB 1
array/dagger/N=1024 (block 512)/broadcast (X .+ 1) 3.71 k allocs: 2.14 MB 3.83 k allocs: 2.14 MB 0.998
array/dagger/N=1024 (block 512)/map (sin.(X)) 3.35 k allocs: 2.12 MB 3.38 k allocs: 2.12 MB 1
array/dagger/N=1024 (block 512)/norm 4.69 k allocs: 0.18 MB 4.82 k allocs: 0.184 MB 0.975
array/dagger/N=1024 (block 512)/reduce (sum) 6.51 k allocs: 0.244 MB 6.6 k allocs: 0.246 MB 0.99
array/dagger/N=1024 (block 512)/transpose (permutedims) 23 k allocs: 3.16 MB 23.1 k allocs: 3.16 MB 1
array/dagger/N=256 (block 128)/add (X + X) 0.0481 M allocs: 2.54 MB 0.0481 M allocs: 2.54 MB 1
array/dagger/N=256 (block 128)/alloc (rand) 9.35 k allocs: 0.563 MB 9.38 k allocs: 0.564 MB 0.999
array/dagger/N=256 (block 128)/broadcast (X .+ 1) 3.7 k allocs: 0.262 MB 3.73 k allocs: 0.263 MB 0.998
array/dagger/N=256 (block 128)/map (sin.(X)) 3.46 k allocs: 0.254 MB 3.43 k allocs: 0.252 MB 1.01
array/dagger/N=256 (block 128)/norm 4.84 k allocs: 0.186 MB 4.71 k allocs: 0.18 MB 1.03
array/dagger/N=256 (block 128)/reduce (sum) 6.51 k allocs: 0.244 MB 6.49 k allocs: 0.242 MB 1.01
array/dagger/N=256 (block 128)/transpose (permutedims) 23 k allocs: 1.28 MB 23.1 k allocs: 1.28 MB 1
array/dagger/N=256 (block 256)/add (X + X) 15.2 k allocs: 1.27 MB 15.2 k allocs: 1.27 MB 0.999
array/dagger/N=256 (block 256)/alloc (rand) 3.94 k allocs: 0.69 MB 3.95 k allocs: 0.69 MB 1
array/dagger/N=256 (block 256)/broadcast (X .+ 1) 1.03 k allocs: 0.538 MB 0.976 k allocs: 0.536 MB 1
array/dagger/N=256 (block 256)/map (sin.(X)) 0.886 k allocs: 0.533 MB 0.888 k allocs: 0.533 MB 1
array/dagger/N=256 (block 256)/norm 1.28 k allocs: 0.05 MB 1.44 k allocs: 0.0566 MB 0.884
array/dagger/N=256 (block 256)/reduce (sum) 1.29 k allocs: 0.05 MB 1.28 k allocs: 0.0499 MB 1
array/dagger/N=256 (block 256)/transpose (permutedims) 7.26 k allocs: 0.872 MB 7.27 k allocs: 0.872 MB 1
linalg/dagger/N=1024 (block 128)/cholesky 1.62 M allocs: 0.0857 GB 1.67 M allocs: 0.0835 GB 1.03
linalg/dagger/N=1024 (block 128)/lu 0.0525 G allocs: 2.45 GB 0.0573 G allocs: 2.6 GB 0.942
linalg/dagger/N=1024 (block 128)/matmul (A*A) 3.46 M allocs: 0.174 GB 3.64 M allocs: 0.174 GB 1
linalg/dagger/N=1024 (block 128)/matvec (A*x) 0.698 M allocs: 0.0333 GB 0.701 M allocs: 0.0334 GB 0.998
linalg/dagger/N=1024 (block 128)/qr 2.46 M allocs: 0.124 GB 2.55 M allocs: 0.125 GB 0.995
linalg/dagger/N=1024 (block 128)/solve (A\b via lu) 0.0537 G allocs: 2.5 GB 0.0585 G allocs: 2.65 GB 0.944
linalg/dagger/N=1024 (block 128)/syrk (A'*A) 2.63 M allocs: 0.136 GB 2.71 M allocs: 0.132 GB 1.02
linalg/dagger/N=1024 (block 512)/cholesky 0.0921 M allocs: 8.61 MB 0.0939 M allocs: 6.68 MB 1.29
linalg/dagger/N=1024 (block 512)/lu 19.7 M allocs: 0.918 GB 21.1 M allocs: 0.953 GB 0.963
linalg/dagger/N=1024 (block 512)/matmul (A*A) 0.103 M allocs: 11.1 MB 0.107 M allocs: 7.26 MB 1.53
linalg/dagger/N=1024 (block 512)/matvec (A*x) 0.0675 M allocs: 5.35 MB 0.0682 M allocs: 3.37 MB 1.59
linalg/dagger/N=1024 (block 512)/qr 0.128 M allocs: 8.66 MB 0.132 M allocs: 8.78 MB 0.986
linalg/dagger/N=1024 (block 512)/solve (A\b via lu) 19.9 M allocs: 0.931 GB 21.2 M allocs: 0.964 GB 0.965
linalg/dagger/N=1024 (block 512)/syrk (A'*A) 0.102 M allocs: 15.1 MB 0.104 M allocs: 13.2 MB 1.15
linalg/dagger/N=256 (block 128)/cholesky 0.0918 M allocs: 4.86 MB 0.0939 M allocs: 4.8 MB 1.01
linalg/dagger/N=256 (block 128)/lu 5.05 M allocs: 0.232 GB 5.39 M allocs: 0.243 GB 0.956
linalg/dagger/N=256 (block 128)/matmul (A*A) 0.103 M allocs: 5.52 MB 0.107 M allocs: 5.39 MB 1.02
linalg/dagger/N=256 (block 128)/matvec (A*x) 0.0675 M allocs: 3.47 MB 0.0682 M allocs: 3.37 MB 1.03
linalg/dagger/N=256 (block 128)/qr 0.128 M allocs: 6.59 MB 0.132 M allocs: 6.71 MB 0.983
linalg/dagger/N=256 (block 128)/solve (A\b via lu) 5.19 M allocs: 0.239 GB 5.52 M allocs: 0.249 GB 0.958
linalg/dagger/N=256 (block 128)/syrk (A'*A) 0.102 M allocs: 5.76 MB 0.105 M allocs: 5.72 MB 1.01
linalg/dagger/N=256 (block 256)/cholesky 25.6 k allocs: 1.82 MB 25.7 k allocs: 1.82 MB 0.999
linalg/dagger/N=256 (block 256)/lu 2.36 M allocs: 0.109 GB 2.36 M allocs: 0.109 GB 1
linalg/dagger/N=256 (block 256)/matmul (A*A) 15.7 k allocs: 1.29 MB 15.7 k allocs: 1.3 MB 1
linalg/dagger/N=256 (block 256)/matvec (A*x) 19.1 k allocs: 0.968 MB 19.2 k allocs: 0.971 MB 0.996
linalg/dagger/N=256 (block 256)/qr 29.5 k allocs: 2.13 MB 29.5 k allocs: 2.13 MB 0.999
linalg/dagger/N=256 (block 256)/solve (A\b via lu) 2.41 M allocs: 0.111 GB 2.41 M allocs: 0.111 GB 1
linalg/dagger/N=256 (block 256)/syrk (A'*A) 23 k allocs: 3.17 MB 23 k allocs: 3.17 MB 1
sparse/dagger/N=1024 (block 64)/cg solve (laplacian) 0.0321 G allocs: 1.54 GB 0.0322 G allocs: 1.54 GB 0.999
sparse/dagger/N=1024 (block 64)/spgemm (S*S) 20 M allocs: 0.944 GB 21.9 M allocs: 1.02 GB 0.93
sparse/dagger/N=1024 (block 64)/spmv (S*x) 2.42 M allocs: 0.116 GB 2.42 M allocs: 0.116 GB 0.998
sparse/dagger/N=256 (block 16)/cg solve (laplacian) 0.0321 G allocs: 1.54 GB 0.0322 G allocs: 1.54 GB 0.998
sparse/dagger/N=256 (block 16)/spgemm (S*S) 19.9 M allocs: 0.928 GB 21.9 M allocs: 0.994 GB 0.933
sparse/dagger/N=256 (block 16)/spmv (S*x) 2.42 M allocs: 0.116 GB 2.42 M allocs: 0.116 GB 0.999
stencil/dagger/N=1024 (block 128)/alloc (neighbors Wrap) 2.18 M allocs: 0.112 GB 4 M allocs: 0.198 GB 0.566
stencil/dagger/N=1024 (block 128)/assign (const) 0.305 M allocs: 15.6 MB 0.305 M allocs: 15.6 MB 1
stencil/dagger/N=1024 (block 128)/multi-expr 0.848 M allocs: 0.0425 GB 0.849 M allocs: 0.0425 GB 0.999
stencil/dagger/N=1024 (block 128)/neighbors (Clamp) 1.88 M allocs: 0.0948 GB 3.54 M allocs: 0.173 GB 0.549
stencil/dagger/N=1024 (block 128)/neighbors (Pad) 1.88 M allocs: 0.0948 GB 3.36 M allocs: 0.162 GB 0.583
stencil/dagger/N=1024 (block 128)/neighbors (Reflect) 1.88 M allocs: 0.0948 GB 3.51 M allocs: 0.172 GB 0.55
stencil/dagger/N=1024 (block 128)/neighbors (Wrap) 2.09 M allocs: 0.105 GB 3.91 M allocs: 0.191 GB 0.552
stencil/dagger/N=1024 (block 128)/update (+) 0.543 M allocs: 27.9 MB 0.543 M allocs: 27.9 MB 1
stencil/dagger/N=1024 (block 512)/alloc (neighbors Wrap) 0.123 M allocs: 14.2 MB 0.319 M allocs: 17.9 MB 0.792
stencil/dagger/N=1024 (block 512)/assign (const) 21.4 k allocs: 1.1 MB 21.5 k allocs: 1.1 MB 1
stencil/dagger/N=1024 (block 512)/multi-expr 0.0577 M allocs: 2.95 MB 0.0578 M allocs: 2.95 MB 0.999
stencil/dagger/N=1024 (block 512)/neighbors (Clamp) 0.115 M allocs: 11.8 MB 0.213 M allocs: 10.7 MB 1.1
stencil/dagger/N=1024 (block 512)/neighbors (Pad) 0.115 M allocs: 11.8 MB 0.149 M allocs: 7.48 MB 1.58
stencil/dagger/N=1024 (block 512)/neighbors (Reflect) 0.115 M allocs: 11.8 MB 0.213 M allocs: 10.7 MB 1.1
stencil/dagger/N=1024 (block 512)/neighbors (Wrap) 0.115 M allocs: 11.8 MB 0.311 M allocs: 15.5 MB 0.76
stencil/dagger/N=1024 (block 512)/update (+) 0.0362 M allocs: 1.86 MB 0.0363 M allocs: 1.86 MB 1
stencil/dagger/N=256 (block 128)/alloc (neighbors Wrap) 0.123 M allocs: 6.68 MB 0.319 M allocs: 16 MB 0.417
stencil/dagger/N=256 (block 128)/assign (const) 21.4 k allocs: 1.09 MB 21.5 k allocs: 1.1 MB 1
stencil/dagger/N=256 (block 128)/multi-expr 0.0577 M allocs: 2.95 MB 0.0578 M allocs: 2.95 MB 0.999
stencil/dagger/N=256 (block 128)/neighbors (Clamp) 0.115 M allocs: 6.17 MB 0.213 M allocs: 10.7 MB 0.577
stencil/dagger/N=256 (block 128)/neighbors (Pad) 0.115 M allocs: 6.17 MB 0.148 M allocs: 7.46 MB 0.826
stencil/dagger/N=256 (block 128)/neighbors (Reflect) 0.115 M allocs: 6.17 MB 0.213 M allocs: 10.7 MB 0.577
stencil/dagger/N=256 (block 128)/neighbors (Wrap) 0.115 M allocs: 6.16 MB 0.311 M allocs: 15.5 MB 0.398
stencil/dagger/N=256 (block 128)/update (+) 0.0362 M allocs: 1.86 MB 0.0363 M allocs: 1.86 MB 1
stencil/dagger/N=256 (block 256)/alloc (neighbors Wrap) 18.6 k allocs: 1.45 MB 18.7 k allocs: 1.45 MB 0.998
stencil/dagger/N=256 (block 256)/assign (const) 6.86 k allocs: 0.356 MB 6.87 k allocs: 0.356 MB 0.999
stencil/dagger/N=256 (block 256)/multi-expr 17.5 k allocs: 0.906 MB 17.5 k allocs: 0.907 MB 0.999
stencil/dagger/N=256 (block 256)/neighbors (Clamp) 15.1 k allocs: 0.777 MB 15.1 k allocs: 0.778 MB 0.999
stencil/dagger/N=256 (block 256)/neighbors (Pad) 15.1 k allocs: 0.775 MB 15.1 k allocs: 0.776 MB 0.999
stencil/dagger/N=256 (block 256)/neighbors (Reflect) 15.1 k allocs: 0.785 MB 15.1 k allocs: 0.785 MB 0.999
stencil/dagger/N=256 (block 256)/neighbors (Wrap) 15 k allocs: 0.768 MB 15.1 k allocs: 0.77 MB 0.998
stencil/dagger/N=256 (block 256)/update (+) 10.6 k allocs: 0.549 MB 10.6 k allocs: 0.55 MB 0.999
time_to_load 0.147 k allocs: 10.8 kB 0.147 k allocs: 10.8 kB 1

⚠️ Regressions (time > 35.0% and outside the reported ±spread; allocs/memory > 25.0%)

  • stencil/dagger/N=1024 (block 128)/alloc (neighbors Wrap) (time): +378.3%
  • stencil/dagger/N=1024 (block 512)/neighbors (Wrap) (allocs): +169.9%
  • stencil/dagger/N=256 (block 128)/neighbors (Wrap) (allocs): +169.8%
  • stencil/dagger/N=1024 (block 512)/alloc (neighbors Wrap) (allocs): +158.9%
  • stencil/dagger/N=256 (block 128)/alloc (neighbors Wrap) (allocs): +158.9%
  • stencil/dagger/N=256 (block 128)/neighbors (Wrap) (memory): +151.6%
  • stencil/dagger/N=256 (block 128)/alloc (neighbors Wrap) (memory): +139.9%
  • stencil/dagger/N=256 (block 128)/neighbors (Wrap) (time): +93.9%
  • stencil/dagger/N=1024 (block 128)/neighbors (Clamp) (allocs): +88.1%
  • stencil/dagger/N=1024 (block 128)/neighbors (Wrap) (allocs): +87.2%
  • stencil/dagger/N=1024 (block 128)/neighbors (Reflect) (allocs): +86.5%
  • stencil/dagger/N=1024 (block 512)/neighbors (Clamp) (allocs): +85.3%
  • stencil/dagger/N=256 (block 128)/neighbors (Reflect) (allocs): +85.3%
  • stencil/dagger/N=1024 (block 512)/neighbors (Reflect) (allocs): +85.2%
  • stencil/dagger/N=256 (block 128)/neighbors (Clamp) (allocs): +85.2%
  • stencil/dagger/N=1024 (block 128)/alloc (neighbors Wrap) (allocs): +83.2%
  • stencil/dagger/N=256 (block 128)/alloc (neighbors Wrap) (time): +83.1%
  • stencil/dagger/N=1024 (block 128)/neighbors (Clamp) (memory): +82.1%
  • stencil/dagger/N=1024 (block 128)/neighbors (Reflect) (memory): +81.7%
  • stencil/dagger/N=1024 (block 128)/neighbors (Wrap) (memory): +81.3%
  • stencil/dagger/N=1024 (block 128)/neighbors (Pad) (allocs): +78.5%
  • stencil/dagger/N=1024 (block 128)/alloc (neighbors Wrap) (memory): +76.6%
  • stencil/dagger/N=256 (block 128)/neighbors (Clamp) (memory): +73.4%
  • stencil/dagger/N=256 (block 128)/neighbors (Reflect) (memory): +73.4%
  • sparse/dagger/N=1024 (block 64)/cg solve (laplacian) (time): +72.8%
  • stencil/dagger/N=1024 (block 128)/neighbors (Pad) (memory): +71.4%
  • stencil/dagger/N=1024 (block 128)/neighbors (Clamp) (time): +65.5%
  • sparse/dagger/N=256 (block 16)/cg solve (laplacian) (time): +58.7%
  • stencil/dagger/N=1024 (block 128)/neighbors (Wrap) (time): +50.9%
  • stencil/dagger/N=1024 (block 128)/neighbors (Reflect) (time): +45.5%
  • stencil/dagger/N=1024 (block 128)/neighbors (Pad) (time): +38.6%
  • stencil/dagger/N=256 (block 128)/neighbors (Reflect) (time): +36.5%
  • stencil/dagger/N=1024 (block 512)/neighbors (Wrap) (memory): +31.6%
  • stencil/dagger/N=1024 (block 512)/neighbors (Pad) (allocs): +29.1%
  • stencil/dagger/N=256 (block 128)/neighbors (Pad) (allocs): +29.0%
  • stencil/dagger/N=1024 (block 512)/alloc (neighbors Wrap) (memory): +26.3%

Improvements

  • array/dagger/N=256 (block 256)/alloc (rand) (time): -94.8%
  • linalg/dagger/N=256 (block 256)/syrk (A'*A) (time): -91.4%
  • linalg/dagger/N=1024 (block 128)/matmul (A*A) (time): -89.8%
  • linalg/dagger/N=1024 (block 128)/syrk (A'*A) (time): -89.5%
  • linalg/dagger/N=1024 (block 128)/qr (time): -85.3%
  • linalg/dagger/N=256 (block 128)/matmul (A*A) (time): -78.1%
  • linalg/dagger/N=256 (block 128)/syrk (A'*A) (time): -76.6%
  • array/dagger/N=1024 (block 512)/alloc (rand) (time): -74.8%
  • linalg/dagger/N=256 (block 128)/cholesky (time): -74.6%
  • linalg/dagger/N=1024 (block 512)/matmul (A*A) (time): -74.5%
  • linalg/dagger/N=1024 (block 512)/syrk (A'*A) (time): -74.5%
  • linalg/dagger/N=1024 (block 128)/cholesky (time): -72.5%
  • linalg/dagger/N=1024 (block 512)/cholesky (time): -71.0%
  • array/dagger/N=256 (block 128)/alloc (rand) (time): -68.3%
  • linalg/dagger/N=256 (block 128)/qr (time): -68.2%
  • linalg/dagger/N=1024 (block 128)/solve (A\b via lu) (time): -38.8%
  • linalg/dagger/N=1024 (block 512)/matvec (A*x) (memory): -37.0%
  • stencil/dagger/N=1024 (block 512)/neighbors (Pad) (memory): -36.7%
  • array/dagger/N=256 (block 256)/transpose (permutedims) (time): -35.8%
  • linalg/dagger/N=1024 (block 128)/lu (time): -35.1%
  • linalg/dagger/N=1024 (block 512)/matmul (A*A) (memory): -34.9%
Within noise (1 metric(s) past threshold but inside the ±spread; not counted)
  • linalg/dagger/N=256 (block 256)/cholesky (time): 137.3%

Full results and plots (download the benchmark-results-* artifacts).

@jpsamaroo
jpsamaroo force-pushed the jps/mpi-bad-scale-stencil branch from 3e7eedb to 3c0b231 Compare August 17, 2026 22:36
@jpsamaroo
jpsamaroo force-pushed the jps/mpi-bad-scale-stencil branch 2 times, most recently from c4cf41e to 03a74d4 Compare September 4, 2026 23:47
@jpsamaroo
jpsamaroo marked this pull request as draft September 4, 2026 23:50
jpsamaroo and others added 7 commits September 5, 2026 10:46
@jpsamaroo
DArray/stencil: Avoid xfer of full chunks for boundary across differe…
1ec0bd0 
…nt spaces
@jpsamaroo
CI/benchmarks: Add Distributed and MPI benchmarks
4c7e6e1
@jpsamaroo
datadeps: Allocate cross-space slots instead of transferring them
febf021 
A Datadeps slot's contents are never assumed to be current. `generate_slot!`
says so outright -- it deliberately does not sync with the owner -- and
`compute_remainder_for_arg!` decides what to copy purely from `arg_history` /
`arg_owner`, never from the buffer. A slot in a space that does not yet appear
in the argument's history is therefore always filled by a copy-to (`FullCopy`,
or a span-exact `MultiRemainderAliasing` once part of it is current) before any
task can read it.

So for a dense, isbits payload crossing a space boundary, `move_rewrap` was
sending bytes that were guaranteed to be overwritten. It now allocates the
destination buffer and sends nothing: only the element type and dimensions
travel (broadcast like a header under MPI, closure arguments under Distributed).
The allocation is Libc-backed straight from `alloc_libc_array` rather than
allocated and then copied into Libc memory by `libc_backed`, since there is
nothing to copy. Under MPI that also fixes a latent no-op -- `libc_backed` was
being applied to a `Chunk`, which hits the identity method.

`from_space != to_space` is the load-bearing guard. When the two coincide the
leaf transfer is the identity, so the "slot" *is* the original data and
`compute_remainder_for_arg!` returns `NoAliasing()`; handing back a fresh buffer
there would silently discard the argument. Allocation also stays routed through
`aliased_object!`, so an already-present parent is still reused rather than
shadowed by a detached buffer -- which is what keeps an array and its view
sharing one destination allocation.

N.B. This depends on the preceding free-syncdeps fix. Transferring the slot was
masking a use-after-free: a recycled Libc block was immediately overwritten with
real data, so a racing read returned plausible values. Without the transfer the
same read hits uninitialized memory, which is how that bug first became visible
(NaN out of a distributed Cholesky).

Measured on 4 MPI ranks, a 1024x1024 `@stencil` sweep with `@neighbors`:
689ms/sweep at 128x128 tiles (872ms before) and 316ms at 512x512 tiles (509ms
before). `test/mpi.jl` 398/403/388/376 passing; Distributed datadeps 1272
passing.

Test contract updated accordingly: cross-space slots are no longer asserted to
hold a copy of the source, only to have the right shape, type, and parent
sharing; same-space slots must still be the real data. Two new testsets cover
the allocation itself and the partial-currency invariant it rests on -- a task
reading spans never written in its own space.
@jpsamaroo
CI: Give the 4-rank MPI CPU job more timeout headroom
268a100 
test/mpi.jl at 4 ranks passes cleanly (398/398) in ~12 minutes on an idle
workstation, matching the ~12m benchmark in c13465a's commit message --
no reproducible deadlock. CI reportedly runs ~40 minutes and occasionally
exceeds the 60-minute timeout, consistent with slower/shared runners eating
the margin rather than a hang. Bump 4-rank's budget to 90 minutes; 2-rank
stays at 60 since it isn't reported as tight.
@jpsamaroo @claude
MPIExt: Broadcast move_rewrap's header via the dispatch-decoupled relay
3daea47 
move_rewrap runs from generate_slot! inside the per-task planning loop,
and each task is handed to eager_launch! (and so starts executing,
concurrently) before the loop moves on to plan the next task's header
broadcast. That means this broadcast is not at the "sequential,
non-overlapping point" the raw bcast_yield tree assumes -- it can race
an already-dispatched task's own execute!/poolget activity, closing the
same forwarder wait-cycle documented above bcast_meta_yield.

Confirmed: a 4-rank mpiexec run of test/mpi.jl hung almost immediately
with the old bcast_yield header broadcast under load from a concurrent
MPI job on the same machine, and passed cleanly (16m44s, 398/398, no
warnings) after switching to bcast_meta_yield.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@jpsamaroo @claude
MPIExt: Drop the blocking barrier that deadlocks the 4-rank MPI job
786cf6b 
The 4-rank CPU MPI job has been running out its CI timeout on Julia 1.10 and
1.11 with nothing in the log to point at. It reproduces once the run is given
CI's CPU budget rather than a workstation's: pinned to 4 CPUs (what
ubuntu-latest provides), Julia 1.10 at HEAD deadlocks outright, and SIGUSR1
backtrace dumps catch all four ranks inside `MPI_Barrier`, called from
`check_uniform`. Nothing here is version-specific -- it is a race whose odds
turn on GC and scheduling timing, which is why 1.12 slips through.

Three defects, in decreasing order of severity:

  * `check_uniform` ended with `MPI.Barrier`. That is a blocking `ccall`, so it
    parks the OS thread inside MPI: the thread runs no other Julia task --
    notably not `bcast_relay_loop`, which other ranks depend on to have a
    broadcast forwarded -- and it never reaches a GC safepoint, so any other
    thread requesting a collection stalls the process until the barrier
    returns. Rank r parks here waiting for rank s while s waits on a broadcast
    only r's relay can forward, and neither can move. The barrier bought
    nothing either: `compare_all` directly above it is already an arrival
    barrier, since it cannot return until every rank has entered it and sent.

  * `bcast_slot_wait` kept a one-shot slot keyed by tag alone. A tag does not
    identify a single broadcast -- `to_tag()` returns the planning task's thunk
    id, so every `move_rewrap` header broadcast issued while generating that
    task's slots shares it, one per wrapper level and one per argument. The
    second delivery overwrote the first before its consumer read it; that
    consumer then took the wrong value and the next one waited forever on a
    payload that had already arrived. Now a FIFO keyed by (root, tag), drained
    in the rank-uniform order MPI's non-overtaking guarantee establishes.

  * The relay busy-spun on `MPI_Improbe`, taking 107 of one thread's 127
    profile samples -- a core per rank, which a 4-vCPU runner hosting 4 ranks
    cannot spare. It now spins hot through a burst and backs off after it,
    keeping the added latency off all but the first message of the next burst.

Both waits that had no deadlock detection now have it, so a future regression
here reports itself instead of silently spending the CI budget.

Verified on Julia 1.10, 4 ranks x 2 threads pinned to 4 CPUs. HEAD deadlocks in
the Stencils testset after ~35 minutes: ranks 1-3 time out on a `compare_all`
recv from rank 0 on tag 1073741823 (`MPI.tag_ub()`) while rank 0, sitting in
the undetected barrier, reports nothing at all, and `mpiexec` never exits. With
this change the suite passes on all four ranks twice over, in 18m46s and
18m19s, with the expected 398/403/376/388 test counts.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@jpsamaroo @claude
datadeps: Compute free syncdeps from the buffer's own recorded aliasing
d744b14 
`gather_free_syncdeps!`'s uniform-execution fallback synced on the object
cache's `key` ainfo, guarded on that key being tracked:

    haskey(state.ainfos_overlaps, wrapped) &&
        get_write_deps!(state, space, wrapped, write_num, syncdeps)

That guard is close to never true. `ainfos_overlaps` is populated by
`populate_ainfo!` only for ainfos tracked as a direct task dependency, and the
buffers reaching this branch are exactly the ones that are not -- they merely
underlie wrapper arguments (the parent array shared by several `view`s, whose
tracked slots are the views rather than the buffer). Worse, the key ainfo
describes the original object in its *source* space: every span carries its
space, and `intersect` skips candidates sharing no space index
(memory-spaces.jl:428), so a source-space ainfo cannot overlap a
destination-space one even when it is tracked. The branch therefore emitted an
`unsafe_free!` with an *empty* syncdep set, racing every task still reading the
buffer.

What the free loop actually needs to know is which tasks touch memory *inside*
the buffer, and that is the tracked ainfos in the buffer's own space overlapping
its extent -- which requires the buffer's *destination-space* aliasing. That
cannot be recomputed here: `aliasing` is a collective under uniform (SPMD)
execution, so reaching it from the free loop costs one broadcast per freed
buffer at a sequential point.

It need not be recomputed, though. `set_stored!` already computes exactly this
ainfo, on every rank, when it allocates the buffer -- it needs it to register
the buffer in `derived` -- and then discards it. Retain it in a new
`value_ainfos` field and hand it to `gather_free_syncdeps!`, which now runs one
exact, space-filtered overlap search for both execution modes. The
uniform/non-uniform split disappears, as does the non-uniform path's own
free-time `aliasing` call.

The result is exact rather than conservative. Syncing against every tracked
ainfo in the space instead would hold every buffer until that space's work had
drained -- the opposite of what deferring frees is meant to buy.

A missing record is now an error rather than a quiet `return`: degrading to an
empty syncdep set is precisely the use-after-free this function exists to
prevent.

Also adopts, from 8469a61 on jps/datadeps-region-async (which closes the same
hole by taking the collective instead), the parts independent of that choice:
`gather_overlap_syncdeps!` as a shared helper, and
`DATADEPS_ASSERT_FREE_SYNCDEPS`, a debug-gated invariant check that re-derives
the answer by linear scan + `will_alias` rather than the interval tree, so a bug
in the fast path cannot also hide from the assertion meant to catch it.

Distributed datadeps suite (4 workers) 2068 passing, 2 broken, no failures --
including the new end-to-end regression test, which asserts every `unsafe_free!`
a shared-parent-views region emits has non-empty syncdeps and runs with the
invariant check forced on. 4-rank `test/mpi.jl` 443/470/455/466 passing, no
failures or errors.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019Pp6hFZNHR44pLhDrR6aXq
@jpsamaroo
jpsamaroo force-pushed the jps/mpi-bad-scale-stencil branch from 03a74d4 to d744b14 Compare September 5, 2026 19:09
@jpsamaroo
jpsamaroo marked this pull request as ready for review September 5, 2026 22:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

performance stencils

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@jpsamaroo