Repository navigation
Conversation
Signed-off-by: Parvm1102 <parvmittal31757@gmail.com>
Signed-off-by: Parvm1102 <parvmittal31757@gmail.com>
`Flux.destructure`'s `re` writes the gradient into the trainable leaves but keeps every other array (e.g. BatchNorm's running stats) at its model value. Zygote returns `nothing` for those fields so `check_equal_leaves` skipped them, but Enzyme returns a full shadow model with the correct zero gradient there and the comparison failed on state that is not a gradient at all. Rebuild the reference from a zeroed model so those leaves compare as zeros. This enables Enzyme on the GNNChain Parallel/BatchNorm test site for non-dense graphs; verified neutral for EdgeConv and GNNlib's propagate. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Parvm1102 <parvmittal31757@gmail.com>
Enzyme differentiated the Chebyshev recursion of `cheb_conv` inline at every call site. `GConvLSTMCell` calls `cheb_conv` eight times and `GConvGRUCell` six, so compiling their gradients was dominated by these inlined copies (a GConvLSTM test site took ~5 min to compile on Julia 1.13.1 with Enzyme 0.13.216). Move the array math of `cheb_conv` into an internal `_cheb_core(W, b, X, L)` and give it an Enzyme reverse rule in a new `GNNlibEnzymeCoreExt`. The scaled Laplacian is a constant (it is already marked inactive for Enzyme), and batched or other annotations fall back to Enzyme's own differentiation. Add a k = 3, bias = false ChebConv test case, which runs the recursion and the constant-bias path of the rule. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With Enzyme 0.13.216 both fail with an IllegalTypeAnalysisException inside the keyword `GNNGraph` constructor that `d_conv` uses to build the transposed graph, the same cause that keeps DConv opted out. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
Author
|
The CI actually passed here. There were lots on changes in Enzyme since #708 and it takes very less time to run now. |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #708. This branch contains #708's three commits; the commits on top of it are the new part (diff against #708). It targets
masterbecause #708's branch lives in my fork, so GitHub can't base a PR on it here. It's a draft for now, mainly to see whether the GNN CI job completes with Enzyme enabled. #708's run in August was killed at minute 77 when the runner ran out of resources.What changes
cheb_conv(newGNNlibEnzymeCoreExt, GNNlib → 1.5.0-DEV). The array math ofcheb_convmoves verbatim into an internal_cheb_core(W, b, X, L), and the extension gives it a hand-written pullback. The scaled Laplacian is treated as a constant, as it already is viaGNNGraphsEnzymeCoreExt. Batched annotations fall back to Enzyme's own differentiation.DCGRUCellandDCGRU. With Enzyme 0.13.216 both fail with anIllegalTypeAnalysisExceptioninside the keywordGNNGraphconstructor thatd_convuses for the transposed graph, the same cause that keepsDConvopted out. A keyword-freed_conv(the Fix Enzyme differentiation of GCNConv/SGConv/TAGConv on adjacency-matrix graphs via a keyword-free COO conversion helper #703 recipe) fixes all three; I've tested that locally and left it for a follow-up.Why
On the current CI stack (Julia 1.13.1, Enzyme 0.13.216), compiling Enzyme gradients is no longer expensive for the message-passing layers (1–13 s per compile). The recurrent layers built on
cheb_convare now the bulk of the cost.GConvLSTMCellcallscheb_conveight times, and Enzyme differentiated each copy inline. With the rule:GConvLSTM(input / model)GConvLSTMCellGConvGRUCellThe whole
GConvLSTMtest item went from 21 min (2.7 GB) to 4 min (1.8 GB). It also makes Enzyme much faster to compile for users ofChebConv,GConvGRUandGConvLSTM.Testing
Locally on Julia 1.13.1 with Enzyme 0.13.216, one test item per process:
ChebConv37/37,GConvLSTMCell5/5,GConvLSTM6/6,GConvGRUCell4/4 andGConvGRU6/6 pass. Theirtest_gradientssites compare Enzyme against finite differences.ChebConvcase withk = 3, bias = falseruns the recursion and the constant-bias branch of the rule.DCGRUCell4/4 andDCGRU6/6 pass with Enzyme skipped.Not yet run: the rest of the suite, and Enzyme on GPU arrays (the GPU tests don't use Enzyme).
🤖 Generated with Claude Code