Repository navigation

[Pytorch] Is the fwd_amx_history of first module is used before required allreduce communication #725

mst9009 started this conversation in General

mst9009
Mar 18, 2024

just as the titile
https://github.com/NVIDIA/TransformerEngine/blob/main/transformer_engine/pytorch/module/base.py#L541

Replies: 0 comments

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment