Implementation of ST-Moe, the latest incarnation of MoE after years of research at Brain, in Pytorch
-
Updated
Jun 17, 2024 - Python
Implementation of ST-Moe, the latest incarnation of MoE after years of research at Brain, in Pytorch
[NeurIPS 24] MoE Jetpack: From Dense Checkpoints to Adaptive Mixture of Experts for Vision Tasks
Code for Dynamic Convolutions: Exploiting Spatial Sparsity for Faster Inference (CVPR2020)
Mixture of Experts from scratch
Google DeepMind: Mixture of Depths Unofficial Implementation.
Topological Manifold Control: A multi-paradigm (PyTorch/JAX) geometric morphing engine utilizing differentiable soft-gating, optimized via XLA fused kernels and autograd-isolated non-blocking pipelines.
通过基于“滞后梯度博弈”的内生演化压力,替代传统 MoE 的外在负载均衡约束,实现神经网络拓扑结构的自组织特化与层级化
[NeurIPS 2026] SpecDrop: Parameter-Free Category-Conditioned Routing for Modular Specialization
Adaptive model-capacity orchestration — dynamically select, load, cache, and budget adapters, experts, models, and future model blocks.
Curated papers on dynamic neural networks, pruning, growing architectures, sparse training, and adaptive computation.
Official implementation of UniF-MoE: a unified framework for token-adaptive MoE computation
Research code for testing whether causal token surprisal can guide adaptive computation, sparse refinement, and learned compute allocation in byte-level language models.
A biologically inspired R&D blueprint for sparse, grounded, continual, energy-efficient AI.
To associate your repository with the conditional-computation topic, visit your repo's landing page and select "manage topics."