ParCIS Lab, BUPT

Popular repositories Loading

1. Magicube Magicube Public

   Magicube is a high-performance library for quantized sparse matrix operations (SpMM and SDDMM) of deep learning on Tensor Cores.

   C++ 91 16

2. Chimera Chimera Public

   Chimera: bidirectional pipeline parallelism for efficiently training large-scale models.

   Python 73 9

3. FlashSparse FlashSparse Public

   FlashSparse significantly reduces the computation redundancy for unstructured sparsity (for SpMM and SDDMM) on Tensor Cores through a Swap-and-Transpose mapping strategy. FlashSparse is accepted by…

   Cuda 41 7

4. Ok-Topk Ok-Topk Public

   Ok-Topk is a scheme for distributed training with sparse gradients. Ok-Topk integrates a novel sparse allreduce algorithm (less than 6k communication volume which is asymptotically optimal) with th…

   Python 27 10

5. Libra Libra Public

   C++ 5 1

6. Spec-K Spec-K Public

   Spec-K: Adaptive Resource Allocation via Speculative Preview for Efficient MoE Inference

   Jupyter Notebook 4