Authors: Victor BERCY, Martyna POREBA, Michal SZCZEPANSKI, Samia BOUCHAFA
Graph-Guided Token Merging (G2TM) is a lightweight one-shot module designed to eliminate redundant tokens early in the ViT architecture. It performs a single merging step after a shallow attention block, enabling all subsequent layers to operate on a compact token set. It leverages graph theory to identify groups of semantically redundant patches.
The main branch only contains this overview. Each adapted model has its own branch: pick the model you want to use below, then follow the README of its branch for installation, training, inference, benchmarking, token visualization, ONNX export and results.
| Task | Model | Branch | Datasets | Get started |
|---|---|---|---|---|
| Semantic segmentation | Segmenter (Linear and Mask Transformer decoders) | segm |
ADE20K, Cityscapes, Pascal Context | README · Results |
| Semantic segmentation | SETR (Naive, PUP and MLA decoders) | setr |
ADE20K, Cityscapes, Pascal Context | README · Results |
| Semantic/Instance/Panoptic segmentation | EoMT | eomt |
ADE20K, Cityscapes, COCO | README · Results |
| Image classification | Vision Transformer | vit |
ImageNet-1k, CIFAR-100, Oxford Flowers, Oxford Pets | README · Results |
You can clone only the branch you need:
# Segmenter + G2TM
git clone -b segm --single-branch https://github.com/vbercy/g2tm g2tm-segm
# SETR + G2TM
git clone -b setr --single-branch https://github.com/vbercy/g2tm g2tm-setr
# EoMT + G2TM
git clone -b eomt --single-branch https://github.com/vbercy/g2tm g2tm-eomt
# ViT + G2TM
git clone -b vit --single-branch https://github.com/vbercy/g2tm g2tm-vitEach branch has its own conda environment (g2tm_segm, g2tm_setr, g2tm_eomt and g2tm_vit respectively), so the four models can be installed side by side.
Every branch ships the same core g2tm Python package, which provides:
- three interchangeable implementations of the merging module, which produce the same token groups: NetworkX connected components, a custom BFS (the fastest for PyTorch inference), and a tensorized FastSV algorithm (the fastest for batched training, and the one that can be exported to ONNX);
- token unmerging to recover the original token grid (used before the decoder for semantic segmentation, and not needed for classification);
- token and attention map visualization utilities.
The model code, training and evaluation pipelines, datasets and dependencies differ from one branch to another.
This work builds upon the official Segmenter, SETR, EoMT and Vision Transformer code, as well as ToMe, ALGM, timm, mmsegmentation and mmcv. The exact third-party code and licences used by each model are listed in the README of its branch.
G2TM is released under the Apache 2.0 licence:
Copyright © 2025 Commissariat à l'Energie Atomique et aux Energies Alternatives (CEA)
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
Each branch contains third-party code under its own licence. Refer to the LICENSE file of the branch you use.
If you cite G2TM or use this repository in your work, please cite:
@conference{bercy2026g2tm,
author={Victor Bercy and Martyna Poreba and Michal Szczepanski and Samia Bouchafa},
title={G2TM: Single-Module Graph-Guided Token Merging for Efficient Semantic Segmentation},
booktitle={Proceedings of the 21st International Conference on Computer Vision Theory and Applications - Volume 2: VISAPP},
year={2026},
pages={43-54},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0014267600004084},
isbn={978-989-758-804-4},
issn={2184-4321},
}