EvidMTL extends SwinMTL with evidential depth estimation and semantic
segmentation. This public release corresponds to the pre-depth-completion
wonka source snapshot used by the combined Evidential Mapping project.
Relative to the upstream SwinMTL architecture, this work adds:
- evidential heads for joint semantic segmentation and monocular depth;
- semantic and depth uncertainty estimates alongside the task predictions;
- an evidential SiLog depth objective with uncertainty regularization and a Normal-Inverse-Gamma prior; and
- the model outputs used by EvidKimera for uncertainty-aware semantic TSDF fusion.
SwinMTL remains the underlying shared encoder-decoder architecture. The original SwinMTL authors and paper are credited below; the modifications above are described in the EvidMTL paper.
Model checkpoints are hosted separately on Hugging Face:
silog_klmuprior_alpha1.01_gamma0.001_betacalc_final_model.ckptuses a noisy ground-truth mean prior.silog_klmuprior_alpha2_beta0.1_gamma1.0_no_noise_final_model.ckptuses the ground-truth mean directly as the prior.
The source code remains licensed under GPL-3.0; see LICENSE.
If you use this repository, please cite both EvidMTL and the upstream SwinMTL paper:
@inproceedings{menon2025evidmtl,
author={Menon, Rohit and Dengler, Nils and Pan, Sicong and Chenchani, Gokul Krishna and Bennewitz, Maren},
title={EvidMTL: Evidential Multi-Task Learning for Uncertainty-Aware Semantic Surface Mapping from Monocular RGB Images},
booktitle={2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
year={2025},
doi={10.1109/IROS60139.2025.11247528}
}
@inproceedings{taghavi2024swinmtl,
author={Taghavi, Pardis and Langari, Reza and Pandey, Gaurav},
title={SwinMTL: A Shared Architecture for Simultaneous Depth Estimation and Semantic Segmentation from Monocular Camera Images},
booktitle={2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
year={2024},
url={https://arxiv.org/abs/2403.10662}
}A multi-task learning framework designed for simultaneous depth estimation and semantic segmentation using the Swin Transformer architecture.
- [30th June] Paper Accepted at the IROS 2024 Conference 🔥🔥🔥
To get started, follow these steps:
-
Only for ROS installation (otherwise skip this part)
cd catkin_ws/src catkin_create_pkg SwinMTL_ROS std_msgs rospy cd .. catkin_make source devel/setup.bash cd src/SwinMTL_ROS/src git clone https://github.com/PardisTaghavi/SwinMTL.git chmod +x inference_ros.py mv ./launch/ ./..
-
Clone the repository:
git clone https://github.com/PardisTaghavi/SwinMTL.git cd SwinMTL -
Create a conda environment and activate it:
conda env create --file environment.yml conda activate prc
To run the testing for the project, follow the below steps:
-
Download the required EvidMTL checkpoint from the Hugging Face model repository.
-
Move Pretrained Models:
- Create a
weightsdirectory and place the EvidMTL checkpoint there. - Third-party Swin backbone checkpoints expected under
model_zooare not redistributed by this repository. - Refer to
testLive.ipynbfor testing.
- Create a
roslaunch SwinMTL_ROS swinmtl_launch.launch- Introduction of a multi-task learning approach for joint depth estimation and semantic segmentation.
- Achievement of state-of-the-art performance on Cityscapes and NYUv2 datasets.
- Utilization of an efficient shared encoder-decoder architecture coupled with novel techniques to enhance accuracy.
We welcome feedback and contributions to the SwinMTL project. Feel free to contact taghavi.pardis@gmail.com.
Special thanks to the authors of the following projects for laying the foundation of this work. Our code relies on:



