Engineering project (3rd year) β Image Processing & Computer Vision, 2026
Cyprien Vial Β· Setthika Sun Β· Bilal Delais
Overview β’ Method β’ Datasets β’ Metrics β’ Results β’ Usage β’ Structure
Domain shift is the statistical gap between the distribution a model is trained on (source) and the one it is deployed on (target). This project builds a full pipeline to measure and rank domain shift across aerial imagery datasets, combining classical distribution-distance metrics with a few-shot separability score: the faster a lightweight classifier learns to tell two domains apart from just a handful of examples, the larger the shift between them.
π‘ Core idea β Train a binary classifier on
1 β 500labeled examples per domain, over many random episodes. A domain pair that becomes separable within a handful of shots is far apart in feature space; a pair that stays confused even at hundreds of shots is nearly indistinguishable.
The study benchmarks 11 heterogeneous source domains (aerial, generic, agriculture, fashion, underwater, synthetic...) against 2 aerial target domains (DOTA and DIOR, with an artificial background-blur degradation), using:
| Approach | Tools |
|---|---|
| π§ Distribution-distance metrics | Wasserstein distance, Multi-Kernel MMD |
| π― Few-shot separability | Logistic Regression Β· MLP Β· Discriminator (1 β 500 shots, 100 episodes) |
| πΊοΈ Manifold visualization | t-SNE, Isomap |
All experiments, raw results, and analysis live in a single notebook: Technical_Study.ipynb.
Fig. Domain-shift ranking is nearly identical whether measured against DOTA or DIOR (r β₯ 0.96 across all metrics).
Aerial imagery is a particularly hard case for domain shift, driven by:
- π‘ differing acquisition conditions (altitude, sensor, resolution)
- π differing geographic context (urban vs. rural, country)
- π·οΈ inconsistent annotation protocols and category taxonomies
- π«οΈ image quality degradation (blur, compression, lighting)
Instead of the usual "train on thousands of samples" setup, the guiding question is:
How quickly can a classifier learn to distinguish two domains when given only a handful of labeled examples per class?
- Low separability, even with many shots β small domain shift β features transfer well
- High separability, even with very few shots β large domain shift β distributions are fundamentally different
Sweeping n_shots from 1 to 500 traces a full learning curve, from which principled, comparable shift scores are derived.
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β 1 Β· DATA LOADING & FILTERING β
β Streamed from the HuggingFace Hub Β· blur/size β
β filtering (Laplacian variance, min bbox size) β
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€
β 2 Β· FEATURE EXTRACTION β
β Frozen ResNet-50 (ImageNet1K_V2) β 2048-d vectors β
β saved as float16 .npy files β
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€
β 3 Β· DISTRIBUTION DISTANCE METRICS β
β Wasserstein-1 (PCA-50) Β· Multi-Kernel MMD β
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€
β 4 Β· FEW-SHOT DOMAIN DISCRIMINABILITY β
β LogReg Β· MLP Β· Discriminator β
β n_shots β {0,1,5,10,25,50,100,250,500} Γ 100 eps β
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββ€
β 5 Β· ANALYSIS & VISUALIZATION β
β DSI score Β· AUC Β· PAD Β· cross-target consistency β
β t-SNE / Isomap projections β
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
π§ͺ Implementation notes worth knowing
| Aspect | Detail |
|---|---|
| Feature extractor | Frozen ResNet-50, IMAGENET1K_V2 weights, final FC replaced with Identity() |
| Preprocessing | Resize 224Γ224, ImageNet mean/std normalization |
| Quality gate | Reject if min(w, h) < 30px, or Laplacian-variance blur score < 20 |
| Class balancing | Downsample both sets to `min( |
| Few-shot protocol | Sample n_shots/class for train, rest for test; StandardScaler fit on train only; averaged over multiple episodes |
| Discriminator by design | Deliberately under-powered (128β32, dropout 0.3) vs. the standard MLP (256β64, dropout 0.2) β an over-confident classifier would make every domain pair look maximally separable, destroying the metric's discriminative range |
| Reproducibility | Global seed = 1, 100 episodes/regime, resumable embedding extraction (skips already-processed datasets) |
All data is streamed directly from the HuggingFace Hub (HichTala/datasets) β no full download required.
| Dataset | Description |
|---|---|
dota-bg_blurry |
DOTA, background artificially blurred, blurry images removed |
dior-bg_blurry |
DIOR, background artificially blurred, blurry images removed |
| Key | Description | Domain |
|---|---|---|
coco |
Common Objects in Context | Generic |
dota |
Aerial imagery (Google Earth) | Aerial |
dior |
Optical remote sensing | Aerial |
dior_bg |
DIOR backgrounds | Aerial |
deepfruits |
Fruit detection in orchards | Agriculture |
xview |
Overhead satellite imagery | Aerial |
oktoberfest |
Crowd & food scenes | Event |
fashionpedia |
Fashion article segmentation | Fashion |
cadot |
Overhead vehicle detection | Aerial |
artaxor |
Arthropod imagery | Synthetic/nature |
uodd |
Underwater object detection | Underwater |
The source pool is deliberately heterogeneous β spanning aerial, ground-level, synthetic, and underwater domains β to stress-test the shift metrics across a wide range of feature-space distances.
| Metric | Definition | Reads as |
|---|---|---|
| DSI-10 | (AUC@10-shot β 0.5) Γ 200, clipped to [0, 100] |
0 = same domain, 100 = maximal shift |
| AUC Learning | Normalized area under the AUC-vs-shots curve | Overall learning efficiency |
| Convergence Rate | perf@10-shot / max_perf |
Relative convergence speed |
| Stability | 1 β 2 Γ mean_std, clipped to [0, 100] |
Inverse of inter-episode variance |
| AUC-500 | Final AUC (at 500 shots) Γ 100 | Asymptotic separability |
| PAD | Proxy A-distance, 2 Γ (1 β 2Ξ΅) |
Classic domain-adaptation shift measure |
| Wasserstein | Mean W-1 distance over 50 PCA components | Location/shape gap between distributions |
| MK-MMD | Multi-Kernel MMD, 30 log-spaced RBF bandwidths | Gap in reproducing-kernel Hilbert space |
Per-pair Γ per-model metrics are aggregated into a consensus score (mean over the 3 classifiers) and joined with the geometric distances for the final report.
| Tier | Sources | DSI-10 |
|---|---|---|
| π΄ Extreme shift | oktoberfest, fashionpedia, uodd, artaxor, deepfruits |
> 85 |
| π Strong shift | coco, cadot, dior, xview, dota |
55 β 85 |
| π’ Moderate / weak shift | dior_bg, dior-bg_blurry |
< 50 |
- β
Near-perfect cross-target consistency β DOTA vs. DIOR correlate at
r = 0.96(DSI-10),r = 0.97(Wasserstein),r = 1.00(MK-MMD): the source ranking barely depends on which target is chosen. - β
Model hierarchy β
LogReg (β83β84%) > MLP (β73%) > Discriminator (β65%)mean DSI-10, suggesting domain shift is largely linearly separable in ResNet-50 feature space. β οΈ Residual asymmetries βdotaanddior_bgshow a notable convergence gap between the two targets, indicating partial distribution overlap with one of them.- π― Practical takeaway β
oktoberfest/fashionpediamaximize shift (best stress-test sources);xview/cocosit closest to the aerial targets; 10 shots are enough to detect shift with LogReg (50 shots for MLP/Discriminator).
All generated figures (learning curves, metric heatmaps, cross-target comparisons, per-epoch DSI/PAD curves) live under domain_shift_analysis/.
Domain_Shift_Study/
βββ Technical_Study.ipynb # Main notebook β full pipeline + analysis
βββ domain_shift_analysis/ # Generated results & figures
β βββ metrics_detailed.csv # Per-pair Γ per-model metrics
β βββ metrics_consensus.csv # Consolidated metrics (3-model average)
β βββ few_shot_learning_curves.png # Aggregated learning curves
β βββ cross_target_comparison.png # DOTA vs. DIOR consistency
β βββ dior-bg_blurry/ # Full report for the DIOR target
β β βββ summary.json / summary.txt
β β βββ metrics_consensus.csv
β β βββ *.png # DSI, heatmap, per-epoch curves, PAD...
β βββ dota-bg_blurry/ # Full report for the DOTA target
β βββ summary.json / summary.txt
β βββ metrics_consensus.csv
β βββ *.png
βββ README.md
Developed and tested on Python 3.9+. A GPU is recommended for feature extraction (the backbone also runs on CPU, just slower).
git clone https://github.com/cyyyp100/Domain_Shift_Study.git
cd Domain_Shift_Study
python -m venv .venv
source .venv/bin/activate # Windows: .venv\Scripts\activate
pip install torch torchvision datasets scikit-learn scipy opencv-python \
pandas numpy matplotlib seaborn tqdm jupyterβΉοΈ No
requirements.txtis currently committed β the list above is reconstructed from the notebook's imports. Pinning exact versions withpip freeze > requirements.txtis recommended before external reuse.
Then open the notebook and run the cells in order:
jupyter lab Technical_Study.ipynb| Section | Content |
|---|---|
| Β§1β3 | Context, imports, global configuration (paths, seed, shot grid) |
| Β§4β13 | Feature extraction β streaming + ResNet-50 inference across all domains (can be long) |
| Β§14β19 | Distribution distances + few-shot evaluation (LogReg / MLP / Discriminator) |
| Β§20β22 | Full analysis report, figures, summary and conclusions |
Intermediate outputs (embeddings, incremental CSVs) make the pipeline resumable: an interruption doesn't require restarting extraction from scratch β already-processed datasets are automatically skipped. Final reports are written to domain_shift_analysis/, one subfolder per target domain.
- Relies on a single backbone (ImageNet ResNet-50); generalization to other architectures (ViT, self-supervised backbones) remains untested.
- Shift metrics operate at the whole-image level, without accounting for the spatial structure relevant to object detection (object scale, localization).
- The asymmetries observed between
dota-bg_blurryanddior-bg_blurryfordotaanddior_bgdeserve dedicated investigation. - A natural extension would connect these shift scores to the actual performance of a few-shot object detector, to validate their predictive power beyond raw feature separability.
@misc{domainshiftstudy2026,
title = {Domain Shift Evaluation in Few-Shot Object Detection: A Systematic
Analysis of Feature Space Divergence Across Aerial Image Datasets},
author = {Vial, Cyprien and Sun, Setthika and Delais, Bilal},
year = {2026},
note = {3rd-year engineering project, Image Processing \& Computer Vision},
howpublished = {\url{https://github.com/cyyyp100/Domain_Shift_Study}}
}No license is currently declared in this repository. Since this is an academic project, adding an explicit license (e.g. MIT, or CC BY-NC 4.0 for non-commercial use) is recommended before any external distribution or reuse.
Technical_Study.ipynb and the results in domain_shift_analysis/.