This is the framework for the configuration and management of federated learning simulation strategies. Federated Learning setup is based on Flower. The framework provides functionality to configure execution environment parameters, such as adversarial attacks, and machine learning parameters, such as number of epochs, ML task, and collect metrics in an uniformed way. It also allows to vary any number of simulation parameters and compare the effects of these parameters on the collected metrics, as well as plug-and-play integration of custom aggregation strategies.
- Sample configuration: located at
config/simulation_strategies/example_strategy_config.json. - Usage: pass the configuration file as a parameter when initializing
SimulationRunnerinsrc/simulation_runner.py.
-
aggregation_strategy_keyword
Defines the aggregation strategy. Options:trust: Trust & Reputation-based aggregation.pid: PID-based aggregation. Initial version of the formula.pid_scaled: PID-based aggregation with Integral part divided by the number of current round, threshold is calculated based on client distances.pid_standardized: PID-based aggregation with the Integral part standardized based on the distribution parameters of all Integral parts, threshold is calculated based on client distances.pid_standardized_score_based: Same as pid_standardized, but threshold is calculated based on pid scores.pid_v5_stateless_mean_based: Same as pid_standardized_score_based, but removal is stateless: all clients are reevaluated each round, and removal decisions apply only to the current round rather than being permanently maintained.pid_v5_stateless_median_based: Same as pid_v5_stateless_mean_based, but threshold is calculated with median instead of mean.pid_made: PID-MADE (PID with Model-Adaptive Deviation Estimation). Uses a lagged centroid reference (geometric median during warmup, then the previous global model), a discounted integral with forgetting factor λ, and an adaptive threshold (fixed, Chebyshev, or Gaussian). Fully stateless: no client is permanently removed — every client is re-evaluated fresh each round.multi-krum: Multi-Krum aggregation. Clients are removed from aggregation only in current round.krum: Krum aggregation works like Multi-Krum, but uses only a single client.multi-krum-based: Multi-Krum-based aggregation where removed clients are excluded from aggregation permanently.rfa: RFA (Robust Federated Averaging) aggregation strategy. Provides Byzantine fault tolerance through weighted median-based aggregation.trimmed_mean: Trimmed-Mean aggregation strategy. Aggregates updates by removing a fixed fraction of the largest and smallest values for each parameter dimension before averaging. Robust against outliers and certain types of attacks.bulyan: Bulyan aggregation strategy. Uses Multi-Krum as the first step of filtering and Trimmed-Mean as the second step to ensure robustness.fed_nga: Fed-NGA (Federated Normalized Gradients Algorithm) aggregation strategy. Performs normalization of client gradients before performing global aggregation to improve robustness against Byzantine faults and data heterogeneity. Fed-NGA paper: https://arxiv.org/pdf/2408.09539freq_fed: FreqFed aggregation strategy. A frequency analysis-based approach that uses 2-D Discrete Cosine Transform (DCT) to extract low-frequency components of client weight updates and clusters them using HDBSCAN to identify and exclude malicious updates. FreqFed paper: https://www.ndss-symposium.org/ndss-paper/freqfed-a-frequency-analysis-based-approach-for-mitigating-poisoning-attacks-in-federated-learning/fl_detector: FLDetector aggregation strategy. Detects malicious clients by predicting each client's expected model update using L-BFGS quasi-Newton approximation and accumulating per-client suspicious scores across rounds. Clients whose score exceeds a z-score threshold are excluded from FedAvg aggregation. FLDetector paper: https://dl.acm.org/doi/10.1145/3534678.3539231lqr_fed: LQR-Fed aggregation strategy. Replaces binary client removal with continuous per-client aggregation weights derived by solving a small Quadratic Program (QP) each round. The QP minimises a cost that penalises assigning weight to clients with high deviation from the global model (Q term) while penalising deviation from uniform weights (R term — control effort). No client is permanently excluded; the weight assignment adapts each round based on the current distance landscape.fedavg: Vanilla Federated Averaging (FedAvg) with no Byzantine-robust mechanism. Serves as an undefended baseline. Records the flattening-and-mean computation time asscore_calculation_timeso overhead can be compared against other strategies.remove_clientsis always forced tofalseregardless of config.fed_dmc: FedDMC aggregation strategy. Detects malicious clients by first reducing the dimensionality of model updates using Principal COmponent Analysis (PCA), then separating benign and malicious updates via Binary Tree-Based Clustering with Noise (BTBCN). It maintains an exponentially weighted trust score for each client across rounds using Self-Ensemble Detection Correction (SEDC), excluding clients whose scores fall below a set threshold before performing aggregation. FedDMC paper: https://doi.org/10.1109/TDSC.2024.3372634signguard: SignGuard aggregation strategy. Detects malicious clients by applying two parallel mechanisms: a norm-based filter to remove gradients with anomalous magnitudes, and a sign-based filter that uses Mean-Shift clustering on gradient sign proportions to identify the honest majority. The updates in the trusted intersection are then aggregated using mean aggregation with norm clipping. SignGuard paper: https://doi.org/10.1109/ICDCS54860.2022.00120rfcl: RFCL aggregation strategy. Designed for highly non-IID environments that clusters model updates using PCA and HDBSCAN, filters intra-cluster outliers via Modified Adaptive Federated Averaging (ModiAFA), and performs similarity analysis to aggregate only the most trusted clusters into the global model while using personalized model sharing to quarantine excluded clients. RFCL paper: https://doi.org/10.3233/FAIA230257
-
strict_mode: ensures that Flower trains and aggregates all available clients at every round. When enabled (default), automatically setsmin_fit_clients,min_evaluate_clients, andmin_available_clientsto equalnum_of_clients. Options:"true","false". -
remove_clients: attempt to remove malicious clients using strategy-specific mechanisms. -
dataset_keyword
Dataset used for execution. Options:femnist_iid: handwritten digit subset (0-9), 10 classes, IID distribution, 100 clients.femnist_niid: same, but the data is distributed in non-iid manner, according to authors' description. 16 clients max.its: intelligent Transportation Systems domain, binary classification (traffic sign vs stop sign), 12 clients.pneumoniamnist: medical imaging (pneumonia diagnosis), binary classification, IID distribution, 10 clients.flair: non-IID distribution (FLAIR dataset, unsupported in current version), 20 clients.bloodmnist: IID distribution, but non-equal number of samples per class, 40 clients.lung_photos: contains images of lung cancer from NLST archive from different CT machines. Data distributed according to the source, with varying number of images representing each stage of cancer. 30 clients.breastmnist: breast ultrasound images for tumor detection, binary classification (malignant vs benign), 10 clients.pathmnist: histopathologic images of colon tissue, 9 classes, IID distribution, 40 clients.dermamnist: dermatological lesion images, 7 classes (various skin diseases), 10 clients.octmnist: optical coherence tomography images of retinal tissue, 4 classes, 40 clients.retinamnist: retina fundus images for diabetic retinopathy classification, 5 classes, 40 clients.tissuemnist: gray-scale microscopic images of human tissue, 8 classes, IID distribution, 40 clients.organamnist: axial view CT scans of abdominal organs, 11 classes, 40 clients.organcmnist: coronal view CT scans of abdominal organs, 11 classes, 40 clients.organsmnist: sagittal view CT scans of abdominal organs, 11 classes, 40 clients.medmentions: PubMed extracts with NER labels, 10 clients (can manually generate more clients using the script insrc/utils/make_medmentions.py)air_quality_uci: Hourly responses averages of a gas multisensor device along with gas concentrations references from a certified analyzer, 12 clients, 4 targets for gas concentration prediction. Source: https://doi.org/10.24432/C59K5Fsmores: Smarter Microbial Observatories for Realtime Experiments datasets for disolved oxygen concentration forcasting introduced at https://github.com/rzv09/smores_proj by Raman Zatsarenko
-
num_of_rounds: total aggregation rounds. -
num_of_clients: number of clients (limited to available dataset clients). -
show_plots: show plots during runtime (true/false). -
save_plots: save plots toout/directory (true/false). -
save_csv: Save metrics as.csvfiles inout/directory (true/false). -
preserve_dataset: save poisoned dataset for verification (true/false). -
training_subset_fraction: fraction of each client's dataset for training (e.g.,0.9for 90% training, 10% evaluation). -
model_type: type of model being trained (e.g.,cnn,linear_regression,mlp,lstm). -
attack_schedule: array of attack configurations for round-based attacks. Each entry requires:start_round: when the attack starts (1 tonum_of_rounds).end_round: when the attack ends (1 tonum_of_rounds).attack_type: type of attack. Both categories are configured asattack_scheduleentries and applied in-memory — no pre-poisoned dataset required:- Data poisoning (applied to training data before local training):
label_flipping,gaussian_noise,target_flipping. - Model poisoning (applied to model weights after local training, before sending to server):
gradient_scaling.
- Data poisoning (applied to training data before local training):
- Attack parameters:
label_flipping: no additional parameters required.gaussian_noise:target_noise_snr(dB, recommended) ormean/std, plus optionalattack_ratio(0.0-1.0, defaults to 1.0).target_flipping:attack_ratio(0.0-1.0, defaults to 0.1). Flips the target values of a percentage of the data points in a clients dataset. Based on Flip attack described in DOI: 10.1109/PRDC50213.2020.00019gradient_scaling:scale_factor(number > 0). The local weight update delta(local - global)is multiplied by this factor before being sent to the server. A value greater than 1 amplifies the update; a value between 0 and 1 dampens it. For example,scale_factor: 10.0makes the malicious client's update 10× larger than its honest counterpart.little_is_enough: applies peturbations to many parameters across a few corrupted clients to control parameter dstribution while bypassing outlier detection. Based on Preventing Convergence Attack descriped in https://doi.org/10.48550/arXiv.1902.06156
- Malicious Clients Selection:
num_malicious_clients: specify number of malicious clients. Specific clients will be selected randomly.proportion_malicious_clients: specify proportion of malicious clients. Specific clients will be selected randomly.specific_malicious_clients: specify array of malicious client ids.
-
save_attack_snapshots: save attack snapshots for analysis (true/false). -
attack_snapshot_format: format for snapshots (pickle,visual, orpickle_and_visual). -
snapshot_max_samples: number of samples per snapshot (1-50, default: 5). -
Flower settings:
training_device:cpu,gpu, orcuda.cpus_per_client: processors per client.gpus_per_client: GPUs per client (ifcudais set as thetraining_device).min_fit_clients,min_evaluate_clients,min_available_clients: client quotas for each round.evaluate_metrics_aggregation_fn: not used.num_of_client_epochs: local client training epochs per round.batch_size: batch size for training.
-
LLM settings:
use_llm: use an llm (true/false)llm_model: the llm model to be usedllm_finetuning: how to finetune the llm (full,lora)llm_task: the task the llm is performing (mlm)llm_chunk_size: size of the token sequences used for training/testing- MLM settings
mlm_probability: specific to mlm tasks, the probability that a token is masked
- Lora settings
lora_rank: rank/size of the low-rank matrices used in loralora_alpha: scaling factor for lora updateslora_dropout: dropout rate applied during traininglora_target_modules: list of model layers where lora should be applied
For air_quality_uci dataset:
target_name: gas concentration to predict (CO, C6H6, NOx, or NO2)
For smores dataset
target_name: predicting next dissolved oxygen concentration in the sequence (DO)sequence_len: amount of time series data to provide to model for its forcasting (default: 12)
For bulyan, krum, multi-krum-based, multi-krum, rfa, trimmed_mean, pid strategies:
For trust strategy:
begin_removing_from_round: start round for removing malicious clients.trust_threshold: threshold for client removal (typically, in the range0-1).beta_value: constant for Trust & Reputation calculus.num_of_clusters: number of clusters (must be1).
For pid, pid_standardized, pid_scaled, pid_standardized_score_based, pid_v5_mean_standard_score_based, pid_v5_median_standard_score_based strategies:
num_std_dev: number of standard deviations used int he calculation of PiD threshold at each round.Kp,Ki,Kd: PID controller parameters.
For krum, multi-krum, multi-krum-based strategies:
num_krum_selections: how many clients the algorithm will select.
For trimmed_mean strategy:
trim_ratio: fraction of extreme values to discard from both ends (lowest and highest) of each parameter dimension before averaging. Must be in the range 0–0.5.
For pid_made strategy:
Kp,Ki,Kd: PID controller gains.begin_removing_from_round: number of warmup rounds (T_warmup). During warmup the centroid reference is the geometric median of client weights; after warmup it switches to the lagged global model.lam: forgetting factor λ ∈ (0, 1) applied to the per-client integral each round (I[i] ← I[i]*λ + Δᵢ). Lower values make the integral forget history faster.threshold_mode: how the per-round threshold τₜ is derived from the score distribution. Options:fixed: τₜ =tau(constant). Requirestauparameter.chebyshev: τₜ = μ + √((1-α)/α)·σ — distribution-free bound. Requiresalphaparameter.gaussian: τₜ = μ + Φ⁻¹(1-α)·σ — parametric bound. Requiresalphaparameter.
tau: constant threshold score (required only whenthreshold_modeisfixed).alpha: per-round false-positive rate α ∈ (0, 1) (required whenthreshold_modeischebyshevorgaussian). Smaller α → higher threshold → fewer clients excluded.
For freq_fed strategy:
min_cluster_size: HDBSCAN min_cluster_size (minimum number of updates to form a cluster).begin_removing_from_round: warmup rounds before filtering is active.
For fl_detector strategy:
fl_detector_history_size: Number of L-BFGS curvature pairs(s, y)retained per client. Paper default:10. Higher values improve curvature approximation at the cost of O(n × N × d) memory. Recommended range:5–20.fl_detector_z_score_threshold: Removal threshold in standard deviations above the per-round mean suspicious score. Lower values are more aggressive (higher false-positive risk). Recommended:1.5–2.0for experiments of 30 rounds or fewer (score distributions become clearly bimodal only around round 60 per the paper);2.0–2.5for 60+ rounds.begin_removing_from_round: Warmup rounds before filtering is active. L-BFGS history still accumulates during warmup. Must be ≥1. Recommended:7–10.
For lqr_fed strategy:
lqr_q: State cost — penalty on assigning aggregation weight to clients with high distance from the reference model. Higher values more aggressively downweight suspicious clients. Start with1.0and tune upward if attacks are not suppressed.lqr_r: Control cost — penalty on deviating from uniform (equal) weights. Higher values keep the weight distribution closer to standard FedAvg. The ratioq/ris the key tuning lever:q/r >> 1→ strong outlier suppression;q/r << 1→ near-FedAvg.begin_removing_from_round: Warmup rounds before the lagged global model is used as the reference centroid. During warmup the arithmetic mean of client weight vectors is used instead. Recommended:5–10.normalize_distances(optional, defaultfalse): Whentrue, each client's L2 distance is divided by the round-wise mean distance before solving the QP. This makes the effectiveq/rsensitivity invariant to the absolute drift scale — useful when comparing across different numbers of local epochs or IID/non-IID settings without re-tuninglqr_q/lqr_r. Withfalse(default), the raw distances are used andq/rmay need adjustment if the training configuration changes.
For fed_dmc strategy:
ensemble_alpha: Controls the weight given to a client's historical trust score versus the current round's detection result in the Self-Ensemble Detection Correction (SEDC) module. Range is [0, 1). (default: 0.5).num_principal_components: Number of dimensions retained after applying PCA to the flattened model updates. (default: 5).min_cluster_size: Noise threshold used in the BTBCN module. (default: 3).removal_threshold: Removal threshold for the SEDC trust score. Clients whose trust score falls strictly below this value are classified as malicious and excluded from aggregation. (default: 0.5).begin_removing_from_round: Warmup rounds before filtering is active. SEDC trust scores still are calculated during warmup.
For signguard strategy:
norm_lower_bound: Loose lower threshold for the Norm-based Filter (default: 0.1)norm_upper_bound: Strict upper threshold for the Norm-based Filter (default: 3.0)coordinate_subset_ratio: Fraction of total gradient coordinates randomly sampled to calculate the sign statistics for the Sign-based Clustering (default: 0.1).bandwidth_quantile: Quantile used to estimate the bandwidth for the Mean-Shift clustering algorithm and controls the window size for discovering clusters. In non-IID environments, a higher quantile (e.g., 0.6-0.7) prevents the algorithm from breaking honest benign clients into many small clusters (default: 0.3).begin_removing_from_round: Warmup rounds before active filtering begins.
For rfcl strategy:
outlier_detection_sensitivity: Sensitivity parameter (ξ) used in the Modified Adaptive Federated Averaging (ModiAFA) module to control the threshold for intra-cluster cosine similarities outlier detection. Lower values make the outlier detection more aggressive. (Suggested Range:1.0-3.0; Default:2.0)num_similar_cluster_centers: The maximum number of clusters (K) selected during the Similarity Analysis phase to form the trusted pool (M_best) for final global aggregation (Suggested Range:1-5; Default:2)min_cluster_size: Parameter used in the HDBSCAN clustering module to dictate the minimum number of clients required to form a valid cluster. (Suggested Range:2-5; Default:2)begin_removing_from_round: warmup rounds before filtering is active.
- Python Environment: Python 3.11.9 is used in the framework. Before attempting to run the code, make sure the Python 3.11.9 is installed in the system.
- Configuration: place configurations in
config/simulation_strategies/. - Specify Configuration: update
src/simulation_runner.pywith the desired configuration file. - Execution:
- On UNIX: run
sh run_simulation.sh(automated virtual environment setup and execution). - On Windows: install dependencies from
requirements.txtand execute manually.
- Output: plots and
.csvfiles (if enabled) saved inout/directory.
Instead of editing configuration files by hand, you can drive the framework from a browser. Run:
sh run_with_ui.shthen open http://localhost:5173. The script sets up the virtual environment, installs the
frontend dependencies on first run, starts the API server and the UI, and stops both when
you press Ctrl+C. Set UI_PORT to use a different port. Node.js 18 or newer is required.
From the UI you can compose and validate configurations, launch simulations and follow their logs live, browse previous experiments, and use the AI assistant.
By default the framework stores every experiment in a local SQLite database
(out/experiments.db) alongside the CSVs and plots in out/. No account, token or
network access is needed, and the whole framework runs offline.
Members of the research group additionally upload their results to the group's
experiment server. To connect, put the token issued to you in .env.private:
echo "EXPERIMENT_SERVER_TOKEN=<your token>" >> .env.privateYou will also be prompted for it the first time you run a simulation or start the
frontend API server from a terminal, and the answer is saved to .env.private for you.
Without a token the framework runs in local mode: results are stored locally and the
UI features that query the group's database show a "not available in local mode"
message instead of failing.
-
Client-Level Metrics:
Loss: Provided by Flower.Accuracy: Evaluation accuracy (provided by Flower).Removal criteria: Used for client removal by the strategy.Distance: From cluster center (via KMeans).Normalized distance: Scaled distance (MinMaxScaler).
-
Round-Level Metrics:
Average loss: Across participating clients.Average accuracy: Across participating clients.
-
air_quality_uciSpecific MetricsR^2: How well a client's data is being modeled.Average R^2: Across participating clients.
-
smoresSpecific MetricsMARE: How accurrate was the continuous prediction?Loss baseline: Loss if predictions made from averaging selection window.Mare baseline: MARE if predictions made from averaging selection window.Average MARE: Across participating clients.Average Loss baseline: Across participating clients.Average MARE baseline: Across participating clients.
The framework allows to execute multiple aggregation strategies one after another and then compare metrics
between these strategies. The metrics such as Average loss and Average accuracy will be plotted on one graph for all
executed strategies to allow the analysis of the effect that changed parameters had on the training process.
The configuration file is the JSON that has the following format:
{
"shared_settings": {
// settings that are not changed between strategies
},
"simulation_strategies": [
{
// settings that are changed between strategies to compare their effects
},
{
...
}
]
}shared_settingsis the section where you can put settings that you do not wish to change for each strategy. It can be number of clients or any other settings.simulation_strategiesis the json array of strategy settings that you wish to alter for each strategy if you believe they may have effects on the learning or attack mitigation process.
- We want to see how the number of local epochs will affect metrics. In that case, the
num_of_client_epochsshould be put into each entry in thesimulation_strategiesarray:
{
"shared_settings": {
"num_of_rounds": 100,
"begin_removing_from_round": 4,
"num_of_clients": 10,
// all other settings according to specification
},
"simulation_strategies": [
{
"num_of_client_epochs": 1
},
{
"num_of_client_epochs": 2
},
{
"num_of_client_epochs": 3
}
]
}This configuration will result in the execution of 3 aggregation strategies in total, with the number of each client's local epochs varied starting form 1 to 3 for each execution. Then, the history of each strategy's loss and accuracy will be displayed within one plot, making it easy to draw conclusions on the efficiency of increasing the number of local client epochs.
- We want to see how the
trust-based mitigation works when we have the different number of malicious clients, and determine if it has any effects on the resulting loss or accuracy. Similarly,num_malicious_clientsinattack_scheduleis now a variable between strategies:
{
"shared_settings": {
"aggregation_strategy_keyword": "trust",
"num_of_clients": 20,
// all other settings according to specification
},
"simulation_strategies": [
{
"attack_schedule": [
{
"start_round": 1,
"end_round": 30,
"attack_type": "label_flipping",
"num_malicious_clients": 2
}
]
},
{
"attack_schedule": [
{
"start_round": 1,
"end_round": 30,
"attack_type": "label_flipping",
"num_malicious_clients": 4
}
]
},
{
"attack_schedule": [
{
"start_round": 1,
"end_round": 30,
"attack_type": "label_flipping",
"num_malicious_clients": 6
}
]
}
]
}- We want to compare
trustwithpid:
{
"shared_settings": {
"num_of_clients": 20,
"attack_schedule": [
{
"start_round": 1,
"end_round": 30,
"attack_type": "label_flipping",
"num_malicious_clients": 2
}
]
// all other settings according to specification
},
"simulation_strategies": [
{
"aggregation_strategy_keyword": "trust",
// the following parameters are specific for trust strategy
"trust_threshold": 0.15,
"beta_value": 0.75,
"num_of_clusters": 1
},
{
"aggregation_strategy_keyword": "pid",
// the following parameters are specific for pid strategy
"num_std_dev": 2.0,
"Kp": 1,
"Ki": 0,
"Kd": 0
}
]
}Since the strategy-specific parameters are not altered between strategies, they can also be put to shared_settigns:
{
"shared_settings": {
"num_of_clients": 20,
"attack_schedule": [
{
"start_round": 1,
"end_round": 30,
"attack_type": "label_flipping",
"num_malicious_clients": 2
}
],
// the following parameters are specific for trust strategy
"trust_threshold": 0.15,
"beta_value": 0.75,
"num_of_clusters": 1,
// the following parameters are specific for pid strategy
"num_std_dev": 2.0,
"Kp": 1,
"Ki": 0,
"Kd": 0,
// all other settings according to specification
},
"simulation_strategies": [
{
"aggregation_strategy_keyword": "trust"
},
{
"aggregation_strategy_keyword": "pid"
}
]
}-- the execution results of these two will be identical.
This design of the configuration file provides the flexibility to put any number of parameters as variables to the array, and compare how they affect the simulation outcome.
One limitation is that as of now it is impossible to vary the number of aggregation rounds, so the parameter
num_of_rounds must always be in the shared_settings section.
Researchers who employ InteFL in their work are kindly asked to reference our peer-reviewed article published in IEEE Intelligent Systems:
@ARTICLE{11366920,
author={Korobeinikov, Dmitrii and Zatsarenko, Raman and Chuprov, Sergei and Barea, Arnaldo and Reznik, Leon},
journal={IEEE Intelligent Systems},
title={InteFL Framework: Optimizing Federated Learning with Metacognition for Application Design and Deployment},
year={2026},
volume={},
number={},
pages={1-10},
keywords={Artificial intelligence;Monitoring;Benchmark testing;Metacognition;Adaptation models;Training;Intelligent systems;Hardware;Federated learning;Data models},
doi={10.1109/MIS.2026.3658072}}