THOR

THOR is a secure Transformer inference framework that uses homomorphic encryption to run a BERT sequence-classification forward pass over encrypted data. It is built on the DESILO FHE library.

The repository exposes three CLI commands:

• encode_weights: generates encoded plaintext weights and masks under light_plaintexts/
• forward: runs one validation example and compares the result with the plain PyTorch model
• forward_batch: runs forward for a range of validation indices across one or more GPUs

Quickstart

Requirements

• desilofhe 2.0.1+ (CUDA 13.2 or 13.4).
• GPU with at least 32 GB of VRAM.

1. Installation

THOR works with Python 3.14+, and any standard Python package manager can be used; the examples below use Poetry.

poetry install

2. Prepare Files

Before running the encrypted forward pass, you need to prepare the following files:

2-1. A fine-tuned BERT checkpoint file including model.safetensors

Download finetuned_models.tar Google Drive Link, then extract the file into the repository root. The code also loads bert-base-uncased components from Hugging Face at runtime.

2-2. Encoded checkpoint files under light_plaintexts/, generated from model weights

encode_weights writes the weights, biases, and masks used by the encrypted forward pass. To generate the light plaintexts for the model:

poetry run encode_weights \
  --model_path ./finetuned_models/mrpc/model.safetensors

This writes the files to ./light_plaintexts/. Files encoded with desilofhe 1.x are not compatible with 2.0 and must be regenerated.

2-3. (Recommended) Cache the encoded files with vmtouch -t light_plaintexts/. The directory is around 110 GB, so make sure you have enough RAM.

You can install vmtouch from your package manager or from source: vmtouch

3. Forward Pass

• Single encrypted forward pass: poetry run forward

You can also run a batch over a range of validation indices:

poetry run forward_batch \
  --start-idx 0 \
  --end-idx 10 \
  --devices 0 1 \
  --output-dir ./forward-batch-results

forward_batch creates one subdirectory per target index and skips indices that already have results in the output directory.

Results

Each forward run writes:

• result.json, which includes the dataset type, target index, device, prediction, plain-model prediction, label, HE logits, and plain logits
• Optional per-layer plots such as layer-00.png through layer-11.png

During execution, the script also prints per-stage timing information from thor.timer.Timer.

Example Output

• HE and PT denote logits from the homomorphically encrypted forward pass and the plain PyTorch model, respectively.
• compute time measures the core encrypted inference execution time only, while total time includes end-to-end overhead such as preprocessing, data transfer, and visualization.

Predicted by HE: 1, Ground Truth: 1
HE A [-3.0007874894011994] B [5.904942303432394]
PT A [-3.12514591217041] B [6.013195514678955]
now: 2026-10-29 06:16:56.145461
----------------------------------------------------------------------------------
           stage time    compute time       total time    stage name
----------------------------------------------------------------------------------
                           1m 13.841s      6m 23.962s
----------------------------------------------------------------------------------

Benchmark

The desilofhe row is measured on validation index 0.

Implementation	CPU	GPU	Compute Time
Original THOR paper	Intel Xeon Platinum 8462Y+	NVIDIA A100-SXM4-80GB	602s
desilofhe 2.0.1	Intel Core i7-10700K @ 3.80GHz	NVIDIA GeForce RTX 5090	73.8s

For optimal performance, cache the encoded files with vmtouch -t light_plaintexts/ before running the forward pass.