A thin local backend for generating images with the Krea 2 model. It runs its own native engine in one process: no ComfyUI and no Wan2GP at runtime. It is exposed as an HTTP API and as a command line.
- Text-to-image with frame resolution (480 / 720 / 1080 / 1440) and aspect ratio
- Image-to-image and inpaint with a mask
- Reference editing: one or two reference pictures, scene or subjects mode
- Outpainting: extend a picture on any side, optionally keeping the original pixels
- Background removal for reference people and objects
- LoRA with per-request weights; the set can change without reloading the model
- Prompt enhancer: Qwen3-VL expands a short idea into a detailed prompt
- Checkpoints in mxfp8, fp8, int8, and GGUF q8_0 formats; switch them without a restart
- Fast inference on NVIDIA GPUs: quantized matrix multiplication, pinned memory, cuDNN attention
- Jobs:
POST /generatereturns a job id at once; a scheduler runs the queue on one or more GPUs, fairly between users; jobs can be cancelled while waiting or running and survive a restart (SQLite) - Access control: API keys per client; users see only their own jobs and pictures, admins see all
- HTTP API with an OpenAPI schema and interactive docs (Swagger UI at
/docs, Scalar at/scalar, ReDoc at/redoc), and a CLI
- Windows (the model-linking script is PowerShell; tested on Windows 11)
- Python 3.11 or 3.12
- uv
- NVIDIA GPU with CUDA. Tested on an RTX 5080 Laptop with 16 GB VRAM (peak use about 14 GB at 1080p).
The fast multiplication modes need a 50-series (Blackwell) GPU; other cards use
exactmode. - About 64 GB of RAM was used for testing. Pinned memory keeps about 19 GB of RAM in use; it can be turned off (see configuration).
- Model files (see Models); the checkpoint alone is about 13 GB.
Clone the repository and install the dependencies (torch is taken from the PyTorch CUDA 12.8 index):
uv syncLink the models into the project (see Models):
pwsh scripts/link_models.ps1Generate an image from the command line:
uv run my-diffusion-generate "a fox in the snow, golden hour" --resolution 720 --aspect-ratio 16:9Start the HTTP server (interactive documentation at http://127.0.0.1:8000/docs or http://127.0.0.1:8000/scalar):
uv run uvicorn my_diffusion.api:app --host 127.0.0.1 --port 8000 --timeout-graceful-shutdown 5Optional settings go into a .env file: copy .env.example to .env and edit it.
The models are not downloaded or copied. The code only knows about the models/ folder in the
project root, and that folder consists of links (junctions) to the real folders - they are created by
scripts/link_models.ps1. On another machine: fix the path in the script (the -ModelsHub
parameter) or simply put the files into the same structure.
| Link in the project | What should be there |
|---|---|
models/checkpoints/ |
Krea 2 checkpoints: .safetensors or .gguf |
models/loras/ |
LoRAs for Krea 2, subfolders allowed |
models/vae/ |
qwen_vae.safetensors and qwen_vae_config.json |
models/text_encoders/Qwen3-VL-4B-Instruct/ |
*_quanto_bf16_int8.safetensors, config.json, tokenizer files |
models/rembg/ |
u2net.onnx (only for background removal) |
Reference editing also needs the edit LoRA krea2_identity_edit_v1_2 in models/loras.
| Area | Support |
|---|---|
| Resolutions | 480, 720, 1080, 1440 pixel budget; 16:9, 4:3, 3:2, 21:9, 1:1 and portrait variants (details) |
| Checkpoints | mxfp8, float8_e4m3fn, int8 tensorwise, bare fp8, GGUF q8_0; turbo and raw kinds (details) |
| LoRA | Standard LoRA files with 7 key naming schemes; LoKr/LoHa not supported (details) |
| Modes | text-to-image, image-to-image, inpaint, reference editing, outpainting, background removal (details) |
| Interfaces | HTTP API (/generate, /result/{id}, /cancel/{id}, /checkpoints/load, ...) and CLI (details) |
| Hardware | NVIDIA CUDA GPUs; fast modes on 50-series only (details) |
Known limitations:
- Edits above 720 (one reference) or 480 (two) are slow rather than impossible: the network is streamed from RAM as needed.
MY_DIFFUSION_EDIT_LIMITS(strict/warn/off, defaultwarn) andMY_DIFFUSION_EDIT_MAX_PIXELSdecide whether such jobs are accepted (details). - A GPU runs one generation at a time; more GPUs run more jobs at once, but each holds its own copy of the models in RAM (multi-GPU is not tested yet). Run the server as a single process.
- Without
MY_DIFFUSION_API_KEYSaccess control is off and every caller is an admin: keep such a server on127.0.0.1. - The prompt enhancer supports only Qwen3-VL-4B.
- Image modes: image-to-image, inpaint, reference editing, outpainting, background removal
- API and CLI: endpoints, request fields, command-line flags, prompt enhancer
- Configuration: all
MY_DIFFUSION_*variables - Checkpoints: formats, turbo and raw, guidance
- Resolutions: pixel budgets and aspect ratios
- LoRA: using LoRA files
- Jobs, queue and access: the generate / result / cancel flow, API keys, fairness, several GPUs, restarts
- Performance: measurements, memory, attention kernel, multiplication modes
- Architecture: how the engine works, design decisions, rejected experiments
- Development: tests, verification scripts, what is not implemented yet