⭐ A PyTorch-based toolkit for 3D face parameterization and reconstruction, integrating multiple face models. ⭐
- [2026.10.29] Release Face3D's fitting, and Face3D's BFM.
- [2026.10.27] 🥳 Release assets link.
- [2026.10.26] Release Deep3DFaceRecon, Deep3DFaceRecon's BFM, and Deep3DFaceRecon's Nvduffrast-based renderer.
- [2026.10.26] Release MediaPipe Blendshapes to FLAME, DECA, MICA, SPECTRE, and SMIRK.
- [2026.10.26] Release FLAME.
- [2026.10.26] 🎉 Release face3d_pytorch.
FLAME-based:
| Name | Paper | Code | |
|---|---|---|---|
| ✅ | MediaPipe Blendshapes to FLAME | —— | Github |
| ✅ | DECA | ACM TOG'21 | Github |
| ✅ | MICA | ECCV'22 | Github |
| ✅ | SPECTRE | arXiv'22 | Github |
| ✅ | SMIRK | CVPR'24 | Github |
| EMOCA | CVPR'22 | Github | |
| INFERNO | —— | Github |
BFM-based:
| Name | Paper | Code | |
|---|---|---|---|
| ✅ | Deep3DFaceRecon | CVPRW'19 | Github |
| ✅ | Face3D's fitting | —— | Github |
| Name | Paper | Code | |
|---|---|---|---|
| ✅ | FLAME | ACM TOG'17 | Github |
| ✅ | Deep3DFaceRecon's BFM | CVPRW'19 | Github |
| ✅ | Face3D's BFM | —— | Github |
| Name | Paper | Code | |
|---|---|---|---|
| ✅ | Deep3DFaceRecon's Nvduffrast-based renderer | CVPRW'19 | Github |
- Face Reconstruction
- Face Reenactment
- ...
Download the assets from here and place them in the project as follows:
face3d_pytorch
|- assets
`- face3d_pytorch
|- asssets // here
| |- blendshapes_to_flame
| |- deca
| `- ...
`- coreLoad a video:
from torchvision.io import read_video
video, _, _ = read_video(filename, output_format="TCHW")
video = (video / 255.0).to(torch.float32).cuda()
# (58, 3, 256, 256), torch.float32Load DECA:
from face3d_pytorch.core.parameterizer.deca import DECAWrapper
deca = DECAWrapper(
flame_encoder_checkpoints="face3d_pytorch/assets/deca/flame_encoder.pth",
detail_encoder_checkpoints="face3d_pytorch/assets/deca/detail_encoder.pth",
detail_decoder_checkpoints="face3d_pytorch/assets/deca/detail_decoder.pth",
)
deca.eval().cuda()Inference:
params_dict, align_dict = deca(video)
for k, v in params_dict.items():
print(f"{k}: {v.shape}")
# shape: torch.Size([58, 100])
# tex: torch.Size([58, 50])
# exp: torch.Size([58, 50])
# pose: torch.Size([58, 6])
# cam: torch.Size([58, 3])
# light: torch.Size([58, 9, 3])
# detail: torch.Size([58, 128])
# uv_z: torch.Size([58, 1, 256, 256])
for k, v in align_dict.items():
print(f"{k}: {v.shape}")
# aligned_images: torch.Size([58, 3, 224, 224])
# align_params: torch.Size([58, 3, 3])Load FLAME:
from face3d_pytorch.core.d3mm.flame import FLAMEWrapper
shape = params_dict["shape"]
exp = params_dict["exp"]
pose = params_dict["pose"]
flame = FLAMEWrapper(
flame_model_path="face3d_pytorch/assets/flame/FLAME2020/generic_model.pkl",
static_landmark_embedding_path="face3d_pytorch/assets/flame/flame_static_embedding.pkl",
dynamic_landmark_embedding_path="face3d_pytorch/assets/flame/flame_dynamic_embedding.npy",
batch_size=images.shape[0],
shape_params=shape.shape[-1],
expression_params=exp.shape[-1],
pose_params=pose.shape[-1],
).cuda()
mesh_dict = flame(shape, exp, pose)
for key, value in mesh_dict.items():
print(f"{key}: {value.shape}")
# verts: torch.Size([58, 5023, 3])
# faces: torch.Size([9976, 3])
# landmarks: torch.Size([58, 68, 3])Visualization:
import matplotlib.pyplot as plt
from torchvision.utils import make_grid
from face3d_pytorch.core.d3mm.utils import render_shape
renderred_mesh = render_shape(vertices=mesh_dict["verts"], faces=mesh_dict["faces"], cam=params_dict["cam"], image_size=224)
aligned_images = align_dict["aligned_images"]
renderred = 0.5 * aligned_images + 0.5 * renderred_mesh
plt.figure(figsize=(40, 40))
plt.imshow(make_grid(renderred, nrow=8).permute(1, 2, 0).detach().cpu().numpy())
plt.axis("off")Inverse-align to original coordinates:
from face3d_pytorch.core.preprocessing.face_alignment import deca_inverse_align
out = deca_inverse_align(renderred_mesh, align_dict["align_params"], output_size=(256, 256))
out = 0.5 * video + 0.5 * out
plt.figure(figsize=(40, 40))
plt.imshow(make_grid(out, nrow=8).permute(1, 2, 0).detach().cpu().numpy())
plt.axis("off")This project is licensed under the Apache License 2.0.



