EdgeFace for 🤗 Transformers

EdgeFace (Idiap Research Institute) packaged as a transformers custom model. All four published variants live in this single repository as subfolders and are accessible through the standard AutoModel / AutoImageProcessor API with built-in MediaPipe face alignment.

EdgeFace replaces the classifier of an edgenext (timm) backbone with a 512-d embedding head trained for face recognition. Two variants additionally apply a static low-rank factorization to their linear layers — EdgeFace's "gamma" trick, baked into the pretrained weights and unrelated to PEFT adapters.

Model variants

Installation

pip install transformers timm torch safetensors huggingface_hub numpy

# Face alignment (do_align=True) also requires:
pip install mediapipe opencv-python

Quick start

Pipeline (default — edgeface-xxs)

from transformers import pipeline

pipe = pipeline("image-feature-extraction", model="anjith2006/edgeface", trust_remote_code=True)

AutoModel (any variant)

import torch
import torch.nn.functional as F
from PIL import Image
from transformers import AutoModel, AutoImageProcessor

repo = "anjith2006/edgeface"
variant = "edgeface-xxs"   # or edgeface-base / edgeface-s-gamma-05 / edgeface-xs-gamma-06

model = AutoModel.from_pretrained(repo, subfolder=variant, trust_remote_code=True).eval()
processor = AutoImageProcessor.from_pretrained(repo, subfolder=variant, trust_remote_code=True)

@torch.no_grad()
def embed(path):
    img = Image.open(path).convert("RGB")
    inputs = processor(img, return_tensors="pt")   # do_align=True by default
    return F.normalize(model(**inputs).embeddings, dim=-1)

score = F.cosine_similarity(embed("a.jpg"), embed("b.jpg")).item()
print(f"{score:.4f}")  # → ~0.9+ same person, lower for different

Face alignment

The image processor detects and aligns the face by default, warping it onto the ArcFace 112×112 template using 5 MediaPipe landmarks — the same alignment the weights were trained with.

# Full image → detect face, align, normalize (default)
inputs = processor(img, return_tensors="pt")

# Pre-aligned 112×112 crop → skip detection, just normalize
inputs = processor(crop, do_align=False, return_tensors="pt")

# Known landmarks → skip detection, align from provided 5 points
inputs = processor(img, landmarks=pts, return_tensors="pt")  # pts: ndarray (5, 2)

If detection fails the processor falls back to a plain resize so batches never crash.

MediaPipe backend

# "auto" (default): try the Tasks API, fall back to legacy solutions.face_mesh
# "tasks":          force the modern API — downloads face_landmarker.task once to ~/.cache/edgeface/
# "solutions":      force the legacy API (older mediapipe installs)
processor = AutoImageProcessor.from_pretrained(
    repo, subfolder=variant, trust_remote_code=True, mp_backend="tasks"
)

# Offline / custom bundle:
processor = AutoImageProcessor.from_pretrained(
    repo, subfolder=variant, trust_remote_code=True,
    mp_model_path="/path/to/face_landmarker.task"
)
# or: export EDGEFACE_MP_MODEL=/path/to/face_landmarker.task

Batch usage

imgs = [Image.open(p).convert("RGB") for p in paths]
inputs = processor(imgs, return_tensors="pt")
with torch.no_grad():
    embs = F.normalize(model(**inputs).embeddings, dim=-1)  # (N, 512)

Local import without trust_remote_code

Clone the source repo and import the package directly:

from edgeface import register_edgeface
register_edgeface()  # wires EdgeFace into AutoConfig / AutoModel / AutoImageProcessor

model = AutoModel.from_pretrained("anjith2006/edgeface", subfolder="edgeface-xxs").eval()
processor = AutoImageProcessor.from_pretrained("anjith2006/edgeface", subfolder="edgeface-xxs")

LoRA fine-tuning

The static low-rank layers in the gamma variants are plain nn.Linear modules, so PEFT targets them without any naming collision:

from peft import LoraConfig, get_peft_model

model = AutoModel.from_pretrained(repo, subfolder=variant, trust_remote_code=True)

# Gamma variants (edgeface-s-gamma-05, edgeface-xs-gamma-06):
lora_cfg = LoraConfig(r=8, lora_alpha=16, target_modules=["linear1", "linear2"])

# Base / XXS variants (no factorized layers — target the backbone linears directly):
# print([n for n, _ in model.named_modules() if isinstance(_, torch.nn.Linear)])
lora_cfg = LoraConfig(r=8, lora_alpha=16, target_modules=["fc1", "fc2"])

model = get_peft_model(model, lora_cfg)
model.print_trainable_parameters()

Source files

License

The pretrained weights and original alignment code are © Idiap Research Institute. The original EdgeFace license governs all weight files and derivative uses. See NOTICE for details. Verify compliance before commercial use or redistribution.

Citation

@article{george2024edgeface,
  title   = {EdgeFace: Efficient Face Recognition Model for Edge Devices},
  author  = {George, Anjith and Ecabert, Christophe and Otroshi Shahreza, Hatef
             and Kotwal, Ketan and Marcel, Sebastien},
  journal = {IEEE Transactions on Biometrics, Behavior, and Identity Science},
  year    = {2024},
  doi     = {10.1109/TBIOM.2024.3352169}
}