LibrePEs16-cls
Perception Encoder (PE) Core PE-Core-S/16-384 packaged for LibreYOLO.
PE Core is a dual-tower vision-language encoder: it maps images, text, and whole videos into one normalized embedding space, and classifies zero-shot from text prompts.
from libreyolo import LibreYOLO
# zero-shot classification
model = LibreYOLO("LibrePEs16-cls.pt")
model.set_classes(["a forklift", "an empty aisle", "a spill"])
r = model.predict("warehouse.jpg")[0]
print(model.names[r.probs.top1], float(r.probs.top1conf))
# embeddings: image, text, and whole video in one space
embedder = LibreYOLO("LibrePEs16-cls.pt", task="embed")
embedder.predict("photo.jpg") # (1, 512) image row
embedder.embed_text(["a dog"]) # (1, 512) text row
embedder.predict("clip.mp4") # (1, 512) row for the whole clip
The bare canonical filename is the download trigger - there is no
from_pretrained.
Specification
| Family / size | pe / s16 |
| Tasks | classify (zero-shot), embed |
| Input resolution | 384x384 |
| Embedding dimension | 512 |
| Text context length | 32 |
| Preprocessing | RGB, bilinear square resize, mean/std = (0.5, 0.5, 0.5) |
| Video pooling | mean of independently encoded frames, L2-normalized once |
| Training | not supported (inference-only; train() raises) |
| Exports | ONNX, TorchScript |
Source
- Upstream repository:
timm/PE-Core-S-16-384 - Upstream revision:
3249a38eb1c432ec19231c6fe1774acb6a4e4efe - Upstream license: Apache-2.0
This is a converted OpenCLIP-compatible PE Core checkpoint, not an
unmodified official facebook/PE-Core-* package checkpoint.
Modifications
State-dict metadata wrap only (weights/convert_pe_weights.py): LibreYOLO
checkpoint metadata is added and tensors are moved to CPU. Learned parameters
are unchanged.
Parity against unmodified open_clip_torch==3.2.0 on float32 CPU is exact
(max_abs_diff == 0.0) for image embeddings, text embeddings, zero-shot logits,
and fixed-frame video embeddings.
License
Apache-2.0, inherited from the upstream weights. See LICENSE and NOTICE in
this repository. You are responsible for complying with those terms.