EgoSieve-S
EgoSieve-S ranks manipulation-ready spans in first-person video. It produces
three readiness logits (KEEP, REVIEW, REJECT), seven observable issue
scores, diagnostic start/end boundary proposals, and a normalized retrieval
embedding. It is a dataset-curation model, not a robot policy.
Usage
from transformers import AutoModelForVideoClassification, AutoProcessor
processor = AutoProcessor.from_pretrained(
"itspublu/EgoSieve-S", revision="v0.1.0", trust_remote_code=True
)
model = AutoModelForVideoClassification.from_pretrained(
"itspublu/EgoSieve-S", revision="v0.1.0", trust_remote_code=True
).eval()
outputs = model(**processor(frames, return_tensors="pt"))
The timestamp-aware scanner and JSONL compiler are provided by the egosieve
package. The checkpoint expects 12 center-sampled
RGB frames per window; use its bundled processor.
Training and evaluation
Data represented in the held-out evaluation: HoloAssist (CDLA-Permissive-2.0), HoloAssist controlled corruptions (CDLA-Permissive-2.0). Splits are grouped by original capture unit. Readiness, calibration, and boundary results use 142 human-grounded readiness rows: 0 direct human and 142 human-derived. Boundary results use 97 human-grounded boundary rows. Issue results use 145 labeled rows: 0 human, 109 human-derived, and 36 programmatic controlled corruptions. Unlabeled task targets are masked. The release bundle includes raw held-out predictions with task-level provenance, split assignments, exact run configuration, and metric provenance.
Human-derived rows are not direct, independent EgoSieve rubric judgments. Treat them as proxy evidence and consult the source dataset cards and their dataset-specific proxy details before comparing or interpreting these metrics.
For v0.1, readiness and boundaries come from a fixed-grid occupancy rule over
reviewed HoloAssist fine-action intervals. low_hand_activity is an occupancy
proxy, and acting_hand_not_visible follows HoloAssist's acting-hand modifier;
it does not assert that every hand is absent. The other five issue metrics
measure injected-corruption versus unmodified-reference discrimination. Those
references were not independently audited as natural issue negatives.
- Readiness macro F1: 0.6739
- Issue macro AUROC: 0.7407
- Issue macro average precision: 0.7222
- Boundary micro F1 at 0.30s: 0.1030
- Readiness ECE: 0.1018
Intended use and limitations
Use the model to rank raw egocentric windows, route uncertain spans for review, and create embeddings for near-duplicate search. Readiness remains dependent on the published rubric and capture domain. RGB cannot establish force, physical success, consent, safety, metric depth, or legal publishability. Boundary scores are proposals and are diagnostic-only in the v0.1 compiler.
First-person recordings can contain faces, screens, homes, and bystanders. Apply a separate privacy and consent review before sharing any media.