fusion-embedding-2-tremor-g1

Tremor-G1 is the Unitree-humanoid specialization of Tremor, the inertial (IMU) sensor pack for the fusion-embedding stack. It maps a window of the robot's head-mounted accelerometer into the Qwen3-VL-Embedding-2B text space, so a Unitree G1/H1's motion becomes searchable and classifiable in plain language: "walking to a door", "picking something up", "adjusting a phone stand".

It is the per-fleet head of Tremor's two-tier design. Where the general Tremor base generalizes zero-shot to sensors it has never seen, Tremor-G1 is trained on one platform's IMU (Humanoid-Everyday, Unitree humanoid) for maximum in-domain accuracy on that platform. Both are built on the same frozen fusion-embedding-2 text space, so their motion embeddings are directly comparable to that model's audio, text, image, and video in one 2048-d space.

General Tremor base | fusion-embedding-2 | Tactus, the touch sense (32x32 pressure/taxel arrays) | Deployment guide

See it in action

A Unitree G1, full-body. The robot performs an activity sequence — sitting, walking, running, climbing — while a window slides across a real, held-out accelerometer stream and Tremor's scores update live, tracking the motion as it changes.

On the robot's own IMU and camera. On a real Unitree humanoid (Humanoid-Everyday), Tremor reads the robot's own IMU as language, live and frame-synced to its egocentric camera — zero simulation. As it switches from adjusting a phone stand to walking to a door, its IMU signature changes from flat to dynamic and Tremor's read tracks the activity.

Model Overview

Tremor-G1 is a frozen UniMTS motion encoder plus a small trained projector. A single accelerometer window is placed at one joint of UniMTS's orientation-invariant skeleton-graph encoder (ST-GCN), producing a 512-d motion feature; a projector of roughly 2.6M parameters maps that into the frozen Qwen base's 2048-d text space. Only the projector is trained — here, on the Unitree humanoid's IMU.

Training and Evaluation

Tremor-G1 trains the projector with InfoNCE against the frozen base's native chat-template text embeddings, on Humanoid-Everyday — the Unitree humanoid's head-mounted accelerometer paired with task descriptions. Only the projector learns.

The metric is k-way activity classification on held-out episodes and held-out tasks of the platform (chance = 1/k). On 1,915 held-out windows spanning 98 distinct tasks:

The cross-human reference is a general human-motion head applied to the same robot data; Tremor-G1's platform specialization roughly doubles in-domain accuracy over it. This is the two-tier lesson: per-fleet specialization buys large in-domain gains but does not improve cross-dataset transfer — for sensors or platforms outside this one, use the general Tremor base (0.502 held-out 5-way, zero-shot to unseen datasets).

Usage

• torch (CUDA recommended), numpy, transformers>=4.46, huggingface_hub
• The frozen UniMTS encoder code and weights (Apache-2.0 code):

  git clone https://github.com/xiyuanzh/UniMTS      # provides model.py::ST_GCN_18
  

  Point UNIMTS_REPO (or unimts_repo=) at the clone. The UniMTS weights download automatically.
• The frozen Qwen3-VL-Embedding-2B base downloads from its original repository.

import numpy as np
from inference import TremorEmbedder

tr = TremorEmbedder.from_pretrained(
    "EximiusLabs/fusion-embedding-2-tremor-g1",
    revision="v0.1-preview",
    unimts_repo="UniMTS",          # path to your git clone of xiyuanzh/UniMTS
)

# accel: a 3-axis accelerometer window from the robot, shape [3, T] (any length / rate)
accel = np.load("window.npy")

for text, score in tr.rank(accel, ["walking to a door", "picking something up",
                                    "adjusting a phone stand", "standing still"]):
    print(f"{score:+.3f}  {text}")

For the sensor contract, streaming, on-device inference (precompute the activity text bank once so the 2B base is not needed on the robot), and how to train your own per-fleet head, see DEPLOYMENT.md.

Engram

This pack is one of the modalities Engram searches. Engram is the open cross-modal memory layer for physical AI: it indexes a robot's video, audio, and motion into one embedding space and answers questions about it in plain language, including temporal reasoning that retrieval alone cannot.

pip install engram-robomem

Repo: https://github.com/Eximius-Labs/engram · PyPI: https://pypi.org/project/engram-robomem · Playground: https://www.eximiuslabs.com/playground

License

The trained projector weights in this repository are released under CC-BY-NC-4.0 (non-commercial).

This reflects the model's lineage. Tremor's frozen encoder is UniMTS, pretrained on synthetic IMU derived from the AMASS / HumanML3D motion corpora, which are released under a non-commercial research license, so Tremor ships non-commercial to respect that lineage.

Limitations

• Platform-specialized. Tuned to the Unitree humanoid's IMU; it does not improve — and should not be used for — zero-shot transfer to other sensors or platforms. Use the general base for that.
• Accelerometer only, single IMU mapped to one joint. Gyroscope is not modeled.
• English text only, and the base's chat-template formatting must be used.
• Research preview, not a high-stakes classifier.

Citation

@misc{tremor2026,
  title  = {Tremor: an inertial-motion sensor pack for the fusion-embedding space},
  author = {Tonmoy, Abdul Basit},
  year   = {2026},
  note   = {Eximius Labs. Model weights CC-BY-NC-4.0.},
  url    = {https://huggingface.co/EximiusLabs/fusion-embedding-2-tremor-g1}
}

Tremor's frozen motion encoder is UniMTS (Zhang et al., NeurIPS 2024), pretrained on synthetic IMU derived from AMASS (Mahmood et al., ICCV 2019). The text space is Qwen3-VL-Embedding-2B. Please cite those works as well.