LightNav-0
LightNav-0: Eliciting VLM Spatial Intelligence for Generalist Embodied Navigation
LightNav-0 is a compact generalist embodied navigation model that elicits the spatial intelligence of a pretrained vision-language model (Qwen3-VL) and aligns it with navigation, without task-specific prediction heads. Diverse tasks share one token interface: dual-channel pointing expresses task-, scene- and embodiment-agnostic spatial intent, and a residual vector-quantized (RVQ) action tokenizer maps that intent to precise, embodiment-specific trajectories — so instruction following, open-vocabulary object navigation and visual tracking live in a single model.
Inference, serving and evaluation code: LightNav-0 repository.
Files
| Path | Content |
|---|---|
config.json, model-*.safetensors, model.safetensors.index.json | Qwen3-VL weights (bf16) with the navigation token embeddings |
tokenizer.json, tokenizer_config.json, chat_template.jinja, processor_config.json | tokenizer / processor |
eval_config.json | processing parameters that must match training (video size, pooling, SlowFast history tiers, task settings); read automatically by the inference code |
action_tokenizer/ | RVQ action-tokenizer bundle shared by navigation and tracking (one 3 × 256 residual codebook, horizon 10) |
The bundle holds manifest.json, the per-level codebooks (codebook_l{0,1,2}.npy, 3 × 256 codes),
jacobian_weights.npy and alpha_per_source.json. The model emits one <act_l{level}_{code}>
token per level; the bundle decodes them into a (10, 3) chunk of robot-local waypoints
[forward_m, lateral_m (+left), yaw_rad (+ccw)]. eval_config.json references the bundle by a
path relative to this directory, so the inference code finds them without extra flags.
Usage
pip install "lightnav[vllm,video] @ git+https://github.com/lightrobo/LightNav-0"
hf download LightOriginsHQ/LightNav-0 --local-dir ./LightNav-0-ckpt
# offline prediction on a clip
lightnav-predict --model_path ./LightNav-0-ckpt --backend vllm_local \
--video clip.mp4 --fps 4 --instruction "follow the person in the red shirt"
# serve (tracking prompt) and drive with the reference client
lightnav-serve --task tracking --model_path ./LightNav-0-ckpt --backend vllm_local --port 8050
lightnav-ws-client --server ws://localhost:8050 --video clip.mp4 --fps 4 \
--instruction "follow the person in the red shirt"
# navigation prompt (instruction following / object navigation)
lightnav-serve --task vln --model_path ./LightNav-0-ckpt --backend vllm_local --port 8051
Habitat VLN-CE / ObjectNav and EVT-Bench evaluation recipes, the WebSocket protocol and the
real-robot deployment guide are in the code repository's docs/.
Model details
- Backbone: Qwen3-VL (4B), bf16; the trajectory / pointing tokens are ordinary rows of the
embedding table, so the checkpoint loads with stock
transformers. - Input: a SlowFast-compressed history of first-person RGB frames (model input 256×448, 4 fps) plus a natural-language instruction.
- Output: dual-channel pointing tokens (
<apos_*>/<opos_*>) followed by three RVQ action tokens (<act_l0_*><act_l1_*><act_l2_*>) decoded to a 10-step waypoint chunk.
Community
Questions, deployment notes and release news — join us on Discord, or scan to join the WeChat group:
License
Apache License 2.0.