3CTeam/fpvla-fsdp-dit-stage1-30k
FP-VLA (Vision-Language-Action) model with trace prediction capability, fine-tuned from openvla/openvla-7b.
Model Architecture
| Component | Type |
|---|---|
| Trace Head | dit |
| Action Head | mlp |
DiT Trace Head
- Iterative denoising diffusion for trajectory generation
- Inference steps: 50-500 (trade-off speed vs quality)
- Input: 66 learnable query tokens (conditioned on image + past trajectory)
- Output: 33-point normalized trajectory [33, 2]
Training Info
- Checkpoint step: 30000
Usage
from prismatic.extern.hf.configuration_prismatic import OpenVLAConfig
from prismatic.extern.hf.modeling_prismatic import OpenVLAForActionPrediction
# Load config + model (constructor + manual weight loading)
config = OpenVLAConfig.from_pretrained("YOUR_REPO_ID")
model = OpenVLAForActionPrediction(config)
import glob
from safetensors.torch import load_file
state_dict = {}
for shard in sorted(glob.glob("YOUR_REPO_ID/*.safetensors")):
state_dict.update(load_file(shard))
model.load_state_dict(state_dict, strict=False)
Evaluation
python vla_scripts/eval_trace_mix.py \
--model_path YOUR_REPO_ID \
--base_vla_path openvla/openvla-7b \
--data_root_dir data \
--dataset_name trace_mix \
--output_dir runs/eval \
--max_samples 64
License
Apache-2.0