ROS 2 documentation embeddings
all-MiniLM-L6-v2 fine-tuned on the ROS 2 documentation, for retrieval over
that corpus.
- Comparison Space: https://huggingface.co/spaces/eoinedge/ros2-embeddings-compare
- Index and pipeline: https://huggingface.co/eoinedge/ros2
- Source: https://github.com/eoinjordan/ros2-coding-assistant
What it is for
The base model is trained on general web text, where "node" means a graph vertex and "action" means a UI event. This copy is adapted to ROS 2's vocabulary so those senses separate.
Use it as a drop-in replacement for the base model when embedding ROS 2 documentation:
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("eoinedge/ros2-docs-embeddings")
vectors = model.encode(passages, normalize_embeddings=True)
Training
| Base | sentence-transformers/all-MiniLM-L6-v2 |
| Pairs | 4,507 mined from the docs' own heading/body structure |
| Loss | MultipleNegativesRankingLoss (in-batch negatives) |
| Epochs | 1 |
| Batch size | 32 |
| Held-out pairs | 400 |
No hand-labelled data and no synthetic questions from a generator — a section heading is a natural query for the body beneath it, and it is already written by the documentation authors.
Results, and how to read them
| Metric | Base | Tuned |
|---|---|---|
| recall@5 on held-out pairs | 0.565 | 0.8225 |
That number flatters the model, and you should treat it with suspicion. The evaluation measures the same heading→body relationship the model was trained on, so it partly measures whether training converged rather than whether retrieval improved.
Spot-checking real questions gives a more mixed picture: some clearly improve, several are unchanged, and at least one regressed. Real questions are not phrased like section headings, which is exactly the gap this evaluation does not cover.
The comparison Space runs both models on the same query so you can judge for yourself rather than trusting the headline.
Limitations
- One epoch, one seed, no hyperparameter search.
- Evaluation is aligned with the training objective, as above.
- Trained on ROS 2 documentation only — it will not help on other corpora and may be worse than the base model there.
- This adapts the retriever, not a generator. It does not teach a coding model anything about ROS 2; it changes which passages that model is handed.
Licence
Apache-2.0, matching the base model.