Loom Spark 1.5
Second of the Loom models · Textile Labs · successor to Loom Spark v1 (which remains available, unchanged)
Loom Spark 1.5 keeps everything that made the first Loom different — trained from scratch to trade actual knowledge for wisdom: self-aware, honest about its limits, warm, and skilled at exactly one real superpower — forming clean web-search queries for its agent harness to execute.
What changed from v1
| v1 | 1.5 | |
|---|---|---|
| parameters | 7.6M | 12.3M (384d × 6 layers × 6 heads) |
| offline discipline | occasional <lookup> leaks | zero leaks in testing (mode markers on every turn) |
fake <result> blocks | possible outside harness | structurally impossible (turns end at the query) |
| non-sequitur answers | sometimes | fixed at the curriculum level |
| math & small facts | shaky | heavier training weight |
Validation loss 0.326 vs v1's 0.337 — and the wins that matter are behavioral: in a 30-question offline battery, v1 leaked tool markup into plain chat constantly; 1.5 leaked zero times, and answers identity/restraint questions correctly from every angle we tried.
Prompt format (IMPORTANT — it is sensitive to format)
Same protocol as v1, but the mode marker now rides on EVERY user turn:
<tools:off>
<tools:off><user> what year did the Titanic sink?
<loom>
With tools on, the model ends its turn with <lookup>query</lookup> followed by
<|endoftext|>; YOUR harness executes the search and splices <result>…</result>
before continuing with <loom>. Stop generation at <|endoftext|> or <user>.
Option A — plain transformers
from transformers import GPT2LMHeadModel, AutoTokenizer
import torch
tok = AutoTokenizer.from_pretrained("TextileLabs/Loom-Spark-1.5")
model = GPT2LMHeadModel.from_pretrained("TextileLabs/Loom-Spark-1.5")
prompt = "<tools:off>\n<tools:off><user> who are you?\n<loom>"
ids = tok(prompt, return_tensors="pt").input_ids
out = model.generate(ids, max_new_tokens=90, do_sample=True, temperature=0.8,
top_k=50, pad_token_id=tok.eos_token_id)
print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=False))
This is a standalone second-generation model — v1 keeps its own repo and card. Tested on transformers ≥ 4.40 and Python 3.9–3.13. The widget above prefills the correct format — keep both
<tools:…>markers or quality drops sharply.
Option B — llama.cpp / Ollama (offline persona)
loom-spark-1.5-f32.gguf carries the weights plus the custom BPE tokenizer with all
nine special tokens embedded. The ollama/Modelfile in this repo produces the offline
persona (greetings, identity, honest deferrals):
ollama pull hf.co/textilelabs/Loom-Spark-1.5
ollama create loom-spark-15 -f Modelfile
ollama run loom-spark-15 "hi"
Raw runners can't execute searches — if a reply contains a <lookup>…</lookup>
query, that's the model saying "I'd look this up". For real internet answers,
use the harness.
Option C — the harness (with internet)
Same package as v1 (harness/ folder here) — it auto-detects 1.5's format:
pip install ./harness
loom-chat # terminal agent, live DuckDuckGo searches
loom-web --port 7860 # local chat GUI with tools toggle + lookup feed
Architecture & training
Decoder-only transformer, pre-LN GELU blocks, tied embeddings, learned positions. 6 layers / 6 heads / d_model 384 / context 256 / vocab 4096 (fresh custom BPE). 12,318,... ≈ 12.3M params.
Trained entirely on our procedurally generated curriculum (zero external datasets) on a CPU-only Dell OptiPlex i5-4690: 3,914 steps ≈ 32M tokens over ~5.8 hours, final validation loss 0.343 (best checkpoint 0.326, shipped).
Limitations (by design)
Still knows almost nothing — that remains the point. Not for facts, medicine, law, finance, or anything where being wrong costs more than company.