Dia-1.6B
Nari Labs' Dia-1.6B dialogue TTS model, exported for loom.cpp. Family 10: text in, neural-codec tokens out -- pair it with dac-44khz-loom for audio.
This is a loom.cpp export: a single self-describing GGUF that carries its own graph topologies, tokenizer (if any) and driver script, produced by loom-exporter.
Original model
Exported from nari-labs/Dia-1.6B. Weights are unmodified; this repo packages the same parameters into
loom.cpp's GGUF format.
License
apache-2.0, inherited from the base model above.
Language(s)
en
Usage
Run it with loom-py -- loom-py-rt on PyPI:
pip install -U "loom-py-rt[hub]"
import loom
model = loom.Model.from_pretrained("loom-ai-org/dia-1.6b-loom")
# What comes back is codec TOKENS, not audio -- frame-major, one row per frame, `n_codebooks` wide.
# `max_new_tokens` counts AUDIO FRAMES rather than decoder steps.
codes = model.text2codes.infer(
"[S1] Hey, can you shut down the computer, my friend?",
max_new_tokens=260, seed=1234,
)
print(len(codes), "frames x", len(codes[0]), "codebooks")
# The second half of the pair, in a repo of its own: one codec serves many models like this one, and
# the codes are worth having on their own -- cache them, edit them, decode them somewhere else.
codec = loom.Model.from_pretrained("loom-ai-org/dac-44khz-loom")
audio = codec.codes2speech.infer(codes)
print(len(audio), "samples at", audio.sample_rate, "Hz =", round(audio.duration, 2), "s")
audio.save("out.wav")
# Nothing goes between those two calls. Both files declare the width of a frame, so a pair that does
# not fit says so instead of producing audio of the wrong duration:
print(model.hparam("codec.n_codebooks"), "==", codec.hparam("codec.n_codebooks"))
# This model SAMPLES by default, at the settings its own generation config declares. `seed=` above is
# what makes a result reproducible; drop it for a different take, or decode greedily for the same
# answer every time -- greedy is much flatter, and not what this checkpoint was tuned for.
print(model.hparam("sampling.temperature", "f32"), model.hparam("sampling.guidance_scale", "f32"))
The layer underneath
The call above is the high-level door: one per task, named for the modality pair it maps between, with
the windowing, sampling and assembly this model needs already applied. Under it, model.infer(...)
passes your arguments straight to the driver this GGUF embeds -- which is where you go for a knob the
door does not name.
model.driver_source prints that driver, including a header comment documenting every argument it
accepts for this model, and is the authority on it. See loom-py for the API and
loom.cpp for what the engine does between the two.
Known limitations
This model does not produce audio. It emits nine streams of DAC codec tokens, and a codec turns those into a waveform -- dac-44khz-loom, which is the codec this checkpoint was trained against. They stay separate because one codec serves many models like this one, and because the codes are the useful intermediate. The usage snippet above is the whole of the joining.
It samples, and it is high-variance. The export declares this checkpoint's own decoding -- temperature 1.8, top_k 50, top_p 0.9, and classifier-free guidance at 3.0 -- so two runs of the same sentence give two different takes. Some of them are not the sentence: on the snippet's own text, one seed in four gave it back verbatim, one gave laughter and two gave near-silence. That is the model rather than the export (transformers behaves identically), which is why the snippet names a seed. Expect to try several.
Guidance costs a second decoder pass at every step, so generation is about twice the work of a comparable LM. Pass guidance_scale=1.0 to turn it off -- faster, and worse.
max_new_tokens counts audio frames at 86.13 per second, not decoder steps: the two differ by this family's delay pattern, which the driver applies and undoes for you. Reaching the cap forces a clean ending rather than truncating, so a budget that is too small gives you a complete, shorter utterance.
Speaker tags are part of the text. [S1] and [S2] are tokens this checkpoint was trained on and are what make it a dialogue model; text without one is out of distribution. Non-verbal cues like (laughs) work the same way. Voices are not selectable -- without an audio prompt the model picks one, and the seed is what decides it.
It is a big download: 6.4 GB, F32, like every other model in this collection. These are reference exports as much as they are downloads, and one lossy artifact among seventeen faithful ones is a difference nothing in the file would tell you about. loom-export --quantize Q8_0 on the upstream checkpoint packs the eligible weights to about 1.8 GB if you would rather have that -- it moves the logits slightly, which for a sampler means a different take rather than a worse one.
Files
dia-1.6b.gguf-- the model, exported with loom-exporter.