Voxtral-Mini-3B Text-Llama
vxltxrllc/voxtral-mini-3b-text-llama is a standalone, text-only extraction of the language model backbone from Mistral AI's Voxtral-Mini-3B.
By stripping away the speech encoder and cross-attention/audio adapter projections, this repository provides a lightweight (~3B parameters) pure-text decoder converted directly into standard Llama architecture (LlamaForCausalLM). Key Highlights
Standard Llama Compatibility: Fully compatible out-of-the-box with vLLM, llama.cpp, Ollama, SGLang, TGI, and standard transformers pipelines without custom model code or multimodal overrides.
Optimized Footprint: Removes all non-text weights, resulting in faster cold starts, lower RAM/VRAM consumption, and clean quantization pipelines (GGUF, AWQ, GPTQ).
Architecture: Pure Causal LM based on the Transformer architecture using Rotary Position Embeddings (RoPE), SwiGLU activations, and Grouped-Query Attention (GQA).
Model Details Developed by: Extraction & conversion by vxltxr LLC && Original base weights by Mistral AI Model Type: Causal Language Model (LlamaForCausalLM) Parameter Count: ~3B Context Window: 8,192 tokens License: MIT
Quickstart & Usage
- Using Hugging Face transformers
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
model_id = "vxltxrllc/voxtral-mini-3b-text-llama"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype=torch.bfloat16,
device_map="auto"
)
prompt = "Explain the difference between streaming ASR and offline ASR in two sentences."
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(
**inputs,
max_new_tokens=128,
temperature=0.7,
top_p=0.9,
do_sample=True
)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
- Fast Serving with vLLM
Because the model was mapped directly to LlamaForCausalLM, you can launch it with vLLM immediately: Bash
vllm serve vxltxrllc/voxtral-mini-3b-text-llama \
--dtype bfloat16 \
--max-model-len 8192 \
--port 8000
Extraction & Conversion Methodology
The weight conversion pipeline performs the following surgical operations:
Filter Keys: Strips audio encoder blocks (audio_encoder.*), projector layers, and cross-attention keys.
Key Mapping: Remaps internal layer naming to standard LlamaForCausalLM conventions (model.layers.{i}.self_attn.*, model.embed_tokens, lm_head).
Config Synthesis: Generates a standard config.json defining architectures: ["LlamaForCausalLM"] with native RoPE theta and GQA parameters matching the extracted shapes.
Intended Use & Limitations Intended Use
Lightweight text generation and conversational modeling.
Downstream fine-tuning (LoRA, QLoRA, DPO/ORPO).
Edge device deployment via llama.cpp / GGUF quantization.
Limitations
No Audio Processing: This model cannot accept audio/speech inputs. For speech-to-text or multimodal tasks, use the original mistralai/Voxtral-Mini-3B.
Inherited Biases: Retains any baseline capabilities, factual limitations, or language biases present in the original model.
Citation & Acknowledgments
If you use this converted model in your work or benchmarks, please cite the original authors and the extraction repository: Code snippet
@misc{voxtral_mini_3b_text_llama_2026, author = {vxltxr LLC}, title = {Voxtral-Mini-3B Text-Llama: Standalone Llama Extraction}, year = {2026}, publisher = {Hugging Face}, journal = {Hugging Face repository}, howpublished = {\url{https://huggingface.co/vxltxrllc/voxtral-mini-3b-text-llama}} }