🫐 Blueberry-3

Blueberry-3 is a fine-tuned version of openai/gpt-oss-120b, trained as a dedicated multilingual reasoning model across six languages: Farsi, English, Arabic, French, German, and Spanish. It generates coherent chain-of-thought reasoning natively in each of these languages, rather than silently falling back to English regardless of the input or output language.

Blueberry-3 extends the line of work started by Strawberry-1 — which first demonstrated that open-source models could reason natively in Farsi instead of defaulting to English — and broadens it, via Blueberry-1, into a model that reasons natively across a defined set of six languages rather than treating Farsi as the sole non-English focus.

Model Details

• Base model: openai/gpt-oss-120b (117B parameters)
• Architecture: gpt_oss
• Fine-tuned by: artindnr
• License: Apache 2.0
• Languages: Farsi, English, Arabic, French, German, Spanish
• Model type: Causal decoder-only language model with reasoning ("thinking") traces
• Related models: Strawberry-1 (20B, Farsi + multilingual) · Blueberry-1 (120B, Farsi + multilingual) · Blueberry-2 (120B, Farsi-English bilingual)

What's New

Most open reasoning models generate their chain-of-thought almost exclusively in English, even when the final answer is requested in another language. Blueberry-3 tackles this across six languages at once:

• Generates coherent reasoning traces natively in Farsi, English, Arabic, French, German, or Spanish
• Maintains consistent reasoning quality across all six languages rather than treating one as primary and the rest as an afterthought
• Preserves the general instruction-following and reasoning ability of the gpt-oss-120b base model
• Builds on the larger-scale training established in Blueberry-1, applied to a wider language set

Training

Blueberry-3 was trained using a mix of fine-tuning strategies — including full fine-tuning and LoRA experiments — on top of gpt-oss-120b, with training data spanning all six target languages plus mixed-language examples. The version released here is the fully fine-tuned (merged, dense-weights) checkpoint, not a LoRA adapter.

How to Use

Blueberry-3 uses the gpt-oss chat template (Harmony format) shipped with the base model, so it works with 🤗 Transformers.

Installation

pip install torch --index-url https://download.pytorch.org/whl/cu128
pip install trl peft transformers kernels

Generation

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

MODEL_ID = "artindnr/blueberry-3"

tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
model = AutoModelForCausalLM.from_pretrained(
    MODEL_ID,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

REASONING_LANGUAGE = "English"  # e.g. "English", "Farsi", "Arabic", "French", "German", "Spanish"
SYSTEM_PROMPT = f"reasoning language: {REASONING_LANGUAGE}"
USER_PROMPT = "تو کی هستی و اسمت چیه؟"

messages = [
    {"role": "system", "content": SYSTEM_PROMPT},
    {"role": "user", "content": USER_PROMPT},
]

inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    tokenize=True,
    return_dict=True,
    return_tensors="pt",
).to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=512,
    temperature=0.6,
    do_sample=True,
)

print(tokenizer.decode(outputs[0]))

This prints the full Harmony-formatted output, including the analysis (reasoning) and final (answer) channels and their special tokens. To get just the plain-text final answer, decode with skip_special_tokens=True and parse out the final channel, or use tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True) to only decode the newly generated tokens.

Reasoning in a specific language

Set reasoning language: <Language> as the system message content to control the language of the reasoning trace (the analysis channel), independent of the language the user writes in. Any of the six supported languages can be set here, regardless of the prompt's language.

Note that the model's default chat template also auto-populates a Harmony-format preamble (identity, knowledge cutoff, current date, reasoning effort, valid channels) ahead of your system/developer message — you don't need to set these yourself.

Relationship to the Strawberry and Blueberry series

Choose Blueberry-1 for general Farsi-centric multilingual use, Blueberry-2 when your workload is specifically Farsi-English, and Blueberry-3 when you need reliable reasoning across all six supported languages.

Intended Use

Blueberry-3 is intended for:

• Multilingual assistants and applications serving users across Farsi, English, Arabic, French, German, and Spanish
• Research on cross-lingual chain-of-thought consistency
• Reasoning-heavy applications that need to support several of these languages without a strong English bias

Limitations

• Reasoning quality is optimized for the six listed languages; performance in other languages is not guaranteed and may fall back toward the base model's default behavior.
• As with any fine-tune, Blueberry-3 inherits the general capabilities and limitations of the gpt-oss-120b base model, including the possibility of hallucinated facts and reasoning errors.
• No formal safety fine-tuning beyond what is inherited from the base model has been applied; use appropriate safeguards in production settings.
• Spreading training across six languages may trade off some peak per-language quality against Blueberry-2's narrower Farsi-English focus.

License

This model is released under the Apache 2.0 license, consistent with the base gpt-oss-120b model.

Citation

If you use Blueberry-3 in your work, please cite:

@misc{blueberry3,
  title  = {Blueberry-3: A Multilingual Reasoning Model Fine-tuned from GPT-OSS-120B},
  author = {artindnr},
  year   = {2026},
  url    = {https://huggingface.co/artindnr/blueberry-3}
}

Acknowledgements

Built on top of openai/gpt-oss-120b. Blueberry-3 builds on the groundwork laid by Strawberry-1, Blueberry-1, and Blueberry-2.