Noema 1.5 2B

Noema 1.5 2B is an open-weight, local-first language model optimized for broad knowledge retention, precise instruction following, multi-turn constraint persistence, short-form code generation, and efficient reasoning.

It is the most balanced release in the Noema 2B lineage.

Compared with the previous Noema release, Noema 1.5 improves pooled knowledge, MMLU-Pro development, HumanEval+, strict IFEval, verified mathematics, and novel constraint following. Against stock Qwen3.5-2B in final evaluation, it improves strict IFEval by 7.02 points, complete three-turn Multi-IF success by 3.11 points, HumanEval+ by 3.05 points, and reasoning efficiency by 13.5%, while passing a preregistered full MMLU-Pro knowledge-retention gate.

No retrieval, external tools, test-time answer repair, or external model calls were used in the reported benchmark results.

Highlights

• Knowledge recovery: +1.42 points on the 1,900-item pooled knowledge development composite over Noema v2.
• MMLU-Pro recovery: +2.86 points over v2 on the matched development panel.
• Stronger code: HumanEval+ increased from 50.00% to 53.66% over v2.
• Better instructions: strict IFEval increased from 70.24% to 73.01% over v2.
• Better multi-turn behavior: complete three-turn Multi-IF success was 21.1% higher than stock Qwen3.5-2B.
• Efficient reasoning: 13.5% fewer completion tokens than stock Qwen3.5-2B on the final thinking panel.
• Broad knowledge retained: passed the preregistered MMLU-Pro non-inferiority gate across 11,332 untouched questions.
• Open-weight deployment: downloadable weights built for private and resource-conscious inference.

Release-over-release results

The table below compares Noema 1.5 with the exact previous Noema checkpoint from which training began.

These are paired development and selection results. They demonstrate release-over-release progress but are not untouched final-lockbox estimates.

Final benchmark results

The final evaluation compared the frozen Noema 1.5 candidate with the exact stock Qwen3.5-2B foundation using identical prompts, generation settings, and graders.

Statistical interpretation

• MMLU-Pro: passed the preregistered non-inferiority test. The one-sided 95% lower bound was -1.438 points against a -1.5-point margin.
• IFEval: +7.02 points, statistically significant (p = 0.000475).
• Multi-IF: +3.11 points on complete three-turn success, statistically significant (p = 4.44e-8).
• HumanEval+ and thinking: favorable point estimates that were not statistically resolved.
• GPQA-Diamond and IFBench: negative point estimates and active areas for future research.

MMLU-Pro was the strict confirmatory endpoint. The final evaluation report is available in the Noema training repository.

Model details

Noema 1.5 is the text-generation trunk. Multimodal capabilities were not trained or evaluated in this release.

Training

Noema 1.5 began from the exact previous Noema 2B release and used a restoration-aware on-policy distillation program:

• 150 optimizer steps;
• 19,200 on-policy trajectories;
• 7,982,979 valid completion tokens;
• 32 unique prompts per step;
• four student rollouts per prompt;
• rank-128 LoRA with alpha 256;
• Gated DeltaNet, ordinary-attention, and MLP projections targeted;
• tied embedding and output matrix kept frozen.

Seventy percent of assignments used Qwen3.5-9B as the capability scorer. Thirty percent used untouched Qwen3.5-2B as a restoration scorer to protect general behavior.

The training mixture contained:

• 52.5% broad knowledge;
• 17.5% mathematics and code;
• 22.5% general-chat restoration;
• 7.5% novel composed constraints.

An experimental secondary constraint-RL stage was tested, failed its first mandatory gate, and was rejected. It did not modify the released checkpoint.

Intended uses

Noema 1.5 is well suited to:

• local and privacy-sensitive assistants;
• structured and format-constrained generation;
• multi-turn conversations;
• short Python program synthesis;
• educational and lightweight reasoning tasks;
• offline applications;
• memory- and compute-constrained deployments.

For knowledge-intensive applications, use retrieval or authoritative tools and verify important factual claims.

Usage

Noema 1.5 requires a transformers version with qwen3_5_text support. The release was built and validated with Transformers 5.12.1.

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "NoemaAI-labs/Noema-1.5-2B"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    dtype=torch.bfloat16,
    device_map="auto",
)

messages = [
    {
        "role": "user",
        "content": "Write a Python function that merges two sorted lists.",
    }
]

inputs = tokenizer.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    enable_thinking=False,
    return_dict=True,
    return_tensors="pt",
).to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=1024,
    do_sample=False,
)

response = tokenizer.decode(
    outputs[0][inputs["input_ids"].shape[-1]:],
    skip_special_tokens=True,
)

print(response)

Non-thinking mode is recommended for concise instruction following, structured generation, and code. The packaged defaults are greedy (do_sample=false) and stop at <|im_end|>.

For harder reasoning tasks, use enable_thinking=True, do_sample=True, temperature=1.0, top_p=0.95, and top_k=20. Deployments should still enforce output limits and repeated-span detection.

Evaluation methodology

The final evaluation used:

• frozen model and tokenizer revisions;
• identical prompts and settings across both arms;
• BF16 inference on NVIDIA L40S hardware;
• vLLM 0.19.1;
• pinned official graders;
• paired bootstrap intervals and McNemar tests;
• deterministic keys and exact-count verification;
• no score inspection before both arms were complete.

Limitations

• The release-over-release v2 comparison uses matched development suites, not a third untouched final arm.
• Broad MMLU-Pro knowledge was statistically retained against stock but not improved in the final point estimate.
• GPQA-Diamond and IFBench produced lower final point estimates.
• The release is based on one primary training seed and does not establish that on-policy distillation beats matched supervised training.
• Evaluation was primarily English and text-only.
• Native 262K context support comes from the Qwen3.5 architecture; Noema independently evaluated contexts only up to 24,576 tokens.
• Thinking mode can still produce long or repetitive responses.
• The model can hallucinate and should not be treated as the sole authority for high-stakes decisions.

Release status

Noema 1.5 2B is an open-weight release, not an open-source release. The weights are publicly downloadable, but no open-source license is granted with this repository at this time.

Citation

@misc{noema15_2b_2026,
  title        = {Noema 1.5 2B: Restoration-Aware On-Policy Distillation for Local AI},
  author       = {NoemaAI Labs},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/NoemaAI-labs/Noema-1.5-2B}}
}