min-spark 1.1

min-spark 1.1 is a 5.76M-parameter language model with native effort levels.

What's new in 1.1

This release is min-spark 1.0 trained for longer. The architecture and vocabulary are untouched, and the parameter count is still 5.76M. What changed is the amount of training. We continued pre-training on another 6.8B tokens, bringing the total up to 16.81B.

That extra training made most of the benchmarks better. ARC-Easy dipped a little, which we consider a fair trade for the gains everywhere else:

Effort levels

If you have used large reasoning models before, effort levels there usually mean a thinking-token budget or a modified system prompt. min-spark takes a different route: each level selects how many passes the model makes through its shared core, so the compute difference happens inside the network itself. The interface is the same one introduced in min-spark 1.0, and nothing about it changed in this release.

In practice, grammar quality benefits most from raising effort. BLiMP jumps noticeably between low and medium, while the other tasks stay roughly flat across settings.

Evaluation

All scores below are zero-shot with lm-eval 0.4.12. BLiMP reports accuracy, ARC-Easy, ARC-Challenge, HellaSwag and PIQA report length-normalized accuracy, and WikiText-2 is byte-level perplexity where lower is better.

Benchmarks

Here is how the best-effort scores compare against current sub-10M entries on the Open SLM leaderboard. Peer numbers come from their leaderboard submissions.

The BananaMind Base Bench tells a similar story. At the best effort setting, version 1.0 reached an Elo of 920 and this release reaches 935:

Splitting those results by effort shows an interesting difference between the two releases. Version 1.0 scored 905 at low effort and then saturated at 920 once you gave it more compute. Version 1.1 keeps improving all the way up: 910 at low, 932 at medium, and 935 at high. In other words, the extra training made higher effort settings actually worth using.

Usage

min-spark 1.1 works with Transformers through remote code loading:

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "MinimaLabs/min-spark-1.1",
    trust_remote_code=True,
).to("cuda")
tokenizer = AutoTokenizer.from_pretrained(
    "MinimaLabs/min-spark-1.1",
    trust_remote_code=True,
)

prompt = "The meaning of life is"
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
outputs = model.generate(
    **inputs,
    effort="high",
    max_new_tokens=64,
)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

The Transformers pipeline API works too:

from transformers import pipeline

pipe = pipeline("text-generation", model=model, tokenizer=tokenizer)
result = pipe(prompt, effort="high", max_new_tokens=64)
print(result[0]["generated_text"])

If you would rather skip Transformers entirely, the repo ships a small standalone script:

python generate.py -p "The meaning of life is" -e high

A few practical notes. Generation processes one sequence at a time, though right-padded batches work fine for evaluation. There is no KV cache yet, so each generated token recomputes the whole prompt, which slows down long generations somewhat. The context window is 512 tokens.

Architecture

Training

Reproducing the evaluation

python run_lmeval.py \
  --effort medium \
  --tasks blimp,arc_easy,arc_challenge,hellaswag,piqa,wikitext

Add --limit N if you want a quicker run while poking at things.

Limitations

This is a small base model. It has not been instruction-tuned, so do not expect conversational behavior, alignment, or safety filtering. Its factual recall, reasoning, and long-form coherence are limited by its size, as you would expect from 5.76M parameters.

The context window is fixed at 512 tokens and generation accepts one sequence at a time with no KV cache. All published results were collected within that window, and we cannot promise anything about behavior beyond it.

License

Apache-2.0. See LICENSE.