Nemotron-3.5-30B-A3B-Antislop-FTPO

A merged, ready-to-serve BF16 checkpoint of NVIDIA's Nemotron 3.5 30B-A3B, tuned with Antislop and FTPO to suppress the overused lexical patterns that make LLM prose recognisable as LLM prose.

66.4% of overused patterns suppressed, with no statistically detectable loss of writing quality and no capability regression.

The LoRA adapter that produced this checkpoint is released separately at thoughtworks/Nemotron-3.5-30B-A3B-Antislop-FTPO-LoRA.

What this is

Language models overuse a characteristic vocabulary: stock dialogue tags, a fixed set of atmospheric props, and a long tail of names like Elara and Kael. Antislop identifies those patterns for this specific model by comparing its output against a human-written baseline, then uses a backtracking sampler to catch the model reaching for one and record what it should have reached for instead. Those records become preference pairs, and FTPO (Final Token Preference Optimization) trains the preference into the weights so it persists with the sampler switched off.

FTPO adjusts only the specific token choices that need adjusting, holding the rest of the vocabulary steady. That precision is what keeps capabilities intact.

What gets banned

The banlist is not a hand-written style guide. It is 4,267 patterns measured as overused in this model's own output relative to human prose. A sample, with raw counts across the 400 held-out prompts, sampler off, so these are weight-level changes:

Every count above is spread across distinct outputs rather than concentrated in one, e.g. said, his voice dropping appears 18 times in 17 different stories before training and in none after.

The name distribution shows the same effect. These are the default protagonists the base model reaches for unprompted:

The banlist also catches assistant register bleeding into fiction, e.g. happy to help craft (4 → 0) and let me know if you'd like (5 → 0), where the model breaks frame to address the user mid-story.

Note that suppression is a reduction, not a hard filter. Patterns still surface occasionally; the sampler is available at inference time if you want them driven closer to zero.

Results

Evaluated on 400 held-out Reddit writing prompts (indices 1000 to 1399, with the pipeline trained on 0 to 999), identical sampling for both models (temp 1.0, top_p 1.0, top_k 50, min_p 0.01), Antislop sampler off. These numbers reflect what training changed in the weights, not what a sampler suppresses at inference.

Lexical diversity at 98.4% sits inside the 95 to 102% band the Antislop paper reports for FTPO, and well clear of the 74 to 92% collapse it measures for DPO.

Agentic and reasoning benchmarks

Single H200, vLLM nightly, tool calling via the qwen3_xml parser, greedy decoding, identical harnesses. The τ-bench user simulator was claude-sonnet-5 for both models.

No benchmark difference is statistically distinguishable from zero. Every confidence interval crosses zero.

Usage

Requires trust_remote_code=True for the nemotron_h architecture.

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "thoughtworks/Nemotron-3.5-30B-A3B-Antislop-FTPO"
tok = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id, torch_dtype="bfloat16", device_map="auto", trust_remote_code=True
)

messages = [{"role": "user", "content": "Write the opening of a story about a lighthouse keeper."}]
text = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
out = model.generate(**tok(text, return_tensors="pt").to(model.device), max_new_tokens=512)
print(tok.decode(out[0], skip_special_tokens=True))

Note: on transformers 5.14.x, apply_chat_template(..., tokenize=True) under the TokenizersBackend tokenizer class returns a truncated sequence. Render with tokenize=False and tokenize the resulting string, as shown above. This affects the upstream base model identically and is not specific to this checkpoint.

vLLM

VLLM_USE_RUST_FRONTEND=1 vllm serve --model thoughtworks/Nemotron-3.5-30B-A3B-Antislop-FTPO \
    --moe-backend flashinfer_cutlass \
    --trust-remote-code \
    --max-num-batched-tokens 4096 \
    --speculative_config.method mtp \
    --speculative_config.num_speculative_tokens 1 \
    --speculative_config.moe_backend flashinfer_cutlass \
    --mamba-backend flashinfer \
    --enable-prefix-caching \
    --mamba-cache-mode align \
    --max-num-seqs 16 \
    --max-model-len 65536

For tool calling, use the qwen3_xml parser, which handles this model's XML tool syntax correctly.

Training

Model architecture

Inherited unchanged from the base model. A hybrid Latent Mixture-of-Experts design with interleaved Mamba-2 and MoE layers plus select attention layers, and Multi-Token Prediction (MTP) layers for speculative decoding. 30B total parameters, 3B active. Context length up to 262,144 in this checkpoint's config.

License

Released under OpenMDW-1.1, matching the license NVIDIA applies to the public Nemotron 3.5 Lightning releases. The Antislop framework itself is MIT-licensed.

Citation

The method was published at ICLR 2026:

@inproceedings{paech2026antislop,
  title     = {Antislop: A Comprehensive Framework for Identifying and Eliminating
               Repetitive Patterns in Language Models},
  author    = {Paech, Samuel and Roush, Allen and Goldfeder, Judah and Shwartz-Ziv, Ravid},
  booktitle = {The Fourteenth International Conference on Learning Representations},
  year      = {2026},
  url       = {https://openreview.net/forum?id=gLcyM1khyp},
  eprint    = {2510.15061},
  archivePrefix = {arXiv},
  primaryClass = {cs.CL}
}

• Paper: ICLR 2026 poster · OpenReview · arXiv:2510.15061
• Code: github.com/sam-paech/auto-antislop (MIT)

Acknowledgements

Built on NVIDIA's Nemotron 3.5, with thanks to the NVIDIA team for the benchmark guidance that shaped the evaluation.