Realtime-Venus-Omni-heretic-ARA
The Realtime-Venus-Omni checkpoint from inclusionAI/Realtime-Venus (itself a fine-tune of openbmb/MiniCPM-o-4_5), with refusal behaviour reduced by abliteration of the language tower. BF16, same file layout and remote code as upstream. An NVFP4 build is at justsumguy/Realtime-Venus-Omni-heretic-ARA-NVFP4.
What changed
Only the language tower (llm.*, Qwen3-8B shape, 36 layers). The vision tower, audio tower,
resampler, audio projection, TTS talker and token2wav assets are upstream's, unmodified.
- 72 matrices were changed:
self_attn.o_projandmlp.down_projin all 36 layers. - The largest relative change is ‖ΔW‖/‖W‖ = 0.379.
The upstream sample media (assets/sample_1_real.mp4, assets/speech_in.mp4) are not included.
Abliteration
- Tool: heretic, the unmerged
arabranch atedc3b12(Arbitrary-Rank Ablation). - Mode: ARA-LoRA with rank-128 adapters. After selection, the adapter delta was merged into the BF16 weights.
- Search setup:
- The language tower was extracted as a standalone Qwen3 model and loaded in bitsandbytes NF4 during the search.
- Hardware: RTX 5060 Ti 16 GB.
- Thinking was disabled in the chat template. System prompt:
You are a helpful assistant.
- Optimiser: 150 trials, 45 of them startup trials. KL target 0.01, row normalisation off, direction not orthogonalised, maximum response length 100.
- Prompts:
- Direction:
mlabonne/harmful_behaviorstrain[:400] andmlabonne/harmless_alpacatrain[:400]. - Evaluation: test[:100] of each.
- Direction:
Selected trial: 83.
| Parameter | Value |
|---|---|
| layers | 17–31 |
| preserve_good_behavior_weight | 0.1400 |
| steer_bad_behavior_weight | 0.1687 |
| overcorrect_relative_weight | 0.4085 |
| neighbor_count | 15 |
| Metric | Refusals (of 100) | KL divergence |
|---|---|---|
| Base model | 98 | – |
| Study (trial 83) | 2 | 0.0696 |
| Recheck (adapter re-fit for export) | 4 | 0.2494 |
The ARA-LoRA re-fit is not bit-reproducible, so the recheck numbers are the ones that describe these weights.
No healing step (fine-tuning after ablation) was applied. Post-abliteration stability needs further testing. In live full-duplex sessions, long repetition loops were observed on ablated builds and not on the unablated model, over a small number of runs.
Usage
Same as upstream: AutoModel.from_pretrained(path, trust_remote_code=True, torch_dtype=torch.bfloat16),
with the pinned requirements in requirements.txt. BF16 weights are 18.7 GB and do not fit a
16 GB card. Use the NVFP4 build, or bitsandbytes NF4 on llm.* only (about 14.8 GB in a duplex session).
License
Apache-2.0, as upstream. See LICENSE and NOTICE. The modification is the abliteration of the language tower described above.