BananaMind 2 AI Detect
Demo: https://huggingface.co/spaces/Banaxi-Tech/BananaMind-2-AI-Detect-Demo
BananaMind 2 AI Detect is a human-vs-AI text detector built by fully fine-tuning Qwen/Qwen3.5-0.8B-Base on the RAID benchmark.
It is our first dedicated AI-text detection model!
The model is a base-model-style completion classifier: given a passage, it completes Label: with either human or ai, and the human/ai token logits at that position give a calibrated confidence score.
Model Details
| Field | Value |
|---|---|
| Base model | Qwen/Qwen3.5-0.8B-Base |
| Parameters | 0.8B |
| Fine-tune type | Full fine-tune (all weights) |
| Architecture | Qwen3.5 hybrid Gated DeltaNet + attention, Qwen3_5ForCausalLM |
| Vocab size | 248,320 |
| Context length (base) | 262,144 |
| Max sequence length (training) | 2,048 |
| Task format | Completion: Text: <passage>\nLabel: human / Text: <passage>\nLabel: ai |
| Weight format | safetensors |
Training Data
Trained on the RAID benchmark (ACL 2024), restricted to two classes: human and gpt4.
| Class | Examples |
|---|---|
| Human | 150,000 |
| GPT-4 | 150,000 |
| Total prepared | 300,000 |
| Train / Eval split (prepared) | 294,000 / 6,000 |
| Train examples actually seen | ~57,600 (~20% of the prepared train set, due to compute limits — see Max steps below) |
RAID is licensed MIT. This restricts training to a single AI source (GPT-4) rather than the full 11-generator set — see Limitations below for what that means in practice.
Training Setup
| Field | Value |
|---|---|
| Hardware | 1x RTX PRO 6000 Blackwell (96GB) |
| Fine-tune type | Full fine-tune, no LoRA |
| Precision | bf16 |
| Sequence length | 2,048 |
| Micro batch | 12 |
| Gradient accumulation | 4 |
| Effective batch | 48 sequences |
| Max steps | 1,200 — capped due to compute limits (≈57,600 examples seen, ~20% of one epoch over the 294,000-example train set) |
| Optimizer | AdamW |
| Peak learning rate | 1e-5 |
| LR schedule | Cosine, warmup ratio 0.03 |
| Weight decay | 0.01 |
| Gradient checkpointing | Off |
| Kernels | flash-linear-attention, causal-conv1d (fused Gated DeltaNet path) |
Evaluation
Self-reported. Label accuracy measured by comparing the argmax of the human / ai logits against ground truth on held-out RAID examples.
| Eval | Result |
|---|---|
| In-distribution label accuracy (RAID gpt4/human held-out set, n=200) | 99.5% |
| Eval loss @ step 300 | 1.82 |
| Eval mean token accuracy @ step 300 | 60.1% |
Manual spot checks (single examples, not a formal benchmark):
| Input | Prediction | P(ai) |
|---|---|---|
| GPT-4 poem, unedited | ai | 99% |
| Human-written poem | human | 2% |
| Gemini poem | ai | 90% |
| Claude poem | human | 1.2% |
| GPT-4 self-edited poem | ai | 96% |
| GPT-4 poem, prompted to "sound human" | ai | 96% |
Limitations
This model was trained for 1,200 steps (~57,600 examples, roughly 20% of one epoch over the 294,000-example train set) rather than to convergence over the full dataset — the run was deliberately capped to fit a time budget. Loss and eval-loss were still trending downward at the final step, so further training on the same data would likely improve results further, particularly on harder or out-of-distribution cases.
This model was also trained on a single AI generator (GPT-4) versus human text, not RAID's full 11-model roster. In practice this means:
- It is closer to a "GPT-4 vs. not-GPT-4" classifier than a general AI-text detector. Text from other AI systems (Claude, Gemini, Llama, Mistral, etc.) was never shown as a positive "ai" example during training, so "human" predictions on non-GPT-4 AI text should not be read as verified human authorship.
- It has not been evaluated against paraphrasing or other adversarial rewriting. RAID's own adversarial-attack subset (
paraphrase,synonym,article_deletion, etc.) was not included in training or in the spot checks below, so robustness to text that has been reworded or lightly edited is untested. - It is comparatively robust to GPT-4 self-editing and "sound human" prompting — both stayed correctly classified as ai in spot testing, suggesting it picked up signal beyond surface phrasing for at least the GPT-4 case.
- Evaluation numbers above are drawn from the same RAID distribution used for training (in-distribution), plus a handful of manual examples — not a comprehensive out-of-distribution benchmark.
Usage
pip install -U transformers torch
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
model_id = "BananaMind/BananaMind-2-AI-Detect"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
dtype=torch.bfloat16,
device_map="auto",
).eval()
HUMAN_ID = tokenizer.encode(" human", add_special_tokens=False)[0]
AI_ID = tokenizer.encode(" ai", add_special_tokens=False)[0]
def classify(text: str, max_chars: int = 6000):
prompt = f"Text: {text[:max_chars]}\nLabel:"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
with torch.no_grad():
logits = model(**inputs).logits[0, -1]
probs = torch.softmax(logits[[HUMAN_ID, AI_ID]].float(), dim=0)
label = "human" if probs[0] > probs[1] else "ai"
return label, probs[1].item() # (prediction, P(ai))
label, p_ai = classify("Your text here.")
print(label, p_ai)
Intended Use
BananaMind 2 AI Detect is intended for lightweight, local experimentation with AI-text detection, research into detector robustness, and as a baseline for comparison against RAID's leaderboard detectors. It is not intended as a sole or authoritative source for high-stakes decisions (academic integrity, content moderation, legal contexts) given the single-generator training scope and known paraphrase fragility described above.
License
Apache 2.0 (inherited from the base model, Qwen/Qwen3.5-0.8B-Base). Training data (RAID) is MIT-licensed; underlying human-authored source texts retain their original third-party rights.
Citation
If you use RAID, please cite the original paper:
@inproceedings{dugan-etal-2024-raid,
title = "{RAID}: A Shared Benchmark for Robust Evaluation of Machine-Generated Text Detectors",
author = "Dugan, Liam and
Hwang, Alyssa and
Trhl{\'\i}k, Filip and
Zhu, Andrew and
Ludan, Josh Magnus and
Xu, Hainiu and
Ippolito, Daphne and
Callison-Burch, Chris",
booktitle = "Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
month = aug,
year = "2024",
address = "Bangkok, Thailand",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2024.acl-long.674",
pages = "12463--12492",
}
🍌