saji bsrs/raging1.0-0.5B-fp16
A lightweight 0.5B (494M) parameter model fine-tuned specifically for RAG ingestion pipelines. This is the full-precision FP16 fused version, suitable for conversion to GGUF, further quantization, or continued fine-tuning.
Purpose
raging1.0-0.5B-fp16 is a formatting specialist designed to sit at the front door of your RAG pipeline. It handles three core ingestion tasks:
- Knowledge Graph Extraction - Entity-relation triples as
[subject, relation, object]JSON arrays - Structured JSON Formatting - Strict schema adherence with correct data types
- Concise Summarization - Factual compression in 1-3 sentences
For Apple Silicon users, a pre-quantized 4-bit MLX version is available at sajibsrs/raging1.0-0.5B-MLX-4bit.
Benchmarks vs Base (Qwen2.5-0.5B-Instruct)
| Metric | Base | raging1.0-0.5B | Delta |
|---|---|---|---|
| KG Triple Precision | 0.33 | 0.67 | +103% |
| Throughput (tok/s) | 64 | 122 | +90% |
| TTFT (ms) | 680 | 338 | -50% |
| Output Length Stability | 22.1 | 17.1 | -23% |
| JSON Schema Accuracy | 100% | 100% | - |
| Summarization Compression | 0.69 | 0.68 | - |
Benchmarks run on Apple M4 using the 4-bit MLX quantization, 10-sample subset, averaged over 3 independent runs. FP16 quality is identical; speed metrics will vary by hardware.
Quick Start
from mlx_lm import load, generate
model, tokenizer = load("sajibsrs/raging1.0-0.5B-fp16")
messages = [
{"role": "system", "content": "Extract entity-relation triples as JSON array of [subject, relation, object]. Keep elements 1-4 words. Output ONLY JSON array."},
{"role": "user", "content": "Microsoft was founded by Bill Gates and Paul Allen in Albuquerque."}
]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True)
response = generate(model, tokenizer, prompt=prompt, max_tokens=256, verbose=False)
print(response)
Training Details
- Base Model: Qwen/Qwen2.5-0.5B-Instruct
- Method: QLoRA (4-bit quantized base, LoRA rank 16, all 24 layers)
- Trainable Parameters: 2.93M (0.59%)
- Dataset: ~19.5K curated examples (summarization + structured JSON + KG extraction)
- Iterations: 1,000 (batch size 2, LR 1e-5)
- Hardware: Apple M4 Mac Mini (16GB unified memory)
- Framework: mlx-lm
- Post-training: LoRA adapters fused into base weights, saved as FP16
Known Limitations
- KG Boundary Detection: May produce compound objects (e.g., "Bill Gates and Paul Allen in Albuquerque" instead of separate triples). Recommend post-processing with regex splitting.
- Pronoun Resolution: May output pronouns ("he", "the company") as subjects instead of resolved entity names.
- Not a Chat Model: Fine-tuned for single-turn structured extraction. Multi-turn conversational quality is degraded compared to base.
- JSON Complexity: Handles flat and simple nested schemas reliably. Deeply nested or conditional schemas may require fallback to a larger model.
Recommended Pipeline Integration
Use raging1.0-0.5B-fp16 as a fast first-pass processor. Validate outputs with Pydantic/jsonschema and fall back to a 7B+ model on validation failure:
import json
from pydantic import BaseModel, ValidationError
def safe_extract(raw_output, schema):
try:
return schema.model_validate(json.loads(raw_output))
except (json.JSONDecodeError, ValidationError):
return None # Fallback to larger model
License
Apache 2.0 (inherited from Qwen2.5-0.5B-Instruct)