LaboAI-0.4.0-3B
LaboAI-0.4.0-3B is a 3B-parameter code model fine-tuned for Android / Kotlin / Jetpack Compose development. It is built on top of Qwen2.5-Coder-3B-Instruct using QLoRA with Unsloth, and the LoRA weights are merged into the released model.
Model details
| Developer | LaboAI |
| Base model | Qwen/Qwen2.5-Coder-3B-Instruct |
| Parameters | ~3B |
| Fine-tuning method | QLoRA (4-bit base, LoRA r=32, alpha=64) |
| Context used in training | 1024 tokens |
| Languages | Kotlin, Java, general code; English and Spanish prompts |
| Available formats | Merged safetensors, GGUF q4_k_m |
Intended use
- Generating and explaining Kotlin code for Android apps
- Jetpack Compose UI snippets, ViewModels, state handling, Room, Retrofit, coroutines
- Fixing and refactoring small to medium Kotlin snippets
- Local inference on modest hardware via the GGUF build (llama.cpp, Ollama, LM Studio)
Prompt format
The model was trained with a plain instruction/response template (not the Qwen chat template), so for best results use:
### Instruction:
{your request}
### Response:
How to use
Transformers
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch
repo = "LaboAI/LaboAI-0.4.0-3B"
tokenizer = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(
repo, torch_dtype=torch.float16, device_map="auto"
)
prompt = (
"### Instruction:\n"
"Write a Jetpack Compose screen with a counter and a button to increment it.\n\n"
"### Response:\n"
)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=512, temperature=0.2, do_sample=True)
print(tokenizer.decode(out[0], skip_special_tokens=True))
llama.cpp / GGUF
llama-cli -m LaboAI-0.4.0-3B-Q4_K_M.gguf -c 2048 --temp 0.2 \
-p "### Instruction:\nCreate a Room DAO for a Note entity.\n\n### Response:\n"
Training data
The training mix was filtered for Android/Kotlin/Jetpack content (keyword filter on the text fields) and formatted into the instruction/response template above:
| Dataset | Notes |
|---|---|
| giggiovpg/ornith-android-instruct | Android instruct data |
| giggiovpg/android-kotlin-compose-compiler-verified | Compose code verified by compiler |
| microsoft/NextCoderDataset | Kotlin subset, capped at 10,000 |
| glaiveai/glaive-code-assistant-v3 | Android/Kotlin subset, capped at 6,000 |
| theblackcat102/evol-codealpaca-v1 | Android/Kotlin subset |
| sahil2801/CodeAlpaca-20k | Android/Kotlin subset |
| genqa/GenQA (code split) | Android/Kotlin subset, capped at 4,000 |
Examples with empty or near-empty text were removed, and the final dataset was shuffled (seed 42).
Training procedure
| Hyperparameter | Value |
|---|---|
| LoRA rank / alpha / dropout | 32 / 64 / 0 |
| Target modules | q, k, v, o, gate, up, down projections |
| Max sequence length | 1024 |
| Per-device batch size | 2 |
| Gradient accumulation | 4 (effective batch 8) |
| Steps | 4,500 |
| Warmup steps | 225 |
| Learning rate | 1e-4 (cosine) |
| Weight decay | 0.01 |
| Optimizer | AdamW 8-bit |
| Precision | fp16 (T4) |
| Hardware | 1x NVIDIA T4 (Kaggle) |
| Framework | Unsloth + TRL SFTTrainer |
Limitations and risks
- It is a small 3B model: it can produce code that does not compile, uses deprecated or non-existent APIs, or contains subtle bugs. Always review and test the output.
- Trained with a 1024-token context, so very long files or multi-file projects may degrade quality.
- The keyword-based filtering may have let in some off-topic examples.
- Android and Compose APIs change quickly; the model may not know the latest library versions.
- Not evaluated on formal benchmarks yet; no quantitative results are claimed.
License
This model is derived from Qwen2.5-Coder-3B-Instruct and inherits its license (Qwen Research License). Please check the base model's license for terms of use, especially for commercial use. The training datasets have their own licenses, which you should also review.
Acknowledgements
- Qwen team for the base model
- Unsloth for efficient fine-tuning
- The authors of the datasets listed above
Citation
@misc{laboai2026,
title = {LaboAI-0.4.0-3B: a Kotlin/Android code model},
author = {LaboAI},
year = {2026},
url = {https://huggingface.co/LaboAI/LaboAI-0.4.0-3B}
}