LipidOS — Phi-3.5 Mini, domain-adapted for lipid/Raman spectroscopy literature
Standalone merged model (base + QLoRA adapter, no peft required) for microsoft/Phi-3.5-mini-instruct,
trained via Unsloth for domain adaptation — next-token/causal-LM
objective on lipid biochemistry and Raman/IR spectroscopy abstracts, not an instruction-tuned task.
Part of LipidOS, a citation-grounded RAG system for the Raman/lipid literature where every generated claim must cite retrieved evidence it actually came from.
Training data
107,665 PubMed abstracts (two MeSH-term pools: bulk lipid biochemistry/lipidomics/membrane biophysics, and Raman/IR/vibrational spectroscopy vocabulary), trained on 64,000 of them (60.6% of one epoch, step-capped to a fixed compute budget).
Results (measured, not estimated)
Held-out perplexity on the same 2,000-example set, base vs. fine-tuned:
| Perplexity | |
|---|---|
| Base (microsoft/Phi-3.5-mini-instruct) | 4.911 |
| Fine-tuned (this model) | 3.955 |
| Improvement | 19.5% lower |
Citation-grounding was separately verified to survive the fine-tune: 0/3 hallucinated evidence
citations across real domain questions plus a deliberate out-of-domain probe (details in the
main repo's docs/solutions.md).
Usage
This model is designed to answer only through a grounded-retrieval + citation-checking
pipeline — it should not be used for open-ended generation without retrieved evidence attached.
See scripts/build_evidence.py
and scripts/generate_finetuned_answer.py
for the reference pipeline.
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("srikarjy025/lipidos-phi3-domain-adapt-merged")
tokenizer = AutoTokenizer.from_pretrained("srikarjy025/lipidos-phi3-domain-adapt-merged")
The LoRA adapter alone (120MB, requires loading the base model + peft separately) is
available at srikarjy025/lipidos-phi3-domain-adapt.
This llama model was trained 2x faster with Unsloth.