SmolLM2 1.7B Fungi RAG Agent Distill LoRA GGUF

This repository contains a fine-tuned LoRA adapter for HuggingFaceTB/SmolLM2-1.7B-Instruct, exported in GGUF format for llama.cpp and also included as PEFT adapter files.

This is not a standalone full GGUF model. Load it with a compatible SmolLM2 1.7B Instruct base GGUF, such as the Hugging Face SmolLM2 1.7B Instruct Q4_K_M GGUF.

Files

• gguf/smollm2-1.7b-fungi-rag-agent-distill-checkpoint-800-lora-f16.gguf: llama.cpp LoRA adapter exported from the best measured checkpoint.
• peft/adapter_model.safetensors and peft/adapter_config.json: original PEFT LoRA adapter files for Transformers/PEFT workflows.
• evaluation/: evaluation summary, report, and plots from the real Fungi RAG multi-agent workflow.

Intended Use

The adapter was tuned for a fungi research-agent workflow that uses:

• rag.search for evidence-grounded mycology answers,
• safety.review for wild-mushroom edibility, field-identification, and safety-sensitive prompts,
• numeric source citations from retrieved evidence,
• explicit uncertainty handling when evidence is partial, missing, or conflicting.

It is best treated as a planner/safety/research-agent adapter, not as a general-purpose chat model.

Usage With llama.cpp

llama-cli \
  -m /path/to/smollm2-1.7b-instruct-q4_k_m.gguf \
  --lora gguf/smollm2-1.7b-fungi-rag-agent-distill-checkpoint-800-lora-f16.gguf \
  -p "<|im_start|>system
You are a fungi RAG assistant. Use rag.search for evidence-backed answers and safety.review for safety-sensitive claims.<|im_end|>
<|im_start|>user
Tell me whether a white-gilled wild mushroom under oak is safe to eat.<|im_end|>
<|im_start|>assistant
" \
  -n 180 -c 2048 -ngl 99 --temp 0

Training Data

Fine-tuning used the linked dataset:

JacktheLander/fungi-rag-agent-sft-database

Prepared split counts:

The dataset was generated for the project-owned fungi RAG system and validated for schema, tool-call contract, citation integrity, and safety boundaries before training.

Source Code

Project codebase:

JacktheLander/FunghiResearchAgent

Evaluation report in the GitHub repo:

evaluation/smollm2_fungi_sft_20260611

Training Summary

Base model: HuggingFaceTB/SmolLM2-1.7B-Instruct
Adapter type: LoRA
LoRA rank: 16
LoRA alpha: 32
LoRA dropout: 0.05
Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Best measured checkpoint: step 800
Final run step: 846

Validation results:

Evaluation Summary

The adapter improved sampled agentic behavior accuracy from 33.3% to 47.6%, with the strongest gains in:

• unsafe wild-mushroom field-ID refusal,
• uncertainty handling,
• safety-sensitive answer style.

In the real multi-agent workflow, the adapter produced safer and more qualified prose, but some outputs failed the current validator because they used parenthetical citations like (5) instead of required bracket citations like [5]. See evaluation/report.md for details.

Runtime on an NVIDIA GeForce GTX 1660 SUPER through llama.cpp CUDA:

Safety And Limitations

This model is for academic fungi research-assistant experiments only.

Do not use it for:

• mushroom edibility decisions,
• field identification,
• medical advice,
• dosage or treatment decisions,
• emergency toxicology decisions.

For wild-mushroom exposure or ingestion, contact poison control or emergency medical services. For identification, use a qualified local expert with full specimen evidence.