ANMOLGPT-4B-v0.2

Second public release of the ANMOLGPT family of open-source Large Language Models.

Overview

ANMOLGPT-4B-v0.2 is an instruction-following language model built by fine-tuning Qwen3.5-4B using QLoRA with Unsloth Studio.

This release builds upon the foundation established by ANMOLGPT-3B-v0.1, introducing a more capable base model while preserving the project's focus on reproducible training, transparent evaluation, and open-source development.

The objective of v0.2 is to improve reasoning, instruction following, and overall language understanding while providing a stronger foundation for future releases.

Base Model

• Model: Qwen3.5-4B
• Fine-tuning Method: QLoRA
• Framework: Unsloth Studio
• Export Format: Hugging Face Transformers

Training Details

Dataset

• Databricks Dolly 25K

Training Configuration

Benchmark Results

Benchmark Comparison

Example Usage

from transformers import AutoTokenizer, AutoModelForCausalLM

model_name = "anmoldhandhania93/ANMOLGPT-4B-v0.2"

tokenizer = AutoTokenizer.from_pretrained(model_name)

model = AutoModelForCausalLM.from_pretrained(
    model_name,
    device_map="auto"
)

prompt = "Explain how reinforcement learning differs from supervised learning."

inputs = tokenizer(prompt, return_tensors="pt")

outputs = model.generate(
    **inputs,
    max_new_tokens=256,
    temperature=0.7
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Intended Uses

ANMOLGPT-4B-v0.2 is suitable for:

• Conversational AI
• Learning and experimentation
• Prompt engineering
• Educational applications
• Software development assistance
• General-purpose text generation

Limitations

This model is an experimental open-source release.

Limitations include:

• May produce inaccurate or fabricated information.
• Performance has not yet been fully benchmarked across all standard evaluations.
• Not intended for safety-critical or production use without additional validation.
• Outputs should be reviewed before use in professional environments.

Roadmap

v0.2

• ✅ Upgraded to Qwen3.5-4B
• ✅ Improved training pipeline
• 🔄 Comprehensive benchmark evaluation

v0.5

• Mixed high-quality instruction datasets
• Longer training schedule
• Improved reasoning and coding capabilities

v1.0

• Domain-specific continued pre-training
• Advanced instruction tuning
• Human evaluation
• Retrieval-Augmented Generation (RAG)
• Tool calling
• Production-ready deployment

Project Goals

The long-term goal of ANMOLGPT is to explore efficient and transparent development of open-source language models while documenting the complete engineering lifecycle:

• Training
• Fine-tuning
• Benchmarking
• Deployment
• Continuous improvement

Acknowledgements

ANMOLGPT is built using the outstanding work of the open-source AI community.

Special thanks to:

• Qwen Team
• Unsloth AI
• Hugging Face
• EleutherAI
• Databricks (Dolly Dataset)

Citation

@misc{anmolgpt2026,
  title={ANMOLGPT-4B-v0.2},
  author={Anmol Dhandhania},
  year={2026},
  publisher={Hugging Face},
  url={https://huggingface.co/anmoldhandhania93/ANMOLGPT-4B-v0.2}
}

Future Work

Upcoming releases will focus on:

• Higher-quality instruction datasets
• Continued pre-training for domain adaptation
• Stronger reasoning capabilities
• Coding improvements
• Comprehensive benchmark comparisons
• Efficient inference optimization

Feedback, issues, and contributions are always welcome.