AhiskaAI-308m-IT-v0.2
AhiskaAI-308m-IT-v0.2 is the instruction-tuned version of our 308M parameter Small Language Model. Fine-tuned on 16,000+ curated Turkish instruction-response pairs, it is designed to provide stronger conversational ability and improved instruction following while remaining efficient enough to run on consumer hardware.
Base Model: AhiskaAI-308m-Base-v0.2
Model Details
- Architecture: Llama-based architecture.
- Fine-tuning: Supervised Fine-Tuning (SFT).
- Format: ChatML.
- Parameters: 308M.
- Context Window: 1024 tokens.
- Tokenizer: Custom BPE Tokenizer (Vocabulary Size: 32,000).
- Training Framework: PyTorch & Transformers.
- Hardware: NVIDIA RTX 4050 6GB Laptop GPU.
Fine-tuning Dataset
The model was fine-tuned using more than 16,000 carefully curated Turkish instruction-response pairs.
The dataset includes tasks such as:
- Question answering
- General conversation
- Summarization
- Text generation
- Instruction following
- Basic reasoning
Design Goal
The 308M-IT model serves as the flagship conversational model of the AhiskaAI v0.2 family.
Its primary objectives are:
- Improved Turkish instruction following.
- Better contextual understanding.
- More natural conversational responses.
- A strong research foundation for future preference alignment methods such as DPO.
Training Logs
The graph above demonstrates the supervised fine-tuning convergence of AhiskaAI-308m-IT-v0.2.
Usage (ChatML Format)
Recommended System Prompt
Sen kibar, sorulan soruları tam cümlelerle yanıtlayan Türkçe bir asistansın.
Example Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("AhiskaAI/AhiskaAI-308m-IT-v0.2")
tokenizer = AutoTokenizer.from_pretrained("AhiskaAI/AhiskaAI-308m-IT-v0.2")
SYSTEM_PROMPT = "Sen kibar, sorulan soruları tam cümlelerle yanıtlayan Türkçe bir asistansın."
user_query = "Ahıska Türkleri hakkında bilgi verir misin?"
prompt = (
f"<|im_start|>system\n{SYSTEM_PROMPT}<|im_end|>\n"
f"<|im_start|>user\n{user_query}<|im_end|>\n"
f"<|im_start|>assistant\n"
)
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=200)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Known Limitations
- Optimized primarily for Turkish.
- Context window is limited to 1024 tokens.
- Factual accuracy is still limited by model size and pretraining data.
- May generate incorrect or incomplete responses on complex reasoning tasks.
Future Plans
- Preference alignment using DPO.
- Larger, higher-quality Turkish datasets.
- Expanded evaluation benchmarks.
- Future AhiskaAI v0.3 model family.
About AhiskaAI
AhiskaAI is an independent open-source initiative dedicated to developing efficient Turkish Small Language Models trained completely from scratch.
Follow us on Hugging Face for updates and future releases.